Extreme Internet Software forum

When scrapping images from cloudflare sites once the initial signup with the browser it is find for a bit, but after awhile cloudflare kicks in again and needs to be verified, instead of getting a prompt to do that, the whole project stops and fails. I feel like this should be fixed, unless there is some workaround for that.

It depends greatly on the website, but in most cases, adding a delay between downloads of 3 - 5 seconds and limiting the number of simultaneous connections to 2 - 3 prevents the CloudFlare protection from kicking back in during the download. Almost all templates for the CloudFlare-protected websites use this technique.