Problems in downloading images from a sub-directory pages.

Post Reply
Ahmad M
Posts: 2
Joined: 24 Jul 2013, 13:11

Problems in downloading images from a sub-directory pages.

Post by Ahmad M » 24 Jul 2013, 14:28

Dear all,
Thanks for this great program.

I have a problem regarding to download full size images from many websites that specialized in photos.

For example, when I create new project to download pictures from (http://www.autowp.ru) , it will only download pictures that are on the main page and never continue downloading from sub-pages (such as http://www.autowp.ru/alfa_romeo/ ,and deeper). And if I create a new project, starting from a sub-link pages as "Starting Address (URL)", it will not download anything at all, announcing that the search results are zero.

Another problem occurred when I want to download full-size images from a Flickr accounts , (such as : http://www.flickr.com/photos/triggerscarstuff/). If I want to download all full-size images from this account, to project will start to download irrelevant photos, starting from the Flickr's main page. And if I want to download images from a specific set or a collection (such as : http://www.flickr.com/photos/triggersca ... 143448306/), the project will start searching images from irrelevant links (scanning like : http://www.flickr.com/about/ .. http://www.flickr.com/help/guidelines/ ...etc) and eventually it will not download any single photo from these sets of collections.

I've faced the same kind of problems in other websites, I know it is probably a configuration problem, but I don't know how to fix it.

Thank you for your help,
With my respectful greetings and good wishes.

Maksym
Site Admin
Posts: 2077
Joined: 02 Mar 2009, 17:02

Re: Problems in downloading images from a sub-directory page

Post by Maksym » 25 Jul 2013, 12:40

Hmmm. This is strange. www.autowp.ru works with all default settings. Do you have the latest version of the program? Please make sure you've uodated to the most recent version, create a new project with all default settings and start it. Seems to be working just fine. Just in case - make sure you don't have anything in the "Excluded URL" filters. If it won't work again - write me to support@exisoftware.com and I'll send you a ready-to-use project.

As for the flickr - try these settings:

Startimg address: http://www.flickr.com/photos/triggerscarstuff/
Target files: *.jp*
Exploration: Regular site, Follow all addresses
Excluded URLs: .*
Included URLs:

triggerscarstuff
\.jpg

You can also play with file size limits, add more sophisticated "Included URLs" to skip thumbnails, but the general idea is shown above.

For other flickr.com users simply change the filter which contains the user name.

Ahmad M
Posts: 2
Joined: 24 Jul 2013, 13:11

Re: Problems in downloading images from a sub-directory page

Post by Ahmad M » 25 Jul 2013, 20:10

Thank you Maxim for you quick reply :)

I already create the project of http://www.autowp.ru with all default settings, and it worked, but downloaded only about 4000 photos of more than 350000 photos and stopped, notifying that the project is complete, and when I resume downloading, the program inform me that the project is complete and ask me if whether I want to update the project of restart downloading again.

When downloading from a sub-page (links of car companies, e.g : http://www.autowp.ru/chevrolet/), an error message appeared :
[ Your project finished but zero target files were saved. Please go to the project options and make sure you entered correct starting addresses.]

sometimes, whether I start the project from the main link of sub-link , I face another error message :
[Your project finished but zero target files were saved. All starting addresses in the project returned the same HTTP error code: 509. Please go to the project options and make sure you entered correct starting addresses.].


At the moment I am trying to download from Flickr using the same setting that you suggest, it will take some time to inform you about the results.

Maksym
Site Admin
Posts: 2077
Joined: 02 Mar 2009, 17:02

Re: Problems in downloading images from a sub-directory page

Post by Maksym » 27 Jul 2013, 12:44

Addresses like http://www.autowp.ru/chevrolet/ are not allowed on this website. Simply try opening this address in browser and you'll see it. So there is no point in making this address one of the startng addresses in the program.

As for the number of images from autowp.ru - this site has a built-in bandwidth limit for every IP address and error 509 means the limit was exceded. So I think it'll be impossible to grab all images from this website in one go. Try clicking "Update" on this project tomorrow.

Post Reply