tomvoyeur.com

Post Reply
aoaoa
Posts: 3
Joined: 07 May 2023, 13:00

tomvoyeur.com

Post by aoaoa » 10 May 2023, 07:53

I would like to download enlarged photos of this page.
http://www.tomvoyeur.com/forums/album.php?albumid=102
It looks very simple but I have tried many template and many ways but failed.
Could you help me?

Maksym
Site Admin
Posts: 2077
Joined: 02 Mar 2009, 17:02

Re: tomvoyeur.com

Post by Maksym » 10 May 2023, 14:41

Well, this one is not that trivial as it seems. You see, there are no links to JPG files anywhere. If you right-click any full-size photo and select [ Open image in new tab ] - you will see that photos have URLs like this:

tomvoyeur.com/forums/attachment.php?attachmentid=XXXXXX&d=XXXXXX

Plus, every photo is shown on it's own page, so you can't just set the [ Exploration mode ] to [ Current page only ] or even to [ Current directory and deeper ] because the directory of you Starting URL is "/forums/", which covers the majority of the website.

In such cases I do the following:

1. I set the [ Exploration mode ] to [ Regular site -> Entire site]
2. I exclude all addresses by adding

.

(just a single point) to the [ Excluded URLs ] filters to make it so that Extreme Picture Finder crawls only addresses that I allow in the [ Included URLs ].

3. Now it's time to add filters to the [ Included URLs ]. They must allow crawling of the full-size photo pages as well as the photo links. Here are the filters:

/album\.php\?albumid=\d+&attachmentid=\d+$
/attachment\.php\?attachmentid=\d+&d=\d+$

Basically that's it, but when I ran the project it turned out that this website does not allow too many simultaneous connections. So I used 3 connections and added a delay between downloads of 1 - 3 seconds. This can be done in the [ Site exploration -> Options ] section of the project properties.

After that I successfully download all photos from several different albums.

aoaoa
Posts: 3
Joined: 07 May 2023, 13:00

Re: tomvoyeur.com

Post by aoaoa » 10 May 2023, 18:25

Thanks for your kind guidence.
Even though your detailed explanation, it is not easy for me as I am not very good at this.
Anyway I will try to see what happen.
Or is it possible for you to supply us template for this so that we can use for this and other similar web sites.

Maksym
Site Admin
Posts: 2077
Joined: 02 Mar 2009, 17:02

Re: tomvoyeur.com

Post by Maksym » 10 May 2023, 18:31

Please try the settings I provided. It should work perfectly.

aoaoa
Posts: 3
Joined: 07 May 2023, 13:00

Re: tomvoyeur.com

Post by aoaoa » 10 May 2023, 19:18

I tried with success. It's amazing and wonderful.
Thank you very much.

Maksym
Site Admin
Posts: 2077
Joined: 02 Mar 2009, 17:02

Re: tomvoyeur.com

Post by Maksym » 11 May 2023, 10:41

You are welcome!

Post Reply