Site Exploration

Amino
Posts: 21
Joined: 17 Jan 2024, 16:14

Re: Site Exploration

Post by Amino »

if i do current page only
https://www.warhammer.com/en-US/shop/wa ... s/tyranids
from that url it downloads nothing
in my way of understanding it should have grabbed

https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371
https://www.warhammer.com/app/resources ... =360&h=371

https://www.warhammer.com/en-US/shop/ty ... e4fa9dcc53
from this url it downloads the images

that aside what i want it to do is grab the jpg without webp formatting

so it would download https://www.warhammer.com/app/resources ... XTRA13.jpg
instead of https://www.warhammer.com/app/resources ... =360&h=371
Maksym
Site Admin
Posts: 2084
Joined: 02 Mar 2009, 17:02

Re: Site Exploration

Post by Maksym »

What I'm trying to tell you is that Extreme Picture Finder works with what you see in the page source, not with what you see on a rendered page. So, I'm asking you once again - show me the part of the source page of the category page with links to those images. Make a screenshot.

If you want to use links from a rendered page - use the [ Address list ] tab of the built-in browser as I described above.
Maksym
Site Admin
Posts: 2084
Joined: 02 Mar 2009, 17:02

Re: Site Exploration

Post by Maksym »

And cutting a part of any link that follows the "?" character is done very easily with the [ Custom Parsers ]. For example:

Expression: ="(https?://[^\?"]+)\?
Result: [#1]
Amino
Posts: 21
Joined: 17 Jan 2024, 16:14

Re: Site Exploration

Post by Amino »

Image
i think that's what you want?
Maksym
Site Admin
Posts: 2084
Joined: 02 Mar 2009, 17:02

Re: Site Exploration

Post by Maksym »

So I'd suggest using the [ Address list ] tab of the built-in browser with the following Regular Expression in the [ Manual login options ] section of the project properties:

Expression: ^(https://[^/]+/app/resources/catalog/product/[^/]+/[^/\?]+)\?
Result: [#1]

Image

Here is the result:

Image
Maksym
Site Admin
Posts: 2084
Joined: 02 Mar 2009, 17:02

Re: Site Exploration

Post by Maksym »

Image
Unfortunately, that's what YOU want. That's not a page source. That's the elements inspector. It uses data from a gazillion of Fetch/XHR requests that were used to render a page and show all those product links and images. Page source, the one that Extreme Picture Finder sees, is available only if you press [ Ctrl + U ] in your browser.
Amino
Posts: 21
Joined: 17 Jan 2024, 16:14

Re: Site Exploration

Post by Amino »

So by default EPF uses source code to grab images and by using the built in browser it grabs data from the requests?
Maksym
Site Admin
Posts: 2084
Joined: 02 Mar 2009, 17:02

Re: Site Exploration

Post by Maksym »

No, not really. You'll need to re-create those requests with [ Custom Parsers ] to get and then parse data from the response. And you'll need to set the correct request type, fill POST data (if needed), and copy values of all meaningful request headers from the original requests. If you want an example - take a look at the instagram or twitter template.
Amino
Posts: 21
Joined: 17 Jan 2024, 16:14

Re: Site Exploration

Post by Amino »

i will do that thanks for all the help
Post Reply