Extreme Internet Software forum

Posted: **28 Nov 2024, 20:58**

Hi,

I was creating a custom template and I ended up with a lot of links that end with multiple "\\\\" chars.
I believe the correct link is still parsed, but this generates a lot of needless links.

Do you have any idea what could be causing this issue?

Thanks

EDIT: I should have added a bit more info.

Take this link (NSWF)

Code: Select all

https://forum.candidgirls.io/t/fit-latina-in-onepiece-flex-gif/499923

In the Excluded links I have:

Code: Select all

^https?://(www\.)?candidgirls\.io
https://forum.candidgirls.io/privacy
https://forum.candidgirls.io/login
https://forum.candidgirls.io/tos
https://forum.candidgirls.io/guidelines
https://forum.candidgirls.io/categories
https://forum.candidgirls.io/u
https://forum.candidgirls.io/tag
https://forum.candidgirls.io/letter_avatar_proxy
images/emoji/
https://forum.candidgirls.io/c

In the Included links I have:

Code: Select all

https://forum.candidgirls.io/t/
https://forum.candidgirls.io/uploads/    (this one is where the attachments are)

I have played a bit with the excluded pages parts in order to parse out needless code other than the image links, but I am still getting these extra links.

Once again, thank you.

Posted: **29 Nov 2024, 11:24**

It looks like all the links with the extra "\" characters are located before the "</header>" tag, so adding one [ Excluded Page Part ] like this should do the job:

Code: Select all

From:
To: </header>

Another note. Why do you exclude www.candidgirls.io? There are no links to that website from the forum. I think the only Excluded URL (if you want to go that way) that you need is

Code: Select all

candidgirls\.io

And the only [Included URL] that you need is

Code: Select all

/original/

I think you are overcomplicating things.

Posted: **29 Nov 2024, 17:08**

Maksym wrote: 29 Nov 2024, 11:24 It looks like all the links with the extra "\" characters are located before the "</header>" tag, so adding one [ Excluded Page Part ] like this should do the job:
Code: Select all
From:
To: </header>
Another note. Why do you exclude www.candidgirls.io? There are no links to that website from the forum. I think the only Excluded URL (if you want to go that way) that you need is
Code: Select all
candidgirls\.io
And the only [Included URL] that you need is
Code: Select all
/original/
I think you are overcomplicating things.

Thanks for your input, Maksym!

The idea I had in mind was to crawl a category, for example:

https://forum.candidgirls.io/c/boobs

and download each thread from that category.

Perhaps I'm not going in the right direction and there is an easier way to do that?

Posted: **30 Nov 2024, 11:51**

You never mentioned you wanted the categories :( How did you plan to handle the pagination of the category pages?

Posted: **30 Nov 2024, 13:11**

So far I was allowing links in the form of " ...\t\ " and I was using the "Excluded page parts" to remove the links for other threads other than &page=.

But yeah, I am a noob with this, so I'm sure there are better ways to do this.

Posted: **30 Nov 2024, 13:21**

Yeah, that's the easiest way. Simply add

/t/

and

page=\d+

to the [ Included URLs ]

Posted: **30 Nov 2024, 13:38**

Thank you for all your help!

Extreme Internet Software forum

Links with extra "\\\"

Links with extra "\\\"

Re: Links with extra "\\\"

Re: Links with extra "\\\"

Re: Links with extra "\\\"

Re: Links with extra "\\\"

Re: Links with extra "\\\"

Re: Links with extra "\\\"