Filenames are not being produced fully

Post Reply
iBot27
Posts: 4
Joined: 23 Jul 2018, 18:50

Filenames are not being produced fully

Post by iBot27 » 24 Jul 2018, 13:08

Software Version is use: Extreme Picture Finder Version 3.42.7.0

This strange thing has been happening, downloaded files do not contain the full original name, i.e. it seems to be getting cut off. Example: Full name is thispicture.jpg - Downloaded/saved file reads: ispicture.jpg

I've also noticed that the program becomes very sluggish after a few hours into my project which is when I believe this issue occurs. This issue is very bothersome as my project contains 100K+ images so having filenames become "garbled" so to speak is going to make my end of work a lot more troublesome than i'd like.

I've seen a previous thread where someone described a similar issue to which I am having in regards to the program becoming sluggish after a like 10K+ images. It doesn't necessarily freeze or hang but becomes very sluggish. Processes slow to a crawl and images are downloaded/saved at a very slow rate. The program continues to chug along but not at an optimal performance.

I've been comparing this software alongside another one and EPF manages to keep up until about the 8K+ mark where it just falls off in links being Processed/queued/downloaded. I've been sticking with EPF due to it's overall extensive options and the fact that when scraping the same site, EPF manages to properly structure files/folders as intended, minus this issue with the filenaming and the program becoming sluggish.

Any help would be greatly appreciated, thanks. Also the site in questions www.nudecollect.com. I'm trying to scrape all the MetArt Categories.

Regards

Maksym
Site Admin
Posts: 2071
Joined: 02 Mar 2009, 17:02

Re: Filenames are not being produced fully

Post by Maksym » 25 Jul 2018, 19:57

Did you use the www.nudecollect.com - category template or did you set up the project yourself?

The only case when the program is designed to cut file names of is when the full file name (including drive and all folders) is longer than allowed by Windows (512 characters).

iBot27
Posts: 4
Joined: 23 Jul 2018, 18:50

Re: Filenames are not being produced fully

Post by iBot27 » 26 Jul 2018, 05:07

My project is based off of the https://www.webimagedownloader.com/temp ... -template/. I believe I may have changed/added/removed a few expressions. At this point I've done so many tests to try to get it working properly I really can't remember them off the top of my head while writing this post but yes my project is based off of the https://www.webimagedownloader.com/temp ... -template/.

A few changes I made to https://www.webimagedownloader.com/temp ... -template/ was the expert expression for subfolders. I just wanted the folder structure of the website. I tried https://www.webimagedownloader.com/temp ... -template/ with no changes and the galleries being placed into their own folder was fine but not grouping them with the month is why I changed it back to the site's folder structure. The other change was to excluded url "/_data/", I removed it from the exclusion because I found that some contactsheets.jpg for galleries are stored there as well as the thumbs. I added "-cu_e260x350.jpg" to the exclusion of urls to prohibit EFP from downloading any of the thumbs.

About the statement of the Windows character limit, I thought about that as well. I enabled "LongPathsEnabled" from the registry and it still didn't change anything. And forgive me in advance as I am not trying to promote a different software at all, I am merely testing to see which will perform to my needs, but, with the other software "noname.exe", it doesn't create the naming issues with the same folder depth/structure. Although, now that I think of it, "noname.exe" doesn't save the images to the created subfolders it places them in the base download folder and I then have to move thousands of files to the correct subfolders manually, an arduous undertaking with 10k+ images.

Here's what I'll do for a test, I'll move my directory to the root of my drive to see if anything changes and I will report back. I'll also do another test with https://www.webimagedownloader.com/temp ... -template/ unchanged to see if I can just work with that and to see if the names get cut short at all. It's just the image filenames that get cut short, none of the folders have that I've noticed.

EDIT:

TEST #1

I used https://www.webimagedownloader.com/temp ... -template/ with no changes to "subfolders" against the basic option "Reproduce folder structure of a website". When using the Regex for subfolders created by the template there were no character cuts to any filenames but when using the basic option "Reproduce folder structure of a website" I reproduced the filename character cuts. This test was done with the current folder-depth/structure I've been using to save my images.

TEST #2

I changed the download destination to a shallower folder-depth/structure and this time used the Subfolder basic option "Reproduce folder structure of a website" and lo and behold, filenames are being properly reproduced with no character cuts.

Conclusion:

The issue was due to Windows character limitations, it seems. Which, explains why "noname.exe" was saving my images to the base folder and none of the subfolders. At least EPF did what it said it was gonna do via project settings with a minor issue. :D

Now my question to you is, from a dev standpoint, is there anyway to do a check to produce a warning to the user that the following may occur. I believe it would help anybody in the future having the same issue. And now knowing this, lol, II might have to rearrange my storage drives with this in mind. I'm surprised windows didn't warn me. Maybe, it was EPF just cutting the characters that prevented a Windows warning?

Either way, my issue with the file naming seems to be resolved. Now to see if I can reproduce the software becoming sluggish. Thanks for the suggestion of windows character limits. And although I had previously thought of that, your re-suggestion made me come up with different ideas for testing.

Maksym
Site Admin
Posts: 2071
Joined: 02 Mar 2009, 17:02

Re: Filenames are not being produced fully

Post by Maksym » 28 Jul 2018, 18:43

Thank you very much for your tests and reports. Glad we sorted that out. I'll have to think how to warn users when the names are being cut off.

Post Reply