Files not downloading completely

Beniled
Posts: 27
Joined: 05 Nov 2021, 18:52

Files not downloading completely

Post by Beniled » 31 May 2022, 23:08

If it takes a while for a video to download the download just stops and leaves the video only partially downloaded. I don't know if this has to do with the Timeout in Tools>Settings or if it is just some sort of timeout from the site. However, it is leaving a bunch videos only partially downloaded. Is there a way to make is so that the file is considered failed and not saved if it doesn't download completely so that we can retry them with the Update feature?

This also happens to images but those are easy to tell because they have artifacts on the preview. It would just be nice if it just rejects the file rather than download a partial file. Also can we set the timeout longer than 10 minutes? maybe not set an upper limit?

Maxim
Site Admin
Posts: 1694
Joined: 02 Mar 2009, 17:02

Re: Files not downloading completely

Post by Maxim » 01 Jun 2022, 11:26

OK, thank you for reporting it. I'll see what can be done here.

Beniled
Posts: 27
Joined: 05 Nov 2021, 18:52

Re: Files not downloading completely

Post by Beniled » 20 Jun 2022, 20:37

Cool, thank you. This would be HUGE! :)

Beniled
Posts: 27
Joined: 05 Nov 2021, 18:52

Re: Files not downloading completely

Post by Beniled » 18 Aug 2022, 19:53

I just want to bump this as this is still an issue I have been finding on websites with larger file sizes, particularly videos, that limit their upload speeds. I was just wondering where it was in the dev timeline?

Maxim
Site Admin
Posts: 1694
Joined: 02 Mar 2009, 17:02

Re: Files not downloading completely

Post by Maxim » 19 Aug 2022, 18:26

The feature is already implemented but it still needs debugging and something needs to be done about user "awareness" about partial files. If you'd like to try it - I'll send you a download link to your email. Just let me know. In fact, your feedback would help a lot. And if anyone else wants to try - just reply to this thread and I'll send a download link.

What is already done: when the program finds out it has downloaded a file only partially and the download process is already finished for this file - it does not save the partially downloaded content, adds the "206" HTTP response code to the file address in the download log and moves on to the next URLs if they are present. Once all URLs are done - the project is finished and you can use the "Update" button to re-download all missing files.

Still to do: how to let the user know the project is finished with such "unsaved" files, and he needs to [ Update ] the project to try to re-download them? Right now you can only see it in the download log of the project.

Beniled
Posts: 27
Joined: 05 Nov 2021, 18:52

Re: Files not downloading completely

Post by Beniled » 20 Aug 2022, 00:18

That sounds really cool! I would definitely take a look.

I am really happy to hear it.

I think, from what I understand you are saying, that maybe a popup or something letting the user know that their project only partially finished would handle letting the user know. Maybe even have a button on the popup to retry the missing files?

Also, being able to set the timeout to higher limits would help, I think, as it might just continue to fail on large files from sites that throttle upload speeds. This is something I noticed especially with downloading videos.

I mean, I am looking at the setting and it says, "Abort download if no data is received for (seconds)".

However, I've watched a video downloading and it was receiving data, the % was increasing as was the file size on disk, but after about ten minutes of downloading total, the download just stops and the video is only partially downloaded. Maybe this function is bugged? As I read it it sounds like as long as it receives data the download will run as long as it takes to finish, correct? If that is the case, then there appears to be a bug as the download will stop after downloading for the set timeout regardless of if it is receiving data or not.

Beniled
Posts: 27
Joined: 05 Nov 2021, 18:52

Re: Files not downloading completely

Post by Beniled » 26 Oct 2022, 19:18

Did you add a popup to let people know that some of the downloads didn't download in the latest version?

Maxim
Site Admin
Posts: 1694
Joined: 02 Mar 2009, 17:02

Re: Files not downloading completely

Post by Maxim » 26 Oct 2022, 19:26

Nope. Currently download log is the only place where you can see you have such files.

Beniled
Posts: 27
Joined: 05 Nov 2021, 18:52

Re: Files not downloading completely

Post by Beniled » 31 Oct 2022, 19:02

Ok, is that something that is still on the todo list? The issue I am having with the Download log is that on some of the larger projects the Download log just sits on loading, and if they load you can't filter, search, or sort the HTTP response column so it makes finding partial downloads difficult for projects with 10s of thousands of urls.

Is there a way to look at the log in say excel? Without using the application, like can it just be opened externally?

Maxim
Site Admin
Posts: 1694
Joined: 02 Mar 2009, 17:02

Re: Files not downloading completely

Post by Maxim » 01 Nov 2022, 10:55

Yes, it is in the to-do list.

Post Reply