Downloading Failed Files with Rhilip Patch # of Failed Files Increases #329
Replies: 2 comments
-
|
Some insight into this, it looks like the files it thinks it needs to download have been downloaded. As I can see that the script is successfully downloading and then deleting things that already exist in the folder. So it isn't updating the YAML in that instance or something. |
Beta Was this translation helpful? Give feedback.
-
|
OK. So, this was my fault. When I copied the code over I only did it a section at a time, I must have messed it up. Because when I went to the code compare page and just copied the entire script over. It seems to work correctly, please ignore my ignorance here. LOL It still seems to create duplicate entries for the retries in the YAML, but it does seem to clean up some, and the numbers have only been going down since. So it must filter out the dups on it's own. |
Beta Was this translation helpful? Give feedback.
-
!!!!!!!!!! UPDATE - THIS WAS CAUSED BY NOT FULLY COPYING THE RHILIP SCRIPT - MUST HAVE MISSED SOME LINES
So I have downloaded a few channels with this tool, and it is a god send. However, I am experiencing a few bugs with the Rhilip patch.
So the script is supposed to mark any downloads that do not complete fully or leave a 0kb file behind, or just simply time out 3 times. Which seems to be working sort of.
The last couple of channels I have pointed it at are never able to fully download all of the files and I have disabled the video_note downloading since that seems broken entirely.
What happens is I will run the script, wait for completion and then it tells me "Downloading of #### files failed", that will be retried on the next run. So I run the script again as designed and when it completes again, the number of retries doesn't always go down. Sometimes it goes up. For example I just ran it four times in a row, and the "Downloading of #### files failed" values were #1 = 183, #2 = 171, #3 = 173, #4 = 175.
So, I looked at the YAML file after one of the runs and I copied out the ids_to_retry into an excel spreadsheet, and for one there were more lines than the script reported, but also there were many duplicate message ID's. I used Excel to sort, and checked it out, out of 600 or so ids, almost 200 were duplicates, I knew because I used the remove duplicate function to see the difference and it was staggering.
Here is my current state of my YAML ids_to_retry after probably 5 - 8 runs, sorted a few ways. The last run when this was copied out stated "Downloading of 175 files failed"
Original Order:
Total ids_to_retry in YAML: 388
Sorted
ids_to_retry:
1721
1721
1737
1737
1746
1746
1751
1751
1762
1762
1763
1763
1765
1765
1768
1771
1772
1773
1774
1802
1814
1816
1817
1947
1953
1955
1958
1960
1973
2440
2440
2463
2463
2467
2467
2468
2468
2470
2470
2476
2476
2486
2486
2493
2493
2495
2495
2498
2498
2502
2502
2503
2503
2509
2509
2510
2510
2513
2513
2514
2514
2515
2515
2517
2517
2543
2543
2544
2544
2545
2545
2554
2554
2575
2575
2577
2577
2579
2579
2582
2582
2583
2583
2589
2589
2592
2592
2595
2595
2597
2597
2598
2598
2599
2599
2602
2602
2608
2608
2609
2609
2612
2612
2614
2614
2617
2617
2619
2619
2621
2621
2622
2622
2625
2625
2629
2629
2632
2632
2634
2634
2638
2638
2639
2639
2640
2640
2647
2647
2650
2650
2652
2652
2657
2657
2658
2658
2660
2660
2661
2661
2662
2662
2663
2663
2665
2665
2668
2668
2669
2669
2671
2671
2673
2673
2674
2674
2676
2676
2679
2679
2680
2680
2682
2682
2683
2683
2684
2684
2685
2685
2688
2688
2689
2689
2690
2690
2691
2691
2692
2692
2694
2694
2695
2695
2696
2696
2697
2697
2698
2698
2699
2699
2701
2701
2703
2703
2704
2704
2705
2705
2706
2706
2707
2707
2709
2709
2711
2711
2714
2714
2719
2719
2723
2723
2724
2724
2726
2726
2727
2727
2763
2763
2822
2825
2827
3133
3133
3145
3145
3178
3178
3222
3222
3340
3340
3373
3373
3409
3409
3460
3460
3500
3500
3538
3538
3540
3540
3552
3552
3558
3558
3567
3567
3568
3568
3571
3571
3572
3572
3573
3573
3575
3575
3589
3589
3590
3590
3592
3592
3595
3595
3596
3596
3599
3599
3600
3600
3601
3601
3602
3602
3604
3604
3610
3610
3612
3612
3633
3633
3635
3635
4470
4470
4537
4537
4541
4541
4747
4747
4749
4749
4760
4760
4761
4761
4765
4765
4776
4776
4778
4778
4779
4779
4782
4782
4784
4784
4785
4785
4786
4786
4790
4790
4792
4792
4805
4805
4807
4807
4808
4808
4809
4809
4810
4810
4811
4811
4813
4813
4814
4814
4817
4817
4818
4818
4822
4822
4824
4824
4827
4827
4828
4828
4830
4830
4831
4831
4833
4833
4836
4836
4838
4838
4841
4957
5004
5006
5068
5076
5102
5113
5119
5121
5121
5149
5149
5962
5965
5966
5980
5982
5984
5987
6014
6048
6050
6058
Deduplicated
Total Deduplicated ids_to_retry: 175 duplicate values found and removed; 213 unique values remain
ids_to_retry:
1721
1737
1746
1751
1762
1763
1765
1768
1771
1772
1773
1774
1802
1814
1816
1817
1947
1953
1955
1958
1960
1973
2440
2463
2467
2468
2470
2476
2486
2493
2495
2498
2502
2503
2509
2510
2513
2514
2515
2517
2543
2544
2545
2554
2575
2577
2579
2582
2583
2589
2592
2595
2597
2598
2599
2602
2608
2609
2612
2614
2617
2619
2621
2622
2625
2629
2632
2634
2638
2639
2640
2647
2650
2652
2657
2658
2660
2661
2662
2663
2665
2668
2669
2671
2673
2674
2676
2679
2680
2682
2683
2684
2685
2688
2689
2690
2691
2692
2694
2695
2696
2697
2698
2699
2701
2703
2704
2705
2706
2707
2709
2711
2714
2719
2723
2724
2726
2727
2763
2822
2825
2827
3133
3145
3178
3222
3340
3373
3409
3460
3500
3538
3540
3552
3558
3567
3568
3571
3572
3573
3575
3589
3590
3592
3595
3596
3599
3600
3601
3602
3604
3610
3612
3633
3635
4470
4537
4541
4747
4749
4760
4761
4765
4776
4778
4779
4782
4784
4785
4786
4790
4792
4805
4807
4808
4809
4810
4811
4813
4814
4817
4818
4822
4824
4827
4828
4830
4831
4833
4836
4838
4841
4957
5004
5006
5068
5076
5102
5113
5119
5121
5149
5962
5965
5966
5980
5982
5984
5987
6014
6048
6050
6058
Beta Was this translation helpful? Give feedback.
All reactions