Can't handle videos that are scheduled for later and added to playlist early

[Fabian42]

The latest video in this playlist is private: https://www.youtube.com/playlist?list=PL7VmhWGNRxKi1ikLc2b_pi6EIGOCdF10d

Since that video got added, the script always outputs an error message for it in the debug sheet and adds all other videos, but does not update the timestamp, which is why it then adds those same videos to the playlist again.

[Fabian42]

Also, some output about which playlist it is would be nice. I can't use the video ID for anything, because it's a private video. And manually searching through my 83 source playlists and channels is pretty tedious.

[victorjzsun]

Sorry for the late response, but I just tried the script with that playlist and it works fine. Do you have the updated script? Try running your script with the playlist in #74 (comment) and see if you get the same output as me.

I'll look into adding extra output on source of video but since we add all videos to a gigantic list before adding, it might take a decent amount of changes

[Fabian42]

Yes, that is the output in the current version. But then the timestamp doesn't change and the next time, the script adds all other videos (which are not private) to the target playlist, over and over again.

[Fabian42]

The playlist no longer has any private videos, that's why the script now works fine with it. It could probably be reproduced with my example playlist from #74, if the latest update timestamp is set to August 2015.

[victorjzsun]

Gotcha, had to fix the error count, try it now with the new version.

[Fabian42]

I just got a mail saying that in the last 24 hours, there was 1 error, at "6/27/21 12:52:03 PM UTC". The sheet has both of its last update timestamps set to "2021-06-28T12:26:25+00:00" and the debug sheet shows no errors at all. There were also no new videos added to either of the two target playlists in the last hours. Does it just silently skip the unavailable videos now? Wouldn't that lead to it missing all videos that are scheduled for later, but added to the source playlist earlier?

[victorjzsun]

Yeah, at least for channel uploads, private videos that are scheduled for later will get picked up by the script after it has been publicly published. Still, you should see a log as shown below in the debug sheet

What makes playlists trickier is that it uses PlaylistItems rather than Videos and so, the items have publishedAt timestamps of when they were added to the playlist, not when the video was published (scheduled or not). Therefore, as you said, it'll miss videos scheduled for later but added to the playlist early. We'd probably need to create some sort of cache to remember the videos that have not been added properly and retry at next update. I'll look into a possible implementation.

[Fabian42]

In that case the previous version was actually better, because there I at least noticed that something was up and could manually do it. Right now it is causing me to miss videos. All I see is "Error: 1 video(s) were not added to playlists correctly, please check Debug sheet. Timestamps for respective rows has not been updated." and then I go to the sheet, but it's hours later and the script has already done more rounds and removed that debug text again.

[victorjzsun]

I was also thinking that we probably want to persist debug logs, just for my information, about how many times do you run the script every day?

[Fabian42]

Exactly 24 times. Much more would run into rate limits anyway.

[victorjzsun]

If you have time, could you try using https://docs.google.com/spreadsheets/d/1Idv6VLjQbzkJ5ToGFJui6o-qhPlZaWfBBKqflGt5kUA/edit?usp=sharing. I implemented persistence of debug logs along with a new sheet to view a particular execution's logs. Feel free to do whatever you want to the script, I have another version with the implementation saved.
P.S. private videos still fail without error, haven't changed that yet, at least it should show up in debug logs

[Fabian42]

Am I supposed to use that one or make a copy? That sheet has no script triggers and I can't add one without linking it to my account, which gives control over my YouTube account to that sheet.

[victorjzsun]

Yeah, make a copy, it's probably easier that way

[Fabian42]

The case didn't occur in the last week, but now I have a different issue: The script suddenly tries to go through tons of rows that don't exist and eventually runs into a timeout. Maybe it's because I added another column? I'm at column "BT" now. But even after removing the column again, it still has that issue. Here is the start of the log:

Row: 4
Cannot query for user Row: 5
Cannot query for user Tue Jul 13 16:35:33 UTC 2021
Cannot query for user Row: 5
Acquired 0 videos
Row: 5
Cannot query for user Acquired 0 videos
Cannot query for user Tue Jul 13 16:35:33 UTC 2021
Cannot query for user Cannot query for user Acquired 0 videos
Acquired 0 videos
Row: 6
Cannot query for user No new videos yet.
Cannot query for user Tue Jul 13 16:35:34 UTC 2021
Cannot query for user Acquired 0 videos
Acquired 0 videos
Row: 8

The row numbers are 4, 5, 6, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 49, 50, … After the first group of 3, those are always groups of 4 and 6, alternating. Pretty strange. In reality I have 2 rows.

[victorjzsun]

It's reading the debugData sheet instead of the correct sheet. Try unhiding the debugData sheet and moving the playlist sheet to the left

[Fabian42]

The order is correct and the only thing hidden are the columns C to E in "Sheet1". The debug sheet is called "Debug", not "debugData" BTW.

[victorjzsun]

I created a new sheet called debugData to store all of the past debug data (hid it to prevent confusion), the "Debug" sheet is now just a convenient viewer which parses the data in the hidden sheet. You'll be able to see it if you click on the sheet menu button

[Fabian42]

Now it works again. Weird that that randomly started happening. I'll keep monitoring for what this ticket is actually about.

[victorjzsun]

No problems with the new debug logger/viewer so far right? other than the sheet reordering

[Fabian42]

I can't see the log of a currently running script in it, so I have to wait to see what did happen instead of seeing it live.
Otherwise, only minor stuff. The dropdown list is getting quite long, so having the latest one on top would be nice. And maybe having it selected by default so that the log is already on the right side, that could speed up doing many tests in a row.

[victorjzsun]

Makes sense, will make some changes