Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot connect to the database or query failed unexpectedly #113

Open
pghpete opened this issue Jan 23, 2023 · 6 comments · May be fixed by #118
Open

Cannot connect to the database or query failed unexpectedly #113

pghpete opened this issue Jan 23, 2023 · 6 comments · May be fixed by #118
Labels
bug

Comments

@pghpete
Copy link

pghpete commented Jan 23, 2023

ytcc version 2.6.1 is failing its update process with an error message "Cannot connect to the database or query failed unexpectedly"

Running ytcc with a log level of debug...

ytcc -l debug -c ytcc.conf update

[1674486976.230407] [MainProcess/MainThread] ytcc.cli.ERROR: Cannot connect to the database or query failed unexpectedly
[1674486976.230621] [MainProcess/MainThread] ytcc.cli.DEBUG: Unknown database error
Traceback (most recent call last):
  File "/home/pete/.local/lib/python3.9/site-packages/ytcc/cli.py", line 742, in main
    exit_code = cli.main(standalone_mode=False)
  File "/home/pete/.local/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/pete/.local/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/pete/.local/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/pete/.local/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/pete/.local/lib/python3.9/site-packages/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/pete/.local/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/pete/.local/lib/python3.9/site-packages/ytcc/cli.py", line 357, in update
    ytcc.update(max_fail, max_backlog)
  File "/home/pete/.local/lib/python3.9/site-packages/ytcc/core.py", line 152, in update
    updater.update()
  File "/home/pete/.local/lib/python3.9/site-packages/ytcc/updater.py", line 246, in update
    asyncio.run(self.do_update())
  File "/usr/lib64/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib64/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/home/pete/.local/lib/python3.9/site-packages/ytcc/updater.py", line 243, in do_update
    await asyncio.gather(*map(self.update_playlist, playlists))
  File "/home/pete/.local/lib/python3.9/site-packages/ytcc/updater.py", line 237, in update_playlist
    self.database.add_videos([video], playlist)
  File "/home/pete/.local/lib/python3.9/site-packages/ytcc/database.py", line 349, in add_videos
    cursor.execute(insert_video, asdict(video))
sqlite3.IntegrityError: UNIQUE constraint failed: video.extractor_hash

bug report info...

ytcc bug-report
---ytcc version---
2.6.1

---youtube-dl version---
2021.06.06

---yt-dlp version---
2023.01.06

---Click version---
8.0.3

---SQLite version---
SQLite system library version: 3.26.0
Python module version: 2.6.0

---python version---
3.9.13 (main, Nov  9 2022, 13:16:24) 
[GCC 8.5.0 20210514 (Red Hat 8.5.0-15)]

---mpv version---
mpv is not installed

---config dump---
[ytcc]
download_dir = /archive2/dlna
download_subdirs = false
mpv_flags = --really-quiet --ytdl --ytdl-format=bestvideo[height<=?1080]+bestaudio/best
order_by = playlists:asc, publish_date:desc
video_attrs = id, title, publish_date, duration, playlists
playlist_attrs = name, url, tags, reverse
db_path = ~/.local/share/ytcc/ytcc.db
date_format = %Y-%m-%d
max_update_fail = 5
max_update_backlog = 20
age_limit = 18

[tui]
alphabet = sdfervghnuiojkl
default_action = play_video

[theme]
prompt_download_audio = 2
prompt_download_video = 4
prompt_play_audio = 2
prompt_play_video = 4
prompt_mark_watched = 1
table_alternate_background = 245
plain_label_text = 244

[youtube_dl]
format = bestvideo[height<=?1080]+bestaudio/best
output_template = %(uploader)s.%(title)s.%(ext)s
ratelimit = 0
retries = 0
subtitles = off
thumbnail = true
skip_live_stream = true
merge_output_format = mkv
max_duration = 0
restrict_filenames = false
@pghpete pghpete added the bug label Jan 23, 2023
@pghpete
Copy link
Author

pghpete commented Jan 23, 2023

I do see that the bug report lists youtube-dl but I've been using yt-dlp instead as its far more advanced and less buggy. Does't ytcc now use yt-dlp?

@woefe
Copy link
Owner

woefe commented Jan 23, 2023

Ytcc uses yt-dlp over youtube-dl, if both are found on your system. The error is probably related to something else. It seems there is a problem in how ytcc or yt-dlp calculate unique identifiers for some videos. Somehow it calculates the same id for two different URLs. I cannot reproduce the issue though. To reproduce, I'd need a list of subscriptions that trigger the error. Did you add new subscriptions recently (especially something not from youtube) that cause the issue?

@pghpete
Copy link
Author

pghpete commented Jan 23, 2023

hum, not that I know of. I did remove the very last sub I performed and re-ran it but it produced the same issue. I then removed the last sub before the error and that didn't help. I ended exporting my existing subscriptions, moving the DB out of the way, and resubscribing to all via my exported list.

I wasn't able to find out what specifically was the issue so I ended up using a heavy hand and moving the DB out of the way and I still ended up having the same error happen.

I have attached a text file of the subscriptions where this is happening.

ytcc-subscriptions.txt

Please let me know if there is anything I can do to assist with tracking down what might be the issue here.

@chemicalstorm
Copy link

I can reproduce the issue with an empty DB and adding only one subscription:

$ ytcc bug-report
---ytcc version---
2.6.1

---youtube-dl version---
youtube-dl not found

---yt-dlp version---
2023.07.06

---Click version---
8.1.6

---SQLite version---
SQLite system library version: 3.42.0
Python module version: 2.6.0

---python version---
3.11.3 (main, Jun  5 2023, 09:32:32) [GCC 13.1.1 20230429]

---mpv version---
mpv is not installed

---config dump---
[ytcc]
download_dir = /mnt/stufz/Videos/Youtube
download_subdirs = true
mpv_flags = --ytdl --ytdl-format=bestvideo[height<=?1080]+bestaudio/best
order_by = playlists:asc, publish_date:desc
video_attrs = id, title, publish_date, duration, playlists
playlist_attrs = name, url, reverse, tags
db_path = ~/.local/share/ytcc/ytcc.db
date_format = %Y-%m-%d
max_update_fail = 5
max_update_backlog = 20
age_limit = 1000

[tui]
alphabet = sdfervghnuiojkl
default_action = download_video

[theme]
prompt_download_audio = 2
prompt_download_video = 4
prompt_play_audio = 2
prompt_play_video = 4
prompt_mark_watched = 1
table_alternate_background = 245
plain_label_text = 244

[youtube_dl]
format = (bestvideo[height<=?1080][vcodec^=avc1]+bestaudio/best)/bestvideo[height<=?1080]+bestaudio/best
output_template = %(upload_date)s - %(title)s.%(ext)s
ratelimit = 0
retries = 0
subtitles = en, fr
thumbnail = true
skip_live_stream = true
merge_output_format = mp4
max_duration = 9000
restrict_filenames = false

[storm@velona ~]$ rm ~/.local/share/ytcc/ytcc.db

[storm@velona ~]$ ytcc -l debug subscribe "HappilyEverHanks" "https://www.youtube.com/@HappilyEverHanks"                                       [1692413350.220437] [MainProcess/MainThread] ytcc.config.DEBUG: Trying to read config from following locations: [PosixPath('/etc/ytcc/ytcc.conf'), PosixPath('/home/storm/.config/ytcc/ytcc.conf'), PosixPath('/home/storm/.ytcc.conf')][1692413350.222454] [MainProcess/MainThread] ytcc.config.DEBUG: Config was read from following locations: ['/home/storm/.config/ytcc/ytcc.conf'][1692413353.930687] [MainProcess/MainThread] ytcc.core.INFO: Performing update check on 10 playlist items[1692413353.931664] [MainProcess/MainThread] asyncio.DEBUG: Using selector: EpollSelector
[1692413354.171524] [MainProcess/MainThread] ytcc.updater.INFO: Checking playlist 'HappilyEverHanks'...
[1692413360.430457] [MainProcess/MainThread] ytcc.updater.WARNING: Publication date of video 'HappilyEverHanks - Live' of playlist 'HappilyEverHanks' is unknown
[1692413360.430896] [MainProcess/MainThread] ytcc.updater.WARNING: Duration of video 'HappilyEverHanks - Live' of playlist 'HappilyEverHanks' is unknown
[1692413360.431155] [MainProcess/MainThread] ytcc.updater.INFO: Processed video 'HappilyEverHanks - Live' of playlist 'HappilyEverHanks'
[1692413394.533273] [MainProcess/MainThread] ytcc.updater.WARNING: Publication date of video 'HappilyEverHanks - Videos' of playlist 'HappilyEverHanks' is unknown
[1692413394.533809] [MainProcess/MainThread] ytcc.updater.WARNING: Duration of video 'HappilyEverHanks - Videos' of playlist 'HappilyEverHanks' is unknown
[1692413394.534087] [MainProcess/MainThread] ytcc.updater.INFO: Processed video 'HappilyEverHanks - Videos' of playlist 'HappilyEverHanks'
[1692413394.573304] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA foreign_keys = ON;
[1692413394.574256] [MainProcess/MainThread] ytcc.database.DEBUG: CREATE TABLE tag ( name VARCHAR NOT NULL, playlist INTEGER REFERENCES playlist (id) ON DELETE CASCADE, CONSTRAINT tagKey PRIMARY KEY (name, playlist) );
[1692413394.614747] [MainProcess/MainThread] ytcc.database.DEBUG: CREATE TABLE playlist ( id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, name VARCHAR UNIQUE, url VARCHAR UNIQUE, reverse BOOLEAN );
[1692413394.672630] [MainProcess/MainThread] ytcc.database.DEBUG: CREATE TABLE content ( playlist_id INTEGER NOT NULL REFERENCES playlist (id) ON DELETE CASCADE, video_id INTEGER NOT NULL REFERENCES video (id) ON DELETE CASCADE, CONSTRAINT contentKey PRIMARY KEY (playlist_id, video_i
d) );                                                                                                                                         
[1692413394.724542] [MainProcess/MainThread] ytcc.database.DEBUG: CREATE TABLE video ( id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, title VARCHAR NOT NULL, url VARCHAR UNIQUE NOT NULL, description VARCHAR, duration FLOAT, publish_date FLOAT, watch_date FLOAT, thumbnail_url VARCHAR,
 extractor_hash VARCHAR UNIQUE );     
[1692413394.755450] [MainProcess/MainThread] ytcc.database.DEBUG: CREATE TABLE extractor_meta ( extractor_hash VARCHAR PRIMARY KEY, failure_count INTEGER );
[1692413394.807044] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA user_version = 5;
[1692413394.830349] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA user_version;
[1692413394.830911] [MainProcess/MainThread] ytcc.database.DEBUG: BEGIN      
[1692413394.831041] [MainProcess/MainThread] ytcc.database.DEBUG: INSERT INTO playlist (name, url, reverse) VALUES ('HappilyEverHanks', 'https://www.youtube.com/@HappilyEverHanks', 0);
[1692413394.831681] [MainProcess/MainThread] ytcc.database.DEBUG: COMMIT


[storm@velona ~]$ ytcc -l debug update                                                                                                         
[1692413400.438851] [MainProcess/MainThread] ytcc.config.DEBUG: Trying to read config from following locations: [PosixPath('/etc/ytcc/ytcc.conf'), PosixPath('/home/storm/.config/ytcc/ytcc.conf'), PosixPath('/home/storm/.ytcc.conf')]
[1692413400.440528] [MainProcess/MainThread] ytcc.config.DEBUG: Config was read from following locations: ['/home/storm/.config/ytcc/ytcc.conf']
[1692413400.444888] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA foreign_keys = ON;
[1692413400.445291] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA user_version;
[1692413400.445850] [MainProcess/MainThread] asyncio.DEBUG: Using selector: EpollSelector
[1692413400.447014] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT p.id AS id, p.name AS name, p.url AS url, p.reverse AS reverse, t.name AS tag FROM playlist AS p LEFT OUTER JOIN tag AS t ON p.id = t.playlist;
[1692413400.452170] [MainProcess/MainThread] ytcc.database.DEBUG: WITH ids as ( SELECT v.id FROM video AS v JOIN content c ON v.id = c.video_id JOIN playlist p ON p.id = c.playlist_id LEFT JOIN tag AS t ON p.id = t.playlist WHERE v.publish_date > 0 AND v.publish_date < Inf AND p.name
 IN ('HappilyEverHanks') ) SELECT v.id AS id, v.title AS title, v.url AS url, v.description AS description, v.duration AS duration, v.publish_date AS publish_date, v.watch_date AS watch_date, v.thumbnail_url AS thumbnail_url, v.extractor_hash AS extractor_hash, p.name AS playlist_nam
e, p.url AS playlist_url, p.reverse AS playlist_reverse FROM video AS v JOIN content c ON v.id = c.video_id JOIN playlist p ON p.id = c.playlist_id WHERE v.id in ids
[1692413401.578072] [MainProcess/MainThread] ytcc.updater.INFO: Checking playlist 'HappilyEverHanks'...
[1692413403.952282] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'youtubetab UChEzR-bdxoEpWdDLVC9wWew'
[1692413403.952661] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'youtubetab UChEzR-bdxoEpWdDLVC9wWew'
[1692413409.173781] [MainProcess/MainThread] ytcc.updater.WARNING: Publication date of video 'HappilyEverHanks - Live' of playlist 'HappilyEverHanks' is unknown
[1692413409.173978] [MainProcess/MainThread] ytcc.updater.WARNING: Duration of video 'HappilyEverHanks - Live' of playlist 'HappilyEverHanks' is unknown
[1692413409.174135] [MainProcess/MainThread] ytcc.updater.INFO: Processed video 'HappilyEverHanks - Live' of playlist 'HappilyEverHanks'
[1692413479.361800] [MainProcess/MainThread] ytcc.updater.WARNING: Publication date of video 'HappilyEverHanks - Videos' of playlist 'HappilyEverHanks' is unknown
[1692413479.362046] [MainProcess/MainThread] ytcc.updater.WARNING: Duration of video 'HappilyEverHanks - Videos' of playlist 'HappilyEverHanks' is unknown
[1692413479.362200] [MainProcess/MainThread] ytcc.updater.INFO: Processed video 'HappilyEverHanks - Videos' of playlist 'HappilyEverHanks'
[1692413479.362614] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT id FROM playlist WHERE name = 'HappilyEverHanks'
[1692413479.363093] [MainProcess/MainThread] ytcc.database.DEBUG: BEGIN
[1692413479.363455] [MainProcess/MainThread] ytcc.database.DEBUG: INSERT INTO video (title, url, description, duration, publish_date, watch_date, thumbnail_url, extractor_hash) VALUES ('HappilyEverHanks - Videos', 'https://www.youtube.com/@HappilyEverHanks/videos', 'WELCOME to HappilyEverHanks! We are Kyle (Carl), Renee (Ney Ney), & Dexter (Sexy Dexy) 😸 In 2017 we started traveling as ER nurses and soon realized that life had bigger plans for us. We have been living on the road full-time for over 5 years now and WOW...what a journey it has been! We are here to show you that life can be anything that you make it. We keep it real and fun every Thursday, and we encourage you to subscribe along for the journey! #ThanksHanks Learn more about our craziness below! ⬇️ ', -1, 169201.0, NULL, 'https://yt3.googleusercontent.com/8D4cgAbhYS-VsJ9IhejI2b7VA6pA2WOLnQ7UWrkgmSaDgaYrx7s_-TunpTHN9lwrjg3bH9jBMA=w2120-fcrop64=1,00000000ffffffff-k-c0xffffffff-no-nd-rj', 'youtubetab UChEzR-bdxoEpWdDLVC9wWew') ON CONFLICT (url) DO UPDATE SET title = 'HappilyEverHanks - Videos', url = 'https://www.youtube.com/@HappilyEverHanks/videos', description = 'WELCOME to HappilyEverHanks! We are Kyle (Carl), Renee (Ney Ney), & Dexter (Sexy Dexy) 😸 In 2017 we started traveling as ER nurses and soon realized that life had bigger plans for us. We have been living on the road full-time for over 5 years now and WOW...what a journey it has been! We are here to show you that life can be anything that you make it. We keep it real and fun every Thursday, and we encourage you to subscribe along for the journey! #ThanksHanks Learn more about our craziness below! ⬇️ ', duration = -1, publish_date = 169201.0, thumbnail_url = 'https://yt3.googleusercontent.com/8D4cgAbhYS-VsJ9IhejI2b7VA6pA2WOLnQ7UWrkgmSaDgaYrx7s_-TunpTHN9lwrjg3bH9jBMA=w2120-fcrop64=1,00000000ffffffff-k-c0xffffffff-no-nd-rj', extractor_hash = 'youtubetab UChEzR-bdxoEpWdDLVC9wWew'
[1692413479.364290] [MainProcess/MainThread] ytcc.database.DEBUG: INSERT OR IGNORE INTO content (playlist_id, video_id) VALUES (1,(SELECT id FROM video WHERE url = 'https://www.youtube.com/@HappilyEverHanks/videos'));
[1692413479.364528] [MainProcess/MainThread] ytcc.database.DEBUG: COMMIT
[1692413479.416085] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT id FROM playlist WHERE name = 'HappilyEverHanks'
[1692413479.416528] [MainProcess/MainThread] ytcc.database.DEBUG: BEGIN
[1692413479.416800] [MainProcess/MainThread] ytcc.database.DEBUG: INSERT INTO video (title, url, description, duration, publish_date, watch_date, thumbnail_url, extractor_hash) VALUES ('HappilyEverHanks - Live', 'https://www.youtube.com/@HappilyEverHanks/streams', 'WELCOME to HappilyEverHanks! We are Kyle (Carl), Renee (Ney Ney), & Dexter (Sexy Dexy) 😸 In 2017 we started traveling as ER nurses and soon realized that life had bigger plans for us. We have been living on the road full-time for over 5 years now and WOW...what a journey it has been! We are here to show you that life can be anything that you make it. We keep it real and fun every Thursday, and we encourage you to subscribe along for the journey! #ThanksHanks Learn more about our craziness below! ⬇️ ', -1, 169201.0, NULL, 'https://yt3.googleusercontent.com/8D4cgAbhYS-VsJ9IhejI2b7VA6pA2WOLnQ7UWrkgmSaDgaYrx7s_-TunpTHN9lwrjg3bH9jBMA=w2120-fcrop64=1,00000000ffffffff-k-c0xffffffff-no-nd-rj', 'youtubetab UChEzR-bdxoEpWdDLVC9wWew') ON CONFLICT (url) DO UPDATE SET title = 'HappilyEverHanks - Live', url = 'https://www.youtube.com/@HappilyEverHanks/streams', description = 'WELCOME to HappilyEverHanks! We are Kyle (Carl), Renee (Ney Ney), & Dexter (Sexy Dexy) 😸 In 2017 we started traveling as ER nurses and soon realized that life had bigger plans for us. We have been living on the road full-time for over 5 years now and WOW...what a journey it has been! We are here to show you that life can be anything that you make it. We keep it real and fun every Thursday, and we encourage you to subscribe along for the journey! #ThanksHanks Learn more about our craziness below! ⬇️ ', duration = -1, publish_date = 169201.0, thumbnail_url = 'https://yt3.googleusercontent.com/8D4cgAbhYS-VsJ9IhejI2b7VA6pA2WOLnQ7UWrkgmSaDgaYrx7s_-TunpTHN9lwrjg3bH9jBMA=w2120-fcrop64=1,00000000ffffffff-k-c0xffffffff-no-nd-rj', extractor_hash = 'youtubetab UChEzR-bdxoEpWdDLVC9wWew'
[1692413479.417094] [MainProcess/MainThread] ytcc.database.DEBUG: ROLLBACK
[1692413479.422798] [MainProcess/MainThread] ytcc.cli.ERROR: Cannot connect to the database or query failed unexpectedly
[1692413479.423356] [MainProcess/MainThread] ytcc.cli.DEBUG: Unknown database error
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/ytcc/cli.py", line 742, in main
    exit_code = cli.main(standalone_mode=False)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ytcc/cli.py", line 357, in update
    ytcc.update(max_fail, max_backlog)
  File "/usr/lib/python3.11/site-packages/ytcc/core.py", line 152, in update
    updater.update()
  File "/usr/lib/python3.11/site-packages/ytcc/updater.py", line 246, in update
    asyncio.run(self.do_update())
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ytcc/updater.py", line 243, in do_update
    await asyncio.gather(*map(self.update_playlist, playlists))
  File "/usr/lib/python3.11/site-packages/ytcc/updater.py", line 237, in update_playlist
    self.database.add_videos([video], playlist)
  File "/usr/lib/python3.11/site-packages/ytcc/database.py", line 349, in add_videos
    cursor.execute(insert_video, asdict(video))
sqlite3.IntegrityError: UNIQUE constraint failed: video.extractor_hash

Note that the issue still occurs even when using the channel id (UChEzR-bdxoEpWdDLVC9wWew) instead of the @ version.

Issue does not occur with eg. this channel: https://www.youtube.com/channel/UCSqUTKab5R1I4B5frDEjXzA

The only notable difference I can see with a quick glance is that the channel causing the bug has the "Live" tab while the second one has not.

@chemicalstorm
Copy link

I've dig a bit in the code and seems to be indeed linked to a difference in what is returned by yt-dlp for the 2 channels:

Channel that works as expected

The info structure retrieved in https://github.com/woefe/ytcc/blob/master/ytcc/updater.py#L82 contains an entries field that is a generator that returns object of _type set to url and whose url field is indeed the video.

Channel that causes the issue

The entries field of the info structure contains 2 objects of type playlist, one for the "Videos" tab and one for the "Live" tab. Obviously, the url field for these only points to the channel page's tab, not to any video. These objects then follow the same pattern as for the other channel, ie. the entries field is a generator that does return a list of objects with the url corresponding to actual video url.

chemicalstorm added a commit to chemicalstorm/ytcc that referenced this issue Aug 19, 2023
@chemicalstorm
Fix Youtube channel with tabs
dee1f1f 
Some Youtube channels will have tags (Videos, Lives, Streams, Shorts,
etc), and updating such a channel was broken as the object returned by
YoutubeDL.extract_info will contains a "sub playlist".
Fix the issue by checking if we have sub playlists and finding the one
that is used for videos.

Note: this means we don't support any types other than Videos for now.

Fixes woefe#113
chemicalstorm added a commit to chemicalstorm/ytcc that referenced this issue Aug 19, 2023
@chemicalstorm
Fix Youtube channel with tabs
d46c3be 
Youtube channel can enable additional features (eg. Shorts) which
results in the channel having corresponding tabs.
Updating such a channel was broken as the object returned by
YoutubeDL.extract_info will contains a "sub playlist".
Fix the issue by checking if we have sub playlists and finding the one
that is used for videos.

Note: this means we don't support any types other than Videos for now.

Fixes woefe#113
@chemicalstorm chemicalstorm linked a pull request Aug 19, 2023 that will close this issue
Draft
@chemicalstorm
Copy link

There is an easy fix: update the subscription to use the playlist/link to the "videos" tab. Unfortunately, for some users that means having to update multiple subscriptions and there is currently no way of being notified that this even needs to be done.

I've tried to do a small workaround in #118 so it's possible to support channels that were already added to ytcc and that enabled the new features at some points, making them use "tabs".

Let me know what you think!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix Youtube channel with tabs chemicalstorm/ytcc
3 participants
@chemicalstorm @woefe @pghpete