Improve accuracy of show language identification. #7
Labels
bug
Something isn't working
Python - scraper
Issues related to the python scraping / data processing portion of the code base.
At present, in the DB
shows
.lang
is populated during the ai_ep_summary process.Generally this seems to work well. However, the results are not always what is desired. For example: An English show that plays world music may be incorrectly labelled. This is currently the case for
show
.'id
= 146152.A tricky component of this is it's computationally expensive to transcribe an entire hour+ long episode. So I have been determining language based on short snippets. A show may also contain station ads that don't necessarily reflect the language of the show itself.
There is also one unusual case where a show is officially trilingual (
show
.'id
= 3033), and a show that is half English and half French(show
.id
= 269938)It's probably too much to handle all of the rare special cases, but some improvement on the current situation might be possible.
A loosely related issue is that I don't wish to create summaries for shows that are music only. As a result I've manually entered the value 'skip' in the lang column for such shows as a short term band aid solution.
The text was updated successfully, but these errors were encountered: