Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update MARC importer language mapping table #8140

Closed
tfmorris opened this issue Jul 27, 2023 · 1 comment · Fixed by #9344
Closed

Update MARC importer language mapping table #8140

tfmorris opened this issue Jul 27, 2023 · 1 comment · Fixed by #9344
Labels
Lead: @hornc Issues overseen by Charles (Staff: Data Engineering Lead) [managed] Needs: Response Issues which require feedback from lead Needs: Triage This issue needs triage. The team needs to decide who should own it, what to do, by when. [managed] Theme: MARC records Type: Bug Something isn't working. [managed]

Comments

@tfmorris
Copy link
Contributor

The language mapping table in the MARC importer should be updated to remap the obsolete language codes listed in #8139

Additionally any code prefixed with a hyphen on this LoC MARC code list (or which doesn't appear on the list) should be prohibited on import.

openlibrary/openlibrary/catalog/marc/parse.py

Lines 278 to 291 in b100bf0

lang_map = {
'ser': 'srp', # https://www.archive.org/details/zadovoljstvauivo00lubb
'end': 'eng',
'enk': 'eng',
'ent': 'eng',
'cro': 'chu',
'jap': 'jpn',
'fra': 'fre',
'gwr': 'ger',
'sze': 'slo',
'fr ': 'fre',
'fle': 'dut', # Flemish -> Dutch
'it ': 'ita',
}

Stakeholders

@hornc
@tfmorris tfmorris added Needs: Lead Needs: Triage This issue needs triage. The team needs to decide who should own it, what to do, by when. [managed] Type: Bug Something isn't working. [managed] labels Jul 27, 2023
@cdrini cdrini added Lead: @cdrini Issues overseen by Drini (Staff: Team Lead & Solr, Library Explorer, i18n) [managed] Lead: @hornc Issues overseen by Charles (Staff: Data Engineering Lead) [managed] and removed Needs: Lead Lead: @cdrini Issues overseen by Drini (Staff: Team Lead & Solr, Library Explorer, i18n) [managed] labels Jul 31, 2023
@stopregionblocking stopregionblocking mentioned this issue Jun 17, 2024
Open
@tfmorris
Copy link
Contributor Author

What additional information is needed for this to be triaged, or, better yet, resolved?

@github-actions github-actions bot added the Needs: Response Issues which require feedback from lead label May 24, 2024
hornc added a commit to hornc/openlibrary-1 that referenced this issue May 26, 2024
@hornc
update MARC import language mapping
fa48f2b 
closes internetarchive#8140
@hornc hornc mentioned this issue May 26, 2024
Merged
mekarpeles pushed a commit that referenced this issue May 28, 2024
@hornc
Update MARC importer language mapping table (#9344)
3b5ef43 
* remove uncommented /unjustified replacements (one off typos?)
* update MARC import language mapping
closes #8140
@hornc hornc mentioned this issue Jun 1, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Lead: @hornc Issues overseen by Charles (Staff: Data Engineering Lead) [managed] Needs: Response Issues which require feedback from lead Needs: Triage This issue needs triage. The team needs to decide who should own it, what to do, by when. [managed] Theme: MARC records Type: Bug Something isn't working. [managed]
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update MARC importer language mapping table hornc/openlibrary-1
3 participants
@tfmorris @mekarpeles @cdrini