$pip install LatinFixer
from LatinFixer import LatinFix
lfix = LatinFix(text, debug=False)
nw = (
lfix.apply_n_escaped()
# ('\\226\\132\\162', '™'),('\\226\\130\\172', '€'),('\\226\\128\\186', '›'),('\\226\\128\\185', '‹'),('\\226\\128\\176', '‰'),('\\226\\128\\166', '…'),('\\226\\128\\162', '•')
.remove_non_printable_chars()
# \x00 ...
.apply_wrong_chars()
# ('’', '’'),('—', '—'),('–', '–'),('‘', '‘'),('„', '„'),('‚', '‚'),('“', '“') ...
.apply_x_69_lower_case_escaped()
# ('\\xe2\\x84\\xa2', '™'),('\\xe2\\x82\\xac', '€'),('\\xe2\\x80\\xba', '›'),('\\xe2\\x80\\xb9', '‹'),('\\xe2\\x80\\xb0', '‰'),('\\xe2\\x80\\xa6', '…') ...
.apply_x_69_upper_case_escaped()
# ('\\xE2\\x84\\xA2', '™'),('\\xE2\\x82\\xAC', '€'),('\\xE2\\x80\\xBA', '›'),('\\xE2\\x80\\xB9', '‹'),('\\xE2\\x80\\xB0', '‰')...
.apply_x_3_lower_case_escaped()
# ('\\xff', 'ÿ'),('\\xfe', 'þ'),('\\xfd', 'ý'),('\\xfc', 'ü'),('\\xfb', 'û'),('\\xfa', 'ú'),('\\xf9', 'ù'),('\\xf8', 'ø')...
.apply_x_3_upper_case_escaped()
# ('\\xFF', 'ÿ'),('\\xFE', 'þ'),('\\xFD', 'ý'),('\\xFC', 'ü'),('\\xFB', 'û'),('\\xFA', 'ú'),('\\xF9', 'ù'),('\\xF8', 'ø'),('\\xF7', '÷') ...
.apply_u_4_upper_case_escaped()
# ('\\u2122', '™'),('\\u20AC', '€'),('\\u203A', '›'),('\\u2039', '‹'),('\\u2030', '‰'),('\\u2026', '…'),('\\u2022', '•') ...
.apply_u_4_lower_case_escaped()
# ('\\u2122', '™'),('\\u20ac', '€'),('\\u203a', '›'),('\\u2039', '‹'),('\\u2030', '‰'),('\\u2026', '…'),('\\u2022', '•'),('\\u2021', '‡')...
.apply_zerox_unescaped_lower()
# ('0xff', 'ÿ'),('0xfe', 'þ'),('0xfd', 'ý'),('0xfc', 'ü'),('0xfb', 'û'),('0xfa', 'ú'),('0xf9', 'ù'),('0xf8', 'ø')....
.apply_zerox_unescaped_upper()
# ('0xFF', 'ÿ'),('0xFE', 'þ'),('0xFD', 'ý'),('0xFC', 'ü'),('0xFB', 'û'),('0xFA', 'ú'),('0xF9', 'ù')...
.apply_html_character_reference()
# (' ', ' '),('!', '!'),('"', '"'),('#', '#'),('$', '$'),('%', '%'),('&', '&'),(''', "'"),('(', '('),(')', ')'),('*', '*')...
.apply_html_entity_reference()
# ('ÿ', 'ÿ'),('¥', '¥'),('ý', 'ý'),('|', '|'),('ü', 'ü'),('¨', '¨'),('ù', 'ù'),('û', 'û')...
.delete_all_non_latin_chars()
.replace_multispaces()
)
Original: Suzy & John "
Repaired: Suzy & John "
Original: £682m
Repaired: £682m
Original: \u00FF\u00FF\u00F0\u00f0\x95\xFF
Repaired: ÿÿððÿ
Original: Smörgås
Repaired: Smörgås
Original: Non ti suscita niente la parola pietÃ\xa0
Repaired: Non ti suscita niente la parola pietí
Original: RosŽ
Repaired: Ros
Original: RUF MICH ZURÃœCK.
Repaired: RUF MICH ZURÜCK.
Original: aqu\195\173
Repaired: aquí
Original: 09. Bát Nhã Tâm Kinh
Repaired: 09. Bát Nhã Tâm Kinh
Original: criança
Repaired: criança
Original: Koç University
Repaired: Koç University
Original: Technische Universität Dresden
Repaired: Technische Universität Dresden
Original: Universität für Musik und darstellende Kunst Wien
Repaired: Universität für Musik und darstellende Kunst Wien
Original: Technische Universität Wien
Repaired: Technische Universität Wien
Original: Ã\x89cole Nationale Supérieure des Beaux-Arts Paris
Repaired: ícole Nationale Supérieure des Beaux-Arts Paris
Original: Universidad Simón BolÃ\xadvar (USB)
Repaired: Universidad Simón Bolí var (USB)
Original: PontifÃ\xadcia Universidade Católica do Rio Grande do Sul (PUCRS)
Repaired: Pontifí cia Universidade Católica do Rio Grande do Sul (PUCRS)
Original: Bogaziçi Ã\x9cniversitesi
Repaired: Bogaziçi íniversitesi
Original: UniversitÃ\xa0 degli Studi di Udine
Repaired: Universití degli Studi di Udine
Original: Universitat Autònoma de Barcelona
Repaired: Universitat Autònoma de Barcelona
Original: Université de Rennes 1
Repaired: Université de Rennes 1
Original: Ã\x89cole Normale Supérieure de Lyon
Repaired: ícole Normale Supérieure de Lyon
Original: Ã\x89cole Nationale Supérieure de Création Industrielle
Repaired: ícole Nationale Supérieure de Création Industrielle
Original: ENSCI Les Ateliers
Repaired: ENSCI Les Ateliers
Original: Universität Bremen
Repaired: Universität Bremen
Original: Institut National des Sciences Appliquées de Lyon (INSA)
Repaired: Institut National des Sciences Appliquées de Lyon (INSA)
Original: Université Laval
Repaired: Université Laval
Original: Universität des Saarlandes
Repaired: Universität des Saarlandes
Original: Universität Konstanz
Repaired: Universität Konstanz
Original: Philipps-Universität Marburg
Repaired: Philipps-Universität Marburg
Original: El Colegio de México A.C.
Repaired: El Colegio de México A.C.
Original: Humboldt-Universität zu Berlin
Repaired: Humboldt-Universität zu Berlin
Original: PontifÃ\xadcia Universidade Católica do Rio de Janeiro
Repaired: Pontifí cia Universidade Católica do Rio de Janeiro
Original: Universidade Federal do Paraná - UFPR
Repaired: Universidade Federal do Paraná - UFPR
Original: Universität Potsdam
Repaired: Universität Potsdam
Original: USI - Università della Svizzera italiana
Repaired: USI - Universití della Svizzera italiana
Original: Palacký University Olomouc
Repaired: Palacký University Olomouc
Original: CentraleSupélec
Repaired: CentraleSupélec
Original: Arts et Métiers ParisTech
Repaired: Arts et Métiers ParisTech
Original: Université de Sherbrooke
Repaired: Université de Sherbrooke
Original: UniversitÃ\xa0 degli studi Roma Tre
Repaired: Universití degli studi Roma Tre
Original: Westfälische Wilhelms-Universität Münster
Repaired: Westfälische Wilhelms-Universität Münster
Original: Universidad Politécnica de Madrid (UPM)
Repaired: Universidad Politécnica de Madrid (UPM)
Original: Universidad Adolfo IbÃ\xa0ñez
Repaired: Universidad Adolfo Ibíñez
Original: Ã\x89cole Centrale de Lille
Repaired: ícole Centrale de Lille
Original: Université Paris 13 Nord
Repaired: Université Paris 13 Nord
Original: Università degli Studi di Udine
Repaired: Universití degli Studi di Udine
Original: Universidade Federal de São Paulo
Repaired: Universidade Federal de São Paulo
Original: Instituto Nacional de Matemática Pura e Aplicada (IMPA)
Repaired: Instituto Nacional de Matemática Pura e Aplicada (IMPA)
Original: Universität Mannheim
Repaired: Universität Mannheim
Original: Université Toulouse 1 Capitole
Repaired: Université Toulouse 1 Capitole
Original: Technische Universität Braunschweig
Repaired: Technische Universität Braunschweig
Original: Eberhard Karls Universität Tübingen
Repaired: Eberhard Karls Universität Tübingen
Original: Universität Rostock
Repaired: Universität Rostock
Original: Université Grenoble Alpes
Repaired: Université Grenoble Alpes
Original: Université de Fribourg
Repaired: Université de Fribourg
Original: Universität Innsbruck
Repaired: Universität Innsbruck
Original: Universidad Adolfo Ibà ñez
Repaired: Universidad Adolfo Ibí ñez
Original: Université du Québec
Repaired: Université du Québec
Original: Universidad de la República (Udelar)
Repaired: Universidad de la República (Udelar)
Original: Universitat Politècnica de Catalunya · BarcelonaTech (UPC)
Repaired: Universitat Politècnica de Catalunya · BarcelonaTech (UPC)
Original: Universität Regensburg
Repaired: Universität Regensburg
Original: Université de Paris
Repaired: Université de Paris
Original: Université Paris 1 Panthéon-Sorbonne
Repaired: Université Paris 1 Panthéon-Sorbonne
Original: Universidad Técnica Federico Santa MarÃ\xada (USM)
Repaired: Universidad Técnica Federico Santa Marí a (USM)
Original: Ruprecht-Karls-Universität Heidelberg
Repaired: Ruprecht-Karls-Universität Heidelberg
Original: Pontificia Universidad Católica Argentina
Repaired: Pontificia Universidad Católica Argentina
Original: UniversitÃ\xa0Â\xa0di Padova
Repaired: Universití di Padova
Original: Technische Universität Berlin (TU Berlin)
Repaired: Technische Universität Berlin (TU Berlin)
Original: Universität Stuttgart
Repaired: Universität Stuttgart
Original: Fundação Getulio Vargas (FGV)
Repaired: Fundação Getulio Vargas (FGV)
Original: Universidade de São Paulo
Repaired: Universidade de São Paulo
Original: Universidad Nacional Autónoma de México (UNAM)
Repaired: Universidad Nacional Autónoma de México (UNAM)
Original: Universidade Federal de São Carlos (UFSCar)
Repaired: Universidade Federal de São Carlos (UFSCar)
Original: Ã\x89cole Centrale de Nantes
Repaired: ícole Centrale de Nantes
Original: Technische Universität Kaiserslautern
Repaired: Technische Universität Kaiserslautern
Original: Università degli studi Roma Tre
Repaired: Universití degli studi Roma Tre
Original: Pontificia Universidad Católica del Perú
Repaired: Pontificia Universidad Católica del Perú
Original: UniversitÃ\xa0 degli Studi di Pavia
Repaired: Universití degli Studi di Pavia
Original: Université PSL
Repaired: Université PSL
Original: Université de Montréal
Repaired: Université de Montréal
Original: Pontificia Universidad Católica de ValparaÃ\xadso
Repaired: Pontificia Universidad Católica de Valparaí so
Original: University Paris 2 Panthéon-Assas
Repaired: University Paris 2 Panthéon-Assas
Original: Université Paris-Nanterre
Repaired: Université Paris-Nanterre
Original: Universidad Autónoma de San Luis de PotosÃ\xad
Repaired: Universidad Autónoma de San Luis de Potosí
Original: Universität Leipzig
Repaired: Universität Leipzig
Original: Ruhr-Universität Bochum
Repaired: Ruhr-Universität Bochum
Original: Université Lumière Lyon 2
Repaired: Université Lumière Lyon 2
Original: Université de Lille
Repaired: Université de Lille
Original: Université Claude Bernard Lyon 1
Repaired: Université Claude Bernard Lyon 1
Original: Université catholique de Louvain (UCLouvain)
Repaired: Université catholique de Louvain (UCLouvain)
Original: Université de Technologie Troyes (UTT)
Repaired: Université de Technologie Troyes (UTT)
Original: Universidad de San Andrés - UdeSA
Repaired: Universidad de San Andrés - UdeSA
Original: Martin-Luther-Universität Halle-Wittenberg
Repaired: Martin-Luther-Universität Halle-Wittenberg
Original: University of Tromsø The Arctic University of Norway
Repaired: University of Tromsø The Arctic University of Norway
Original: Rheinische Friedrich-Wilhelms-Universität Bonn
Repaired: Rheinische Friedrich-Wilhelms-Universität Bonn
Original: Universidad de Alcalá
Repaired: Universidad de Alcalá
Original: USI - UniversitÃ\xa0 della Svizzera italiana
Repaired: USI - Universití della Svizzera italiana
Original: Linköping University
Repaired: Linköping University
Original: Universidad Nacional de Córdoba - UNC
Repaired: Universidad Nacional de Córdoba - UNC
Original: UniversitÃ\xa0 degli Studi di Perugia
Repaired: Universití degli Studi di Perugia
Original: Università degli Studi di Pavia
Repaired: Universití degli Studi di Pavia
Original: Johannes Gutenberg Universität Mainz
Repaired: Johannes Gutenberg Universität Mainz
Original: Università Iuav di Venezia
Repaired: Universití Iuav di Venezia
Original: Friedrich-Alexander-Universität Erlangen-Nürnberg
Repaired: Friedrich-Alexander-Universität Erlangen-Nürnberg
Original: Université de Nantes
Repaired: Université de Nantes
Original: Universidad de Córdoba
Repaired: Universidad de Córdoba
Original: Universidade de BrasÃ\xadlia
Repaired: Universidade de Brasí lia
Original: Université de Strasbourg
Repaired: Université de Strasbourg
Original: Universidad Autónoma de Nuevo León
Repaired: Universidad Autónoma de Nuevo León
Original: Pontificia Universidad Católica de Chile (UC)
Repaired: Pontificia Universidad Católica de Chile (UC)
Original: Université Paris-Est Créteil Val de Marne
Repaired: Université Paris-Est Créteil Val de Marne
Original: Universidad Autónoma del Estado de México (UAEMex)
Repaired: Universidad Autónoma del Estado de México (UAEMex)
Original: Université de Montpellier
Repaired: Université de Montpellier
Original: Universität der Künste Berlin
Repaired: Universität der Künste Berlin
Original: Università  di Padova
Repaired: Universití di Padova
Original: Université Paris-Saclay
Repaired: Université Paris-Saclay
Original: Eötvös Loránd University
Repaired: Eötvös Loránd University
Original: Technische Universität Bergakademie Freiberg
Repaired: Technische Universität Bergakademie Freiberg
Original: Technische Universität Hamburg
Repaired: Technische Universität Hamburg
Original: Universidade Católica Portuguesa - UCP
Repaired: Universidade Católica Portuguesa - UCP
Original: Ã\x89cole Nationale Supérieure des Industries Chimiques (ENSIC) Nancy
Repaired: ícole Nationale Supérieure des Industries Chimiques (ENSIC) Nancy
Original: Instituto Tecnológico Autónomo de México (ITAM)
Repaired: Instituto Tecnológico Autónomo de México (ITAM)
Original: Université de Limoges
Repaired: Université de Limoges
Original: Université Sorbonne Nouvelle Paris 3
Repaired: Université Sorbonne Nouvelle Paris 3
Original: Université Paul Sabatier Toulouse III
Repaired: Université Paul Sabatier Toulouse III
Original: Julius-Maximilians-Universität Würzburg
Repaired: Julius-Maximilians-Universität Würzburg
Original: Université de Poitiers
Repaired: Université de Poitiers
Original: Universitat Politècnica de València
Repaired: Universitat Politècnica de València
Original: UniversitÃ\xa0Â\xa0Cattolica del Sacro Cuore
Repaired: Universití Cattolica del Sacro Cuore
Original: Université Nice Sophia Antipolis
Repaired: Université Nice Sophia Antipolis
Original: University of Jyväskylä
Repaired: University of Jyväskylä
Original: Bauhaus-Universität Weimar
Repaired: Bauhaus-Universität Weimar
Original: Université de Liège
Repaired: Université de Liège
Original: Universität Jena
Repaired: Universität Jena
Original: University of Göttingen
Repaired: University of Göttingen
Original: Technische Universität Ilmenau
Repaired: Technische Universität Ilmenau
Original: Ã\x89cole Centrale de Lyon
Repaired: ícole Centrale de Lyon
Original: Ludwig-Maximilians-Universität München
Repaired: Ludwig-Maximilians-Universität München
Original: Université de Lorraine
Repaired: Université de Lorraine
Original: Université de Technologie de Compiègne (UTC)
Repaired: Université de Technologie de Compiègne (UTC)
Original: Universität Siegen
Repaired: Universität Siegen
Original: Universität Duisburg-Essen
Repaired: Universität Duisburg-Essen
Original: Université de Savoie
Repaired: Université de Savoie
Original: Universidad Autónoma de Madrid
Repaired: Universidad Autónoma de Madrid
Original: Università  Cattolica del Sacro Cuore
Repaired: Universití Cattolica del Sacro Cuore
Original: Ankara Ã\x9cniversitesi
Repaired: Ankara íniversitesi
Original: Universidade da Coruña
Repaired: Universidade da Coruña
Original: Università degli Studi di Perugia
Repaired: Universití degli Studi di Perugia
Original: Hochschule für Gestaltung und Kunst Zürich
Repaired: Hochschule für Gestaltung und Kunst Zürich
Original: Universität Hamburg
Repaired: Universität Hamburg
-
Notifications
You must be signed in to change notification settings - Fork 0
Repair badly decoded latin strings \x00 | \226\130\172 | “ | \xe2\x84\xa2 | \u2122 | | ÿ
License
hansalemaos/LatinFixer
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
About
Repair badly decoded latin strings \x00 | \226\130\172 | “ | \xe2\x84\xa2 | \u2122 | | ÿ