A powerful and modular toolkit for record linkage and duplicate detection in Python
-
Updated
Feb 21, 2024 - Python
Entity resolution (also known as data matching, data linkage, record linkage, and many other terms) is the task of finding entities in a dataset that refer to the same entity across different data sources (e.g., data files, books, websites, and databases). Entity resolution is necessary when joining different data sets based on entities that may or may not share a common identifier (e.g., database key, URI, National identification number), which may be due to differences in record shape, storage location, or curator style or preference.
A powerful and modular toolkit for record linkage and duplicate detection in Python
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
Record linking package that fuzzy matches two Python pandas dataframes using sqlite3 fts4
A list of free data matching and record linkage software.
🔎 Finds fuzzy matches between CSV files
Resources for tackling record linkage / deduplication / data matching problems
PyTorch library for transforming entities like companies, products, etc. into vectors to support scalable Record Linkage / Entity Resolution using Approximate Nearest Neighbors.
An open-source library that leverages Python’s data science ecosystem to build powerful end-to-end Entity Resolution workflows.
Link Wikidata items to large catalogs
A browser user interface for manual labeling of record pairs.
Compound AI toolchain for fast and accurate entity matching, powered by LLMs.
Fuzzy string matching in R. Inspired by Python's thefuzz (but without the Python).
Curated list of awesome software and resources for Senzing, The First Real-Time AI for Entity Resolution.
Welcome to Snowman App – a Data Matching Benchmark Platform.
This projects aims to provide lists containing only great movies to users based only a gew filters and search parameters.
Crawl, matching and explore data about jobs in Viet Nam.
WInte.r is a Java framework for end-to-end data integration. The WInte.r framework implements well-known methods for data pre-processing, schema matching, identity resolution, data fusion, and result evaluation.
ProxCluster is a framework for Incremental Entity Resolution that leverages concepts similar to K-Means for clustering duplicates. This work was developed as the final paper for my Bachelor degree in Computer Science
Repository for CS 838 (Spring 2017) Data Science project
Created by Halbert L. Dunn
Released 1946