Technical Deep Dives
The Many Rodgers: How Entity Resolution Handles Scanning Chaos
TLDR A software tool that uses statistics to decide which database records refer to the same person (called Splink) merged eight or more scanning variants of...
#entity resolution
4 investigations
Data Quality
The Singleton Crisis: 197,945 Entities That Appear Once
TLDR Nearly one in four entities in the corpus (197,945 out of 821,633) appears in only one document. An entity that appears in only one document — called a...
Data Quality
Garbage Cluster Dissolution: When Entity Resolution Connects Noise to Noise
TLDR A cleanup script (Script 19b) dissolved 8 entity groups containing 3,576 records that had been incorrectly merged because unreadable text that the...
Methodology
519K Splink Clusters: Turning 2.38 Million Fragments Into One Namespace
TLDR Probabilistic entity resolution using a statistical matching method reduced 2.38 million raw Named Entity Recognition entries to 519,000 unified clusters...