In words
What it is, why it matters, and what it is like.
Why am I learning this?
You will learn how to recognize that two separate rows of information actually belong to the same person, company, or product — even when those rows come from different software systems and share no common identification number. This skill is the key to fixing duplicate customer entries in a sales database, linking a bank’s internal records with external credit reports, and combining data after one business acquires another. Without this step, your reports will show inflated customer counts, missed opportunities for personalized service, and confused financial records. It matters now because clean, unified data is the only way to trust any analysis or automated process that relies on it.
The idea, in plain terms
Think of the last time you tried to find a friend on a social network or a professional directory. You typed a name and received several profiles as results. You had to look at the details to decide which one was actually them. Maybe one result had the correct name but a profile photo from a different city, suggesting it was someone else. Another result had the same job title and company but no profile photo, making it ambiguous. A third result matched the name and the city perfectly but showed a different last name because the person had recently changed their marital status. You combined these clues — name similarity, location match, professional history — to make a judgment call. Entity resolution is simply automating that specific judgment call for millions of records at once.
It is the process of deciding when two rows of data refer to the same real-world thing, like a person, a company, or a product, when neither row carries a unique identifier you can just match on directly. In an ideal world, every system would use one standard ID, like a national identity number for people or a tax identification number for businesses. In reality, the same company might be listed as 'Reliance Industries' in one database, 'RIL' in another, and 'Reliance Industries Ltd.' in a third. The same customer might be listed as 'A. Sharma' in one system and 'Anita Sharma' in another, with a different phone number because they moved house. Entity resolution is the discipline of bringing those records together without a shared key by comparing the fields you do have and scoring how likely it is that they are the same thing.
An analogy
Imagine a busy railway station in a city like Mumbai, where a single person has bought tickets from three different counters, each run by a separate company. The first ticket says 'A. Sharma, Phone 98765 43210'. The second says 'Anita Sharma, Phone 98760 43210' — a typo in the last digit. The third says 'Anita S., Address: Malad West'. There is no ticket number shared across all three, so the system cannot just match on that identifier. A human clerk at a single desk, if they had access to all three, could look at the name, the phone number, and the address, and say, 'These are clearly the same person, let me merge them.' But if the station handled a million passengers a day, no team of clerks could do this by eye. Entity resolution is the automated clerk. It looks at pairs of records, compares each field — name, phone, address — and assigns a similarity score to each comparison. Then it combines those field-level scores into an overall score, and decides whether the records match, are definitely different, or are too close to call and need a human to check.
The analogy holds well because the clerk uses common sense, not an exact key. It breaks down in two crucial ways. First, a clerk can understand context: they know 'S.' is a common abbreviation for 'Sharma'. An automated system does not, unless you teach it. Second, a clerk can handle a handful of records; the station handles a million. The automated clerk must compare records without checking every possible pair, because a million records would mean five hundred billion comparisons, which would take too long. To solve this, the system uses 'blocking': a technique where records are grouped into smaller buckets based on loose criteria (for example, only comparing names that start with the same letter) so it never has to check pairs that could possibly be different.
The analogy stops working when you ask the clerk to do the impossible: check every pair. Automated systems must be smarter about which pairs to look at.
Definition
Entity resolution is the process of determining when different data records refer to the same real-world entity, in the absence of a shared unique identifier, by comparing the fields in the records and deciding whether they match.
Where this sits
This topic sits inside your Data Engineering note, which you already have. It connects directly to defining strict rules for incoming data: when teams agree on exactly which fields must be populated and how they should be formatted, entity resolution becomes reliable because there is clear information to compare. It also connects to viewing all your data as one unified map: when you know that 'customer 42' in one system is the same person as 'customer 73' in another, you can merge queries and see the complete history of that customer rather than fragmented glimpses.
