A bookshop receives the same novel from three suppliers.
One calls it Things Fall Apart. Another sends Things Fall Apart: A Novel. A third puts the author’s name first, adds the publisher, and gives the edition an ISBN.
The shop wants a reader to find the book. It wants staff to see its stock. It wants to avoid showing the same edition three times.
So it gives the book a name it can use consistently.
That sounds like cleaning up. It is more than that.
The shop has made a decision about what those incoming references mean, which differences matter, and which name will stand for the thing in its own catalogue. It has also had to be careful. A paperback, a hardback, an audiobook, and a new edition may all be connected to the same work. They are not necessarily the same thing to sell.
This is the part of entity resolution I have been trying to understand better. We talk about matching records, grouping products, and using AI to make sense of messy data. Before any of that, somebody has to decide what a name should mean here.
Four jobs that get put in one box
The words are close enough to be confusing. The jobs are not.
| Job | The question | Example |
|---|---|---|
| Normalization | How can we make this easier to compare? | COCA-COLA becomes coca cola for comparison. |
| Entity resolution | Do these references point to the same real-world thing? | Does this supplier row describe the same company as that invoice row? |
| Canonicalization | Which representation will we use for this thing, for this purpose? | Use The Coca-Cola Company as the preferred company name. |
| Categorization | What kind of thing is this? | This company is a beverage manufacturer. |
Normalization cleans a reference. It may remove a full stop, standardise a postcode, or make two kinds of apostrophe behave alike. It does not prove that two records refer to one thing.
Entity resolution makes that harder claim. It asks whether two references, written by different people or systems, point to the same entity in the world. What kind of thing are you resolving? is the distinction I keep coming back to: the entity is the thing in the world; the row, listing, or label is only a reference to it.
Canonicalization happens when a system says, this is how we will represent that entity here. A category does something else again. It puts an entity into a useful group for a job.
The order often looks like this:
raw reference → normalize → find candidates → resolve → choose a canonical representation → categorize
But real systems loop.
Once a bookshop has learned that a supplier’s T. Fall Apart means a particular book, that known variant can help it find the right candidate next time. The remembered name helps the later match. The later match may give the shop a new name to remember.
So canonicalization is not simply one tidy step before or after entity resolution. It is part of the memory that makes resolution improve.
A name is not the thing
Take these references:
COCA-COLA
Coca Cola
Coca-Cola Co.
The Coca-Cola Company
Changing the first three into a lower-case, punctuation-free form is normalization. It makes comparison less distracted by typing style.
Choosing The Coca-Cola Company as the preferred display name is canonicalization. But it is only sensible after the system has enough reason to believe that these references are being used for the company.
That last part matters because names move between levels.
Coca-Cola Classic 330ml can
├─ GTIN: identifies this sellable variant
├─ size: 330ml
├─ container: can
└─ belongs to: Coca-Cola Classic product family
Coke might mean the company, a brand, a product family, one particular drink, or something else entirely. A good system does not solve that by choosing one favourite answer and hoping it stays right. It uses the surrounding information: where the reference came from, what kind of record it is, the identifiers nearby, and when it was written.
This is why the original reference should stay in the record. A canonical name is useful for search, reporting, and presentation. The original tells us what somebody actually wrote. Losing it makes later correction harder, and makes a confident mistake look like a fact.
What frequency can, and cannot, tell us
Some names are much more useful than others.
High Street appears in many addresses. 12 High Street appears in many more places than people expect. Agreement on those words is weak evidence when the town, postcode, or flat number is missing.
The same is true in a catalogue. USB Cable and Black T-Shirt are common descriptions. They bring many possible products into view, but identify very little by themselves.
This is the useful role of frequency. It tells us how much an agreement should change our mind. A rare, distinctive value may be strong evidence. A common value may barely narrow the field. Splink’s term-frequency documentation describes the same practical idea: matches on common values carry less weight than matches on uncommon ones.
Frequency is not confidence. It does not tell us that an alias is correct. It tells us that some agreements are less surprising, and therefore less informative, than others.
Nor is it a number to collect once and forget. Coke might be a very common way to refer to a drink in one marketplace and a poor way to identify a company in supplier data. Its meaning depends on the type of record, the source, place, and time.
Robin Linacre makes this concrete for addresses with discriminating tokens amongst neighbouring addresses. On a street full of candidates, the shared street name is often not the useful part. The flat number, house number, or building name may be what separates one address from the next. His address-matching examples are a good reminder that a missing detail can make a plausible match unsafe.
Normalizing Flat 2, 12 High St can make it easier to compare with other addresses. It cannot turn it into a verified place. It can create a candidate list. The evidence still has to decide whether the reference belongs to one address or another.
Canonicalization is organisational memory
Imagine a marketplace that has been operating for years. It has seen sellers call one product Coke Can 330ML, Coca Cola 330 ml, and Coca-Cola Classic 330ml can. It has learned which spelling tends to arrive from which supplier. It knows which identifiers are reliable. It has retained the decisions that a person reviewed when the answer was not clear.
The valuable thing is not only a clean label in the catalogue.
It is a memory of how the world refers to that thing.
That memory has parts:
- the raw reference and where it came from;
- the aliases that were linked to the entity;
- the identifiers and attributes that helped distinguish it;
- the decision, its reason, and who or what made it;
- the period in which the decision was valid.
This can become valuable infrastructure. The next unknown reference is no longer entirely unknown. It is an unfamiliar way of speaking that may fit something the organisation already knows.
There is a business consequence here. Search becomes less repetitive. Supplier reports can be joined more carefully. A recall can find the affected sellable items without treating an entire product family as one product. A customer can find a book even when the supplier’s title is awkward. None of this comes from one preferred name alone. It comes from keeping the path from a reference to the thing it meant.
The same memory has a limit. A canonical representation is a convention, not an eternal truth. Companies rename themselves. Streets change name. A later edition replaces an earlier one. The Ship of Theseus is nearby: a name can stay while the thing changes, and a thing can continue while its name changes. A system needs dates, sources, and the ability to revise its own answer.
The semantic layer is how the memory travels
One team may call something a product. Another may call it an offer. A third may be looking at a sellable variant. All three may be looking at related data and still produce different numbers, search results, and decisions.
That is not only a naming problem. It is a meaning problem.
A semantic layer is the shared explanation of what the organisation means when it uses words such as customer, product, active supplier, or sales. It can say which entity is being counted, which relationships matter, which attributes are current, and which definition a report or application should use.
That changes entity resolution too. Before a system compares two references, it needs to know what it is trying to understand.
Is this record a book work, a particular edition, or a copy that can be sold? Is Coca-Cola naming a company, a product family, or a 330ml can? Should a supplier’s item code be treated as an identifier, a useful clue, or just text?
Without those meanings, a resolver can find strings that look alike and still ask the wrong question. It may compare a product family with a sellable variant. It may treat a category as an identity. It may merge two records because their names agree, even though the identifiers and level of detail say they cannot be the same thing.
The semantic layer can give the matching process its guardrails:
| It tells the resolver | So the resolver can ask |
|---|---|
| what kind of entity this record may refer to | Are these two references even candidates for the same kind of thing? |
| the level at which identity matters | Are we matching a book, an edition, or a physical copy? |
| which attributes and identifiers mean what | Is this GTIN evidence about a sellable variant, or is this colour only descriptive? |
| which relationships are possible | Could these be different variants in one family, rather than one entity? |
It does not decide the match. It makes the decision intelligible. The evidence still has to say whether these two references refer to the same entity. But now the system knows what a valid comparison looks like, what differences it may tolerate, and where it should stop.
Canonicalization gives that layer something dependable to point at. It says that these supplier references currently lead to this company entity, and that this is the preferred name for showing it. Entity resolution keeps those links honest. Categories and relationships say how the entity connects to other things.
raw references and source records
↓
normalization and candidate finding
↓
entity resolution ← semantic layer: entity types, levels, and valid relationships
↓
entity and alias memory
↓
semantic layer: shared meanings for product, customer, supplier, and sales
↓
search, reports, applications, and AI assistants
The semantic layer does not make the underlying decisions disappear. It exposes them. If product means a product family in a dashboard but a sellable variant at checkout, that difference should be visible. Otherwise two teams can use the same word, appear to agree, and make incompatible decisions.
This is where organisational memory becomes useful beyond the team that built it. The marketplace does not have to teach every analyst, search service, dashboard, or assistant what Coke Can 330ML means. They can ask for the recognised product, then use the shared definition that fits their job. And when a new supplier record arrives, the resolver can use the same definition before it makes a link.
Does AI make this disappear?
AI changes the work. It does not make the question go away.
A language model can be much better than a simple string rule at noticing that two phrases may be related. It can read the words around a reference. It can help propose variants, extract details from messy listings, translate names, and make a better candidate list.
Those are real gains.
But there is a difference between saying, these expressions seem related, and saying, this reference points to entity E12345 and we can explain why.
The second claim may need an identifier, a source with authority, a date, the level of the thing being named, a business rule, and a way to correct the result. It may matter whether the record names a product family, a sellable variant, a company, or a physical item. Language alone cannot choose that boundary for a business.
This is also why fine-tuning is not the whole answer. A model may become better at recognising patterns in language. But an organisation’s changing knowledge about suppliers, books, products, and addresses needs somewhere that can be updated, inspected, and reused tomorrow. It should not have to be rediscovered from scratch whenever a model is asked the same question.
Maybe organisations do not always need their models to know more. Maybe they need better ways to tell models what their data refers to. A semantic layer can be part of that answer, if it gives the model the same governed entities, relationships, and definitions that people use.
That is the role I see for canonicalization now. Not a cosmetic clean-up before the interesting work begins. A maintained record of names, variants, decisions, and limits, so a system can meet the next messy reference with some memory of the world.
The canonical name is not the entity. It is the name the system has learned to use, until it has reason to learn better.
Related
- Sameness and similarity, on why records that look alike do not automatically refer to one thing.
- Similarity is not transitive, on why candidate groups are not entity groups.
- A category is a decision, on why grouping still needs a purpose after resolution.