
Semantic cannibalization: what vectors detect that keyword audits miss
How to use embeddings to identify pages competing for the same semantic territory, even when they don’t share keywords. Concept, thresholds and how to fix it.
highlights
- Cannibalization isn’t just two pages fighting over the same keyword. Two pages can compete for the same territory without sharing a single word.
- Vector analysis identifies this kind of overlap by comparing the meaning of pages, not their text.
- When two articles are semantically very similar, Google may not know which one to prioritize, and both end up ranking worse than a single well-structured article would.
- Detecting the problem is the easy part. Deciding whether to consolidate, differentiate or remove one of the pages requires analyzing traffic and search intent.
- Tools like Screaming Frog v22.0 already run this kind of analysis without any code.
what semantic cannibalization is (and why it goes beyond keywords)
Keyword cannibalization can be detected with any SEO tool: you filter the pages ranking for the same query and decide which one to consolidate. The process is manual, but the problem is visible.
Semantic cannibalization is different. Two pages can use completely different vocabularies and still cover the same territory of meaning. From the standpoint of a semantic retrieval system, they’re near-duplicates. But no keyword audit will flag it.
Screaming Frog v22.0 introduced semantic similarity detection for exactly this reason: to go beyond textual duplication and capture real topical overlap.
how vector detection works
The process is straightforward. You generate an embedding for each page on the site and calculate the similarity between every pair. Pairs with a very high score are candidates for semantic cannibalization.
The practical thresholds, referenced in Screaming Frog and validated in practice:
Semantically very close content. It’s worth checking whether the pages really cover distinct angles or are duplicating effort.
Practically a semantic duplicate. Consolidation is almost always the right call.
One important detail: vectorizing the entire page dilutes the embedding. Section-level analysis, using H2s as the unit, reveals more granular overlaps: two pages can have completely different introductions but nearly identical entire sections.
what to do with the flagged pairs
Detecting is the easy part. Deciding how to resolve it is an editorial call and depends on context the vector score doesn’t provide.
Consolidate
Merge the two pages into a more complete version and redirect the less relevant one to the page you keep. This makes sense when both cover the same angle and neither has a clear differentiator.
Differentiate
Rewrite one of them to explore a different aspect of the topic. This resolves the overlap without losing the content.
Remove
When neither page has meaningful traffic or authority and the topic is already covered by a third, stronger page. Redirect to it and retire both.
The vector score identifies the problem. GSC traffic data, external links and the search intent of each URL are the criteria that determine which path to take. iPullRank documented that content pruning based on semantic alignment produced a 2 to 3% lift in a site’s overall semantic relevance after removing more than 500 articles.
a hypothesis about what happens in google’s index
This section is explicitly speculative.
If Google maintains something analogous to an internal vector index, and there are hints of this in the 2024 documentation leak and in the known evolution of its ranking systems, then pages with very similar embeddings may be competing in semantic space even before they compete for explicit keywords.
It’s an argument that is structurally consistent with what we know about semantic retrieval. But it’s still a hypothesis, with no conclusive public documentation.
tools to identify semantic cannibalization
Screaming Frog v22.0 with its integration with embedding models (OpenAI, Gemini or a local Ollama instance) is the most direct path. You configure the semantic similarity analysis and export the pairs above the threshold.
If you prefer more control, Python with sentence-transformers generates the similarity matrix and filters pairs by threshold. It allows analysis by page section, not just by entire URL.
final thoughts
Semantic cannibalization is a real problem that is systematically ignored because traditional SEO tools can’t see it. It isn’t the fault of manual processes; it’s a limitation of the lexical model.
The good news is that detection has become accessible. Screaming Frog handles much of the work without code. What still requires attention is the decision stage: consolidating, differentiating or removing are choices that depend on understanding the site, the content and what Google is already rewarding in each case.

Lucas Cassapula
Partner & Head of SEO at Wesearch and Co-founder of Mentionflow
I am a partner at Wesearch and co-founder of Mentionflow. I have worked with SEO for almost 10 years. I am a data-driven geek who is always testing hypotheses, looking for patterns and turning ideas into products. I share studies, experiments and automations focused on SEO and GEO.