Duplicate Content
Duplicate content is identical or very similar content available at more than one URL. It rarely causes a penalty but splits signals and wastes crawling.
What is Duplicate Content?
Duplicate content is content that appears at more than one URL, either word for word or with only minor differences. It can exist within one website (for example, the same page with and without tracking parameters) or across domains (scraped or syndicated articles). Search engines show only one version and must choose which.
Most duplication is accidental and technical, not plagiarism. The real costs are diluted link signals, wasted crawl activity and Google sometimes picking the wrong URL as the version to rank.
How it works
When Google finds several URLs with the same main content, it clusters them and selects a canonical URL. It uses signals such as:
rel="canonical"link elements- 301 redirects
- Internal links and sitemap entries
- HTTPS over HTTP, and cleaner URLs over parameterized ones
Ranking signals from the cluster are consolidated to the chosen canonical. If your signals conflict (for example, the canonical tag says one URL but internal links point to another), Google may ignore your preference.
Common technical causes:
| Cause | Example |
|---|---|
| URL parameters | ?utm_source=linkedin, sort or filter parameters |
| Protocol and host variants | http vs https, www vs non-www |
| Trailing slashes and case | /pricing vs /pricing/ |
| Printer or AMP versions | Separate URLs for the same article |
| Boilerplate-heavy pages | Location or integration pages with one changed word |
Examples
- A SaaS blog shares posts with UTM tags, and some tagged URLs get indexed. Adding self-referencing canonicals on every post consolidates them.
- A security vendor publishes 40 “SIEM for [industry]” pages that differ only in the industry name. Google indexes a few and ignores the rest as near duplicates.
- A partner republishes your research report in full and outranks you. Asking for a link to the original and a noindex on their copy resolves it.
Best practices
- Add a self-referencing canonical tag on every indexable page.
- Redirect protocol, host and trailing slash variants to one format with 301s.
- Link internally only to canonical URLs, and list only canonicals in your XML sitemap.
- When you build programmatic pages, give each one genuinely unique data or guidance, not a swapped keyword.
- Consolidate overlapping articles into one stronger page and redirect the rest.
Common mistakes
- Blocking duplicates in robots.txt, which stops Google from seeing the canonical tag on them.
- Canonicalizing paginated pages to page one, which hides deeper content.
- Using noindex and canonical on the same URL, which sends mixed signals.
Duplicate content and AI search
AI engines retrieve passages from indexed pages. When the same content lives at several URLs, the version an engine retrieves and cites may not be the one you want, and link signals that help a page rank are split. Consolidating to one clear canonical makes it more likely the right page is retrieved. See how AI engines cite sources.
Related terms
Sources
- How to specify a canonical URL with rel=“canonical” and other methods, Google Search Central
- Demystifying the “duplicate content penalty”, Google Search Central Blog
- Page indexing report, Search Console Help
- Duplicate content, Moz
Frequently asked questions
Is there a duplicate content penalty?
Not for normal duplication. Google has said for years that duplicate content alone does not trigger a penalty. It groups duplicates, picks a canonical and shows that one. Penalties apply only when duplication is deceptive or manipulative, such as scraping other sites at scale, which falls under spam policies.
How do I find duplicate content on my site?
Crawl the site with a tool such as Screaming Frog to find matching titles, descriptions and page hashes. In Google Search Console, the Page indexing report shows statuses like "Duplicate without user-selected canonical" and "Google chose different canonical than user", which point to URLs Google treats as duplicates.
Is syndicated content duplicate content?
Yes, technically. If you republish articles on partner sites, the partner version may outrank yours. Ask partners to link back to the original, and where possible to noindex their copy. Google notes a cross-domain canonical is not recommended for syndication, since the pages are often not equivalent.