How to Prevent Duplicate Slugs in a Large SEO Site

Duplicate slugs are usually not a content problem first. They are a data integrity problem that becomes a routing problem, then an indexation problem, and finally an operational problem when teams start overwriting URLs by accident.

On a large SEO site, the question is not whether two records can produce the same slug. They will. The real question is where you enforce uniqueness, how you recover when collisions happen, and how you keep imports and concurrent publishing from creating broken URLs.

Make the database the source of truth

If a slug is meant to identify a page route, it should be enforced by the database, not just by application code. Application-level checks are useful, but they are advisory. Two workers can both read "available" at the same time and then both try to insert the same slug.

A proper uniqueness constraint closes that gap. In most systems, that means a unique index on the slug column, or on the tuple that actually defines uniqueness. For example, a slug may only need to be unique within a tenant, a locale, or a content type. In that case, the constraint should reflect the real routing rule:

CREATE UNIQUE INDEX uniq_pages_tenant_locale_slug
ON pages (tenant_id, locale, slug);

That design matters because global uniqueness is often too blunt. If your site structure allows /en/ and /fr/ to reuse the same human-readable slug, forcing one global namespace creates unnecessary collisions. This is one of the same architectural boundaries that matter in multi-tenant SEO publishing systems.

If your pages use a canonical URL field separate from the slug, be clear about what is actually unique. Some teams try to make the slug unique, then later discover the route is composed from category + slug + locale, not slug alone. The constraint should match the generated URL, not an abstract idea of cleanliness.

Generate slugs deterministically, then normalize aggressively

Slug generation should be deterministic enough that the same source string produces the same base slug every time. That makes imports and reprocessing predictable.

The usual pipeline is:

1. Convert to lowercase.
2. Remove or transliterate characters that should not appear in the URL.
3. Replace spaces and separators with hyphens.
4. Collapse repeated hyphens.
5. Trim leading and trailing hyphens.

The important part is not the exact regex. It is consistency. If one code path turns C++ for Beginners into c-for-beginners and another turns it into cpp-for-beginners, you no longer have a stable slugging system. You have two naming policies pretending to be one.

Stable base slugs also help when you are detecting duplicates before publication. A good preflight check can compare the proposed slug against existing rows and against the currently reserved queue, which is especially useful if you already have validation gates like the ones described in how to validate SEO pages before publishing.

Collision handling should be explicit, not accidental

Collisions are inevitable when multiple pages share the same title, city name, product name, or topic. You need a policy.

The simplest policy is suffixing:

• best-running-shoes
• best-running-shoes-2
• best-running-shoes-3

This works, but only if the suffix is assigned by a single authority. If application code invents the suffix based on a stale read, collisions will still happen under load.

A safer approach is to reserve the slug inside a transaction, or let the database reject the insert and then retry with a new candidate. That retry loop should be bounded and deterministic. For example:

• try the base slug
• if conflict, try base + -2
• if conflict, try base + -3
• stop after a fixed limit and escalate

Do not make the fallback random unless you are comfortable with unstable URLs. Randomized slug suffixes are fine for internal IDs, not for search pages that need predictable paths.

The best policy depends on page semantics. If the slug is primarily for display and the canonical URL is built from a deeper hierarchy, then a suffix is usually acceptable. If the slug is part of the visible page identity, you may need a stronger disambiguator in the source data itself, such as a parent category, region, or product family.

Imports are where duplicate slugs usually enter quietly

Manual page creation is not the biggest risk. Bulk imports are.

CSV loads, partner feeds, CMS migrations, and programmatic page backfills often bypass the same code path used by the normal UI. That is how teams end up with duplicate slugs created by a migration job that did not know about current production state.

Imports should never write directly to the table without validation. They need the same slug generation and uniqueness logic as the rest of the system, plus a reconciliation step for existing data.

A useful pattern is a staging table:

1. Import raw records into staging.
2. Normalize and generate candidate slugs.
3. Detect collisions against both staging and production.
4. Resolve conflicts before inserting into the live table.
5. Record the final slug chosen and the reason for any change.

This gives you a place to inspect problematic rows before they become public URLs. It also makes rollback easier. If the import job fails halfway through, you can rerun it without guessing which rows were already assigned a route.

If the import source already contains a slug field, treat it as an input, not a promise. External slugs often contain punctuation, duplicates, or obsolete URL patterns that do not fit your current routing rules. The safest migration strategy is usually to remap them into your own namespace rather than preserve them blindly.

Concurrency is the part that breaks careful plans

Even with perfect slug rules, concurrent writers can still produce duplicates if uniqueness is not enforced atomically.

This shows up in:

• parallel imports
• worker queues generating similar pages
• editors publishing at the same time
• retries after transient failures
• backfills running while new content is being added

The fix is to move slug allocation into a transaction that the database can protect. In practical terms, that means one of three approaches:

• unique index plus retry on conflict
• lock the candidate namespace before insert
• centralize slug reservation in a service or stored procedure

The first option is usually the most practical at scale. It keeps the database as the arbiter and lets your application retry safely. The critical part is that the retry must regenerate or re-reserve the slug inside the same logical publish attempt. If you compute the slug once and reuse it after a conflict, you are just repeating the failure.

This is closely related to making an SEO pipeline idempotent. If your content jobs can be replayed, they need to arrive at the same final state without creating duplicate URLs. That is the same class of problem discussed in how to make an SEO content pipeline idempotent.

Don’t confuse slug uniqueness with URL stability

A unique slug is not automatically a stable slug.

Stability means a URL stays the same after publication unless you intentionally change it. If you regenerate slugs whenever the title changes, you will create avoidable redirects and break internal references. For a large site, that is not just messy; it increases maintenance burden across sitemaps, internal links, and analytics.

A better rule is to separate the editable title from the route slug. Let the title change freely. Let the slug change only through an explicit migration or editorial action. That gives you a clean place to decide whether the page should keep its current URL, be redirected, or be reissued under a new path.

When you do need to change slugs, update the redirect map, internal links, and sitemap output as one coordinated change. Otherwise, you will solve duplicate slugs at the database layer and introduce URL drift everywhere else.

Operational safeguards that actually help

For large sites, prevention should include a few simple checks:

• a unique database constraint on the real URL namespace
• pre-insert slug reservation or retry-on-conflict logic
• staging imports before live writes
• collision logging with the source record ID and final slug
• automated checks for duplicate or near-duplicate routes before publishing

The goal is not to eliminate every collision. It is to make collisions visible, deterministic, and recoverable.

If the system has to choose between silent duplication and explicit failure, prefer explicit failure. A failed insert is annoying. A duplicated slug that overwrites the wrong page is much more expensive.

The cleanest slug system is the one that assumes collisions will happen, then makes the database, the import path, and the publisher handle them in the same way every time.