hreflang on a Bilingual Site Is Three Invariants: One URL Per Document, Absolute Values, Real Alternates

Why does a bilingual site lose its own language associations?

A bilingual site loses its language associations for three reasons that share one property: the crawler reports none of them. A relative hreflang value is discarded rather than resolved. A second URL serving the same document competes with the first instead of merging with it. An alternate that points at a page nobody translated takes its valid siblings down with it. Every one of those is well-formed markup. Every one of them passes the lint rules I have seen applied to this problem, and none of them produces an error you can find by reading your own HTML.

hreflang is not a hint a crawler weighs; it is an assertion set, and a single alternate that resolves relative or lands on a 404 can cost you the association between every URL in that cluster, with nothing reported anywhere.

The invariants that prevent this are small enough to state in three lines:

• One URL per document. The bare path is English. /zh/... is Chinese. /en/... is not a page, it is a redirect.
• Absolute alternate values anywhere a crawler reads the value directly, which means sitemap XML, feed XML and any Link header.
• An alternate only where a translation exists as a file. Not where one is planned, and not where one is promised in a CMS field.

None of these is difficult. All three regress quietly, because the failure looks identical to success when you read the source instead of the resolution. The rest of this article is the implementation I run, in the order the failures actually appear.

What does one URL per document require at the routing layer?

The cheapest way to create duplicate content on a bilingual site is to make the language prefix optional. If both /about and /en/about return 200, you have two documents where you intended one, and no amount of correct hreflang fixes it, because the annotation is describing a structure you have already broken.

My contract is asymmetric on purpose. English is primary and owns the shortest URLs, so the bare path is canonical and /en exists only to be redirected away from. Chinese is a real prefix. The mechanism is a single middleware file with an internal rewrite: the public URL stays clean, one route tree serves both locales, and crawlers still receive complete HTML.

import { NextResponse } from 'next/server';
import type { NextRequest } from 'next/server';

const LOCALE_PREFIX = /^\/zh(?=\/|$)/;

export function proxy(request: NextRequest): NextResponse {
  const { pathname, search } = request.nextUrl;

  // `/en` and `/en/anything` are never canonical. 308 preserves the method
  // and body, and tells crawlers and caches this is permanent.
  if (pathname === '/en' || pathname.startsWith('/en/')) {
    const stripped = pathname.slice(3); // '' | '/about' | ...
    const target = new URL(stripped === '' ? '/' : stripped, request.url);
    target.search = search;
    return NextResponse.redirect(target, 308);
  }

  // A non-default locale owns a real prefix and needs no rewriting.
  if (LOCALE_PREFIX.test(pathname)) return NextResponse.next();

  // Bare paths are English. The rewrite is invisible to the crawler.
  const rewriteUrl = request.nextUrl.clone();
  rewriteUrl.pathname = `/en${pathname === '/' ? '' : pathname}`;
  return NextResponse.rewrite(rewriteUrl);
}

The distinction between a rewrite and a redirect is the whole point. A rewrite means one URL, one response, one route tree. A redirect from /en/about means the second URL never becomes indexable in the first place, which is why the canonicalising branch returns 308 rather than serving the page and declaring a canonical. If you serve the page and declare a canonical, you have shipped two URLs and asked politely for one of them to be ignored.

That last row is the one teams under-plan for. If your CMS happily generates a Chinese URL for every entry regardless of whether a translation exists, you have manufactured a 404 for every article you have not translated yet, and you have probably published an alternate pointing at it.

Why must the sitemap use absolute alternates when the metadata does not?

Next.js resolves metadata.alternates.languages against metadataBase, so relative values there are correct and I use them. Sitemap XML has no such resolution step. A /zh/blog/... value inside an xhtml:link element is not a partial URL a crawler completes; it is an annotation the crawler drops.

I keep two functions rather than one with a flag, because the two call sites cannot then be confused:

export type Locale = 'en' | 'zh';

/** Google prefers `zh-Hans` over a bare `zh`. */
export const hreflang: Record<Locale, string> = { en: 'en', zh: 'zh-Hans' };

/** Bare paths are English; Chinese lives under `/zh`. */
export function localeHref(locale: Locale, path = '/'): string {
  const clean = path === '/' ? '' : `/${path.replace(/^\/+|\/+$/g, '')}`;
  return locale === 'en' ? clean || '/' : `/zh${clean}`;
}

/** Relative values, for `metadata.alternates` only: Next resolves them. */
export function alternateLanguages(path: string): Record<string, string> {
  return {
    [hreflang.en]: localeHref('en', path),
    [hreflang.zh]: localeHref('zh', path),
    'x-default': localeHref('en', path),
  };
}

// Absolute form, for sitemap and feed XML: relative values are silently dropped.
export function absoluteLanguageAlternates(
  path: string,
  toAbsolute: (p: string) => string,
): Record<string, string> {
  return Object.fromEntries(
    Object.entries(alternateLanguages(path)).map(([key, value]) => [key, toAbsolute(value)]),
  );
}

That map is correct for chrome routes, which exist in both locales by construction. It is wrong for articles, and the reason is the third invariant.

How do you emit an alternate only where a translation exists?

The set of locales that actually serve a given article must be computed from the filesystem, not declared in frontmatter. Frontmatter is a promise; the filesystem is a fact, and only one of them can be checked at build time.

I read every post in both locales, then resolve coverage in a second pass, because a Chinese file declares translationOf holding the English slug of its original, and whether that English file exists is only knowable after the whole tree has been read.

interface Post {
  slug: string;
  locale: Locale;
  /** Set on translated files; coverage itself comes from the filesystem. */
  translationOf?: string;
  availableLocales: readonly Locale[];
}

/** hreflang for one article, absolute, for sitemap and feed XML. */
export function articleAlternates(
  post: Post,
  toAbsolute: (path: string) => string,
): { canonical: string; languages: Record<string, string> } {
  const englishSlug = post.locale === 'en' ? post.slug : (post.translationOf ?? post.slug);
  const hrefFor = (locale: Locale): string =>
    locale === 'zh'
      ? toAbsolute(localeHref('zh', `/blog/${post.translationOf ?? post.slug}`))
      : toAbsolute(localeHref('en', `/blog/${englishSlug}`));

  // An alternate only for a locale whose file exists.
  const languages = Object.fromEntries(
    post.availableLocales.map((locale) => [hreflang[locale], hrefFor(locale)]),
  );

  // `x-default` must resolve too: prefer English, fall back to the Chinese file.
  languages['x-default'] = hrefFor(post.availableLocales.includes('en') ? 'en' : 'zh');

  return { canonical: toAbsolute(localeHref(post.locale, `/blog/${post.slug}`)), languages };
}

The arithmetic on this repository is the argument for the check. The tree holds 57 English articles and 3 Chinese files. If every English article emitted a zh-Hans alternate on the assumption that translation follows, 54 of those alternates would point at 404s. Emitting alternates from availableLocales instead means the 54 untranslated articles publish en and x-default only and claim nothing they cannot deliver.

x-default is the subtle case and it is why this is filtering rather than a boolean. The obvious implementation hardcodes x-default to the English URL. That is correct until you have a Chinese-only article, at which point the default advertises a page that does not exist, and the coverage filter does not catch it because coverage only guards the en and zh-Hans pair. Pointing x-default at whichever variant actually exists keeps the invariant intact for articles that only exist in one language.

There is a second-order consequence worth naming, because it is where the engineering meets the business case. Two language versions of one article are supposed to be one work in two languages. If the alternates are missing, a retrieval system sees two documents competing for the same query, which splits citation and ranking between them instead of consolidating. That merge is exactly the problem an entity graph with a shared identifier for both language versions is built to prevent, and no language annotation survives on its own if the entity layer disagrees with it.

What does each failure look like from the crawler's side?

I keep this taxonomy next to the code, because every row is a decision someone makes under deadline pressure and each one is invisible in review.

The reciprocity row is the one people miss because it is not a per-page bug. hreflang is a mutual declaration: if page A names B as its Chinese version, B must name A as its English version. A one-way annotation is discarded, so the failure looks like the annotation was never emitted at all, on both pages.

How do you verify the annotation rather than reading the source?

Three checks, and the third is the only one that keeps working after I stop looking. Writing them as an assertion rather than as a set of commands is not ceremony, because this area has one specific trap: grep -c exits non-zero precisely when the count is zero, so a count-based check reports a correct site as a failure and, in a pipeline that inverts the test, a broken one as a success. The gate below extracts each alternate and checks it individually, so the two outcomes cannot be confused.

set -euo pipefail
BASE=https://willchan.me
SLUG=i18n-routing-without-duplicate-content

# 1. Both language URLs must answer 200. A 404 here is a dead alternate.
for path in "/blog/$SLUG" "/zh/blog/$SLUG"; do
  code=$(curl -sS -o /dev/null -w '%{http_code}' "$BASE$path")
  [ "$code" = "200" ] || { echo "$path answered $code"; exit 1; }
done

# 2. Alternates must be in the server-rendered HTML, not injected after hydration.
curl -sS "$BASE/blog/$SLUG" | grep -q 'rel="alternate"' \
  || { echo 'no alternates in the server-rendered HTML'; exit 1; }

# 3. Every sitemap alternate must be absolute and resolve. A 308 is not a pass:
#    an alternate that redirects is not a served page for that language.
for href in $(curl -sS "$BASE/sitemap.xml" \
  | grep -o 'hreflang="[a-z-]*" href="[^"]*"' | sed 's/.*href="//; s/"$//'); do
  case "$href" in
    https://*) ;;
    *) echo "relative alternate: $href"; exit 1 ;;
  esac
  code=$(curl -sS -o /dev/null -w '%{http_code}' "$href")
  [ "$code" = "200" ] || { echo "$href answered $code"; exit 1; }
done

echo 'every alternate absolute and resolving'

The third loop is the expensive one, and its size is worth stating because it decides where the check belongs. Six chrome routes in two locales contribute 36 alternates, and the 60 article entries contribute 126 more, since the 3 translated pairs emit three each and the 54 untranslated English articles emit en and x-default but nothing else. That is 162 requests, which is a nightly job against production plus a manual run before a content release rather than a step on every pull request. It is also the only check here that catches an alternate pointing at a page that has since been renamed.

When is a language prefix the wrong approach?

Four cases where I would not build this:

• Region, not language. If the distinction is en-US against en-GB pricing, currency or legal terms, hreflang is the wrong instrument on its own. Region targeting wants a country-code domain or a region path with genuinely different content; two URLs for the same English text split across regions is duplicate content with extra steps.
• Machine-translated content you would not publish. Emitting hreflang asserts that a human-quality equivalent exists at the other end. If it does not, noindex on the translated tree is more honest and less expensive than a language cluster that fails its own claim when someone reads it.
• A single translated landing page. The apparatus here costs an afternoon, which is a poor trade for one page. Publish the translation at a stable URL with a self-referencing canonical and no alternates at all; an absent annotation is neutral, while a half-maintained cluster is a liability.
• A platform that owns your routing. On hosted CMS and e-commerce platforms, the prefix, the redirect behaviour and the sitemap are generated for you, and fighting them costs more than the annotation is worth. Fix what you own: absolute values in the sitemap, alternates only for real translations, and one canonical URL per document as far as the platform allows.

There is also a cost to the asymmetric contract itself. Keeping /en as a permanent redirect rather than a served alias means every internal link must be generated through one helper, and a hand-written /en/... href in a component is a bug that only shows up in logs. That is a real maintenance tax, and on a site with two engineers and one language I would not pay it.

What is the smallest correct version to ship this week?

Write the URL contract into one helper and route every internal link through it, so that /en cannot be produced by accident. Compute availableLocales from the filesystem at build time, and emit alternates from that set rather than from a declared list. Then wire the assertion above to run nightly against production, because the failure it defends against is invisible by construction and a check that only runs when someone remembers is not a check. The whole thing is roughly an afternoon, and the first time it turns red you will have caught an alternate that would otherwise have sat in production pointing at nothing for a year.