Revalidate a stored HTTP source with ETag

AI assisted the preparation of this article, and its Python example was tested locally with synthetic fixtures.

A content importer can retain a previously fetched representation and ask whether the server still considers it current. A conditional GET sends the saved entity tag, allowing a server to respond with 304 rather than transferring the representation again.

Flow: Stored body, then Conditional GET, then 304 or 200, then Reuse or replace.

Keep the old body until a new one succeeds

This deliberately narrow client accepts either no stored record or a record containing the requested URL, a body and an entity tag. It returns a new record on 200 and reuses the prior record on 304 only when the server returns the same tag. It never mutates the supplied record.

from urllib.request import Request, urlopen
from urllib.error import HTTPError

def fetch(url, stored):
    if stored is not None and (stored.get("url") != url or not stored.get("etag") or not isinstance(stored.get("body"), bytes)):
        raise ValueError("stored representation and tag required")
    headers = {"If-None-Match": stored["etag"]} if stored else {}
    try:
        with urlopen(Request(url, headers=headers), timeout=5) as response:
            tag = response.headers.get("ETag")
            if response.status != 200 or not tag:
                raise ValueError("expected a tagged 200 response")
            return {"url": url, "etag": tag, "body": response.read()}
    except HTTPError as error:
        if error.code == 304:
            if stored is None or error.headers.get("ETag") != stored["etag"]:
                raise ValueError("304 requires a matching stored representation")
            return stored
        raise

A local loopback server can return a tagged first body, return 304 when it receives that exact tag, then change both the tag and content for the next response. The client should preserve the original bytes on 304 and replace both fields on the new 200. A server error should propagate without changing the old record. An unexpected 304 without a record fails explicitly.

This fixture's exact-tag check is intentionally narrower than general HTTP validation. RFC 9110 specifies weak comparison for If-None-Match. Keep the server's quoted tag as received; it is an opaque validator, not necessarily a content hash.

Do not treat this fixture as a cache

A complete cache must account for response selection, Vary, freshness, authorization scope, redirects and metadata updates on a 304. RFC 9111 describes the stored-response update rules. This example neither implements those rules nor handles arbitrary weak-tag equivalence.

The test server binds an ephemeral loopback port and shuts down in a finally block. It uses synthetic bytes and no external provider. In production, also bound response size and time, verify source permissions, and define retention and failure reporting. A 304 says the server's selected representation matches the request condition; it does not prove provenance, authorization, successful parsing or successful indexing.