Cache AI responses using all inputs that affect the result

A response cache can return a wrong AI summary even when its storage works perfectly. If the key includes only the visible question, it may reuse an answer produced with another prompt, another source revision or another tenant's evidence.

Start with an explicit specification of everything that affects the result. For a synthetic summary endpoint, that can include input text, prompt version, model configuration, source revision and authorization scope. Decide which values may be normalized without changing the task.

import hashlib
import json

def response_key(spec):
    encoded = json.dumps(
        spec, sort_keys=True, separators=(",", ":"), ensure_ascii=False
    ).encode("utf-8")
    return "summary:" + hashlib.sha256(encoded).hexdigest()

a = {"text": "hello", "prompt": "v1", "scope": "tenant-a"}
b = {"scope": "tenant-a", "prompt": "v1", "text": "hello"}
assert response_key(a) == response_key(b)
assert response_key(a) != response_key({**a, "prompt": "v2"})

The example shows deterministic serialization for a JSON-compatible specification. It does not define the full production specification for you.

Each behavior-relevant input contributes to the scoped response-cache key.

Separate identity from expiry

A time-to-live limits how long a cached item can remain available. It does not make two different inputs equivalent. A short TTL still allows a cross-tenant answer leak during that interval if scope is absent.

Django's caching documentation describes cache keys, prefixes and versioning. Use those mechanisms to manage the storage layer, while keeping the application's result identity complete.

A hash makes the key compact. It is not encryption or a reason to store sensitive outputs without access controls. Keep cached values within the same retention and authorization rules as fresh responses.

Exercise meaningful changes

Change one input at a time in a fixture: prompt version, source revision, tenant scope and model configuration. Each behavior-relevant change should produce a different key. Reordering dictionary properties should not, under this serialization.

Do not normalize away meaningful whitespace from code or discard punctuation from identifiers without a task-specific justification. Limit accepted numeric values too; exotic or non-finite representations can make cross-language serialization inconsistent.

Permission revocation and source deletion need invalidation or a fresh access check even if the cached answer's key still matches. Keep dependencies on source IDs so those changes can be traced.

This function checks key construction, not cache concurrency, provider nondeterminism or answer freshness. If users expect a newly sampled response, caching may conflict with that expectation. Define when reuse is permitted before treating every matching key as a cache hit.