Separate model failures from application failures in an AI endpoint
A failed AI endpoint needs enough internal detail to identify which layer failed. A provider timeout, a model refusal and an application parsing bug require different investigation even when all three prevent an answer.
For a synthetic endpoint, define an internal result classification before writing retry logic. Distinguish transport failure, explicit provider-reported refusal, malformed output, schema validation failure and an unexpected application exception. Keep a correlation ID that follows the request through those layers.
Do not infer a refusal merely because prose contains an apology. Provider contracts vary, and some expose explicit refusal fields or response events. Verify the selected API's current contract before writing provider-specific handling.
AI request branching into input, provider and validation errors.
Place each failure at its boundary
A timeout occurs while waiting for a response. Invalid JSON occurs after bytes arrive but parsing fails. A schema error occurs after parsing succeeds but the object violates the expected contract. An internal exception might happen when reading a field the application incorrectly assumed was present.
Pydantic's model documentation explains validation of structured input. Such validation can identify a contract mismatch; it does not tell the application why the upstream model produced that mismatch.
Use a local provider stub with explicit modes for timeout, a refusal-shaped fixture and malformed JSON. Include one valid response that triggers a deliberate downstream application error in a disposable test. Confirm that these modes produce separate internal categories.
Match recovery to the failure
A transient transport problem may justify a bounded retry. A refusal may require explaining the limitation or letting the user revise the request. Repeating the same invalid schema response indefinitely consumes resources without creating a reliable result.
Avoid treating every exception as a provider outage. A programming error in your own serializer can otherwise trigger repeated paid requests and hide the actual bug. Preserve exception details in restricted diagnostics while showing the user a concise message and a safe next step.
A timeout can leave the provider-side operation's outcome uncertain. If the workflow triggers external side effects, an automatic retry needs an idempotency and reconciliation design. Text generation alone has different consequences, but still incurs duplicated work.
Log timings and categories without storing unnecessary prompt contents, credentials or full user records. A useful incident record often needs request identity and stage information more than raw text.
This example is a classification and test plan, not an integration with any current provider SDK. The stub verifies application branching. Live verification still needs the chosen provider's documented error behavior, network settings and response schema, with failure cases exercised under the versions the application actually runs.