Load an inference resource through FastAPI lifespan

A predictor that is expensive to initialize should have an explicit application lifetime. Loading it inside every request repeats work, while loading it at module import can make unrelated tests and utilities pay the initialization cost.

FastAPI's lifespan interface provides startup and shutdown boundaries. A lightweight fake predictor makes the lifecycle visible without downloading a model.

from contextlib import asynccontextmanager
from fastapi import FastAPI

class FakePredictor:
    def __init__(self):
        self.closed = False

    def predict(self, value):
        if self.closed:
            raise RuntimeError("Predictor is closed")
        return value * 2

    def close(self):
        self.closed = True

@asynccontextmanager
async def lifespan(app):
    predictor = FakePredictor()
    app.state.predictor = predictor
    try:
        yield
    finally:
        predictor.close()
        del app.state.predictor

app = FastAPI(lifespan=lifespan)

@app.get("/predict")
def predict(value: float):
    return {"result": app.state.predictor.predict(value)}

FastAPI's lifespan documentation describes work before and after the context manager's yield. The fake's multiplication represents a deterministic local operation, not a model inference benchmark.

A fake predictor is acquired at startup and released during normal application shutdown.

Verify startup and cleanup

Enter the application's lifespan in a test and confirm the predictor is available. Call its prediction method with a known number, then leave the lifespan and check that the captured resource is closed. A request-level test should also verify the endpoint's input and output behavior using a client that runs lifespan.

If initialization raises, the service should fail readiness instead of accepting requests with an absent predictor. Clean up partially acquired resources when startup has several steps. A context manager around each resource can make failure paths easier to manage.

Account for worker boundaries

A lifespan resource belongs to an application process. Multiple worker processes can each load their own copy, increasing memory use. A diagram with one resource per application object does not promise one shared model across an entire deployment.

Check whether the actual inference library supports concurrent calls and how it uses threads or accelerators. A synchronous endpoint moves ordinary blocking work away from an async handler, but it does not make a non-thread-safe model safe to share.

Shutdown can also be abrupt. The finally block handles normal context exit; process termination or machine failure can prevent graceful cleanup. External jobs and durable state need their own recovery behavior.

This example covers ownership of one in-process fake resource. It does not implement model downloading, batching, rate limits or production readiness checks. Verify those separately for the deployed model and server configuration before replacing the fake predictor.