Set a bounded budget for an AI workflow

An AI workflow that can ask for another step needs a stopping rule outside the model. A prompt saying "finish quickly" cannot enforce request cost or elapsed time. Bound the loop in application code and record why it stopped.

A local fake model makes the control behavior testable without a provider:

import asyncio

async def fake_step():
    await asyncio.sleep(0)
    return {"done": False}

async def run(max_steps=3, seconds=1):
    if max_steps < 1 or seconds <= 0:
        raise ValueError("Budgets must be positive")
    steps = 0
    try:
        async with asyncio.timeout(seconds):
            for _ in range(max_steps):
                result = await fake_step()
                steps += 1
                if result["done"]:
                    return {"reason": "completed", "steps": steps}
    except TimeoutError:
        return {"reason": "deadline", "steps": steps}
    return {"reason": "step_limit", "steps": steps}

if __name__ == "__main__":
    print(asyncio.run(run()))

The default fake step never completes the task, so the default run stops at the step limit. Python's asyncio task documentation describes the timeout context manager and its cancellation behavior.

The application checks time and step limits before allowing further workflow steps.

Budget the whole operation

Use one overall deadline across retries and tool calls. Starting a fresh full timeout for every retry can multiply the time a user waits. Check remaining time before beginning another expensive operation.

A step limit does not directly express a money limit. Different steps can use different models, input sizes or tools. Track authorized cost categories separately and refuse another call when its permitted allocation is exhausted. Estimates and settled provider usage can differ, so define what the budget actually guarantees.

Cancellation is cooperative. Blocking work that does not yield to the event loop can exceed a timeout, and aborting a local wait may not cancel work already accepted by a remote service. Use a worker or service-level limit for workloads that need stronger isolation.

Test every stop reason

Change the fake step to finish immediately and verify the completed outcome. Make it wait longer than a tiny test deadline and verify the deadline outcome. Reject zero or negative configuration before any work begins.

If a step performs a side effect, a timeout can leave its outcome uncertain. Save an operation ID and reconcile the result before retrying. Do not label a timed-out send as definitely unsent.

This example tests loop control only. It does not measure tokens, billable cost, tool permissions or model quality. Those limits need explicit accounting and enforcement around the real integrations.