Expose task progress without treating it as completion

A progress value of 100 can arrive before an exported file has been closed or validated. If the UI treats that number as success, it can offer a download that does not exist. Keep progress and terminal state separate.

Success follows validated export; failures and cancellation are separate terminal states.

Define permitted state transitions

This synthetic state machine describes one processing attempt:

ALLOWED = {
    "queued": {"running", "cancelled"},
    "running": {"exporting", "failed", "cancelled"},
    "exporting": {"succeeded", "failed", "cancelled"},
    "succeeded": set(),
    "failed": set(),
    "cancelled": set(),
}

def transition(job, new_state, *, output_path=None):
    if new_state not in ALLOWED[job["state"]]:
        raise ValueError("Invalid job transition")
    if new_state == "succeeded" and not output_path:
        raise ValueError("Success requires validated output")
    return {**job, "state": new_state, "output_path": output_path}

job = {"id": "fixture-1", "state": "queued"}
job = transition(job, "running")
job = transition(job, "exporting")
job = transition(job, "succeeded", output_path="fixture-output.mp4")

A path string is only a marker in this pure-Python example. The real worker must close and verify the output before making the successful transition. An application should provide an authorized download endpoint rather than exposing an internal filesystem path.

Stage progress can be stored separately, such as an encoding percentage within running. It should be labeled by stage so a reset during exporting does not look like the job unexpectedly went backward.

Make concurrent updates conditional

The dictionary function above is a demonstration. A database implementation should update only if the recorded previous state and attempt identifier still match. Otherwise a stale worker can overwrite a cancelled job or the result of a newer attempt.

Give each retry attempt its own identity. Decide whether a retried job remains one user-visible job with several attempts, and expose that relationship consistently. A terminal state for one attempt should not silently become running again without a recorded transition.

Test cancellation near completion and a failure during final export. Confirm neither creates a success state merely because an earlier stage reached 100 percent. Also test a missing worker heartbeat and define when the job becomes stalled for operator review.

Celery result state can help operations, but a business workflow often needs its own persisted lifecycle and output validation.

The processing stages in ShortForge provide related context. Celery documents task execution and states.