Validate a CSV import before changing application data

An import that saves each row as soon as it parses can leave the database half updated when a later record is invalid. For a small administrative CSV, a useful policy is to validate the entire input first and apply changes only when the error list is empty.

CSV rows flowing through a validation tray with rejected row numbers.

Separate parsing from changes

This example accepts synthetic contact records. The email check is deliberately a simple application rule, not a complete email-address validator.

import csv
from io import StringIO

def validate_contacts(text):
    reader = csv.DictReader(StringIO(text, newline=""))
    required = {"name", "email"}
    columns = set(reader.fieldnames or [])
    if not required <= columns:
        return [], [{"row": 1, "error": "Missing name or email column"}]

    contacts, errors = [], []
    for row_number, row in enumerate(reader, start=2):
        name = (row.get("name") or "").strip()
        email = (row.get("email") or "").strip()
        if None in row:
            errors.append({"row": row_number, "error": "Too many values"})
        elif not name or email.count("@") != 1:
            errors.append({"row": row_number, "error": "Invalid name or email"})
        else:
            contacts.append({"name": name, "email": email})
    return contacts, errors

Using the CSV parser preserves quoted commas. Splitting a line on commas would turn a name such as "Lee, Sam" into extra fields. The parser also handles quoted newlines, so the displayed record number above is a logical record number. If an operator needs physical line positions, report the reader's line counter and explain its behavior for multiline records.

The caller should refuse to apply the returned contacts whenever errors are present. Do not silently import the valid subset unless partial imports are an explicit product feature.

Check more than the happy path

Use fixtures for a missing header, an empty input, a quoted comma and a row with too many values. Decide whether duplicate headers are allowed; for a strict import, reject them before constructing records. Set an input-size limit so collecting all records cannot consume unlimited memory.

Validation can become stale between this step and the eventual write. Another request may create a duplicate email while the import waits for confirmation. Keep required uniqueness rules in the database and apply the accepted batch inside a transaction with an intentional conflict policy.

For very large imports, stage validated records in separate storage and record batch status. That allows review without keeping the whole dataset in RAM. A staging area still needs access controls and a retention policy.

Postzyo is related background on administrative automation. Python's CSV documentation describes the parsing behavior and dialect controls.