Reject duplicate JSON object keys in Python

A configuration can contain {"role":"viewer","role":"admin"} and still be accepted by Python's default JSON decoder. The resulting dictionary keeps the last value. That makes the input ambiguous to a human reviewer and can hide a disagreement between systems that choose different duplicate-key policies. If your configuration contract requires unique names, reject the repetition while decoding, before a dictionary has discarded the evidence.

Python's JSON documentation describes the default behavior and the object_pairs_hook extension point. The hook receives an ordered list of pairs for each decoded object. It can return the mapping the application will use, or raise an error when it encounters the same decoded name twice.

Each decoded object goes through a pairs hook; a repeated key raises an error.

Reject duplicates during decoding

import json

def unique_json(text):
    def unique(pairs):
        result = {}
        for key, value in pairs:
            if key in result:
                raise ValueError(f'duplicate key: {key}')
            result[key] = value
        return result
    return json.loads(text, object_pairs_hook=unique)

unique_json('{"a":{"b":1}}') returns {'a': {'b': 1}}. The input '{"a":{"b":1,"b":2}}' raises ValueError: duplicate key: b. The nested example matters: the hook runs for each JSON object, not only the outermost one. Repeated names in different objects are allowed because each object gets its own dictionary. An object inside a list is checked the same way.

The decoder has already interpreted escapes before the hook compares names. Therefore a literal name and its equivalent Unicode escape compare as the same Python string. The error names the repeated key but does not provide its path or position. If diagnostics need a location, use a parser designed to retain location information; do not guess the path from this callback.

Test the contract at the boundary

Use a successful nested object and a duplicate inside that nested object. For example, assert the successful nested result and check that the duplicate raises an exception rather than returning a mapping. A malformed JSON document raises json.JSONDecodeError, while this policy raises ValueError; catch them at the input boundary if the caller needs a controlled rejection. Since JSONDecodeError is also a ValueError, catch it first if distinguishing the two.

Duplicate detection is only one part of validation. A unique object can still have missing fields, wrong types, unexpected fields or values the caller cannot authorize. Apply the schema and business rules after decoding. Python also accepts non-finite numeric constants by default; a contract that forbids them needs its own parse_constant policy. The hook does not impose a size limit, and Python warns that untrusted JSON can consume substantial CPU and memory. Limit incoming bytes before parsing and avoid logging sensitive input in full. These checks make the behavior of this particular boundary explicit without claiming that every JSON producer follows the same policy.

Choose an error policy

Consider a batch of configuration files. Rejecting the whole file prevents an operator from deploying a partially interpreted setting, but it also means a single typo blocks that file. Return a useful validation error and retain the original file under the application's normal access controls. If compatibility requires accepting duplicates, document which value wins and test that policy across every reader. Do not convert to a dictionary first and then try to detect duplicates: by that point the repeated occurrence is no longer available. Test both escaped and unescaped equivalent names if external producers use escapes.