Python: Reject Ambiguous JSON Settings
A settings file with two retries keys should fail review instead of silently choosing one value. Python’s default JSON decoder keeps the last occurrence. This example adds duplicate detection, rejects non-finite numbers, and checks a small application schema.
Prepare a scratch directory
Tested on Linux with Python 3.12.14. Use Python 3.12 or newer, Bash, a text editor, and the mktemp, rm, and rmdir utilities. No third-party packages are needed. Keep the same terminal open. Stop if setup fails, before saving any source:
start_directory="$PWD"
work_directory="$(mktemp -d)" || exit 1
cd -- "$work_directory" || exit 1Parse first, then validate the settings
Save as validate-json-settings.py:
import json
import math
def unique_object(pairs):
result = {}
for key, value in pairs:
if key in result:
raise ValueError(f"duplicate key: {key}")
result[key] = value
return result
def reject_constant(token):
raise ValueError(f"non-finite constant: {token}")
def finite_float(token):
value = float(token)
if not math.isfinite(value):
raise ValueError("number outside finite float range")
return value
def load_settings(text):
if len(text) > 4096:
raise ValueError("settings exceed 4096 characters")
settings = json.loads(
text,
object_pairs_hook=unique_object,
parse_constant=reject_constant,
parse_float=finite_float,
)
if type(settings) is not dict or set(settings) != {"retries"}:
raise ValueError("expected an object containing only retries")
retries = settings["retries"]
if type(retries) is not int or not 0 <= retries <= 5:
raise ValueError("retries must be an integer from 0 to 5")
return settings
samples = [
("valid", '{"retries": 2}'),
("duplicate", '{"retries": 1, "retries": 4}'),
("nested duplicate", '{"retries": {"n": 1, "n": 2}}'),
("constant", '{"retries": NaN}'),
("overflow", '{"retries": 1e400}'),
("boolean", '{"retries": true}'),
("extra key", '{"retries": 2, "debug": true}'),
]
for label, text in samples:
try:
print(f"{label}: {load_settings(text)}")
except ValueError as error:
print(f"{label}: rejected ({error})")Run:
python3 validate-json-settings.pyObserved output:
valid: {'retries': 2}
duplicate: rejected (duplicate key: retries)
nested duplicate: rejected (duplicate key: n)
constant: rejected (non-finite constant: NaN)
overflow: rejected (number outside finite float range)
boolean: rejected (retries must be an integer from 0 to 5)
extra key: rejected (expected an object containing only retries)Why the checks are separate
object_pairs_hook receives each object’s pairs before they collapse into a dictionary, including nested objects. Duplicate detection therefore happens while both values are still visible. Escaped keys are decoded first, so two spellings of the same decoded key also collide.
parse_constant handles the decoder’s NaN and infinity extensions. A syntactically ordinary number such as 1e400 follows a different path: conversion to a float can overflow to infinity. finite_float handles that path. It does not make binary floating-point arithmetic exact.
The schema then requires exactly one key and a bounded integer. An explicit type check excludes True, because Python otherwise treats booleans as instances of int. A valid JSON array or a floating-point value such as 2.0 is still invalid for these settings.
Keep the boundary narrow
This function accepts an already-loaded string. The length check limits characters passed to the parser; it does not limit the bytes read from disk or a network. Apply an input-byte limit while reading, and handle syntax errors, excessive nesting, and decoding failures at the application’s input boundary. JSONDecodeError is a ValueError; excessive nesting may instead raise RecursionError.
Do not describe strict=True as a duplicate-key or finite-number check: that option controls unescaped control characters in strings. See the Python JSON decoder reference for the hooks used here.
Clean up
rm -- validate-json-settings.py
cd -- "$start_directory" || exit 1
rmdir -- "$work_directory"
unset start_directory work_directoryIf the directory is not empty, inspect what remains before removing anything else.
Acknowledgements
This post was created with assistance from AI.