Skip to content
Distinguish missing rows from None with zip_longest

Distinguish missing rows from None with zip_longest

November 19, 2026

When two exports contain optional values, an extra None row is still an extra row. itertools.zip_longest() pads shorter inputs with None by default, which can make that extra row look equal to the padding. Use a distinct sentinel to identify missing positions.

Set up

Tested on Linux with CPython 3.12.14. Use Bash, Python 3.12 or newer, a text editor, and the mktemp, rm and rmdir utilities. No packages or network access are needed. Keep the same terminal open and stop if setup fails. Run without -O so the assertions remain enabled.

start_dir="$PWD"
work_dir="$(mktemp -d)" || exit 1
cd -- "$work_dir" || exit 1

Run the example

Save as compare_optional_rows.py:

from itertools import zip_longest

left = ["ready", None]
right = ["ready"]
default_pairs = list(zip_longest(left, right))
print("Default padding:", default_pairs)
assert default_pairs[-1] == (None, None)

MISSING = object()


def differences(left_rows, right_rows):
    result = []
    for index, (left_value, right_value) in enumerate(
        zip_longest(left_rows, right_rows, fillvalue=MISSING)
    ):
        if left_value is MISSING:
            result.append((index, "missing-left"))
        elif right_value is MISSING:
            result.append((index, "missing-right"))
        elif left_value != right_value:
            result.append((index, "different-value"))
    return result


assert differences(left, right) == [(1, "missing-right")]
assert differences(right, left) == [(1, "missing-left")]
assert differences([None], [None]) == []
assert differences([], []) == []
assert differences(["ready"], ["failed"]) == [(0, "different-value")]
print("Left has extra None:", differences(left, right))
print("Right has extra None:", differences(right, left))
print("Two real None values:", differences([None], [None]))

Run:

python3 -B compare_optional_rows.py

Observed output:

Default padding: [('ready', 'ready'), (None, None)]
Left has extra None: [(1, 'missing-right')]
Right has extra None: [(1, 'missing-left')]
Two real None values: []

Test sentinel identity before comparing values

The default pair at index one contains two null values, although only the left input contains a row there. The comparison helper supplies a fresh sentinel object instead. Identity checks classify a missing left or right position before ordinary values are compared.

Two real null values still compare as equal. The assertions also cover empty inputs and unequal non-null values. The zip_longest reference documents padding and continuation until the longest input is exhausted.

Keep the sentinel private to the comparison code. It must never be a legitimate input value. Identity checks avoid confusing it with an ordinary value that merely compares equal to something else; use the same sentinel object for padding and testing.

Match the comparison to the data

Rows are aligned by position. This is appropriate only when both exports use the same ordering and positions have meaning. If records have stable IDs and either export can be reordered, match by ID before comparing values.

The helper expects finite inputs containing strings or null values and collects all differences in a list. Other value types may need their own equality rules. Large reports can emit differences incrementally. An infinite input would keep the alignment running, so bound such sources before using this loop.

Clean up

Remove the saved file and empty scratch directory. If the directory is not empty, inspect its contents before removing anything else.

rm -- compare_optional_rows.py
cd -- "$start_dir" || exit 1
rmdir -- "$work_dir"
unset start_dir work_dir

Acknowledgements

This post was created with assistance from AI.

Last updated on