Python Basics: Packing Binary Records with struct

A small sensor record might contain a device number and a reading. Python’s struct module lets you turn those numbers into a defined sequence of bytes and read them back.

These examples use Linux, Bash, and Python 3.12 or 3.13, with no extra packages. The three scripts are independent and only print results. Create a disposable working directory, then save all files there using the names shown:

start_dir="$PWD"
struct_demo_dir=$(mktemp -d) || exit 1
cd "$struct_demo_dir" || exit 1

Keep this Bash session open through cleanup. Check that python3 --version reports the version you want to use.

Pack one record

Save as demo_struct_record.py:

import struct

record = struct.Struct(">Hh")
packed = record.pack(513, -125)
device_id, reading = record.unpack(packed)
print("Bytes per record:", record.size)
print("Hex:", packed.hex(" "))
print("Device:", device_id)
print("Reading:", reading)

Run it:

python3 demo_struct_record.py

Output:

Bytes per record: 4
Hex: 02 01 ff 83
Device: 513
Reading: -125

> selects big-endian byte order: the most significant byte comes first. H is a two-byte unsigned integer; h is a two-byte signed integer. This format uses four bytes without padding. Avoid an omitted prefix for exchange formats, since it selects platform-dependent native layout.

Our fictional sensor stores the reading as an integer. The producer and consumer must also agree on its unit; the bytes alone do not tell you that.

Read consecutive records

Save as demo_struct_batch.py:

import struct

record = struct.Struct(">Hh")
payload = b"".join(record.pack(*row) for row in [(1, 230), (2, -45)])
for device_id, reading in record.iter_unpack(payload):
    print(f"Device {device_id}: {reading}")

print("Empty batch:", list(record.iter_unpack(b"")))

Run it:

python3 demo_struct_batch.py

Output:

Device 1: 230
Device 2: -45
Empty batch: []

iter_unpack() yields one tuple per record. The payload length must be a multiple of the record size. An empty payload contains zero records. Both scripts define their own format and sample data, so this example does not need the first script.

Reject incomplete or out-of-range data

Save as demo_struct_errors.py:

import struct

record = struct.Struct(">Hh")
for device_id in [0, 65535, 65536, -1]:
    try:
        record.pack(device_id, 10)
    except struct.error:
        print(f"Device {device_id}: out of range")
    else:
        print(f"Device {device_id}: accepted")

try:
    record.unpack(b"\x00\x01\x00")
except struct.error:
    print("Incomplete record: rejected")

Run it:

python3 demo_struct_errors.py

Output:

Device 0: accepted
Device 65535: accepted
Device 65536: out of range
Device -1: out of range
Incomplete record: rejected

These failures are intentional and caught. H accepts 0 through 65535; h accepts -32768 through 32767. unpack() requires exactly one record’s bytes, including rejecting extra bytes. Length checks do not validate a record’s meaning: a device number can fit the format and still be unknown to your application.

Clean up

From the same working directory and Bash session, remove only the three files you created, then the empty directory:

rm -- demo_struct_record.py demo_struct_batch.py demo_struct_errors.py
cd "$start_dir" || exit 1
rmdir -- "$struct_demo_dir"
unset start_dir struct_demo_dir

If you saved extra files there, rmdir leaves the directory in place. Inspect it rather than deleting unrelated files.

References

• Python struct documentation

Acknowledgements

This post was created with assistance from AI.