Decode partial subprocess output after a timeout
A command runner can return text on success and bytes in its timeout exception. A timeout can also interrupt a multibyte character. Handle those cases explicitly before adding captured output to a diagnostic string.
Set up
Tested on Linux with CPython 3.12.14. Use Bash, Python 3.12 or newer, a text editor, and the mktemp, rm and rmdir utilities. No packages or network access are needed. Keep the same terminal open and stop if setup fails. Run without -O so the assertions remain enabled.
start_dir="$PWD"
work_dir="$(mktemp -d)" || exit 1
cd -- "$work_dir" || exit 1Run the example
Save as timeout_output.py:
import subprocess
import sys
def diagnostic_text(value):
if value is None:
return ""
if isinstance(value, bytes):
return value.decode("utf-8", errors="backslashreplace")
return value
child_code = "import os, time; os.write(1, b'ready \\xe2\\x82'); time.sleep(60)"
try:
subprocess.run(
[sys.executable, "-c", child_code],
capture_output=True,
text=True,
encoding="utf-8",
timeout=2,
check=True,
)
except subprocess.TimeoutExpired as error:
assert isinstance(error.stdout, bytes), "child did not produce output in time"
assert error.stdout == b"ready \xe2\x82"
print("Captured type:", type(error.stdout).__name__)
print("Diagnostic:", diagnostic_text(error.stdout))
print("No stderr:", diagnostic_text(error.stderr) == "")
else:
raise AssertionError("expected timeout")
assert diagnostic_text(None) == ""
assert diagnostic_text("complete") == "complete"
assert diagnostic_text(b"caf\xc3\xa9") == "caf\u00e9"Run:
python3 -B timeout_output.pyObserved output:
Captured type: bytes
Diagnostic: ready \xe2\x82
No stderr: TrueDecode the exception output explicitly
The child writes an incomplete UTF-8 sequence directly to standard output, then sleeps. The two-second timeout interrupts that sleep. The caught exception contains bytes despite the text-mode request. Strict decoding would fail on the unfinished character, so the helper renders invalid bytes as escape sequences.
The subprocess exception reference specifies bytes for captured timeout output and permits None when no output was observed. The helper accepts either case, plus a string from a normal text result. It assumes UTF-8; choose the encoding promised by the child process.
This is diagnostic decoding. It does not recover the missing character or make partial output a complete application record. Keep the original bytes if later analysis needs their exact values. The escaped diagnostic alone is not a lossless serialization because literal backslashes can look like generated escapes.
Keep timeout and capture limits explicit
The direct child is killed and waited for by subprocess.run() before it re-raises the timeout. This example starts no grandchildren; it does not demonstrate terminating a process tree. Process creation can delay timeout delivery, and a heavily loaded machine might not run the child before the deadline. The assertions fail explicitly if the expected bytes were not captured.
Captured output is buffered in memory. A timeout does not impose an output-size limit. Use a different capture strategy for a command that can produce unbounded data. The argument list uses the default shell=False, and the child source is a fixed literal; do not substitute untrusted input into that source.
Clean up
Remove the saved file and empty scratch directory. If the directory is not empty, inspect its contents before removing anything else.
rm -- timeout_output.py
cd -- "$start_dir" || exit 1
rmdir -- "$work_dir"
unset start_dir work_dirAcknowledgements
This post was created with assistance from AI.