Python Basics: Run a Spawned Worker with Explicit Result Handling

A spawned process imports the program in a fresh interpreter. Put startup behind the main guard, keep the target function importable, and make parent-side result and shutdown handling explicit.

Prepare a scratch directory

Tested on Linux with CPython 3.12.14. You need Bash, Python, a text editor, and the standard mktemp, rm, and rmdir utilities. No external packages are required. Run without -O so assertions remain enabled.

Keep the same terminal open. Stop if setup fails, before saving the source:

start_dir="$PWD"
work_dir="$(mktemp -d)" || exit 1
cd -- "$work_dir" || exit 1

Run the example

Save this as multiprocessing-spawn-demo.py:

import multiprocessing as mp


def summarize(values, sender):
    try:
        sender.send({"count": len(values), "total": sum(values)})
    finally:
        sender.close()


def main():
    context = mp.get_context("spawn")
    receiver, sender = context.Pipe(duplex=False)
    worker = context.Process(target=summarize, args=([4, 7, 9], sender))
    started = False
    try:
        worker.start()
        started = True
        sender.close()
        if not receiver.poll(10):
            raise TimeoutError("worker did not provide a result")
        result = receiver.recv()
        worker.join(timeout=10)
        if worker.is_alive():
            raise TimeoutError("worker did not exit")
        if worker.exitcode != 0:
            raise RuntimeError(f"worker exit code: {worker.exitcode}")
        assert result == {"count": 3, "total": 20}
        print("Start method:", context.get_start_method())
        print("Count:", result["count"])
        print("Total:", result["total"])
        print("Worker exit:", worker.exitcode)
    finally:
        sender.close()
        receiver.close()
        if started:
            if worker.is_alive():
                worker.terminate()
            worker.join()
        worker.close()


if __name__ == "__main__":
    main()

Run it:

python3 -B multiprocessing-spawn-demo.py

Observed output:

Start method: spawn
Count: 3
Total: 20
Worker exit: 0

Read the result

The parent receives the result before joining, then verifies that the child exited successfully. Explicitly selecting spawn avoids relying on a platform default. The worker function lives at module level so the new interpreter can find it. The main guard prevents the child import from starting another worker.

Apply it carefully

Run this saved file rather than pasting it into an interactive prompt. Pipe messages use pickle; communicate only with trusted processes. The ten-second checks detect slow result delivery and shutdown, but are not a strict whole-program deadline. Forced termination is a fallback for this isolated worker and can skip its cleanup; workers holding shared resources need a cooperative cancellation design.

The official multiprocessing reference documents the APIs used here.

Clean up

Remove the saved source, return to the starting directory, and remove the empty practice directory:

rm -- multiprocessing-spawn-demo.py
cd -- "$start_dir" || exit 1
rmdir -- "$work_dir"
unset start_dir work_dir

If the directory is not empty, inspect its remaining contents before removing anything else.

Acknowledgements

This post was created with assistance from AI.