How to send a box to another process¶

A process that didn't create a box can get it in two ways: you hand it over as an argument when the process starts, or the process opens it by name with attach. This guide shows both with multiprocessing, and when to prefer which.

1. Pass the box as an argument¶

Hand the box to the new process like any other argument:

import multiprocessing as mp

from sharedbox import SharedBox


class Status(SharedBox, name="example-status"):
    last_item: int = -1


def report(status: Status) -> None:
    status.last_item = 99
    status.close()

if __name__ == "__main__":
    context = mp.get_context("spawn")
    with Status() as status:
        child = context.Process(target=report, args=(status,))
        child.start()
        child.join()
        print(status.last_item)  # 99
        with context.Pool(4, initializer=start_worker, initargs=(status.name,)) as pool:
            print(sum(pool.map(work, range(100))))  # 9900
    Status.unlink()

This works however the child is started. multiprocessing can start a child in three ways, its start methods; switch between them to see what the child gets:

What the child gets

With spawn or forkserver, the box travels as a pickle of its name and ids. The child unpickles it and opens the same shared memory.

With fork, the child starts as a copy of the parent and keeps using the parent's box object. Nothing is pickled.

multiprocessing.shared_memory.SharedMemory travels the same way. See SharedBox for what the pickle holds, and the handle entry for what a second box object on the same data means.

2. In a pool, attach once per worker¶

Each time a box is unpickled, the child opens its segment again. If you pass the box with every task of a pool, it gets opened once per task, which is wasted work. Pass the box's name to the pool's initializer instead, and attach there, once per worker:

worker_status: Status


def start_worker(name: str) -> None:
    global worker_status
    worker_status = Status.attach(name)


def work(item: int) -> int:
    worker_status.last_item = item
    return item * 2

main above starts the pool with initargs=(status.name,).

3. Choose the start method¶

Windows only has spawn. On Linux the default is fork before Python 3.14 and forkserver from 3.14 on. To choose one yourself, use multiprocessing.get_context, as the script does.

Forking a box with callbacks can hang

If the box has callbacks connected to its events, a child forked while a callback runs can hang, and a forked child forwards nothing from follow until it calls follow again. Use spawn or forkserver for such a box.

Keeping a pickle for later¶

A pickled box only says which box it is; it doesn't hold the values, so it's no good for saving them. Loading it after the box was removed raises SegmentNotFoundError, and loading it after a new box was created under the same name raises SchemaMismatchError. To keep the values, store a snapshot instead.

The whole script

"""The script of the guide "How to send a box to another process"."""

import multiprocessing as mp

from sharedbox import SharedBox


class Status(SharedBox, name="example-status"):
    last_item: int = -1


def report(status: Status) -> None:
    status.last_item = 99
    status.close()




worker_status: Status


def start_worker(name: str) -> None:
    global worker_status
    worker_status = Status.attach(name)


def work(item: int) -> int:
    worker_status.last_item = item
    return item * 2




if __name__ == "__main__":
    context = mp.get_context("spawn")
    with Status() as status:
        child = context.Process(target=report, args=(status,))
        child.start()
        child.join()
        print(status.last_item)  # 99
        with context.Pool(4, initializer=start_worker, initargs=(status.name,)) as pool:
            print(sum(pool.map(work, range(100))))  # 9900
    Status.unlink()