Skip to content

PyPI PyPI - Python Version License CI uv Ruff Checked with mypy Conventional Commits

sharedbox

sharedbox lets several Python processes share one record, as if they all held the same dataclass. You declare the fields once, and every process that opens the record reads and writes the same values. The values sit in shared memory, a block of memory the operating system lets several processes use at once, so no process has to send them to another. Such a record is a box.

A first look

Here a parent process creates a box, starts a child that moves the motor, and prints the change the moment it happens:

import multiprocessing as mp
from typing import Annotated

from sharedbox import Capacity, SharedBox


class Motor(SharedBox):
    position: int
    enabled: bool
    label: Annotated[str, Capacity(32)]


def worker() -> None:
    motor = Motor.attach()          # finds the box by its class
    motor.position = 10
    motor.close()


if __name__ == "__main__":
    with Motor(1, False, "x-axis") as motor:
        motor.events.position.connect(lambda new, old: print(old, "->", new))
        child = mp.Process(target=worker)
        child.start()
        child.join()                      # prints: 1 -> 10
    Motor.unlink()

Capacity(32) gives the text field room for 32 bytes, since every field has a fixed size. Running the script prints:

1 -> 10

Step through what happens, and point at a shape to read more:

The parent creates the box and connects a callback to position.

parent processCreates the box with Motor(1, False, "x-axis") and connects a callback to events.position.the boxin shared memoryOne named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes.1 -> with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3        child.start()1 -> with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3        child.start()1    motor = Motor.attach() 2    motor.position = 10 3    motor.close()1    motor = Motor.attach() 2    motor.position = 10 3    motor.close() createsattaches by classposition = 10prints 1 -> 10Creates the box with Motor(1, False, "x-axis") and connects a callback to events.position. One named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes.
parent processCreates the box with Motor(1, False, "x-axis") and connects a callback to events.position.the boxin shared memoryOne named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes.1 -> with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3        child.start()1 -> with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3        child.start()1    motor = Motor.attach() 2    motor.position = 10 3    motor.close()1    motor = Motor.attach() 2    motor.position = 10 3    motor.close() createsattaches by classposition = 10prints 1 -> 10Creates the box with Motor(1, False, "x-axis") and connects a callback to events.position. One named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes.

The child starts and attaches to the box. Nobody hands it the box: Motor.attach() works out its name from the class.

parent processCreates the box with Motor(1, False, "x-axis") and connects a callback to events.position.the boxin shared memoryOne named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes.child processRuns worker() in a separate process.1    with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3 ->     child.start()1    with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3 ->     child.start()1 -> motor = Motor.attach() 2    motor.position = 10 3    motor.close()1 -> motor = Motor.attach() 2    motor.position = 10 3    motor.close() createsattaches by classposition = 10prints 1 -> 10Creates the box with Motor(1, False, "x-axis") and connects a callback to events.position. One named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes. Runs worker() in a separate process.
parent processCreates the box with Motor(1, False, "x-axis") and connects a callback to events.position.the boxin shared memoryOne named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes.child processRuns worker() in a separate process.1    with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3 ->     child.start()1    with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3 ->     child.start()1 -> motor = Motor.attach() 2    motor.position = 10 3    motor.close()1 -> motor = Motor.attach() 2    motor.position = 10 3    motor.close() createsattaches by classposition = 10prints 1 -> 10Creates the box with Motor(1, False, "x-axis") and connects a callback to events.position. One named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes. Runs worker() in a separate process.

The child writes position. The new value is in shared memory at once, for every process.

parent processCreates the box with Motor(1, False, "x-axis") and connects a callback to events.position.the boxin shared memoryOne named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes.child processRuns worker() in a separate process.1    with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3 ->     child.start()1    with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3 ->     child.start()1    motor = Motor.attach() 2 -> motor.position = 10 3    motor.close()1    motor = Motor.attach() 2 -> motor.position = 10 3    motor.close() createsattaches by classposition = 10prints 1 -> 10Creates the box with Motor(1, False, "x-axis") and connects a callback to events.position. One named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes. Runs worker() in a separate process.
parent processCreates the box with Motor(1, False, "x-axis") and connects a callback to events.position.the boxin shared memoryOne named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes.child processRuns worker() in a separate process.1    with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3 ->     child.start()1    with Motor(1, False, "x-axis") as motor: 2        motor.events.position.connect(lambda new, old: ...) 3 ->     child.start()1    motor = Motor.attach() 2 -> motor.position = 10 3    motor.close()1    motor = Motor.attach() 2 -> motor.position = 10 3    motor.close() createsattaches by classposition = 10prints 1 -> 10Creates the box with Motor(1, False, "x-axis") and connects a callback to events.position. One named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes. Runs worker() in a separate process.

The parent's watcher thread wakes on the write and calls the callback, which prints 1 -> 10.

parent processCreates the box with Motor(1, False, "x-axis") and connects a callback to events.position.the boxin shared memoryOne named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes.child processRuns worker() in a separate process.1    with Motor(1, False, "x-axis") as motor: 2 ->     motor.events.position.connect(lambda new, old: ...) 3        child.start()1    with Motor(1, False, "x-axis") as motor: 2 ->     motor.events.position.connect(lambda new, old: ...) 3        child.start()1    motor = Motor.attach() 2    motor.position = 10 3    motor.close()1    motor = Motor.attach() 2    motor.position = 10 3    motor.close() createsattaches by classposition = 10prints 1 -> 10Creates the box with Motor(1, False, "x-axis") and connects a callback to events.position. One named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes. Runs worker() in a separate process.
parent processCreates the box with Motor(1, False, "x-axis") and connects a callback to events.position.the boxin shared memoryOne named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes.child processRuns worker() in a separate process.1    with Motor(1, False, "x-axis") as motor: 2 ->     motor.events.position.connect(lambda new, old: ...) 3        child.start()1    with Motor(1, False, "x-axis") as motor: 2 ->     motor.events.position.connect(lambda new, old: ...) 3        child.start()1    motor = Motor.attach() 2    motor.position = 10 3    motor.close()1    motor = Motor.attach() 2    motor.position = 10 3    motor.close() createsattaches by classposition = 10prints 1 -> 10Creates the box with Motor(1, False, "x-axis") and connects a callback to events.position. One named block of shared memory holding position, enabled and label. Every process that opens it reads and writes the same bytes. Runs worker() in a separate process.
One box, two processes

Is it for you?

sharedbox fits when separate processes need the same small set of values, such as the state of a device that one process controls and another shows. Reading or writing a number or a short string takes well under a microsecond, and a process that is reading never holds up one that is writing.

If your code runs in threads of one process instead, you can share ordinary Python objects, which is simpler; When to use sharedbox compares the two.

sharedbox runs on CPython 3.11 or newer, on Windows and Linux; How to install sharedbox has the details.

Where to go next

  • Tutorials: start here if sharedbox is new to you, and build a small script one step at a time.
  • How-to Guides: install sharedbox, then get one task done, such as storing an array or naming a box.
  • Explanations: how a box works inside, and why it is built that way.
  • Reference: the API, the memory layout and the C and C++ interface.
  • Changelog: what changed in each release.
  • Contributing: set up a copy you can change, run the tests and send your change.