Skip to content

Reference fields

A reference field lets one box point at another, the way an attribute of one Python object can hold another object. Pointing across processes is harder than inside one, though: there are no shared Python objects to point at, only names. This page explains what the field stores, how a process finds the class to open the other box with, and how a reference can break. The rules for declaring, assigning and reading reference fields are in SharedBox.

What a reference stores

A reference field doesn't copy the other box into this one. The other box keeps its own segment, lock and lifetime, and the field stores just enough to open it again, from any process:

  • the box's name, which attach takes;
  • the schema hash of the box's own class, to find the class to attach it with;
  • the box's create id, to tell it apart from a box created later under the same name.

Because the outer box only points at the other box, each keeps its own sequence lock. That means no single write can change both boxes at once, and closing or unlinking the outer box leaves the other alone.

Finding the class

The stored schema hash says which class the box was created with, but to open the box a process needs that class itself, defined in its own code. So each process keeps a list of the SharedBox classes it has defined, looked up by schema hash, and a class joins the list as soon as it is defined. Reading a reference field looks the stored hash up there.

Import the class before you read the reference

If a process hasn't imported the module that defines the other box's class, the lookup finds nothing and reading the field raises UnknownBoxClassError. Import that module in every process that reads the reference.

The list holds each class weakly, so a class nothing else uses, such as one defined inside a function, can still be freed. That's why the lookup takes the first matching class "that is still alive".

Broken references

A reference can outlive the box it names, and the name alone can't show it. Step through a stage whose motor is replaced under the same name:

A reference that breaks

The field stage.motor stores the name motor-x and the create id 41 of the box it points at.

stage.motorThe reference field stores the motor's name, schema hash and create id.motor-xcreate id 41The box the reference was made for.1    Motor.unlink("motor-x") 2    Motor.create("motor-x")1    Motor.unlink("motor-x") 2    Motor.create("motor-x")1    stage = Stage.attach() 2    motor = stage.motor1    stage = Stage.attach() 2    motor = stage.motor points atThe reference field stores the motor's name, schema hash and create id. The box the reference was made for.
stage.motorThe reference field stores the motor's name, schema hash and create id.motor-xcreate id 41The box the reference was made for.1    Motor.unlink("motor-x") 2    Motor.create("motor-x")1    Motor.unlink("motor-x") 2    Motor.create("motor-x")1    stage = Stage.attach() 2    motor = stage.motor1    stage = Stage.attach() 2    motor = stage.motor points atThe reference field stores the motor's name, schema hash and create id. The box the reference was made for.

Another process unlinks motor-x and creates a new box under the same name. The new box gets create id 77.

stage.motorThe reference field stores the motor's name, schema hash and create id.motor-xcreate id 77The box the reference was made for.1 -> Motor.unlink("motor-x") 2 -> Motor.create("motor-x")1 -> Motor.unlink("motor-x") 2 -> Motor.create("motor-x")1    stage = Stage.attach() 2    motor = stage.motor1    stage = Stage.attach() 2    motor = stage.motor points atThe reference field stores the motor's name, schema hash and create id. The box the reference was made for.
stage.motorThe reference field stores the motor's name, schema hash and create id.motor-xcreate id 77The box the reference was made for.1 -> Motor.unlink("motor-x") 2 -> Motor.create("motor-x")1 -> Motor.unlink("motor-x") 2 -> Motor.create("motor-x")1    stage = Stage.attach() 2    motor = stage.motor1    stage = Stage.attach() 2    motor = stage.motor points atThe reference field stores the motor's name, schema hash and create id. The box the reference was made for.

A process that now reads stage.motor finds create id 77 where the field stored 41, so it raises BrokenReferenceError instead of opening the wrong box.

stage.motorThe reference field stores the motor's name, schema hash and create id.motor-xcreate id 77The box the reference was made for.1    Motor.unlink("motor-x") 2    Motor.create("motor-x")1    Motor.unlink("motor-x") 2    Motor.create("motor-x")1    stage = Stage.attach() 2 -> motor = stage.motor1    stage = Stage.attach() 2 -> motor = stage.motor 41, not 77The reference field stores the motor's name, schema hash and create id. The box the reference was made for.
stage.motorThe reference field stores the motor's name, schema hash and create id.motor-xcreate id 77The box the reference was made for.1    Motor.unlink("motor-x") 2    Motor.create("motor-x")1    Motor.unlink("motor-x") 2    Motor.create("motor-x")1    stage = Stage.attach() 2 -> motor = stage.motor1    stage = Stage.attach() 2 -> motor = stage.motor 41, not 77The reference field stores the motor's name, schema hash and create id. The box the reference was made for.

The first time your box object reads the field, it compares the stored create id with that of the box under that name now, and raises BrokenReferenceError if the box is gone or is a different one. Once it has opened the other box, your box object keeps it open and hands you the same one on later reads, so it keeps working even if someone unlinks the name in the meantime; it only looks again when the field is pointed at a different box. See SharedBox for the details.

A reference can break at any time, from another process, long after you called follow, when there's no call of yours to raise an error into. So follow doesn't raise for a broken reference: it logs a warning and forwards nothing until the field is pointed at another box.

A class that refers to itself

A reference field may name the very class it's in, which lets you build a chain of boxes. If the field were required, you couldn't create the first box, because it would need an existing box of the same class, just as with a dataclass. So declare such a field with an empty default:

from __future__ import annotations

from sharedbox import SharedBox


class Node(SharedBox):
    value: int = 0
    next: Node | None = None

Before Python 3.14, and without from __future__ import annotations, put the annotation in quotes instead: next: "Node | None" = None.