How to clean up segments¶

A box lives in shared memory, and that memory outlives the Python object you use it through. So cleaning up takes two steps: each process closes the boxes it opened, and once no process needs a box any more, you remove the name of its segment. This guide also shows how to recover when a process crashed before cleaning up.

1. Close each box you open¶

Leave a with block, or call close yourself:

from sharedbox import LockTimeoutError, SharedBox


class Job(SharedBox, name="example-job"):
    done: bool = False


job = Job()
job.events.done.connect(lambda new, old: print(job.name, "done:", new))
job.done = True
job.close()

close lets go of this process's handle and stops its watcher thread, while the data stays for the other processes.

A callback that uses the box keeps it open

A callback that refers to the box, as the lambda above does, keeps the box alive, so garbage collection never closes it for you. Call close yourself on such a box.

2. Remove the name¶

unlink removes the name, so no process can attach the box from then on:

Job.unlink()

Call it once, from the process that created the box; Closing and lifetime says why. On Linux the segment stays until it is unlinked. On Windows unlink does nothing, because the segment goes away by itself when its last handle is closed. Either way you write the same code, and it works on both.

Remove a segment left by a crash¶

On Linux, a process that crashed before calling unlink leaves its segment behind as a file in /dev/shm. You can list them:

ls /dev/shm/sharedbox.*

If you try to create a box under a name that is left over, you get SegmentExistsError, and its message says whether the process that created it is still running. If it isn't, remove the segment with unlink and the part of the file name after sharedbox., for example Job.unlink("job-1"), or simply delete the file.

On Windows nothing is left behind: a segment goes away with the last process that has it open.

Release a lock left by a crash¶

Only one process can write to a box at a time, so a writer holds the box's write lock while it writes. A process that dies in the middle of a write leaves that lock taken. From then on, every read and write waits for the class's lock_timeout and then raises LockTimeoutError, whose message names the process that holds the lock. Once you know that process is gone, call force_unlock on any box attached to the same segment, then write again:

job = Job.attach()
try:
    job.done = False
except LockTimeoutError as error:
    print(error)  # ... locked by pid 1234 ...
    # only once process 1234 is no longer running
    job.force_unlock()
    print(job.done)  # True: the write that timed out did not happen
    job.done = False
print(job.done)  # False

The write that timed out didn't happen, so the field keeps its old value until you write it again.

Only unlock a lock whose owner is gone

force_unlock doesn't check that the process holding the lock has stopped. If it is still running, its half-finished write becomes visible to readers. Check the process id (pid) in the error message first, with ps -p <pid> on Linux or tasklist /FI "PID eq <pid>" on Windows; force_unlock has the details.

The whole script

"""The script of the guide "How to clean up segments"."""

from sharedbox import LockTimeoutError, SharedBox


class Job(SharedBox, name="example-job"):
    done: bool = False


job = Job()
job.events.done.connect(lambda new, old: print(job.name, "done:", new))
job.done = True
job.close()

Job.unlink()

creator = Job(True)

job = Job.attach()
try:
    job.done = False
except LockTimeoutError as error:
    print(error)  # ... locked by pid 1234 ...
    # only once process 1234 is no longer running
    job.force_unlock()
    print(job.done)  # True: the write that timed out did not happen
    job.done = False
print(job.done)  # False

job.close()
creator.close()
Job.unlink()