Periodic Framework — Python API
Write a Python function, name its period and its topics, and TAIL OS runs it on time.
Read this first
- What you need: the TAIL OS image for QEMU. Its disk image carries Python and the
periodicmodule, so there is nothing to install.import periodicloads the module from the standard library archive/usr/lib/python314.zip. The standard payloads,tail_messages, are a file of their own beside it,/usr/lib/python3.14/tail_messages.py. - What you get: a node that runs on a fixed period, reads and publishes topics, and needs no loop, thread, sleep or retry.
- Time to a first node: one file of 21 lines, written with
vion the booted image and started withpython. There is no build.
This page is the Python API. The Rust page describes the Rust API. Both use the same words for the same things. Where the two differ, this page says why.
Contents
- Quick start
- Your first nodes
- Declaring a node
- Run-time parameters
- Actions
- Recording and replay
- Payloads
Cyclerun()- Safety
- Cost
- Errors
inithas no equivalent, and does not need one- What is verified
- Glossary
A node is a function that TAIL OS runs once every period. Each run is a release. The framework runs the timing loop for you: it reads the clock, calls the function, and sleeps for the rest of the period.
A node has inputs, the topics it reads, and outputs, the topics it publishes. You declare each one on the parameter that receives it. Each input has its own freshness bound. A node can also take run-time parameters that you tune from the shell, and serve one action: a goal a client sends it, which takes many releases.
Quick start
counter is one file and one node. It runs every 100 ms, counts its releases, and
publishes the count:
# counter.py
from periodic import periodic, publish, run, now, Payload, Skip, Topic
# What the counter publishes on every release: releases run so far, and when this one ran.
Releases = Payload("count:u64 stamp_ns:u64", "Releases")
# The counter's releases, published by the counter alone.
COUNTER_RELEASES = Topic("counter/releases", Releases, lane="kernel")
count = 0
@periodic(every="100ms", on_overrun=Skip)
def counter(releases=publish(COUNTER_RELEASES)):
global count
count += 1
releases.publish(Releases.instance(count=count, stamp_ns=now()))
run()
@periodicturns the function into a node, andrun()runs it. The file has no loop and no sleep.releases=publish(COUNTER_RELEASES)is the node's output. The value the body hands toreleases.publish(..)is published oncounter/releaseswhen the body returns.countlives at module level. That is how a Python node keeps state between releases.Releasesis a payload, the type a topic carries.Payloadlays it out exactly as the same struct in Rust.
Run and check it
Python and the periodic module are on the disk image of the
QEMU download. On the booted image, write the file with
vi /counter.py, then start it in the background and look at it:
tsh> python /counter.py &
tsh> node list
counter pid 9 python
tsh> topic echo -n 3 --as "count:u64 stamp_ns:u64" counter/releases
# counter/releases: kernel lane, --as, 16 B, layout 0x79e81151
count=2 stamp_ns=3888115504
count=3 stamp_ns=3988053776
count=4 stamp_ns=4088056880
A Python node takes a few seconds to start while the interpreter loads, so wait a moment
before node list. node list shows each running node: its name, its process and its
program. Your pid and counts differ.
topic echo reads the topic as any other program would. --as names the payload's
fields, because Releases is not a standard payload that topic echo decodes on its own.
Each count is one more release. The layout identity, 0x79e81151, is the Rust quick
start's, because the two Releases declarations have the same fields. The
shell tools page describes both tools.
Your first nodes
The main example is drive, a node that reads two topics and publishes one. A topic
is a named channel that carries one payload type. The lane is how a topic's messages
travel from publisher to readers (Topics and lanes).
drive reads sensor/range and cmd/velocity and publishes drive/velocity; the other
ends can be Rust nodes or Python nodes.
The section shows four programs:
| Program | What it does |
|---|---|
publisher.py |
publishes one topic |
subscriber.py |
reads one topic |
drive.py |
reads two topics and publishes one |
steer.py |
reads one topic and publishes two |
None of them writes a loop or a sleep. None declares a standard payload: each imports it
from tail_messages. That module is generated from the Rust robot_messages crate, so a
Python node and a Rust node hold one definition. tail_messages is a file on the QEMU
image. The Raspberry Pi 3 image built from the BSP does not include Python.
A topic
from tail_messages import Range
# The front ranger's readings, in the `range_front` frame.
SENSOR_RANGE = Topic("sensor/range", Range, lane="safe")
A topic is defined once, with three facts: its name, its payload and its lane. All three
belong to the topic, not to a node, because every end of a topic must agree on them. The
lane is "kernel", "safe" or "fast", and it has no default. A payload carries no
frame: the frame belongs to the topic, and its comment names it.
A Python publisher
examples/publisher.py:
from periodic import periodic, publish, run, monotonic_ns, Skip, Topic
from tail_messages import Range, RANGE_INFRARED
# The front ranger's readings, in the `range_front` frame.
FROM_PYTHON = Topic("pair/from_python", Range, lane="kernel")
count = 0
@periodic(every="20ms", on_overrun=Skip)
def sensor(reading=publish(FROM_PYTHON)):
global count
count += 1
reading.publish(Range.instance(stamp_ns=monotonic_ns(), radiation_type=RANGE_INFRARED,
field_of_view=0.05, min_range=0.02, max_range=4.0,
range=0.25 * count, variance=0.0))
run()
An output is declared on the parameter that publishes it: reading=publish(FROM_PYTHON).
On every release the body gets an Out in that parameter. reading.publish(value) hands
it the release's value, and the framework publishes it once the body returns. A body
returns nothing.
RANGE_INFRARED is one of Range's named values, generated beside it. count keeps
state at module level; below is why
Python needs no init for that.
Pick the stamp clock by what the node does:
| The node | Stamps with |
|---|---|
reads hardware, like sensor |
monotonic_ns(): nanoseconds since boot, read when called, the instant it read the device |
| decides from its inputs | now(): the current release's one clock reading in nanoseconds, the instant every input was judged against; 0 before run() has opened the node |
| passes an input on | the input's own stamp_ns |
Both read the clock every stamp_ns field is in, the same one as periodic::monotonic_ns()
and periodic::now() in Rust. A replay of the node hands the body the recorded now(), so
a stamp from now() replays exactly; monotonic_ns() reads anew. See
Recording and replay.
A process is one node, so now() takes no argument. Call it from the body: from a thread
you started, it raises RuntimeError.
A Python subscriber
The declaration and the fail-safe branch of examples/subscriber.py. The rest of its body
counts correct readings and exits with a verdict:
from periodic import periodic, run, subscribe, Skip, Topic
from tail_messages import Range, RANGE_ULTRASOUND
# The front ranger's readings, in the `range_front` frame.
SENSOR_RANGE = Topic("sensor/range", Range, lane="safe")
@periodic(every="20ms", on_overrun=Skip)
def brake(lidar=subscribe(SENSOR_RANGE, stale_after="210ms")):
global seen
reading = lidar.get()
# `get()` is None when nothing has arrived yet and when what last arrived is older than
# `stale_after`. The two collapse on purpose, so this is the fail-safe branch.
if reading is None:
return
An input is declared on the parameter it feeds:
lidar=subscribe(SENSOR_RANGE, stale_after="210ms"). On every release the body gets a
Cycle in that parameter. stale_after says how old a reading may be before it
counts as absent.
A node that reads two topics
examples/drive.py, the whole program. It passes the commanded velocity on while the
range reading says the way is clear, and publishes a stop otherwise:
from periodic import periodic, publish, run, subscribe, Skip, Topic
from tail_messages import Range, Twist, Vector3
# Velocity commands, in the `base_link` frame.
CMD_VELOCITY = Topic("cmd/velocity", Twist, lane="kernel")
# The front ranger's readings, in the `range_front` frame.
SENSOR_RANGE = Topic("sensor/range", Range, lane="safe")
# The velocity the drive is given, in the `base_link` frame.
DRIVE_VELOCITY = Topic("drive/velocity", Twist, lane="safe")
STOP_WITHIN_M = 0.3
STILL = Vector3.instance(x=0.0, y=0.0, z=0.0)
STOP = Twist.instance(linear=STILL, angular=STILL)
@periodic(every="20ms", on_overrun=Skip)
def drive(command=subscribe(CMD_VELOCITY, stale_after="310ms"),
lidar=subscribe(SENSOR_RANGE, stale_after="70ms"),
velocity=publish(DRIVE_VELOCITY)):
twist, reading = command.get(), lidar.get()
if twist is not None and reading is not None and reading.range > STOP_WITHIN_M:
velocity.publish(twist)
else:
# The command or the range is absent or stale, or an obstacle is close.
velocity.publish(STOP)
run()
Two inputs on two lanes, each with its own bound, and one output. The body gets one
Cycle per input and one Out per output, in the order the parameters are written. A
command may be 310 ms old and still be used; a range reading only 70 ms.
get() returns None both when nothing has arrived and when what it holds is older than
stale_after. So one else branch covers a sensor that goes quiet and a sensor that never
came up.
Twist nests two Vector3s, read as twist.linear.x, the name Rust uses. You build a
nested payload from instances of the payloads it nests.
Start the nodes in any order. A safe-lane input waits up to ten seconds for its publisher; a kernel-lane input opens at once.
A node that publishes two topics
examples/steer.py, the whole program. It steers towards x = 2 m. It commands a velocity
on every release that has a usable odometry reading. On the releases the robot stands
still, it also reports where it stopped:
from periodic import periodic, publish, run, subscribe, Payload, Skip, Topic
from tail_messages import Odometry, Twist, Vector3
DriveState = Payload("stamp_ns:u64 x:f64 y:f64", "DriveState")
# The robot's estimated pose and velocity, in the `odom` frame.
ODOMETRY = Topic("odometry", Odometry, lane="safe")
# Velocity commands, in the `base_link` frame.
CMD_VELOCITY = Topic("cmd/velocity", Twist, lane="kernel")
# Where the robot last stood still, in the `odom` frame.
DRIVE_STATE = Topic("drive/state", DriveState, lane="kernel")
GOAL_X_M = 2.0
GAIN_PER_S = 0.5
STILL_BELOW_M_S = 0.01
# Built once and rewritten on every release: an output keeps a reference until the body
# returns, and packs it then.
velocity = Twist.instance(linear=Vector3.instance(x=0.0, y=0.0, z=0.0),
angular=Vector3.instance(x=0.0, y=0.0, z=0.0))
stop = DriveState.instance(stamp_ns=0, x=0.0, y=0.0)
@periodic(every="20ms", on_overrun=Skip)
def drive(odometry=subscribe(ODOMETRY, stale_after="50ms"),
command=publish(CMD_VELOCITY),
state=publish(DRIVE_STATE)):
reading = odometry.get()
if reading is None:
return # absent or stale: neither output publishes
velocity.linear.x = GAIN_PER_S * (GOAL_X_M - reading.pose.position.x)
command.publish(velocity)
if abs(reading.twist.linear.x) < STILL_BELOW_M_S:
stop.stamp_ns = reading.stamp_ns
stop.x, stop.y = reading.pose.position.x, reading.pose.position.y
state.publish(stop)
run()
Each output is its own parameter with its own Out. When the body returns, the framework
publishes every output that was handed a value, in the order the outputs are declared:
command, then state. The order the body handed them over in does not matter.
An output handed nothing publishes nothing at that release. Its readers keep what it last
published: here drive/state holds the last stop. A release with no usable reading
publishes neither output.
publish keeps a reference, not a copy, and packs the value when the body returns. So
velocity and stop can be built once and rewritten on every release. A change made after
publish but before the body returns is published. An output carries one publication per
release: a second publish at the same release raises ValueError. DriveState is the
example's own payload, declared with Payload (Payloads).
Which examples are tested
All four programs run in the project's tests. publisher.py, subscriber.py, drive.py,
steer.py and balance.py (Run-time parameters) also run on the
emulated Raspberry Pi 3, each against Rust nodes. None of them has run on real hardware.
Declaring a node
@periodic(every="20ms", on_overrun=Skip)
def drive(command=subscribe(CMD_VELOCITY, stale_after="310ms"),
lidar=subscribe(SENSOR_RANGE, stale_after="70ms"),
velocity=publish(DRIVE_VELOCITY)):
...
The decorator states what is true of the whole node. Each input, output, run-time parameter and served action is declared on its own parameter, as the parameter's default.
| Where | Written | Required |
|---|---|---|
| decorator | every="<duration>" |
always |
| decorator | on_overrun=Skip, CatchUp or Fault |
always |
| parameter | name=subscribe(TOPIC, stale_after="<duration>") |
once per input |
| parameter | name=publish(TOPIC) |
once per output |
| parameter | name=param(default, range=(minimum, maximum)) |
once per run-time parameter |
| parameter | name=serve(ACTION, lease="<duration>") |
at most once per node |
A node reads, publishes or serves an action. One that does none of them is refused:
NodeError: a periodic node must `publish`, take an input, or serve an action; one that does none of them is a timer, and the framework has nothing to offer it
Every rule on this page is checked at decoration, which is import time: before a topic is opened and before a release is taken. Rust refuses the same mistakes at compile time; import is the earliest point Python has.
The node's keys
@periodic(every="20ms", on_overrun=Skip)
every is the period: the time from one release to the next. Releases fall on
start + n * period, so they never drift.
on_overrun says what happens on an overrun, a release whose body runs past the next
release: Skip, CatchUp or Fault, imported from periodic. There is no default. The
policies are the Rust ones (OverrunPolicy).
NodeError: a periodic node must declare `on_overrun`, one of Skip, CatchUp or Fault; there is no default, because a cycle that misses its deadline is a safety-relevant event and silence is not an answer to it
A duration is an integer and a unit, ns, us, ms or s, with nothing between them
(Durations). A mistyped one is refused at
decoration, naming the key and, for an input, its topic:
lidar=subscribe(SENSOR_RANGE, stale_after="70 ms")
NodeError: the `stale_after` of topic 'sensor/range' is '70 ms'; write an integer and one of ns, us, ms, s -- as in "20ms". There is no fractional form: write "1500us", not "1.5ms"
Any other key is refused, safety among them (Safety).
A topic
SENSOR_RANGE = Topic("sensor/range", Range, lane="safe")
lane is "kernel", "safe" or "fast"
(Topics and lanes says which to choose).
It has no default. A topic without one can be constructed, and subscribe and publish
refuse it:
NodeError: topic 'sensor/range' needs its lane: where the topic is defined, write Topic('sensor/range', Range, lane="kernel"), or "safe", or "fast". The lane belongs to the topic -- both ends must use the same one -- and it has no default
Define a topic in a module both programs import, so its two ends cannot be written differently. Two separate definitions, or one in Python and one in Rust, are compared by the payload's size and layout identity (Payloads). A payload with two fields exchanged is refused.
The lane is not compared. A Python reader that writes lane="kernel" for a topic its
Rust writer defines on the safe lane starts and receives nothing, and neither side is told.
The Rust page lists every lane mismatch
and what it does.
An input
lidar=subscribe(SENSOR_RANGE, stale_after="70ms")
The parameter is one input: the topic, then stale_after, how old a reading may be
before this node reads it as absent. On every release the body is handed a
Cycle in that parameter. These are the words of the Rust attribute,
#[subscribe(TOPIC, stale_after = "..")], in the one place Python offers beside a
parameter.
- Every input needs both parts. There is no default bound and no node-wide bound.
- Each input has its own bound. Two nodes may read one topic with different bounds.
- At each release the framework reads the clock once and judges every input against its
own
stale_afterat that one reading, so two inputs never disagree about the time.
| Written | NodeError |
|---|---|
subscribe(SENSOR_RANGE) |
the input on topic 'sensor/range' must declare `stale_after`; without it there is no answer to "is this data still usable", and the node would act on whatever it was last given |
subscribe("sensor/range", stale_after="70ms") |
a topic is a definition, not a name: define TOPIC = Topic('sensor/range', YourPayload, lane="safe") once, and write subscribe(TOPIC, stale_after="<duration>") |
subscribe(SENSOR_RANGE, stale_after="70ms", lane="safe") |
`lane` is not an input key: the lane belongs to the topic. Write Topic("some/name", YourPayload, lane="safe") where the topic is defined |
lidar=SENSOR_RANGE |
`drive` parameter `lidar` names topic 'sensor/range' but not whether the node reads or publishes it: write `lidar=subscribe(TOPIC, stale_after="<duration>")` to read it, or `lidar=publish(TOPIC)` to publish it |
Sizing stale_after. Use the Rust page's rule:
stale_after >= (lost publications tolerated + 1) x the publisher's period + this node's period
Then move it half a node period off, never onto a multiple of the node's period. drive.py
runs every 20 ms and tolerates one lost reading from a 20 ms sensor: 2 x 20 + 20 = 60,
written "70ms". Nothing checks the rule. A bound shorter than the node's period is
accepted, and makes the input usable only on the release a publication arrives.
The bound is measured from receipt at this node. The data may be older by the gap since the node's previous release, which is one period unless a release ran late.
A node reads a topic once. Two inputs that name one topic are refused, naming both:
NodeError: `drive` inputs `front` and `rear` name the same topic 'sensor/range': a node reads a topic once
An output
velocity=publish(DRIVE_VELOCITY)
The parameter is one output. publish takes the topic and nothing else; the lane is the
topic's. On every release the body is handed an Out in that parameter:
| Call | Effect |
|---|---|
velocity.publish(value) |
Hands the output this release's value, an instance of the topic's payload. A reference is kept, not a copy |
velocity.publish(None), or no call |
Nothing is published on that output at this release |
a second velocity.publish(...) in one release |
ValueError: output 'drive/velocity' was already published at this release; an output carries one publication per release |
- Nothing is sent at the call. When the body returns, every output holding a value is packed and published, in the order the outputs are declared. A served action's status goes out after them.
- A body that raises publishes nothing at that release. The exception leaves
run(). - An output handed nothing publishes nothing. Its readers go on holding what it last published. That is not an overrun.
A body returns nothing. The output is the parameter, never the return value. The first release whose body returns a value stops the node:
NodeError: `drive` returned Twist, and a node publishes through its outputs, not its return value. Declare one on a parameter, output=publish(TOPIC), write output.publish(value) where `drive` returned the value, and return nothing
A node publishes a topic once. Two outputs that name one topic are refused, naming both:
NodeError: `drive` outputs `front` and `rear` name the same topic 'drive/velocity': a node publishes a topic once
One node may name the same topic as an input and as an output. Each release then reads what an earlier release published.
A refused publish does not stop the node. When the transport refuses a publication, nothing is raised. The other outputs are still published, and the node goes on. The sixteenth refusal in a row on one output writes one record to the console:
drive: 16 consecutive publishes to drive/state refused (Refused); still trying
A publish the transport takes ends the run, and so does a release that hands that output
nothing. A refused value is not sent again. A Rust node writes the same record to its log
ring; a Python node writes it to the console, so logview does not show it.
What the body may look like
run() calls the body with one argument per parameter, positionally, in the order they
are written: a Cycle for an input, an Out for an output, the value in force for a
run-time parameter, the node's Goal for a
served action. So every parameter must be one of the four.
| Written | NodeError |
|---|---|
def drive(gain, lidar=subscribe(...)) |
`drive` parameter `gain` is not fed by anything. A parameter of a periodic body is an input, `gain=subscribe(TOPIC, stale_after="<duration>")`, an output, `gain=publish(TOPIC)`, a run-time parameter, `gain=param(<default>, range=(<min>, <max>))`, or a served action, `gain=serve(ACTION, lease="<duration>")` |
def drive(*readings) |
`drive` takes `*readings`, and a periodic body is called with exactly one argument per declared input or parameter: ... |
def drive(**options) |
`drive` takes `**options`, and a periodic body is called positionally and is never passed a keyword. Remove `**options` |
async def drive(...) |
`drive` is declared `async def`, so calling it would return a coroutine and run nothing. A periodic body is an ordinary function, called once per release; remove `async` |
a body that contains yield |
`drive` contains `yield`, so calling it would return a generator and run nothing. ... |
- A method is declared bound to its object:
periodic(every="10ms", on_overrun=Skip)(Estimator().step), as inCycle. Its first parameter receives the object.@periodicon a method inside its class body is refused, with this bound form as the fix. - A wrapper made by another decorator, around the function that declares the inputs or
run-time parameters, is refused: what it wraps declares them, not the wrapper. Put
@periodicdirectly on the function that declares them. - A body called by hand finds its parameters still holding their declarations. Using
one as a
Cycleraises aNodeErrorthat saysrun()is what calls a periodic body.
Three differences from Rust, each deliberate
| Rust | Python | Why | |
|---|---|---|---|
| A topic's payload and lane | read from the topic's type | arguments of Topic |
Python has no type to read them from |
| A mistyped duration | builds, and the node stops at start-up | NodeError at decoration |
decoration runs before anything is opened |
safety |
a key | refused | a Python node is Quality Managed by design (Safety); nothing keeps it off the fast lane |
A declaration in the earlier grammar
Before a node could read and publish several topics, subscribe, stale_after, lane,
payload and publish were keys of @periodic. A declaration that still writes one is
refused with one message. It lists every such key, says where each went, and prints the
declaration as it is written now, with its own values:
`sensor` is declared in an earlier grammar: `publish` is no longer a key of @periodic.
publish an output is declared on the parameter that publishes it
Written as it is now, this declaration is:
OUTPUT = Topic('pair/from_python', Range, lane='kernel')
@periodic(every='20ms', on_overrun=Skip)
def sensor(output=publish(OUTPUT)):
...
output.publish(value) # the value `sensor` returned
Paste it, and give the output a name of your own. A body that already declares its parameters in the current form and still carries an old key is told to remove the key.
A node that converts
Each topic names its own payload, so a node that reads one payload and publishes another needs nothing special:
from periodic import periodic, publish, run, subscribe, Skip, Payload, Topic
Range = Payload("stamp:u64 range_mm:u32 quality:u32", "Range") # 16 bytes
Distance = Payload("mm:u32", "Distance") # 4 bytes
PILOT_RANGE = Topic("pilot/range", Range, lane="safe")
PILOT_ECHO = Topic("pilot/echo", Distance, lane="safe")
@periodic(every="20ms", on_overrun=Skip)
def bridge(lidar=subscribe(PILOT_RANGE, stale_after="210ms"), echo=publish(PILOT_ECHO)):
reading = lidar.get()
if reading is None:
return
echo.publish(Distance.instance(mm=reading.range_mm))
run()
Run-time parameters
from periodic import param, periodic, publish, run, subscribe, Skip, Topic
from tail_messages import Imu, Twist, Vector3
# The IMU's readings, in the `imu_link` frame.
IMU_DATA = Topic("imu/data", Imu, lane="safe")
# Velocity commands, in the `base_link` frame.
CMD_VELOCITY = Topic("cmd/velocity", Twist, lane="kernel")
STILL = Vector3.instance(x=0.0, y=0.0, z=0.0)
STOP = Twist.instance(linear=STILL, angular=STILL)
forward = Twist.instance(linear=Vector3.instance(x=0.0, y=0.0, z=0.0), angular=STILL)
@periodic(every="20ms", on_overrun=Skip)
def balance(imu=subscribe(IMU_DATA, stale_after="70ms"),
command=publish(CMD_VELOCITY),
kp=param(42.0, range=(0.0, 200.0)),
enabled=param(True)):
reading = imu.get()
if not enabled or reading is None:
command.publish(STOP) # disabled from the shell, or the IMU is absent or stale
return
forward.linear.x = kp * reading.angular_velocity.y
command.publish(forward)
run()
A parameter is a value you tune while the node runs: a gain, a limit, a flag. You
declare it on the argument that receives it, with param(default, range=(minimum, maximum)). The body is handed each value by value on every release, and reading one
costs nothing. examples/balance.py is the whole program above.
| Rule | Detail |
|---|---|
| Name | The argument's name, at most 23 bytes. |
| Kind | The default's type: bool, int (held as a 64-bit integer) or float (a 64-bit float). |
range |
A tuple of two numbers, both ends included. Required for int and float; a bool takes none: param(True). An int parameter's bounds are ints; a float parameter's may be either. |
| Count | At most 16 parameters per node. |
Change one from the shell
param writes a request; the node applies it and answers with its state.
tsh> param list balance
kp = 42.0 (0.0 .. 200.0)
enabled = true
tsh> param set balance kp 45.5
balance: kp = 45.5 from release 112
tsh> param set balance kp 250
param: balance kp = 250.0 is outside 0.0 .. 200.0; nothing was set
tsh> param set balance enabled false
balance: enabled = false from release 140
tsh> param list balance
kp = 45.5 (0.0 .. 200.0, default 42.0)
enabled = false (default true)
tsh> param list
balance 2 parameters
The release numbers are illustrative.
- A set takes effect at the start of a release. The release that
param setnames is the first whose body gets the new value, counting from 0 at the node's first release. A body never sees a value change while it runs. param setreports success only when the node answers. It exits 1 when the node refuses, or does not answer within eight of its periods plus 250 ms.- Set coupled values in one command.
param set balance kp 50 enabled trueis one request, applied whole at one release. Two commands are two requests, applied at two releases. - The node checks every request against the declared range, both ends included,
whoever sent it. It refuses a value outside the range, a NaN, or an unknown name, and
nothing changes.
paramrefuses the same values before it sends. - Values last for the node's life. A restart starts at the defaults again. To keep a
value, change its default in the source;
param listshows each changed value beside its default. - Every Python node with parameters is settable. There is no
safetykey, so nothing marks a node whose configuration must not change at run time. Give such a node no parameters, and its values as constants. paramlogs each applied change in its own ring of the system log:balance: kp 42.0 -> 45.5 from release 112(logview -s param).- Replay hands each body the values it ran with (Recording and replay).
node listshows every node;param listshows only nodes with parameters.node info balanceprints what the node declares.
What a declaration may not say
Each rule is checked at decoration. The refusal names the function, the parameter and the rule:
| Written | NodeError |
|---|---|
mode=param("fast") |
`balance` parameter `mode` has default 'fast', a str; a parameter's default is a bool, an int or a float |
kp=param(42.0) |
`balance` parameter `kp` is a float parameter and declares no range: a value a shell can set needs its limits, kp=param(42.0, range=(<min>, <max>)) |
armed=param(True, range=(False, True)) |
`balance` parameter `armed` is a bool parameter, whose only values are False and True: remove `range` |
kp=param(42.0, range=[0, 200]) |
`balance` parameter `kp` has range [0, 200]; write a pair of numbers, range=(<min>, <max>) |
steps=param(3, range=(0, 2.5)) |
`balance` parameter `steps` is an int parameter and has range (0, 2.5); write a pair of ints, range=(<min>, <max>) |
kp=param(250.0, range=(0.0, 200.0)) |
`balance` parameter `kp`'s default 250.0 is outside its range 0.0 .. 200.0 |
kp=param(42.0, range=(200.0, 0.0)) |
`balance` parameter `kp`'s range is empty: its minimum 200.0 exceeds its maximum 0.0 |
max_torque_newton_metres=param(..) |
`balance` parameter `max_torque_newton_metres` is 24 bytes long; a parameter's name is at most 23 |
| a seventeenth parameter | `balance` declares 17 parameters; a node declares at most 16 |
A bound that is not finite, and an int that does not fit 64 bits, are refused the same
way.
What parameters cost
- One non-blocking read per release, of the request topic, made by
run()before the body. Almost always it finds nothing. A release that finds a request also checks up to 16 values and publishes the node's state. All of it counts inside the release that overrun accounting measures. - No allocation on a release without a change. The body's arguments are rebuilt only on a release whose values differ from the previous release's (Cost).
- Two kernel-lane topics per node,
<node>/__paramsand<node>/__param_set, named after the decorated function. Count them in the image's topic budget, as the Rust page gives it. - Without those topics the node runs on its defaults and cannot be set. This happens when topic storage is full, for example. The node says so once, on the console:
balance: no parameter topics (topic storage full); runs with its defaults
- The first request may be slow. The interpreter is loaded from the disk, so the code that handles a request may be paged in from the file system during the release that handles the first one.
- Any program can set a parameter, within its range, by writing the request topic, as any program can publish any kernel-lane topic. A value outside the range never reaches a body.
Actions
The client sends a goal; the node answers with a status on every release.
An action is a request that takes many releases: "drive to (2.0, 0.5)", "home the arm", "dock". A client sends a goal. The node accepts or refuses it at once, reports feedback on every release, and ends it with a result. The client can cancel it. The node stops it on its own when the client is gone.
It is the mechanism of the Rust page's actions. Either language serves either: a Python node serves a Rust client, and a Python client sends to a Rust node.
Define the action once, in a module the node and its clients import:
# robot_actions.py
from periodic import Action, Payload
# Where to drive, in the `odom` frame [m]; how far the base still has to go [m]; where it
# stopped, in the `odom` frame [m].
DriveTo = Payload("x:f64 y:f64", "DriveTo")
DriveProgress = Payload("remaining_m:f64", "DriveProgress")
DriveDone = Payload("x:f64 y:f64", "DriveDone")
# Drive the base to a point.
DRIVE_TO = Action("drive_to", DriveTo, DriveProgress, DriveDone)
The name and the three payloads belong to the action. When a Rust definition of
drive_to meets this one, the framework compares them payload by payload. Two that
disagree are refused, never misread.
Action(...) raises NodeError at the definition for any of these:
- a goal larger than 4072 bytes;
- a feedback and a result that together pass 3920 bytes, each rounded up to 8;
- a name longer than 246 bytes.
The programs in this section show the whole surface. They are not run as shown, but a node and a client of every form here run on the emulated Raspberry Pi 3 in the project's tests.
Serving it
A node serves an action on one more parameter, serve(ACTION, lease=...):
import math
from periodic import periodic, publish, run, serve, subscribe, Skip, Topic
from robot_actions import DRIVE_TO, DriveDone, DriveProgress
from tail_messages import Odometry, Twist, Vector3
# The base's odometry, `base_link` in `odom`.
ODOMETRY = Topic("odometry", Odometry, lane="safe")
# Velocity commands, in the `base_link` frame.
CMD_VELOCITY = Topic("cmd/velocity", Twist, lane="kernel")
ARRIVED_WITHIN_M = 0.05
STILL = Vector3.instance(x=0.0, y=0.0, z=0.0)
STOP = Twist.instance(linear=STILL, angular=STILL)
command = Twist.instance(linear=Vector3.instance(x=0.0, y=0.0, z=0.0), angular=STILL)
progress = DriveProgress.instance(remaining_m=0.0)
@periodic(every="20ms", on_overrun=Skip)
def drive(odometry=subscribe(ODOMETRY, stale_after="50ms"),
velocity=publish(CMD_VELOCITY),
goal=serve(DRIVE_TO, lease="100ms")):
active, reading = goal.active(), odometry.get()
if active is None or reading is None:
velocity.publish(STOP) # no goal, cancelled, its client gone, or odometry stale
return
target, here = active.goal, reading.pose.position
progress.remaining_m = math.hypot(target.x - here.x, target.y - here.y)
if progress.remaining_m < ARRIVED_WITHIN_M:
active.succeed(DriveDone.instance(x=here.x, y=here.y))
velocity.publish(STOP)
return
active.feedback(progress)
command.linear.x = min(0.5, progress.remaining_m) # your control law
velocity.publish(command)
run()
goal / active |
Meaning |
|---|---|
goal.active() |
The goal this release runs, as an ActiveGoal, or None. None means idle, cancelled, client gone, or replaced by a newer goal from the same client. It collapses like Cycle.get() on a stale input, so the stop is the branch you write anyway. |
active.goal |
What the client asked for. Like Cycle.get()'s reading, it is one object, rewritten. |
active.arrived() |
True on the goal's first release. Reset your controller here. |
active.feedback(value) |
This release's feedback. The last call in a release wins. |
active.succeed(result) |
Ends the goal with its result. goal.active() is None from then on, this release included. |
active.abort(code) |
Ends the goal with code, an int from 0 to 4294967295. On the goal's first release the client gets 'refused' with the code: this is how you refuse a goal by its content. |
- Ask
goal.active()on every release. The sameactiveobject comes back on every release that runs a goal, and stands for whichever goal that is. Do not keep it. - A call on
activewith no goal running raisesNodeError, and so does one aftersucceedorabort. Rust refuses the same code at build time. - Call it from the body. From a thread you started,
feedback,succeedandabortraiseRuntimeError. - One goal at a time. A goal from another process while one runs is refused
'busy', and the running goal is untouched. A new goal from the process that owns the running goal replaces it from the same release; the old one ends'preempted'. There is no queue. - The status follows the outputs. The node publishes one status per release, idle included, after all its outputs. A client that sees a goal stopped knows the stop went out first.
- A node serves at most one action. A second
serve(...)parameter raisesNodeError.
Sending a goal
A program sends a goal in one line:
done = DRIVE_TO.send(DriveTo.instance(x=2.0, y=0.5)).wait()
With feedback and cancel:
from periodic import GoalError
from robot_actions import DRIVE_TO, DriveTo
with DRIVE_TO.send(DriveTo.instance(x=2.0, y=0.5)) as drive: # GoalError('busy' | 'refused', code)
for progress in drive: # one per server release, until it ends
print(f"{progress.remaining_m:.2f} m to go")
if bumper_pressed():
drive.cancel() # returns once the server has stopped it
try:
done = drive.wait()
except GoalError as ended:
print(f"drive_to: {ended}") # drive_to: cancelled
| A goal's handle | Meaning |
|---|---|
ACTION.send(goal) |
Waits until the goal is accepted or refused. A client may start before its server: it waits up to ten seconds for the server's first status, then sends the goal once. Raises GoalError for a goal that was not accepted. |
ACTION.start(goal) |
Returns at once. The answer comes through poll. |
poll() |
Never blocks. Returns Progress.Pending, Progress.Running or Progress.Ended. Sends the keep-alive when one is due, and takes a new feedback into feedback. |
feedback |
The newest feedback the handle took, or None until one came. |
| iterating it | Each feedback once, waiting for the next, until the goal ends. |
wait() |
Waits for the end. Returns the result, or raises GoalError. |
cancel() |
Returns once a status shows the goal stopped. |
close(), the end of a with block, or the last reference dropped |
Cancels a goal that has not ended. |
Every feedback and result a handle hands you is a new object, which you may keep.
GoalError has .reason and .code; .code is 0 unless the node gave one. str() of it
is the phrase in the last column.
.reason |
Raised by | Meaning | str() |
|---|---|---|---|
'busy' |
send |
Another process's goal is running. | busy: another process's goal is running |
'refused' |
send |
The node aborted the goal on its first release. | refused by the server, code 3 |
'aborted' |
wait |
The node aborted the goal later. | aborted by the server, code 3 |
'canceled' |
wait |
The client cancelled it. | cancelled |
'preempted' |
wait |
A newer goal from the same client replaced it. | replaced by a newer goal of this process |
'abandoned' |
wait |
Its keep-alives lapsed past the node's lease. | abandoned: its keep-alives lapsed past the server's lease |
'lost' |
send, wait, cancel |
It is no longer running, and how it ended was not seen. | lost: no longer running, its ending not seen |
'no_server' |
send, or wait after start |
No server answered within 10 s. | no server answered within 10 s |
A topic of the action that refuses the client raises OSError from send or start, and
the console says which topic and why. Two causes are practical: the node holds another
definition of the action, or no seat is free on one of its topics.
A periodic node as a client never blocks. It starts a goal, polls it once per release, and cancels by dropping it:
from periodic import periodic, run, subscribe, Progress, Skip, Topic
from robot_actions import DRIVE_TO
from tail_messages import Range
# The front ranger's readings, in the `range_front` frame.
SENSOR_RANGE = Topic("sensor/range", Range, lane="safe")
STOP_WITHIN_M = 0.3
drive = None
@periodic(every="50ms", on_overrun=Skip)
def patrol(front=subscribe(SENSOR_RANGE, stale_after="120ms")):
global drive
reading = front.get()
if reading is None or reading.range < STOP_WITHIN_M:
drive = None # dropping a goal that has not ended cancels it
return
if drive is None or drive.poll() is Progress.Ended:
drive = DRIVE_TO.start(next_waypoint())
run()
The lease
The lease is how long a goal survives without a keep-alive from its client. The handle
sends a keep-alive once per server period while you use it: iterating it, wait, cancel,
or poll once per release. When the keep-alives stop, the node stops the goal at the end of
the lease, and the goal ends 'abandoned'. That covers a client that was killed, crashed,
hung, or simply stopped using its handle.
-
The lease is required and must be at least twice the period. Both are checked when
@periodicruns:Written NodeErrorgoal=serve(DRIVE_TO)action 'drive_to' is served with no lease: write serve(ACTION, lease="<duration>"). A goal whose client has gone is stopped when its lease runs out, so the lease is what stops the robot, and it has no defaultgoal=serve(DRIVE_TO, lease="39ms")atevery="20ms"`drive` serves action 'drive_to' with lease '39ms', shorter than twice the period '20ms': a lease of two periods lets a client be one period late, and about five is the advicegoal=DRIVE_TO`drive` parameter `goal` names action 'drive_to' but not its lease: write `goal=serve(ACTION, lease="<duration>")` -
Choose about five periods, for example
lease="100ms"on a 20 ms node. That rides out a lost keep-alive, a client one period late, and a client node up to four periods slow. -
A killed client's goal stops within the lease plus two periods. At 20 ms and 100 ms that is about 140 ms after the last keep-alive, plus release lateness. No death signal stops it sooner.
-
A periodic client slower than the lease loses every goal. It polls once per its own release. A client every 150 ms against a 100 ms lease sees each goal end
'abandoned'. Make the lease longer than the slowest client's period plus one server period. -
A program that stops using its handle loses its goal. In
drive = DRIVE_TO.send(goal); long_work(); drive.wait(), the goal is abandoned duringlong_work. Iterate the handle, or callpollorwait, while the goal should run.
What it costs and what it takes
- At most eight requests read and one status published per release, idle releases included. Both count inside the cycle that overrun accounting measures.
- No allocation per release. The
Goal, itsActiveGoaland the goal's object are built once, before the first release. A client'spollthat takes no feedback allocates nothing. - Two kernel-lane topics per action,
<action>/__goaland<action>/__status. Count them in the image's topic budget. - Replay hands each body the goal it ran with, and
arrived()as recorded. In a replay,feedback,succeedandabortkeep their checks and publish nothing. - Any program can send, keep, cancel or replace a goal by writing the request topic with another process's id. No topic has writer authorisation yet. The lease, and the device's own lease, remain the stop.
Recording and replay
tsh> topic record -o /recordings/run7.mcap & FIRST: before the nodes
tsh> lidar & joystick & python /drive.py &
... the run; stop the nodes, then the recorder ...
tsh> python /drive.py --replay /recordings/run7.mcap
replay: drive life 1: 1512 releases, 1511 compared, identical
facts: cmd/velocity: 0 releases without an arrival, 12 missed, 0 rejected; sensor/range: 1, 0, 0; 0 overruns
replay: running release 1511, the life's last; its output is in no record
You record a run, then run one node of it again, offline, against exactly what it saw.
Here /drive.py is the obstacle stop of your first nodes. lidar and
joystick stand for whatever publishes sensor/range and cmd/velocity; the counts are
illustrative.
Nothing in the node changes. It needs no line of code for replay: run() reads the
options, and replay reads the node's release journal. The lines are
the ones a Rust node's replay prints. The mechanism is the
Rust page's recording and replay.
Start the publishers too. run() waits up to ten seconds for a publisher of each
safe-lane input (run()). Without one, run() raises OSError, and the node
records nothing.
The release journal
The record of release k goes out before the body runs.
Every node with an output publishes one record per release on a kernel-lane topic of its
own, <function name>/__releases: drive publishes drive/__releases. You write nothing
for it. A node with no output has no journal.
Record k is the one a Rust node of the same declaration publishes, and it carries the same facts (the Rust page's release journal): the release, each input, each parameter, the goal, and release k-1's outputs.
- One kernel-lane publish per release, of at most 4096 bytes, before the body. It counts inside the cycle that overrun accounting measures.
- One topic per node with an output in the image's topic budget: one page for a record up to 192 bytes, two up to 384 bytes.
When the journal cannot be opened, the node runs and publishes exactly as it would with one. It says so once on the console:
drive: no release journal (topic storage full); this run cannot be replayed
Every node, with an output or not, also publishes its description on
<function name>/__node, for node list. That is one more topic in the budget. When it
cannot be opened, the node runs unlisted and says so once:
drive: no node description (topic storage full); node list does not show this node
A node's function name must be unique within an image, as in Rust: two nodes of one name share one journal topic.
Replaying
python node.py [ARGS...] --replay FILE [--life N]
| Option | Meaning |
|---|---|
--replay FILE |
Replay the node from the recording. Goes at the end of the command line. |
--life N |
After --replay FILE: pick a later life of the node. A restart during the recording is life 2. |
run() reads only that trailing part of sys.argv. The module's code runs as in a live
run. Then run() replays:
| Live | Replay | |
|---|---|---|
| Opening | Admits the topics, waiting up to ten seconds. | Reads the file and checks the life. Opens no topic. |
| Each release | Sleeps to the release, reads the inputs. | Never sleeps. Hands the body record k. |
An input's Cycle |
The newest publication. | The recorded reading. Its age and staleness are judged by this script's stale_after, at the recorded clock reading. |
overruns(), parameters, goal.active() |
As they happen. | As recorded. Fault never stops a replay. |
now() |
The release's clock reading. | The recorded reading. |
out.publish(value) |
Published when the body returns. | Published nowhere. Compared with what the node published at that release. |
A difference names the release, the output by its topic, and its first differing byte.
The command exits 1. Here drive.py was edited to STOP_WITHIN_M = 0.4, and replayed
against the same recording:
tsh> python /drive.py --replay /recordings/run7.mcap
replay: drive life 1: first divergence at release 803:
output drive/velocity byte 0: recorded 9a 99 99 99 99 99 c9 3f 00 00 00 00 00 00 00 00, replayed 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
replay: 41 of 1511 compared releases differ
facts: cmd/velocity: 0 releases without an arrival, 12 missed, 0 rejected; sensor/range: 1, 0, 0; 0 overruns
replay: running release 1511, the life's last; its output is in no record
Up to 16 bytes are shown. The first eight are linear.x: the recording passed 0.2 m/s on,
and the edited body publishes a stop. That is a what-if: "would drive have stopped earlier
at 0.4 m?"
Replay recomputes each input's age and staleness, and the body, with the script you run. It refuses a changed declaration and names both versions:
| Changed | Replays |
|---|---|
| The body | yes |
stale_after |
yes |
| A parameter's default or range | yes |
| An input's or output's payload, order or count | no |
| The period or the overrun policy | no |
| A parameter's name, kind or order | no |
The end of a replay
A body that prints release K shows the order, read through a pipe:
release 0
...
release 1510
replay: drive life 1: 1512 releases, 1511 compared, identical
facts: ...
replay: running release 1511, the life's last; its output is in no record
release 1511
The verdict comes before the last release runs. The last release's output is in no
record, because the node was stopped, crashed, or the recording ended. So run() prints
the verdict, runs that release uncompared, and ends the process at the next wait: exit 0
for identical, 1 otherwise.
run()neither returns nor raises at the end of a replay. The end is a process exit. It skipsatexitcallbacks,finallyandwitharoundrun(), finalizers, non-daemon threads, and files the node buffered itself.- What the body prints is not lost.
run()flushessys.stdoutandsys.stderrbefore every release, live and in a replay.
| The script, in a replay | Exit | Says |
|---|---|---|
| Runs to the life's end | 0 identical; 1 a divergence, or a journal that lost releases | The verdict, on standard output. |
Ends while recorded releases remain: sys.exit in a body, or an exception out of run() |
1 | On standard error: replay: drive life 1: the program ended before release 61 of 0..1511; cannot claim the rest |
| Raises in the last release's body | The interpreter's 1, with its traceback | The verdict, already on standard output. |
Calls os._exit, or is killed |
Its own | Nothing. |
Only the identical line on standard output is a pass. A script that leaves through
os._exit(0) has passed nothing.
Writing a node that replays
Write nothing for replay. Leave the trailing arguments to run(), and keep out of the
outputs what no record holds. What replay cannot see shows up as a divergence or as your
script's own error, never as a false identical:
| What the script does | In a replay | What to do |
|---|---|---|
Module code before run() touches a device |
It runs, even before a refused replay is refused. | Keep device access out of module code. Replay a node that drives a device only where the device is absent, or replay the nodes downstream of it. |
Calls argparse's parse_args() |
It sees --replay FILE and fails with your own usage error. |
Use parse_known_args(). Give the same arguments at replay: they are not recorded. |
Reads sys.argv[1] as its own word |
It reads --replay when you gave no word. |
Treat --replay there as no word. |
Builds an output from a set of str or bytes |
String hashing is seeded per process, so the order differs. | Sort first. |
Uses random, os.urandom or uuid4 |
Other values. | Seed random with a constant, or keep randomness out of the outputs. |
Stamps with time, datetime or monotonic_ns() |
Read when called, so other values. | Stamp what the node decides with now(). |
| Publishes from a thread it started | Not the body's, and in no record. | Publish only from the body. |
A refused replay ends the process inside run(), before any topic is opened. It prints one
line on standard error and exits 1; a malformed --replay ... tail exits 2. The module's
code has run by then.
| Refusal | Line |
|---|---|
--replay not followed by exactly FILE or FILE --life N (exit 2) |
replay: usage: drive [ARGS...] --replay FILE [--life N] |
| A node with no output | replay: drive declares no output; replay compares a node's published output and has nothing to compare |
| The file cannot be opened or read | replay: cannot read /recordings/run7.mcap: open failed, or read failed |
| Not a recording | replay: /recordings/run7.mcap is not a recording (not an MCAP file) |
| No journal of this node in it | replay: /recordings/run7.mcap has no drive/__releases (the node ran without a journal, or the recorder did not record it) |
| Another declaration | replay: drive was built from another declaration (recorded 0x…, … B; this binary 0x…, … B): an input or an output changed layout, size or order, a parameter its name, kind or order, or the period or overrun policy changed |
| The recorder attached late | replay: life 1 starts at release 37: the recorder attached after drive started; start topic record before the nodes |
| No such life | replay: no life 9; /recordings/run7.mcap holds life 1 |
- Start
topic recordbefore the nodes. Otherwise the journal is not whole from the first release, and replay refuses with the late-attach line above. - A journal the recorder lost part of replays up to the lost stretch, and claims nothing for the rest.
Payloads
A payload is the type a topic carries. Import a standard payload when one fits, and declare your own when none does.
Standard payloads
from tail_messages import Range, RANGE_INFRARED
The module tail_messages has these:
| Kind | Payloads |
|---|---|
| geometry | Vector3, Point, Quaternion, Pose, Twist |
| sensor | Range, Imu, and Range's named values RANGE_ULTRASOUND and RANGE_INFRARED |
| navigation | Odometry |
The image build prints the module from the Rust crate robot_messages, field for field. Each field's comment gives its unit. So a Python node and a Rust node hold one definition.
The module checks itself when it is imported. Each declaration asserts that its size, alignment and layout identity are the Rust struct's. A module edited by hand stops at AssertionError: Range.
The conventions
The geometry conventions are in the module, in each record's comment, in the words the Rust records carry:
| Item | Convention |
|---|---|
| Axes in a body frame | right-handed: x forward, y left, z up |
| An angle | positive turns counter-clockwise about its axis: a yaw of +15 degrees turns left |
Quaternion |
a Hamilton quaternion stored x, y, z, w, with w the scalar; the identity is (0, 0, 0, 1); all zero means "not estimated" |
A Pose used as a transform |
the pose of the child frame in the parent frame |
A Range reading |
lies on the sensor frame's +x axis |
Odometry's pose |
base_link in odom |
Limit: the arithmetic on these records (composition, inversion, a rotation from roll, pitch and yaw) and frame sets with their lookup are Rust only in this release (Frames and transforms). They will reach Python as bindings that call the Rust code, never as a pure-Python copy. Until then a Python node computes them by hand, against these conventions.
Your own payload
Declare the fields, and the framework computes the layout:
Gate = Payload("stamp_ns:u64 state:u32=OPEN:0|CLOSED:1 reserved:u32", "Gate")
Gate.size # 16 -- the same as the Rust struct
Gate.align # 8
Gate.canonical # "{stamp_ns:u64,state:u32=OPEN:0|CLOSED:1,reserved:u32}"
Gate.layout # 0xa39598f2 -- the identity the Rust derive computes for the same fields
A payload crosses shared memory to a peer that may be written in Rust. So its layout is that of a Rust #[repr(C)] struct with the same fields.
| Declaration | Rust |
|---|---|
u8 i8 u16 i16 u32 i32 u64 i64 |
the same |
f32 f64 |
the same |
name:u8[32] |
[u8; 32]; arrives as bytes |
name:u32[4] |
[u32; 4]; arrives as a tuple |
name:Vector3, with using=(Vector3,) |
a field of a nested record; read as name.x |
name:u32=OPEN:0|CLOSED:1 |
a field with named values, #[wire_plain(values(OPEN = 0, CLOSED = 1))] |
Limit: an array of arrays or of payloads has no declaration here.
Payload.instance(**fields) makes a value to publish. Every field must be given, and a nested field is given an instance of its payload:
Gate.instance(stamp_ns=1)
NodeError: Gate needs every field; missing reserved, state
Why not a struct format string. struct pads with native alignment only under @. Under <, >, = and ! it packs tight. So a Rust #[repr(C)] struct of a u32 then a u64 is 16 bytes, while struct.calcsize("<IQ") is 12. A wrong prefix gives a subscriber that reads a shifted field. Payload(..) has no prefix to get wrong. (ctypes is not available on TAIL OS.)
What Payload(..) refuses
A declaration a Rust payload could not have is refused where it is written, with what to do. Take the row that matches what you wrote:
Written in Payload(.., "Refused") |
NodeError |
|---|---|
"stamp:u64 counter:u32" |
payload 'Refused' has padding at its end; name it with a reserved field |
"flag:u8 count:u64" |
payload 'Refused' has padding before 'count'; reorder the fields or name the gap with a reserved field |
"pressed:bool .." |
payload 'Refused': 'pressed' is a bool, which cannot cross the wire: only 0 and 1 are valid and a sender may send 2. Use u8. |
"from:u32 to:u32" |
payload 'Refused': 'from' is not a usable field name; a field is declared in C, C++ and Python alike, .. |
"align:u64[0]" |
payload 'Refused': 'align' is an array of no elements |
"gain:f32=UNITY:1 .." |
payload 'Refused': only an integer field has named values |
Padding is refused because a gap would carry the sender's uninitialised bytes. A field name must be usable in every language the payload is declared in, so a keyword such as from is refused.
Both ends must hold one definition
Payload(..) computes the payload's layout identity: a 32-bit hash of the canonical text, which covers the field names, their order, their types, nesting, array lengths and named values. It is the identity the Rust derive computes for the same fields. On every lane, the size and the identity are checked against the other end when the topic opens. A node whose payload disagrees does not start: run() raises an OSError naming the topic.
Here a node's payload disagrees with the one the kernel topic range/front carries. On kernel, the reason and the console record end with what the topic carries:
could not start: topic `range/front` refused the open: this is not the payload type the topic carries: it carries 32-byte payloads of layout 0x6772d100
So two declarations with the same fields in another order are refused, in Python and Rust alike.
Not compared: the payload's name (Point read as Vector3 is admitted, because the bytes are the same three f64), a field's unit, and the lane.
A kernel topic lives while an end holds it. Once every end of the old payload has closed, the corrected declaration creates the topic anew.
Cycle
A Cycle is one input's view of one release. The node gets one per input, passed to its body in the order the inputs are declared.
def brake(lidar=subscribe(SENSOR_RANGE, stale_after="70ms")):
An input is in one of three states:
| State | get() |
was_stale() |
since_received() |
|---|---|---|---|
| nothing received since the node started | None |
False |
None |
received, and no older than stale_after |
the sample | False |
a number of seconds |
received, and older than stale_after |
None |
True |
a number of seconds |
| Method | Returns |
|---|---|
get() |
the sample, or None if absent or stale |
copied() |
the same sample, in an object of its own |
arrived() |
whether a newer publication was taken at this release |
topic() |
the name of the topic this input reads |
since_received() |
seconds since a sample last arrived at this node, or None |
was_stale() |
whether something was received and is now older than stale_after |
missed() |
publications the lane lost for this input since the previous release |
rejected() |
publications the lane refused on this input since the previous release |
overruns() |
releases the node missed since its previous release ran |
is_healthy() |
True when missed(), rejected() and overruns() are all zero |
get()
reading = lidar.get()
if reading is None:
velocity.publish(STOP) # absent or stale
Returns the sample if one was received and is no older than stale_after. A stale sample returns None, the same as no sample, so the fail-safe branch is the default branch.
get() borrows. It returns the same object on every release, with its fields rewritten. That mirrors Rust's get() -> Option<&T>. A body that keeps the object and reads it on a later release sees new data. Rust's borrow checker catches that mistake; Python cannot, so it is stated here. The Cycle objects are also the same ones on every release.
This reuse is what keeps a release free of allocations the garbage collector has to walk (Cost).
copied()
self.last_reading = lidar.copied()
Returns the same sample in a new object you own. Use it to keep a sample past this release.
arrived()
wheels = odometry.get()
if odometry.arrived() and wheels is not None:
self.pose.x_mm = wheels.travelled_mm # once, on the release it arrives
True if a newer publication was taken at this release. get() answers "is there a usable reading?"; arrived() answers "is it new?". A node fed by a slower input sees a usable reading on every release and a new one on some.
topic()
if lidar.get() is None:
print(f"no usable reading on {lidar.topic()}")
Returns the name of the topic this input reads: "sensor/range". A parameter's name is only its author's choice; topic() is the binding itself.
since_received()
gap = lidar.since_received()
if gap is not None and gap > 0.040:
raise_sensor_warning()
Returns the seconds since a publication last arrived at this node, as a float. It is not the publication's age: no lane timestamps a publication. It is None until something arrives, and exactly zero on the release arrived() is true.
was_stale()
if lidar.was_stale():
print("using fail-safe: last reading too old")
True if a sample was received and has aged past stale_after. Use it only when you need the difference that get() hides.
Limit: never received is never stale. An input that has received nothing returns False, however long the node has run. stale_after is not a start-up deadline. A node that must act when a publisher never comes up tests get() is None, which covers "not yet" and stale alike.
missed()
if lidar.missed() > 0:
dropped += lidar.missed()
Returns the publications the lane lost for this input since the previous release. It is not a running total. Each lane counts its own losses:
| Lane | missed() counts |
|---|---|
kernel |
publications the topic overwrote before the node read them, counted from the input's open. The topic keeps at least its newest 8. |
safe, fast |
publications the lane itself reports skipped |
Limit: missed() == 0 does not mean the node saw every publication. An input is handed the newest publication at each release. The ones a newer one replaced are handed to nobody and are not counted.
rejected()
if lidar.rejected() > 0:
rejected_total += lidar.rejected()
Returns the publications the lane refused on this input since the previous release. It is not a running total. On kernel it is always 0, because a publisher of another payload is refused when it opens (Payloads).
overruns()
if lidar.overruns() > 0:
velocity.publish(last_command)
Returns the releases the node missed since its previous release ran. It is one count for the node, the same on every input of a release. Read it from any one input; adding them up counts each overrun once per input.
Limit: a Python node writes no overrun record to the system log. A node with no input has no Cycle to read the count from. So under Skip or CatchUp, the body of a Python node that only publishes cannot observe its overruns.
is_healthy()
if not lidar.is_healthy():
raise_degraded()
True if this input saw no loss, no rejection and no overrun on this release: missed(), rejected() and overruns() all zero, in one call.
An input is a value, the newest one
examples/estimator.py reads a 10 ms IMU and a slower wheel odometry. It predicts on every release and applies the odometry correction once, on the release it arrives:
from periodic import periodic, publish, run, subscribe, Skip, Payload, Topic
Imu = Payload("yaw_rate_mrad_s:i32 forward_mm_s2:i32", "Imu")
Odometry = Payload("travelled_mm:i64 heading_mrad:i32 reserved:u32", "Odometry")
Pose = Payload("x_mm:i64 heading_mrad:i32 corrections:u32", "Pose")
IMU_DATA = Topic("imu/data", Imu, lane="safe")
WHEEL_ODOMETRY = Topic("wheel/odometry", Odometry, lane="safe")
ESTIMATE_POSE = Topic("estimate/pose", Pose, lane="safe")
class Estimator:
"""A dead-reckoned heading, corrected by the wheels whenever they report."""
def __init__(self):
self.pose = Pose.instance(x_mm=0, heading_mrad=0, corrections=0)
def step(self,
imu=subscribe(IMU_DATA, stale_after="35ms"),
odometry=subscribe(WHEEL_ODOMETRY, stale_after="65ms"),
estimate=publish(ESTIMATE_POSE)):
"""Predict from the IMU, correct from the wheels when they report, publish the pose."""
rates = imu.get()
if rates is None:
return # no usable IMU: publish nothing this cycle
self.pose.heading_mrad += rates.yaw_rate_mrad_s // 100
wheels = odometry.get()
if odometry.arrived() and wheels is not None:
self.pose.x_mm = wheels.travelled_mm # once, on the cycle the reading arrives
self.pose.heading_mrad = wheels.heading_mrad
self.pose.corrections += 1
estimate.publish(self.pose)
periodic(every="10ms", on_overrun=Skip)(Estimator().step)
run()
The payloads here are the example's own, not the standard ones from tail_messages. This file runs on the development host only, against a stand-in for the transport.
A node behind a faster publisher is handed the newest publication at each release. So a topic that carries increments must not be read as an input: the increments the node passed over are lost, and missed() does not count them. The odometry above is a cumulative reading for that reason.
run()
run()
run() runs the declared node. It does not return until the node's policy stops it.
When the command line ends in --replay FILE [--life N], run() replays the node instead and never returns. The process ends inside it, after the last release or at a refused replay (Recording and replay).
run() |
What it means for you |
|---|---|
reads the whole of sys.argv as it is when run() is called, and uses only the trailing --replay .. part |
a script that consumes its own arguments by rewriting sys.argv first hands over what is left |
flushes sys.stdout and sys.stderr before it opens the node and before every release, skipping one that is None |
what the body printed is written before any call that may end the process |
| refuses a second declared node, naming both | a process is one node, as a Rust node is one program |
The decorator cannot run the node itself. A module whose import never finishes is never entered into sys.modules, and every traceback from it would be reported inside an import.
The order it opens topics in
open outputs, in declaration order -> open inputs, in parameter order -> gc.freeze() -> release 0
run() opens the topics the node publishes first, in the order they are declared. Then it opens its inputs, in the order the parameters are written.
| Input's lane | When nobody publishes it yet |
|---|---|
safe, fast |
the node waits, for ten seconds in all: one wait for the node, not one per topic |
kernel |
the node never waits: the input is admitted with no publisher and reads "not yet" until one publishes |
While the node waits for a later input, it holds nothing. What was published meanwhile is dropped at the first release.
The node starts at its first release, not when run() opens its topics. Whatever was published before that is thrown away unread: when it was received is not known, and it must not be presented as fresh. The time between opening the topics and the first release, gc.freeze() included, is not charged to the node as overruns.
Limit: an input reads "not yet" on the first release, unless another program published in the instant between that discard and the release's own read.
Safety
A Python node cannot declare a topic safety-relevant, and this is deliberate.
@periodic(every="20ms", on_overrun=Fault, safety=True)
NodeError: a Python node cannot declare `safety`, and this is deliberate.
A safety classification asserts that a bound on cycle time exists and is known. The garbage collector's pauses, measured on the emulated Raspberry Pi 3:
| Collection | Pause |
|---|---|
| A full collection, against a bare interpreter heap | 6.6 ms worst, 4.4 ms median |
| An incidental collection inside a 20 ms release grid | 2.5 ms: 12.7% of the period |
Both the pause and its frequency scale with your live heap, which the framework neither observes nor bounds. There is no bound to assert, so the claim is not available to make.
A Python node is Quality Managed by construction, as the fast lane is by its own design.
| A Python node may | A Python node may not |
|---|---|
| subscribe to a topic a Rust node has classified safety-relevant: consuming safety data at QM is an ordinary architecture | claim the classification for what it publishes |
Put a node that must publish safety data in Rust.
Cost
Measured on the emulated Raspberry Pi 3, on a node with one input. These are not board numbers: the emulator's CPU is not a Raspberry Pi 3's. Use the ratios to budget, not the absolute milliseconds.
| Cost | How much |
|---|---|
| Smallest period with no overruns | 5 ms; at 1 ms the node overran 35 releases in 200 |
| Lateness against the grid | median about 0.9 to 1.0 ms, p99 about 1.7 ms, at every period |
| Time to the first release | cold about 4.2 s, warm about 1.4 to 1.9 s |
| Allocations the collector walks, per release | none, whatever the number of inputs |
| Topics per node, beyond its own inputs and outputs | the release journal, <function name>/__releases, for a node that publishes; the node description, <function name>/__node, for every node (The release journal) |
Limit: the period and lateness figures were not measured again for a node with several inputs.
The time to the first release matters to a system integrator: a Python node is not ready for seconds after it is spawned. That time is interpreter start-up. The node's ten-second admission wait already spans it, so a Python subscriber may still be started before its Rust publisher.
What the framework does about the collector
You do not have to do any of this yourself:
run()callsgc.freeze()before the first release. That moves the interpreter's whole start-up heap out of the generations the collector walks. A full collection costs 6.6 ms before that call and 0.018 ms after it.- Nothing the collector walks is allocated per release. The
Cycleobjects, the sample eachget()returns and the payload buffers are created once. - A node's run-time parameters are handed over in one argument tuple. It is built at the first release, and rebuilt only on a release whose values differ from the previous release's. Every other release makes nothing for them.
What you can do
A body that does not keep what it makes triggers no collection at all: CPython collects on allocations minus deallocations. Keeping a rolling window of samples is what makes the collector run. If you need one, that is the cost to measure.
Errors
| Exception | Means | When |
|---|---|---|
NodeError |
something is wrong with a declaration | where it is written, or at decoration |
OSError |
the node cannot start; it names the topic and why | from run() |
NodeError from run() |
the Fault policy stopped the node, or the body did something no node may do |
at that release |
An uncaught exception exits non-zero. So a supervisor can tell a node that never started from one that stopped cleanly.
Refused at the declaration
The node never runs. Take the row that matches:
| Cause | Raised |
|---|---|
| A duration that will not parse | NodeError, naming the key and, for an input, its topic |
| A missing or unknown key, a topic without its lane, a body the loop cannot call | NodeError, naming it |
| A run-time parameter whose default, range or name breaks a rule, or a seventeenth | NodeError, naming the function, the parameter and the rule (Run-time parameters) |
A declaration in the earlier grammar, publish= among its keys |
NodeError, once, with its rewrite (A declaration in the earlier grammar) |
Two outputs on one topic, or a publish(...) without a laned topic |
NodeError, naming it (An output) |
safety=True |
NodeError, with the reason in Safety |
A payload with padding, a bool, an empty array, a keyword for a name, or named values on a float |
NodeError, naming the field (Payloads) |
An action definition, a serve(...), a lease under twice the period, or a second served action that breaks a rule |
NodeError, naming it (Actions) |
Three of them, as written and as refused:
@periodic(every="20 ms", on_overrun=Skip)
NodeError: `every` is '20 ms'; write an integer and one of ns, us, ms, s -- as in "20ms". There is no fractional form: write "1500us", not "1.5ms"
lidar=subscribe(Topic("sensor/range", Range), stale_after="70ms")
NodeError: topic 'sensor/range' needs its lane: where the topic is defined, write Topic('sensor/range', Range, lane="kernel"), or "safe", or "fast". The lane belongs to the topic -- both ends must use the same one -- and it has no default
@periodic(every="20ms", on_overrun=Skip)
def drive(command=publish(CMD_VELOCITY), state=publish(CMD_VELOCITY)):
NodeError: `drive` outputs `command` and `state` name the same topic 'cmd/velocity': a node publishes a topic once
Raised while the node starts or runs
| Cause | Raised |
|---|---|
| A topic that never appears | OSError from run() after the node's ten-second wait: errno is -3, filename is the topic |
| A payload that disagrees with the publisher's, in size or in layout | OSError from run(), naming the topic, on every lane; on kernel the console record also says what the topic carries |
| A body that returns a value | NodeError from run() at that release, nothing published, with the output form to write |
A second out.publish(...) on one output in one release |
ValueError from the call, naming the output |
| A publish the lane refused | nothing raised: the outputs after it are published and the node goes on; the 16th refusal in a row on one output writes one console record naming the node and the topic (An output) |
A release overrunning under Fault |
NodeError from run(): <node>: a cycle overran its deadline, and this node declared on_overrun=Fault |
active.feedback, succeed or abort from a thread the body started |
RuntimeError, on every such call: the node is reached only from the thread that called run() (Actions) |
A body's feedback, succeed or abort with no goal running at its release |
NodeError from the call |
| A goal that did not succeed, or a send no server answered | GoalError(reason, code) from the client's call (Actions) |
A topic that refused a goal's send or start |
OSError from the call; the console says why |
A refused replay: a malformed --replay ... tail, a file that is not a recording of this node, another declaration, no such life |
no exception: one line on standard error, and the process ends inside run() before any topic is opened, exit 2 for the tail and 1 for the rest (Recording and replay) |
| The end of a replay | no exception: the process ends inside run(), exit 0 when identical and 1 otherwise |
An entry of sys.argv holding a NUL byte |
ValueError from run(), before the node is opened |
A flush of sys.stdout or sys.stderr that fails |
that error, such as BrokenPipeError for a closed pipe, from run() |
A node that cannot start names the topic
This node declares two inputs; nobody publishes the second, py_two_inputs/nobody. It catches what run() raises:
try:
run()
except OSError as reason:
print(f"[never_appears] {reason}")
if reason.filename != MISSING.name:
sys.exit(5)
if reason.errno != NEVER_APPEARED:
sys.exit(3)
sys.exit(0)
On the emulated Raspberry Pi 3 it printed, about eleven seconds after the interpreter started:
[never_appears] [Errno -3] the topic never appeared: 'py_two_inputs/nobody'
Before the exception is raised, the console gets one record per fact. This node also publishes a topic, which it created before it waited:
could not start: topic `py_two_inputs/nobody` never appeared
its publication `py_two_inputs/orphan` is left without a writer
A node with several outputs gets one "left without a writer" record per output it had created, in declaration order.
Limits:
- These records are on the console only, not in the system log, and they name no node.
- Only the first topic that could not be opened is named.
- The topic the node had created stays owned by the process until it exits.
After Fault stops a node, a kept Cycle reads as nothing
This node keeps the Cycle it is handed, overruns once on purpose under Fault, and asks the kept cycle afterwards:
kept = []
@periodic(every="200ms", on_overrun=Fault)
def node(first=subscribe(FIRST, stale_after="500ms")):
kept[:] = [first]
... # overruns once, on purpose
try:
run()
except NodeError as stopped:
print(f"[fault] {stopped}")
cycle = kept[0]
# get() is None, arrived() is False, since_received() is None
[fault] node: a cycle overran its deadline, and this node declared on_overrun=Fault
A sample that was usable on the last good release is not usable once the node has stopped.
init has no equivalent, and does not need one
Rust needs init because a bare fn cannot own state between releases. In Python, a closure, a module global or a bound method already does:
| Where the state lives | Example |
|---|---|
| a module global | publisher.py keeps its count at module level |
| an object, with the node declared as its bound method | estimator.py keeps its pose on an Estimator |
periodic(every="10ms", on_overrun=Skip)(Estimator().step)
A method decorated inside its class body is refused. Its first parameter is self, which no topic feeds:
class Estimator:
@periodic(every="10ms", on_overrun=Skip)
def step(self, imu=subscribe(IMU_DATA, stale_after="35ms")):
NodeError: `step` is written in a class, and its first parameter `self` is not fed by anything. A method is declared BOUND to its object, which is what supplies `self`: periodic(...)(instance.step). ..
What is verified
| Where | What you can rely on |
|---|---|
| On the development host | Payload layouts and identities equal what Rust computes. Every refusal on this page is checked with its whole text. Every example on this page that is a file runs. |
| On the emulated Raspberry Pi 3, beside Rust nodes | The Quick start runs: node list shows counter, and topic echo sees its count climb. So do the publisher, the subscriber, drive.py, steer.py and balance.py; actions in both directions; run-time parameters set with param; recording and replay; a topic that never appears; a node stopped by Fault; and a payload of another size refused at open. |
Not verified:
- Nothing ran on real hardware, and the Raspberry Pi 3 image built from the BSP carries no Python interpreter.
estimator.pyruns on the development host only.- Staleness is checked inside a window above
stale_after, not at the exact bound. - That exporting one
PYTHONHASHSEEDfor both the live run and the replay makes aset's order repeat is not checked.
Glossary
action: a request that takes many releases, such as "drive to (2.0, 0.5)": a client sends a goal, and the serving node reports feedback on every release and ends it with a result.
frame (coordinate frame): one of the robot's coordinate systems. Frame sets and their lookup are Rust only in this release; a Python node uses the conventions in tail_messages.
goal: one request of an action, sent by a client and handed to the serving node's body on every release until it ends.
input: a topic a node reads, declared on a parameter with subscribe(TOPIC, stale_after=".."); the body is handed a Cycle there.
lane: how a topic's messages travel, written in the topic's definition: "kernel" copies through the kernel (up to 4 writers, 4 KiB), "safe" shares memory with one writer and may carry safety data, "fast" shares memory with one writer and may not.
layout identity: a 32-bit hash of a payload's fields, their order and types, which Payload(..) computes as the Rust derive does; an end whose identity differs from the topic's is refused at open.
lease: how long a served goal lives without its client's keep-alive, at least twice the period, after which the node ends the goal abandoned.
node: a function, or a bound method, given to @periodic(..), whose body run() calls once on every release.
output: a topic a node publishes, declared on a parameter with publish(TOPIC); the body is handed an Out there, and what it hands out.publish(..) is published when the body returns.
overrun: a release that runs past the next one; on_overrun says what happens then.
parameter: a run-time parameter, a value declared with param(default, range=(minimum, maximum)) that the shell can change while the node runs.
payload: the type a topic carries, a standard tail_messages record or your own declared with Payload("fields", "Name").
period: the interval every sets; releases fall on start + n * period.
release: one scheduled instant on the period's grid, at which the body runs once.
release journal: the record every node that publishes writes on <function name>/__releases at every release, which replay reads.
replay: running one node again, offline, against a recording, with --replay FILE at the end of its command line, and comparing its outputs with the recorded ones.
stale_after: an input's bound on how old a sample may be, measured from receipt at this node, before it reads as absent.
topic: a named channel that carries one payload type on one lane, defined once with Topic(name, payload, lane=..).