To achieve arbitrary N+1 redundancy across physical, transport, and logical
layers, the robot must abandon centralized master-slave polling. Instead,
treat the chassis as a distributed, zero-trust cluster of edge nodes driven by
an event-sourced architecture.
Here is how to design the system for total fault tolerance.
Multi-Homed Physical Topology
Hardware redundancy requires that no single severed wire or fried
microcontroller can isolate a limb.
 * Dual-Spine Buses: Run two physically isolated, parallel networks (e.g., CAN
FD A and CAN FD B) down different physical routes in the chassis.
 * A/B Node Transceivers: Every joint controller (e.g., the RP2040 at the
socket) is equipped with dual transceivers, tapping into both Bus A and Bus B
simultaneously.
 * Daisy-Chain Bypasses: If using a daisy-chain topology, include passive
hardware bypass relays. If a node loses logic power, the relay snaps shut,
keeping the bus unbroken for the remaining downstream nodes.
Event-Sourced Messaging Layer
Standard robotic state updates (e.g., "move to X") are fragile if a packet is
lost. The transport layer must be built on idempotency and immutable logs.
 * Append-Only State: Treat all sensor readings, motor commands, and mode
switches as immutable events with strict sequence IDs.
 * Lock-Free Ring Buffers: Implement SPSC (Single-Producer, Single-Consumer)
ring buffers at the network boundary of every node. Nodes do not wait for
acknowledgments; they continuously publish their latest state to the buffer.
 * Deterministic Catch-Up: If a node loses its comlink and reconnects, it
simply broadcasts its last known sequence ID. The network immediately replays
the missing event deltas, allowing the node to achieve state synchrony in
microseconds without blocking execution.
Mesh Routing and Transport
To survive comlink losses, the internal network must route around damage
dynamically, treating the robot's limbs like a localized mesh network.
 * Path Redundancy: If the torso cannot reach the left hand via the left arm
due to a severed cable, but the left hand can talk to the right hand, traffic
is seamlessly routed through the intact pathway.
 * Publish-Subscribe Sockets: Abstract the physical buses. Nodes do not send
messages to "IP 10" or "CAN ID 5". They publish to topics (e.g.,
chassis.locomotion.velocity). Any node subscribed to that topic pulls the
data, regardless of which physical bus it arrives on.
Floating Logical Compute
Control logic (kinematics, balance, SLAM) should not be physically pinned to a
single "brain" in the torso.
 * Active/Standby Arbiters: Deploy N+1 logic nodes across the robot (e.g., one
in the torso, one in the pelvis). They all subscribe to the same event streams
and compute the kinematics simultaneously.
 * Heartbeat Failover: Only the elected "Active" node publishes motor commands
to the bus. It emits a high-frequency heartbeat. If the heartbeat stops, the
Standby node immediately begins publishing commands. Because it has been
reading the same event log, it has the exact same internal state, resulting in
zero downtime.
 * Graceful Degradation: If all high-level compute nodes fail, the socket
nodes must detect the silence and autonomously drop into a localized safe
state (e.g., locking joints to maintain balance or safely collapsing to the
ground).