Download Game! Currently 74 players. Last logged in:BiubiuSloppyStylusFloppy

Blitzer's Blog >> 73098

Back to blogs index
Posted: 24 Oct 2026 18:24 [ permalink ]
To build a real-time "radar" without relying on GPS, the robot needs a spatial
perception pipeline based on SLAM (Simultaneous Localization and Mapping).
Since you are using a cluster of Android phones as the orchestration layer,
you can leverage their massive compute power to fuse external hardware sensors
with the phones' native hardware.
Here is the top-level architecture for building a distributed, real-time world
model:
1. The Sensory Layer (Ingestion)
You cannot rely on a single sensor type. A robust robot uses a fused approach,
combining high-bandwidth data with low-latency spatial data.
 * Android Native Sensors: The phones already possess high-resolution cameras,
high-frequency IMUs (accelerometer/gyroscope), and in some cases, native ToF
(Time of Flight) LiDAR.
 * External Spatial Sensors: Connect a 360-degree 2D or 3D LiDAR (e.g., an
affordable RPLIDAR) and a stereo depth camera (like an Intel RealSense)
directly to your USB-C Ethernet switch.
 * UDP Broadcast: These external sensors do not send data to a specific phone.
They broadcast their raw point clouds and distance metrics over the local
network via UDP multicast. Every Android phone on the switch can ingest the
raw data simultaneously.
2. The Fusion & SLAM Engine (The "Radar")
The Android phones run a VIO (Visual-Inertial Odometry) and SLAM pipeline.
This is where the actual "radar map" is generated.
 * Pose Estimation: The IMU and camera feeds are fused to mathematically
calculate the robot's exact 6-DOF (Degrees of Freedom) movement in 3D space.
Even if the wheels slip, the visual tracking knows exactly how far the torso
actually moved.
 * The Occupancy Grid: The LiDAR point clouds are processed into a 3D voxel
grid (or a 2D occupancy grid). This divides the world into three states: Free
Space (safe to drive), Occupied Space (obstacle), and Unknown Space
(uncharted).
 * Raycasting: As the LiDAR spins, it casts "rays" into the grid. If a ray
hits something 2 meters away, it marks that cell as "Occupied" and clears all
the cells between the robot and the obstacle as "Free Space."
3. Semantic Layering (AI Vision)
A raw 3D point cloud only tells you where things are, not what they are. The
Android cluster handles the semantic understanding.
 * Set-of-Mark / Object Detection: The active Android phone runs a lightweight
vision model on its NPU (Neural Processing Unit). It draws bounding boxes
around objects in the camera feed.
 * Spatial Projection: Because the camera is calibrated to the LiDAR grid, the
AI projects its 2D bounding boxes into the 3D space. The "radar" no longer
just shows a solid block; it tags that block as "Couch" or "Human Leg."
4. The N+1 Distributed World Model
If Phone A is calculating the SLAM map, Phone B (the standby) must have the
exact same map so it can take over instantly if Phone A dies. You cannot
stream a massive 3D map back and forth efficiently.
 * Event-Sourced Spatial Deltas: Instead of syncing the whole map, the active
SLAM processor broadcasts only map deltas over the internal IP mesh (e.g.,
event: obstacle_added, coords: [x, y, z]).
 * Replicated State: Every standby phone subscribes to these spatial events
and applies them to its own internal memory. They all independently build
identical 3D maps.
 * Multi-Perspective Merging: If you mount Phone A facing forward and Phone B
facing backward, both phones can process their own camera feeds and broadcast
their spatial discoveries to the network. The world model seamlessly merges
the front and rear data into a single omnidirectional map.