You are describing a true server-meshing distributed architecture,
structurally similar to Improbable's SpatialOS or the high-density physics
engines used in modern MMOs. Decoupling the visual representation from the
physical simulation is exactly how you achieve that kind of massive scale.
Because you have already established the memory segmentation with your 8x8x8
voxel quadrants, your environment is perfectly structured to act as the static
source of truth for this distributed network. Here is how that architecture
breaks down.
1. Primitive-Only Collision (The Math Engine)
Restricting the physics simulation strictly to Oriented Bounding Boxes (OBBs)
and Spheres/Ellipsoids is computationally brilliant. Physics engines spend the
vast majority of their CPU cycles in the "narrow phase," calculating contact
manifolds for complex concave geometry.
By forcing the world into elongated cubes and stretched balls, you reduce
collision detection to simple algebraic intersections. For ragdolls, this maps
perfectly to standard multi-sphere or capsule skeletal setups. The headless
nodes never need to know what a "character" or a "tree" looks like; they only
calculate the intersections of these invisible geometric bounds.
2. Distributed Spatial Partitioning (The Nodes)
Splintering the world into computational bounding regions means each headless
node only simulates the voxel chunks and dynamic entities within its specific
3D volume.
The hardest engineering challenge here is boundary handoffs. When a
fast-moving ball crosses from Node A's region into Node B's region, collisions
cannot simply stop at the border. You will need an overlap margina "ghost
region." Node A must stream read-only proxy data of the moving object to Node
B just before it crosses the threshold so Node B can accurately resolve border
collisions before taking full computational authority. Routing these regional
node handoffs and telemetry streams through edge delivery pipelines or a
Tailscale-secured backend mesh will keep the internal distributed topology
extremely fast and secure.
3. The Predictive Protocol (Dead Reckoning)
To keep the visual clients perfectly smooth regardless of server tick rates or
network jitter, the physics nodes cannot simply stream absolute spatial
coordinates. They must stream kinematic state vectors.
The network payload for a dynamic entity becomes a highly compressed binary
packet containing:
 * Position
 * Linear Velocity
 * Linear Acceleration
 * Rotation (Quaternion)
 * Angular Velocity
The rendering client utilizes these vectors for dead reckoning. Instead of
freezing while waiting for the next network packet, the visual client
continuously integrates the position using P(t) = P_0 + V_0t + \frac{1}{2}at^2
and applies spherical linear interpolation (slerping) to the rotation. When
the next true server packet arrives, the client smoothly interpolates any
minor discrepancies over the next few frames, completely masking the network
latency from the observer.
4. Visual Skinning (The Illusion)
Because the client handles the predictive interpolation locally, the rendering
engine can attach highly detailed glTF meshes or skinned armatures to these
invisible physics primitives. The renderer simply maps the root bone of a
high-poly visual mesh to the predicted coordinates of the underlying physics
ellipsoid.
You have the chunk-based memory architecture fully operational in the browser
to define the static terrain bounds. To prototype this, the most logical first
step is to establish a single headless authority, feed it a subset of your
OPFS voxel chunks to generate static box colliders, and drop a single dynamic
sphere into it to test the predictive streaming protocol back to your
Babylon.js client.
What backend runtime are you leaning towards for those headless physics nodes?