Download Game! Currently 60 players and visitors. Last logged in:DraxxWandererGralgarrFonkensemble

Blitzer's Blog >> 72231

Back to blogs index
Posted: 10 Sep 2026 06:52 [ permalink ]
Start with preprocessor imports. Add runtime `eval` only after sandboxing and
resource limits exist.

### 6. Async and determinism: yes, this is the right stage, but be precise

You cannot get true wall-clock deterministic total execution times with worker
threads and OS scheduling. You can get:
- Deterministic logical time via gas/instruction budgets.
- Bounded wall-clock time via deadlines and polling.
- Pollable intermediate state via shared memory.

Recommended architecture:
- Main VM spawns a worker via `SPAWN`.
- Worker runs an isolated VM with a `SharedArrayBuffer`.
- Worker writes snapshots to an SPSC ring buffer or seqlock-protected slot.
- Main VM uses `POLL` to synchronously read the latest complete snapshot.
- `CANCEL` sets an atomic flag; worker checks it every N instructions.
- `GAS` decrements a shared or local counter; exhaustion triggers
yield/terminate.

For the ring buffer:
- Header: `writeSeq`, `readSeq`, `status`, `cancelFlag`, `gasRemaining`.
- Data: fixed-size slots or circular byte buffer.
- Producer writes payload, then atomically publishes sequence.
- Consumer reads sequence, payload, rechecks sequence for consistency.
- If only latest state matters, use double-buffer + atomic flip instead of a
full ring.

This gives you time-boxed execution with partial results, which is usually
what deterministic total program execution actually needs.

## Implementation plan

### Phase 0  Spec and conformance suite
- Write formal grammar for JSON-Lisp AST.
- Define symbol vs literal rules.
- Define special forms and arity.
- Define evaluation order, scoping, `STORE` vs `SET`, error behavior.
- Build a conformance suite of ASTs and expected results.

### Phase 1  Interpreter MVP
- Implement `evaluate(ast, env)`.
- Support primitives, `if`, `def`, `let`, `set`, arithmetic, comparison.
- Use it to validate semantics before compiling.

### Phase 2  Compiler and stack VM
- Implement `compile(ast)` to flat bytecode.
- Implement VM with data stack, call stack, instruction pointer.
- Add `CONST`, `LOAD`, `STORE`, `ADD`, `MUL`, `JMP`, `JMP_FALSE`.
- Add `ENTER_SCOPE`, `EXIT_SCOPE`.

### Phase 3  Functions, closures, and mutation
- Add `MAKE_FUNC`, `CALL`, `RET`.
- Capture closure environment at definition time.
- Implement `SET` with correct scope-chain traversal.
- Add tail-call optimization if deep recursion is expected.
- Replace prototype envs with `Map`-based envs for safety.

### Phase 4  Macros, imports, and runtime eval
- Add preprocessor for `import` and macro expansion.
- Add `quote` and list manipulation.
- Add `EVAL` only after sandboxing and gas limits exist.

### Phase 5  Async and parallel execution
- Add `SPAWN`, `POLL`, `CANCEL`, `GAS`.
- Implement worker-thread VM.
- Implement `SharedArrayBuffer` snapshot buffer.
- Add task registry, status codes, and deadline handling.
- Add tests for partial results, cancellation, and gas exhaustion.

### Phase 6  Hardening and tooling
- Add resource limits: memory, stack depth, gas, wall-clock deadline.
- Add security: no host access unless explicit opcodes; sanitize variable
names.
- Add debugger: source maps, telemetry, instruction tracing.
- Add benchmarks: compile time, VM throughput, token efficiency, async poll
latency.
- Add docs generated from opcode/special-form registry.

## Final recommendation

Keep JSON-Lisp as the canonical executable AST. Add a human/AI surface syntax
later if token efficiency becomes criticalraw S-expressions are often more
compact than JSON because they avoid quotes and commas. For now, focus on
formal semantics, safe environments, and a clean compiler/VM split.

The async design is sound, but frame it as bounded, pollable, cooperative
execution rather than strict wall-clock determinism. That will give you the
practical benefits: time-boxed execution, partial results, cancellation, and
parallelism without fighting Nodes event loop.