Start with preprocessor imports. Add runtime `eval` only after sandboxing and resource limits exist. ### 6. Async and determinism: yes, this is the right stage, but be precise You cannot get true wall-clock deterministic total execution times with worker threads and OS scheduling. You can get: - Deterministic logical time via gas/instruction budgets. - Bounded wall-clock time via deadlines and polling. - Pollable intermediate state via shared memory. Recommended architecture: - Main VM spawns a worker via `SPAWN`. - Worker runs an isolated VM with a `SharedArrayBuffer`. - Worker writes snapshots to an SPSC ring buffer or seqlock-protected slot. - Main VM uses `POLL` to synchronously read the latest complete snapshot. - `CANCEL` sets an atomic flag; worker checks it every N instructions. - `GAS` decrements a shared or local counter; exhaustion triggers yield/terminate. For the ring buffer: - Header: `writeSeq`, `readSeq`, `status`, `cancelFlag`, `gasRemaining`. - Data: fixed-size slots or circular byte buffer. - Producer writes payload, then atomically publishes sequence. - Consumer reads sequence, payload, rechecks sequence for consistency. - If only latest state matters, use double-buffer + atomic flip instead of a full ring. This gives you time-boxed execution with partial results, which is usually what deterministic total program execution actually needs. ## Implementation plan ### Phase 0 Spec and conformance suite - Write formal grammar for JSON-Lisp AST. - Define symbol vs literal rules. - Define special forms and arity. - Define evaluation order, scoping, `STORE` vs `SET`, error behavior. - Build a conformance suite of ASTs and expected results. ### Phase 1 Interpreter MVP - Implement `evaluate(ast, env)`. - Support primitives, `if`, `def`, `let`, `set`, arithmetic, comparison. - Use it to validate semantics before compiling. ### Phase 2 Compiler and stack VM - Implement `compile(ast)` to flat bytecode. - Implement VM with data stack, call stack, instruction pointer. - Add `CONST`, `LOAD`, `STORE`, `ADD`, `MUL`, `JMP`, `JMP_FALSE`. - Add `ENTER_SCOPE`, `EXIT_SCOPE`. ### Phase 3 Functions, closures, and mutation - Add `MAKE_FUNC`, `CALL`, `RET`. - Capture closure environment at definition time. - Implement `SET` with correct scope-chain traversal. - Add tail-call optimization if deep recursion is expected. - Replace prototype envs with `Map`-based envs for safety. ### Phase 4 Macros, imports, and runtime eval - Add preprocessor for `import` and macro expansion. - Add `quote` and list manipulation. - Add `EVAL` only after sandboxing and gas limits exist. ### Phase 5 Async and parallel execution - Add `SPAWN`, `POLL`, `CANCEL`, `GAS`. - Implement worker-thread VM. - Implement `SharedArrayBuffer` snapshot buffer. - Add task registry, status codes, and deadline handling. - Add tests for partial results, cancellation, and gas exhaustion. ### Phase 6 Hardening and tooling - Add resource limits: memory, stack depth, gas, wall-clock deadline. - Add security: no host access unless explicit opcodes; sanitize variable names. - Add debugger: source maps, telemetry, instruction tracing. - Add benchmarks: compile time, VM throughput, token efficiency, async poll latency. - Add docs generated from opcode/special-form registry. ## Final recommendation Keep JSON-Lisp as the canonical executable AST. Add a human/AI surface syntax later if token efficiency becomes criticalraw S-expressions are often more compact than JSON because they avoid quotes and commas. For now, focus on formal semantics, safe environments, and a clean compiler/VM split. The async design is sound, but frame it as bounded, pollable, cooperative execution rather than strict wall-clock determinism. That will give you the practical benefits: time-boxed execution, partial results, cancellation, and parallelism without fighting Nodes event loop.