To support parallel execution across the Omega Mesh, we need to upgrade the
Virtual Machine's yield mechanism.
Right now, the call-other macro automatically forces the VM to yield a single
network envelope. To support parallel fetching (like Promise.all), we need to
separate the construction of a request from the yielding of it.
We will introduce a request-other macro that builds the envelope without
pausing the VM, and a yield-all macro that accepts an array of envelopes and
yields them simultaneously. The SupervisorEngine will then act as a parallel
multiplexer: it will fire them all out, track their replies, and resume the VM
with a perfectly ordered array of results.
1. Upgrade the Macros (engine.mjs)
Open engine.mjs and locate your macros dictionary. We are going to add
request-other and yield-all to the list.
Update the macros block to look exactly like this:
const macros = { 'and-then': (args) => ['if', args[0], args[1], false],
'or-else': (args) => ['if', args[0], true, args[1]], 'cond': function
processCond(args) { if (args.length === 1) return args[0]; if (args.length ===
0) return null; return ['if', args[0], args[1], processCond(args.slice(2))];
}, '+=': (args) => ['set', args[0], ['+', args[0], args[1]]], 'until': (args)
=> ['while', ['not', args[0]], args[1]], 'register-object': (args) =>
['yield', ['record', 'action', ['str', 'sys_register'], 'object_id',
args[0]]], 'call-other': (args) => ['yield', ['record', 'action', ['str',
'call_other'], 'target_obj', args[0], 'method', args[1], 'args', ['array',
...args.slice(2)]]], 'timer:sleep': (args) => ['yield', ['record', 'action',
['str', 'sleep'], 'ms', args[0]]], // ---> NEW: PARALLEL DISPATCH MACROS <---
'request-other': (args) => ['record', 'action', ['str', 'call_other'],
'target_obj', args[0], 'method', args[1], 'args', ['array', ...args.slice(2)]],
'yield-all': (args) => ['yield', ['array', ...args]] };
2. Upgrade the Supervisor Engine (mesh.mjs)
Now open mesh.mjs. We have to teach the SupervisorEngine how to multiplex an
array of yields, tag them with an index, and wait for all the replies to
arrive before waking the worker.
Make the following four surgical patches inside mesh.mjs:
Patch A: The Inbox Multiplexer Find this.mailbox.onReceive inside
SupervisorEngine and replace the sys_ack / call_reply block:
this.mailbox.onReceive((envelope) => { if (envelope.action === 'sys_ack' ||
envelope.action === 'call_reply') { const actor = this.actors.get(envelope.targ
et_obj); if (actor) { // ---> HYBRID INBOX: Handle both single and parallel
replies if (actor.pendingYields && envelope.req_index !== undefined) {
actor.pendingYields.results[envelope.req_index] = envelope.data;
actor.pendingYields.count++; if (actor.pendingYields.count ===
actor.pendingYields.total) { const finalResults = actor.pendingYields.results;
actor.pendingYields = null; // Clear the queue actor.worker.postMessage({
type: 'resume', data: finalResults }); } } else { actor.worker.postMessage({
type: 'resume', data: envelope.data }); } } }
Patch B: Forwarding the Request Index Scroll down slightly to the call_other
block in the same function. Add the reqIndex to the context so the executing
node remembers it:
actor.worker.postMessage({ type: 'eval', bytecode: assemble(optimize(compile(c
allAst))), context: { replyTo: envelope.source_obj, replyNode:
envelope.source_node, reqIndex: envelope.req_index // <-- Add this tag! }
});
Patch C: The Return Address (and WAL replay) Scroll down to worker.on('message'
, msg => { ... if (msg.type === 'completed'). Update both the standard
call_reply sender and the WAL replay context to include the index:
this.mailbox.send({ action: 'call_reply', target_obj: msg.context.replyTo,
target_node: msg.context.replyNode, req_index: msg.context.reqIndex, // <--
Add this tag! data: msg.data });
(Also add reqIndex: envelope.req_index to the context object inside the WAL
replay block worker.postMessage({ type: 'eval' ... }) at the bottom of the
function).
Patch D: The Outbox Dispatcher Finally, scroll down to the msg.type ===
'yield' listener and replace it with this array-aware multiplexer:
else if (msg.type === 'yield') { if (Array.isArray(msg.data)) { // --->
PARALLEL DISPATCHER <--- const actor = this.actors.get(objectId);
actor.pendingYields = { results: new Array(msg.data.length), count: 0, total:
msg.data.length }; msg.data.forEach((envelope, idx) => { envelope.source_obj
= objectId; envelope.req_index = idx; // Tag it! this.mailbox.send(envelope);
}); } else { // ---> SINGLE DISPATCHER <--- const envelope = msg.data;
envelope.source_obj = objectId; this.mailbox.send(envelope); } }
3. Deploy the Parallel Master!
Restart your hal-9000 console so the daemon reloads mesh.mjs.
Now, we deploy the upgraded Master node payload. Notice how we swapped
call-other out for the request-other and yield-all macros!
Go to your Mission Control dashboard:
Target Node: Engine#Tampere
Actor ID: Actor#Master
Payload:
[ "do", ["def", "reduce_job", [], ["let", [ ["results", ["yield-all",
["request-other", ["str", "Actor#WorkerHelsinki"], ["str", "crunch"], 5],
["request-other", ["str", "Actor#WorkerOulu"], ["str", "crunch"], 5] ]] ],
["+", ["get", "results", 0], ["get", "results", 1]] ] ], ["register-object",
["str", "Actor#Master"]] ]
Run the Distributed Job!
(Make sure Helsinki and Oulu still have the Worker actors deployed from the
previous step).
Go to the [ Execute Actor (HTTP) ] panel, send a POST to Actor#Master calling
reduce_job.
Watch the terminal telemetry! Tampere will blast out both crunch() operations
simultaneously. Helsinki will sleep for 1s, Oulu will sleep for 1.5s. Exactly
1.5 seconds later (instead of 2.5 seconds), Tampere will resume with an array
of [50, 100], sum them natively in WebAssembly, and return RESULT: 150!