---

## 2. Semantic Delta Telemetry (Diffing)

**Goal:** Instead of retransmitting the whole tree on every change, generate a
stream of *intent-level* events describing *what changed* and *why it
matters*.

### Brainstorming Points

- **What constitutes a semantic delta?**  
  A raw DOM mutation (node inserted) is low-level; you want to interpret it:
  - A new log line appeared rather than A text node was appended to div#log.
  - The submit button became disabled rather than attribute `disabled` was
added.
  
  Thus, each delta should include:
  - **Target:** the affected element (identified by stable ID or path).
  - **Type of change:** e.g., `text_updated`, `state_changed`,
`element_added`, `element_removed`, `attribute_modified`, `focus_moved`,
`scroll_position`, `value_changed`.
  - **New state/value:** the current value of the relevant property.
  - **Old state/value** (optional, for diffing).

- **Using MutationObserver:**  
  MutationObserver gives you raw DOM changes. To turn them into semantic
deltas, you need to:
  1. **Debounce/batch:** Many mutations happen in quick succession; you often
want to coalesce them into a single logical change (e.g., a single frame
update).
  2. **Map mutations to affected accessibility nodes:** Determine which nodes
in your semantic tree were impacted.
  3. **Extract meaningful change:** Compare old/new accessibility properties
(role, name, state, value) to detect what actually changed from a user/agent
perspective.
  4. **Generate a high-level event:** e.g., if a `<div role="status">` gets
new text, produce `{"event": "STATUS_UPDATE", "message": "Operation
completed"}`.

- **Alternative: Use higherlevel browser events**  
  Many changes are already exposed as events: `input`, `change`, `focus`,
`blur`, `scroll`, `click`. You can listen to these and translate them directly
into semantic events without parsing raw mutations. However, not all changes
have corresponding events (e.g., attribute changes via JavaScript). A hybrid
approach works: use events for user interactions, MutationObserver for
programmatic changes, and a periodic accessibilitytree diff for anything
missed.

- **English narrative generation:**  
  The deltas JSON can be used to generate a natural language description via
templates or an LLM at the consumer side. For example:
  - `{"event": "TEXT_APPENDED", "target": "#log", "value": "[SUCCESS] Agent
process finished"}`  The log panel now shows: [SUCCESS] Agent process
finished.
  - `{"event": "STATE_CHANGED", "target": "#submitBtn", "property":
"disabled", "value": true}`  The submit button is now disabled.

- **Consideration: Avoiding noise**  
  Many UI updates are transient or irrelevant (e.g., a spinner animation, a
tooltip appearing/disappearing). You may want to filter deltas based on:
  - Whether the change affects an element that is semantically important
(interactive, status, etc.).
  - Whether the change is likely to affect the agents decisionmaking (e.g., a
new message vs. a background color change).
  - Throttle events by time or importance.