---

## 3. The Accessibility Engine Approach

**Goal:** Ensure the JSON AST and the English narrative are perfectly aligned,
so an LLM reasoning over either modality has an unambiguous understanding of
the UI state.

### Brainstorming Points

- **Single source of truth:**  
  Both representations should be generated from the same underlying semantic
model (the accessibility tree). That way, there is no drift between what the
JSON says and what the English says.  
  Think of it as: the JSON is a *compiled* form; the English is a
*prettyprinted* form.

- **Design principles for English narratives:**
  - **Deterministic & unambiguous:** Avoid pronouns; always refer to elements
by their accessible name and role.  
    Example: The Apply Patch & Resume button is enabled. instead of Its
clickable.
  - **Include only relevant state:** Mention disabled/enabled,
checked/unchecked, visibility, and current value if its an input.  
  - **Use consistent templates:** For each delta type, have a template like
{Role} {Name} {change description}. This makes parsing by the LLM easier.
  - **For full snapshots:** Summarize the screen hierarchically: The page
contains a navigation bar with links Home, Settings. Below is a form with
fields Username (text input, value john_doe), Password (password input,
empty), and a Login button (enabled).

- **Making the JSON LLMfriendly:**  
  The JSON AST should be designed so that an LLM can also reason directly over
it if needed. Use clear field names, consistent types, and avoid deeply nested
structures that are hard to traverse. Consider including a `"description"`
field in each node for convenience, or rely on the LLM to infer it from the
structured data.

- **Accessibility as a philosophy:**  
  By focusing on the accessibility tree, you are automatically capturing the
*intended* semantics of the UI, not just the visual appearance. This aligns
with how screen readers work and is exactly what an agent needs to see the
interface. You might even go further and include `aria-live` regions as
highpriority delta sourcesthey often announce important changes (e.g., form
submitted successfully).

- **Testing the bridge:**  
  Build a test suite where you compare the JSON output and the English output
against a set of known UI scenarios. For any divergence, adjust the templates
or the mapping logic. Also, verify that an LLM given only the English
narrative can correctly answer questions about the UI state (e.g., Is the
submit button enabled?).