You are absolutely right on the money with your intuition. Pulling the raw DOM
directly into your local window is a recipe for a broken mess.
If you just copy the DOM (document.documentElement.outerHTML), you lose all
the CSS (relative paths break), the JavaScript state is completely detached,
canvas elements are blank, and iframe cross-origin policies will throw a fit.
It essentially becomes a static, unstyled, broken webpage.
However, your idea to fetch screenshots and interact via the Chrome DevTools
Protocol (CDP) is brilliant. In fact, this is exactly how tools like
Puppeteer, Playwright, and modern cloud browsers operate under the hood!
Since CDP is literally just JSON sent over WebSockets, you can build a
lightweight, browser-specific "VNC" directly inside your mobile utility
without needing any actual VNC servers.
Here is how we can architect this for your HITL/AI automation sessions:
The Architecture: "CDP Remote"
Instead of sending pixels via an OS-level VNC, we connect your mobile web app
directly to the Linux devbox's Chromium WebSocket.
Visuals: We send a CDP command (Page.captureScreenshot) every X milliseconds.
The remote browser replies with a Base64 image, which we instantly paint onto
an <img> tag inside a virtual display.
Interaction: We put an onclick listener on that <img>. When you tap it on your
phone, we calculate the X/Y coordinates and send a CDP command
(Input.dispatchMouseEvent) back to the Linux box to simulate a real click.
Execution: You can send raw JavaScript (Runtime.evaluate) directly from your
mobile REPL to the Linux browser.
Phase 1: Preparing the Linux Devbox
To allow your mobile phone (http://Elli:33370) to connect directly to the
Linux box's Chromium, you must launch Chromium with three specific flags. By
default, Chromium rejects external WebSocket connections to prevent DNS
rebinding attacks.
chromium \
  --remote-debugging-port=9222 \
  --remote-debugging-address=0.0.0.0 \
  --remote-allow-origins="*"

(Note: Exposing 0.0.0.0:9222 is dangerous on a public network, but totally
fine on a secure private LAN/VPN).
Phase 2: The JavaScript Helper (Run this in your REPL/Console)
Here is a proof-of-concept helper you can add to your windows.html or just
paste directly into your REPL. It discovers the remote tab, connects to its
WebSocket, and sets up a live-updating screenshot display.
class RemoteBrowserViewer {
  constructor(linuxIp, port = 9222) {
    this.baseUrl = `http://${linuxIp}:${port}`;
    this.ws = null;
    this.msgId = 1;
    this.callbacks = new Map();
    
    // Create the virtual display for the remote stream
    this.displayId = 'remote-linux';
    this.imgEl = document.createElement('img');
    this.imgEl.style.width = '100%';
    this.imgEl.style.height = 'auto';
    this.imgEl.style.display = 'block';
    
    const container = document.createElement('div');
    container.style.backgroundColor = '#000';
    container.appendChild(this.imgEl);
    
    DisplayManager.add(this.displayId, container);
  }

  async connect() {
    console.info(`Fetching active tabs from ${this.baseUrl}...`);
    try {
      // 1. Get the list of open tabs from Chromium
      const res = await fetch(`${this.baseUrl}/json/list`);
      const tabs = await res.json();
      
      // Find the first actual page (ignore background extensions)
      const page = tabs.find(t => t.type === 'page');
      if (!page) throw new Error("No active pages found.");

      console.info(`Connecting to: ${page.title}`);
      
      // 2. Connect to the CDP WebSocket
      this.ws = new WebSocket(page.webSocketDebuggerUrl);
      
      this.ws.onmessage = (event) => {
        const data = JSON.parse(event.data);
        if (data.id && this.callbacks.has(data.id)) {
          this.callbacks.get(data.id)(data.result);
          this.callbacks.delete(data.id);
        }
      };

      this.ws.onopen = () => {
        console.info("CDP WebSocket Connected!");
        this.startScreenStream();
      };
      
    } catch (err) {
      console.error("Connection failed:", err);
    }
  }

  // Helper to send CDP Commands
  sendCommand(method, params = {}) {
    return new Promise((resolve) => {
      const id = this.msgId++;
      this.callbacks.set(id, resolve);
      this.ws.send(JSON.stringify({ id, method, params }));
    });
  }

  // Continuously fetch screenshots
  async startScreenStream() {
    const fetchFrame = async () => {
      if (this.ws.readyState !== WebSocket.OPEN) return;
      
      const result = await this.sendCommand('Page.captureScreenshot', { 
        format: 'jpeg', 
        quality: 50 // Keep it lightweight for mobile streaming
      });
      
      if (result && result.data) {
        this.imgEl.src = `data:image/jpeg;base64,${result.data}`;
      }
      
      // Request next frame (approx 10fps to save mobile battery/bandwidth)
      setTimeout(fetchFrame, 100); 
    };
    
    fetchFrame();
  }
}

Phase 3: Trying it out
Once you have your Linux Chromium running on the same network, open your
mobile REPL and type:
/ 1. Initialize with your Linux box's IP address
const remote = new RemoteBrowserViewer('192.168.1.50'); // Replace with actual
IP
/ 2. Connect
remote.connect();
/ 3. Switch your view to watch the stream!
DisplayManager.switchTo('remote-linux');

Why this is a game-changer for AI/HITL Automation
If you are running AI automation agents (like LLMs parsing the DOM or trying
to click things), standard VNC is completely blind to the actual web
structure.
By building this CDP bridge, you get the visual monitoring (via the
screenshots), but from your mobile REPL, you can now seamlessly fire commands
like:
remote.sendCommand('Runtime.evaluate', { expression: 'document.title' })
remote.sendCommand('DOM.getDocument')
You can monitor what the AI is doing visually, jump into the REPL to inspect
the exact DOM state of the Linux box, and tweak variables on the flyall while
laying on the couch with your phone.