WebSocket protocol

Need a hand or have feedback? The developer SDK and these docs are a work in progress, so nothing here is set in stone. If anything is unclear, missing, or wrong — or you'd like to propose a change — reach out directly:

• Email: contact@flyingart.dev
• Discord: https://discord.gg/pmAtpESgbH

The engine's WebSocket at ws://<host>:<port>/ws is the one place where state updates, panel discovery, config bundles, license operations, and panel pixel streams all meet.

Text frames are JSON. Binary frames are panel pixel data with a small header.

Connection

ws://<host>:<port>/ws?name=<display-name>&appKey=<key>&connectionType=<process|viewer>

Query param	Purpose
name	Human-readable name, surfaced in /status and logs.
appKey	App key for analytics and future billing.
connectionType	process keeps the engine alive; viewer doesn't.

On connect, the engine sends a welcome frame:

{
  "type": "welcome",
  "clientId": "c_89d2c1ef",
  "state": {
    /* EngineState */
  },
  "panels": [
    /* Panel[] */
  ],
  "configs": [
    /* ConfigBundle[] */
  ]
}

Server → client (text frames, JSON)

Type	Payload	Sent when
welcome	{ clientId, state, panels, configs }	Immediately after the WS opens.
state	Flat EngineState fields (isAdmin, msfsConnection, directX, markerInjection, …)	Engine state changed (MSFS connection, DirectX, marker-injection cycle, etc.).
panels	{ panels: Panel[] }	Panel list changes.
configs	{ configs: ConfigBundle[] }	A process client pushed / cleared profile bundles.
heartbeat	{}	~10 s app-level keepalive sent alongside the WS ping. Browsers can't observe protocol pongs from JS, so the heartbeat keeps idle viewers from self-evicting.
time.sync.reply	{ clientSentMs, engineMs }	Reply to a time.sync request. clientSentMs is echoed verbatim; engineMs is Date.now() on the engine when the reply was sent.
panel.debug	{ panelId, pendingAgeMs, bufferedBytes, qualityLevel, transitionCount, skipped }	Per-tick backpressure telemetry. Only sent if the client passed debug: true in subscribe.panel; zero cost otherwise.
license.response	{ id, ok, error?, credentials?, product?, deactivatedCount?, source? }	Reply to any license.* request (correlated by id).

Client → server (text frames, JSON)

Type	Fields	Purpose
subscribe.panel	panelId, optional crop: {x,y,w,h}, optional targetFps (10–60), optional debug, optional forceEveryFrame	Start receiving binary frames. See Subscription options below.
unsubscribe.panel	panelId	Stop.
panel.input	panelId, x, y, kind: "click", optional delayMs (default 300)	Synthesise a click at the source panel's native pixel coords. The engine forwards via Coherent's CDP Runtime.evaluate.
config.push	ownerName, profiles: Profile[]	Push profile bundles. Engine keeps them in memory only; broadcasts the aggregated configs list to all clients. Process clients only.
time.sync	clientSentMs (client Date.now())	NTP-style offset probe; engine replies with engineMs so the client can compute clock skew (see Clock sync).
license.activate	id, licenseKey	Activate a license. Response carries fresh credentials for the caller to persist.
license.deactivate	id, licenseKey, deviceToken	Release a license. Engine is stateless — the caller supplies the stored credentials.
license.validate	id, licenseKey, deviceToken	Re-validate on startup / reconnect. On success, the caller's name is added to the engine's validatedOwners set, which gates the DEMO overlay.

License requests are correlated to license.response by the id you pass. The SDK's activateLicense / deactivateLicense / validateLicense methods wrap this for you.

Subscription options

subscribe.panel accepts the following fields beyond panelId:

Field	Type	Default	Meaning
crop	{x, y, w, h}	—	Layout hint only — the engine still streams the full panel; the rect is broadcast so viewers can crop client-side (typical for fragments).
targetFps	number	30	Per-subscription frame rate. Server-clamped to [10, 60]. The engine ticks at the max across all subscribers and decimates per client, so a 30 fps phone and a 60 fps PC sharing a panel run on one 60 fps tick loop.
debug	boolean	false	Enables panel.debug text frames every tick (backpressure, pending-frame age, current quality bucket, transition count, skipped flag). Used by the debug overlay; zero overhead when off.
forceEveryFrame	boolean	false	Bypass the static-frame fingerprint skip. The engine normally hashes ~576 sample pixels per tick and skips encode + send when the hash matches the last frame; setting this forces a send on every tick whose frameId advanced. Per-panel, not per-client — if any subscriber sets it, the engine sends to everyone. Use only on the local PC viewer where bandwidth is free.

Clock sync

The SDK runs a 5-sample time.sync exchange on connect (and every 60 s after) to estimate the engine's wall-clock offset:

rtt          = clientRecvMs - clientSentMs
engineOffset = engineMs + rtt/2 - clientRecvMs

The smallest-RTT sample wins (asymmetric one-way delays cancel only when RTT is small). The result is exposed as engine.engineOffsetMs on the SDK and is the basis for end-to-end wire-latency measurement:

const wireMs = Date.now() - (frame.engineTimestampMs + engine.engineOffsetMs);

Binary panel frames

Whenever you're subscribed to a panel, frames arrive as binary WebSocket messages. The header is little-endian packed; the body is a standard JPEG. Every frame is self-contained, so a dropped frame just means you see the next one a tick later.

offset  size  field
------  ----  -----
0       1     panelId length (u8)
1       N     panelId (UTF-8)
N+1     4     frameNumber (u32 LE)
N+5     2     width  (u16 LE)
N+7     2     height (u16 LE)
N+9     8     engineTimestampMs (u64 LE) — Date.now() at the start of the
              tick that produced this frame. Use it with `time.sync` to
              compute one-way wire latency. Ignore if you don't care.
N+17    ...   JPEG bytes

Pipe the body into createImageBitmap (browser) or any JPEG decoder.

Frame rate

Each subscriber picks its own frame rate (10–60 fps) via the targetFps field on subscribe.panel. The default when omitted is 30 fps. The engine ticks the capture loop at the maximum targetFps any active subscriber requested, then decimates per client so each viewer sees only its requested fraction of ticks — a phone at 30 fps and a PC at 60 fps watching the same panel share one 60 fps tick loop without the phone receiving extra frames.

Identical source frames (nothing changed in MSFS) are fingerprinted and skipped: the engine hashes an NxN grid of pixels per tick and skips the encode + send when the hash matches the last sent frame for that panel. A single full frame is forced every ~1 s as a backstop. New subscribers get the current frame on the next tick regardless.

Adaptive quality

JPEG quality is per-subscriber and adaptive. Each connected client is placed on a discrete quality ladder (Q40 / Q60 / Q75 / Q85) and moved up or down based on its WebSocket buffer state — a phone on weak WiFi naturally settles at a lower bucket while a wired PC stays at the top. You don't subscribe to a quality; the server picks per client and sends the matching bytes. Setting GLASSOUT_JPEG_QUALITY=N on the engine pins all clients to a fixed quality and disables the ladder, useful for A/B testing.

Cropped subscriptions

If you subscribed with a crop rect, the server still streams the full panel — the SDK provides the crop hint purely so the engine can size layouts and so your code can render only the region you care about.

Idle shutdown

The engine shuts down ~5 seconds after the last process-type client disconnects. Viewer clients can linger forever, but they can't stop the engine from winding down when its last "driver" leaves.

Example bare-minimum client

import { WebSocket } from "ws";
import sharp from "sharp"; // or @julusian/jpeg-turbo, or any JPEG decoder

const ws = new WebSocket("ws://localhost:8787/ws?name=demo&appKey=demo&connectionType=viewer");

ws.on("message", async (data, isBinary) => {
  if (!isBinary) {
    const msg = JSON.parse(data.toString());
    if (msg.type === "welcome") {
      ws.send(
        JSON.stringify({
          type: "subscribe.panel",
          panelId: msg.panels[0].id,
          targetFps: 30, // optional — 10..60, default 30
        }),
      );
    }
    return;
  }

  const buf = data as Buffer;
  const panelIdLen = buf.readUInt8(0);
  const panelId = buf.subarray(1, 1 + panelIdLen).toString();
  const frameNumber = buf.readUInt32LE(1 + panelIdLen);
  const w = buf.readUInt16LE(1 + panelIdLen + 4);
  const h = buf.readUInt16LE(1 + panelIdLen + 6);
  // 8 bytes of engineTimestampMs (u64 LE) at offset (1 + panelIdLen + 8) —
  // read with readBigUInt64LE if you're computing wire latency, skip otherwise.
  const jpeg = buf.subarray(1 + panelIdLen + 16);

  const rgb = await sharp(jpeg).raw().toBuffer(); // w * h * 3 bytes
  // render rgb here — fully decoded panel frame
});

In a browser the example collapses to a createImageBitmap(new Blob([jpeg], { type: "image/jpeg" })) followed by ctx.drawImage. No prior-frame state, no XOR step, no decompressor library — every binary frame is a self-contained image.