Physical buttons and other hardware

A tablet is not special. The control channel is a WebSocket carrying JSON. Anything that can open one is a full client with the same rights: a microcontroller with two arcade buttons and a lamp, a phone, a PLC, a building-management system, or a script on a laptop.

This page helps you build that. You need no web technology, no npm package, and nothing from Unreal — just a WebSocket client library and a JSON parser. On an ESP32, that means two Arduino libraries; in Python, the websockets package and the standard library.

Read The layers first if you have not. The rules below are the same ones the tablet follows; the wire protocol is the full specification.

This page is about hardware that drives the kiosk. For hardware the kiosk drives — such as DMX lighting tracking the avatar’s turn-taking state — see DMX lights and physical signalling, an Unreal-side change rather than a client.

What you connect to

ws://<kiosk-ip>:9010/link

Both the port and whether a token is required come from the kiosk’s ini file — see Quick start. Know these two things before plugging anything in:

A token is required from anything on the network. A connection from the kiosk’s own machine arrives on a loopback address and is trusted without one. Anything else must present the token in its first message; a hardware panel is anything else.
You count against MaxClients (four by default). A panel plus two tablets is three.

Try it before you build anything

Ten lines of Python will tell you whether the kiosk is reachable, whether your token is correct, and exactly what this installation publishes — before you cut any metal.

import asyncio, json, websockets

async def main():
    async with websockets.connect("ws://192.168.1.50:9010/link") as ws:
        await ws.send(json.dumps({"v": 2, "t": "hello", "token": "...", "client": "bench"}))

        # Anything you like, once you have seen the hello come back:
        # await ws.send(json.dumps({"v": 2, "t": "cmd", "cmd": "pause",
        #                           "args": {"value": True}}))

        while True:
            frame = await ws.recv()
            if isinstance(frame, bytes):          # the Unreal host sends binary
                frame = frame.decode("utf-8")
            print(frame)

asyncio.run(main())

The first thing printed is the kiosk’s hello, carrying every state key it publishes and its command vocabulary, if declared. Press buttons on the tablet and watch the state patches arrive. That output is the true specification for your panel, and it cannot go stale the way this page can.

The five rules

Everything a hardware client must get right, in one list.

Say hello first. Send {"v":2,"t":"hello","token":"…"} as your first frame. Until you do, every other message is refused. The kiosk answers with its own hello carrying a complete snapshot of every state value it holds. It may send another hello later on the same connection, so treat each one as replacing everything you knew.
Accept binary frames. The Unreal host can only send binary — a limitation of the engine’s WebSocket plugin, not a choice — so decode incoming frames as UTF-8 yourself. A host on another engine may send text instead, and a client must accept either. Your own frames must be text. Some minimal libraries only surface text frames; check yours before designing around it.
Unwrap batch. A frame of type batch carries a list of messages in msgs. Process them in order. A host that can only deliver one frame per tick uses this; the Unreal host does.
Never toggle locally. A button that turns something on and off must send the opposite of what the kiosk last confirmed, then wait. Do not keep your own idea of whether it is paused; you will be wrong the moment anyone touches the tablet.
Stay inside the budget. Twenty commands a second sustained, forty in a burst. A debounced push-button is nowhere near that. A control wired straight to a rotary encoder or an analogue pin easily is. Throttle continuous inputs before they reach the socket.

Wiring a button

There are three kinds, depending on what the kiosk publishes for it rather than what the button looks like.

A plain action. Restart, or a canned phrase. Send it and forget it.

{"v":2,"t":"cmd","cmd":"restart"}

Debounce the contact in your own firmware. The kiosk will happily accept two presses 5 ms apart, because from its perspective, that is indistinguishable from an operator genuinely pressing twice.

A toggle. Pause, microphone, blackout. Read the state key, send the opposite:

/* paused_known is set only from an incoming state frame. Track separately whether
   the key has ever arrived at all. */
if (have_seen_paused) {
    send_cmd_bool("pause", !paused_known);
}

That have_seen_paused check is not defensive clutter. Until the kiosk publishes the key, there is no honest value to send. The tablet keeps such a button disabled rather than guessing, and a hardware button should refuse to fire for the same reason.

An action that takes time. Call assistance. Send it once; the kiosk publishes assistancePending while the work is outstanding and clears it when done. Light the button while that key is true and ignore presses meanwhile.

Wiring a lamp

A lamp is a client that only listens. Connect, say hello, then drive the output from state.

Key Good for

Key	Good for
`sessionActive`	a conversation is happening — the obvious "in use" light
`paused`	paused, on the panel that paused it
`blackout`	out of service. The one worth putting on a big red lamp
`health.failed`	the names of the kiosk’s failing self-checks; empty means it believes it works
`connection`	the conversation state — `Ready`, `Listening`, `Speaking`, or `Unavailable`

sessionActive

a conversation is happening — the obvious "in use" light

paused

paused, on the panel that paused it

blackout

out of service. The one worth putting on a big red lamp

health.failed

the names of the kiosk’s failing self-checks; empty means it believes it works

connection

the conversation state — Ready, Listening, Speaking, or Unavailable

Three details decide whether the lamp is ever wrong:

Track rev. Each state frame carries a revision number increasing by one. If you ever see a gap, send {"v":2,"t":"cmd","cmd":"sys.resync"} — the one command the kiosk answers itself — and take the fresh hello that comes back. Skipping this is the one shortcut that eventually leaves a lamp stuck on.
A null value means the key was removed, not that it became false.
Silence is not "fine". Send {"v":2,"t":"ping"} every few seconds and expect a pong. A kiosk whose main thread has wedged leaves the socket open while serving nothing. Since nothing disconnects, the last state you saw stays on your lamp forever. The tablet treats eight seconds without a frame as stale and greys itself out. A lamp should similarly go dark or amber, not keep showing a conversation that ended an hour ago.

A box that can take the kiosk out of service

This is the most useful non-button device to build, needing no cooperation from anybody. Send blackout with true, and the kiosk ends any conversation, refusing to start another until you send false. Every tablet in the room fogs over at the same moment, because they all render one published key.

{"v":2,"t":"cmd","cmd":"blackout","args":{"value":true}}
{"v":2,"t":"cmd","cmd":"blackout","args":{"value":false}}

A key-switch on the wall is one use. A better one is a supervisor: something watching a sensor, a camera process, or a turnstile can hold the whole installation out of service while that component is broken, lifting the hold when its own restart succeeds. Promethist Watchdog is the supervisor we ship for processes on the box; a client like this extends that idea to things the watchdog cannot see.

One trap. The blackout command latches only the operator’s half of the lockout, while the blackout key reflects that half OR the kiosk’s own failing self-checks. Send false while a self-check is failing and the key correctly stays true — your command did land. Watch blackoutManual to see your own half. See Taking itself out of service.

Do not hard-code our vocabulary

The hello you printed earlier is the honest list for that installation. A host may also declare its commands in server.commands, and each entry can carry a reflects field naming the state key confirming the command landed.

Read reflects rather than assuming a command is named after the key it changes. Most of ours are not; blackout actively is not, as above.

Before you install it

Reconnect forever, with a delay. A panel screwed to a wall must come back on its own — the kiosk will be restarted without anyone telling your panel about it.
Fail safe. Decide what your outputs do when the socket is down, and make that the power-on state too.
Keep the token out of anything readable. It is a shared secret, not a password per device.
Test with the reference tablet connected at the same time. Two clients is the normal case, and exactly where a client keeping its own idea of the state gets caught.