Taking itself out of service

An unattended kiosk cannot tell a healthy backend from a dead one. The reason is the whole problem: between visitors there is no socket open. The tablet stays green and the avatar sits idle; the first thing to discover that the backend died in the night is a visitor pressing Start and waiting. Everything the component already notices — a recovery that gave up, a connect that failed, a namespace dropped mid-conversation — it spots only because somebody was standing there when it happened.

This page covers the automatic half. For the feature as a whole — the operator’s own latch, how the two combine, and what the visitor’s screen shows meanwhile — see Blackout: taking the kiosk out of service.

It works this out for itself, with no setup. bAutoBlackoutOnUnhealthy is on by default. It used to be off, on the grounds that a false positive costs a day of visitors. This is incorrect: the probe runs only while no conversation is up, avoiding interruptions. One success lifts the latch, and a latched box re-probes every five seconds. A false trip merely lasts the length of the outage plus a few seconds. Shipping with detection switched off meant deployments discovered its absence at the worst possible moment.

These last two settings dictate how long a real outage takes to show: up to one interval to catch the first failure, then the 30-second window. The interval is the one to shorten if this feels slow. From the first failure onward, the box re-probes every two seconds regardless. Thus, the window is the entire condemnation threshold, and shortening it means an access-point roam or a router taking twenty seconds to come back reads as an outage.

All of them are overridable from Saved/PromethistLinkConfig.ini without a rebuild. This matters because they live on a component inside a Blueprint asset; otherwise, a deployed box could only turn the feature off. This section also records the two things the box must remember across a process restart, so expect to see them appear:

BlackoutManual survives a Watchdog restart, a power cut and a nightly reboot. Somebody shut the kiosk on purpose, and a 3am crash does not invalidate that decision. This is also how you clear a box whose tablet is lost. LastBackendUrl is recorded when a conversation actually reaches a backend, never when it merely attempts one. Therefore, a single ?region=US visitor cannot leave a European installation probing an unused host.

One check ships, and its shape reflects the design: net.backend — nothing answers where the backend should be. This is the only check that discovers an outage before a visitor does, and the only one that records a verdict.

The OS’s own network signal is deliberately not a check. UE broadcasts it on Windows through the Network List Manager and the plugin listens — but only to ask for a probe, never to conclude anything. It cannot serve as a verdict in either direction. The engine calls any live adapter "connected" regardless of what is reachable through it, and it reports "no network" for a gateway-less venue LAN with the backend on the same switch, potentially condemning a working installation. Furthermore, the signal only changes when the OS dictates. A steady-state reading produces no further events, meaning a latch from it could never clear. As a trigger, however, it is free, instant, and cannot be wrong: a cable going back in is exactly the moment to re-ask the question of something that can answer it.

The failures the component already detects report nothing. They ask for a probe. This is the load-bearing decision here. A recovery give-up has no matching "and now it works again" event. A verdict it latched could therefore only be cleared by something else, letting one bad afternoon hold a kiosk out of service permanently. Escalating instead gives every check exactly one authority and one way back. It also makes the answer faster where it matters: a give-up followed by three quick probes settles the question in seconds rather than waiting out the interval.

Four rules keep the probe from being worse than the problem, and the second makes it safe to have at all:

With the neutral URL set, a confirmed failure asks one more question before wording itself. If ours is silent but the neutral answers, our backend is down; if neither answers, the building lacks working internet. Same verdict, different sentence — and a different person to call. Leave it empty on a locked-down kiosk LAN, where a blocked third party provides evidence of nothing.

-NoPromethistHealth on the command line disables the whole thing regardless of the component — the latch as well as the probe. This is because its failure mode is an installation that refuses to serve anybody, which must be stoppable on a box nobody can rebuild for.

With no tablet attached, bind OnHealthChanged or the feature is invisible. This is the common partner case; otherwise, the log remains its only output. The event fires whether or not the latch is armed, which is why it exists separately from OnBlackoutChanged.

One wire your project owes this, and it is not optional if you speak mid-sentence. A blackout tears the socket down, so the backend’s #exit never arrives. Speech2Face leaves its animating state only on that message or an explicit stop. Without a wire, it plays out the already inferred frames over silence and then holds the last one: a mouth frozen mid-word. Bind On State Changed (Disconnected) to S2F’s Stop Animation to cover every teardown at once — a blackout, the browser’s X, the AFK timeout, and a recovery give-up — not just this feature.

A streamed viewer gets a landing page, not a frozen frame. Any blackout — either latch — sends the browser the same recoveryFailed message a lost backend does, with reason: "outOfService". This tears down WebRTC and lands on the disconnected overlay with its own wording. Disconnect() sends nothing by itself, so previously, a lockout mid-session left a still image of a live-looking avatar on screen with nothing to explain it.

Reporting your own checks is one node, and it is the extension point the rest of this is built on:

Anything can call it — a graph, or an external process reaching in over this very tablet channel, which is already the box’s control plane. Two obligations come with it. Own the check’s name, since two reporters sharing one will overwrite each other. And report the recovery too: nothing expires a verdict, so a check reported bad stays bad until the same reporter says otherwise. This is right for something watched continuously but wrong for a one-off failure. For those, copy the component’s own pattern: ask for a probe rather than recording a verdict.

OnHealthChanged fires on the edge either way, whether or not the installation goes out of service, so a project can react to a fault without latching anything.