A safety layer that measures itself before it judges a robot.

Frames in, reviewable events out.

Capture, inference and posting run as three independent loops. A capture thread per camera decodes at the camera's native rate and streams video to the console. An inference worker samples the latest frame on its own interval, asks a model adapter for detections and evaluates them against the safety policy. An asynchronous poster sends the resulting safety events to a FastAPI backend with an operator review queue. Every event carries the rule, severity, confidence, a human-review flag and its evidence.

The adapter boundary is the point of the design. A deterministic mock adapter measures the pipeline alone. An OpenAI-compatible adapter puts a real vision-language model behind the same interface, so the same run artifact records both the runtime cost and what the model returned.

Every run writes one JSON artifact with device provenance, per-frame timings, token usage, event and detection counts, peak memory and a 1 Hz tegrastats summary. Nothing on this page is typed by hand.

What was built.

• Safety event schema and policy engine: PPE missing, zone intrusion, human-robot proximity, with severity, confidence gating and human-review flags.
• Edge worker with mock, OpenAI-compatible and Cosmos-Reason2 adapters; an asynchronous event poster with a bounded queue, retries and a drop counter; run reports with provenance.
• Operator console served by the API: Live, Cameras, Model, Runs and Events. Cameras, restricted zones, rules, model endpoint and inference cadence are set in the UI and applied without a restart.
• Camera setup for ten vendor profiles with a connection test that decodes one frame or names the failure. Passwords are encrypted at rest, never returned by the API, and masked in logs, events and run reports.
• A constrained-mode guard that refuses a JSON schema the model server would ignore, proven by a live canary request before any worker uses it.
• FastAPI backend with SQLite store, single and batch event endpoints, operator review, and Alembic migrations.
• Run recorder and tegrastats sampler that turn each run into a committed artifact; an evidence page generated from those artifacts.

The pipeline keeps camera rate; the backend does not.

With the mock adapter at 30 fps pacing, the worker sustained 28.588 frames/s over 1800 frames with rule evaluation at 0.9877 ms p95 and board power at the idle level, 24,604 mW p50 (`reports/thor/mock_30fps_no_post.json`). Posting each event to the backend synchronously dropped throughput to 4.357 frames/s at 160.9 ms per frame; batching a frame's events into one request lifted it to 7.075 frames/s at 68.2 ms (`mock_30fps_backend.json`, `mock_30fps_backend_batched.json`). Posting has since moved to an asynchronous poster off the capture and inference threads; its effect on the Thor is not measured yet, so these synchronous figures remain the committed numbers.

Cosmos-Reason2-2B served by vLLM on the same device cost 4.35 s p50 and 12.85 s p95 per frame with its reasoning block, at 66.6 W board power (`cosmos2b_video_60.json`). Reasoning length drove the tail: the slowest frames produced near 1000 completion tokens at about 61 tokens/s.

The grammar fixed the vocabulary and exposed the real problem.

Four runs on the same 60 frames. Disabling the think block halved the p95 to 6.58 s. Adding a JSON-schema grammar to the decoder removed every out-of-vocabulary label, 59 before, 0 after, and brought the p50 to 2.69 s and p95 to 3.36 s (`cosmos2b_video_60_schema.json`). It also made the 2B report six persons on footage that has none, and five of those tripped PPE and proximity rules: seven false safety events.

Cosmos-Reason2-8B under the same grammar and prompt reported zero persons and zero events on the same frames at 3.20 s p50 (`cosmos8b_video_60_schema.json`), decoding at roughly 13 tokens/s against 60 for the 2B. Sixty frames of one video is a signal, not a rate. A server detail is recorded too: with vLLM's `--reasoning-parser qwen3` enabled, `response_format` JSON schema was accepted but not enforced, so the constrained runs used a second container without the parser.

The config now refuses a schema the server would ignore.

The vLLM finding above is now enforced in code instead of documented. Saving constrained mode against a server started with a reasoning parser is refused, and before any worker uses constrained mode a canary request must prove the server honours the schema. The canary asks for one word while the schema allows only a different one.

On 2026-10-02 a probe drove the application on the Thor against the real vLLM 0.14 containers (`reports/thor/constrained_guard_probe.json`). The reasoning-parser server answered the canary with `banana`, a value the schema forbids, and the guard refused constrained mode. The container without the parser returned the schema value and was allowed.

What went wrong and what was corrected.

The first real-model run failed at frame 26 on a 60 s adapter timeout; the log is kept (`cosmos2b_video_60_attempt1_timeout.log`), the timeout became a command-line option, and a failed run now writes a partial artifact with its error. With the grammar and the original prompt, which still asked for an `<answer>` wrapper the grammar forbids, the model emitted an empty list on every frame; that run is kept as `cosmos2b_video_60_schema_v1prompt.json` and the prompt was rewritten for JSON-only output.

The unconstrained 2B echoed the schema string itself as a label on 12 detections. That is in the artifact, not edited out.

What is not established.

• No precision or recall for any rule. There is no labelled ground truth, so the false events above are counted, not rated.
• No committed live-camera run; the model footage is an Isaac Sim recording with no people in it.
• Runs are 60 to 390 seconds, not sustained operation.
• The asynchronous poster's effect on throughput and packet-to-event latency is not measured on the Thor.
• Zone geometry is configured, not derived from calibration.

Next work is a live-camera evidence run on the Thor in the existing run-report format, Cosmos-Reason2-8B on live video, and authored simulation scenes with known PPE and zone states so both model sizes get a precision and recall, labelled simulation.

Evidence page and raw records.

The evidence page is generated from the committed run artifacts by the repository's own script and served by GitHub Pages; every row links to its JSON.

Sources and links.