The full build
Cameras watch a robot workcell. A vision-language model on the device reports who is in the frame and what they are wearing, safety rules turn that into structured events, and an operator reviews them. The full build adds the ways video gets in, an access gate that checks a worker's protective gear, a labeled scorer, and Cosmos 3 runs. It measures what it costs to run, and it records what the model gets wrong. The source of the full build is not public; the open core is.
reports/thor/The video shows the access gate reading a worker's protective gear with Cosmos Reason 2 on the Thor.
Paste an RTSP link with its credentials taken out and stored encrypted, pick a USB camera with its sizes and frame rates listed, upload an MP4, MOV or MKV, or share a phone or laptop camera from the browser. All of them run through the same rules and events.
Each camera, event and run report records where its video came from. Events and evidence frames from uploaded or synthetic sources carry a visible "not a live camera" note on Live, Events and in the run report.
A front-scan check of hard hat, glasses, vest and boots, read region by region with two question wordings that must agree. Camera loss, staleness, tailgating, departure, timeout, uncertainty and disagreement all deny. It is a compliance aid, not a safety interlock, and its output is simulated and closed by default.
Cosmos 3 Edge is a selectable backend beside Cosmos Reason 2, which stays the default. Both receive the same prompt, schema and settings.
Scores the rules against 60 labeled simulation pictures, with every request and answer kept as evidence.
The Runs page records a run in the same format as the committed Thor runs. A run refuses to start when the model server is not ready and stops, marked failed, if the server goes down.
The operator draws restricted zones on a snapshot from each camera and picks which rules run on it. Every rule works on what the model reports for that frame: labels, boxes, and for each person whether a hard hat and a high-visibility vest are worn.
A person without a hard hat or without a high-visibility vest. The blue circle is the ISO 7010 shape for a mandatory action.
A person's box overlaps a zone the operator drew as restricted, such as the robot cell.
The centre of a person's box within 90 pixels of the centre of a robot's box.
A pallet, cart or box the model reports as blocking an emergency path.
Anything else the model itself calls unsafe, passed through for review.
One thread per camera decodes the stream and keeps the newest frame. The console shows that video at the camera's own rate.
A worker samples the newest frame, asks the model for people, objects and PPE, and runs the rules bound to that camera.
Events go to a bounded queue drained by its own thread, with retries, a drop counter, and latency measured from frame capture to the backend's acknowledgement.
If the model server stops, the video keeps playing and the console shows "inference unavailable". Events only come from frames the model actually answered.
Read from rules/policies.py and rules/severity.py. The rule functions are pure: no I/O, the same detections always give the same findings.
Hard hat and vest only. Gloves, glasses and boots are not checked.
A degraded runtime lowers confidence by 0.08, a critical one by 0.18, and each dropped frame by 0.01 up to 0.10.
Every number below is read from a committed run artifact. The model runs used 60 frames of an Isaac Sim workcell recording with a robot arm and no people in it.
Hover a dot for its value. A sixth run is left off on purpose: a grammar with a mismatched prompt made the model return nothing on every frame, so its 0.31 s p50 is the cost of generating nothing. The README keeps it as a labelled failure case.
With a JSON-schema grammar, out-of-vocabulary labels went from 59 to 0 on the 2B. On footage with no people, it then reported six persons and fired seven false PPE and proximity events.
Under the same grammar and prompt, the 8B reported zero persons and zero events on the same frames, at 3.20 s p50. Sixty frames of one video is a signal, not a rate.
With vLLM 0.14's reasoning parser on, a JSON schema is accepted but not enforced. That is now blocked in code: before any worker uses constrained mode, a canary request asks for one word while the schema allows only another, and the server has to answer with the schema's value. The probe is committed as reports/thor/constrained_guard_probe.json.
Constrained mode refused when starting the reasoning-parser container.
Constrained mode refused when saving settings with the reasoning parser on.
The reasoning-parser server's answer to the canary. The guard refused.
The container without the parser honoured the grammar and was allowed. Ready 55.6 s after launch.
60 Isaac Sim pictures in four conditions: a worker with a hard hat and vest, a worker with neither, a worker inside a restricted zone, and an empty scene. Measured on the Thor with the full build's labeled scorer; the summary reports are published in the open core under reports/thor/. Simulation, not real-world footage.
hard_hat and vest became required fields. All 15 real ones keptmodel_accuracy_ppe_v1_2026-10-17.jsonIt found the worker in 44 of 45 pictures against 29, found 14 of 15 zone entries against 9, and raised none of the false proximity alarms. Median answer time 1.83 s against 1.54 s.
Cosmos 3 Edge kept 13 of 15 real gear alarms and raised 3 false ones, where Reason 2 kept 15 and raised none on the people it did find. It also needed disable_any_whitespace in the server's grammar settings to answer at all.
Cosmos3-Nano-Sim-Transfer redrew the full build's access-gate test clips as daylight, dusk and night on the Thor. The gate's decision held at 15 of 15 in every version. The safety-glasses read fell from 45 to 27 of 45 by night.
The gate's reader scored 15 of 15 on five daylight test clips. Cosmos3-Nano-Sim-Transfer redrew the same clips as daylight, dusk and night on the Thor, keeping each worker's position and gear, and the gate's own check scored every version three times. Generated test clips and model redraws of them, not field evidence.
| Version | Mean brightness (0 to 255) | Open or denied right (of 15) | Let through without gear | Safety-glasses reads right (of 45) |
|---|---|---|---|---|
| Original, resized to 832 x 480 | 141 to 145 | 15, 15, 15 | 0 | 45 |
| Daylight redraw | 132 to 140 | 15, 15, 15 | 0 | 39 |
| Dusk | 84 to 91 | 15, 15, 15 | 0 | 38 |
| Night | 54 to 72 | 15, 15, 15 | 0 | 27 |
With Cosmos Reason 2 as the reader, open or denied was right on every sample in every lighting, and no worker without gear was let through. Hard hat, vest and boots were read correctly throughout.
The safety glasses, the smallest and faintest item, lost 18 of 45 reads by night. Part of that is the redraw itself: the daylight redraw already lost 6. Glasses only raise a warning at this gate.
Behind the same gate with nothing else changed, it denied every compliant worker in every version, the untouched originals included. The gate's question wordings were written and checked against Cosmos Reason 2.

The constrained Cosmos Reason 2 configuration repeated as three passes behind a warm-up, with the server image, model files, source commit and the board's other load recorded before each run.
The server returned 512 tokens cut off mid-string, and the worker fails a run on the first answer that breaks the output contract, by design. The 44 frames before it are the cost measurement; the 45th is the finding.
Screens from the running app, captured with its built-in synthetic workcell feed and the mock detector. The console labels both on screen so they are never mistaken for a real camera or a real model.
Each run writes one artifact with the device, date, inputs, per-frame timings, token use, event counts, peak memory and a 1 Hz tegrastats record.
A timed-out run, an empty-output run and a model echoing its own schema are all committed and labelled, not edited out.
Ten vendor profiles, a connection test that decodes one frame or names the problem, and zones drawn on a live snapshot.
Passwords are encrypted at rest, never returned by the API, only reused for the host they were saved for, and masked in logs, events and reports.
The model container keeps running across restarts. Stopping it needs an explicit, logged confirmation.
The system flags and records. It never stops a robot or a line; a person decides.
Out of scope by design: stopping machines or acting without a person. This is observability with a human in the loop.