Physical AI · Jetson · Sim-to-real evidence

Closing the gap between simulation and hardware constraints.

This case study presents the measurements behind the Physical AI program: what runs on Jetson hardware, what remains simulation evidence, what was corrected, and what is still not established.

Hardware Jetson AGX Thor 128GB and Jetson Orin NX

Dates 2026-08-20 measurements, 2026-09-07 corrections

Scope Inference, probes, simulation validation, correction ledger

// PROBLEM

A program has to survive contact with the machine.

The core problem was closing the gap between a simulator workflow and a system that runs on real edge hardware under timing, power, thermal, camera, and safety constraints.

The repo is not treated as proof by itself. The useful evidence is the committed record: device identity, runtime versions, power mode, timing distributions, thermal samples, simulation summaries, and corrections when a claim failed inspection.

// BUILT

What was built.

Measured: a Jetson AGX Thor path for GR00T eager and TensorRT inference benchmarking.
Measured: an Orin NX hardware probe path for CUDA fp16 matmul, V4L2 camera timing, thermal, and power readings.
Measured simulation: an Isaac Sim Ludo pick-and-place executor with provenance, per-attempt records, summaries, and correction sidecars.
Implemented: evidence charts generated from committed repo artifacts rather than notebook-only screenshots.

Digital twin workspace image — Digital twin workspace used to show the simulation context behind the case study.

Isaac Sim and OpenUSD scene image — Isaac Sim and OpenUSD scene context for the simulation side of the Physical AI workflow.

// MEASUREMENTS

Thor leads the evidence.

On 2026-08-20, Jetson AGX Thor 128GB in 120W mode ran 100 timed GR00T iterations after 10 warmups, BF16, batch 1. PyTorch eager measured 126.6 ms median end-to-end, or 7.9 Hz. The recovered TensorRT path measured 101.6 ms median end-to-end, or 9.8 Hz. The same run recorded 42.0 C GPU peak, 38.218 C sustained median, VDD_GPU peak 10,676 mW, VIN total peak 39,937 mW, and no observed throttling.

126.6 ms PyTorch eager median, AGX Thor, 2026-08-20

101.6 ms TensorRT median, same run conditions

9.8 Hz TensorRT throughput from median latency

42.0 C GPU thermal peak, no observed throttling

Thor latency chart showing PyTorch eager median 126.6 ms and TensorRT median 101.6 ms — Thor end-to-end latency, sourced from `reports/thor_trt_benchmark/thor_trt_benchmark.json`.

Thor component timing chart with data processing, backbone, and action head timings — Component timing from the same Thor benchmark. Data processing stayed 7.95 ms median; TensorRT reduced backbone and action-head time.

Thor thermal and recorded rail peak chart — Thermal samples and recorded rail peaks from the Thor benchmark artifacts.

Ludo placement chart comparing pre-fix and post-fix simulation groups — Simulation placement grouped by configuration: post-fix seeds 10-12 recorded 35/36 turn-level successes and 19.2 mm p50 error.

// CORRECTIONS

What went wrong and what was corrected.

The 128 ms / 7.8 Hz TensorRT attribution was retracted. It was a torch.compile baseline from a run where the TensorRT stage failed on a missing Matplotlib import. The corrected Thor TensorRT measurement is 101.6 ms median / 9.8 Hz, and numerical parity with PyTorch remains unrecorded.

The GR00T eval01 placement result was also corrected. A raw geometric success counted an untouched cup as successful. The corrected rule requires a grasp, making the result 0/20 corrected. The target was not observable to the policy, so the placement error is not used as a fair goal-conditioned comparison.

Correction chart showing old geometric rule and corrected grasp-required rule — Correction record from `reports/ludo_groot17_eval01/session_summary_corrected.json` and `docs/SYNRIA_EXPERIMENT_LEDGER.md`.

// DEMO VIDEO

The demo is simulation evidence.

The video shows the Ludo simulation workflow, not autonomous physical-arm proof. It belongs beside the simulation placement records, not beside the Thor hardware inference benchmark.

// LIMITS AND NEXT

What is not established.

Autonomous real-arm pick and place is not established.
TensorRT/PyTorch numerical parity is not recorded.
The Thor and Orin measurements are short runs, not sustained thermal or safety validation.
Physical object-success ground truth is not established for the command-derived hardware path.

Next work is a parity run with recorded tolerances, sustained thermal and safety validation, and a physical manipulation run with observed object-success ground truth.

Primary sources: Thor benchmark JSON, Thor thermal samples, Orin day-one smoke JSON, Ludo post-fix stats, GR00T correction sidecar, pipeline notes, Jetson deployment notes, and not-established claims.

Repository ↗ docs/evidence ↗ reports ↗