ROS 2 Experiments & reproducible results.
A public registry for ROS 2, Kubernetes, Jev, agentic AI and edge/cloud robotics comparisons. Research questions are real; performance results are published only when their raw evidence can be reviewed.
What has actually been measured?
At initial publication, no scenario has a verified Jev run or robotics performance result. Performance cells deliberately show N/A rather than estimates. The registry can be updated through reviewed pull requests once the experiment protocol and evidence requirements are met.
| Experiment | Primary endpoint to measure | Evidence status | Result | Next action |
|---|---|---|---|---|
| ROS 2 DDS/QoS fault triage Research issue #84 | Macro-F1 and coverage/abstention | Not measured | N/A | Join experimentInspect evidence |
| Edge/cloud inference placement Research issue #85 | Sensor-to-valid-action p95 | Not measured | N/A | Join experimentInspect evidence |
| LLM simulator skill proposal screening Research issue #86 | Unauthorized proposal recall | Not measured | N/A | Join experimentInspect evidence |
| Fair multi-robot simulated dispatch Research issue #87 | Simulated task completion and fairness | Not measured | N/A | Join experimentInspect evidence |
Client-side enhancement reads data/experiments.json from this static site. Without JavaScript, all rows remain visibly labelled "Not measured" by default. Changes to verified status require a provenance-complete artifact and maintainer review.
What a result submission must contain.
Frozen experiment
Task and null hypothesis, exact dataset/split and license, source and container commits, model/revision and question criteria hash, hardware/clock and network profile.
Raw and failed outcomes
Run IDs, seeds, repeats, all valid/invalid/unknown predictions, provider outages, latency with units, actual measurement source and cost assumptions.
Independent review
Raw shareable files, SHA-256 hashes, scripts and exact replay commands, calculated uncertainty and limitations, human-reviewed evidence URL.
Status rules
- Planned: question and methodology exist; measured values must remain null.
- Collecting: trial evidence is under review; measured values remain undisplayed.
- Verified: public evidence, immutable run manifest hash and numerical metric exist and were independently reviewed.
Research safeguards
Use synthetic or licensed data; never commit secret tokens, private ROS telemetry, real robot addresses or unapproved human recordings. Independent deterministic safety controls and simulator-only boundaries are mandatory. Neither TypeSafe nor this website certifies a robot controller.
Publishing a negative finding is valuable too.
Propose a reproducible experiment
Before submitting claims, read the scenario, preregister metrics and create a small issue/PR that includes scripts, synthetic or licensed raw data, failures, and an evidence manifest. See the dedicated results issue and Claude Code/Codex instructions.
Evidence workflow issue #88 ↗Next: Browse scenarios and select one study to extend with independently reproducible evidence.