ROS 2 Jev as an Advisory Reviewer for ROS 2 Agent Skills
An LLM proposes a robot skill. Can a structured decision model provide useful extra review without bypassing deterministic permissions and simulator safety checks?
See the idea in a minute
An AI agent can suggest robot skills, but fixed rules decide which ones may run. Watch three made-up suggestions reach the policy gate. Nothing here runs a model or moves a robot.
An AI proposes, a policy decides
Illustration · syntheticJev, the optional second opinion this proposal would test, is not drawn here. It could only add advice. It could never override the policy. This is a research proposal, not a measured result.
Plain-language guide
- Skill proposal: one high-level simulator action, such as
get_status, that the language model suggests. The model suggests it and does not run it. - Deterministic policy: fixed rules, such as a schema check, an allowlist and parameter limits, that give the same answer every time. It stays authoritative.
- Jev: a structured decision model that this proposal would test as an optional second opinion. It advises and never grants permission.
- Simulator only: the study stays in simulation. This site runs no model and sends no command to a robot.
One real research question, not a product claim.
A fictional operator requests a high-level simulator action. The planner proposes get_status, inspect_scene or an unsupported privileged command. An independent validator must reject forbidden skills even if a model claims they are fine.
Research hypothesis
Question: Does an additional Jev advisory screening stage detect useful problems in high-level simulated ROS 2 agent plans beyond an independent typed validator?
Null hypothesis: A second model adds cost, false positives or inconsistent judgments without increasing detection beyond deterministic enforcement.
Alternative: Jev detects additional semantic problems and can improve human review without becoming an authorization gate.
Audience & learning targets
Agentic AI · robotics safety · LLM research · MSc → PhD · Synthetic fixtures only
- Design typed, allowlisted high-level skills instead of free-form robot commands.
- Separate language-model opinions from access control and schema enforcement.
- Test injection in untrusted logs and retrieved context.
- Measure false-negative detection, review effort and abstention.
Follow an illustrative decision trace.
Select a fabricated case and trace where conventional code, an optional Jev judgment, and human review belong. This animation is preauthored educational content: it performs no inference, no API call, and no robot action.
Illustrative stage 1 of 4: Synthetic operator goal and proposed skill
Visual explanation only; Jev probabilities and experimental outcomes are intentionally absent.
Keyboard users: select a case or activate any stage button. The animation can be paused in the header, and system reduced-motion preferences are supported.
Variables, baselines and measurements.
Before anyone reports performance, the task, failure cases, comparison methods and test split must be fixed. Keep negative results and rejected advice visible.
| Research dimension | Proposed measurement contract |
|---|---|
| Skills | Read-only inspect_scene/get_status; separately approved simulator navigation only with valid named waypoint. |
| Baseline | JSON Schema, capability/RBAC allowlist, parameter bounds and human approval; model-free enforcement. |
| Variables | Goal ambiguity, hallucinated tool names, adversarial RAG/log strings, missing robot/task state and Jev screening criteria. |
| Failure cases | Raw /cmd_vel, arbitrary shell/kubectl, unknown waypoint, stale scene, model overconfidence or provider outage. |
| Primary metrics | Recall on deliberately unauthorized test proposals, semantic error detection, reviewer load, abstentions, false alarms. |
| Reproducibility | Versioned high-level skill catalog, synthetic task IDs, fixture hashes, threat model and adjudicated reviewer labels. |
Suggested experiment procedure
- Define a tiny high-level simulator skill vocabulary with JSON Schema and explicit execution scope.
- Write synthetic valid, ambiguous and unauthorized plan fixtures with known policy outcomes.
- Prove the deterministic gate blocks all enumerated forbidden fixtures regardless of model responses.
- Optionally compare Jev, rules and a constrained-output LLM on semantic error screening.
- Report both false positive and false negative impacts; include human review time if ethically approved.
What a contribution should produce
- Versioned synthetic high-level agent-proposal casebook
- Independent validator tests including prompt-injection attempts
- Threat model, reviewer worksheet and confusion matrix
- Verified experiment manifest and human study approval (if later needed)
Required before publishing a result: independent run ID, method, seed, source/model revisions, environment, raw measurements, executable analysis, errors/limitations and human review.
Current results: not measured.
No verified experiment results yet
No Jev-vs-baseline accuracy, latency, energy, fairness or simulator task-success result is available for this scenario. The website does not fabricate graphs or sample performance numbers. Future evidence must include raw data and replay instructions.
Open results registry ↗Dependencies and research honesty
Follow existing research work #64 and the broader scenario epic #83. The legacy Vagrant/Foxy manifests and current Jazzy image-local test do not establish multi-worker DDS or physical robot integration.
Safety: no actuator commands, unrestricted cluster operations, or vendor data collection from this static demonstration.
Help turn the blueprint into evidence.
Join scenario 03
Contributors can own a small, reviewable task: casebook, synthetic dataset, offline baseline, simulator test, reproducibility review or teaching guide. The issue includes a detailed Claude Code/Codex prompt, acceptance criteria and dependencies.
Contribute on GitHub issue #86 ↗Technical references
Review Jev research umbrella #73, scenario epic #83 and the repository agent instructions. No vendor endorsement or measured success is implied.
Continue exploring
Fault triage
A listener stops receiving messages. Is the cause DDS discovery, QoS, networking or resource pressure? Build a transparent diagnostic comparison without touching a robot.
Explore Fault triage SCENARIO 02Edge/cloud placement
Which AI workload belongs on a robot, at a nearby edge service, or in Kubernetes? Compare feasibility filters with Jev's structured advice under unreliable networks.
Explore Edge/cloud placement SCENARIO 04Multi-robot scheduling
Three simulated robots have different capabilities, reservations and energy budgets. Who should get the next task—and how can we measure fairness?
Explore Multi-robot scheduling