ROS 2 / K8S LAB
SCENARIO 02 · P2 / NETWORK RESEARCH

Edge vs Cloud AI Inference Placement for ROS 2

Which AI workload belongs on a robot, at a nearby edge service, or in Kubernetes? Compare feasibility filters with Jev's structured advice under unreliable networks.

Proposal · not benchmarkedMSc → PhDStart with synthetic inputs
CONCEPTUAL WORKFLOW / NOT LIVEROBOTICS · CLOUD · AI
SIMULATED / RESEARCH BLUEPRINTNO ACTUATION

See the idea in a minute

A robot has a sensor and a model to run. The model could run on a nearby edge box or in a faraway cloud. Press play or pick a step to see how a time budget decides. The picture is a sketch, and no timing has been measured.

Where should the model run?

Illustration · synthetic
A made-up comparison of running a model on an edge box or in the cloud A robot with a sensor sits on the left, an edge box beside it and a cloud far to the right. Two bars show how long each placement takes, drawn as lengths only with no numbers, against a dashed budget line. Step one: the network is good and both bars end before the budget line, so both are within budget. Step two: the network degrades, the cloud bar grows past the budget line and is marked over budget, while the edge bar stays short and within budget. Step three: a placement rule chooses the edge box. Step four: the loop is reported as a measurement to collect later. The picture is illustrative and no measured latency exists. SensorRobot Edge boxnearby Cloudfar away Chosen: edge Network: good Network: degraded Time budget · bar length only, no numbers Edge within budget within budget · chosen Cloud within budget over budget over budget · not chosen Budget limit To collect latermeasured timings
Step 1 of 4

Illustrative only. Bar lengths are invented and carry no units. No latency has been measured, and no real robot, camera, model or network is involved. The edge/cloud study is a proposal.

Plain-language guide

Edge and cloud
The edge box is a computer close to the robot. The cloud is a faraway service, here a Kubernetes cluster or a hosted model service, reached over a network that can slow down or drop.
Latency budget
The longest time a result may take to come back and still be useful. A placement that goes past the budget is over budget.
Hard gate
An ordinary code check that removes placements that cannot work, such as ones that are unreachable, too slow or not allowed, before any ranking happens.
Jev
A structured decision model this research proposes to try as an optional adviser. It would only rank placements that already pass the gate. Its answers are advice, not commands.
01 / What are we studying?

One real research question, not a product claim.

A fictional camera-equipped robot needs to run a task-compatible perception policy. Local CPU and remote hosted inference have different cost and network risks. A deterministic gate removes impossible or stale placements before any model comparison.

Research hypothesis

Question: Does Jev's preference among already eligible placements improve simulated task outcomes over deterministic heuristics once action freshness and network overhead are measured?

Null hypothesis: Under the same hard constraints and fixed workload, Jev advisory selection yields no net outcome gain after hosted latency, outage and cost.

Alternative: A bounded soft preference helps choose among equally eligible locations in a reproducible simulation; no hard real-time claim.

Audience & learning targets

Edge AI · systems researchers · Kubernetes · MSc → PhD · Start with synthetic inputs

  • Separate compute placement policy from hard real-time robot control.
  • Apply numeric deadlines, data residency and memory bounds in code.
  • Design repeatable RTT, jitter, loss, disconnect and recovery profiles.
  • Collect action age, availability and cost without invented benchmarks.
Interactive explanation

Step through one placement decision and see what gets rejected

A robot request has a deadline. Plain code decides whether the model runs locally or on a remote service, and a non-actuating sink then accepts or rejects each answer. The five cases follow the logic of the deterministic simulator in experiments/edge-cloud-placement. The numbers are small declared inputs chosen for teaching. They are not simulator output and not measurements.

  • Status: simulated
  • Storyboard: illustrative, hand-authored
  • Measured results: N/A (not run)
  • Registry status: planned, nothing measured
  1. dataRequest
  2. plain codeEligibility
  3. plain codePlacement policy
  4. advisory only, optionalAdvisory model
  5. plain codeNon-actuating sink

Predicted fit: remote

The remote service is eligible and the predicted round trip fits inside the deadline, so the deadline-aware policy sends the request there.

Declared inputs (made up for teaching, not measured)
policydeadline_aware
deadline_ms300 ms
remote model_memory_mb8192 MB
remote free memory16384 MB
predicted round trip (EWMA)100 ms
margin_ms20 ms
predicted plus margin (by hand)120 ms
remote answer arrives at130 ms
  1. A request arrives with a deadline info

    The request has a 300 ms budget, counted from capture.

    deadline_ms: 300
  2. The remote service is eligible passed

    16384 MB are free and the model needs 8192 MB, so the remote can take it, as far as the policy knows.

    remote: 16384 MB free, needs 8192 MB
  3. The prediction fits the budget passed

    Predicted round trip 100 ms plus a 20 ms margin is 120 ms, which is within 300 ms. The route is remote, reason predicted_fit.

    route: remote (predicted_fit)
  4. Optional advisory model: not run N/A (not run)

    It is not part of the simulator. Advice could not make an ineligible remote eligible.

  5. The sink accepts the first answer passed

    The answer arrives at 130 ms, before the deadline, and nobody has answered yet. The sink records would_apply with actuated: false.

    decision: would_apply
    actuated: false

Outcome: accepted Remote placement accepted by plain code. No action was applied to anything: the sink only records would_apply.

Predicted miss: local

Earlier timeouts raised the round-trip estimate. It no longer fits the deadline, and no probe is due, so the policy keeps the request on the robot.

Declared inputs (made up for teaching, not measured)
policydeadline_aware
deadline_ms300 ms
remoteeligible
predicted round trip (EWMA)320 ms (raised by earlier timeouts)
margin_ms20 ms
predicted plus margin (by hand)340 ms
time since last remote send400 ms
probe_interval_ms1000 ms
local compute150 ms
  1. A request arrives with a deadline info

    The budget is 300 ms.

    deadline_ms: 300
  2. The remote is eligible passed

    Eligibility alone does not send the request there.

  3. The remote does not fit, so it is rejected rejected

    Predicted 320 ms plus a 20 ms margin is 340 ms, above 300 ms. The last remote send was 400 ms ago, so the 1000 ms probe interval has not passed and no probe is sent. The route is local, reason predicted_miss.

    route: local (predicted_miss)
  4. Optional advisory model: not run N/A (not run)

    The policy decided without it.

  5. The sink accepts the local answer passed

    Local compute takes 150 ms, inside the deadline. Decision: would_apply.

    decision: would_apply

Outcome: accepted Local placement accepted. The estimate can recover later because a probe is allowed once the probe interval has passed.

Remote says no: explicit reject

The policy's view of the remote is out of date. The remote refuses because it has too little memory, and the hedged policy falls back to the robot.

Declared inputs (made up for teaching, not measured)
policyhedged_fallback
deadline_ms300 ms
remote model_memory_mb8192 MB
remote free memory before the drop16384 MB
remote free memory now4096 MB
memory dropped300 ms ago
advert_lag_ms500 ms
remote reject arrives at70 ms
local answer arrives at230 ms
  1. A request arrives with a deadline info

    The budget is 300 ms.

    deadline_ms: 300
  2. The remote refuses: not enough memory rejected

    Memory dropped 300 ms ago, but the capability advertisement lags by 500 ms, so the policy still saw 16384 MB. The remote has 4096 MB and needs 8192 MB, so it rejects at 70 ms with ineligible_memory.

    remote reply: reject
    reason: ineligible_memory
  3. The hedged policy launches the local fallback passed

    After the explicit reject the policy starts the local copy at once and holds the remote back for the probe interval.

    route: local (fallback)
  4. Optional advisory model: not run N/A (not run)

    It cannot add free memory to the remote.

  5. The sink accepts the fallback answer passed

    The local answer arrives at 230 ms, before the 300 ms deadline. Decision: would_apply, marked as a fallback.

    decision: would_apply
    fallback: true

Outcome: rejected The remote placement was rejected for memory. The request still met its deadline through the local fallback.

Duplicate answer: second one rejected

Both the remote and the local fallback answer. The first accepted answer wins, and the second is refused even though it arrived before the deadline.

Declared inputs (made up for teaching, not measured)
policyhedged_fallback
deadline_ms300 ms
hedge timer fires at110 ms
local answer arrives at270 ms
remote answer arrives at285 ms
  1. A request arrives with a deadline info

    The budget is 300 ms.

    deadline_ms: 300
  2. The remote is tried, and a hedge is armed passed

    The remote has not answered when the hedge timer fires at 110 ms, so the local fallback starts in parallel.

    hedge: local fallback at 110 ms
  3. Optional advisory model: not run N/A (not run)

    The timer, not a model, decides when to hedge.

  4. The local answer is accepted first passed

    It arrives at 270 ms, within the deadline. Decision: would_apply.

    decision: would_apply
  5. The remote answer is rejected as a duplicate rejected

    It arrives at 285 ms, also within the deadline, but this request is already applied. The sink checks for duplicates before lateness.

    decision: rejected
    reason: duplicate_response_id

Outcome: rejected One action is applied per request. The slower copy is rejected with duplicate_response_id.

Local-only request: remote answer rejected

The request must stay on the robot. The remote is never asked, and an unexpected remote answer is rejected by the sink.

Declared inputs (made up for teaching, not measured)
request local_onlytrue
deadline_ms300 ms
local compute150 ms
unexpected remote answer arrives at100 ms
  1. A request that must stay local arrives info

    The local_only flag is set. The deadline is 300 ms.

    local_only: true
  2. The remote is ruled out up front rejected

    A local_only request never considers the remote. No remote request is sent.

    remote: not considered
  3. The route is local passed

    Reason: local_only.

    route: local (local_only)
  4. Optional advisory model: not run N/A (not run)

    Advice cannot override local_only.

  5. A stray remote answer is rejected rejected

    An unexpected answer from the remote arrives at 100 ms. The sink rejects it with ineligible_source before the local answer arrives.

    decision: rejected
    reason: ineligible_source
  6. The local answer is accepted passed

    Local compute takes 150 ms, within the 300 ms deadline. Decision: would_apply.

    decision: would_apply

Outcome: rejected The remote source was rejected for a local_only request. The local answer was accepted.

What gets rejected, and why

Each row is a negative case in the storyboard above. Reason names are proposals for this study, not an implemented interface, unless the simulator source below defines them.

Rejections in the storyboard
CaseStopped atReasonWhat it guards against
Predicted miss: localPlacement policypredicted_miss: the remote does not fit the deadline.A remote answer that would probably be late.
Remote says no: explicit rejectEligibilityineligible_memory: reported by the remote.A remote with too little memory for the model.
Duplicate answer: second one rejectedSinkduplicate_response_idA second answer for an already-applied request.
Local-only request: remote answer rejectedEligibility and sinklocal_only, then ineligible_sourceAny remote involvement in a request that must stay local.

Experimental protocol

  1. Fix the request stream, the deadlines and the network profile in a config file before running anything. Every parameter there is declared, not measured.
  2. Run all policies on the same seeded request stream, so differences come from the policy.
  3. Compare always_local, always_remote, deadline_aware and hedged_fallback on deadline misses and on the cause of each miss.
  4. Treat every number the simulator prints as SIMULATED. Real-network or real-robot evidence needs a separate, reviewed run.

The simulator never actuates anything: its sink only records would_apply or rejected. There are two sites only, the robot and one remote service, and no hold action: a request with no accepted answer is a deadline miss. Do not assume the remote wins. Under the simulator's placeholder parameters, running locally is not worse than always going remote, and other parameters could change that ordering. The registry primary metric (sensor-to-valid-action p95) has no measured value. See also the experiment blueprint on this page.

Measured values

Every value is N/A until a reviewed run exists
MetricValueWhy
Sensor-to-valid-action p95 (registry primary metric)N/ANo live measurement. Simulated values are not results.
Deadline-miss rate on a real networkN/ANot run.
Local versus remote energyN/ANot measured.

See the evidence registry. Nothing on this page is a benchmark.

Where the real code and tests live

Paths are relative to the repository root and the links open the main branch on GitHub. A link resolves once the change that adds the path has been merged.

Illustration only. The steps are fixed text written by hand. This page runs no model, sends no network request, publishes no ROS 2 message and moves no robot. Use the Pause decorative motion button near the top of the page to stop all motion; the player never starts by itself.

02 / Interactive walkthrough

Follow an illustrative decision trace.

Select a fabricated case and trace where conventional code, an optional Jev judgment, and human review belong. This animation is preauthored educational content: it performs no inference, no API call, and no robot action.

SYNTHETIC ONLY
A fictional camera-equipped robot needs to run a task-compatible perception policy. Local CPU and remote hosted inference have different cost and network risks. A deterministic gate removes impossible or stale placements before any model comparison.

Illustrative stage 1 of 4: Synthetic observation + resource profile

Visual explanation only; Jev probabilities and experimental outcomes are intentionally absent.

Keyboard users: select a case or activate any stage button. The animation can be paused in the header, and system reduced-motion preferences are supported.

03 / Experiment blueprint

Variables, baselines and measurements.

Before anyone reports performance, the task, failure cases, comparison methods and test split must be fixed. Keep negative results and rejected advice visible.

Research dimensionProposed measurement contract
PlacementsLocal process, same-LAN remote policy, and Kubernetes remote policy after verified DDS/network prerequisites.
BaselineDeterministic hard constraints and latency/cost-aware heuristic; explicitly define an oracle for retrospective regret.
VariablesRTT, jitter, packet loss, disconnection duration, model service queue and approved data locality.
Failure casesLossy link, stale action chunk, provider timeout, remote policy unavailable, infeasible accelerator memory.
Primary metricsp50/p95/p99 sensor-to-valid-action age, missed deadlines, reject counts, availability and eligible-placement regret.
ReproducibilitySource SHA, simulator/model version, hardware/clock, network profile seed, placement eligibility snapshot and failures.

Suggested experiment procedure

  1. Describe a specific compatible simulator, model and task; otherwise remain on synthetic decision examples.
  2. Implement deterministic eligibility and establish a no-JeV local baseline with unit tests.
  3. Validate network fault injection and timestamp assumptions before collecting measurements.
  4. Compare placement advisories under matched workloads and fixed budgets, including outages.
  5. Publish raw timing/event records and explain what cannot be generalized to physical robotics.

What a contribution should produce

  • Synthetic placement matrices with eligible/ineligible candidates
  • Latency/network impairment run protocol
  • Immutable inference request and eligibility logs
  • p95/action-age/failure plots with raw runs (future)

Required before publishing a result: independent run ID, method, seed, source/model revisions, environment, raw measurements, executable analysis, errors/limitations and human review.

04 / Evidence not marketing

Current results: not measured.

No verified experiment results yet

No Jev-vs-baseline accuracy, latency, energy, fairness or simulator task-success result is available for this scenario. The website does not fabricate graphs or sample performance numbers. Future evidence must include raw data and replay instructions.

Open results registry ↗

Dependencies and research honesty

Follow existing research work #62 and the broader scenario epic #83. The legacy Vagrant/Foxy manifests and current Jazzy image-local test do not establish multi-worker DDS or physical robot integration.

Safety: no actuator commands, unrestricted cluster operations, or vendor data collection from this static demonstration.