End-to-end pipeline from specification to scored output. A
Task Specification (role, query, evaluation criteria, gold trajectory)
and a deterministic Environment Bundle (SLURM scheduler state, RBAC
policy, telemetry snapshots) are consumed by the BenchmarkRunner, which
drives a pluggable Agent Adapter through a bounded tool-call loop (at
most 10 rounds) against a role-filtered Tool Registry backed by five mock
HPC tool families. Each round is captured by the TraceWriter; the final
Trace is evaluated across six dimensions by the AggregateScorer using
profile default_hpc_v01 — a Governance hard-fail zeroes the full task score.
Per-task BenchmarkResult records feed the run-level
CLEAR scorecard (Efficacy, Assurance, Reliability, Cost, Latency) and
downstream report artifacts.
Benchmark comprises 80 tasks across 10 QCATs and 5 roles in 23 deterministic
environment snapshots (59 dev / 21 test).