Mohsen Seyedkazemi Ardebili

Agent Operations Benchmark — a trace-driven, role-aware, RBAC-enforced benchmark for LLM agents that operate HPC systems. It asks the blunt question: can an autonomous agent be trusted to run a supercomputer? — and scores the answer against real operator roles and hard policy constraints.

• 88 tasks across 10 question categories × 5 operator roles
• 29 deterministic environment bundles — 23 synthetic, 6 built from real Marconi100 telemetry
• Permission-enforced: a policy violation hard-fails the task, whatever the answer said
• 12 scorers over 6 dimensions, rolled into a CLEAR scorecard (Cost · Latency · Efficacy · Assurance · Reliability)
• 16 model systems evaluated on the 59-task dev split, every headline number tied to a frozen run ID

Verified swarm engineering — an open research laboratory asking whether humans and AI agents can turn uncertain goals into trustworthy software by separating generation from acceptance. Its first installable tool, idkmesh gate-audit, measures what a review panel is actually worth: effective independent votes rather than head-count, error-correlation structure, and the breach rate of seeded known-bad candidates.

• E017 measured 25 independently seeded test oracles — programs, over a 72-candidate corpus with ground truth from executing hidden tests — at an effective panel size of 1.00
• The standard N/(1+(N−1)ρ) heuristic predicted 1.66, so it overstates exactly where it is relied on
• Changing the aggregation rule cut error 3.7×; growing the panel bought nothing
• E016 is retained as a negative result: 20 live LLM verifiers did not verify, so no AI review panel has been measured here
• Negative results and failed experiments are kept, not hidden — evidence must survive independent scrutiny

Graph anomaly-anticipation for exascale HPC — topology-aware node-failure prediction running in production on CINECA's Tier-0 Marconi100, published in FGCS.

• Models the machine as a graph and predicts, per node, the probability of an anomaly in a future window
• Benchmarked by AUC against per-node DNN, gradient-boosting, random-forest and decision-tree baselines
• Trained offline, then served online as a Kubeflow pipeline reading live ExaMon telemetry over MQTT
• The paper reports negligible added overhead versus running the monitoring system alone

Thermal-hazard prediction for datacenters — multi-modal deep learning forecasting thermal failures fast enough to act, with explanations operators trust.

• A year of inlet/outlet temperature and power telemetry from 3,312 nodes of CINECA's Marconi A2
• Temporal Convolutional Network, LSTM and SVM models over a rolling time window
• Six-hour prediction horizon — chosen with the facility manager, not with a grid search
• Training data published openly on Zenodo so the result can be reproduced

An end-to-end MLOps platform for HPC workload management, built for the EuroHPC SEANERGYS project and now running in production at LuxProvide on the MeluXina supercomputer. Architect, lead designer and main developer; WP3 task lead at Bologna. It is model-agnostic and multi-tenant — any of the sixteen consortium partners registers a model and the platform auto-discovers and operationalises it (train → version → govern → serve → monitor) without ever owning the model code. The whole loop, on a real supercomputer, behind an operator approval gate.

• Auto-discovery training pipelines in Prefect: every registered model × dataset pair runs train → evaluate → log → promote
• Slurm adapter for HPC job orchestration; a DataPlane bridge carries messages off the machine
• MLflow registry with a multi-stage lifecycle and a YAML model registry with per-environment overlays
• Ray Serve multi-model serving with batched inference; MinIO for artifacts
• A sysadmin approval gate stands between a promoted model and the serving fleet — nothing goes live unattended
• Prometheus, Grafana and Loki for metrics and logs; React 19 + FastAPI dashboard; exa operator CLI

The operator's companion for KubeIntellect — a CLI and Python SDK (kq) exposing the full agent API with streaming output and a human-approval UX, built for CI/CD.

• Full KubeIntellect API coverage from the terminal
• Streaming Rich TUI · pipeline-friendly output