Pressure
Agents generate faster than anyone can review
Generation capacity is growing faster than trustworthy review capacity. IDKMesh treats verification debt and evidence quality as first-class constraints rather than an afterthought.
What IDKMesh is
An open-source research and engineering project asking whether a large open community of humans and AI agents can discover goals, decompose work, execute bounded tasks, verify results independently, and maintain complex systems better than isolated developers or agents can. It treats that as an empirical question.
Goal → bounded Work Unit → isolated attempts → independent verification → evidence → explicit integration decision.
The project deliberately separates generation from acceptance. A successful worker, a popular proposal, or a large swarm does not automatically become canonical truth. See the six ideas that follow from that →
Capability truth: what works today?
This compact projection is generated from the repository's canonical capability truth JSON, not maintained as a second claim list.
9 implemented · 6 experimental · 5 planned. Highest evidence level currently represented: 3/5.
Try it: audit a review gate
The first installable tool cut from this research measures what a panel of verifiers is actually worth. The finding it packages is an observed one: twenty-five independently seeded test oracles, run over a real 72-candidate corpus, were measured to be worth about one — and the standard N/(1+(N−1)ρ) correction overstated even that by 1.66×. Growing the panel bought nothing; changing how the votes were combined cut error 3.7×. Source: E017.
git clone https://github.com/MSKazemi/idkmesh && cd idkmesh
pip install .
idkmesh gate-audit examples/gate-audit/panel-votes.example.json --prettyWhat that command prints is not the E017 measurement. The bundled example is a small hand-authored input that labels itself "evidence_class": "synthetic" in its own output: fifteen candidates, five invented reviewers, 1.69 effective votes from a nominal five against a heuristic 3.66, and two of three seeded known-bad probes accepted. It shows you the shape of the answer. Point the tool at verdicts you actually collected to get an observed one. The contract is Gate Audit v0.1, and the audit is diagnostic only — it grants no acceptance or merge authority. Full quickstart, with the output explained →
What has actually been measured
Every number below is an observed measurement from E017, produced by code in this repository that you can re-run. A verifier there is a partial test oracle — a program, not a person and not a language model — and ground truth comes from executing hidden tests. Most results elsewhere on this site are synthetic mechanism tests, and are labelled as such.
1.66×
how much the standard
N/(1+(N−1)ρ) correction overstated that same panel — it said 1.66 effective verifiers where the measurement said 1.00.0.587
mean pairwise error correlation across 300 pairs of 25 independently seeded test oracles — programs, graded on real defects (E017). Not a knob set by hand.
1.00
effective panel size of those 25 oracles under majority vote — panel error 0.2083 against a single verifier's 0.2044. Growing the panel bought nothing (E017).
3.7×
error reduction from fixing the aggregation rule instead — 0.2083 → 0.0556, with an irreducible floor no quorum reaches (E017).
Read this honestly
Implemented infrastructure is evidence of a capability to run experiments — not evidence that the research hypotheses are true. Most results on this site are synthetic mechanism tests, clearly labelled as such, and the negative ones are kept. The full experiment atlas, with each result's status →
Why now
Failure mode
Different agents fail together
More votes are not automatically more independent evidence. The project measures verifier reliability, dependence, and failure correlation — and has found the standard correction optimistic exactly where it matters.
Method
The repository is the laboratory
Bounded work, public evidence, reproducible decisions, and community growth measured without Goodharting activity counts. Negative results stay in the tree.
What do you want to do?
Start with the smallest surface that matches your goal. You can use the shipped diagnostic without understanding the research stack, and you can contribute without learning every subsystem first.
Use it now
Audit a review gate
Install the dependency-free CLI, run the bundled example, then point gate-audit at verdicts your human or AI reviewers already produced.
Start in 15 minutes →
Understand it
Learn the trust model visually
See bounded Work Units, verifier independence, verification debt, evidence, and integration authority before reading the deeper architecture.
How IDKMesh works →
Research it
Reproduce or challenge the evidence
Open the experiment atlas, follow results back to their retained records, rerun the code, and try to falsify the claims that matter.
Explore the evidence →
Build it
Take a bounded open task
Code, tests, review, documentation, security, research, and community work all count. Check assignees and recent comments before claiming a task.
Browse good first issues ↗
What exists, and what does not
The project keeps these two lists apart on purpose. Both are maintained in the repository README.
On main today
- versioned WorkUnit contracts, with
work-unit-v0.2as the current semantic task contract; - ResultManifest, EvaluatorPlan and VerificationResult contracts that separate worker claims, verifier evidence and integration authority;
- cross-object provenance and integrity validation;
- protocol-neutral worker adapters plus A2A/MCP bindings and conformance helpers;
- zero-project-spend compute admission and routing experiments;
- IDKGraph repository modelling, observability and link-integrity machinery;
- an installable
idkmeshCLI whosegate-auditcommand packages the measured verifier-panel results as a review-gate diagnostic; - protected
mainwith the PR gate required on Python 3.11 and 3.13.
Not yet a finished capability
- no claim that IDKMesh can safely coordinate thousands or millions of real machines;
- the reference Verified Swarm Runner is not a polished install-and-run product with multiple production worker adapters;
- canonical real-node integration remains behind its independent-review and evidence gates;
- A2A/MCP support is an interoperability layer, not a claim that every external agent framework is production-integrated;
- autonomous repository and community actuation stays policy- and authority-gated;
- benchmark infrastructure is not scientific proof until controlled observed runs exist.
AI agent verification and orchestration topics
Ten substantial guides cover the questions most people ask about trustworthy agentic systems. They connect search-friendly explanations to the repository's canonical contracts, experiments, and evidence.
AI agent verification
Validate agent output with bounded tasks, independent evaluation, provenance, and explicit acceptance authority.
Multi-agent orchestration
Coordinate heterogeneous agents through decomposition, capability routing, isolated candidates, and verification backpressure.
AI code review
Use coding agents and AI pull-request review without collapsing generation, CI evidence, review, and merge authority.
LLM-as-a-judge reliability
Reason about evaluator calibration, bias, stability, correlated errors, and what a judge is actually allowed to decide.
Verifier panels
Understand effective independent votes, correlated reviewer errors, quorums, and reliable review gates.
AI agent governance
Place human approval, scoped permissions, audit evidence, and integration authority at meaningful boundaries.
AI provenance and evidence
Bind agent claims to exact artifacts, identities, evaluations, and reproducible checks.
MCP and A2A interoperability
See how MCP, Agent2Agent, connectors, work contracts, and evidence layers fit together.
Verification debt and scaling
Control the review bottleneck when autonomous generation grows faster than trustworthy verification capacity.
Verified swarm engineering
Build Git-native collaborative agent systems where useful work earns trust through evidence rather than activity volume.
Explore
Explained simply
How it works
Six animated figures: bounded work, correlated verifiers, effective votes, the authority ladder, verification debt, and the evidence ladder.
Read →
Architecture
Four pipelines
Animated flowcharts of the work/evidence path, the two-attempt orchestrator, zero-spend compute admission, and the PR gate — read off the executable code.
Read →
Evidence
Experiment atlas
Every experiment from E011 to E042 with its real status — positive, negative, or prototype — and what it changed.
Read →
Everything
Documentation library
The full material index: contracts and schemas, architecture, specifications, research, findings, audits, governance, and project memory.
Browse →
The operating principle
Optimise verified useful work per unit of scarce human attention and compute — not commits, agent count, stars, or activity volume.
A proposal is not true because it came from a strong model, many models, an expert human, or a popular repository. Evidence must survive independent scrutiny.
IDKMesh is intentionally unfinished. The repository is the public laboratory, and well-supported disagreement is a contribution. If you can improve a question, falsify an assumption, reproduce an experiment, write a test, find a security problem, make a contract clearer, or reduce reviewer burden, you can contribute.