Pressure
Agents generate faster than anyone can review
Generation capacity is growing faster than trustworthy review capacity. IDKMesh treats verification debt and evidence quality as first-class constraints rather than an afterthought.
What IDKMesh is
An open-source research and engineering project asking whether a large open community of humans and AI agents can discover goals, decompose work, execute bounded tasks, verify results independently, and maintain complex systems better than isolated developers or agents can. It treats that as an empirical question.
Goal → bounded Work Unit → isolated attempts → independent verification → evidence → explicit integration decision.
The project deliberately separates generation from acceptance. A successful worker, a popular proposal, or a large swarm does not automatically become canonical truth. See the six ideas that follow from that →
Try it: audit a review gate
The first installable tool cut from this research measures what a panel of verifiers is actually worth. The finding it packages is an observed one: twenty-five independently seeded test oracles, run over a real 72-candidate corpus, were measured to be worth about one — and the standard N/(1+(N−1)ρ) correction overstated even that by 1.66×. Growing the panel bought nothing; changing how the votes were combined cut error 3.7×. Source: E017.
git clone https://github.com/MSKazemi/idkmesh && cd idkmesh
pip install .
idkmesh gate-audit examples/gate-audit/panel-votes.example.json --prettyWhat that command prints is not the E017 measurement. The bundled example is a small hand-authored input that labels itself "evidence_class": "synthetic" in its own output: fifteen candidates, five invented reviewers, 1.69 effective votes from a nominal five against a heuristic 3.66, and two of three seeded known-bad probes accepted. It shows you the shape of the answer. Point the tool at verdicts you actually collected to get an observed one. The contract is Gate Audit v0.1, and the audit is diagnostic only — it grants no acceptance or merge authority. Full quickstart, with the output explained →
What has actually been measured
Every number below is an observed measurement from E017, produced by code in this repository that you can re-run. A verifier there is a partial test oracle — a program, not a person and not a language model — and ground truth comes from executing hidden tests. Most results elsewhere on this site are synthetic mechanism tests, and are labelled as such.
1.66×
how much the standard
N/(1+(N−1)ρ) correction overstated that same panel — it said 1.66 effective verifiers where the measurement said 1.00.0.587
mean pairwise error correlation across 300 pairs of 25 independently seeded test oracles — programs, graded on real defects (E017). Not a knob set by hand.
1.00
effective panel size of those 25 oracles under majority vote — panel error 0.2083 against a single verifier's 0.2044. Growing the panel bought nothing (E017).
3.7×
error reduction from fixing the aggregation rule instead — 0.2083 → 0.0556, with an irreducible floor no quorum reaches (E017).
Read this honestly
Implemented infrastructure is evidence of a capability to run experiments — not evidence that the research hypotheses are true. Most results on this site are synthetic mechanism tests, clearly labelled as such, and the negative ones are kept. The full experiment atlas, with each result's status →
Why now
Failure mode
Different agents fail together
More votes are not automatically more independent evidence. The project measures verifier reliability, dependence, and failure correlation — and has found the standard correction optimistic exactly where it matters.
Method
The repository is the laboratory
Bounded work, public evidence, reproducible decisions, and community growth measured without Goodharting activity counts. Negative results stay in the tree.
Pick a path
You do not need to understand the whole repository to do something useful in it. Check a task's assignees and recent comments before claiming it.
15 minutes
Run the first installable tool
Clone, pip install ., and audit a review gate — effective votes, correlation structure, seeded-probe breach rate.
Quickstart →
Build or research
Take a bounded open task
Live good first issue and help wanted queues. Confusion you hit on the way in is itself useful evidence.
Browse live issues ↗
Expert reviewer
Try to falsify one claim
Independent review is first-class work here. Audit the evolution control plane, its checkpoint trust boundary, and its non-compensation guards.
Control-plane audit #151 ↗
What exists, and what does not
The project keeps these two lists apart on purpose. Both are maintained in the repository README.
On main today
- versioned WorkUnit contracts, with
work-unit-v0.2as the current semantic task contract; - ResultManifest, EvaluatorPlan and VerificationResult contracts that separate worker claims, verifier evidence and integration authority;
- cross-object provenance and integrity validation;
- protocol-neutral worker adapters plus A2A/MCP bindings and conformance helpers;
- zero-project-spend compute admission and routing experiments;
- IDKGraph repository modelling, observability and link-integrity machinery;
- an installable
idkmeshCLI whosegate-auditcommand packages the measured verifier-panel results as a review-gate diagnostic; - protected
mainwith the PR gate required on Python 3.11 and 3.13.
Not yet a finished capability
- no claim that IDKMesh can safely coordinate thousands or millions of real machines;
- the reference Verified Swarm Runner is not a polished install-and-run product with multiple production worker adapters;
- canonical real-node integration remains behind its independent-review and evidence gates;
- A2A/MCP support is an interoperability layer, not a claim that every external agent framework is production-integrated;
- autonomous repository and community actuation stays policy- and authority-gated;
- benchmark infrastructure is not scientific proof until controlled observed runs exist.
Explore
Explained simply
How it works
Six animated figures: bounded work, correlated verifiers, effective votes, the authority ladder, verification debt, and the evidence ladder.
Read →
Architecture
Four pipelines
Animated flowcharts of the work/evidence path, the two-attempt orchestrator, zero-spend compute admission, and the PR gate — read off the executable code.
Read →
Evidence
Experiment atlas
Every experiment from E011 to E042 with its real status — positive, negative, or prototype — and what it changed.
Read →
Everything
Documentation library
The full material index: contracts and schemas, architecture, specifications, research, findings, audits, governance, and project memory.
Browse →
The operating principle
Optimise verified useful work per unit of scarce human attention and compute — not commits, agent count, stars, or activity volume.
A proposal is not true because it came from a strong model, many models, an expert human, or a popular repository. Evidence must survive independent scrutiny.
IDKMesh is intentionally unfinished. The repository is the public laboratory, and well-supported disagreement is a contribution. If you can improve a question, falsify an assumption, reproduce an experiment, write a test, find a security problem, make a contract clearer, or reduce reviewer burden, you can contribute.