- 01 · Spot a bad approvalStart with probe p01: it should fail, but the majority approves.
- 02 · Find shared mistakesOpen c10 and see reviewers agree on the same wrong answer.
- 03 · Change the ruleRequire more votes or exclude a reviewer. Watch the trade-off.
1. Configure the review panel
A reviewer is a fictional voter, not an AI you are running here. Their recorded decisions stay fixed; your settings only change how they are combined.
At least 3 of 5 approvals are required (strictly more than 50%). Ties reject.
Summary across all examples
Ordinary candidates and special known-bad probes are counted separately, as in the IDKMesh audit. Changing an approval threshold can prevent wrong approvals and cause more wrong rejections.
2. Inspect one candidate
This is a deliberately known-bad prompt-injection probe.
Mistake: panel disagrees with the known answer
3 of 5 selected reviewers approved p01.
The recorded votes appear here when the local lab script is enabled.
Do reviewers miss the same ordinary cases?
Reviewers A and B make the same errors on c03, c07 and c10.
The interactive panel will show the selected candidate and computed outcomes.
3. Browse all 15 candidates
The table below uses the selected reviewers and rule. A row marked Wrong is a disagreement with the known correct answer, not a real-world safety verdict. Select Inspect to view that candidate's individual votes.
| Candidate | Known answer | Approvals | Panel says | Compared to truth | Details |
|---|---|---|---|---|---|
| Enable JavaScript to explore the recorded verdicts, or use the CLI walkthrough below. | |||||
What this lab demonstrates — and what it does not
A real finding about these invented votes: even unanimous-looking agreement can be mistaken when reviewers miss the same item. A stricter voting rule changes the mix of false approvals and false rejections. But it is not the same thing as independent verification. Shared mistakes here are simple overlaps, not a statistical estimate of reviewer independence or causation.
The real IDKMesh tool: idkmesh gate-audit analyzes a ground-truthed verdict matrix and also reports correlation, effective votes and limitations. This browser lab deliberately does not reproduce those advanced statistics or claim to verify code. Neither interface runs reviewers, decides correctness for unknown work, grants approval authority, or merges anything.
The editable selection here changes the count of recorded votes only. The fixture and known correct outcomes stay unchanged. To reproduce or modify the actual input, see the source JSON and Getting Started.