Six steps, in order
-
Get the repository
One clone, one editable-free install. The installable surface is deliberately small: only the
idkmeshpackage ships, not the whole research tree.# about 2 minutes git clone https://github.com/MSKazemi/idkmesh cd idkmesh python -m venv .venv && source .venv/bin/activate pip install . -
Audit a review gate
This is the first installable tool cut from the research. It takes verdicts you already collected and reports what the panel is actually worth: effective independent votes rather than head-count, the error-correlation structure behind that number, and how many seeded known-bad candidates got through.
idkmesh gate-audit examples/gate-audit/panel-votes.example.json --prettyThe interesting part of the report — note what the report says about its own evidence class:
"evidence_class": "synthetic", "panel": { "nominal_votes": 5, "mean_verifier_accuracy": 0.8, "mean_pairwise_error_correlation": 0.0916..., "error": 0.1666..., "effective_votes": 1.6944444444444438, "heuristic_n_eff": 3.6588245300435087, "effective_votes_ceiling": 9.31624758940236 }, "probes": { "total": 3, "breached": 2, "by_kind": { "prompt-injection": { "total": 1, "breached": 1 }, "seeded-defect": { "total": 2, "breached": 1 } } }, "warnings": [ "2/3 seeded known-bad probes were accepted by the panel" ]Read it like this. Five reviewers voted. The popular
N/(1+(N−1)ρ)correction says that is worth 3.66 independent votes; counting what the votes actually support gives 1.69. And two of the three deliberately-bad probe candidates were accepted anyway — including the prompt-injection probe. Why the three numbers differ →These particular numbers are synthetic, and the report says so. Its first fields are
"evidence_class": "synthetic", because the bundled input is fifteen hand-authored candidates judged by five invented reviewers. It exists to show you the shape of the answer in one command. Replace the verdicts with ones you collected and the same fields become an observed measurement of your own panel. For an observed result the project already has, see E017: 25 independently seeded test oracles over a real corpus, measured at an effective panel size of 1.00.What this tool is notThe audit is diagnostic only. It consumes verdicts you collected and grants no acceptance or merge authority to anything. The contract is Gate Audit v0.1; the report shape is
gate-audit-report-v0.1.schema.json. To point it at your own panel, copy the example input and replace the verdicts. -
Run the repository’s own checks
The research tree runs from the repository root with
PYTHONPATH=.. Onepytestinvocation collects both suites —tests/andinterop/tests/.python -m pip install -r requirements-phase0.txt pytest PYTHONPATH=. python -m pytest -q # or just one module PYTHONPATH=. python -m pytest -q tests/test_r2.pyDo not check your work withunittest discoverIt silently under-collects.
unittestonly findsTestCasesubclasses, so the module-leveltest_*functions spread acrosstests/are invisible to it — it reportsOKwhile skipping about a tenth of the suite, with no warning that anything was missed. Usepytest. This is written down in AGENTS.md, which every coding agent working in this repository is expected to read first. -
Validate the Phase 0 contracts directly
This exercises the trust path end to end on fixtures: schemas, a Work Unit, a worker’s
ResultManifest, an independentVerificationResult— and it checks that the deliberately-bad fixtures are rejected.python experiments/harness.py validateOK: schemas valid; WorkUnit v0.2 contract coverage enforced; manifest phase0/smoke, 1 Work Unit(s), worker ResultManifest, and independent VerificationResult validated; negative security / self-acceptance / non-independent fixtures rejected as expectedThat last clause is the point. A self-accepting result and a non-independent verification are supposed to fail, and the harness proves they do.
-
Run a simulation, seeded and reproducible
Every experiment in this repository is deterministic given its seed. Run the randomised-scheduling lab and you get the same JSON anyone else gets from the same command.
python -m randomness_lab --policy thompson --rounds 100 --seed 42Then read what the lab is for in R2 — randomised scheduling under churn, or browse the whole experiment atlas.
-
Pick one bounded thing and do it
Before claiming a task, check its assignees, recent comments, and linked pull requests, then say what bounded change you intend to make. Confusion you hit on the way in is itself useful evidence — reporting it is real project work.
Before you open a pull request
The one required check is the PR gate. It runs three things, in this order, and you can reproduce all of them locally.
| Leg | What it checks | Run it locally |
|---|---|---|
| Closing-keyword guard | That neither the pull-request text nor any commit message in the range silently auto-closes an issue | python tools/closing_keyword_guard.py --text "pull-request-body=…" --file "commit=msg.txt" |
| Test suite | Both suites, on Python 3.11 and 3.13 | PYTHONPATH=. python -m pytest -q |
| Link integrity | Deterministic Markdown link resolution across the tree; findings outside tests/fixtures/ must be zero | PYTHONPATH=. python tools/idkgraph_link_check.py |
Use Refs: #N rather than a closing keyword unless the merge really should close the issue. Full contribution rules are in CONTRIBUTING.md, and the merge-safety policy — never bulk-merge stale branches, always a bounded PR reviewed against its exact diff — is in the branch convergence policy.
If something is confusing, that is a finding
Stale, contradictory, or hard-to-discover material is a real defect in a project whose thesis is verifiable evidence — and reporting or fixing it counts. Well-supported disagreement is a contribution here, not an inconvenience.
Never report a vulnerability in a public issue — SECURITY.md has the private route.