Date: 2026-10-13
Status: pre-outcome computational-method amendment; no X6/X7 confirmatory verifier outcome is used here
Amends: v1 preregistration and v1.1 amendment
Issue: #1080
Parent program: #971
The v1.1 amendment froze two separate requirements:
A literal implementation that performs both 10,000-replicate confirmatory bootstraps inside every one of 10,000 power-simulation cohorts is a nested bootstrap with roughly 100 million bootstrap replicates per clustering level and per candidate sample size, before accounting for the work inside each replicate. At the larger task counts it is not a credible zero-project-spend planning path.
No scientific outcome motivated this amendment. The problem was identified during implementation review of #1080 before any X6/X7 confirmatory verifier matrix existed.
This amendment changes only the sample-size power approximation. It does not change the final confirmatory estimator, decision threshold, effect of interest, number of confirmatory bootstrap replicates, or the allowed interpretation of X6/X7.
The confirmatory H1 decision remains exactly the v1.1 rule:
The power planner records this final rule but does not execute 20,000 nested bootstrap replicates inside each simulated cohort.
The following v1.1 quantities remain frozen:
Calibration effects are still centered separately within the official-pass and official-fail strata before +delta0 is injected. The observed calibration effect direction therefore cannot make the planned sample smaller.
For each simulated cohort, compute the class-macro point estimate as the equal average of the official-pass and official-fail mean log-score differences.
For each clustering level — repository and task — delete each cluster in turn and recompute the macro estimate. The standard delete-one-cluster jackknife variance is used, and the planning lower bound is the full-cohort estimate minus the two-sided 95% Student-t critical value times the jackknife standard error.
A simulated cohort counts as a power success only when both the repository-level and task-level lower bounds are above zero.
The estimated power is the success fraction over the 10,000 simulated cohorts.
This is a sample-size planning approximation, not a replacement for the confirmatory percentile bootstrap.
Repository counts can be small, so the planner uses the two-sided 95% Student-t critical value for the repository/task cluster degrees of freedom rather than the Normal 1.96 threshold. Values through 30 degrees of freedom are fixed from the standard t table; above 30 the deterministic implementation uses a Cornish-Fisher expansion around the Normal 0.975 quantile.
This makes the planning interval more conservative than a Normal approximation when the repository count is small.
The simulated cohort must not contain more independent repository clusters than the actual eligible confirmatory universe.
For candidate task count N, the simulated repository count is the smaller of N and the eligible repository count, subject to at least eight repositories.
Calibration repository profiles may be resampled with replacement to represent an unseen repository drawn from the same calibration population, but each simulated cohort contains no more than the eligible repository count.
The eligible repository count is frozen from the pre-confirmatory candidate inventory; it is not inferred from verifier outcomes.
For each task count on the frozen ladder:
Choose the smallest task count reaching power at least 0.90.
If no ladder value reaches 0.90, evaluate the complete eligible task universe.
If the complete universe remains below 0.90, the frozen result is underpowered_full_universe. The study may still run and publish descriptively, but it does not claim that the positive confirmatory design was adequately powered for delta0.
If the admissibility requirements fail — including fewer than eight eligible repositories or absence of either execution-label stratum — the result is infeasible.
Neither status can be repaired after confirmatory unblinding by changing delta0, alpha, cluster level, or the task-count ladder.
The machine-readable plan is canonical only when all of these match this amendment:
Smaller simulation counts may be used for unit tests and implementation development, but the output must carry canonical=false and cannot satisfy the S3 freeze gate.
A freeze manifest must re-parse the bound power plan and reject canonical=false, an invalid self digest, or drift in any frozen setting.
The confirmatory hypothesis test is not made cheaper or less strict. The final repository/task bootstrap decision remains exactly as preregistered.
This amendment only replaces a computationally explosive power-analysis approximation with a standard cluster-jackknife/cluster-t approximation that:
The approximation and its limitation must be reported in Paper A’s methods if the power plan is used.
Nothing in this amendment is evidence that E017 replicates.
Still required:
This amendment was created during implementation review of #1080 on 2026-10-13. The reviewer noticed that a literal nested 10,000-by-10,000 resampling design was not operationally credible under the project’s zero-spend constraint.
The change was made before any X6/X7 confirmatory verifier outcome was generated or inspected. It is AI-assisted research design under maintainer direction and is not independent peer review.