---

DEVLOG #7 (2026-07-02, V3 BENCHMARK REBUILD)

Project: SL-LLM-R
Tracked Work Time: Approximately 8 hours

I rebuilt the Stage 1A benchmark from zero while keeping the proposal engine
frozen as the measurement target.

The V1/V2 tasks were mostly all-pass or all-fail. V3 introduces structured
difficulty axes, different hidden-rule forms, and verifier-opacity conditions.
The goal is to create cases where structured search can use information that
random search cannot.

ADR-0012 documents the design. The important rule is that I cannot modify the
proposal kernels, ranking, scheduler, or verifier to make V3 look better. The
benchmark is the ruler; changing the ruler to favor the system would invalidate
the result.

Development Round 1 exposed more weak tasks. Round 2 adjusted the calibration
and froze the generator configuration, splits, and parameters.

The frozen development evaluation then produced a discrimination signal that
V1/V2 lacked. The development gate returned DEVELOPMENT_GO: the engine
produced valid, diverse candidates under budget and the benchmark was no
longer purely all-pass/all-fail.

I also found an environment bug: tests sometimes imported a stale editable
installation instead of the current repository. A new verifier now fails when
imports resolve outside the active checkout.

---

What Works

• V3 generator with structured difficulty axes
• Two documented calibration rounds
• Frozen development configuration
• Provenance reconciliation and editable-install isolation
• Development gate: DEVELOPMENT_GO

---

Tests

The recorded V3 suites, provenance audit, and editable-install checks passed.

---

Changed Files

Commits: d4884b6 through cdb2038

---

Next Steps

Preregister the confirmatory protocol, freeze the generator and sealer, and
prepare a held-out split.

---

Summary (AI)

V3 produced a useful development-level discrimination signal. This was a
Development GO, not a confirmatory or final scientific result.