---

DEVLOG #7 (2026-07-02, V3 BENCHMARK REBUILD)

Project: SL-LLM-R
Tracked Work Time: Approximately 8 hours

I rebuilt the Stage 1A benchmark from zero while keeping the proposal engine
frozen as the measurement target.

The V1/V2 tasks were mostly all-pass or all-fail. V3 introduces structured
difficulty axes, different hidden-rule forms, and verifier-opacity conditions.
The goal is to create cases where structured search can use information that
random search cannot.

ADR-0012 documents the design. The important rule is that I cannot modify the
proposal kernels, ranking, scheduler, or verifier to make V3 look better. The
benchmark is the ruler; changing the ruler to favor the system would invalidate
the result.

Development Round 1 exposed more weak tasks. Round 2 adjusted the calibration
and froze the generator configuration, splits, and parameters.

The frozen development evaluation then produced a discrimination signal that
V1/V2 lacked. The development gate returned DEVELOPMENT_GO: the engine
produced valid, diverse candidates under budget and the benchmark was no
longer purely all-pass/all-fail.

I also found an environment bug: tests sometimes imported a stale editable
installation instead of the current repository. A new verifier now fails when
imports resolve outside the active checkout.

---

What Works

• V3 generator with structured difficulty axes
• Two documented calibration rounds
• Frozen development configuration
• Provenance reconciliation and editable-install isolation
• Development gate: DEVELOPMENT_GO

---

Tests

The recorded V3 suites, provenance audit, and editable-install checks passed.

---

Changed Files

Commits: d4884b6 through cdb2038

---

Next Steps

Preregister the confirmatory protocol, freeze the generator and sealer, and
prepare a held-out split.

---

Summary (AI)

V3 produced a useful development-level discrimination signal. This was a
Development GO, not a confirmatory or final scientific result.

---

DEVLOG #6 (2026-07-02, CANDIDATE TRACING AND FAILURE ANALYSIS)

Project: SL-LLM-R

After the V1/V2 no-go, I wanted to see exactly what the proposal engine was
doing candidate by candidate.

I built deterministic tracing and wired it into the engine. Every candidate now
records its kernel source, parent, depth, score history, verifier decision, and
budget consumption. Trace files are canonically serialized and hashed.

Then I evaluated the main remediation hypotheses in isolation:

• Mutation kernel only
• Combination kernel only
• Restart kernel only
• Adaptive mixture

None showed an advantage over random. I also compared the strongest single
kernel and a uniform mixture. They behaved the same way.

The traces did reveal a real efficiency problem: duplicate candidates consume a
significant part of the nominal budget before deduplication. That is fixable,
but the oracle results from the previous devlog show that it cannot solve the
main benchmark problem by itself.

This was still useful. Instead of only knowing that the engine failed, I can now
replay the candidate path and see where each unit of budget went.

---

What Works

• Deterministic candidate-level traces
• Isolated evaluation of the proposal kernels
• Budget-use analysis
• Duplicate-pressure failure identified

---

Tests

.venv/bin/python3 -m pytest tests/stage1a_v2/test_round2_tracing.py

The recorded tracing tests passed. Sixty trace files were generated.

---

Changed Files

Commits: 8f0988c, dc9f233, e5b2b34

---

Next Steps

Use the tracing infrastructure in a new V3 benchmark with intermediate task
difficulty.

---

Summary (AI)

Tracing now explains candidate origin, scoring, verifier outcomes, and budget
use. No proposal kernel beat random on the old benchmark, reinforcing the
benchmark-discrimination NO-GO.

---

DEVLOG #5 (2026-07-02, FIRST EVALUATION AND NO-GO)

Project: SL-LLM-R

The proposal engine did not outperform random. That is the headline.

I built the evaluation runner and compared the engine with four baselines over
four seeds and two budget levels. The result contained 991 data rows, and the
engine’s HitRate was indistinguishable from random proposal.

My first reaction was to blame the engine, so I built a separate no-go analysis
with a failure taxonomy, kernel contribution matrix, and oracle upper bounds.
The oracle result changed the interpretation: even a system with knowledge of
the hidden rule could barely solve the tasks above chance.

The benchmark was not discriminative. Tasks were mostly either trivial for
every system or impossible for every system. There was no useful intermediate
band where structured search could beat random search.

I also found a split-integrity bug. One split-ID path depended on Python hash
randomization and could change between versions. I replaced it with canonical
JSON and SHA-256, then documented the root cause.

I defined remediation hypotheses H1-H5 and tested the V2 development changes.
They also ended in NO-GO. At that point, changing the proposal engine further
would have been tuning against a broken ruler.

---

What Works

• Deterministic evaluation runner
• Failure taxonomy and oracle-bound analysis
• Split-integrity bug fixed and tested
• Negative result preserved instead of hidden

---

Tests

make stage1a-test
make stage1a_v2-test

The recorded evaluation and remediation tests passed. The scientific result was
still NO-GO.

---

Changed Files

Commits: 4de0165, 268c508, 33e64e2, 5ee6178

---

Next Steps

Build a V3 benchmark with real intermediate difficulty and measurable
separation between structured and random search.

---

Summary (AI)

Stage 1A did not beat random on the original benchmark. Oracle analysis showed
that the benchmark lacked discrimination, so the correct result was NO-GO and
a benchmark redesign.

---

DEVLOG #4 (2026-07-01, STAGE 1A PROPOSAL ENGINE)

Project: SL-LLM-R
Tracked Work Time: Not reliably tracked for this entry

Before the update: I lost track of time and forgot to post a devlog. Sorry
about that. I kept working, but I should have documented it properly instead
of reconstructing everything afterwards. I will post devlogs more regularly
from now on.

This session was where the specification first met reality: I built the Stage
1A proposal engine.

The engine takes a ProposalRequest, applies a fixed candidate budget, runs a
configured kernel mixture, and returns a deterministic candidate set. Every
candidate gets a SHA-256 digest, parent reference, depth, and full lineage. No
wall-clock values or floating-point data enter the ID path.

I implemented five proposal kernels:

• Random
• Enumeration
• Mutation
• Combination
• Restart

Budget accounting was more annoying than the algorithms. Rejected and duplicate
candidates still consume budget. That prevents degenerate kernels from running
forever, although it can reduce the effective search budget.

The metrics layer now reports HitRate, diversity, coverage, and novelty. I used
Jaccard distance over intervention fingerprints for the first diversity metric.
It is deterministic, but it may confuse syntactic difference with meaningful
semantic difference.

The Stage 1A schemas include proposal candidates, requests, manifests, split
manifests, decision points, kernel results, and metric reports. Invalid outcomes
use typed reason codes instead of arbitrary strings.

---

What Works

• Deterministic candidate generation under a fixed budget
• Five proposal kernels with traceable lineage
• Reproducible splits and typed validation
• Initial HitRate, diversity, coverage, and novelty metrics

---

Tests

make stage1a-test

The recorded Stage 1A test run passed.

---

Changed Files

Commit: d9f5a10

41 files and about 2,014 added lines.

---

Next Steps

Run the engine against baselines and find out whether it actually does anything
useful.

---

Summary (AI)

The Stage 1A proposal engine is implemented, deterministic, and tested. Its
scientific value is still unknown until baseline evaluation.

---

DEVLOG #3 (2026-07-01, ARCHITECTURE, TOOLCHAIN, AND STAGE 0)

Project: SL-LLM-R
Tracked Work Time: 14 hours

Today started with architecture decisions and ended with the first real Stage 0
implementation.

I wrote six ADRs for the repository structure, toolchain, dependencies,
determinism, and low-level code.

The language split is:

• Python for research logic and orchestration
• C for stable ABI boundaries
• Assembly only for measured hot paths

I wanted assembly earlier, but that would add complexity before the system is
testable. SHA-256 stays in Python’s hashlib.

Central object IDs must not depend on time, process IDs, filesystem ordering,
thread scheduling, or uuid4(). Objects are hashed from canonical JSON:

json.dumps(data, sort_keys=True, separators=(",", ":"))

Core must not depend on infrastructure. I also added a machine-readable Stage
0 Definition of Done. Stage 0 only passes when schemas, validation, roundtrip
tests, golden vectors, and provenance exist.

Then I started coding.

The result was 67 files and around 2,514 added lines.

The schemas cover events, snapshots, manifests, verifier results, registry
commits, resource accounting, and Governor decisions. Each central object is
immutable, validated, encoded, hashed, and linked to provenance.

The RNG uses SplitMix64 with deterministic seed partitioning. The same seed
produces the same sequence across processes and thread counts.

The event store is append-only. Every event has a parent_hash, so history
cannot be silently overwritten.

The registry was the hardest part. Publication uses a temporary file and
atomic rename. Crash tests require recovery to either the old valid root or
the new valid root. My first version left temporary files behind, so I added
startup cleanup and root validation.

I also added an x86-64 RNG kernel and checked it against the C reference with
36,008 generated vectors. Both produce identical output.

Finally, stage0_status.py now checks real files, tests, manifests, vectors,
and audit evidence instead of trusting manual passed flags.

---

What Works

• Six ADRs accepted
• Deterministic schemas and canonical encoding
• Append-only event storage
• Atomic registry commit, rollback, and crash recovery
• C and x86-64 parity
• Evidence-driven Stage 0 audit

---

Tests

make test
make conformance

All current Stage 0 tests and the audit pass.

---

Changed Files

• Initial implementation: bf59395
• Status-checker fix: 9d6d353
• Assembly integration: 90a1f70

---

Next Steps

Begin Stage 1A and implement the proposal engine.

---

Summary (AI)

Stage 0 is implemented and tested: deterministic schemas, canonical encoding,
append-only events, rollback, native parity, and evidence-based status checks.

This is an engineering result, not evidence that VGCP or autonomous learning
works.

DEVLOG #2 - Project Start: Spec V1.2.2 Frozen

Project: SL-LLM-R

Session Goal

Finalize Version 1.2.2 of the SL-LLM-R specification before
repository implementation begins.

Approximate Work Time

9h 41m

Introduction

Today was almost entirely specification work.

I finished V1.2.2 of the SL-LLM-R specification, now 135 pages long, and froze
it under SHA-256 453fb34a...

Most of the work was not adding ideas, but making the claim boundary stricter.
I do not want this project to sound like “self-improving AI solved” when it is
still a pre-experimental architecture. Nothing has been tested empirically.

The core idea is VGCP: verifier-grounded counterfactual plasticity.

Instead of letting an LLM blindly update itself, the system should save its
state, try controlled alternatives, compare them from the same starting point,
and use verifier evidence to decide whether a change should be kept.

A positive result alone is not enough. The system also needs uncertainty,
rollback, retention tests, fixed budgets, and checks against verifier
exploitation.

I defined six operations for the integration claim:

• Typed proposal generation
• Sealed trajectory snapshots
• Matched counterfactual suffixes
• Anytime-valid inference
• Module-specific objectives
• Transactional shadow validation

The possible contribution is combining all six into one auditable path, not
claiming every component is new.

I also defined hypotheses, null hypotheses, success conditions, and a
contribution ladder from L0 to L6.

The non-goals matter too. The project does not claim consciousness,
unrestricted self-improvement, guaranteed convergence, open-world safety, or
backbone rewriting in P0.

P0 must not modify the frozen backbone, its Governor, the evaluator, or its
permissions. The learner can propose changes, but trusted components decide
whether they may be tested, promoted, or rejected.

Permission violations are hard failures, not lower scores.

No code yet. No tests. Just the frozen specification.

Completed Changes

• Finished and froze Version 1.2.2 of the SL-LLM-R specification.
• Defined the six operations used for the integration claim.
• Defined hypotheses, null hypotheses, success conditions, and the L0-L6
  contribution ladder.
• Made the project’s claim boundary, non-goals, and P0 restrictions explicit.

Technical Details

• Specification: SL-LLM-R V1.2.2, 135 pages
• Frozen SHA-256: 453fb34a..
• Core mechanism: VGCP
• Six operations define the integration claim

What Works

• V1.2.2 is frozen
• Research question and claim boundary are defined
• Success conditions, falsifiers, and non-goals are explicit

Open Problems

• Can all six operations be implemented practically?
• Is the full mechanism better than simpler baselines?

Tests

None. No implementation exists yet.

Changed Files

None in the repository. The specification is still PDF-only.

Next Steps

Choose the repository structure and toolchain, write the first ADRs, and begin
Stage 0.

Summary

The specification is frozen. The hardest part was not writing more ideas, but
removing claims I could not yet support.

Next: architecture decisions and the first real code.

DEVLOG

#1 - Why I Started SL-LLM-R

Project: SL-LLM-R

---

Introduction

I originally just wanted a project for the summer.

I had a few ideas, but most already existed or sounded like something I would abandon after a few days.

Then I started thinking about AI.

I use AI a lot for coding. It can understand large projects and solve difficult problems, but it can also repeat the same simple mistake several times. Even when it notices that something failed, it usually does not actually learn from it.

The context becomes longer, but the model itself stays almost the same.

Current models are trained on human-created data and updated through processes designed by humans. They can generate answers, but they normally cannot decide what to learn next, which decision caused a failure, or whether a change would damage another capability.

So I started thinking about a system that could learn from its own verified experiences.

That became SL-LLM-R.

---

The Idea

The core mechanism is called VGCP:

Verifier-Grounded Counterfactual Plasticity.

The simplified process is:

1. Save the state before a decision.
2. Let the model act.
3. Check the result with an external verifier.
4. Generate possible repairs.
5. Replay the same situation with and without each repair.
6. Compare the outcomes.
7. Keep only changes supported by evidence.

In simplified form:

Δ = E[Y(change) - Y(original)]

If the repaired version performs better across controlled tests, the change may be useful. If the evidence is weak, the system should not update.

The goal is controlled, local, reversible, and verifiable learning.

---

The PDF

I first planned to write a short document.

That did not go as planned.

Every answer created more questions: How do I save the exact state? How do I compare runs fairly? How do I prevent leakage, forgetting, broken rollbacks, or the system approving its own changes?

The document became the SL-LLM-R V1.2.2 specification.

It is now 135 pages long and contains the architecture, mathematics, safety rules, algorithms, experiments, and failure conditions.

The PDF does not prove that the system works. It defines what I need to build and what would count as failure.

---

Completed Changes

• Defined the project idea.
• Created the VGCP concept.
• Wrote the V1.2.2 specification.
• Started the initial implementation.

---

Next Steps

• Continue the deterministic foundation.
• Implement snapshots and rollback.
• Build the proposal system.
• Compare it against simple baselines.

---

Summary

I started with a few random project ideas, got annoyed by how intelligent and stupid AI can be at the same time, and ended up designing a system that should learn from verified mistakes.

It might become something important.

It might also be another stupid idea that takes over my entire summer holiday.

Right now, I do not know.

That is why I am building it.

---

Summary (AI)

SL-LLM-R is a pre-experimental project about controlled, verifier-grounded continual learning. No validated self-improvement result is currently claimed.

---

Note

I spent way too long writing this devlog and the PDF.

I also started coding parts of the project, and it was… different from what I expected.

More soon.