By
@Samuelm
Building a LLM system that can learn from verified outcomes by replaying counterfactual choices, identifying what caused success or failure, and safely turning that evidence into future improvements.
Project: SL-LLM-R
Tracked Work Time: Approximately 8 hours
I rebuilt the Stage 1A benchmark from zero while keeping the proposal engine
frozen as the measurement target.
The V1/V2 tasks were mostly all-pass or all-fail. V3 introduces structured
difficulty axes, different hidden-rule forms, and verifier-opacity conditions.
The goal is to create cases where structured search can use information that
random search cannot.
ADR-0012 documents the design. The important rule is that I cannot modify the
proposal kernels, ranking, scheduler, or verifier to make V3 look better. The
benchmark is the ruler; changing the ruler to favor the system would invalidate
the result.
Development Round 1 exposed more weak tasks. Round 2 adjusted the calibration
and froze the generator configuration, splits, and parameters.
The frozen development evaluation then produced a discrimination signal that
V1/V2 lacked. The development gate returned DEVELOPMENT_GO: the engine
produced valid, diverse candidates under budget and the benchmark was no
longer purely all-pass/all-fail.
I also found an environment bug: tests sometimes imported a stale editable
installation instead of the current repository. A new verifier now fails when
imports resolve outside the active checkout.
DEVELOPMENT_GO
The recorded V3 suites, provenance audit, and editable-install checks passed.
Commits: d4884b6 through cdb2038
Preregister the confirmatory protocol, freeze the generator and sealer, and
prepare a held-out split.
V3 produced a useful development-level discrimination signal. This was a
Development GO, not a confirmatory or final scientific result.
Project: SL-LLM-R
After the V1/V2 no-go, I wanted to see exactly what the proposal engine was
doing candidate by candidate.
I built deterministic tracing and wired it into the engine. Every candidate now
records its kernel source, parent, depth, score history, verifier decision, and
budget consumption. Trace files are canonically serialized and hashed.
Then I evaluated the main remediation hypotheses in isolation:
None showed an advantage over random. I also compared the strongest single
kernel and a uniform mixture. They behaved the same way.
The traces did reveal a real efficiency problem: duplicate candidates consume a
significant part of the nominal budget before deduplication. That is fixable,
but the oracle results from the previous devlog show that it cannot solve the
main benchmark problem by itself.
This was still useful. Instead of only knowing that the engine failed, I can now
replay the candidate path and see where each unit of budget went.
.venv/bin/python3 -m pytest tests/stage1a_v2/test_round2_tracing.py
The recorded tracing tests passed. Sixty trace files were generated.
Commits: 8f0988c, dc9f233, e5b2b34
Use the tracing infrastructure in a new V3 benchmark with intermediate task
difficulty.
Tracing now explains candidate origin, scoring, verifier outcomes, and budget
use. No proposal kernel beat random on the old benchmark, reinforcing the
benchmark-discrimination NO-GO.
Project: SL-LLM-R
The proposal engine did not outperform random. That is the headline.
I built the evaluation runner and compared the engine with four baselines over
four seeds and two budget levels. The result contained 991 data rows, and the
engine’s HitRate was indistinguishable from random proposal.
My first reaction was to blame the engine, so I built a separate no-go analysis
with a failure taxonomy, kernel contribution matrix, and oracle upper bounds.
The oracle result changed the interpretation: even a system with knowledge of
the hidden rule could barely solve the tasks above chance.
The benchmark was not discriminative. Tasks were mostly either trivial for
every system or impossible for every system. There was no useful intermediate
band where structured search could beat random search.
I also found a split-integrity bug. One split-ID path depended on Python hash
randomization and could change between versions. I replaced it with canonical
JSON and SHA-256, then documented the root cause.
I defined remediation hypotheses H1-H5 and tested the V2 development changes.
They also ended in NO-GO. At that point, changing the proposal engine further
would have been tuning against a broken ruler.
make stage1a-test
make stage1a_v2-test
The recorded evaluation and remediation tests passed. The scientific result was
still NO-GO.
Commits: 4de0165, 268c508, 33e64e2, 5ee6178
Build a V3 benchmark with real intermediate difficulty and measurable
separation between structured and random search.
Stage 1A did not beat random on the original benchmark. Oracle analysis showed
that the benchmark lacked discrimination, so the correct result was NO-GO and
a benchmark redesign.
Project: SL-LLM-R
Tracked Work Time: Not reliably tracked for this entry
Before the update: I lost track of time and forgot to post a devlog. Sorry
about that. I kept working, but I should have documented it properly instead
of reconstructing everything afterwards. I will post devlogs more regularly
from now on.
This session was where the specification first met reality: I built the Stage
1A proposal engine.
The engine takes a ProposalRequest, applies a fixed candidate budget, runs a
configured kernel mixture, and returns a deterministic candidate set. Every
candidate gets a SHA-256 digest, parent reference, depth, and full lineage. No
wall-clock values or floating-point data enter the ID path.
I implemented five proposal kernels:
Budget accounting was more annoying than the algorithms. Rejected and duplicate
candidates still consume budget. That prevents degenerate kernels from running
forever, although it can reduce the effective search budget.
The metrics layer now reports HitRate, diversity, coverage, and novelty. I used
Jaccard distance over intervention fingerprints for the first diversity metric.
It is deterministic, but it may confuse syntactic difference with meaningful
semantic difference.
The Stage 1A schemas include proposal candidates, requests, manifests, split
manifests, decision points, kernel results, and metric reports. Invalid outcomes
use typed reason codes instead of arbitrary strings.
make stage1a-test
The recorded Stage 1A test run passed.
Commit: d9f5a10
41 files and about 2,014 added lines.
Run the engine against baselines and find out whether it actually does anything
useful.
The Stage 1A proposal engine is implemented, deterministic, and tested. Its
scientific value is still unknown until baseline evaluation.
Project: SL-LLM-R
Tracked Work Time: 14 hours
Today started with architecture decisions and ended with the first real Stage 0
implementation.
I wrote six ADRs for the repository structure, toolchain, dependencies,
determinism, and low-level code.
The language split is:
I wanted assembly earlier, but that would add complexity before the system is
testable. SHA-256 stays in Python’s hashlib.
Central object IDs must not depend on time, process IDs, filesystem ordering,
thread scheduling, or uuid4(). Objects are hashed from canonical JSON:
json.dumps(data, sort_keys=True, separators=(",", ":"))
Core must not depend on infrastructure. I also added a machine-readable Stage
0 Definition of Done. Stage 0 only passes when schemas, validation, roundtrip
tests, golden vectors, and provenance exist.
Then I started coding.
The result was 67 files and around 2,514 added lines.
The schemas cover events, snapshots, manifests, verifier results, registry
commits, resource accounting, and Governor decisions. Each central object is
immutable, validated, encoded, hashed, and linked to provenance.
The RNG uses SplitMix64 with deterministic seed partitioning. The same seed
produces the same sequence across processes and thread counts.
The event store is append-only. Every event has a parent_hash, so history
cannot be silently overwritten.
The registry was the hardest part. Publication uses a temporary file and
atomic rename. Crash tests require recovery to either the old valid root or
the new valid root. My first version left temporary files behind, so I added
startup cleanup and root validation.
I also added an x86-64 RNG kernel and checked it against the C reference with
36,008 generated vectors. Both produce identical output.
Finally, stage0_status.py now checks real files, tests, manifests, vectors,
and audit evidence instead of trusting manual passed flags.
make test
make conformance
All current Stage 0 tests and the audit pass.
bf59395
9d6d353
90a1f70
Begin Stage 1A and implement the proposal engine.
Stage 0 is implemented and tested: deterministic schemas, canonical encoding,
append-only events, rollback, native parity, and evidence-based status checks.
This is an engineering result, not evidence that VGCP or autonomous learning
works.
Project: SL-LLM-R
Finalize Version 1.2.2 of the SL-LLM-R specification before
repository implementation begins.
9h 41m
Today was almost entirely specification work.
I finished V1.2.2 of the SL-LLM-R specification, now 135 pages long, and froze
it under SHA-256 453fb34a...
Most of the work was not adding ideas, but making the claim boundary stricter.
I do not want this project to sound like “self-improving AI solved” when it is
still a pre-experimental architecture. Nothing has been tested empirically.
The core idea is VGCP: verifier-grounded counterfactual plasticity.
Instead of letting an LLM blindly update itself, the system should save its
state, try controlled alternatives, compare them from the same starting point,
and use verifier evidence to decide whether a change should be kept.
A positive result alone is not enough. The system also needs uncertainty,
rollback, retention tests, fixed budgets, and checks against verifier
exploitation.
I defined six operations for the integration claim:
The possible contribution is combining all six into one auditable path, not
claiming every component is new.
I also defined hypotheses, null hypotheses, success conditions, and a
contribution ladder from L0 to L6.
The non-goals matter too. The project does not claim consciousness,
unrestricted self-improvement, guaranteed convergence, open-world safety, or
backbone rewriting in P0.
P0 must not modify the frozen backbone, its Governor, the evaluator, or its
permissions. The learner can propose changes, but trusted components decide
whether they may be tested, promoted, or rejected.
Permission violations are hard failures, not lower scores.
No code yet. No tests. Just the frozen specification.
453fb34a..
None. No implementation exists yet.
None in the repository. The specification is still PDF-only.
Choose the repository structure and toolchain, write the first ADRs, and begin
Stage 0.
The specification is frozen. The hardest part was not writing more ideas, but
removing claims I could not yet support.
Next: architecture decisions and the first real code.
#1 - Why I Started SL-LLM-R
Project: SL-LLM-R
I originally just wanted a project for the summer.
I had a few ideas, but most already existed or sounded like something I would abandon after a few days.
Then I started thinking about AI.
I use AI a lot for coding. It can understand large projects and solve difficult problems, but it can also repeat the same simple mistake several times. Even when it notices that something failed, it usually does not actually learn from it.
The context becomes longer, but the model itself stays almost the same.
Current models are trained on human-created data and updated through processes designed by humans. They can generate answers, but they normally cannot decide what to learn next, which decision caused a failure, or whether a change would damage another capability.
So I started thinking about a system that could learn from its own verified experiences.
That became SL-LLM-R.
The core mechanism is called VGCP:
Verifier-Grounded Counterfactual Plasticity.
The simplified process is:
In simplified form:
Δ = E[Y(change) - Y(original)]
If the repaired version performs better across controlled tests, the change may be useful. If the evidence is weak, the system should not update.
The goal is controlled, local, reversible, and verifiable learning.
I first planned to write a short document.
That did not go as planned.
Every answer created more questions: How do I save the exact state? How do I compare runs fairly? How do I prevent leakage, forgetting, broken rollbacks, or the system approving its own changes?
The document became the SL-LLM-R V1.2.2 specification.
It is now 135 pages long and contains the architecture, mathematics, safety rules, algorithms, experiments, and failure conditions.
The PDF does not prove that the system works. It defines what I need to build and what would count as failure.
I started with a few random project ideas, got annoyed by how intelligent and stupid AI can be at the same time, and ended up designing a system that should learn from verified mistakes.
It might become something important.
It might also be another stupid idea that takes over my entire summer holiday.
Right now, I do not know.
That is why I am building it.
SL-LLM-R is a pre-experimental project about controlled, verifier-grounded continual learning. No validated self-improvement result is currently claimed.
I spent way too long writing this devlog and the PDF.
I also started coding parts of the project, and it was… different from what I expected.
More soon.
