Renma Devlog #1 

Dataset built, smoke test passed, full QLoRA run not started. 

Dataset:
12,000 examples (11,760 train / 240 val) from MathNet (18,823 kept of 27,817) + Hendrycks MATH (2,354 kept of 7,500, tiers 1–2 only), 149 dupes removed.
Tier mix: 1—2.5%, 2—8.7%, 3—43.9%, 4—36.1%, 5—8.8%.
Open question: tier 1 is thin (2.5%) — decide before full run whether to upsample or accept the skew. 

LoRA targeting:
Qwen3.5-4B is hybrid — 24/32 layers are Gated-DeltaNet (in_proj_qkv/in_proj_z/out_proj), only 8 are standard attention (q/k/v/o_proj).
Targeting just the usual four names would silently train ¼ of the network with no error.
qlora.yaml now targets both.
Still need to verify empirically via trainable param count after get_peft_model(). 

ROCm/bitsandbytes:
gfx1030 + rocm70 build segfaults with no traceback (no gfx1030 code objects despite support matrix).
Bisected to rocm714 build working — now forced via BNB_ROCM_VERSION=714 in rocm.py.
Also: flash-linear-attention/causal_conv1d are CUDA-only, so Gated-DeltaNet falls back to torch_chunk_gated_delta_rule (correct, 2–3x slower). 

Next:
Run full QLoRA training.