EDITH AGENT

EDITH devlog #2
—Hello again proggers—
Sup. Still here, still yapping.
Quick recap for anyone new: EDITH is what happens when you decide JARVIS wasn’t cursed enough and you need a five-tier microkernel hierarchy to fix that. Kernel > EDITH > Atlas > Agents > Services. It’s fine. Everything’s fine.
Cat or dog? torch.cuda
print(f”I try to make stu{2f} which {actually_works} {2night}”)
Okay enough cold open, let’s get into it~
—Yap—
First real training run for EDITH is done. Not “done done” — done in the sense that a QLoRA finetune on Phi-4 14B exists on disk and produces tokens instead of an error traceback, which at this point I am counting as a win.
lower your standards, it’s the only way to survive
This is the run that’s supposed to teach base Phi-4 how to actually be EDITH instead of a generic assistant that thinks it has no idea what Atlas is.
—The Great Suffering—
1 — The EOS Token Incident
Model would NOT stop talking. Like, at all. Turns out Phi-4’s chat template expects <|im_end|> and I had been happily feeding it <|endoftext|> this whole time like some kind of caveman.
What do you think fixed it?
A. Retraining from scratch immediately
B. Screaming into a pillow
C. Actually reading the tokenizer config instead of assuming
D. Bribing the model with more epochs
Option C. Unfortunately.
2 — Loss Masking, or: Why Is It Learning My Prompts
Spent a genuinely embarrassing amount of time confused why the model was so good at reciting system prompts back at me. Loss masking was straight up missing, so it was dutifully backpropagating on tokens it should’ve been ignoring. It learned to talk like the prompt because I told it, structurally, that the prompt was the thing worth learning.
in my defense nobody told me
yes they did, it’s in every QLoRA guide ever written
3 — Chat Template Drift
This one’s still a little bit alive. The template drifts juuust enough between what I define and what the tokenizer actually applies at inference time that formatting gets weird on longer contexts. Not broken, just… off. Like a shirt buttoned one hole wrong.
Haven’t fully solved this yet. That’s a devlog #3 problem.
4 — The Hardware Wasn’t The Bottleneck For Once
RTX 5080 laptop chewed through the QLoRA steps fine once Forge Neo and friends stopped fighting Blackwell support. First time in a while the GPU wasn’t the thing yelling at me. Felt weird. Suspicious, even.
—FAQ (no one asked these but they are here)—
Q: What is a “progger”?
A: A human programmer. You. The one reading this. Congrats on the promotion from “hooman.”
Q: Is EDITH sentient yet?
A: No. EDITH is a 14B parameter model that, as of this week, has just learned to stop talking when it’s supposed to. Baby steps.
Q: Why not just use a bigger model and skip all this pain?
A: Because then I wouldn’t get to write devlogs about my pain.
Q: What’s Atlas?
A: The bounded-router tier that handles the satellite nodes so EDITH doesn’t have to babysit every timer and wake-word ping. It’s the responsible sibling.
Q: Will chat template drift actually get fixed in devlog #3?
A: Unknown. Ask again in devlog #3.
Q: Is OP “devlarping”?
A: No, at least I hope not. This is a progger devlog. I have some dignity.
—This is the end*—
*end for now, EOS token permitting
Next devlog is either “chat template drift: solved” or “chat template drift: I have accepted my fate.” No in between.
CYA!!!

#DEVLOG 001

built a simple scraper for training data
we currently scrape:

• open github projects open for LLM training
• internal code and docs
• wikipedia’s LLM training page

these are all going to be used to train the first iteration of edith agents