Devlog 2: Tinyapper (an llm from scratch) is done with its first train! After an epoch of training it averaged ~3.8 nats/token for training data, which is decently close to GPT 2s 3.62… I hope to turn this into a frivolous Wikipedia article generator this weekend for a first ship this weekend, so get ready for that!