Devlog 4 — Phase 1 Complete!
Picking up where I left off, I added softmax to the output layer instead of ReLU, which converts the 10 outputs into actual probabilities that sum to 1. I also switched from MSE loss to cross entropy loss, which is the standard for classification problems since it heavily penalizes confident wrong predictions.

I implemented batch training, splitting the 60,000 image dataset into chunks of 256 instead of feeding all of it through at once. This made training way faster and way more effective. Accuracy jumped to over 90% in just a couple of cycles instead of taking 100 cycles like before.

I also added training accuracy printed alongside the loss each cycle so I could watch both improve together in real time. Then I built save and load functionality so NeuroLab doesn’t have to retrain from scratch every time, plus auto load logic that checks if a saved model already exists and loads it instead of retraining.
Final results from the full 100 cycle run. Training accuracy hit 100% and stayed there, and test accuracy on 10,000 images NeuroLab had never seen came out to 98.02%.

Last thing today was writing test_project.py with full pytest coverage. I wrote 9 test functions covering sigmoid, relu, relu_derivative, preactivation, init_weights, mse_loss, simple_loss, softmax, and cross_entropy_loss, with multiple test cases each. All 9 passed.

That officially wraps up Phase 1 of NeuroLab. Next up is Phase 2. A second dataset, more optimizers, and regularization with dropout and L2.