Devlog #001

Today I started a pretty big project. I really like math, and I thought this could be the perfect way to combine it with Python.

I started by learning how neural networks work and then moved on to the math behind them. I didn’t just want to memorize the formulas, I wanted to actually understand them, because without understanding the theory, implementing them in code becomes much harder (if not impossible).

The hardest part was, by far, understanding how to implement backpropagation and how all the mathematical concepts connect together.

To finish today’s coding session, I created a small demo to test if I had actually understood everything. I decided to build a simple neural network that predicts the XOR gate (a problem that cannot be solved using a linear function).

For this model, I used Xavier initialization, the hyperbolic tangent (tanh) activation function for the hidden layer, a sigmoid activation function for the output layer, and Binary Cross-Entropy (BCE) as the loss function.

In the next devlog, I will learn how to load and start processing the MNIST dataset.

Thank you for reading!