Dycosystem
Hardware- 10 Devlogs
- 116 Total hours
Got the Neural_network class doing an actual feedforward pass now, but honestly most of my time this session wasn’t spent typing, it was spent on a whiteboard (well, my head) trying to actually understand the math before I wrote a single line.
Matrix multiplication only works if the columns of the first matrix match the rows of the second, and figuring out WHY that has to be true, not just memorizing it, is what ate most of my time. Once that clicked, the actual network made a lot more sense: weights between input and hidden layer have to be shaped (hidden_nodes, input_nodes) specifically so the multiplication lines up correctly, same for hidden to output. It’s not arbitrary, the shape IS the math.
feedforward(input): turn the input into a matrix, multiply by weights, add bias, sigmoid, repeat for the next layer, turn the result back into a normal arrayFed it [1, 0] and got a real guess back out. Random weights so the guess means nothing yet, but the math pipeline actually works end to end now, and I understand every step of it instead of just copying it.
Weights used to randomize as integers 0-9 which is wrong for a neural network, changed it to floats between -1 and 1, which is what weights are actually supposed to start as.
train() exists but does nothing yet. That’s the next math rabbit hole, figuring out how to actually adjust these weights based on how wrong the guess was.
Slower progress than usual but way more understanding behind it. Thanks for reading, see you next devlog!
(I am posting a picture of my dog since I have nothing to post a picture of, this is probably the devlog with the most pictures!)
Wired the Neural_network class up to a real feedforward pass. Before this it was just remembering how many nodes it had — now it actually takes an input and produces a guess.
feedforward(input) now does the full pass: multiply by weights, add the bias, squash through sigmoid, twice (once per layer), and hands back a real guessRan it with a plain input array [1, 0] and got an actual output back. First time this thing has done anything resembling thinking, even if the “thinking” is currently just random weights.
Matrix.make_from_array() turns a plain array into a column matrixmake_to_array() turns a matrix back into a plain arrayNeeded both since the network has to convert real inputs into matrix form to do the math, then convert the result back into something normal code can use.
randomize_matrix() used to fill values with random integers 0-9, which makes no sense for neural network weights. Changed it to random floats between -1 and 1, which is the actual range weights should start in.
train() exists but it’s empty right now — the network can guess, it just can’t learn from being wrong yet. That’s next.
Genuinely exciting one, this is the first time the neural network side of the project has produced output instead of just being scaffolding. Thanks for reading, see you in the next one!
I went back through my Matrix class and actually fixed a bug that was sitting in the matrix multiplication code — I was resetting my running sum inside the innermost loop instead of before it, so it was only ever keeping the last multiplication instead of adding them all up. Classic off-by-scope mistake.
scalar_mult and made it its own static matrix_mult function, since cramming two different operations into one method never made much sensemap and map_index
map(fx) runs a function over every value in the matrixmap_index(fx) does the same but also passes in the row/column, so the function can behave differently depending on positionmap is forprint_table
console.table(this.matrix_values) everywhereFeels like boring plumbing work, but this is the stuff the whole neural network is going to be built on top of. I’d rather get it solid now than debug it later once there’s an actual brain depending on it.
Thanks for reading, see you in the next one!
What’s up guys! Quick but important update this time.
I leveled up from just a single perceptron to actually starting a real multi-layer neural network. First thing I had to do for that was build my own Matrix class, since it turns out neural networks are basically just matrix math wearing a trench coat — the weights, the inputs, the outputs, all matrices, and “guessing” is really just multiplying them together.
So I wrote one from scratch. It can zero itself out, randomize itself, transpose, multiply by a number or by another matrix, and add stuff to itself. Not the most exciting thing to build, but literally every neural network sits on top of this, so I wanted to actually get it instead of just grabbing a library and moving on.
I also started the real Neural_network class, but honestly it’s not doing much yet — right now it just remembers how many input, hidden, and output nodes it has. No actual thinking happening in there yet. Next up is hooking the Matrix class into it so it can take inputs and spit out a real guess, and after that, actually learn from being wrong.
Now that I’ve got two real tracks going I split the project up — Genetic_Algorithms/ for the ecosystem sim, and NeuralNetwork/ for all this neuroevolution groundwork.
Slow progress this time, but it’s the kind of slow that matters — I’d rather actually understand the math than fake my way through it. Thanks for reading, see you in the next one!
I am back with a new project! And it is not small…