@rubfiber

I finished training the network for 25 epochs. It took 2 hrs on pretty old hardware, so I would say it performed decent.
Tracking works almost perfectly. It’s able to distinguish between an object and its background, even if it is not a fruit. I tested this with my finger and it followed pretty good. The network makes errors just picking similar-looking fruits, which makes sense since the training data was not that great (for example blue grapes and blueberries look pretty similar through the eyes of the network).

Previously, 15 other incorrect cells during training were drowning out the 1 correct cell. I decided to boost the correct cell’s weight on training by doubling it. Then, I increased learning rate to 0.01 and epochs to 25. It’s still training, but difference is night and day.

Epoch: 1
loss 102.3050 | conf@object 0.580 | conf@empty 0.296 | cell 37/100 (within 1: 87) | class 9/100 (chance 3%) | p(true class) 0.382 | p(top class) 0.532
Epoch: 2
loss 94.2365 | conf@object 0.619 | conf@empty 0.257 | cell 52/100 (within 1: 99) | class 13/100 (chance 3%) | p(true class) 0.414 | p(top class) 0.587
Epoch: 3
loss 86.3697 | conf@object 0.699 | conf@empty 0.223 | cell 58/100 (within 1: 98) | class 12/100 (chance 3%) | p(true class) 0.446 | p(top class) 0.644
Epoch: 4
loss 84.6920 | conf@object 0.752 | conf@empty 0.189 | cell 60/100 (within 1: 100) | class 16/100 (chance 3%) | p(true class) 0.496 | p(top class) 0.654
Epoch: 5
loss 85.4948 | conf@object 0.796 | conf@empty 0.166 | cell 61/100 (within 1: 100) | class 24/100 (chance 3%) | p(true class) 0.532 | p(top class) 0.676

Previously, the number of times the network got the correct object in the cell was 3-9%. Now, in only the 5th epoch (out of 25), it is up to 24. There is still a long way to go, but I found this change significant.

I wan’t able to do much, but I did manage to accomplish these tasks:

I changed training the model based x and y positions to splitting the image into a 4x4 grid - I don’t know if it works since training is taking a while, but it should work fine

addDataGrids added (does the same as addData but for the grids I was talking about)
new trainCNN and testing classes

Tracking was off for fruits, and there was a bug where the overlay is overwritten. However, fixing the bug revealed the model is not yet ready, which either could be a code issue or a dataset issue.

Imrpoved diagnostics, made some changes to addData, and fixed typos in FFN class

I haven’t been able to do much because of schoolwork, but I worked on GPU acceleration, which I am not able to finish for a bit. I made an im2col and col2im GPU acceleration program, and I will add more . However, for now, this is all I am able to complete.

These past few days, I have been working on almost entirely training/testing and fine-tuning.

• I added the ability to change activation function per neuron, which can be used in rare cases
• I mainly added a webcam ability, which is shown in the pictures below.
  It’s not very accurate, which can be a bad selection of network configurations or a bad dataset, which I’m leaning towards since the majority of it was just fruits on a white background with a few pictures I took myself.
  It also means the example weights and biases or not very accurate for fruits, as seen in the 3rd picture. It keeps thinking random objects are lime and tracking the wrong things, but with the right settings and a good dataset, I’m sure it can be good.
• The predictions are also happening in 3 ms, which is really good.

Over these past 3 days I was able to add a better prediction and training for my convolutional neural network.

• im2col: I switched from traditional sliding over the image, which was inefficient, to an algorithm that enhances it. It first extracts patches by sliding the filter across the input tensor, extracting every patch (they overlap too). Each patch is 3D, so it must be flattened into a vector and then stacked to form a 2D array. The weights are then also stacked to form a 2D array. Finally, the 2 matrices are multiplied.
• backpropagation: The CNN also needs to learn, so it accumulates gradients, which it uses to calculate how much the error changes by modifying weights and bias. The opposite of im2col is also used to calculate errors. However, in the beginning, the dense layers also need to pass their errors back, so I made a method that passes errors, specifically for CNN backpropagation.
• Pooling layers: pooling layers are very important for CNNs, as they reduce the size of the feature maps, which can help cut down computing costs. It can also help the model recognize an object, even if its position is shifted slightly.
• I might implement a GPU-accelerated CNN using tornadoVM, but this will be challenging, as I may have to modify methods. However, for larger-scale networks, this might add a large performance boost.

Over the past 2 days when I did not post a devlog. I managed to get a lot done.
Instead of relying on hiddenLayersDepth and hiddenLayersWidth, the neteork now uses an array of layer sizes, allowing for more flexibility.
I also added training based on Dataset classes, which are much easier for data organization by using ArrayLists (It didnt change the network architecture so I still am using arrays).
I needed to convert 2D weight/bias arrays into ArrayLists, which made a static method in the util class for.
After that, I did some cleanup by removing redundant variables, which made the code much less messy.
Today, I worked on allowing to customize the activation functions used. For now you can only change one function for the whole hidden layer section, but I will add the ability to make an array later. I used an Enum and a bunch of methods from the util class to add relu, leaky relu, tanh, sigmoid, and softmax to the Activation enum.
Also today, I started work on my convolutional neural network, which is the architecture used for object recognition. I finished most of the layers necessary, but I am yet to test its prediction for errors, so I have not pushed it to github yet. I have the filters and the feed forward network already done, so it shouldn’t be that hard.

I added saving and loading the neural network - The network saves to a .bin file with a desired path, saving space. In a previous project I made a while ago, I used JSON files, which grew very large. This will hopefully save space.
I made a new method in my util class. This helps convert a 1D array into a 2D one with a key, telling how many elements per layer. This will help when I switch over from hiddenLayersWidth and hiddenLayersDepth to an array of lengths.

I added a better training method - Mini-Batch Gradient Descent. It trains the network in small batches instead of the whole dataset. I spent a few hours fixing bugs from implementing this.
Attatched are screenshot of the network’s training performance. Although it is a small network being trained on a simple task, it is still nice to see that the network is running well.
After that, I decided to try to implement training from the GPU using TornadoVM. It did not work, so I did not push to my repository. Once I get it working, however, I will.

Neural network from scratch - added basic training

I added a way to train the model, calculating deltas and adjusting for the errors. Here, I trained it to output the XOR values based on the inputs, where -1 is false and 1 is true. The network outputted answers which appeared to be true (very close to -1 and 1), and before they were outputting random noise. This shows that the network learned something.

Neural network has a predict() method

Although it currently outputs a random value since it can’t be trained yet, it is still a good milestone.

I tested a network with 3 input neurons, 4 output neurons, and 2 hidden layers containing 4 neurons each. I ran the predictions 1 billion times based on random inputs. Here are the results:

Total time: 167380.87 ms
Speed per prediction: 0.000167 ms
Per second: 5,974,399 predictions per second

Neural network - neuron compute function added and changed double to float for better performance