@Clark

These past 10 hours of coding have been absolute hell. I have tried to make images with the same label closer in the embedding space, but then everything else broke. After about a week, I just gave up on it entirely and accepted that it was only going to aim for differentiating images, not caring about the image labels. After making this change(a very saddening one, pouring all that time down the drain), I was able to spike model performance greatly.
I have practically come full circle, with my model hardly different from what it was at my last devlog, but I have learnt some valuable stuff. I have improved the loss function, giving a much cleaner way to compare embedding vectors and calculate their (dis)similarity. Additionally, I have created a few pieces of code that allow me to visualise various aspects of my model. All in all, I wouldn’t consider this time to be a total waste, as I have learned some valuable stuff. If you want to take a look at the various parts I have added, check out the images attached below and read my explanation of what they do:

1. The first image shows all the kernels in the first convolutional layer of my network.
2. The second image shows a tree with all the operations applied to the images, converting them from an image to an embedding vector.
3. The third image shows the process of my model being trained

I have made a couple of changes to the embedding of this network. Firstly, I have increased the embedding dimension from 16 to 128, allowing it to capture much more information about the image. I have added a couple of new files to my Github repo, allowing me to view the corrupted images, run the model already trained and have started work on a classifier based on my Siamese Network.

Additionally, I realised what was causing my model to be terrible at predicting two similar images: I had accidentally weighted it to focus on different images at a 10x higher rate than similar images. With these out of the way, I have been able to lower the error rate of my model drastically.
Another thing I have done is change how the corruption function works. Previously, it would normalise the RGB colours to 0-1 by simply dividing by a factor. This caused the images to get darker, lowering my model’s performance. By switching this to a sigmoidal function, I was able to achieve much better results.
I also changed the different img target to rely on the labels. If the two images had the same label but were different, they would be related with a dot product of about 0.9, drastically improving my model’s organizational abilities. By which I mean that the embeddings for similar concepts are themselves similar. Seems obvious, but I forgot to do it until this point.

I have been working on an improved optimization algorithm for my Siamese network. I have done this using a dot matrix. What this means is that I have taken the pairwise dot product between the embeddings of two images. These two images are either the corrupted and non-corrupted versions of the same image, or two different images. The two different images will be referred to as Set A, and the two same images will be referred to as Set B.

Set A, the cosine similarity matrix(same image) in the image below, shows very promising results, matching up with what is hoped. A majority of the image is random noise, representing a low correlation between different parts of the embedding vector. This is intended because I want to maximize entropy, meaning it can store the most information in each part of the embedding vector. The main diagonal of Set A(the one going from the top left to the bottom right) shows a very high value. This means that the embedding vectors of the corrupted and non-corrupted versions of an image are very similar. Set B is very noisy, meaning that the embedding vectors of unrelated images are very dissimilar, exactly what I want.

These results are very promising, allowing me to work on interesting uses of this model, which will be the focus of my next few devlogs.

I have added an optimization function to my Siamese Network!

For some information on these images, the difference error graph(the one on the right) is measuring the difference between the high-dimensional representations, or embeddings of different images. I want this graph to be as high as possible, meaning each image gets its own unique embedding. The graph on the left, however, measures the difference in embeddings between corrupted and uncorrupted versions of the *same * image. This should be as low as possible, meaning corrupted and uncorrupted versions of an image are converted into very similar embeddings.

Unfortunately, it is struggling to make the embedding of different images distinct, shown through a very low difference. This may seem bad, but it is working EXACTLY AS EXPECTED.

In many papers on these types of neural networks, cosine similarity is used to reduce feature collapse. I will get into cosine similarity in a later devlog, but for now, I will explain feature collapse. Feature collapse is a phenomenon in neural networks in which all outputs of a network collapse to a single value. This causes the network to always spew the same output value, regardless of the input. In my network, this would show up as a very low similarity error and a very low difference.

This is exactly what is shown in the image below. Therefore, I am on the right track!

Hey there! I have been working on a Siamese network, and this is my first devlog. A Siamese network is a type of neural network akin to an advanced compression algorithm. It works by converting an image into an abstract list of numbers, known as an embedding. It teaches the network to make these abstract numbers as close as possible for corrupted and uncorrupted versions of the same image, and as different as possible for different images. At the moment, I have just been working on the corruption algorithm, which I have got working. Siamese Networks were some of the earliest neural networks applied to real problems, that of recognising handwritten digits for an ATM.

Just a test devlog