Heyyy!

I have been making some changes to my Neural Compression Algorithm, resulting in a greatly improved model. These changes fall in 3 major ones:

1. Architecture shift:
   Previously, I had many Convolutional layers that shrunk the spatial dimensions because they had no padding. It seems that adding padding to prevent Convs from shrinking spatial dimensions, and using a pool on top of that is far better.
   ○ I reduced the number of conv layers,
   ○ added more pooling layers,
   ○ added padding, and
   ○ Implemented upscaling.
   This upscaling is really important and possibly the main reason for my model’s newfound performance. Previously, I was relying on linear layers to increase the spatial dimensions, but linear layers are not that good for images. Thus, it was best for me to implement the opposite of a pooling layer, an upsampling layer. Basically, this just splits each pixel into 4 new pixels, where each of these 4 pixels have the same colour as the original. From here, I can use a convolutional layer to smooth out the data.

2. Compression size increase:
   The second major improvement came from increasing the size of the compressed image. However, it may seem wrong to improve the model simply by reducing the work it must do, but I argue this is necessary. Before, I was shrinking an image from 32 * 32 * 3 (32 pixels wide, 32 pixels tall, and 3 for RGB) to just 64 numbers. This results 48x fewer data to store, which is simply too much. If these images were higher quality (like if they were 4K resolution), it would be easier to shrink the images by a similar factor. However, I do not have such images, but that may come in a future iteration

3. Rewrite:
   I have rewritten the entire training file from scratch, removing any unnecessary lines as I go. This cut down on bloat and reduced training time by ~2-3x. A lower training time allowed me to run the training loop for more epochs, greatly speeding up the rate at which I can improve my code.

If you took the time to read this, thank you so much! Feel free to comment and ask me any questions about this project.