ResNet Implementation

After debugging for a week. It’s finally done. There’s a whole story behind this though.

1. Bundling a python project with pyinstaller

To ship my project I had to create an exe file out of my project, and for that I used pyinstaller. Ran into multiple micro-dependencies issues like groovy and safehttpx. And the exe file was over 6 GBs

2. Pytorch CPU hell

To reduce the exe file size, I had to use the cpu only torch packages. But when pyinstaller bundled those, I ran into some dependency issue by c10.dll file. And no matter how many workarounds and things I tried I just couldn’t fix it. I just knew at that moment that this is just not the way. There were also some issues regarding Intel OpenMP thread collision, I don’t even know what that means! :(

3. ONNX the savior

Researching a bit I stumbled upon onnx runtime engine for pytorch models. So, I created another app script this time with onnx model loading and numpy based preprocessing (sadly I couldn’t even use the preprocessing pipeline from pytorch). But this was simply the beginning….

4. ONNX dependency hell and incompatibility with other packages

When I tried building the onnx based app, the first issue was a silent failure, no log, no warning just exe file not running at all… It was some silent C level crash that python couldn’t detect, so I changed my app script again and added fault_handler. There was a raw memory collision between gradio and onnx for dll space. I changed the import order for onnx first but pyinstaller still hooked up onnx runtime dll’s to some third party package’s dll files.

Finally after that, I manually wrote a hook script and grabbed the dll files manually myself to inject into pyinstaller analysis object.

Devlog 1.5

• Wrote readme
• Had to commit a couple of times cuz I kept misspelling/misformatting my file names xD.

Devlog 1.4

• Added app usage description
• Added examples for quick testing
• Chose a better theme

Devlog 1.3

• Made a gradio based wrapper around the model
• Debugged the app deployment on hugging face
• Handled edge cases and refactored the code

Devlog 1.2

I mainly focused on profiling the model’s performance, and I found the model’s inference to be taking ~200 ms on GTX 1650. But I compiled the model with torch.compile() which basically fuses all the cuda operations in one big monolithic kernel and my jaw dropped seeing 20x performance boost taking only ~8 ms.

Oh and I also profiled the compiled model wrong the first time and also recorded the lazy jit compilation of the model during the first pass. The solution of it was to synchronise cuda with CPU and then run the profiler.

(This took wayyy more than just 31 mins, I had to install and set up WSL to run torch.compile because pytorch’s backend uses triton which doesn’t support windows).

DevLog 1.1

• Fixed Bugs
• Wrote a training ipynb file so I can train the model on google colab or kaggle
• Tried different hyperparameters for better generalization and training efficiency
• Edited .gitignore

First off, I needed to understand how ResNet worked so I read the paper: https://arxiv.org/abs/1512.03385

After reading made a plan and visual to follow and implement in the code, starting off simple I choose to implement ResNet-18.

In code:

• Downloaded Cifar-10 (this took a long time somewhy)
• Defined the Residual Blocks
• Encapsulated the Resnet Model
• Wrote training script and some utility functions
• Tested inference and shape match