ResNet Implementation
- 6 Devlogs
- 6 Total hours
An Image Classifier using the ResNet18 architecture.
Ship #1
Pending review
@Muhammad_Ahmad
on
ResNet Implementation
Made a image classifier model based on the resnet-18 architecture. I trained it on cifar-10 (which is a dataset for 10 classes of images airplanes, cars, trucks, ships, cats, dogs, horses, birds, frogs, deer). The hardest part was to understand the why behind the model’s inductive bias and translating the maths into code I guess. You can check the code out on github and try out the deployed wrapper around the model on hugging face spaces :>
@Muhammad_Ahmad
on
ResNet Implementation
17m 4s logged
Devlog 1.5
- Wrote readme
- Had to commit a couple of times cuz I kept misspelling/misformatting my file names xD.
@Muhammad_Ahmad
on
ResNet Implementation
24m 59s logged
Devlog 1.4
- Added app usage description
- Added examples for quick testing
- Chose a better theme
@Muhammad_Ahmad
on
ResNet Implementation
1h 0m 18s logged
Devlog 1.3
- Made a gradio based wrapper around the model
- Debugged the app deployment on hugging face
- Handled edge cases and refactored the code
@Muhammad_Ahmad
on
ResNet Implementation
31m 26s logged
Devlog 1.2
I mainly focused on profiling the model’s performance, and I found the model’s inference to be taking ~200 ms on GTX 1650. But I compiled the model with torch.compile() which basically fuses all the cuda operations in one big monolithic kernel and my jaw dropped seeing 20x performance boost taking only ~8 ms.
Oh and I also profiled the compiled model wrong the first time and also recorded the lazy jit compilation of the model during the first pass. The solution of it was to synchronise cuda with CPU and then run the profiler.
(This took wayyy more than just 31 mins, I had to install and set up WSL to run torch.compile because pytorch’s backend uses triton which doesn’t support windows).
@Muhammad_Ahmad
on
ResNet Implementation
1h 38m 15s logged
DevLog 1.1
- Fixed Bugs
- Wrote a training ipynb file so I can train the model on google colab or kaggle
- Tried different hyperparameters for better generalization and training efficiency
- Edited .gitignore
@Muhammad_Ahmad
on
ResNet Implementation
2h 9m 3s logged
First off, I needed to understand how ResNet worked so I read the paper: https://arxiv.org/abs/1512.03385
After reading made a plan and visual to follow and implement in the code, starting off simple I choose to implement ResNet-18.
In code:
- Downloaded Cifar-10 (this took a long time somewhy)
- Defined the Residual Blocks
- Encapsulated the Resnet Model
- Wrote training script and some utility functions
- Tested inference and shape match
