By
@Ash-ioL
A web app that allows users to snap a picture of a number and get a prediction on what number the number is. Original purpose is to demonstrate machine learning.
@Ash-ioL
on
Handwritten Digit Prediction (MNIST) Webpage
SHIPWRIGHTS: Please read the instructions below 🙏❤️. And please take the time to try this out fully; I promise you’ll like it :D
I made a web app with a frontend and backend that demonstrates an AI model that predicts numbers.
The most challenging part was preprocessing the image, there were so many conditions like shadows, blur, center, size, and properties of the input image that needed to be nullified as much as possible.
I am proud of everything honestly… not much to say. But I am especially proud of how usable it is. I had a couple friends test it out and they said it was really friendly with all the guides and facts inside the webpage, and it worked like a charm.
Shipwrights please read below:
Instructions to try:
Grab a quick white sheet of paper and with a black marker or pen, write a random digit from 0 to 9. Make sure the digit is relatively thick.
Now here’s the tricky part. It’s a bit easier on phone. If your trying it out on a PC, you might have to take a picture on your phone and transfer the image to your PC and upload it in the website. But on phone, you can take a picture directly on the website.
When taking a picture, make sure you have distanced the number from the camera so there aren’t any shadows, try to zoom in and keep it centered, focused, and clear.
Once you’ve uploaded the image or have taken the picture in the website, It will load for a bit. You might run into an issue here and it’s not in my control. The backend service I’m using, Render, falls asleep every 15 minutes and will take up to 1 minute to wake. So if you’re confused and its loading for over a minute, refresh the page and retake/reupload the image. Sorry for this inconvienience.
After it’s done loading, it will show you the prediction and the confidence and as well as a couple of facts on how the project works.
Thank you for reading till the end!
@Ash-ioL
on
Handwritten Digit Prediction (MNIST) Webpage
4h 35m 5s logged
I spend the majority of my time in the preprocessing code trying to get it to work. I tried things like finding contours, adaptive threshold, and other things but I ended up just using thresh otsu and multiplying that threshold value by something near 2 and it ended up working out.
After that I was setting up cors allowed origins with the production domain, and I went through all my css and made sure its responsive and supports mobile devices.
My biggest challenge was honestly finding time to do this. I’m taking a heavy classload this year and with other commitments I could only squeeze out like an hour of my afternoon on this every day and I worked on it mostly in class and during lunch. But however it bugged me so much (and still does) of how fragile this model is to shadows and such, although I made it less impactful.
Signing off,
Ash-ioL
@Ash-ioL
on
Handwritten Digit Prediction (MNIST) Webpage
2h 35m 37s logged
I created a simple loading screen to transition into the final result after the image is uploaded. For the loading screen I just put the image they uploaded and underneath it the name of the image they uploaded. For the animated ray, I used a parent container to remove overflow and a pseudo-element to have the moving ray. The biggest challenge was messing around with the perfect amounts of blur, opacity, ray width, and direction to make it look put together.
I also started working on the results UI. It displays the preprocessed image, prediction, confidence score, and some information about the pipeline behind this project. Since I am done with the UI, now I just need to use the fetch api in Javascript to get the response and feed it into the results. And some other polishing.
In the backend, I still need to fix the preprocessing bug but I figured it would be easier to debug if I finish the frontend first since I can use it as a tool to take a picture and see what it gives.
@Ash-ioL
on
Handwritten Digit Prediction (MNIST) Webpage
1h 12m 5s logged
So I added a footer in the bottom with social links and in the top I added a logo, along with a big transparent one in the background. I also made the file input button a html file input tag so it utilizes the file inputting and camera use. I’ve really just been adding stuff and not worrying too much about organization. I’m hoping I’ll be done with the project before that impacts me.
Signing off,
Ash-ioL
@Ash-ioL
on
Handwritten Digit Prediction (MNIST) Webpage
2h 8m 51s logged
I generated the svgs for the icons and the diagrams with AI. I manually did the colors and layout. Next I am going to make it responsive and compatible with phones. After that, I need to fix an issue with the preprocessing of the image in the backend.
Testing it out was kind of a hassle with taking a picture, uploading it to drive, using my laptop to send it to the model. But since I tested it out, I was able to know that something was wrong.
@Ash-ioL
on
Handwritten Digit Prediction (MNIST) Webpage
6h 25m 24s logged
I completed the API part of the project. It allows users send a post request with the image and gives the prediction as the response.
I also have it deployed publicly.
I spent a huge amount of time in the preprocessing step for the image since there were a lot of steps involved and lots of places for things to go wrong.
Started testing with a frontend to make sure it works but I haven’t added any elements to it yet.
Signing off,
Ash-ioL
