Self-Fish Devlog #2

My Scraper is on steroids !!!

I scraped 150,000 images

Yooooo guys the scraper worked too good now, it scraped roughly 150,000 images in just 6 hours, that is insane, the speed was around 7 images per second or 420 images per minute, that is really crazy, my old scraper took a full day for 75k images, this is like a million times faster!!!

In the last devlog, I acheived a speed of around 1079 images in 10 minutes or around 100 images per hour, so since then I have made it 4x times faster.

Here’s what I did:

• Added premature scrolling - earlier the scraper would miss a lot of images or would wait for a long period of time for the images to load and then scroll, but now as soon as the main div appears on the screen, the scraper scrolls down a bit triggering lazy load instantly and then scrolls down to the bottom triggering te lazy load 2 times in a cycle, this made the whole scrolling thing so much quicker.

• Fixed downloads not cancelling - I am storing some extra urls so that if one or more of the urls failed to download, I have backup for completing the target, but I am using a ThreadPoolExecutor to download images concurrently, when it reaches the target the executor used to just break but this didnt prevent it from downloading all other images stored in my array even tho they aren’t saved on the disc, this caused the scraper to wait indefinitely for a long time, since it now has to fetch double or sometimes more the amount, but I have now fixed that

Next steps?

I will now clean the dataset, and since by cleaning the dataset I am actively contributing to the completion of the project, I have got permission to use lapse to record that time

Self-Fish Devlog #2

My Scraper is on steroids !!!

I scraped 150,000 images

Yooooo guys the scraper worked too good now, it scraped roughly 150,000 images in just 6 hours, that is insane, the speed was around 7 images per second or 420 images per minute, that is really crazy, my old scraper took a full day for 75k images, this is like a million times faster!!!

In the last devlog, I acheived a speed of around 1079 images in 10 minutes or around 100 images per hour, so since then I have made it 4x times faster.

Here’s what I did:

• Added premature scrolling - earlier the scraper would miss a lot of images or would wait for a long period of time for the images to load and then scroll, but now as soon as the main div appears on the screen, the scraper scrolls down a bit triggering lazy load instantly and then scrolls down to the bottom triggering te lazy load 2 times in a cycle, this made the whole scrolling thing so much quicker.

• Fixed downloads not cancelling - I am storing some extra urls so that if one or more of the urls failed to download, I have backup for completing the target, but I am using a ThreadPoolExecutor to download images concurrently, when it reaches the target the executor used to just break but this didnt prevent it from downloading all other images stored in my array even tho they aren’t saved on the disc, this caused the scraper to wait indefinitely for a long time, since it now has to fetch double or sometimes more the amount, but I have now fixed that

Next steps?

I will now clean the dataset, and since by cleaning the dataset I am actively contributing to the completion of the project, I have got permission to use lapse to record that time

Poke-tabs Devlog

Hey everyone!!!

I made poke-tabs, my very own custom new tab layout with Pokemons!!!

It has the following features:

• A large Clock showing the current time!
• A random pokemon roll that gives you your very own pokemon with a very cool animation, be sure to check that out, it looks really cool!!!
• A shortcut section that every new tab has
• A guess the pokemon game
• The flavor text of your pokemon

The entire project was made in react and uses pokeapi to get pokemon data and animated + high quality sprites!!!

Our Site

While everyone is busy making their personal site, I made the site of the State, This isn’t your average personal site, the state owns this one.

It is pretty basic, but I wanted some un-convential design, I do not like the plain and minimalistic designs that every site has now a days, they all look and feel the same and lack a character so that’s why I went for this design, it’s still pretty basic and gives off the 2000s vibe but I love it.

As much as I love using react, the mission demanded for a html and css site, and it’s been years since I last wrote something in html so I decided to give it a try, It felt really nostalgic and I enjoyed it too.

I actually really love the color scheme, it feels very warm.

Since the site was pretty simple, I have completed it quickly, I dont think there is anything else I would like to add now so I will ship after this devlog even tho it’s the first one!!!

Shelf-Fish Devlog #1

What is Shelf-Fish?

It is a computer vision based cross platform inventory management and billing system that I am making with react native as frontend and probably a fastapi backend.

How will the project work?

The project would be made in several phases that are listed below:

• Phase 01 - Scraper

  In this phase I will make a fast scraper with selenium that would scrape 1000-1500 images for over 130 grocery items that I have listed.

• Phase 02 - Training

  In this phase I would clean the dataset and train the model!!

• Phase 03 - Backend

  I would build a basic cors backend in fastapi and add some user relations

• Phase 04 - Frontend

  This phase involves building a cross platform fronend with react native.

• Phase 05 - ESP 32

  This one is not confirmed yet, but if i get the time, instead of using a phone, I would use the esp 32 and esp32 cam to register objects and create bills, or perhaps I should make this the main focus of the project and make this a hardware project, I am still not sure.

What have I done till now?

I am modifying my old scraper that I used for my previous pokedex project, which was very very slow, I have changed it completely, It now directly goes to the image search url instead of going to google.images.com and then manually typing the words. It now also downloads images concurrently using the ThreadPoolExecutor, earlier this part was the slowest but in just 2 seconds, it downloads 125 images,

But it is still pretty slow, It took over 10 min to scrape 1079 images, and it was a bit short of the 1125 target images which is due to slow network that caused the images to not load. For the next devlog I will try to make it sub 8 min or faster

Cosmidex App is now finished !!!!!

Finally, after 2 months, the app is finished and I am ready to ship!!!

Here’s what I have added :

• Added Guess the Pokemon, It is addictive as hell, I used the imageTint property of the React native’s Image tag to make the whole silhouette of the pokemon black. After the user guesses it, it displays the orignal pokemon.

• Added cries, everytime you open the pokemon page for a pokemon, it’s latest or legacy cry plays once, you can toggle to not play it using the button on top right corner of the pokemon image.

• Added shiny, you can now toggle to show shiny or default sprites of the pokemon.

That’s it, the project is finished :(

I always want to add more, but I have to study for entrances.

Cosmidex App is now finished !!!!!

Finally, after 2 months, the app is finished and I am ready to ship!!!

Here’s what I have added :

• Added Guess the Pokemon, It is addictive as hell, I used the imageTint property of the React native’s Image tag to make the whole silhouette of the pokemon black. After the user guesses it, it displays the orignal pokemon.

• Added cries, everytime you open the pokemon page for a pokemon, it’s latest or legacy cry plays once, you can toggle to not play it using the button on top right corner of the pokemon image.

• Added shiny, you can now toggle to show shiny or default sprites of the pokemon.

That’s it, the project is finished :(

I always want to add more, but I have to study for entrances.

You can now get a Medal!!!

Cosmidex Devlog #8

You will now get a medal everytime you catch all 151 pokemon in Generation 1!!!

I have added the catch mechanishm

Everytime you successfully predict a pokemon using your camera, gallery or from the Not Confident page, that pokemon would be added to your caught list. The caught list is stored in Local storage.

Beside the catch mechanism here are other changes:

• Fixed the camera not closing issue on web by starting the Camera only when the Camera page is focused, otherwise closing it.
• Removed most of the unnecessay console.log()
• On android, the Tab was covering pokemon info on the Pokemon Page, to fix that, I made the tab shake toggled on that page, the user now needs to shake their
• Changed the Home page of android to match that of the web.
• Added a spawn tab that shows spawn locations along with methods to catch in the generation 1 games, ie- Pokemon Red, Blue and Yellow.
• Added the README file.
• Deployed the project on vercel properly, The project is now live on this link

What’s next?

• I now want to add a daily Who’s that pokemon? Game, that would be so fun i guess.
• I also will add pokemon cries and show shiny versions of pokemons.

You can now get a Medal!!!

Cosmidex Devlog #8

You will now get a medal everytime you catch all 151 pokemon in Generation 1!!!

I have added the catch mechanishm

Everytime you successfully predict a pokemon using your camera, gallery or from the Not Confident page, that pokemon would be added to your caught list. The caught list is stored in Local storage.

Beside the catch mechanism here are other changes:

• Fixed the camera not closing issue on web by starting the Camera only when the Camera page is focused, otherwise closing it.
• Removed most of the unnecessay console.log()
• On android, the Tab was covering pokemon info on the Pokemon Page, to fix that, I made the tab shake toggled on that page, the user now needs to shake their
• Changed the Home page of android to match that of the web.
• Added a spawn tab that shows spawn locations along with methods to catch in the generation 1 games, ie- Pokemon Red, Blue and Yellow.
• Added the README file.
• Deployed the project on vercel properly, The project is now live on this link

What’s next?

• I now want to add a daily Who’s that pokemon? Game, that would be so fun i guess.
• I also will add pokemon cries and show shiny versions of pokemons.

Cosmidex Web app is complete !!!

Devlog #7

The web version of the Cosmidex app is also finished, FINALLYYY!!!

Here is what I have added:

• Converted the tflite model to TFJS, took a lot of time, many libraries were deprecated, so I had to manually remove imports for those.
• Created seperate web components using .web.js to upload/capture images -> preprocess -> predict, my codebase for both the android and the web version of the app is still the same.
• Created seperate page components for the web and changed the design for the Pokemon Page and the Home page.

What’s left?

The app is mostly ready for it’s first ship, I have hit the 10 hr mark thats why I am uploading a devlog early, but there’s not much left to do, I know I have been saying this for the last 2 devlogs. Here’s what is left:

• Remove all the console logs that still linger in components for debug.
• The camera doesn’t close in the web version after capturing and predicting so I have to fix that.
• Probably change the home page for the android version to match the web.
• Add some cool animations to it.
• At last I want to push this project one bit further and add a capturing feature, like you capture a photo of a pokemon and it gets added to your caught pokemon in the pokedex and after capturing all 151 pokemon, you get a medal.

Cosmidex Web app is complete !!!

Devlog #7

The web version of the Cosmidex app is also finished, FINALLYYY!!!

Here is what I have added:

• Converted the tflite model to TFJS, took a lot of time, many libraries were deprecated, so I had to manually remove imports for those.
• Created seperate web components using .web.js to upload/capture images -> preprocess -> predict, my codebase for both the android and the web version of the app is still the same.
• Created seperate page components for the web and changed the design for the Pokemon Page and the Home page.

What’s left?

The app is mostly ready for it’s first ship, I have hit the 10 hr mark thats why I am uploading a devlog early, but there’s not much left to do, I know I have been saying this for the last 2 devlogs. Here’s what is left:

• Remove all the console logs that still linger in components for debug.
• The camera doesn’t close in the web version after capturing and predicting so I have to fix that.
• Probably change the home page for the android version to match the web.
• Add some cool animations to it.
• At last I want to push this project one bit further and add a capturing feature, like you capture a photo of a pokemon and it gets added to your caught pokemon in the pokedex and after capturing all 151 pokemon, you get a medal.

Pokedex Mobile app is finally finished

Pokedex Devlog #6

The Pokedex mobile app is now finished and I am working on finishing the web version of the app too.

Here is what I have added:

• A NotConfident page that is displayed when the model can’t give a solid prediction about the image.
• A ’AllPokemon` page that displays all 151 pokemons from generation 1 in a flatList.
• I was using a a very messy state based navigation so I updated it to a Stack navigation.
• I also add a tab at the bottom of the Pokemon, Not confident and All Pokemon pages, though it sometimes overlaps the page content like that of the mewtwo picture.
• Create a seperate AppContext for the web version to load the model in the web.

What’s left in the project before the ship?

• The Mobile app is finished, I just have to make the Camera and Upload components for the web version of the app and then the project would officially be completed.

I was having my exams so I wasn’t able to code for a while :(

Pokedex Mobile app is finally finished

Pokedex Devlog #6

The Pokedex mobile app is now finished and I am working on finishing the web version of the app too.

Here is what I have added:

• A NotConfident page that is displayed when the model can’t give a solid prediction about the image.
• A ’AllPokemon` page that displays all 151 pokemons from generation 1 in a flatList.
• I was using a a very messy state based navigation so I updated it to a Stack navigation.
• I also add a tab at the bottom of the Pokemon, Not confident and All Pokemon pages, though it sometimes overlaps the page content like that of the mewtwo picture.
• Create a seperate AppContext for the web version to load the model in the web.

What’s left in the project before the ship?

• The Mobile app is finished, I just have to make the Camera and Upload components for the web version of the app and then the project would officially be completed.

I was having my exams so I wasn’t able to code for a while :(

Pokedex App is finally complete (mostly)

Devlog #5

The app is now mostly complete, this phase of the project went very smoothly.

Here is what I have added:

• After predicting the pokemon, The app sends a request to PokeAPI at the https://pokeapi.co/api/v2/{pokemon_name} endpoint and gets the detailed information of the pokemon.

• The information is displayed on a pokemon page which includes basic stats, the type of the pokemon, height , wight et.

• I have also worked a lot of ui, majority of the time was spent in designing the ui. However the home page still looks empty, it is just a div with ImageBackground and a logo with two buttons and 1 scanner icon that I designed in canva.

• I have also added typed backgrounds, so that every pokemon type has a seperate space themed background, they are mostly nebula images that i found in the NASA Gallery but boyyyy do i like the themes, they look soooo gooood!!!. I am so proud of myself….

What’s left?

While the app is mostly ready, there a few things that I still need/want to add:

• A page displaying all the 151 pokemons in generation 1
• A page that is displayed when there is no confident prediction
• A web version of the app, which would be pretty easy, I just need to change the uploading and capturing logic for it to work on web

So hopefully this is my second last devlog for this project, stay tuned !!!!

Pokedex App is finally complete (mostly)

Devlog #5

The app is now mostly complete, this phase of the project went very smoothly.

Here is what I have added:

• After predicting the pokemon, The app sends a request to PokeAPI at the https://pokeapi.co/api/v2/{pokemon_name} endpoint and gets the detailed information of the pokemon.

• The information is displayed on a pokemon page which includes basic stats, the type of the pokemon, height , wight et.

• I have also worked a lot of ui, majority of the time was spent in designing the ui. However the home page still looks empty, it is just a div with ImageBackground and a logo with two buttons and 1 scanner icon that I designed in canva.

• I have also added typed backgrounds, so that every pokemon type has a seperate space themed background, they are mostly nebula images that i found in the NASA Gallery but boyyyy do i like the themes, they look soooo gooood!!!. I am so proud of myself….

What’s left?

While the app is mostly ready, there a few things that I still need/want to add:

• A page displaying all the 151 pokemons in generation 1
• A page that is displayed when there is no confident prediction
• A web version of the app, which would be pretty easy, I just need to change the uploading and capturing logic for it to work on web

So hopefully this is my second last devlog for this project, stay tuned !!!!

Pokedex Devlog #4

Heyyyyy Everyone!!!

The training of the model is finally completed and I have made the first version of the Pokedex app, It captures an image using react-native-vision-camera, crops only the middle portion, resizes the image to match my model’s image size of 224*224 px, then gets the raw pixelbuffer, converts into rgb and normalises the values from 0-255 to 0-1.0. This normalised pixel data is then fed to the model which predicts the class of the image. There were a lot of problems in making the app and there are still lot more to come , but here is everything I did:

1. An Ironical Training

After collecting 75,000 images from my custom scraper, I zipped all of those images, and uploaded them directly to my colab notebook, uploading them took half an hour though😫 . I ran the training script at 50 epochs and training the model took another 8 hours . It was really ironical to see the model train in google colab on the same images that google was trying to hide from my little scraper.

2. A Successful Model

After the training was completed, I had to test the model to see how it performed and whether I need to retrain it. The results shocked me, I gave the model 12 images from pintrest that it had never seen before and it predicted all of them correctly. The accuracy was way better than what I expect from a model trained on raw data. However, if needed I will clean the dataset and train again.

3. React native Camera Hell

Everything was going very smoothly at this point, but I was never prepared for the next phase. I built a blank react native project, installed react-native-vision-camera and added a camera to the app.js, the camera was working, great, so I built a function to capture the photo and display it on the screen, that too worked. But the next step was to process the raw image pixel data into the format that my model required.

I searched online, how to resize an image in react-native-vision-camera, the ai results told me to use the resizer package that comes with it. I spent three, three whole days!!! figuring why the resizer was not able to resize the image only to realise that it took a frame output not an image, so I rewrote the whole camera code again to use a live stream instead of capturing a photo. The resizer was working, I then normalized the pixel values while converting it into rgb format and fed the data to my model which outputed complete gibberish, all that work for nothing!!! 😭 I tried logging in the shape of input and output of the model, and the pixelformat of the bufffer and the raw pixel data. But all those were giving either undefined or 0. Nothing seemed to be working at this about, there was something seriously wrong with how I was processing the images. But then going through the docs of react-native-vision-camera, I saw that the image object that my previous capturing images code, that i had dumped, was outputting was a react-native-nitro-image object and it had an inbuilt resizeAsync function.

I got frustrated again and rewrote the entire code again, this time with the previous capture photo logic. After cropping, resizing and getting the rgb data in to the required format, I tested the app with a photo of Gloom displayed on my laptop screen and it predicted it correctly. I was sooooo happyyy that it worked and is still working tho the accuracy is low due to pixel noises.

What’s next?

The hardest parts of the project are completed now, I just need to an api call, after the model predicts the pokemon to, get the details of the pokemon, like it’s type, pokedex entry and other things. The UI also sucks so I will work on that, I think I want to make it space theme, I don’t know how it would look but I will try. Till then… stay tuned!!!

Pokedex Devlog #4

Heyyyyy Everyone!!!

The training of the model is finally completed and I have made the first version of the Pokedex app, It captures an image using react-native-vision-camera, crops only the middle portion, resizes the image to match my model’s image size of 224*224 px, then gets the raw pixelbuffer, converts into rgb and normalises the values from 0-255 to 0-1.0. This normalised pixel data is then fed to the model which predicts the class of the image. There were a lot of problems in making the app and there are still lot more to come , but here is everything I did:

1. An Ironical Training

After collecting 75,000 images from my custom scraper, I zipped all of those images, and uploaded them directly to my colab notebook, uploading them took half an hour though😫 . I ran the training script at 50 epochs and training the model took another 8 hours . It was really ironical to see the model train in google colab on the same images that google was trying to hide from my little scraper.

2. A Successful Model

After the training was completed, I had to test the model to see how it performed and whether I need to retrain it. The results shocked me, I gave the model 12 images from pintrest that it had never seen before and it predicted all of them correctly. The accuracy was way better than what I expect from a model trained on raw data. However, if needed I will clean the dataset and train again.

3. React native Camera Hell

Everything was going very smoothly at this point, but I was never prepared for the next phase. I built a blank react native project, installed react-native-vision-camera and added a camera to the app.js, the camera was working, great, so I built a function to capture the photo and display it on the screen, that too worked. But the next step was to process the raw image pixel data into the format that my model required.

I searched online, how to resize an image in react-native-vision-camera, the ai results told me to use the resizer package that comes with it. I spent three, three whole days!!! figuring why the resizer was not able to resize the image only to realise that it took a frame output not an image, so I rewrote the whole camera code again to use a live stream instead of capturing a photo. The resizer was working, I then normalized the pixel values while converting it into rgb format and fed the data to my model which outputed complete gibberish, all that work for nothing!!! 😭 I tried logging in the shape of input and output of the model, and the pixelformat of the bufffer and the raw pixel data. But all those were giving either undefined or 0. Nothing seemed to be working at this about, there was something seriously wrong with how I was processing the images. But then going through the docs of react-native-vision-camera, I saw that the image object that my previous capturing images code, that i had dumped, was outputting was a react-native-nitro-image object and it had an inbuilt resizeAsync function.

I got frustrated again and rewrote the entire code again, this time with the previous capture photo logic. After cropping, resizing and getting the rgb data in to the required format, I tested the app with a photo of Gloom displayed on my laptop screen and it predicted it correctly. I was sooooo happyyy that it worked and is still working tho the accuracy is low due to pixel noises.

What’s next?

The hardest parts of the project are completed now, I just need to an api call, after the model predicts the pokemon to, get the details of the pokemon, like it’s type, pokedex entry and other things. The UI also sucks so I will work on that, I think I want to make it space theme, I don’t know how it would look but I will try. Till then… stay tuned!!!