Dropping the plan

Self-fish Devlog #4

Heyy gng, since the past few days I have been just editing and reviewing images and they don’t seem to finish any time soon but stardance is finishing soon and I want to finish this project before that so I am dropping the workplan of using a classification model for identifying items

That approach was flawed, I had not thought about that earlier instead I was biased by my previous successful project using the classification approach to identify pokemon

I think a better approach would be to use an ocr and then ask an llm to parse that into brand name, item name and weight and other things

Despite the my orignal plan failing, I want to highlight that engineering, in all forms, is itterative, no one finds the solution to the problem in their first try and the things I have made before, ie- my scraper, is not obselete at all, it works so good, like it scraped 150k images in 6 hours, that’s really insane and very useful too, I will make it as another seperate project in the future cuz I think it deserves some more recognition and will improve it too

first 3 images- images of clean dataset, products are cropped and all noises are deleted
last image - for comparison, an unclean dataset, a lot of noises in it like people and some images dont even have the product

Who is Who - Self-Fish devlog #3

There were so many duplicate images

So in the last devlog, I told my scraper scraped nearly 150k images, but when I began cleaning them I noticed a lot of images were exact duplicates, I thought it was just for a class or two but It was for every single one of them , except Oreo

After manually cleaning 14 classes with duplicates, I gave up, and decided to use python to delete duplicates, I used hashlib and md5 hash to detect exact duplicate images inside all folders of my root train images folder which each represent one class, and I used os.remove to delete those duplicates.

I ended up deleting around 34k images which brings my dataset from 150k to 100k images, but this would change too cuz the 100k maybe unique but they still have a lot of discrepencies, like random products and all.

I also used matplotlib to plot a graph of the images remaining, and the results dont look good for a few classes which have now less than 300 images, others are in range 500 - 800, remember they were around 1100 images earlier, so around 800 images were duplicates of the remaining 300.

I know what’s causing this, multiple search queries, since google images only displays a fixed 400 images per search, I made my scraper in a way that it searches for the same product multiple times but with a different query which causes popular images to be duplicates

One more thing, the dataset would not only decrease in number, but after cleaning the images, I would augment them which would drastically increase the number of images and also make the model less likely to overfit.

Also the graph is really ugly cuz there are like 130 classes on the x axis and they dont really fit well and I am not an expert in pyplot so I dont know how to make it look good

Self-Fish Devlog #2

My Scraper is on steroids !!!

I scraped 150,000 images

Yooooo guys the scraper worked too good now, it scraped roughly 150,000 images in just 6 hours, that is insane, the speed was around 7 images per second or 420 images per minute, that is really crazy, my old scraper took a full day for 75k images, this is like a million times faster!!!

In the last devlog, I acheived a speed of around 1079 images in 10 minutes or around 100 images per hour, so since then I have made it 4x times faster.

Here’s what I did:

• Added premature scrolling - earlier the scraper would miss a lot of images or would wait for a long period of time for the images to load and then scroll, but now as soon as the main div appears on the screen, the scraper scrolls down a bit triggering lazy load instantly and then scrolls down to the bottom triggering te lazy load 2 times in a cycle, this made the whole scrolling thing so much quicker.

• Fixed downloads not cancelling - I am storing some extra urls so that if one or more of the urls failed to download, I have backup for completing the target, but I am using a ThreadPoolExecutor to download images concurrently, when it reaches the target the executor used to just break but this didnt prevent it from downloading all other images stored in my array even tho they aren’t saved on the disc, this caused the scraper to wait indefinitely for a long time, since it now has to fetch double or sometimes more the amount, but I have now fixed that

Next steps?

I will now clean the dataset, and since by cleaning the dataset I am actively contributing to the completion of the project, I have got permission to use lapse to record that time

Self-Fish Devlog #2

My Scraper is on steroids !!!

I scraped 150,000 images

Yooooo guys the scraper worked too good now, it scraped roughly 150,000 images in just 6 hours, that is insane, the speed was around 7 images per second or 420 images per minute, that is really crazy, my old scraper took a full day for 75k images, this is like a million times faster!!!

In the last devlog, I acheived a speed of around 1079 images in 10 minutes or around 100 images per hour, so since then I have made it 4x times faster.

Here’s what I did:

• Added premature scrolling - earlier the scraper would miss a lot of images or would wait for a long period of time for the images to load and then scroll, but now as soon as the main div appears on the screen, the scraper scrolls down a bit triggering lazy load instantly and then scrolls down to the bottom triggering te lazy load 2 times in a cycle, this made the whole scrolling thing so much quicker.

• Fixed downloads not cancelling - I am storing some extra urls so that if one or more of the urls failed to download, I have backup for completing the target, but I am using a ThreadPoolExecutor to download images concurrently, when it reaches the target the executor used to just break but this didnt prevent it from downloading all other images stored in my array even tho they aren’t saved on the disc, this caused the scraper to wait indefinitely for a long time, since it now has to fetch double or sometimes more the amount, but I have now fixed that

Next steps?

I will now clean the dataset, and since by cleaning the dataset I am actively contributing to the completion of the project, I have got permission to use lapse to record that time

Poke-tabs Devlog

Hey everyone!!!

I made poke-tabs, my very own custom new tab layout with Pokemons!!!

It has the following features:

• A large Clock showing the current time!
• A random pokemon roll that gives you your very own pokemon with a very cool animation, be sure to check that out, it looks really cool!!!
• A shortcut section that every new tab has
• A guess the pokemon game
• The flavor text of your pokemon

The entire project was made in react and uses pokeapi to get pokemon data and animated + high quality sprites!!!

Our Site

While everyone is busy making their personal site, I made the site of the State, This isn’t your average personal site, the state owns this one.

It is pretty basic, but I wanted some un-convential design, I do not like the plain and minimalistic designs that every site has now a days, they all look and feel the same and lack a character so that’s why I went for this design, it’s still pretty basic and gives off the 2000s vibe but I love it.

As much as I love using react, the mission demanded for a html and css site, and it’s been years since I last wrote something in html so I decided to give it a try, It felt really nostalgic and I enjoyed it too.

I actually really love the color scheme, it feels very warm.

Since the site was pretty simple, I have completed it quickly, I dont think there is anything else I would like to add now so I will ship after this devlog even tho it’s the first one!!!

Shelf-Fish Devlog #1

What is Shelf-Fish?

It is a computer vision based cross platform inventory management and billing system that I am making with react native as frontend and probably a fastapi backend.

How will the project work?

The project would be made in several phases that are listed below:

• Phase 01 - Scraper

  In this phase I will make a fast scraper with selenium that would scrape 1000-1500 images for over 130 grocery items that I have listed.

• Phase 02 - Training

  In this phase I would clean the dataset and train the model!!

• Phase 03 - Backend

  I would build a basic cors backend in fastapi and add some user relations

• Phase 04 - Frontend

  This phase involves building a cross platform fronend with react native.

• Phase 05 - ESP 32

  This one is not confirmed yet, but if i get the time, instead of using a phone, I would use the esp 32 and esp32 cam to register objects and create bills, or perhaps I should make this the main focus of the project and make this a hardware project, I am still not sure.

What have I done till now?

I am modifying my old scraper that I used for my previous pokedex project, which was very very slow, I have changed it completely, It now directly goes to the image search url instead of going to google.images.com and then manually typing the words. It now also downloads images concurrently using the ThreadPoolExecutor, earlier this part was the slowest but in just 2 seconds, it downloads 125 images,

But it is still pretty slow, It took over 10 min to scrape 1079 images, and it was a bit short of the 1125 target images which is due to slow network that caused the images to not load. For the next devlog I will try to make it sub 8 min or faster

Cosmidex App is now finished !!!!!

Finally, after 2 months, the app is finished and I am ready to ship!!!

Here’s what I have added :

• Added Guess the Pokemon, It is addictive as hell, I used the imageTint property of the React native’s Image tag to make the whole silhouette of the pokemon black. After the user guesses it, it displays the orignal pokemon.

• Added cries, everytime you open the pokemon page for a pokemon, it’s latest or legacy cry plays once, you can toggle to not play it using the button on top right corner of the pokemon image.

• Added shiny, you can now toggle to show shiny or default sprites of the pokemon.

That’s it, the project is finished :(

I always want to add more, but I have to study for entrances.

Cosmidex App is now finished !!!!!

Finally, after 2 months, the app is finished and I am ready to ship!!!

Here’s what I have added :

• Added Guess the Pokemon, It is addictive as hell, I used the imageTint property of the React native’s Image tag to make the whole silhouette of the pokemon black. After the user guesses it, it displays the orignal pokemon.

• Added cries, everytime you open the pokemon page for a pokemon, it’s latest or legacy cry plays once, you can toggle to not play it using the button on top right corner of the pokemon image.

• Added shiny, you can now toggle to show shiny or default sprites of the pokemon.

That’s it, the project is finished :(

I always want to add more, but I have to study for entrances.

You can now get a Medal!!!

Cosmidex Devlog #8

You will now get a medal everytime you catch all 151 pokemon in Generation 1!!!

I have added the catch mechanishm

Everytime you successfully predict a pokemon using your camera, gallery or from the Not Confident page, that pokemon would be added to your caught list. The caught list is stored in Local storage.

Beside the catch mechanism here are other changes:

• Fixed the camera not closing issue on web by starting the Camera only when the Camera page is focused, otherwise closing it.
• Removed most of the unnecessay console.log()
• On android, the Tab was covering pokemon info on the Pokemon Page, to fix that, I made the tab shake toggled on that page, the user now needs to shake their
• Changed the Home page of android to match that of the web.
• Added a spawn tab that shows spawn locations along with methods to catch in the generation 1 games, ie- Pokemon Red, Blue and Yellow.
• Added the README file.
• Deployed the project on vercel properly, The project is now live on this link

What’s next?

• I now want to add a daily Who’s that pokemon? Game, that would be so fun i guess.
• I also will add pokemon cries and show shiny versions of pokemons.

You can now get a Medal!!!

Cosmidex Devlog #8

You will now get a medal everytime you catch all 151 pokemon in Generation 1!!!

I have added the catch mechanishm

Everytime you successfully predict a pokemon using your camera, gallery or from the Not Confident page, that pokemon would be added to your caught list. The caught list is stored in Local storage.

Beside the catch mechanism here are other changes:

• Fixed the camera not closing issue on web by starting the Camera only when the Camera page is focused, otherwise closing it.
• Removed most of the unnecessay console.log()
• On android, the Tab was covering pokemon info on the Pokemon Page, to fix that, I made the tab shake toggled on that page, the user now needs to shake their
• Changed the Home page of android to match that of the web.
• Added a spawn tab that shows spawn locations along with methods to catch in the generation 1 games, ie- Pokemon Red, Blue and Yellow.
• Added the README file.
• Deployed the project on vercel properly, The project is now live on this link

What’s next?

• I now want to add a daily Who’s that pokemon? Game, that would be so fun i guess.
• I also will add pokemon cries and show shiny versions of pokemons.

Cosmidex Web app is complete !!!

Devlog #7

The web version of the Cosmidex app is also finished, FINALLYYY!!!

Here is what I have added:

• Converted the tflite model to TFJS, took a lot of time, many libraries were deprecated, so I had to manually remove imports for those.
• Created seperate web components using .web.js to upload/capture images -> preprocess -> predict, my codebase for both the android and the web version of the app is still the same.
• Created seperate page components for the web and changed the design for the Pokemon Page and the Home page.

What’s left?

The app is mostly ready for it’s first ship, I have hit the 10 hr mark thats why I am uploading a devlog early, but there’s not much left to do, I know I have been saying this for the last 2 devlogs. Here’s what is left:

• Remove all the console logs that still linger in components for debug.
• The camera doesn’t close in the web version after capturing and predicting so I have to fix that.
• Probably change the home page for the android version to match the web.
• Add some cool animations to it.
• At last I want to push this project one bit further and add a capturing feature, like you capture a photo of a pokemon and it gets added to your caught pokemon in the pokedex and after capturing all 151 pokemon, you get a medal.

Cosmidex Web app is complete !!!

Devlog #7

The web version of the Cosmidex app is also finished, FINALLYYY!!!

Here is what I have added:

• Converted the tflite model to TFJS, took a lot of time, many libraries were deprecated, so I had to manually remove imports for those.
• Created seperate web components using .web.js to upload/capture images -> preprocess -> predict, my codebase for both the android and the web version of the app is still the same.
• Created seperate page components for the web and changed the design for the Pokemon Page and the Home page.

What’s left?

The app is mostly ready for it’s first ship, I have hit the 10 hr mark thats why I am uploading a devlog early, but there’s not much left to do, I know I have been saying this for the last 2 devlogs. Here’s what is left:

• Remove all the console logs that still linger in components for debug.
• The camera doesn’t close in the web version after capturing and predicting so I have to fix that.
• Probably change the home page for the android version to match the web.
• Add some cool animations to it.
• At last I want to push this project one bit further and add a capturing feature, like you capture a photo of a pokemon and it gets added to your caught pokemon in the pokedex and after capturing all 151 pokemon, you get a medal.

Pokedex Mobile app is finally finished

Pokedex Devlog #6

The Pokedex mobile app is now finished and I am working on finishing the web version of the app too.

Here is what I have added:

• A NotConfident page that is displayed when the model can’t give a solid prediction about the image.
• A ’AllPokemon` page that displays all 151 pokemons from generation 1 in a flatList.
• I was using a a very messy state based navigation so I updated it to a Stack navigation.
• I also add a tab at the bottom of the Pokemon, Not confident and All Pokemon pages, though it sometimes overlaps the page content like that of the mewtwo picture.
• Create a seperate AppContext for the web version to load the model in the web.

What’s left in the project before the ship?

• The Mobile app is finished, I just have to make the Camera and Upload components for the web version of the app and then the project would officially be completed.

I was having my exams so I wasn’t able to code for a while :(

Pokedex Mobile app is finally finished

Pokedex Devlog #6

The Pokedex mobile app is now finished and I am working on finishing the web version of the app too.

Here is what I have added:

• A NotConfident page that is displayed when the model can’t give a solid prediction about the image.
• A ’AllPokemon` page that displays all 151 pokemons from generation 1 in a flatList.
• I was using a a very messy state based navigation so I updated it to a Stack navigation.
• I also add a tab at the bottom of the Pokemon, Not confident and All Pokemon pages, though it sometimes overlaps the page content like that of the mewtwo picture.
• Create a seperate AppContext for the web version to load the model in the web.

What’s left in the project before the ship?

• The Mobile app is finished, I just have to make the Camera and Upload components for the web version of the app and then the project would officially be completed.

I was having my exams so I wasn’t able to code for a while :(

Pokedex App is finally complete (mostly)

Devlog #5

The app is now mostly complete, this phase of the project went very smoothly.

Here is what I have added:

• After predicting the pokemon, The app sends a request to PokeAPI at the https://pokeapi.co/api/v2/{pokemon_name} endpoint and gets the detailed information of the pokemon.

• The information is displayed on a pokemon page which includes basic stats, the type of the pokemon, height , wight et.

• I have also worked a lot of ui, majority of the time was spent in designing the ui. However the home page still looks empty, it is just a div with ImageBackground and a logo with two buttons and 1 scanner icon that I designed in canva.

• I have also added typed backgrounds, so that every pokemon type has a seperate space themed background, they are mostly nebula images that i found in the NASA Gallery but boyyyy do i like the themes, they look soooo gooood!!!. I am so proud of myself….

What’s left?

While the app is mostly ready, there a few things that I still need/want to add:

• A page displaying all the 151 pokemons in generation 1
• A page that is displayed when there is no confident prediction
• A web version of the app, which would be pretty easy, I just need to change the uploading and capturing logic for it to work on web

So hopefully this is my second last devlog for this project, stay tuned !!!!

Pokedex App is finally complete (mostly)

Devlog #5

The app is now mostly complete, this phase of the project went very smoothly.

Here is what I have added:

• After predicting the pokemon, The app sends a request to PokeAPI at the https://pokeapi.co/api/v2/{pokemon_name} endpoint and gets the detailed information of the pokemon.

• The information is displayed on a pokemon page which includes basic stats, the type of the pokemon, height , wight et.

• I have also worked a lot of ui, majority of the time was spent in designing the ui. However the home page still looks empty, it is just a div with ImageBackground and a logo with two buttons and 1 scanner icon that I designed in canva.

• I have also added typed backgrounds, so that every pokemon type has a seperate space themed background, they are mostly nebula images that i found in the NASA Gallery but boyyyy do i like the themes, they look soooo gooood!!!. I am so proud of myself….

What’s left?

While the app is mostly ready, there a few things that I still need/want to add:

• A page displaying all the 151 pokemons in generation 1
• A page that is displayed when there is no confident prediction
• A web version of the app, which would be pretty easy, I just need to change the uploading and capturing logic for it to work on web

So hopefully this is my second last devlog for this project, stay tuned !!!!