Devlog 6: Before I used UMAP to reduce dimensions, but when I was unpickling it from joblib into my normal .py scripts, it started to crash my app and scripts because UMAP is not compatible with python 3.13.2 for some reason. I did not want to downgrade my python, so I switched to PCA, which is another dimension reducer mechanism. PCA is fast, linear, and it preserves global variance. After, I start working on integrating my GUI with it. As you know, the plan is to get a current NEO asteroid and input it through the model to see what group it is, so I created that specific script to do that. Based on the means of the cluster groups, I figured out their characteristics, which was needed in order to tell the user what the asteroid group means. I also created a class that plots the new point into the existing PLT plot, which will be shown in the streamlit app. After all this, I thought I was done, but I realized I made a very big mistake. I didn’t analyze my dataset too much, until I started to work on a script that gets the top 5 most similar asteroids. I noticed the 5 asteroids were all the same. When I looked at my dataset, I saw that I had a bunch of duplicate asteroid names, but with different approach dates, so in reality, I only had about 20 unique asteroids. Thus, to mitigate this, I found a new dataset that has over 300k entries, and 30k+ unique asteroids. I made sure I did everything correct when training the unsupervised model. Finally, I once again, integrated the model to my GUI. Now everything looks nice and clean.
