AI Mail Cleaner

Sigh
Update #3

I implemented all of the feature extraction part. The TF-IDF vectorization, the unsupervised kmeans, the labelling the custom folders.

The problem is… it’s just not accurate. I’m not very sure what to do. At this point, I have a few options, I guess.

Option #1: Hyperparameterize the vectorization. It could possible immensely boost my accuracy. Kmeans is already essentiallly hyperparametrized. But this is REALLY computationally expensive.

Option #2: Just make API calls ot GPT or something. Lame. Boring. No.

idk what to do.

Update #2

I’m really stupid.

I found this library called simplegmail which handles the oauth and the parsing for me, so far a particular message I can just call message.id or message.html to get content. I quit on classifier.ipynb and just started a new one, and I managed to get a dataframe for ALL the mails which is really exciting!

next stage: convert the combined text feature into numerical data via tf-idf(Term Frequency-Inverse document Frequency) vectorization. Then, I need to implement unsupervised KNN and have the user label the clusters.

image -> short snippet of the dataframe I generated, im so proud ^^

Update #1

So I got the google Oauth figured out, it was surprisingly easy considering I usually struggle with auth lol. The next thing I need to do is extract features from the user’s mailbox in order to train the model.

I thought this would be easy. Key word, thought.

The problem is, some mails contain content in HTML, some in plain text, and some are a mix. So each have a different method of parsing.

There are 5 main types, which you can see in the notebook in the repo. I’m almost done with the first one(multipart/alternative) though only the plain text has been retrieved, I need to parse the html too.

I got a doctor appointment today, and I have other courses to do, so I might not be able to grind.

P.S this is a visualization of the all the information the gmail API returns for ONE mail. Please help me.