YouTube Metadata changes

Getting the metadata for each youtube video has proven hard. yt-dlp + youtube_transcript_api used to be fine but as i starting embedding many videos i got rate limited and ip banned from transcript (hopefully lifted within 2 days, i watch yt w/ captions on)
So i switched to meta_yt and that worked so much better but eventually i got rate limited again. I have now finally swapped to the official google API so i can get a bunch of metadata from videos quickly, but no captions from it since it costs 200 quota credits (I have a max of 10,000 a day)

Actual Algorithm

I have currently saved 501 video embeddings to my database and have generated a user embedding for me. 12 out of the ~378 values for that embedding is in the image below.
For each video you watch your embedding changes like so. The video’s embedding gets multiplied by 0.05 and by the percentage of the video you watched (0-1). Then that gets added to your user embedding that was multiplied by 0.95.
The formula looks like this:
userEmbedding = (userEmbedding * 0.95) + (videoEmbedding * 0.05 * watchPercentage)

Up Next:

Next I want to implement FAISS, and the cosine function to compare similarity of vectors. Then using that I want to try selecting some videos I am, and am not intrested in and see how well it knows me.
After it works well enough I want to either make myself a browser extention or website to serve me videos.
I also gotta make some way to find new videos (popularity, trending, what else?) and then embed them and store it

I made most of the parts I need to get this working

YouTube History

YouTube, in their infinite wisdom, does not expose an api endpoint for your watch history w/o using google takeout (which takes forever)
So I make a quick firefox extention to read my watched videos and time watched and then save it to a json file for later

YouTube Video Info

I am using yt_dlp and youtube_transcript_api to take a YT video’s ID and then retrieve the following info:

• title
• description
• duration
• categories
• tags
• was_live
• transcript

Embedding Model

I’ve downloaded and tested the BAAI/bge-small-en-v1.5 text transformer to generate text embeddings for any text I want. The goal is that users have an embedding for things they like, and each video has an embedding based on it’s title, description, category, tags, and the transcript.
To recomend videos I can see how similar a video’s embedding is to the user’s embedding, and then rank them. I can then give them choices based on that ranking.
Watching and liking videos will nudge the user’s embedding towards that video’s embedding, while leaving early or disliking nudges it away.

Database

I’m using a Lightning Memory-Mapped Database (LMDB) w/ 2 tables, one for all the user’s embeddings, and the other for all the video’s embeddings.
I am going to try and use FAISS for the similary search from the user embedding to all the videos

Whats next?

I need to do a few things, mostly just connecting each of these systems.
First of all I need to go through my entire watch history and get the info for them, then embed them, and then store it in my database. Also repeating this process with any other video I can find on YouTube

I also need to connect the system to take my user embedding, and get the most similar videos and then return those.

I would also like to make a custom UI/homepage for my algorithm.