DEV LOG 3: Personal Language Trainer

development has been slow because of me researching a bunch on linguistics

what’s new?

• updates on the user database to include more specifications of words used
• updates on text processor being a pure classifier rather than heavily filtering the input
• initiating a scoring system to determine the importance of each word.
• a whole lot of testing with other NLP libraries.

That’s pretty much it the other days are mostly filled with research in minor updates that slowly refactor the code but mostly everything stays the same.

DEV LOG 2: Personal Language Trainer

It’s been a few days after my last dev log and I managed to redo and fix multiple things.

what’s new?

Updates on text processing:

• removed tier0 bucket
• reconfigured how the words get filtered and process

Updates on database:

• initialized the database with ‘sqlite3’
• three tables are created: ‘words’, ‘sentences’ and ‘words_sentence_links’
• ‘words’ record all the words inputted. (if new it will increment an id and count)
• ‘sentences’ record all sentences with no counts
• ‘word_sentence_links’ contain word and sentence ids and has the purpose to show which word is present inside a sentence.

Testing:

• I also added some minor things like an entire dedicated test script to test the program.
• There are also test cases in both audio and text to test recording and transcribing.

That’s pretty much it, It still looks boring so I have screenshots of the tests and the database

DEV LOG 1: Personal Language Trainer

This is the start of my project Personal Language Trainer.

Initializing

• Created the transcribing component of the code using OpenAI’s open sourced model whisper.
• Created the text processor for the words transcribed to prepare it for the database. I used spaCy an NLP to clear unwanted words from the transcript of each sentence.
• The text processor contains a filter that will take valid words for communication then, put them into tiers so that it would be easier to calculate which type of words should be prioritized based of importance and usage.

That’s all for today I don’t really have much for a demo other than a screenshot (it doesn’t look really interesting tbh).