Compare the words! ++

I have reworked how words are compared (again)!

The best CLI EVER (well, now at least)

SpellCLI has been out for some time! I have been using it and I have found a problem. Words that are anagrams of each other often flood the top results and if the word is spelled incorrectly, give (seemingly) completely random words. This is because the placement of a word was not considered at all in the algorithm described in Compare the words! +

The Great Algorithm

The new algorithm now considers the placement of words (how much they moved) and if words are missing!
Take the words “great” and “rat”. ‘r’ haves to move 1 place and ‘a’ and ‘t’ move 2:

G|R|E|A|T 
-|R|   |   |
_|- |- |A|
_|_ |- |- |T

For a total of 5 movements. Than, the ‘g’ and ‘e’ are left, for a total of 2 leftovers.
Leftovers than get a ×max_len multiplier, this wights them so that moving letters is “more similar” that extra letters.
This gives a total of 15 points of differences. The max amount of difference is calculated as (len1+len2) * max_len (40 in this case). this gives a total of 0.375 difference or 0.625 similarity (63% similar)!

The future of SpellCLI

I don’t know how common devlogs will be for SpellCLI now that the project is done. I don’t have any plans to ship the project again, if I make several changes by the end of the summer I might.

See you in my next project 👀, bye!

Out Now!

SpellCLI is out on crates.io!
The program can be installed with

cargo install spellcli

See you soon(er than you think)!

Output and away!

I have reconfigured how information is outputted and I am very close to shipping.

Out(put)

I have changed the output to display

Is the input word correct?
- if so it's definition
Definitions of similar words

Not a very complex addition but definitely a needed one!

And Out!

Now that everything is cleaned up and working, the project is just about ready to ship! All I need to do now is a few more tests, to work on the README, and get the CLI on crates.io/downloadable with cargo intall SpellCLI!

Good bye!

Multi-threaded Optimizations and a life time of lifetimes

The project is now multi-threaded, there is no more acync code, and there are way fewer lifetimes. I can’t wait to show you all of it!

Taking our time

When I started 3 hours and 44 minutes ago (this is the most amount of coding time I’ve added in a devlog this project!) I tested the speed of the program using the word “anagram”, set to give the top 5 results, I got the speeds: 6314, 5514, and 5488 milliseconds, (from input of anagram to end of program) that’s about 5.5 seconds! That is not very fast, at all.

My Client

The first optimization I implemented is the use (or reuse) of a Client when requesting definitions from the web. A Client is like a connection to the internet (I think). And every time I got a word’s definition, I created a new Client. (You could imagine how inefficient that can get)
Moving over to reuse a Client was very easy (compared to what’s next )
After this change, anagram returned 4332, 3944, and 4038 milliseconds; about 4 seconds, getting better!

My Client requested multiple threads

Next I made each search for a definition run on a separate thread. Implementing this wasn’t too bad. I did have some trouble with lifetimes but a few .clone()s patched up those errors! But, when the code ran, nothing happened. After panting plenty println!()s I found out that the threads weren’t running, at all . I came to the conclusion that the threads weren’t running because they were using acync closures, and everything acync in rust in lazy and needs to be told to ran. After getting frustrated with tokio I switched the acync part of the acync function with it’s blocking equivalent and everything worked! And with that, I didn’t need tokio anymore ¯\_(ツ)_/¯.
This change returned 3628, 3659, and 3982 milliseconds; about 3.5 seconds, good.

My teacher never thought me how to read a dictionary.

Lastly, I decided to make the code that checks every word against the input word, you guessed it, I’m tired multi-threaded!
This is where lifetimes decided to step in .
(in 2 secs: a reference (ref or &) points to an owned value; so if the owned value 💥, the ref . To stop this, lifetimes say: “just don’t 💥 when there’s a ref (+ some 🪄 so things aren’t easy))
I made the code that splits dictionary searching into multiple jobs, easy.
Than, I tried to make the the system run on multi-threaded, not easy.
After fighting &str for too long I finally decided to just turn all the interrupting &strs into Strings (making the values owned; owned values have no lifetimes (I mean, well…)).
The results after all that are 1769, 1666, and 1665 milliseconds; a little over 1.5 seconds, this is very good!

4 paragraphs, 3 improvements, 2 seconds to run, 1 week to launch.

That was a wild ride! I hope you enjoyed! I do have some after thoughts if you would like to continue reading:
When running with run --release, “anagram” took 800 milliseconds, which I consider fast enough. I do have an idea for something that could make the app run even faster, but I’ll add it if I feel like it.
And I am thinking of shipping within the coming 7 days (maybe, sooner).

Bye!!!

Definitions and JSON

I have completed the code that gets definitions for words. Even though the idea is simple, the code is not!

Dictionary

The long journey of getting definitions starts at dictionaryapi.dev. This site allows anyone get a large amount of info on any given word!
At the end of the URL I can add a word and it will give me a word’s synonyms, example uses, and definitions.
Getting the website data is easy enough using reqwest :

reqwest::get(/* URL */).await.unwrap()
        .text().await.unwrap();

This gets all the data that I need (oh, ya, I’ll explain the use of async code later)!

Info

But, the real tricky part is that the data is in JSON and not Rust (who would have though).
Luckily enough, there is a library, serde_json, that helps with Rust-JSON communication!
interfacing with JSON wasn’t too hard, I just find it unusual how everything was wrapped in a vector; I had to access the data like this:

let word_map = &json_body[0]["meanings"][def_num as usize];

let word_def = &word_map["definitions"][0]["definition"];
let part_of_speech = &word_map["partOfSpeech"];

This could be completely normal but, I don’t know. By the way, [def_num as usize] was originally just [def_num] witch is invalid and silently, always returns null. This took too long to find out .

Async and whats next

And now, about the async code. The program is taking longer than I would like to run, so, I’m planing on adding some async and multi-threaded code next.

Bye!

Compare the words! +

I have reworked how words are compared!
before, if I tried to compare “different” and “diferent”
the code would see

| d | i | f | f | e | r | e | n | t |
| d | i | f | e | r | e | n | t |
| ^ | ^ | ^ |X |X | X | X | X | X |

For a total of 33% accuracy (very wrong)!
Now, the new code only compares the difference in letters, without caring about where each letter is placed!
The code now sees:

| d | i | f  | r | e | n | t |  <- all letters used
| 1 | 1 | 2  | 1 | 2 | 1 | 1 |  <- count for "different"
| 1 | 1 | 1  | 1 | 2 | 1 | 1 |   <- count for "diferent"

then calculates the accuracy per letter:

| d | i | f  | r | e | n | t |
| 1 | 1 | .5  | 1 | 1 | 1 | 1 |

witch gives an average (aka accuracy) of 0.928(93%)! This is far better than before!
This can give some results that you wouldn’t think like how the code sees “rustq” as most similar to “quarts” (as seen in the image below).
You can see the function code here.
Now, I’m thinking of getting word definitions for the top i most similar words.
Can’t wait for next time, bye!

Every word in the dictionary

I have created a function that compares an input word against every word in the library wordnik_list (very useful! you should check it out!) and retrieves the top n most similar words. The function is very fast already so I probably won’t need to do anything multi-threaded, but the term “multi-threaded” in a project’s description generally creates a better vibe!
I’m planing on reworking how words are compared because there is an issue with how the current comparison works (I’ll go into more detail once the fix is implemented).
See you later!

Compare the words!

I have added a struct that allows words to be compared for similarity!
It works by checking if each letter in two strings are the same.
So, if I compare “tall” and “wall”: the code will see
t | a| l | l |
w | a | l | l |
❌|✅|✅|✅
for a total of ¾/0.75/75% similarity!
I’m planing on using this functionality to compare the input word against every English word to get the word the user is most likely trying to spell. So I might have to get into multi-threaded code down the line (or should I say twine XD).
Bye!

Setup

I set up a basic utility for the app, being the first_word function. The function isn’t supper complex. What it dose is goes through each character in a string, adds the character to the output if it isn’t white space. If it is white space that the output string is returned.

Idea

My vision for this app is that you will input a word and the CLI will tell you if it is spelled correctly, and definitions of similarly spelled words. It isn’t supper complex but I do see myself using it in the future quite a bit.