LLM

final: redesign the webpage for easier reach and interaction, also readjust the training pipeline as the old one have many flaws that the I myself couldnt get respond when testing, and it’s fixed ig

Modify training configurations, pipeline and fix some error in src/model/mlp.py and src/model/rope.py

Change the dataset from entirely scratch to Dolci-Instruct-SFT, then also add serializer (src/utils/serialize.py) to serialize the dataset into .txt file and then to train it with spm.SentencepieceTrain() (src/utils/tspm.py) also add bin.py to turn tokenized into token IDs for faster dataloder, also removed unnecessary file

#1 - Been on configurating the project, also after configuration I go on getting the dataset, while you can search for them online, I wanna be really type of ‘from scratch’ so in this case I also create the dataset from scratch as well. And all those 10hrs 44m, I’ve been manually entering random wikipedia articles to the code to scrap the texts, The main obstacle that make this so long is that sometimes I have typos, wrong words, or even not found which I’ve to also manually resolve all of them. Thats all but been going on around ~1600 articles