sparq

left: regular (no future prediction)
right: dynamic (future prediction)

devlog #1: machine learning for dummies 😦

my first devlog!
i probably shouldve done another devlog earlier since its now been 11 hours into this project but unfortunately i have the foresight of a seal so… anyway! join me on my machine learning learning (haha get it) journey! this is my first real foray into actually learning ML beyond using preexisting models, and its been super interesting so far.

ive genuinely learned so much including but not limited to (very generally):

• DQN model: how to create a custom model and train and test it
• dynamic model: imagined states and being able to predict futures
• the usefulness of a configs file LOL
• TODO bc i am genuinely the most forgetful person you’ll ever meet

anyway im not sure how much more i’ll work on this specific project since cartpole is super simple and i want to try nondiscrete action spaces which are more complicated and applicable. im working on learning offline training rn but thats probably going to be a separate project

repo: https://github.com/ela1nex/customDQN

below images:

• left: tested wo dynamic model
• right: tested w dynamic model (makes a super big difference!!)