Rail Engine

Alright! So today was very productive:

1. I trained a lot more models for benchmarking based off MSE

2. I increased model size, because with seeding support, I could find the limits of my model size and the most optimal seeds for training and…

3. Added seeding support for benchmarking. Seeding lets me actually make sure that my models are consistent during training, and I can repeat a trial if something goes wrong

4. Added logging; Each training cycle now also creates a csv file with the epoch number, the train score, and the value score (how well it generalized vs how close it is to the training data)

5. I doubled my dataset, just by inverting the current data, and fixed the training script. The training script would cause spikes in the model’s val score, making it worse, and it couldnt recover from those spikes

But… with the now ~6x bigger model and double the data, training on just my cpu is taking forever, but theres not much I can do about that for now :P

Okay, so I was able to actually train an ml model off 25 games that were recorded, but it was a pain
(this is technically v2, v1 was a test, trained off one game)

So ive started working on my next part of my FTC automation network, and thats with Rail Engine.
Rail engine is a small IM based transformer model designed to work with the preexisting logic engine framework. Its a model that takes in ~0.5 second of context window, and it outputs what it thinks the next coordinate pair should be.
Im getting the data via JuiceAutoScout, over on the FTC discord, and I take a video for it and it outputs robot positions and robot numbers.
To maximize training data, im treating each robot in the data the robot, so I get 4 times the training data :D