I also tried PID as a benchmark (only in simulation of course). It did pretty well as shown below, so I might try either imitation learning or a different model that just identifies ball and beam position to then apply PID on top.

In parallel, I also tested QT-Opt on hardware and it failed… I’m trying more training with visual distortions in the simulation environment so it becomes more robust