NOTE: I had not planned on working on this for stardance and was originally doing it independently, which is why this devlog has so many hours in it; it represents my work over the past few days, but I couldn’t figure out a way to start from scratch and only record hours from now on. I also plan on making this more detailed soon, since this devlog is covering a lot of ground!
First attempts at training model with custom reward structures. So far, I have built a custom gridworld environment and wrapper for the teacher agent which allows you to pass a number of custom regions (to-be generated randomly) and special rewards for them (distinct from the step penalty, or the wall penalty if you decide to include it). The teacher has access to these custom rewards through the teacher wrapper (later on, when I add the student, the student won’t have access to them and will have to infer them from the teacher). I also created a custom deep Q neural network model for the teacher agent (using a CNN to allow for spatial reasoning). Recently, I successfully trained an agent that could navigate any grid size to get to the target when no special regions are present (a win for generalization) and I just tested this model with a single special region with a large penalty (half of goal reward, but negative) and it did not perform very well.
