I realised I made a mistake in my previous implementation of the DQN model.
I forgot to inherit from the required pytorch module and I forgot to feed the state_size and action_size to the constructor method.
To further evaluate and fix my code, I will be using an implementation from GeeksForGeeks to guide me while modifying some sections to suit my specific scenario.
I have completed the rest of the code which involved coding the training process and initialising the policy and target networks.
A challenge I am facing now is modifying the code to suit my specific environment and updating the GUI to visually showcase the result of the AI’s choices.
A few changes I made were:
Added gravity and rocket acceleration variables
Removed the previous actions and introduced only two choices for the model which is either to apply thrust or choose to not apply thrust.
Fixed the returns from the step function so that it returns a numpy array that contains the rocket’s y coordinate and the rocket’s velocity
Regenerated the AI generated code to include a crash animation
The specific challenge with the modifications is that the input requirements are very specific and the manipulation of these inputs relies on complex functions.
What are the next steps:
I will use AI to debug my codes, specifically the errors associated with incorrect input and output dimensions and structures. I will also use it to debug my GUI screen.
Once the model is working, I will change the acceleration to make it relative to time and introduce a time variable to reflect acceleration due to gravity.
I have implemented the neural network that will be used by the Deep Q Network model.
As the task is not very complex, I decided to utilise 32 neurons. In the constructor method, I am essentially stating that one piece of data will be given to the 32 neurons (y coordinate) and I expect two q values to be outputted (as the rocket can only go up or down).
The Basic map has been created
