I created a basic transformer model for the classification task and started working on training it.
I need to think about what augmentations to use on the training set.
Also I need to find a way to balance the dataset (I am using the FMA medium dataset). The problem is that there is a class (Rock music) that has much more samples, than all of the other classes combined. I need to find a way, to balance this, because right now the model learns only to always predict the rock class.