I increased the block size from 8 characters to 64. This means that instead of the model taking in the previous 8 characters and outputting the next one, it takes in the previous 64 characters and outputs the next. This means that it should be able to keep track of longer words so it doesn’t get lost in them. However, it doesn’t seem to be improving the model too much, although that might be because I haven’t trained it as much as the others. I’ve done roughly 200,000 epochs, which took a long time, since this model is much larger (110,415 parameters vs 45,000), and has 6 hidden layers as apposed to 2, which means it is a lot slower. As you can see in the loss graph, it probably could make further gains, although it seems to be mostly good.