Attention Kernels and Feature Inteference Under a Representational Bottleneck

This project was made for the Non-Trivial Fellowship. It is a machine learning paper testing various attention kernels and their effects on mitigating superposition, and whether this depends on compression size.