About us
Stay in touch
The neural network design introduced by Google researchers in the 2017 paper 'Attention Is All You Need'. It lets a model weigh every part of its input against every other, and it underlies GPT, Claude and most modern AI.
So we need to engage in acts of interpretation to explain the output, and these have to operate at a different scale of abstraction, not just at the level of the transformer architecture or next token prediction, but even beyond that.
AA Cavia, AA Cavia on Summoning Worlds