Attention map
i
Attention map
A grid showing how much information each word takes from every other word while the model builds its internal representation.
Pick a token and inspect how strongly it looks at every other token in the sentence.
Attention head
Tracks grammatical relationships between words.
Query tokenSwitch heads or tokens to see how the same sequence can produce different attention patterns.
Strongest connectionsat → cat
Keys →
Thecatsatonthewarmmat
Rows are queriesDarker = stronger attention