Transformer block
i
Transformer block
A repeated model layer that mixes information between tokens and transforms their internal numerical features.
Step one token through the operations repeated throughout a transformer model.
Active operation · 1/5
Embed
Token ID becomes a dense vector.
Embedding[token_id] + position
Residual stream vector
0.2-0.70.40.9