Docs/Graph
Graph view
Notes become connections. Press Mod+G.
Transformers
Transformers
Sequence models that replace recurrence with attention.
Built on . Introduced in [@vaswani2017].
Self-attention
Every token attends to every other token in one step:
Attention(Q,K,V) = softmax(QK⊤ / √dk) V
See the paper, page 3:
LocalOffline readyMarkdown
60 words · saved
| Node | a note |
| Edge | a link |
| Colour | a folder |
Click a node to highlight its links, double-click to open it. Scroll to zoom, drag to pan. Current note shows only what's nearby; Options change layout, colour and filters.
No connections yet?
Create one with [[Another note]].