Docs/Links
Links
Type [[ and a note's name. Links work both ways.
Transformers
Transformers
Sequence models that replace recurrence with attention.
Built on . Introduced in [@vaswani2017].
Self-attention
Every token attends to every other token in one step:
Attention(Q,K,V) = softmax(QK⊤ / √dk) V
See the paper, page 3:
LocalOffline readyMarkdown
60 words · saved
[[Note name]] links to a note. The folder isn't needed.
| Write | Links to |
|---|---|
[[Transformers]] |
the note |
[[Transformers|this paper]] |
the note, shown as "this paper" |
[[Transformers#Attention]] |
a heading |
[[Lecture 3.pdf#page=12]] |
a page of a PDF |
[[2026-10-07]] |
that day on the calendar |
- Not written yet? The link is dashed. Click it to create the note.
- Backlinks in the context sidebar (
Mod+Shift+\) list every note that links here, plus mentions you haven't linked. - Renaming is safe. Neoma shows every link it will update before changing anything.
![[Note]]shows a note inside another.aliasesin properties give a note other names.