Docs/Search
Search
Search runs on your device. No remote index.
Transformers
Transformers
Sequence models that replace recurrence with attention.
Built on . Introduced in [@vaswani2017].
Self-attention
Every token attends to every other token in one step:
Attention(Q,K,V) = softmax(QK⊤ / √dk) V
See the paper, page 3:
LocalOffline readyMarkdown
60 words · saved
| Do | Press |
|---|---|
| Open a note by name | Mod+O |
| Search every word | Mod+Shift+F |
| Find in this note | Mod+F |
| Type | Finds |
|---|---|
"exact phrase" |
that phrase |
transformer -draft |
without "draft" |
tag:thesis |
a tag, nested tags included |
path:Courses |
a folder |
type:experiment |
a property value |