Confidence Is a Budget: A Small Study of Selective Prediction

Mon, 24 Aug 2026 00:00:00 +0000

This is a short, synthetic paper written to demonstrate Nota rather than to report an empirical result.

The production build replaces this Hugo fallback with the server-rendered Nota document at the same permalink. Hugo retains this companion page so the article still participates in section listings, categories, tags, RSS, and the sitemap.

Attention as Learned Routing

Sun, 12 Jul 2026 00:00:00 +0000

This is an AI-generated placeholder article.

Attention is often introduced as a metaphor: a model “looks at” the relevant tokens. A more useful engineering picture is learned routing. Each token produces a query, a key, and a value; the query-key match determines where information flows.

For a sequence represented by $X$ , one attention head computes

\operatorname{Attention}(Q,K,V) = \operatorname{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.

The matrix inside the softmax is a table of routing scores. Dividing by $\sqrt{d_k}$ keeps those scores from growing too sharp as the key dimension increases.

Math on ejuarezg.com

Confidence Is a Budget: A Small Study of Selective Prediction

Attention as Learned Routing