NonLinear-1 is a research group.
Most of us are around twenty.
We think and discuss together as equals, and publish every interesting finding here.
Sparse Attention: Algorithm–Infra Co-design ↗
From block-level selection in NSA to token-level selection in DSA.
Sparse Linear Attention: Viewing SWA from the Perspective of Linear Attention ↗
Understanding SWA, state-based linear attention, and Raven through memory slots, FIFO, and sparse routing.
K3's KDA Precision Problem: Why Lower-Bounded Decay Is Necessary ↗
Understanding the joint design of decay lower bounds, 16-token tiles, and Tensor Cores through the split-exponent and Neumann-inverse precision paths.
MLA, dim by dim
A unified tensor view of MLA, and the conclusions it reveals.