DISTILLATIONS Dec 3, 2025 9 min read Exploring Linear Attention Variants Can we have capable limitless-context foundation models? Read Article
DISTILLATIONS Nov 25, 2025 7 min read Why Small Models Get Stuck: Exploring the Softmax Bottleneck Learn about the "Stupidity Wall" and how to parameterize your small models efficiently. Read Article
RESIDUALS Nov 23, 2025 1 min read Welcome Not every problem requires a trillion-parameter model behind an API call. Let’s figure out what does, and what doesn't. Read Article
