Open library
The essential reading of generative AI and agents, open in a clean reading view, with Meow and Standard summaries when available. No account needed to read.
560 papers · 558 with a Meow summary · 333 posts
Key publications from Anthropic, OpenAI, Google DeepMind, Meta, DeepSeek, Qwen, Z.ai, Kimi and other labs, plus the most influential essays and blogs.
Google Research
Speculative decoding speeds autoregressive LLM inference by computing several likely tokens in parallel while preserving the target model’s exact output distribution. Its key is probabilistic checking, not simply accepting or rejecting a draft.
Lilian Weng
Transformer inference gets cheaper only when the fix targets the actual bottleneck—memory, computation, or latency—and fits the hardware that must run it.
Stuart Sul · Cursor
Cursor has open-sourced MoK, a deterministic NVL72 MoE training kernel that fuses communication and computation into one kernel.
Less Wright · Cursor
For small-batch MoE decode on Blackwell, warp decode assigns work by output rather than expert; its gain comes from removing data-layout bookkeeping, not merely fusing the old path.
Brendan O'Donoghue · Google DeepMind
DiffusionGemma generates text in parallel blocks, trading some output quality for up to 4x faster inference on dedicated GPUs.
Olivier Lacombe · Google DeepMind
A small Gemma 4 drafter proposes several tokens while the larger target model checks them in parallel.
Mistral
Mistral Small 4 combines instruct, reasoning, agentic coding and multimodal input in one model rather than asking users to choose among specialized models.
Mistral
UCX’s memory hook was meant to speed InfiniBand transfers, but in this vLLM setup it helped anonymous memory accumulate outside the heap. Disabling the hook fixed the leak without a performance hit in this use case.
Z.ai
The Infra Agent’s contribution was not just writing code: the authors credit a feedback loop that made its systems work testable. They describe this as an early step, not recursive self-improvement itself.
ben burtenshaw · DeepSeek
DeepSeek-V4 aims to make a 1M-token window practical for agents by combining cheaper long-context attention with tool-aware reasoning and execution.
Gergely Orosz · The Pragmatic Engineer
Inference engineering turns open-model access into control over serving—but its payoff depends on balancing latency, throughput, quality, reliability, and cost for a real workload.
Vicki Boykis
In the author’s experience, local models have moved from development lookups to agentic coding; the results are promising, but not benchmark proof.