Infrastructure & deployment
ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference
ASPIRE lets requests in a batch draft and verify on separate schedules, targeting the mismatch between a shared schedule and requests whose best draft length changes.
Why it matters: If requests can verify when their own estimates and batch costs favor it, a batch need not wait for one shared draft-verify schedule. The reported gains make this approach worth examining for long-context decoding.