Infrastructure & deployment
Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM
Push-based NIXL transfer overlaps decode-side setup with prefill and moves transfer preparation off the decode worker’s critical path, targeting lower TTFT and TPOT.
Why it matters: Push is a fit when selecting the decode instance early is acceptable and lower TTFT is the priority. Pull remains preferable when routing after prefill or bidirectional KV transfer is required.