Infrastructure & deployment
TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
TOPAS jointly schedules cached agent prefixes and requests, aiming to improve task-level job completion time (JCT) under a shared KV-cache budget.
Why it matters: For multi-agent serving, the abstract suggests cache residency and request selection should be considered together rather than optimizing locality or workflow progress in isolation. The reported gains are measured on three synthetic DAGs and two MetaGPT workflows.