Open library
The essential reading of generative AI and agents, open in a clean reading view, with Meow and Standard summaries when available. No account needed to read.
563 papers · 561 with a Meow summary · 333 posts
Every paper prepared on Reading + AI, newest first.
Zhengqi Li, Richard Tucker, Noah Snavely et al.
We present an approach to modeling an image-space prior on scene motion. Our prior is learned from a collection of motion trajectories extracted from real video sequences depicting natural, oscillatory dynamics of objects such as trees, flowers, candles, and clothes swaying in the wind. We model dense, long-term motio…
Xiaodong Chen, Yuxuan Hu, Jing Zhang et al.
This paper introduces LLM-Streamline, a pioneer work on layer pruning for large language models (LLMs). It is based on the observation that different layers have varying impacts on hidden states, enabling the identification of less important layers to be pruned. LLM-Streamline comprises two parts: layer pruning, which…
Leyang Xue, Yao Fu, Zhan Lu et al.
This paper presents MoE-Infinity, an efficient MoE inference system designed for personal machines with limited GPU memory capacity. The key idea for MoE-Infinity is that on personal machines, which are often single-user environments, MoE-based LLMs typically operate with a batch size of one. In this setting, MoE mode…
Sanket Shah, Arun Suggala, Milind Tambe et al.
The declining participation of beneficiaries over time is a key concern in public health programs. A popular strategy for improving retention is to have health workers ‘intervene’ on beneficiaries at risk of dropping out. However, the availability and time of these health workers are limited resources. As a result, th…
Nicholas Gao & Stephan Günnemann
Solving the Schrödinger equation is key to many quantum mechanical properties. However, an analytical solution is only tractable for single-electron systems. Recently, neural networks succeeded at modeling wave functions of many-electron systems. Together with the variational Monte-Carlo (VMC) framework, this led to s…
Michael Scherbela, Leon Gerard, Philipp Grohs
Deep neural networks have become a highly accurate and powerful wavefunction ansatz in combination with variational Monte Carlo methods for solving the electronic Schrödinger equation. However, despite their success and favorable scaling, these methods are still computationally too costly for wide adoption. A signific…
Junjie Wang, Yuchao Huang, Chunyang Chen et al.
Pre-trained large language models (LLMs) have recently emerged as a breakthrough technology in natural language processing and artificial intelligence, with the ability to handle large-scale datasets and exhibit remarkable performance across a wide range of tasks. Meanwhile, software testing is a crucial undertaking t…
Tanishq Kumar Zachary Ankner* Benjamin F. Spector Blake Bordelon Niklas Muennighoff Mansheej Paul Cengiz Pehlevan Christopher Ré Aditi Raghunathan
Low precision training and inference affect both the quality and cost of language models, but current scaling laws do not account for this. In this work, we devise “precision-aware” scaling laws for both training and inference. We propose that training in lower precision reduces the model’s effective parameter count,…
Runming Yang Taiqiang Wu Jiahao Wang Pengfei Hu
Knowledge distillation (KD) has been a predominant method for compressing Large Language Models (LLMs). In this paper, we first revisit KD and Low-Rank Adaption (LoRA) and demonstrate that they follow the same paradigm. Inspired by this observation, we propose a parameter-efficient knowledge distillation method, Llm-N…
Wail Y. Alkowaileet, Michael J. Carey
In the last decade, document store database systems have gained more traction for storing and querying large volumes of semi-structured data. However, the flexibility of the document stores’ data models has limited their ability to store data in a columnar-major layout — making them less performant for analytical work…
Zane Durante, Qiuyuan Huang, Naoki Wake et al.
Multi-modal AI systems will likely become a ubiquitous presence in our everyday lives. A promising approach to making these systems more interactive is to embody them as agents within physical and virtual environments. At present, systems leverage existing foundation models as the basic building blocks for the creatio…
Ekin Akyürek, Mehul Damani, Adam Zweiger et al.
Language models (LMs) have shown impressive performance on tasks within their training distribution, but often struggle with structurally novel tasks even when given a small number of in-context task examples. We investigate the effectiveness of test-time training (TTT)—temporarily updating model parameters during inf…
Evan Miller
Evaluations are critical for understanding the capabilities of large language models (LLMs). Fundamentally, evaluations are experiments; but the literature on evaluations has largely ignored the literature from other sciences on experiment analysis and planning. This article shows researchers with some training in sta…
Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, Kaifu Zhang
Currently OpenAI o1 sparks a surge of interest in the study of large reasoning models (LRM). Building on this momentum, Marco-o1 not only focuses on disciplines with standard answers, such as mathematics, physics, and coding—which are well-suited for reinforcement learning (RL)—but also places greater emphasis on open…
Zhuoqun Li, Xuanang Chen, Haiyang Yu et al.
Retrieval-augmented generation (RAG) is a key means to effectively enhance large language models (LLMs) in many knowledge-based tasks. However, existing RAG methods struggle with knowledge-intensive reasoning tasks, because useful information required to these tasks are badly scattered. This characteristic makes it di…
Stefan Szeider
The MCP Solver bridges Large Language Models (LLMs) with symbolic solvers through the Model Context Protocol (MCP), an open-source standard for AI system integration. Providing LLMs access to formal solving and reasoning capabilities addresses their key deficiency while leveraging their strengths. Our implementation o…
Mingxing Xu, Wenrui Dai, Chunmiao Liu et al.
Traffic forecasting has emerged as a core component of intelligent transportation systems. However, it still remains an open challenge for timely accurate traffic forecasting, especially long-term forecasting, due to the highly nonlinear and dynamical spatial-temporal dependencies of traffic flows. In this paper, we p…
Yong Liu, Guo Qin, Zhiyuan Shi et al.
We introduce Sundial, a family of native, flexible, and scalable time series foundation models. To predict the next-patch’s distribution, we propose a TimeFlow Loss based on flow-matching, which facilitates native pre-training of Transformers on continuous-valued time series without discrete tokenization. Conditioned…
Wyatt Meldman-Floch
We follow existing distributed systems frameworks employing methods from algebraic topology to formally define primitives of blockchain technology. We define the notion of cross chain liquidity, sharding and probability spaces between and within blockchain protocols. We incorporate recent advancements in synthetic hom…
Majeed Kazemitabaar, Oliver Huang, Sangho Suh et al.
Novice programmers are increasingly relying on Large Language Models (LLMs) to generate code for learning programming concepts. However, this interaction can lead to superficial engagement, giving learners an illusion of learning and hindering skill development. To address this issue, we conducted a systematic design…
Christopher Kane, Christoph Lehner, Amarjit Soni
Adding a hard photon to the final state of a leptonic pseudoscalar-meson decay lifts the helicity suppression and can provide sensitivity to a larger set of operators in the weak effective Hamiltonian. Furthermore, radiative leptonic $B$ decays at high photon energy are well suited to constrain the first inverse momen…
Guancheng Lin, Xiao Yu, Jacky Keung et al.
Pre-trained or fine-tuned on large code corpora, Large Language Models (LLMs) have demonstrated strong performance in code completion tasks. However, their embedded knowledge is constrained by the timeliness of training data, which often includes code using deprecated APIs. Consequently, LLMs frequently generate depre…
Weijun Luo, Kelvin Luu, Xinyi Liu et al.
Agentic benchmarks guide model selection and training. Yet an agent can pass a task without demonstrating the intended capability. Such outcomes constitute unearned passes, their proportion among all passes defines the integrity gap. As agents improve, benchmark surfaces that once seemed harmless can become exploitabl…
Michael Chen, Hong Chul Nam, Xi Deng et al.
Planning under uncertain control execution noise requires reliable feedback policies across diverse environments. We introduce **Planning Under Noise (PUN)**, a neural operator that learns noise-aware value fields for planning in unseen geometries. Building on an established screened-Poisson formulation of stochastic…
Ben Mildenhall Pratul P. Srinivasan Matthew Tancik, Jonathan T. Barron Ravi Ramamoorthi Ren Ng
We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene using a fully-connected (non-convolutional) deep network, whose input is a singl…
Yeyin (Eva) Zhu, Rishabh Ranawat
RTL repair benchmarks should measure whether a model can fix broken RTL given a specification. To this end, we construct and hash-bind RTLRepairDataset, 5,784 (broken, fixed) training pairs; RTLRepairBench, 459 held-out cases with 374 syntactic and 85 behavioral mutations; and an evaluation harness supporting simulati…
Jitendra Solanki, Anand Todkar, Ziran Min
Modern distributed microservice deployments generate massive volumes of heterogeneous, multi-format observability logs across physically and logically partitioned infrastructure. Isolating root causes during cascading failures remains an arduous, cognitively demanding task for Site Reliability Engineers (SREs), heavil…
Zehao Wang Jianpeng Yao Yuping Wang Yiping Li, Jiaqi Li Zhixu Li Kanghoon Lee Jinfeng Wang, Cong Liu Zhaowei Tan Jinkyoo Park Jiachen Li
Reinforcement learning enables mobile robots to make fast and scalable navigation decisions in constrained human-shared environments. However, directly extending single-robot approaches to multi-robot settings often fails to ensure safety and efficiency, especially under occlusions and out-of-distribution (OOD) scenar…
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux et al.
We introduce Mixtral 8x7B, a Sparse Mixture of Experts (SMoE) language model. Mixtral has the same architecture as Mistral 7B, with the difference that each layer is composed of 8 feedforward blocks (i.e. experts). For every token, at each layer, a router network selects two experts to process the current state and co…
Huawei Lin, Jinting Liu, Tan Yu et al.
Systems that combine several large language models (LLMs) often use one context for many questions. When the context is passed between models as text, the receiving model processes all of its tokens again for every question. Avoiding this cost requires a context representation that is compact, reusable across question…
Zekai Li, Jiayi Yu, Jinlu Zhang et al.
In this paper, we systematically investigate the mechanisms, scaling behavior, and generalization of physical Recursive self-improvement. We introduce PhysRSI-eval to quantify physical RSI through a five-axis metric: First-Attempt Performance, Efficiency, Success, Budget and Improvement from failure. Different VLMs ar…
Shiao Xie, Siyu Chen, Jianwei Lv et al.
Multimodal reinforcement learning with verifiable rewards (RLVR) provides reliable task-level supervision, yet offers limited guidance for localizing errors in visual reasoning. On-policy self-distillation (OPSD) complements this supervision with dense token-level corrections from a privileged teacher equipped with an…
Jiaxin Ge, Junyi Zhang, Tianning Chai et al.
A seemingly reasonable push in Sokoban can make the puzzle unsolvable. To avoid such mistakes, an agent has to anticipate the consequences of its actions before execution, especially when feedback from the real environment is sparse or actions are irreversible. We introduce Agentic Imagination (Agima), a training fram…
Wenqian Zhang, Yangyi Huang, Zigeng Xu et al.
While LLM agents can generate visually plausible CAD assemblies, these designs often fail to fulfill their intended physical functions. We study functional CAD design, a setting where agents interpret underspecified requests to construct and iteratively revise mechanisms that satisfy functional requirements in physica…
Rui Shi, Xunjian Yin, Jiacheng Sang et al.
Long-horizon agents must learn heterogeneous capabilities from interleaved trajectories. Updating shared parameters to learn new capabilities can degrade previously acquired ones, resulting in catastrophic forgetting. As a trajectory unfolds, different stages may require different capabilities, yet conventional joint…
MemTensor Team
Memory-native evolution requires making memory an intrinsic capability of foundation models, enabling them to continuously accumulate and utilize information. To this end, we introduce **MemT-1.0**, a new architecture of memory foundation models that internalize memory as part of model computation. Building on Metis,…
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch et al.
We introduce Mistral 7B, a 7–billion-parameter language model engineered for superior performance and efficiency. Mistral 7B outperforms the best open 13B model (Llama 2) across all evaluated benchmarks, and the best released 34B model (Llama 1) in reasoning, mathematics, and code generation. Our model leverages group…
Pang Wei Koh, Percy Liang
How can we explain the predictions of a black-box model? In this paper, we use influence functions — a classic technique from robust statistics — to trace a model’s prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. To s…
Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin et al.
OpenAI o1 represents a significant milestone in Artificial Inteiligence, which achieves expert-level performances on many challanging tasks that require strong reasoning ability. OpenAI has claimed that the main techinique behinds o1 is the reinforcement learining ([OpenAI, 2024a](#bib.bib113); [OpenAI, 2024b](#bib.bi…