Reading + AI
LibrarySign in
Open the original paper on arXiv (new tab)

Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence

NVIDIA

arXiv:2604.24954v2 · This workspace stays on this version. Check arXiv for newer versions

Some source details need review 1 to check

Original formulas, table values and figures are preserved. Comparison can miss differences; use the original PDF to check details.

Reading view checked: 104 math expressions, 0 rendering failures; 16 of 16 images available.

Compared 138 of 138 source passages against 27 of 27 PDF pages.

  • S3_F2: the arXiv HTML draws 6 text or formula block(s) of this graphic outside its frame; each was scaled into its frame (to 76% or more) without changing any text or formula. Text may look smaller than in the PDF.

16 of 16 figure, table and graphic images captured. Open original PDF

Abstract

Abstract. We introduce Nemotron 3 Nano Omni, the latest model in the Nemotron multimodal series and the first to natively support audio inputs alongside text, images, and video. Nemotron 3 Nano Omni delivers consistent accuracy improvements over its predecessor, Nemotron Nano V2 VL, across all modalities, enabled by advances in architecture, training data and recipes. In particular, Nemotron 3 delivers leading results in real-world document understanding, long audio-video comprehension, and agentic computer use. Built on the highly efficient Nemotron 3 Nano 30B-A3B backbone, Nemotron 3 Nano Omni further incorporates innovative multimodal token-reduction techniques to deliver substantially lower inference latency and higher throughput than other models of similar size. We are releasing model checkpoints in BF16, FP8, and FP4 formats, along with portions of the training data and codebase to facilitate further research and development.

Model-BF16  |  Model-FP8  |  Model-FP4  |  Dataset  |  Megatron-Bridge | NeMo-RL  |  Example Data Pipeline

In this work, we present Nemotron 3 Nano Omni, an efficient omni-modal model built on the Nemotron 3 Nano 30B-A3B (NVIDIA et al., 2025a) language model backbone, augmented with the C-RADIOv4-H1^{1} (Ranzinger et al., 2026; Heinrich et al., 2025) vision encoder and the Parakeet-TDT-0.6B-v22^{2}(Xu et al., 2023; Rekesh et al., 2023; Sekoyan et al., 2025) audio encoder. Nemotron 3 Nano Omni extends the Nemotron multimodal family with native audio support and improved reasoning capability across all supported modalities. It is particularly effective in practical multimodal settings, including real-world document understanding, long audio-video comprehension, and agentic computer use. In addition, Nemotron 3 Nano Omni incorporates innovative multimodal token-reduction techniques that substantially reduce inference latency and increase throughput, enabling efficient deployment without sacrificing model quality.

Compared to our previous release, Nemotron Nano V2 VL (NVIDIA et al., 2025c), Nemotron 3 Nano Omni introduces several key design choices and architectural advances:

  1. 1.

    Improved LLM Backbone. We replace the dense Nemotron Nano V2 12B hybrid backbone with the Nemotron 3 Nano 30B-A3B Mixture-of-Experts (MoE) hybrid backbone, enabling more efficient processing of long multimodal sequences and higher inference throughput.

  2. 2.

    Native Audio Support. We extend the model to natively support audio inputs in addition to text, images, and video.

  3. 3.

    Dynamic Image Resolution. We replace the tiling-based image processing approach with a dynamic resolution strategy that better preserves native aspect ratios.

  4. 4.

    Temporal Video Compression. We introduce Conv3D-based temporal compression for video, achieving a 2×\times reduction in temporal tokens.

  5. 5.

    Extended Context Length. We increase the maximum context length from 128K to 256K tokens, improving performance on long-context multimodal reasoning tasks.

Training an omni-modal MoE model introduces challenges in modality alignment, training stability, and data balancing across heterogeneous sources. To preserve the strong text reasoning capabilities of the base LLM while improving multimodal performacne, we adopt a multi-stage training strategy that progressively introduces new modalities and scales context length. This staged approach mitigates catastrophic forgetting and stabilizes cross-modal alignment during training.

Driven by these technical improvements, Nemotron 3 Nano Omni achieves substantial gains over Nemotron Nano V2 VL across a wide range of tasks. In particular, it attains leading results in document understanding, audio-visual reasoning, and audio benchmarks, ranking at or near the top of leaderboards such as OCRBench-V2 (Liu et al., 2024), MMLongBench-DOC (Ma et al., 2024), VoiceBench (Chen et al., 2024a), WorldSense (Hong et al., 2026), and DailyOmni (Zhou et al., 2026).

These improvements also translate into higher inference efficiency and lower latency. On NVIDIA B200, Nemotron 3 Nano Omni achieves 3×\times higher single-stream output token throughput than Qwen3-Omni (Xu et al., 2025) and 9×\times higher output token throughput per GPU at a fixed interactivity target. Compared with Nemotron Nano V2 VL, Nemotron 3 Nano Omni provides 3×\times higher throughput at the same interactivity target and 2×\times higher single-stream output token throughput. Nemotron 3 Nano Omni ranks as the most cost-efficient open video understanding model on MediaPerf.

Along with this report, we are releasing the model checkpoints on HuggingFace:

We are also releasing part of our training datasets, pipelines, and code:

The remainder of this paper is organized as follows. Section 2 describes the model architecture. Section 3 details the training pipeline and datasets. Section 4 presents evaluation results across all modalities.

Our model follows an encoder-projector-decoder design, combining the Nemotron 3 Nano 30B-A3B (NVIDIA et al., 2025a) language model with modality-specific encoders for vision and audio, connected via MLP projectors. An overview of the architecture is shown in Figure 1. The vision encoder is based on C-RADIOv4-H (Ranzinger et al., 2026; Heinrich et al., 2025), while the audio encoder is initialized with Parakeet-TDT-0.6B-v2 (Xu et al., 2023; Rekesh et al., 2023; Sekoyan et al., 2025).

Figure 1
Figure 1: Nemotron 3 Nano Omni architecture. For encoding images and videos we use dynamic resolution. Additionally, videos use Conv3D and optionally Efficient Video Sampling for higher throughput. Audio inputs are encoded using Parakeet v2 audio encoder. Visual, audio, and text tokens are concatenated and fed to the LLM.

To handle varying image resolutions, we replace the tiling strategy used in Nemotron Nano V2 VL (NVIDIA et al., 2025c) with dynamic resolution processing that preserves the native aspect ratio. Each image is decomposed into a variable number of 16×1616\times 16 patches, with the total number of visual tokens per image constrained between 1,024 and 13,312. This equates to an image size of 512×512512\times 512 and 1840×18401840\times 1840, respectively, for square images. Prior to projection, we apply a pixel shuffle operation with 4×\times downsampling to reduce the token count presented to the language model. For video frames, we use a Conv3D patch embedder that compresses every two frames into one, leading to a 2×2\times reduction in the total number of tokens for video inputs.

Audio inputs are resampled to 16 kHz mono and encoded using the Parakeet-TDT-0.6B-v2 FastConformer encoder. We first compute log-mel spectrogram features with a 10 ms hop size, followed by three stride-2 convolutional subsampling layers, resulting in an overall ∼\sim8×\times temporal downsampling. This yields approximately 12.5 tokens per second of audio (i.e., ∼\sim80 ms per token). Audio streams are segmented into 30-second clips (corresponding to ∼\sim375 tokens per clip) with the last clip accounting for the remainder. Streams shorter than 30 seconds are not padded. We train the model to handle inputs ranging from 0.5 second to 20 minutes, but the model context length can accommodate audio input of over 5 hours.

For multimodal inputs containing both visual and audio streams (e.g., videos with audio), modality tokens are interleaved in temporal order during sequence construction to enable joint temporal reasoning across modalities.

Figure 2
Figure 2: Staged training recipe for the v3 omni-modal model. The pipeline first performs vision SFT, then joint omni SFT while progressively extending context length, followed by omni-modal RL training.

Training an omni-modal reasoning model with heterogeneous encoders requires careful orchestration. To this end, we adopt a staged training strategy that first performs supervised fine-tuning (SFT) to progressively align modalities, improve multi-modal instruction-following and extend context capacity, followed by reinforcement learning (RL) to further refine reasoning and safety. Figure 2 illustrates the overall progression of these stages.

Our SFT pipeline is split into seven stages that progressively introduce new modalities and increase context length. This curriculum is designed to promote stable cross-modal alignment and mitigate catastrophic forgetting while improving multi-modal understanding. Detailed descriptions of each stage are provided in Sections 3.1.1–3.1.7, and an overview is provided in Table 1.

Table 1
Number of SamplesNumber of TokensPrimary Data Domains
Stage 09.35M15.5BCaptioning, OCR, document, VQA
Stage 186.3M214.8BComprehensive vision-language SFT
Stage 259.2M11.4BASR (Granary)
Stage 3242.0M100.5BASR, sound, music, speech understanding
Stage 430.5M57.3BVision, video, audio, text, omni, safety
Stage 56.08M33.5BLong video, omni, reasoning
Stage 6623K34.0BUltra-long documents, long-context text
Total (all stages)434.1M466.9B
Table 1: Approximate values for the total number of samples and tokens (including masked tokens from the prompt) in the training datasets across the SFT stages. This includes any sample repetitions.
View original tableOriginal Table 1

We begin by training only the vision MLP projector to align the vision and language modalities with a maximum context length of 16384, while keeping all other components frozen. This stage uses approximately 9.35 million vision–text samples (∼\sim 15.5B tokens), including a portion of the Stage 1 dataset (see Section 3.1.2) and covering a diverse set of tasks, including image captioning, visual grounding, OCR, document understanding, GUI understanding, and general visual question answering.

After training the vision projector, we unfreeze both the language model and the vision encoder for joint vision–language fine-tuning. During this stage, the model develops its core vision-language capabilities. The training data builds upon the SFT Stage 1 dataset used in Nemotron Nano V2 VL (NVIDIA et al., 2025c), with several key enhancements.

First, we replace its text-only subset with a portion of the SFT dataset from Nemotron 3 Nano 30B-A3B, resulting in higher-quality text reasoning samples. Second, we improve label quality by re-annotating noisy subsets using models from the Qwen3-VL series (Yang et al., 2025). Third, we enhance the availability and quality of reasoning traces by incorporating both human-annotated and model-generated chains of thought, leveraging models from the Qwen3-VL (Yang et al., 2025), Qwen3.5 (Qwen Team, 2026), and Kimi-K2.5 (Kimi Team et al., 2026) families.

Finally, we expand coverage across domains, including GUI understanding, visual grounding, charts, tables, document understanding, and video understanding, as well as across multiple languages. This is achieved through a combination of publicly available datasets, as well as internally curated data, including human annotation. To increase domain coverage, we additionally develop fully-synthetic data pipelines ensuring broad representation across domains, question types, and visual diversity. Guided by the gaps identified in the training blend, we source relevant data and generate synthetic question-answer pairs at scale using frontier open-source models such as Qwen3-VL (Yang et al., 2025), Qwen 3.5 (Qwen Team, 2026), GPT-OSS (OpenAI, 2025), Nemotron-Parse (Chumachenko et al., 2025), and DeepSeek-OCR (Wei et al., 2025). For each domain, we generate question-answer pairs from images, videos, or OCR extracted from images using domain-specific instructions. This is followed by distillation of reasoning traces and strict filtering of the resulting samples to ensure data correctness, usefulness, and overall quality.

The resulting dataset comprises approximately 86.3M samples (∼\sim 214.8B tokens), including sample repetitions.

Analogous to Stage 0 for vision, this stage warms up the audio MLP projector (Chen et al., 2024b) while keeping the LLM, vision encoder, and Parakeet-TDT audio encoder all frozen.

The training data consists of the Granary v1.1 ASR dataset (Koluguri et al., 2025), comprising approximately 59.2M samples (∼\sim11.4B tokens) of diverse automatic speech recognition data across varied acoustic conditions, speaking styles, and languages.

Building on Stage 2, this stage unfreezes the Parakeet-TDT audio encoder while keeping the LLM backbone and vision encoder frozen. The audio encoder and its associated projector are jointly trained on an expanded audio corpus.

As shown in Table 2, this stage is trained using a mixture of ASR data along with sound, music, and speech understanding. Audio samples are paired with captions, multiple-choice questions, and open-ended questions, with a subset further augmented with reasoning traces. Our synthetic data generation pipeline leverages open models like Qwen3-Omni-30B-A3B to produce captions and specialized music tools to produce metadata. These outputs are then used to generate QA pairs via GPT-OSS-120B.

Table 2
Dataset typeNumber of samples% of total tokensNumber of tokens
ASR113.8M22.7%22.8B
Sound understanding61.0M24.4%24.5B
Music understanding19.8M43.3%43.5B
Speech understanding47.5M9.6%9.6B
Total242.0M100.5B
Table 2: Dataset composition for the audio pretraining stage.
View original tableOriginal Table 2

This is the first stage that jointly trains on all modalities. All model parameters, including the LLM backbone, are trainable. The data mixture combines vision SFT, text instruction following, safety, video understanding, omni (audio+video) QA and captioning, ASR, and audio reasoning data.

Table 3
Dataset typeNumber of samples% of total tokensNumber of tokens
Vision data14.6M53.4%30.6B
Text data948K6.1%3.5B
Text safety data14K0.02%10.4M
Image safety data9K0.02%10.0M
Short video data1.3M11.0%6.3B
Short video reasoning data388K4.2%2.4B
Short video omni data251K2.8%1.6B
ASR data2.9M1.1%640M
Audio reasoning data765K4.4%2.5B
Audio data9.3M16.9%9.7B
Total30.5M57.3B
Table 3: Dataset composition for Stage 4: Joint Omni SFT at 16k context length.
View original tableOriginal Table 3

Table 3 summarizes the data composition. The dominant sources are the vision dataset (30.6B tokens), the audio dataset (9.7B tokens), and the short video data (6.3B tokens). The omni-modal data used in this stage is a blend of audio-visual captions, open-ended QA and MCQ style QA. Videos less than 2 minutes length are used as source media for this data. The question-answer pairs and captions are synthetically generated by first extracting audio-visual metadata from videos and then using that metadata for question-answer generation and summarization using open-source models Qwen3-Omni-30B-A3B and GPT-OSS-120B. The audio reasoning dataset comprises speech-to-text conversations synthesized by converting text SFT user turns into spoken form and generating LLM responses to a curated subset of ASR prompts.

This stage extends the context length to 49,152 tokens with all model parameters trainable. The data mixture is rebalanced to emphasize longer sequences, with reduced sampling of short-context data and increased weight on medium and long video, omni, and reasoning data.

Table 4
CategoryNumber of samples% of total tokensNumber of Tokens
ASR650K0.4%0.12B
Audio2.84M11.3%3.80B
Vision1.17M9.8%3.28B
Text101K7.2%2.42B
Safety45K0.1%0.04B
Video (short)25K0.6%0.21B
Video (medium)96K5.8%1.95B
Video (long)74K3.3%1.11B
Video reasoning167K10.2%3.42B
Omni (short)6K0.3%0.09B
Omni (medium+long)710K39.1%13.10B
Omni reasoning198K11.8%3.94B
Total6.08M100%∼\sim33.5B
Table 4: Stage 5 (Omni SFT 48k) data composition by category.
View original tableOriginal Table 4

Table 4 shows the per-category breakdown. Compared to Stage 4, this stage has a much higher proportion of long-context data: medium and long video, omni data with joint audio-video understanding, and reasoning traces. Short video and omni data are downsampled, while medium/long omni data and reasoning data receive the bulk of the training budget. To improve safety coverage, we additionally incorporate the safety data blend from Nemotron 3 Super (NVIDIA et al., 2026).

For the 48k SFT stage, omni-modal data comprising reasoning and non-reasoning single-turn QA is synthesized from diverse domains and categories. The pipeline segments videos into 20-second clips, extracts audio-visual metadata using multimodal models such as Qwen3-Omni-30B-A3B, and generates QA pairs and reasoning traces via open-source reasoning models such as GPT-OSS-120B.

This stage extends the context length to 262,144 and is intended to significantly increase the model’s long context capabilities. The data for this stage consists of ∼\sim34.0B tokens across long-context text-only and vision domains such as long-context reasoning and long document understanding (see Table 5). It particularly improves the model’s ability to analyze documents spanning 10 to 100+ pages, including reasoning over text, charts, and complex tables. We assemble a diverse collection of long-form documents, including academic papers, financial reports, and presentations, and leverage vision-language models to generate synthetic question-answer pairs and reasoning traces at the page, multi-page, and full-document levels. To support long-document understanding, we release runnable data pipeline recipes3^{3} using NeMo Data Designer (The NeMo Data Designer Team, 2025).

Table 5
Dataset typeNumber of samples% of total tokensNumber of tokens
Long Context Vision508K90.9%30.9B
Text63K7.3%2.5B
Long Context Text2.2K1.5%506M
Vision50K0.3%106M
Total623K100%34.0B
Table 5: Dataset composition for the ultra-long context Stage 6.
View original tableOriginal Table 5

The audio encoder and projector are frozen during this stage to focus model capacity on long-context text and document understanding.

Table 6
Stage 0Stage 1Stage 2Stage 3Stage 4Stage 5Stage 6
Context Length16K48k256k
Max Video Frames–64––64256256
Global BS128256512256128
CP-216
LR10−310^{-3}5×10−55\times 10^{-5}10−310^{-3}2.5×10−52.5\times 10^{-5}10−510^{-5}10−610^{-6}
Minimum LR10−510^{-5}010−510^{-5}010−710^{-7}0
Linear Warmup Fraction0.10.010.1
Weight Decay0.010.050.010.05
Trainable ModulesVision ProjectorAll except audioAudio ProjectiorAudio Encoder & ProjectorAllAll except audio
# GPU Nodes326412864
Table 6: Summary of the SFT training hyperparameters. All stages use the AdamW optimizer (β1=0.9\beta_{1}{=}0.9, β2=0.999\beta_{2}{=}0.999), a cosine LR decay, BF16 precision, TP=2 and EP=32
View original tableOriginal Table 6

All SFT stages are trained using the Megatron framework (Shoeybi et al., 2019)4^{4} , together with Transformer Engine5^{5} and the Megatron Energon6^{6} dataloader. Training is conducted on 32–128 nodes of NVIDIA H100 GPUs, depending on the stage.

We employ 2-way tensor parallelism (TP), 32-way expert parallelism (EP), and sequence parallelism to efficiently scale training. All stages are trained in BF16 mixed precision and use online sequence packing with a balanced greedy knapsack algorithm to maximize GPU utilization.

To fit long sequences in GPU memory, we use selective activation recomputation for the LLM backbone (recomputing core attention, MLP, LayerNorm, and MoE activations) and full block-level recomputation for all 32 vision encoder layers. Sound model activations are recomputed starting from Stage 4. Vision projection and sound projection recomputation are enabled from Stage 5 onward to support the increased memory requirements of longer sequences. Additionally, context parallelism is introduced in later stages, with 2-way and 16-way CP in Stages 5 and 6, respectively, to accommodate increasingly long sequence lengths.

The vision encoder’s CPE layers are kept in eval mode in stages 1, 4 and 5 to stabilize training. For videos, we sample up to 64 frames in Stages 1 and 4, and up to 256 frames in Stages 5 and 6. We also employ video augmentation that randomly selects the target number of patches per video frame from {256,512,768,1024}\{256,512,768,1024\}. This allows us to reduce the image resolution at inference time, while scaling up the number of frames, to improve temporal information without increasing the number of tokens. We use the AdamW optimizer with β1\beta_{1} and β2\beta_{2} set to 0.90.9 and 0.9990.999, respectively, and a cosine annealing schedule with a linear warmup. Table 6 summarizes the training hyperparameters for SFT stages.

After SFT, we apply multiple rounds of reinforcement learning to further improve instruction following, reasoning, and safety-alignment for text, image, and video modalities. We design a curriculum learning pipeline for post-training: (1) Preference Optimization, (2) Text-RL-stage-1, (3) Image-RL, (4) Omni-RL, and (5) Text-RL-stage-2.

To align our model using both preference-level and quality-level supervision, we adopt Mixed Preference Optimization (MPO) (Wang et al., 2024a), which combines a preference loss and a quality loss during the offline reinforcement learning stage. Specifically, we employ Direct Preference Optimization (DPO) (Rafailov et al., 2023) as the preference loss and Binary Classifier Optimization (BCO) (Wang et al., 2024a) as the quality loss. To construct the training data, we apply rejection sampling to generate candidate responses in the vision domain and assign binary labels based on outcome correctness, yielding positive samples for accepted responses and negative samples for rejected ones.

During text-only RL, we only train the LM parameters of the model via multi-environment RLVR/RLHF for improving general capabilities. We reuse the RL data and infrastructure from the post-training of Nemotron 3 Nano and Super (NVIDIA et al., 2025b; NVIDIA et al., 2026). As part of our staged multi-modal training, during text-only RL stages we additionally freeze the LM input token embedding parameters to mitigate representational drift between multi-modal stages.

ImageRL is the first stage of our multimodal RL pipeline. We employ outcome-based RL on visual reasoning tasks, which can be divided into the following categories

  • •

    Chart, document, and text-rich image reasoning: numerical, comparative, and trend reasoning over plots, tables, diagrams, infographics, and natural images containing text (∼\sim28K).

  • •

    STEM and mathematical problems: geometry, algebra, functions, and counting, in both English and Chinese (∼\sim19K).

  • •

    Game and puzzle reasoning: rule-based reasoning over rendered game-board states (∼\sim12K).

  • •

    Visual question answering: open-ended and multiple-choice questions covering spatial relations, attribute recognition, and yes/no judgements (∼\sim8K).

  • •

    Visual grounding: click-coordinate prediction on desktop, mobile, and web screenshots (∼\sim7K).

During training, each prompt is graded by a [0,1][0,1] scalar that linearly combines an outcome score. The outcome score comes from one of four rule-based verifiers, chosen per prompt: string-match for free-form text answers, mathruler for symbolic equivalence on numeric and algebraic answers, multiple-choice for selected-letter answers, and gui-coordinate for click-target predictions, where the reward decays smoothly with distance from the target. The format score rewards a single <think> reasoning block followed by a single \boxed answer, with partial credit when the policy emits extra reasoning or boxed entries. This keeps correct answers from being zeroed out by the surface format errors common in VLM checkpoints after SFT, while still discouraging verbose multi-answer outputs.

To ensure an informative learning signal, we apply pass-rate filtering using 8 rollouts per prompt from the initial policy checkpoint, retaining only prompts whose empirical pass rate is below 0.8; prompts that are trivially solvable at initialization are discarded. The filtering is based on the same verifiers that are used during training. We additionally include a small set of unanswerable or image-text-mismatched prompts to train the policy to abstain when visual evidence is insufficient.

The resulting corpus and verifier suite are inherited by OmniRL as the image component of its mixed-modality training mixture.

Understanding is inherently challenging, and extending it to multiple modalities further increases complexity due to the need for sophisticated cross-modal reasoning. Prior advances in text reasoning have demonstrated the effectiveness of structured reasoning in improving model performance (Wei et al., 2022; Shao et al., 2024). More recently, omni-modal reasoning has also been shown to be beneficial to omni and video tasks (Ye et al., 2026). Motivated by these findings, we develop a unified reinforcement learning training stage aimed at enhancing the model’s capacity for coherent reasoning across images, videos, and audio modalities.

To make omni RL training possible, we curate a diverse, omni-modal training corpus of approximately 120K prompts spanning 113 sub-datasets across four modality groups: image, video, audio, and text-only reasoning. The dataset is constructed by aggregating and filtering data from multiple sources: (1) Omni RL data (∼\sim17.6K samples): synthetic data generated from video content with accompanying audio, covering diverse visual understanding and temporal reasoning tasks; (2) Video RL data (∼\sim8.5K): video-only question–answer pairs targeting spatial, temporal, and causal reasoning. (3) Image RL data (∼\sim32K): a large-scale image understanding set drawing from OCR (∼\sim10.5K), chart analysis (∼\sim8.9K), game-related visual QA (∼\sim11.9K), GUI grounding (∼\sim7.1K), and additional curated domains; (4) Audio RL data (∼\sim4.2K) and ASR (∼\sim3.8K): audio question-answering and automatic speech recognition tasks at various utterance lengths. We incorporate an ASR verifier to stabilize the speech recognition capability of our model. The reward is 1 - WER, where WER is computed after text normalization.

To ensure balanced difficulty and effective learning signals, we apply pass-rate filtering based on the initial policy checkpoint. We retain only prompts on which the base model achieves a pass rate between 0.1 and 0.9 (with stricter 0.3–0.7 bands for AudioQA), thereby excluding prompts that are either trivially solvable or entirely intractable for the current policy. The verification pipeline supports five task types: multiple-choice (34%), string matching (31%), mathematical rule-based verification (26%), GUI coordinate grounding (6%), and ASR evaluation (3%). We additionally include a small set of unanswerable or mismatched samples (∼\sim4K) to train the model to appropriately abstain when evidence is insufficient.

Training is conducted on NVIDIA B200 and H100 GPU cluster using a Ray-based distributed training framework built on NeMo-RL (NVIDIA, 2025). The global batch size is set to 4,096 with 16 rollouts for each prompt and a micro-batch size of 1. We apply an adapted version of Group Sequence Policy Optimization (GSPO) (Zheng et al., 2025; Shao et al., 2024) as the RL training algorithm.

We use a multimodal deduplication strategy during the generation phase to leverage a unique multimodal tensor with rollouts associated with each prompt. We leverage tensor, expert, and context parallelism during training. All experiments are run with the AdamW optimizer with β1\beta_{1} and β2\beta_{2} set to 0.90.9 and 0.9990.999, respectively, and a linear warmup.

In Sections 4.1–4.4, we conduct a comprehensive evaluation of the model’s ability to reason over vision, audio, and text inputs, and present the corresponding results. In Section 4.6, we analyze the efficiency gains achieved through Efficient Video Sampling (EVS) for video inputs. In Sections 4.7 and 4.8, we examine the impact of quantization on model accuracy and efficiency.

For the vision and audio evaluations in Sections 4.1–4.3, we use the VLMEvalKit (Duan et al., 2025)7^{7} framework with a vLLM (Kwon et al., 2023)8^{8} backend. Text evaluations are conducted using the NeMo-Skills9^{9} framework.

We conduct a comprehensive evaluation of our model on the following broad categories:

  1. 1.

    STEM Reasoning: MMMU (Yue et al., 2024), MathVista-Mini (Lu et al., 2024)

  2. 2.

    Document Understanding, OCR & Charts: MMLongBench-Doc (Ma et al., 2024), OCRBench(Liu et al., 2024), OCRBench-V2 (Fu et al., 2024), ChartQA (Masry et al., 2022), AI2D (Kembhavi et al., 2016), TextVQA (Singh et al., 2019), DocVQA (Mathew et al., 2021b), InfoVQA (Mathew et al., 2021a), OCR-Reasoning (Huang et al., 2025), CharXiv (Wang et al., 2024b)

  3. 3.

    Visual Grounding & Spatial Reasoning: TreeBench (Wang et al., 2025), CV-Bench (Tong et al., 2024), RefCOCO(Kazemzadeh et al., 2014)

  4. 4.

    GUI Understanding: ScreenSpot (Cheng et al., 2024), ScreenSpot-v2 (Wu et al., 2024), ScreenSpot Pro (Li et al., 2025), OSWorld (Xie et al., 2024)

  5. 5.

    Video Understanding: Video-MME (Fu et al., 2025)

As shown in Table 7, we observe significant improvements compared to Nemotron Nano V2 VL across all benchmarks and even outperforms Qwen3-Omni on several categories.

Table 7
TaskBenchmarkNemotron 3 Nano OmniNemotron Nano V2 VLQwen3-OmniQwen3.5-Omni
Open-Source✓✓✓✗
Size30B-A3B12B30B-A3BFlash
ModeReasoning offReasoning onReasoning offReasoning on
STEM ReasoningMMMU (val)55.270.855.367.875.676.9
MathVista-Mini71.982.869.075.580.082.9
Document Understanding, OCR & ChartsMMLongBench-Doc46.157.532.138.049.553.6
OCRBench88.386.685.683.586.089.1
OCRBenchV2 (EN/ZH)65.8/52.067.0/52.762.0/44.254.8/39.8--
ChartQA (Test)89.990.389.884.989.5-
DocVQA (Test)93.395.694.793.295.3-
AI2D (Test)88.588.587.284.786.6289.0
TextVQA (Val)85.181.085.476.181.7-
InfoVQA (Test)83.686.879.480.483.31-
OCR-Reasoning22.254.1421.033.949.9-
CharXiv (RQ/DQ)49.1/81.963.6/88.941.7/76.541.3/77.261.1/-64.4/-
Visual Grounding & Spatial ReasoningTreeBench43.751.638.542.5--
CV-Bench84.284.081.078.3--
RefCOCO80.690.5---92.6
GUIScreenSpot90.389.339.442.5--
ScreenSpot-v293.492.841.742.8--
ScreenSpot-Pro59.357.84.85.559.7-
OSWorld-47.4-11.129.0-
Video UnderstandingVideoMME (w/o sub)70.872.266.063.070.577.0
Table 7: Comparison of Nemotron 3 Nano Omni with our previous release, Nemotron Nano V2 VL, as well as other state-of-the-art omni-modal models.
View original tableOriginal Table 7

We evaluate our model across three broad categories:

  1. 1.

    Automatic Speech Recognition (ASR): We use the OpenASR leaderboard (Srivastav et al., 2026), and report word error rate on its English subset, including AMI, Earnings22, GigaSpeech, LibriSpeech, SPGISpeech, TED-LIUM and VoxPopuli. For long-form ASR, we additionally evaluate on TED-LIUM Longform (Fox et al., 2024), which tests transcription quality and long-context consistency on continuous speech.

  2. 2.

    Audio Understanding: We evaluate on MMAU (Sakshi et al., 2024), a benchmark of ∼\sim10k audio clips with QA pairs spanning speech, environmental sounds, and music, covering 27 skills in information extraction and multi-step reasoning.

  3. 3.

    Voice Interaction & Reasoning: We use VoiceBench (Chen et al., 2024a), which assesses LLM-based voice assistants on realistic spoken interactions, evaluating knowledge, instruction following, and safety across diverse speakers and environments.

Table 8
TaskBenchmarkSubtaskNemotron 3 Nano OmniQwen3-OmniQwen3.5-Omni
Open-Source✓✓✗
Size30B-A3B30B-A3BFlash
ASROpenASR (Reasoning off)AMI11.0912.52–
Earnings2211.2712.3–
GigaSpeech9.668.49–
LibriSpeech (clean)1.571.521.3
LibriSpeech (other)2.963.222.4
SPGISpeech1.983.69–
TED-LIUM3.442.38–
VoxPopuli5.68.26–
OpenASR Avg5.956.55–
Long-form ASRTED-LIUM (Reasoning off)–3.112.4–
Audio UnderstandingMMAU (Reasoning off)Music74.2––
Audio76.9––
Speech72.8––
MMAU Avg74.677.580.4
Voice InteractionVoiceBench (Reasoning on)IFEval88.780.6–
BBH91.188.9–
AdvBench10097.2–
AlpacaEval95.096.4–
CommonEval91.390.5–
WildVoice91.790.5–
OpenBookQA93.094.3–
MMSU82.383.0–
SD-QA71.478.1–
VoiceBench Avg89.488.887.8
Table 8: Comparison of Nemotron 3 Nano Omni with other state-of-the-art open source models on diverse audio and speech tasks, ASR (OpenASR), long-form ASR (TED-LIUM), MMAU, and VoiceBench. ASR tasks are measured by word error rate (lower is better). For MMAU and VoiceBench, higher is better. ASR and MMAU use non-reasoning settings, while VoiceBench uses reasoning.
View original tableOriginal Table 8

As shown in Table 8, Nemotron 3 Nano Omni outperforms Qwen family models on ASR and VoiceBench benchmarks.

We evaluate our model on audio-visual perception and reasoning using two complementary benchmarks:

  1. 1.

    DailyOmni (Zhou et al., 2026): an audio-visual QA benchmark for cross-modal reasoning in daily scenarios, with 684 videos (segmented into 30 and 60 secoond clips) and 1,197 multiple-choice questions across six tasks, testing temporal alignment, event understanding, causal reasoning, and cross-modal consistency.

  2. 2.

    WorldSense (Hong et al., 2026): a large-scale omni-modal benchmark with 1,662 long-context videos and 3,172 multiple-choice questions across 26 tasks, evaluating long-range dependencies, sound grounding, temporal reasoning, and complex cross-modal inference.

As shown in Table 9, Nemotron 3 Nano Omni outperforms Qwen3-Omni on both reasoning on and off modes.

Table 9
BenchmarkNemotron 3 Nano OmniQwen3-OmniQwen3.5-Omni
Open-Source✓✓✗
Size30B-A3B30B-A3BFlash
ModeReasoning offReasoning onInstructThinking
DailyOmni74.574.171.973.681.8
WorldSense55.255.454-57.8
Table 9: Comparison of Nemotron 3 Nano Omni with other state-of-the-art open source models on Video+Audio (Omni) benchmarks, measured by accuracy (higher is better).
View original tableOriginal Table 9

We conduct all pure-text evaluations with a maximum output length of 131,072 tokens, temperature set to 1.0, and top-p of 1.0. We report Pass@1 average of 8 runs for AIME-2025; an average of 4 runs for GPQA-Diamond (Rein et al., 2023); and score of 1 run for SciCode (Tian et al., 2024), LiveCodeBench v5 (07/24 - 05/25) (Jain et al., 2024), IFBench (Zhou et al., 2023), and TauBench V2 (telecom). We additionally include MMLU-Pro to assess general academic and knowledge-intensive reasoning.

Table 10 shows the evaluation on a selected text benchmarks compared to the Nemotron 3 Nano 30B-A3B LLM that is used as the LLM backbone. The goal of the Omni model is maintain text benchmarks of the LLM while adding vision and audio understanding capabilities.

Table 10
BenchmarkNemotron 3 Nano OmniNemotron 3 Nano 30B-A3BQwen3-Omni
Open-Source✓✓✓
Size30B-A3B30B-A3B30B-A3B
MMLU-Pro77.378.361.6
GPQA (no tools)72.273.073.1
LiveCodeBench63.268.3-
AIME25 (no tools)82.189.173.7
IFBench (prompt)74.271.5-
AA-LCR41.035.9-
TauBench V2 (Telecom)42.742.2-
SciCode32.033.3-
Table 10: Comparison of Nemotron 3 Nano Omni, Nemotron 3 Nano LLM, and Qwen3-Omni across selected text-only benchmarks.
View original tableOriginal Table 10

We study the effect of inference-time reasoning budgets by evaluating model performance under two settings: (1) a base configuration with a maximum sequence length of 16,384 tokens, and (2) a reasoning-enabled configuration with a 13K reasoning budget, a 1,024-token grace period, and a maximum sequence length of 16,384 tokens (Table 11).

Our results suggest that reasoning budget adjustment yields accuracy gains on select benchmarks under reasoning-on mode, with no degradation observed on the remaining ones. These gains with budget control may arise from the early termination of malformed reasoning traces with repetition loops on out-of-distribution tasks, as well as the truncation of overly verbose reasoning chains for problems requiring minimal or straightforward reasoning.

Table 11
BenchmarkMathVista-MiniMMLongBench-DocDocVQA (Val)Charxiv(RQ)RefCOCOVideoMME
w.o. reasoning budget80.354.595.361.890.467.5
w. reasoning budget82.856.895.26490.670.3
Table 11: Effect of reasoning budget across several key benchmarks.
View original tableOriginal Table 11

Nemotron 3 Nano Omni reduces the cost of long video inputs through two stacked mechanisms on the vision side. Conv3D is an architecture change applied during both training and inference: every T=2T=2 consecutive frames are fused into a single “tubelet” before the first ViT block. This halves the number of vision tokens flowing through the ViT and the LLM, cutting both ViT prefill cost and LLM-side prefill, attention compute, and KV-cache footprint. EVS (Efficient Video Sampling)(Bagrov et al., 2025) is a runtime-only feature that drops video tokens after the ViT blocks and the vision adapter, immediately before they reach the LLM. For each spatial position (h,w)(h,w) it computes the cosine dissimilarity between consecutive tubelets and keeps the globally most-dissimilar tokens up to a budget set by the pruning rate qq; the entire first tubelet is pinned to maximum dissimilarity so it is always retained as an anchor. The two mechanisms compose multiplicatively: Conv3D halves the number of tokens in the temporal dimension, EVS prunes the remaining tokens in the spatial dimension.

Table 12 compares the four combinations of Conv3D and EVS for both BF16 and NVFP4 checkpoints, with EVS fixed at q=0.5q=0.5. Each accuracy column reports per-benchmark scores at 128 and 256 sampled frames, with reasoning off. Accuracy is averaged across three runs with identical settings. TTFT is averaged across five concurrency-1 aiperf runs against a synthetic 512-frame, 512×512512\times 512 video at 30 fps.

Table 12
ConfigurationDailyOmniLongVideoBenchVideo-MMEWorldSenseAvgTTFT
128f256f128f256f128f256f128f256f128f256f(ms)
BF1674.7474.7766.2367.9069.1369.7054.8054.5066.2366.727969
BF16 + EVS74.4674.3865.7067.8069.8070.1054.8755.4066.2166.926452
BF16 + Conv3D74.4174.2466.3067.2068.7070.7054.8354.4366.0666.645984
BF16 + Conv3D + EVS73.7473.5465.7066.6068.6070.7055.0754.4365.7866.325313
NVFP471.7171.6866.0766.9369.2369.9753.2752.4565.0765.266885
NVFP4 + EVS71.6571.7665.5067.3069.8070.9352.9052.6064.9665.655977
NVFP4 + Conv3D70.3770.8465.9066.4368.7070.3052.6352.2764.4064.965635
NVFP4 + Conv3D + EVS70.7670.6564.9766.5068.4770.1752.5052.7064.1765.005083
Table 12: Per-benchmark accuracy (128 frames / 256 frames) and TTFT at concurrency 1 across Conv3D and EVS combinations, with EVS rate q=0.5q=0.5, reasoning off.
View original tableOriginal Table 12

Both mechanisms significantly reduce TTFT on BF16: Conv3D alone drops it from 7969 ms to 5984 ms (−25-25%), EVS alone drops it to 6452 ms (−19-19%), and stacking them yields 5313 ms (−33-33% versus the baseline) at a cost of about half a point of average accuracy. The same ordering holds on NVFP4. Underlying these gains is a substantial reduction in the number of input tokens for the LLM: a 512-frame video produces ∼\sim141k input tokens without either mechanism, drops to ∼\sim75k with Conv3D enabled (−47-47%), and drops further to ∼\sim42k with Conv3D combined with EVS at q=0.5q=0.5 (−70-70% versus the baseline).

Table 13 sweeps the EVS pruning rate qq on BF16 with Conv3D enabled. Per-benchmark accuracy is essentially flat through q=0.7q=0.7, slightly reduces at q=0.8q=0.8, and drops noticeably beyond, with LongVideoBench being the most sensitive benchmark to aggressive pruning. TTFT improves monotonically through the range, for a ∼\sim 1414% reduction at q=0.7q=0.7 versus no-EVS.

Table 13
EVSDailyOmniLongVideoBenchVideo-MMEWorldSenseAvgTTFT
qq128f256f128f256f128f256f128f256f128f256f(ms)
none74.4174.2466.3067.2068.7070.7054.8354.4366.0666.645984
0.573.7473.5465.7066.6068.6070.7055.0754.4365.7866.325313
0.674.4174.4465.1066.5068.9070.9054.5754.4065.7466.565173
0.773.8273.7765.0065.4068.7070.3054.1053.8065.4165.825124
0.873.3873.2464.3064.8067.8070.1054.0053.7364.8765.475182
0.971.5471.7159.8061.0067.1068.3052.9752.8762.8563.474883
0.9569.3169.3155.6057.9064.6066.3051.6751.9360.2961.364804
Table 13: BF16 with Conv3D enabled, varying EVS pruning rate qq, reasoning off. Same column structure as Table 12.
View original tableOriginal Table 13

Inspired by the quantization recipe from Nemotron 3 Super, we pursued a mixed-precision strategy for FP4: routed MoE experts are quantized to NVFP4 (FP4 E2M1 values with per-block FP8 E4M3 scales over groups of 16 elements and an additional per-tensor FP32 global scale), while the Mamba in_proj / out_proj, shared experts, and attention o_proj are quantized to FP8 (per-tensor E4M3 values with a per-tensor FP32 scale). All remaining language-model layers are left in BF16, as are the vision and audio encoders and their MLP projectors. For the KV cache we use FP8, while the Mamba SSM state cache is kept at FP32 at serving time. This gives a model-weight footprint of 4.98 effective bits per weight (20.9 GB vs the 61.5 GB BF16 reference). For FP8 we quantize every linear layer in the language model to per-tensor E4M3 (with a per-tensor FP32 scale), with the exception of the MoE router and lm_head, and pair it with an FP8 KV cache. The vision and audio encoders and their MLP projectors are excluded entirely. This yields ∼\sim8.5 bpw (32.8 GB). We evaluated the quantized models across 25 text, image, video, and audio benchmarks and found a median accuracy drop of less than 1%1\% vs BF16 for both FP8 and NVFP4.

Table 14
BenchmarkBF16FP8NVFP4
Size (GB)61.532.820.9
Effective bpw16.008.54.98
MathVista-Mini71.9071.0571.30
CharXiv (RQ)49.1048.0547.95
OCR-Reasoning22.2023.4322.78
MMLongBench-Doc46.1045.8445.78
OCRBenchV2 (EN)65.8065.6365.77
OCRBenchV2 (ZH)52.0050.2450.39
CV-Bench84.2085.6285.27
VideoMME70.8069.4069.60
DailyOmni74.5074.0674.23
WorldSense-AVLM55.2054.4054.60
MMAU74.6274.5674.34
TedLium-Longform (WER↓\downarrow)3.113.123.04
HF-ASR avg, 8 short-form (WER↓\downarrow)5.955.975.95
Mean (11 non-ASR)60.5860.2160.18
Median (11 non-ASR)65.8065.6365.77
Δ\Delta vs BF16 (mean)—−0.37-0.37−0.40-0.40
Table 14: Accuracy of Nemotron 3 Nano Omni at BF16, FP8, and NVFP4 across the eval suite.
View original tableOriginal Table 14

NVFP4. Compared to BF16 precision, NVFP4 on NVIDIA B200 provides up to 7.5×\times the output token throughput at iso-interactivity (18200 tok/s versus 2400 tok/s, at 150 tok/s/user) on an single-image reasoning usecase.

Low-latency single-stream inference. Nemotron 3 Nano Omni delivers strong single-stream inference performance on NVIDIA B200, reaching more than 500 output tokens/s at a concurrency of 1. This low-latency generation rate is sustained at longer sequence lengths and with larger multimodal inputs, such as long videos or multi-document workloads, due to the hybrid architecture. This is approximately 2.4–2.9×\times faster than Qwen3-Omni, which reaches 175–210 output tokens/s depending on input size and sequence length, and 2×\times faster than Nemotron Nano V2 VL, which reaches 250 output tokens/s.

For a multi-document workload, Nemotron 3 Nano Omni achieves a time-to-first-token (TTFT) of approximately 1.3 s, compared to more than 2.5 s for Qwen3-Omni.

High-throughput serving. At maximum concurrency on a single NVIDIA B200, Nemotron 3 Nano Omni reaches 5000 output tokens/s on a multi-document workload. At an iso-interactivity target of 50 output tokens/s per user, the deployment provides 9×\times higher output throughput than Qwen3-Omni on long-video workloads and 7.5×\times higher output throughput on multi-document workloads. Compared with Nemotron Nano V2 VL, Nemotron 3 Nano Omni provides 3×\times higher throughput at the same interactivity target.

Experimental setup. All measurements use a single NVIDIA B200 GPU and vLLM nightly as of 2026-04-19 with EVS 50%. Nemotron 3 Nano Omni is evaluated in NVFP4, Qwen3-Omni with dynamic FP8 quantization, and Nemotron Nano V2 VL in NVFP4. Text ISL=50 and OSL=8000. The multi-document workload contains 32 images at 1024×\times1536 resolution. The long-video workload contains 512 frames at 512×\times512 resolution.

We introduced Nemotron 3 Nano Omni, an efficient omni-modal model that extends the Nemotron multimodal family with native audio support and consistently stronger reasoning across text, images, video, and audio. Built on the Nemotron 3 Nano 30B-A3B MoE hybrid backbone and augmented with the C-RADIOv4-H vision encoder and the Parakeet-TDT audio encoder, the model combines dynamic image resolution, Conv3D-based temporal video compression, and a 256K context length to process long, heterogeneous multimodal inputs with high accuracy. We use a multi-stage training recipe that progressively introduces new modalities and extends context to enable robust cross-modal alignment while preserving the text reasoning ability of the base LLM.

Across a broad evaluation suite, Nemotron 3 Nano Omni delivers consistent gains over Nemotron Nano V2 VL and achieves leading or competitive results on document understanding (OCRBench-V2, MMLongBench-Doc, ChartQA, CharXiv), agentic GUI use (ScreenSpot, ScreenSpot-Pro, OSWorld), long audio-video comprehension (WorldSense, DailyOmni), and voice interaction (VoiceBench), while retaining the text reasoning performance of the Nemotron 3 Nano 30B-A3B backbone. Combined with innovative multimodal token-reduction techniques, these capabilities translate into substantially lower inference latency and several-fold higher throughput than comparably sized models. We release model checkpoints in BF16, FP8, and FP4 formats alongside a large portion of our training data and code, with the goal of enabling the community to further advance efficient omni-modal modeling.

Core Model Development

Amala Sanjay Deshmukh, Kateryna Chumachenko, Tuomas Rintamaki, Matthieu Le, Tyler Poon, Danial Mohseni Taheri, Ilia Karmanov, Guilin Liu, Jarno Seppanen, Arushi Goel, Mike Ranzinger, Greg Heinrich, Guo Chen, Lukas Voegtle, Philipp Fischer, Timo Roman, Karan Sapra, Collin McCarthy, Shaokun Zhang, Fuxiao Liu, Hanrong Ye, Yi Dong, Mingjie Liu, Yifan Peng, Piotr Zelasko, Zhehuai Chen, Nithin Rao Koluguri, Nune Tadevosyan, Lilit Grigoryan, Ehsan Hosseini Asl, Pritam Biswas, Leili Tavabi, Yuanhang Su, Zhiding Yu, Peter Jin, Alexandre Milesi, Netanel Haber

Data Generation and Curation

Yao Xu, Sarah Amiraslani, Nabin Mulepati, Eric Tramel, Jaehun Jung, Ximing Lu, Brandon Cui, Jin Xu, Zhiqi Li, Shihao Wang, Yuanguo Kuang, Shaokun Zhang, Huck Yang, Boyi Li, Hongxu (Danny) Yin, Song Han, Bilal Kartal, Pavlo Molchanov, Adi Renduchintala, Charles Wang,, David Mosallanezhad, Soumye Singhal, Luis Vega, Katherine Cheung, Sreyan Ghosh, Yian Zhang, Alexander Bukharin, Venkat Srinivasan, Johnny Greco, Andre Manoel, Maarten Van Segbroeck, Suseella Panguliri, Rohit Watve, Divyanshu Kakwani, Shubham Pachori, Jeffrey Glick, Radha Sri-Tharan, Aileen Zaman, Khanh Nguyen, Shi Chen, Jiaheng Fang, Qing Miao, Wenfei Zhou, Yu Wang, Zaid Pervaiz Bhat, Varun Praveen, Arihant Jain, Ramanathan Arunachalam, Tomasz Kornuta, Ashton Sharabiani, Amy Shen, Wei Huang

Systems, Data and Infrastructure

Yi-Fu Wu, Ali Roshan Ghias, Huiying Li, Brian Yu, Nima Tajbakhsh, Chen Cui, Wenwen Gao, Li Ding, Terry Kong, Manoj Kilaru, Anahita Bhiwandiwalla, Marek Wawrzos, Daniel Korzekwa, Pablo Ribalta, Grzegorz Chlebus, Besmira Nushi, Ewa Dobrowolska, Maciej Jakub Mikulski, Kunal Dhawan, Steve Huang, Jagadeesh Balam, Yongqiang Wang, Nikolay Karpov, Valentin Mendelev, George Zelenfroynd, Meline Mkrtchyan, Qing Miao, Omri Almog, Bhavesh Pawar, Rameshwar Shivbhakta, Sudeep Sabnis, Ashrton Sharabiani, Negar Habibi, Geethapriya Venkataramani, Pamela Peng, Prerit Rodney, Serge Panev, Richard Mazzarese, Nicky Liu, Michael Fukuyama, Andrii Skliar, Roger Waleffe, Duncan Riach, Yunheng Zou, Jian Hu, Hao Zhang, Binfeng Xu, Yuhao Yang, Zuhair Ahmed

Inference and Optimization

Alexandre Milesi, Carlo del Mundo, Chad Voegele, Zhiyu Cheng, Nave Assaf, Andrii Skliar, Daniel Afrimi, Natan Bagrov, Ran Zilberstein, Ofri Masad, Eugene Khvedchenia, Natan Bagrov, Borys Tymchenko, Tomer Asida, Daniel Afrimi, Parth Mannan, Victor Cui

Safety

Michael Evans, Katherine Luna, Jie Lou, Pinky Xu, Guyue Huang, Negar Habibi, Michael Boone, Pradeep Thalasta, Adeola Adesoba, Dina Yared, Christopher Parisien, Leon Derczynski, Shaona Ghosh, Wes Feely, Micah Schaffer, Radha Sri-Tharan, Jeffrey Glick, Barnaby Simkin, George Zelenfroynd, Tomasz Grzegorzek, Rishabh Garg

Evaluation, Product and Legal

Aastha Jhunjhunwala, Sergei Kolchenko, Farzan Memarian, Haran Kumar, Shiv Kumar, Isabel Hulseman, Anjali Shah, Kari Briski, Padmavathy Subramanian, Joey Conway, Udi Karpas, Jane Polak Scowcroft, Annie Surla, Shilpa Ammireddy, Ellie Evans, Jesse Oliver, Tom Balough, Chia-Chih Chen, Sandip Bhaskar, Alejandra Rico, Bardiya Sadeghi, Seph Mard, Katherine Cheung, Meredith Price, Laya Sleiman, Saori Kaji, Wesley Helmholz, Wendy Quan

Leadership

Michael Lightstone, Jonathan Cohen, Jian Zhang, Oleksii Kuchaiev, Boris Ginsburg, Jan Kautz, Eileen Long, Mohammad Shoeybi, Mostofa Patwary, Oluwatobi Olabiyi, Andrew Tao, Bryan Catanzaro, Udi Karpas

References

  1. Bagrov et al. (2025) N. Bagrov, E. Khvedchenia, B. Tymchenko, S. Aharon, L. Kadoch, T. Keren, O. Masad, Y. Geifman, R. Zilberstein, T. Rintamaki, M. Le, and A. Tao Efficient video sampling: pruning temporally redundant tokens for faster VLM inference. External Links: 2510.14624, Link Cited by: §4.6.Source
  2. Chen et al. (2024a) Y. Chen, X. Yue, C. Zhang, X. Gao, R. T. Tan, and H. Li VoiceBench: benchmarking llm-based voice assistants. External Links: 2410.17196, Link Cited by: §1, item 3.Source
  3. Chen et al. (2024b) Z. Chen, H. Huang, A. Andrusenko, O. Hrinchuk, K. C. Puvvada, J. Li, S. Ghosh, J. Balam, and B. Ginsburg Salm: speech-augmented language model with in-context learning for speech recognition and translation. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 13521–13525. Cited by: §3.1.3.
  4. Cheng et al. (2024) K. Cheng, Q. Sun, Y. Chu, F. Xu, Y. Li, J. Zhang, and Z. Wu SeeClick: harnessing gui grounding for advanced visual gui agents. External Links: 2401.10935, Link Cited by: item 4.Source
  5. Chumachenko et al. (2025) K. Chumachenko, A. S. Deshmukh, J. Seppanen, I. Karmanov, C. Chen, L. Voegtle, P. Fischer, M. Wawrzos, S. Motiian, R. Ageev, K. Wu, A. Milesi, M. Moosaei, K. Pawelec, P. Subramanian, M. Samadi, X. Yu, C. Dear, S. Stoddard, J. Diamond, J. Oliver, L. Chraghchian, P. Skelly, T. Balough, Y. Xu, J. P. Scowcroft, D. Korzekwa, D. Hanley, S. Bhaskar, T. Roman, K. Sapra, A. Tao, and B. Catanzaro NVIDIA nemotron parse 1.1. External Links: 2511.20478, Link Cited by: §3.1.2.Source
  6. Duan et al. (2025) H. Duan, X. Fang, J. Yang, X. Zhao, Y. Qiao, M. Li, A. Agarwal, Z. Chen, L. Chen, Y. Liu, Y. Ma, H. Sun, Y. Zhang, S. Lu, T. H. Wong, W. Wang, P. Zhou, X. Li, C. Fu, J. Cui, J. Chen, E. Song, S. Mao, S. Ding, T. Liang, Z. Zhang, X. Dong, Y. Zang, P. Zhang, J. Wang, D. Lin, and K. Chen VLMEvalKit: an open-source toolkit for evaluating large multi-modality models. External Links: 2407.11691, Link Cited by: §4.Source
  7. Fox et al. (2024) J. D. Fox, D. Raj, N. Delworth, Q. McNamara, C. Miller, and M. Jetté Updated corpora and benchmarks for long-form speech recognition. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 13246–13250. Cited by: item 1.
  8. Fu et al. (2025) C. Fu, Y. Dai, Y. Luo, L. Li, S. Ren, R. Zhang, Z. Wang, C. Zhou, Y. Shen, M. Zhang, P. Chen, Y. Li, S. Lin, S. Zhao, K. Li, T. Xu, X. Zheng, E. Chen, C. Shan, R. He, and X. Sun Video-mme: the first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. External Links: 2405.21075, Link Cited by: item 5.Source
  9. Fu et al. (2024) L. Fu, Z. Kuang, J. Song, M. Huang, B. Yang, Y. Li, L. Zhu, Q. Luo, X. Wang, H. Lu, Z. Li, G. Tang, B. Shan, C. Lin, Q. Liu, B. Wu, H. Feng, H. Liu, C. Huang, J. Tang, W. Chen, L. Jin, Y. Liu, and X. Bai OCRBench v2: an improved benchmark for evaluating large multimodal models on visual text localization and reasoning. External Links: 2501.00321, Link Cited by: item 2.Source
  10. Heinrich et al. (2025) G. Heinrich, M. Ranzinger, H. Yin, Y. Lu, J. Kautz, A. Tao, B. Catanzaro, and P. Molchanov RADIOv2.5: improved baselines for agglomerative vision foundation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22487–22497. Cited by: §1, §2.
  11. Hong et al. (2026) J. Hong, S. Yan, J. Cai, X. Jiang, Y. Hu, and W. Xie WorldSense: evaluating real-world omnimodal understanding for multimodal llms. External Links: 2502.04326, Link Cited by: §1, item 2.Source
  12. Huang et al. (2025) M. Huang, Y. Shi, D. Peng, S. Lai, Z. Xie, and L. Jin OCR-reasoning benchmark: unveiling the true capabilities of mllms in complex text-rich image reasoning. External Links: 2505.17163, Link Cited by: item 2.Source
  13. Jain et al. (2024) N. Jain, K. Han, A. Gu, W. Li, F. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica LiveCodeBench: holistic and contamination free evaluation of large language models for code. External Links: 2403.07974, Link Cited by: §4.4.Source
  14. Kazemzadeh et al. (2014) S. Kazemzadeh, V. Ordonez, M. Matten, and T. Berg Referitgame: referring to objects in photographs of natural scenes. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 787–798. Cited by: item 3.
  15. Kembhavi et al. (2016) A. Kembhavi, M. Salvato, E. Kolve, M. Seo, H. Hajishirzi, and A. Farhadi A diagram is worth a dozen images. External Links: 1603.07396, Link Cited by: item 2.Source
  16. Kimi Team et al. (2026) Kimi Team, T. Bai, Y. Bai, Y. Bao, S. H. Cai, Y. Cao, Y. Charles, H. S. Che, C. Chen, G. Chen, H. Chen, J. Chen, J. Chen, J. Chen, J. Chen, K. Chen, L. Chen, R. Chen, X. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Z. Chen, Z. Chen, D. Cheng, M. Chu, J. Cui, J. Deng, M. Diao, H. Ding, M. Dong, M. Dong, Y. Dong, Y. Dong, A. Du, C. Du, D. Du, L. Du, Y. Du, Y. Fan, S. Fang, Q. Feng, Y. Feng, G. Fu, K. Fu, H. Gao, T. Gao, Y. Ge, S. Geng, C. Gong, X. Gong, Z. Gongque, Q. Gu, X. Gu, Y. Gu, L. Guan, Y. Guo, X. Hao, W. He, W. He, Y. He, C. Hong, H. Hu, J. Hu, Y. Hu, Z. Hu, K. Huang, R. Huang, W. Huang, Z. Huang, T. Jiang, Z. Jiang, X. Jin, Y. Jing, G. Lai, A. Li, C. Li, C. Li, F. Li, G. Li, G. Li, H. Li, H. Li, J. Li, J. Li, J. Li, L. Li, M. Li, W. Li, W. Li, X. Li, X. Li, Y. Li, Y. Li, Y. Li, Y. Li, Z. Li, Z. Li, W. Liao, J. Lin, X. Lin, Z. Lin, Z. Lin, C. Liu, C. Liu, H. Liu, L. Liu, S. Liu, S. Liu, S. Liu, T. Liu, T. Liu, W. Liu, X. Liu, Y. Liu, Y. Liu, Y. Liu, Y. Liu, Y. Liu, Z. Liu, Z. Liu, E. Lu, H. Lu, Z. Lu, J. Luo, T. Luo, Y. Luo, L. Ma, Y. Ma, S. Mao, Y. Mei, X. Men, F. Meng, Z. Meng, Y. Miao, M. Ni, K. Ouyang, S. Pan, B. Pang, Y. Qian, R. Qin, Z. Qin, J. Qiu, B. Qu, Z. Shang, Y. Shao, T. Shen, Z. Shen, J. Shi, L. Shi, S. Shi, F. Song, P. Song, T. Song, X. Song, H. Su, J. Su, Z. Su, L. Sui, J. Sun, J. Sun, T. Sun, F. Sung, Y. Tai, C. Tang, H. Tang, X. Tang, Z. Tang, J. Tao, S. Teng, C. Tian, P. Tian, A. Wang, B. Wang, C. Wang, C. Wang, C. Wang, D. Wang, D. Wang, D. Wang, F. Wang, H. Wang, H. Wang, H. Wang, H. Wang, H. Wang, J. Wang, J. Wang, J. Wang, K. Wang, L. Wang, Q. Wang, S. Wang, S. Wang, S. Wang, W. Wang, X. Wang, X. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Z. Wang, Z. Wang, Z. Wang, Z. Wang, Z. Wang, Z. Wang, C. Wei, M. Wei, C. Wen, Z. Wen, C. Wu, H. Wu, J. Wu, R. Wu, W. Wu, Y. Wu, Y. Wu, Y. Wu, Z. Wu, C. Xiao, J. Xie, X. Xie, Y. Xie, Y. Xin, B. Xing, B. Xu, J. Xu, J. Xu, J. Xu, L. H. Xu, L. Xu, S. Xu, W. Xu, X. Xu, X. Xu, Y. Xu, Y. Xu, Y. Xu, Z. Xu, Z. Xu, J. Yan, Y. Yan, G. Yang, H. Yang, J. Yang, K. Yang, N. Yang, R. Yang, X. Yang, X. Yang, Y. Yang, Y. Yang, Y. Yang, Z. Yang, Z. Yang, Z. Yang, H. Yao, D. Ye, W. Ye, Z. Ye, B. Yin, C. Yu, L. Yu, T. Yu, T. Yu, E. Yuan, M. Yuan, X. Yuan, Y. Yue, W. Zeng, D. Zha, H. Zhan, D. Zhang, H. Zhang, J. Zhang, P. Zhang, Q. Zhang, R. Zhang, X. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Z. Zhang, C. Zhao, F. Zhao, J. Zhao, S. Zhao, X. Zhao, Y. Zhao, Z. Zhao, H. Zheng, R. Zheng, S. Zheng, T. Zheng, J. Zhong, L. Zhong, W. Zhong, M. Zhou, R. Zhou, X. Zhou, Z. Zhou, J. Zhu, L. Zhu, X. Zhu, Y. Zhu, Z. Zhu, J. Zhuang, W. Zhuang, Y. Zou, and X. Zu Kimi k2.5: visual agentic intelligence. External Links: 2602.02276, Link Cited by: §3.1.2.Source
  17. Koluguri et al. (2025) N. R. Koluguri, M. Sekoyan, G. Zelenfroynd, S. Meister, S. Ding, S. Kostandian, H. Huang, N. Karpov, J. Balam, V. Lavrukhin, Y. Peng, S. Papi, M. Gaido, A. Brutti, and B. Ginsburg Granary: speech recognition and translation dataset in 25 european languages. External Links: 2505.13404, Link Cited by: §3.1.3.Source
  18. Kwon et al. (2023) W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, Cited by: §4.
  19. Li et al. (2025) K. Li, Z. Meng, H. Lin, Z. Luo, Y. Tian, J. Ma, Z. Huang, and T. Chua ScreenSpot-pro: gui grounding for professional high-resolution computer use. External Links: 2504.07981, Link Cited by: item 4.Source
  20. Liu et al. (2024) Y. Liu, Z. Li, M. Huang, B. Yang, W. Yu, C. Li, X. Yin, C. Liu, L. Jin, and X. Bai OCRBench: on the hidden mystery of ocr in large multimodal models. Science China Information Sciences 67 (12). External Links: ISSN 1869-1919, Link, Document Cited by: §1, item 2.SourceSource
  21. Lu et al. (2024) P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K. Chang, M. Galley, and J. Gao MathVista: evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR), Cited by: item 1.
  22. Ma et al. (2024) Y. Ma, Y. Zang, L. Chen, M. Chen, Y. Jiao, X. Li, X. Lu, Z. Liu, Y. Ma, X. Dong, P. Zhang, L. Pan, Y. Jiang, J. Wang, Y. Cao, and A. Sun MMLongBench-doc: benchmarking long-context document understanding with visualizations. External Links: 2407.01523, Link Cited by: §1, item 2.Source
  23. Masry et al. (2022) A. Masry, D. X. Long, J. Q. Tan, S. Joty, and E. Hoque ChartQA: a benchmark for question answering about charts with visual and logical reasoning. External Links: 2203.10244, Link Cited by: item 2.Source
  24. Mathew et al. (2021a) M. Mathew, V. Bagal, R. P. Tito, D. Karatzas, E. Valveny, and C. V. Jawahar InfographicVQA. External Links: 2104.12756, Link Cited by: item 2.Source
  25. Mathew et al. (2021b) M. Mathew, D. Karatzas, and C. V. Jawahar DocVQA: a dataset for vqa on document images. External Links: 2007.00398, Link Cited by: item 2.Source
  26. NVIDIA et al. (2025a) NVIDIA, :, A. Blakeman, A. Grattafiori, A. Basant, A. Gupta, A. Khattar, A. Renduchintala, A. Vavre, A. Shukla, A. Bercovich, A. Ficek, A. Shaposhnikov, A. Kondratenko, A. Bukharin, A. Milesi, A. Taghibakhshi, A. Liu, A. Barton, A. S. Mahabaleshwarkar, A. Klein, A. Zuker, A. Geifman, A. Shen, A. Bhiwandiwalla, A. Tao, A. Agrusa, A. Verma, A. Guan, A. Mandarwal, A. Mehta, A. Aithal, A. Poojary, A. Ahamed, A. Mishra, A. K. Thekkumpate, A. Dattagupta, B. Zhu, B. Sadeghi, B. Simkin, B. Lanir, B. Schifferer, B. Nushi, B. Kartal, B. D. Rouhani, B. Ginsburg, B. Norick, B. Soubasis, B. Kisacanin, B. Yu, B. Catanzaro, C. del Mundo, C. Hwang, C. Wang, C. Hsieh, C. Zhang, C. Yu, C. Mungekar, C. Patel, C. Alexiuk, C. Parisien, C. Neale, C. Meurillon, D. Mosk-Aoyama, D. Su, D. Corneil, D. Afrimi, D. Lo, D. Rohrer, D. Serebrenik, D. Gitman, D. Levy, D. Stosic, D. Mosallanezhad, D. Narayanan, D. Nathawani, D. Rekesh, D. Yared, D. Kakwani, D. Ahn, D. Riach, D. Stosic, E. Minasyan, E. Lin, E. Long, E. P. Long, E. Segal, E. Lantz, E. Evans, E. Ning, E. Chung, E. Harper, E. Tramel, E. Galinkin, E. Pounds, E. Briones, E. Bakhturina, E. Tsykunov, F. Ladhak, F. Wang, F. Jia, F. Soares, F. Chen, F. Galko, F. Sun, F. Siino, G. H. Agam, G. Ajjanagadde, G. Bhatt, G. Prasad, G. Armstrong, G. Shen, G. Batmaz, G. Nalbandyan, H. Qian, H. Sharma, H. Ross, H. Ngo, H. Hum, H. Sahota, H. Wang, H. Soni, H. Upadhyay, H. Mao, H. C. Nguyen, H. Q. Nguyen, I. Cunningham, I. Galil, I. Shahaf, I. Gitman, I. Loshchilov, I. Schen, I. Levy, I. Moshkov, I. Golan, I. Putterman, J. Kautz, J. P. Scowcroft, J. Casper, J. Mitra, J. Glick, J. Chen, J. Oliver, J. Zhang, J. Zeng, J. Lou, J. Zhang, J. Choi, J. Huang, J. Conway, J. Guman, J. Kamalu, J. Greco, J. Cohen, J. Jennings, J. Daw, J. V. Vialard, J. Yi, J. Parmar, K. Xu, K. Zhu, K. Briski, K. Cheung, K. Luna, K. Wyss, K. Santhanam, K. Shih, K. Kong, K. Bhardwaj, K. Shankar, K. C. Puvvada, K. Pawelec, K. Anik, L. McAfee, L. Sleiman, L. Derczynski, L. Ding, L. Wei, L. Liebenwein, L. Vega, M. Grover, M. V. Segbroeck, M. R. de Melo, M. Nazemi, M. N. Sreedhar, M. Kilaru, M. Ashkenazi, M. Romeijn, M. Chochowski, M. Cai, M. Kliegl, M. Moosaei, M. Kulka, M. Novikov, M. Samadi, M. Corpuz, M. Wang, M. Price, M. Andersch, M. Boone, M. Evans, M. Martinez, M. Khona, M. Chrzanowski, M. Lee, M. Dabbah, M. Shoeybi, M. Patwary, N. Mulepati, N. Nabwani, N. Hereth, N. Assaf, N. Habibi, N. Zmora, N. Haber, N. Sessions, N. Bhatia, N. Jukar, N. Pope, N. Ludwig, N. Tajbakhsh, N. Ailon, N. Juluru, N. Sharma, O. Hrinchuk, O. Kuchaiev, O. Delalleau, O. Olabiyi, O. U. Argov, O. Puny, O. Tropp, O. Xie, P. Chadha, P. Shamis, P. Gibbons, P. Molchanov, P. Morkisz, P. Dykas, P. Jin, P. Xu, P. Januszewski, P. P. Thombre, P. Varshney, P. Gundecha, P. Tredak, Q. Miao, Q. Wan, R. K. Mahabadi, R. Garg, R. El-Yaniv, R. Zilberstein, R. Shafipour, R. Harang, R. Izzo, R. Shahbazyan, R. Garg, R. Borkar, R. Gala, R. Islam, R. Hesse, R. Waleffe, R. Watve, R. Koren, R. Zhang, R. Hewett, R. J. Hewett, R. Prenger, R. Timbrook, S. Mahdavi, S. Modi, S. Kriman, S. Lim, S. Kariyappa, S. Satheesh, S. Kaji, S. Pasumarthi, S. Muralidharan, S. Narentharen, S. Narenthiran, S. Bak, S. Kashirsky, S. Poulos, S. Mor, S. Ramasamy, S. Acharya, S. Ghosh, S. T. Sreenivas, S. Thomas, S. Fan, S. Gopal, S. Prabhumoye, S. Pachori, S. Toshniwal, S. Ding, S. Singh, S. Sun, S. Ithape, S. Majumdar, S. Singhal, S. Sergienko, S. Alborghetti, S. Ge, S. D. Devare, S. K. Barua, S. Panguluri, S. Gupta, S. Priyadarshi, S. N. Akter, T. Bui, T. Ene, T. Kong, T. Do, T. Blankevoort, T. Moon, T. Balough, T. Asida, T. B. Natan, T. Ronen, T. Konuk, T. Vashishth, U. Karpas, U. De, V. Noorozi, V. Noroozi, V. Srinivasan, V. Elango, V. Cui, V. Korthikanti, V. Rao, V. Kurin, V. Lavrukhin, V. Anisimov, W. Jiang, W. U. Ahmad, W. Du, W. Ping, W. Zhou, W. Jennings, W. Zhang, W. Prazuch, X. Ren, Y. Karnati, Y. Choi, Y. Meyer, Y. Wu, Y. Zhang, Y. Qin, Y. Lin, Y. Geifman, Y. Fu, Y. Subara, Y. Suhara, Y. Gao, Z. Moshe, Z. Dong, Z. Zhu, Z. Liu, Z. Chen, and Z. Yan NVIDIA nemotron 3: efficient and open intelligence. External Links: 2512.20856, Link Cited by: §1, §2.Source
  27. NVIDIA et al. (2025b) NVIDIA, :, A. Blakeman, A. Grattafiori, A. Basant, A. Gupta, A. Khattar, A. Renduchintala, A. Vavre, A. Shukla, A. Bercovich, A. Ficek, A. Shaposhnikov, A. Kondratenko, A. Bukharin, A. Milesi, A. Taghibakhshi, A. Liu, A. Barton, A. S. Mahabaleshwarkar, A. Klein, A. Zuker, A. Geifman, A. Shen, A. Bhiwandiwalla, A. Tao, A. Guan, A. Mandarwal, A. Mehta, A. Aithal, A. Poojary, A. Ahamed, A. K. Thekkumpate, A. Dattagupta, B. Zhu, B. Sadeghi, B. Simkin, B. Lanir, B. Schifferer, B. Nushi, B. Kartal, B. D. Rouhani, B. Ginsburg, B. Norick, B. Soubasis, B. Kisacanin, B. Yu, B. Catanzaro, C. del Mundo, C. Hwang, C. Wang, C. Hsieh, C. Zhang, C. Yu, C. Mungekar, C. Patel, C. Alexiuk, C. Parisien, C. Neale, D. Mosk-Aoyama, D. Su, D. Corneil, D. Afrimi, D. Rohrer, D. Serebrenik, D. Gitman, D. Levy, D. Stosic, D. Mosallanezhad, D. Narayanan, D. Nathawani, D. Rekesh, D. Yared, D. Kakwani, D. Ahn, D. Riach, D. Stosic, E. Minasyan, E. Lin, E. Long, E. P. Long, E. Lantz, E. Evans, E. Ning, E. Chung, E. Harper, E. Tramel, E. Galinkin, E. Pounds, E. Briones, E. Bakhturina, F. Ladhak, F. Wang, F. Jia, F. Soares, F. Chen, F. Galko, F. Siino, G. H. Agam, G. Ajjanagadde, G. Bhatt, G. Prasad, G. Armstrong, G. Shen, G. Batmaz, G. Nalbandyan, H. Qian, H. Sharma, H. Ross, H. Ngo, H. Sahota, H. Wang, H. Soni, H. Upadhyay, H. Mao, H. C. Nguyen, H. Q. Nguyen, I. Cunningham, I. Shahaf, I. Gitman, I. Loshchilov, I. Moshkov, I. Putterman, J. Kautz, J. P. Scowcroft, J. Casper, J. Mitra, J. Glick, J. Chen, J. Oliver, J. Zhang, J. Zeng, J. Lou, J. Zhang, J. Huang, J. Conway, J. Guman, J. Kamalu, J. Greco, J. Cohen, J. Jennings, J. Daw, J. V. Vialard, J. Yi, J. Parmar, K. Xu, K. Zhu, K. Briski, K. Cheung, K. Luna, K. Santhanam, K. Shih, K. Kong, K. Bhardwaj, K. C. Puvvada, K. Pawelec, K. Anik, L. McAfee, L. Sleiman, L. Derczynski, L. Ding, L. Liebenwein, L. Vega, M. Grover, M. V. Segbroeck, M. R. de Melo, M. N. Sreedhar, M. Kilaru, M. Ashkenazi, M. Romeijn, M. Cai, M. Kliegl, M. Moosaei, M. Novikov, M. Samadi, M. Corpuz, M. Wang, M. Price, M. Boone, M. Evans, M. Martinez, M. Chrzanowski, M. Shoeybi, M. Patwary, N. Mulepati, N. Hereth, N. Assaf, N. Habibi, N. Zmora, N. Haber, N. Sessions, N. Bhatia, N. Jukar, N. Pope, N. Ludwig, N. Tajbakhsh, N. Juluru, O. Hrinchuk, O. Kuchaiev, O. Delalleau, O. Olabiyi, O. U. Argov, O. Xie, P. Chadha, P. Shamis, P. Molchanov, P. Morkisz, P. Dykas, P. Jin, P. Xu, P. Januszewski, P. P. Thombre, P. Varshney, P. Gundecha, Q. Miao, R. K. Mahabadi, R. El-Yaniv, R. Zilberstein, R. Shafipour, R. Harang, R. Izzo, R. Shahbazyan, R. Garg, R. Borkar, R. Gala, R. Islam, R. Waleffe, R. Watve, R. Koren, R. Zhang, R. J. Hewett, R. Prenger, R. Timbrook, S. Mahdavi, S. Modi, S. Kriman, S. Kariyappa, S. Satheesh, S. Kaji, S. Pasumarthi, S. Narentharen, S. Narenthiran, S. Bak, S. Kashirsky, S. Poulos, S. Mor, S. Ramasamy, S. Acharya, S. Ghosh, S. T. Sreenivas, S. Thomas, S. Fan, S. Gopal, S. Prabhumoye, S. Pachori, S. Toshniwal, S. Ding, S. Singh, S. Sun, S. Ithape, S. Majumdar, S. Singhal, S. Alborghetti, S. Ge, S. D. Devare, S. K. Barua, S. Panguluri, S. Gupta, S. Priyadarshi, S. N. Akter, T. Bui, T. Ene, T. Kong, T. Do, T. Blankevoort, T. Balough, T. Asida, T. B. Natan, T. Konuk, T. Vashishth, U. Karpas, U. De, V. Noorozi, V. Noroozi, V. Srinivasan, V. Elango, V. Korthikanti, V. Kurin, V. Lavrukhin, W. Jiang, W. U. Ahmad, W. Du, W. Ping, W. Zhou, W. Jennings, W. Zhang, W. Prazuch, X. Ren, Y. Karnati, Y. Choi, Y. Meyer, Y. Wu, Y. Zhang, Y. Lin, Y. Geifman, Y. Fu, Y. Subara, Y. Suhara, Y. Gao, Z. Moshe, Z. Dong, Z. Liu, Z. Chen, and Z. Yan Nemotron 3 nano: open, efficient mixture-of-experts hybrid mamba-transformer model for agentic reasoning. External Links: 2512.20848, Link Cited by: §3.2.2.Source
  28. NVIDIA et al. (2026) NVIDIA, :, A. Chandiramani, A. Blakeman, A. Olaoye, A. Gupta, A. Somasamudramath, A. Khattar, A. Adesoba, A. Renduchintala, A. Asif, A. Agrawal, A. Vavre, A. Kiswani, A. Padmakumar, A. Hotchandani, A. Shukla, A. Bercovich, A. Ficek, A. Shaposhnikov, A. Gronskiy, A. Kondratenko, A. Neefus, A. Steiner, A. Yang, A. Bukharin, A. Young, A. Hatamizadeh, A. Taghibakhshi, A. Galiautdinova, A. Liu, A. Kumar, A. S. Mahabaleshwarkar, A. Klein, A. Zuker, A. Geifman, A. Bhiwandiwalla, A. Subramaniam, A. Tao, A. Shrivastava, A. Agrusa, A. Srivastava, A. Verma, A. Guan, A. Shors, A. Chockalingam, A. Mandarwal, A. Ramani, A. Mehta, A. Jain, A. Venkatesan, A. Anoosheh, A. Aithal, A. Poojary, A. Ahamed, A. Mishra, A. S. Demiroz, A. K. Thekkumpate, A. Sohrabizadeh, A. Kaur, A. Dattagupta, B. S. Anandan, B. Sadeghi, B. Simkin, B. Lanir, B. Schifferer, B. Chislett, B. Nushi, B. Kartal, B. Thiede, B. D. Rouhani, B. Chen, B. Ginsburg, B. Norick, B. Kisacanin, B. Yu, B. Catanzaro, B. Mani, C. del Mundo, C. Lee, C. Kim, C. Hwang, C. Ni, C. Wang, C. Truong, C. Hsieh, C. Yu, C. Luo, C. Wang, C. Mungekar, C. Patel, C. Alexiuk, C. Holguin, C. Wing, C. Munley, C. Parisien, C. Desai, C. Sheng, C. Neale, C. Meurillon, D. Kumar, D. Gil, D. Su, D. Corneil, D. Afrimi, D. B. E. Triana, D. Egert, D. Fatade, D. Lo, D. Rohrer, D. Serebrenik, D. Sorokin, D. Gitman, D. Levy, D. Stosic, D. Edelsohn, D. Messina, D. Mosallanezhad, D. Tamok, D. Donia, D. Narayanan, D. O’Kelly, D. Peri, D. Nathawani, D. Wu, D. Rekesh, D. Yared, D. Kakwani, D. K. B. Tuttle, D. Ahn, D. Jiang, D. Poorkay, D. O’Flaherty, D. Riach, D. Stosic, D. V. Stee, E. Minasyan, E. Lin, E. P. Long, E. Segal, E. Lantz, E. Lewis, E. Evans, E. Ning, E. Chung, E. Harper, E. Pham-Hung, E. W. Tramel, E. Galinkin, E. Pounds, E. Etrog, E. Briones, E. Wu, E. Bakhturina, E. Tsykunov, E. Dobrowolska, F. S. Movahed, F. Memarian, F. Wang, F. Jia, F. Soares, F. V. Frujeri, F. Chen, F. Lin, F. Galko, F. Zhang, F. Siino, F. Hou, G. Bhatt, G. Prasad, G. Venkataramani, G. Gupta, G. Armstrong, G. Shen, G. Borghesi, G. Neskovic, G. Batmaz, G. Lam, G. Wu, G. Pauloski, G. Davis, G. Nalbandyan, G. Zhang, G. Farber, G. Huang, H. Qian, H. K. S. Kumar, H. Kim, H. Sharma, H. Iso, H. Ross, H. Hum, H. Sahota, H. Wang, H. Soni, H. Upadhyay, H. Nguyen, I. Cunningham, I. Galil, I. Shahaf, I. Padovani, I. Gitman, I. Shovkun, I. Dhillon, I. Loshchilov, I. Kelly, I. Schen, I. Levy, I. Moshkov, I. Golan, I. Putterman, J. Tu, J. Baczek, J. Kautz, J. P. Scowcroft, J. Rosenberg, J. Casper, J. Pflum, J. Grant, J. Sewall, J. Mitra, J. Glick, J. Chen, J. Oliver, J. Xu, J. Zhu, J. Song, J. Zhang, J. Zeng, J. Lou, J. Milton, J. Chow, J. Zhang, J. Choi, J. Huang, J. Huang, J. Caruso, J. Conway, J. Guman, J. Jatko, J. Kamalu, J. Greco, J. Cohen, J. Raiman, J. Jennings, J. Daw, J. Yu, J. Tapia, J. Yi, J. Parmar, J. Achar, K. Briski, K. Mattoo, K. Cheung, K. Luna, K. Wyss, K. Shih, K. Kong, K. Nguyen, K. Bhardwaj, K. Buryak, K. S. Sivamani, K. Krommydas, K. Murphy, K. C. Puvvada, K. Pawelec, K. Anik, L. Tewari, L. Sleiman, L. Du, L. Derczynski, L. Ding, L. Ilan, L. Wu, L. Wei, L. Vega, L. Su, M. V. Segbroeck, M. R. de Melo, M. Zhang, M. Fathi, M. N. Sreedhar, M. Sreedhar, M. T. Chandran, M. R. Gomez, M. Ashkenazi, M. Cuevas, M. Romeijn, M. Zhang, M. Cai, M. Gabel, M. Kliegl, M. Patelka, M. Moosaei, M. Varacalli, M. Novikov, M. Ferrato, M. Samadi, M. Corpuz, M. Xin, M. Wang, M. Wang, M. Price, M. Schaffer, M. Andersch, M. Boone, M. Evans, M. Z. Wang, M. Martinez, M. Khona, M. Chrzanowski, M. Hollinger, M. Ma, M. Lee, M. Dabbah, M. Shoeybi, M. Patwary, N. Mulepati, N. Khalil, N. Nabwani, N. Agarwal, N. Balasubramaniam, N. Hennouni, N. Kodukula, N. Hereth, N. Pinckney, N. Assaf, N. Habibi, N. Qin, N. Zmora, N. Haber, N. Reamaroon, N. Quak, N. Bhatia, N. Jukar, N. Pope, N. Ludwig, N. Tajbakhsh, N. Ailon, N. Juluru, N. De, N. Pitt, O. Rybakov, O. Hrinchuk, O. Kuchaiev, O. Delalleau, O. Olabiyi, O. U. Argov, O. Almog, O. Puny, O. Tropp, O. Padovani, O. Xie, P. Chadha, P. Shamis, P. Gibbons, P. Molchanov, P. Belcak, P. Jin, P. Xu, P. Januszewski, P. Jannaty, P. Shevate, P. Thalasta, P. P. Thombre, P. Varshney, P. Gambhir, P. Gundecha, P. Tredak, Q. Miao, Q. Wan, Q. T. Minh, R. K. Mahabadi, R. Oberman, R. Garg, R. Kandu, R. Zhong, R. El-Yaniv, R. Zilberstein, R. Shafipour, R. Yao, R. Pi, R. Mazzarese, R. Wang, R. Izzo, R. Singla, R. Shahbazyan, R. Garg, R. Borkar, R. Gala, R. Islam, R. Clark, R. Hesse, R. Waleffe, R. V. Kalidindi, R. Watve, R. Koren, R. Fan, R. Kharwar, R. Cai, R. Zhang, R. J. Hewett, R. Prenger, R. Timbrook, R. Egashira, S. Mahdavi, S. S. A. Joshi, S. Modi, S. Kriman, S. Pombra, S. Kariyappa, S. Satheesh, S. Pombo, S. Kaji, S. Pasumarthi, S. Mishra, S. Muralidharan, S. Hara, S. Narenthiran, S. Rogawski, S. Na, S. Bak, S. Sameni, S. Poulos, S. Mor, S. Acharya, S. G. A. Lord, S. T. Sreenivas, S. Kotek, S. Gharghabi, S. Thomas, S. Lin, S. Likhite, S. Fan, S. Chen, S. Gopal, S. Prabhumoye, S. Pachori, S. Toshniwal, S. Zhang, S. Ding, S. Renjith, S. Prayaga, S. Jain, S. Sun, S. Rella, S. Das, S. Ithape, S. H. S, S. Majumdar, S. Singhal, S. H. Singudasu, S. Niverty, S. Sergienko, S. Gloginic, S. Alborghetti, S. Ge, S. McCullough, S. D. Devare, S. V. Velury, S. Rao, S. K. Barua, S. Gai, S. Panguluri, S. Koundinyan, S. Patnam, S. Priyadarshi, S. Bhendigeri, S. N. Akter, S. Arunagiri, T. Yuan, T. Abramovich, T. Bui, T. Yu, T. Kong, T. Do, T. Gburek, T. Marques, T. Moore, T. Blankevoort, T. Moon, T. Ma, T. Mitra, T. Grzegorzek, T. Asida, T. B. Natan, T. Keren, T. Ronen, T. Rebedea, T. Starkey, T. Konuk, T. Vashishth, T. Condensa, U. Karpas, U. De, V. Noorozi, V. Noroozi, V. A. Shah, V. Vaidyanathan, V. Srinivasan, V. Elango, V. Cui, V. Korthikanti, V. Mehta, V. Adams, V. Wu, V. Kurin, V. Lavrukhin, V. Anisimov, W. Seo, W. Jiang, W. U. Ahmad, W. Du, W. Ping, W. Chen, W. Quan, W. Dai, W. Gao, W. Jennings, W. Zhang, X. Ren, X. Xin, X. Li, Y. Yu, Y. Chen, Y. Galron, Y. Karnati, Y. Choi, Y. Meyer, Y. Wu, Y. Zhang, Y. Lin, Y. Geifman, Y. Fu, Y. Suhara, Y. Kwon, Y. Zhang, Y. Huang, Z. Moshe, Z. Wang, Z. Cheng, Z. Zhu, Z. Yang, Z. Liu, Z. Chen, Z. Yan, and Z. Ahmed Nemotron 3 super: open, efficient mixture-of-experts hybrid mamba-transformer model for agentic reasoning. External Links: 2604.12374, Link Cited by: §3.1.6, §3.2.2.Source
  29. NVIDIA et al. (2025c) NVIDIA, :, A. S. Deshmukh, K. Chumachenko, T. Rintamaki, M. Le, T. Poon, D. M. Taheri, I. Karmanov, G. Liu, J. Seppanen, G. Chen, K. Sapra, Z. Yu, A. Renduchintala, C. Wang, P. Jin, A. Goel, M. Ranzinger, L. Voegtle, P. Fischer, T. Roman, W. Ping, B. Wang, Z. Yang, N. Lee, S. Zhang, F. Liu, Z. Li, D. Zhang, G. Heinrich, H. Yin, S. Han, P. Molchanov, P. Mannan, Y. Xu, J. P. Scowcroft, T. Balough, S. Radhakrishnan, P. Zhang, S. Cha, R. Kumar, Z. P. Bhat, J. Zhang, D. Hanley, P. Biswas, J. Oliver, K. Vasques, R. Waleffe, D. Riach, O. Olabiyi, A. S. Mahabaleshwarkar, B. Kartal, P. Gundecha, K. Nguyen, A. Milesi, E. Khvedchenia, R. Zilberstein, O. Masad, N. Bagrov, N. Assaf, T. Asida, D. Afrimi, A. Zuker, N. Haber, Z. Cheng, J. Xin, D. Wu, N. Spirin, M. Moosaei, R. Ageev, V. A. Shah, Y. Wu, D. Korzekwa, U. K. Sreekumar, W. Jiang, P. Subramanian, A. Rico, S. Bhaskar, S. Motiian, K. Wu, A. Surla, C. Chen, H. Wolff, M. Feinberg, M. Corpuz, M. Wawrzos, E. Long, A. Jhunjhunwala, P. Hendricks, F. Memarian, B. Hall, X. Wang, D. Mosallanezhad, S. Singhal, L. Vega, K. Cheung, K. Pawelec, M. Evans, K. Luna, J. Lou, E. Galinkin, A. Hazare, K. Purandare, A. Guan, A. Warno, C. Cui, Y. Suhara, S. Likhite, S. Mard, M. Price, L. Sleiman, S. Kaji, U. Karpas, K. Briski, J. Conway, M. Lightstone, J. Kautz, M. Shoeybi, M. Patwary, J. Cohen, O. Kuchaiev, A. Tao, and B. Catanzaro NVIDIA nemotron nano v2 vl. External Links: 2511.03929, Link Cited by: §1, §2, §3.1.2.Source
  30. NVIDIA (2025) NVIDIA NeMo rl: a scalable and efficient post-training library. Note: https://github.com/NVIDIA-NeMo/RLGitHub repository Cited by: §3.2.5.Source
  31. OpenAI (2025) OpenAI Introducing gpt-oss. External Links: Link Cited by: §3.1.2.Source
  32. Qwen Team (2026) Qwen Team Qwen3.5: towards native multimodal agents. External Links: Link Cited by: §3.1.2, §3.1.2.Source
  33. Rafailov et al. (2023) R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn Direct preference optimization: your language model is secretly a reward model. External Links: 2305.18290, Link Cited by: §3.2.1.Source
  34. Ranzinger et al. (2026) M. Ranzinger, G. Heinrich, C. McCarthy, J. Kautz, A. Tao, B. Catanzaro, and P. Molchanov C-radiov4 (tech report). External Links: 2601.17237, Link Cited by: §1, §2.Source
  35. Rein et al. (2023) D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman GPQA: a graduate-level google-proof q&a benchmark. External Links: 2311.12022, Link Cited by: §4.4.Source
  36. Rekesh et al. (2023) D. Rekesh, N. R. Koluguri, S. Kriman, S. Majumdar, V. Noroozi, H. Huang, O. Hrinchuk, K. Puvvada, A. Kumar, J. Balam, and B. Ginsburg Fast conformer with linearly scalable attention for efficient speech recognition. External Links: 2305.05084, Link Cited by: §1, §2.Source
  37. Sakshi et al. (2024) S. Sakshi, U. Tyagi, S. Kumar, A. Seth, R. Selvakumar, O. Nieto, R. Duraiswami, S. Ghosh, and D. Manocha MMAU: a massive multi-task audio understanding and reasoning benchmark. External Links: 2410.19168, Link Cited by: item 2.Source
  38. Sekoyan et al. (2025) M. Sekoyan, N. R. Koluguri, N. Tadevosyan, P. Zelasko, T. Bartley, N. Karpov, J. Balam, and B. Ginsburg Canary-1b-v2 & parakeet-tdt-0.6b-v3: efficient and high-performance models for multilingual asr and ast. External Links: 2509.14128, Link Cited by: §1, §2.Source
  39. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. External Links: 2402.03300, Link Cited by: §3.2.4, §3.2.5.Source
  40. Shoeybi et al. (2019) M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro Megatron-lm: training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053. Cited by: §3.1.8.
  41. Singh et al. (2019) A. Singh, V. Natarjan, M. Shah, Y. Jiang, X. Chen, D. Parikh, and M. Rohrbach Towards vqa models that can read. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8317–8326. Cited by: item 2.
  42. Srivastav et al. (2026) V. Srivastav, S. Zheng, E. Bezzam, E. L. Bihan, N. R. Koluguri, P. Żelasko, S. Majumdar, A. Moumen, and S. Gandhi Open asr leaderboard: towards reproducible and transparent multilingual and long-form speech recognition evaluation. External Links: 2510.06961, Link Cited by: item 1.Source
  43. The NeMo Data Designer Team (2025) N. The NeMo Data Designer Team NeMo data designer: a framework for generating synthetic data from scratch or based on your own seed data. Note: https://github.com/NVIDIA-NeMo/DataDesignerGitHub Repository Cited by: §3.1.7.Source
  44. Tian et al. (2024) M. Tian, L. Gao, S. D. Zhang, X. Chen, C. Fan, X. Guo, R. Haas, P. Ji, K. Krongchon, Y. Li, S. Liu, D. Luo, Y. Ma, H. Tong, K. Trinh, C. Tian, Z. Wang, B. Wu, Y. Xiong, S. Yin, M. Zhu, K. Lieret, Y. Lu, G. Liu, Y. Du, T. Tao, O. Press, J. Callan, E. Huerta, and H. Peng SciCode: a research coding benchmark curated by scientists. External Links: 2407.13168, Link Cited by: §4.4.Source
  45. Tong et al. (2024) S. Tong, E. Brown, P. Wu, S. Woo, M. Middepogu, S. C. Akula, J. Yang, S. Yang, A. Iyer, X. Pan, Z. Wang, R. Fergus, Y. LeCun, and S. Xie Cambrian-1: a fully open, vision-centric exploration of multimodal llms. External Links: 2406.16860, Link Cited by: item 3.Source
  46. Wang et al. (2025) H. Wang, X. Li, Z. Huang, A. Wang, J. Wang, T. Zhang, J. Zheng, S. Bai, Z. Kang, J. Feng, Z. Wang, and Z. Zhang Traceable evidence enhanced visual grounded reasoning: evaluation and methodology. External Links: 2507.07999, Link Cited by: item 3.Source
  47. Wang et al. (2024a) W. Wang, Z. Chen, W. Wang, Y. Cao, Y. Liu, Z. Gao, J. Zhu, X. Zhu, L. Lu, Y. Qiao, and J. Dai Enhancing the reasoning ability of multimodal large language models via mixed preference optimization. External Links: 2411.10442, Link Cited by: §3.2.1.Source
  48. Wang et al. (2024b) Z. Wang, M. Xia, L. He, H. Chen, Y. Liu, R. Zhu, K. Liang, X. Wu, H. Liu, S. Malladi, A. Chevalier, S. Arora, and D. Chen CharXiv: charting gaps in realistic chart understanding in multimodal llms. External Links: 2406.18521, Link Cited by: item 2.Source
  49. Wei et al. (2025) H. Wei, Y. Sun, and Y. Li DeepSeek-ocr: contexts optical compression. External Links: 2510.18234, Link Cited by: §3.1.2.Source
  50. Wei et al. (2022) J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, pp. 24824–24837. Cited by: §3.2.4.
  51. Wu et al. (2024) Z. Wu, Z. Wu, F. Xu, Y. Wang, Q. Sun, C. Jia, K. Cheng, Z. Ding, L. Chen, P. P. Liang, and Y. Qiao OS-atlas: a foundation action model for generalist gui agents. External Links: 2410.23218, Link Cited by: item 4.Source
  52. Xie et al. (2024) T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, Y. Liu, Y. Xu, S. Zhou, S. Savarese, C. Xiong, V. Zhong, and T. Yu OSWorld: benchmarking multimodal agents for open-ended tasks in real computer environments. External Links: 2404.07972, Link Cited by: item 4.Source
  53. Xu et al. (2023) H. Xu, F. Jia, S. Majumdar, H. Huang, S. Watanabe, and B. Ginsburg Efficient sequence transduction by jointly predicting tokens and durations. In Proceedings of the 40th International Conference on Machine Learning, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett (Eds.), Proceedings of Machine Learning Research, Vol. 202, pp. 38462–38484. External Links: Link Cited by: §1, §2.Source
  54. Xu et al. (2025) J. Xu, Z. Guo, H. Hu, Y. Chu, X. Wang, J. He, Y. Wang, X. Shi, T. He, X. Zhu, Y. Lv, Y. Wang, D. Guo, H. Wang, L. Ma, P. Zhang, X. Zhang, H. Hao, Z. Guo, B. Yang, B. Zhang, Z. Ma, X. Wei, S. Bai, K. Chen, X. Liu, P. Wang, M. Yang, D. Liu, X. Ren, B. Zheng, R. Men, F. Zhou, B. Yu, J. Yang, L. Yu, J. Zhou, and J. Lin Qwen3-omni technical report. External Links: 2509.17765, Link Cited by: §1.Source
  55. Yang et al. (2025) A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu Qwen3 technical report. External Links: 2505.09388, Link Cited by: §3.1.2, §3.1.2.Source
  56. Ye et al. (2026) H. Ye, C. H. Yang, A. Goel, W. Huang, L. Zhu, Y. Su, S. Lin, A. Cheng, Z. Wan, J. Tian, et al. OmniVinci: enhancing architecture and data for omni-modal understanding llm. ICLR. Cited by: §3.2.4.
  57. Yue et al. (2024) X. Yue, Y. Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y. Sun, C. Wei, B. Yu, R. Yuan, R. Sun, M. Yin, B. Zheng, Z. Yang, Y. Liu, W. Huang, H. Sun, Y. Su, and W. Chen MMMU: a massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of CVPR, Cited by: item 1.
  58. Zheng et al. (2025) C. Zheng, S. Liu, M. Li, X. Chen, B. Yu, C. Gao, K. Dang, Y. Liu, R. Men, A. Yang, J. Zhou, and J. Lin Group sequence policy optimization. External Links: 2507.18071, Link Cited by: §3.2.5.Source
  59. Zhou et al. (2023) J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou Instruction-following evaluation for large language models. External Links: 2311.07911, Link Cited by: §4.4.Source
  60. Zhou et al. (2026) Z. Zhou, R. Wang, Z. Wu, and Y. Jiang Daily-omni: towards audio-visual reasoning with temporal alignment across modalities. External Links: 2505.17862, Link Cited by: §1, item 1.Source