MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods

We initiate our data construction by aggregating a diverse array of multimodal datasets from the open-source community. We first leverage FineVision [46], an extensive collection of visual instruction datasets. We conduct a rigorous manual inspection of each candidate source, filtering out datasets unrelated to STEM or reasoning tasks. To expand the coverage of mathematical reasoning, scientific reasoning, and visual games & puzzles, we further incorporate high-quality datasets such as BMMR [48], Euclid30K [19], Zebra-CoT-Physics [18], and GameQA-140K [42]. This strategic expansion ensures a more balanced and challenging coverage of scientific and puzzle-solving domains. For comprehensive details and inclusion criteria, please refer to Appendix 7.

We implement a comprehensive data cleaning pipeline to guarantee the linguistic consistency, textual cleanliness, and reasoning suitability of the collected samples. The cleaning prompt is detailed in Prompt 11 (see Appendix 10 for representative noise cases). The procedure includes:

We further perform automatic image cleaning [46] by discarding corrupted or unreadable images, resizing those with a longest side exceeding 2048 pixels while preserving aspect ratio, and converting all images uniformly to the RGB color space.

The landscape of multimodal datasets is characterized by significant fragmentation and a lack of standardization. The heterogeneity in file formats and annotation structures across different sources poses a substantial challenge for unified data processing. For datasets that lack explicit ground-truth labels, we employ Qwen3-30B-A3B-Thinking to extract and store the missing answers in the answer field, following the prompt in Prompt 12. To facilitate downstream processes—such as caption generation, reasoning distillation, and correctness evaluation—and to enhance accessibility for the research community, we further convert all collected samples into a unified canonical schema. Each standardized data entry includes the following fields:

The schema fields are defined as follows: source denotes the origin dataset (e.g., Geometry3K), while id serves as the unique sample identifier. The Raw Data fields preserve the original_question and original_answer exactly as obtained from the source, which are then processed into the standardized Input/Output triplet (image, question, and answer) used for training.

Within the Augmented Annotations, qwen3vl_235b_instruct_caption and qwen3vl_235b_thinking_response denote the dense visual description and reasoning steps generated by the teacher model, respectively. The Metrics group includes qwen3vl_4b_pass_rate, which serves as a difficulty proxy based on a smaller model’s performance, alongside is_consistent and consistency_analysis, which provide automated verification of the generated reasoning against the ground truth.

We adopt a simple and light way to construct our MMFineReason dataset for training. Specifically, to ensure high-quality and non-redundant reasoning traces, we apply a multi-stage filtering procedure to the distilled outputs.

Following the above data selection pipeline, we obtain a high-quality MMFineReason-1.8M dataset with totally 5.1B solution tokens. We further uniformly sample 40k instances from MMFineReason-1.8M to construct an RL training subset, while the remaining data are reserved for SFT.

Given the massive scale of our curated data, training on the entire corpus is computationally suboptimal due to the prevalence of redundant or trivial samples. To address this, we employ a difficulty-based filtering strategy [54]. Specifically, we perform inference on each question using Qwen3-VL-4B-Thinking, generating four independent responses. We discard any example where the model provides a correct answer in at least one attempt (pass rate = 0). This conservative filtering criterion ensures that we retain only genuinely challenging problems where a moderate-sized model fails consistently. By discarding samples that contribute negligible training signals, we direct our computational resources toward challenging data points that actively drive optimization, resulting in faster convergence. Specifically, we derive two challenging subsets from the full MMFineReason-1.8M corpus: MMFineReason-123K and MMFineReason-586K, by retaining samples with pass rate = 0 and pass rate $\neq$ 1, respectively, which are well suited for efficient SFT and ablation studies. A detailed analysis of the difficulty score distribution is presented in Section 4.2.

Recent studies [51, 49] demonstrate that detailed image captioning significantly enhances multimodal reasoning capabilities. Building on this insight, we leverage the powerful Qwen3-VL-235B-A22B-Thinking model to generate structured, high-fidelity captions for the entire MMFineReason dataset. As shown in Figure 6 and Table 1, MMFineReason provides significantly denser semantic information, averaging 609 tokens per caption—more than double the length of HoneyBee [4] (299 tokens).

Crucially, distinct from HoneyBee which only incorporates captions for a subset of its samples (approx. 58%), MMFineReason guarantees 100% caption coverage for all image-question pairs. This comprehensive coverage ensures that every reasoning chain is explicitly supported by fine-grained visual details, providing a more consistent and robust foundation for multimodal learning than baselines with partial visual context.

Leveraging the generated captions as high-level semantic tags, we taxonomize the dataset into distinct STEM/diagrammatic and natural-image categories; the distribution is detailed in Table 2. The corpus is predominantly composed of STEM and diagrammatic content: geometric diagrams, mathematical plots, and logic puzzles collectively account for $75.3\%$ of the full set. This reflects the MMFineReason’s emphasis on symbolic and diagram-centric reasoning. While natural images constitute a smaller fraction, they exhibit significant internal diversity—ranging from urban scenes and documents to astronomical visualizations. This intentional distribution ensures MMFineReason prioritizes fine-grained mathematical reasoning while retaining a complementary natural subset to assess generalization beyond synthetic diagrams. We leave the investigation of optimal mixing ratios between these distributions to future work.

We quantify reasoning depth by analyzing the CoT token length distributions using the Qwen3-VL tokenizer (Figure 6 and Table 1). MMFineReason exhibits a substantially more elaborate reasoning process than existing baselines, achieving an average CoT length of 2,910 tokens—approximately $2.7\times$ longer than HoneyBee ($1,063$) and $4.3\times$ longer than OpenMMReasoner ($675$). Notably, the median token count of MMFineReason ($2,038$) is nearly $2.1\times$ that of HoneyBee ($972$) and over $11.3\times$ that of OpenMMReasoner ($180$). This disparity indicates that while baselines often provide concise or superficial rationales, MMFineReason consistently delivers extensive, step-by-step derivations. Furthermore, the extended tail of our distribution (Max: $16,316$) underscores the dataset’s capacity to handle highly complex, multi-stage reasoning tasks requiring deep cognitive traversal.

To evaluate the structural quality and domain adaptability of MMFineReason, we analyze the token length distribution and reasoning density across four curated domains. Using the Qwen3-VL tokenizer on the generated responses (qwen3vl_235b_thinking_response), we observe distinct characteristics (visualized in Figure 6):

We use LLaMA-Factory [62] and VeRL [40] as our SFT and RL training frameworks, respectively. The training configurations are summarized in Table 10 and Table 10. We train the Qwen3-VL-2B/4B/8B-Instruct models under the same experimental setup.

Supervised Fine-Tuning (SFT). As shown in Table 10, we optimize the model using AdamW with a learning rate of $1\text{e-}5$ and a cosine decay scheduler. To maximize training throughput and reduce memory fragmentation, we utilize liger kernel and enable sequence packing with a length of 32,768. The input images are resized to a resolution of $768\times 768$ during this stage to balance efficiency and performance. We train the models for 3 epochs with a global batch size of 32.

Reinforcement Learning (RL). For the RL stage, we adopt the GSPO algorithm [61] to enhance the reasoning capabilities of the model. As detailed in Table 10, we set the learning rate to $1\text{e-}6$ with a constant scheduler to ensure stable convergence. A key component of our setup is the generation of $G=16$ rollouts for each prompt to estimate the group-dependent baseline, which reduces the variance of the gradient estimator. The training spans 300 steps with a batch size of 256.

Evaluation Setup. We evaluate our models on VLMEvalKit [11]. To strictly assess the model’s reasoning reliability, we employ greedy decoding (Temperature $=0$) for all benchmarks. Notably, as shown in Table 10, we increase the maximum image resolution to $2048\times 2048$ during inference. More evaluation details are in Appendix 8.2.

Our baselines fall into three main categories: (1) Closed-source VLMs, including Gemini-2.5-Flash [9] and GPT5-mini High [35]; (2) Open-weight VLMs, specifically Qwen3-VL-8B-Thinking, Qwen3-VL-30B-A3B-Thinking and Qwen3-VL-32B-Thinking; and (3) Open-source VLMs. For MMR1 [17] and HoneyBee [4], we fine-tune Qwen3-VL-8B-Instruct on their respective SFT datasets to ensure a fair comparison. For OMR-7B [58], we directly report the RL results from the original paper and our evaluation using the officially released checkpoints. All models are evaluated in thinking mode to ensure consistent comparison.

To ensure a comprehensive assessment, we evaluate our model across a diverse suite of benchmarks spanning three key domains:

We first examine the impact of data quality by comparing models trained on full datasets under the same parameter scale (8B). MMFineReason-1.8M achieves a peak score of 75.7, establishing a substantial lead over existing open-source datasets such as MMR1-1.6M (67.4) and HoneyBee-2.5M (65.1). Despite utilizing a smaller sample size than HoneyBee (1.8M vs. 2.5M), our model achieves a performance gain of +10.6 points. This significant gap highlights that the fine-grained reasoning logic in our dataset provides much denser supervision signals than standard caption-based or coarse-reasoning datasets.

Beyond peak performance, MMFineReason exhibits remarkable efficiency in low-data regimes. The most striking finding is that our minimal subset, MMFineReason-123K, already achieves a score of 73.3. This result significantly outperforms models trained on the full HoneyBee (2.5M) and MMR1 (1.6M) datasets. By utilizing only $\sim$5% of the data volume of comparable benchmarks, MMFineReason matches or exceeds their performance. This ”cross-over” effect suggests that rigorous filtering and high-quality rationale construction effectively eliminate the redundancy found in large-scale datasets, allowing for faster convergence with a fraction of the computational budget.

Finally, we benchmark our models against leading open-weights and commercial baselines (represented by dashed lines in Figure 7). Our high-quality data enables smaller models to punch above their weight class. Specifically, our 4B model achieves a score of 73.9, surpassing the widely-used Qwen3-VL-8B-Thinking (72.5), which demonstrates that superior data quality can effectively compensate for a $2\times$ reduction in model parameters. Scaling up to 8B parameters, our model establishes a new state-of-the-art for its size class with a score of 75.7. It not only outperforms the significantly larger Qwen3-VL-30B-A3B-Thinking (74.5) but also exceeds the performance of the commercial baseline Gemini-2.5-Flash (75.0). These results validate that MMFineReason enables efficient open-weights models to compete directly with, and even surpass, proprietary and significantly larger foundation models.

Figure 8 (left) presents the effects of introducing caption augmentation (CapAug), specifically by prepending image captions enclosed within <caption>...</caption> tags to the response. The experiments reveal a clear trade-off in model performance. Specifically, CapAug slightly boosts logical reasoning capabilities, with LogicVista improving by 1.1%. This can likely be attributed to captions forcing the model to parse the layout and complete information of images, facilitating a better understanding of structure and logical relationships. However, on STEM visual reasoning tasks like MathVista and MathVision, it does not bring significant performance gains. This is likely because the Long CoT already contains sufficient visual information required for reasoning, rendering the caption information redundant. Consequently, we do not adopt CapAug in our final version.

The analysis of input resolution (Max Pixels) in Figure 8 (right) shows that simply increasing resolution does not always yield performance gains. Surprisingly, ultra-high resolution ($2048^{2}$) does not outperform medium-to-low resolution settings on multiple benchmarks. For example, on MathVista and CharXiv, the performance of $768^{2}$ or even $256^{2}$ is superior to that of $2048^{2}$. Although $2048^{2}$ achieves a slight advantage on MathVision, the overall benefit is limited considering the significantly increased computational cost. This phenomenon suggests that: (1) the core features of many current benchmark problems (especially geometry and charts) do not rely on pixel-level ultra-high definition; and (2) excessive resolution results in overly long visual token sequences, introducing redundancy and increasing the difficulty for the attention mechanism to capture key features. However, a notable exception is observed on RealWorldQA, which focuses on general natural images. On this benchmark, scaling up to $2048^{2}$ yields stable performance gains. This is likely because real-world scenarios often contain fine-grained details, small objects, or dense text embedded in complex backgrounds, which necessitate higher pixel density for precise recognition. In contrast to the sparse and structural nature of diagrams, natural images require high resolution to resolve these subtle visual cues effectively. Balancing performance across domains and computational efficiency, we adopt $768^{2}$ as our default setting.

By juxtaposing MMR1 ($N\approx 1.5\text{M}$) with ViRL39K ($N\approx 39\text{K}$), we observe a significant non-linear relationship between data volume and reasoning performance. While MMR1 establishes the upper bound with $73.60\%$ accuracy, ViRL39K achieves $72.79\%$ accuracy—retaining $98.9\%$ of the relative performance using only $2.4\%$ of the data volume. This constructs a new Pareto frontier, indicating that systematically cleaned, reformatted, and verified data (as seen in ViRL39K) can drastically reduce training costs. It further suggests that blind data scaling encounters severe diminishing returns beyond a certain threshold in multimodal reasoning tasks.

The most significant outlier in our scaling analysis is WeMath2.0-SFT. Despite comprising a negligible $0.05\%$ of the total data volume ($N=814$), it achieves a reasoning accuracy of $70.98\%$, effectively matching the performance of datasets three orders of magnitude larger, such as MMR1 ($73.60\%$). This disproportionate efficiency validates the “Knowledge-Oriented Chain-of-Thought” (KO-CoT) paradigm. We hypothesize that large-scale pre-training endows models with latent domain knowledge, but often leaves them without the specific reasoning syntax required for complex problem-solving. WeMath2.0-SFT functions not as a knowledge source, but as a high-efficiency catalyst, aligning the model’s internal representations with structured reasoning paths. This suggests that for foundation models, a small set of high-quality, reasoning-dense instructions is sufficient to trigger capabilities that otherwise remain dormant in massive, noisy corpora.

Puzzle/game-centric datasets are consistently weaker: GameQA-140K (122.9K, 67.67), Raven (20.3K, 68.08), and PuzzleQA (2.0K, 68.82). Despite their rigorous generation/verification (e.g., engine-derived annotations in GameQA-140K), these tasks emphasize planning-like search, symbolic state transitions, and abstract relational rules that may be mismatched with the dominant evaluation distribution (often math/science QA). Another plausible factor is that many puzzle/game solutions resemble program-execution traces; if the target model is not explicitly trained to internalize algorithmic state updates, these examples contribute less to general multimodal QA performance.

Geo3K (9.0K, 66.38) and Geo170K (11.8K, 67.02) are among the lowest performers, despite being in-domain for diagram reasoning. This highlights a critical nuance: geometry datasets can be structurally narrow (limited diagram styles, repetitive constructions, constrained linguistic patterns), which reduces their marginal utility for broad reasoning. Euclid30K performs substantially better (26.7K, 70.79), supporting the hypothesis that formalized reasoning structure and better-designed problem diversity (plane/solid geometry, proof-like steps) are more important than simply adding more geometry instances.

Comparing GameQA-140K ($67.67\%$) and BMMR ($72.94\%$) highlights the importance of disciplinary diversity in Chain-of-Thought (CoT) data. Although GameQA offers a larger volume of samples ($140\text{K}$), its scope is restricted to closed-world game logic. BMMR, despite being smaller ($80\text{K}$), spans over 300 academic disciplines. This breadth allows the model to internalize a more generalized reasoning structure, suggesting that for general-purpose vision-language models, diversity in the reasoning domain is a stronger driver of performance than the depth of any single task type.

As shown in Table 6, we first applied rigorous filtering based on sequence length and template validity.The ”Filt. (Len)” step removed samples that exceeded the context window limits or were anomalously short, while the ”Filt. (Tem)” step discarded samples with parsing errors or malformed templates.Overall, the raw datasets exhibited high structural integrity, with retention rates exceeding 95% for most subsets. notably, while large-scale datasets like MMR1 contained a higher absolute number of template errors (75,585), the relative loss remained minimal ($<5\%$).Some specific domains, such as VisualSphinx and Geometry3k, showed slightly lower retention rates ($\approx 92-93\%$), primarily due to complex formatting requirements inherent to their tasks.

Following structural cleaning, we evaluated the semantic quality of the remaining data using a ”Pass Rate” (PR) metric and a consistency check, as detailed in Table 7. The Pass Rate serves as an indicator of sample clarity or model competence, where a higher rate implies that the model successfully processed the sample. We observed a clear positive correlation between the Pass Rate and the Consistency Rate:

Ultimately, we identified approximately 1.81 million consistent samples (Total Consistent) out of the processed pool, providing a solid foundation for stable model training.

As shown in Table 10, the SFT stage utilizes a cosine learning rate scheduler with a peak learning rate of $1\text{e-}5$. To optimize training efficiency and memory usage, we employ sequence packing with a length of 32,768 and integrate Liger Kernel support. We also accommodate high-resolution inputs by setting the maximum pixel limit to $768\times 768$.

Following SFT, we further align the model using Reinforcement Learning. The detailed hyperparameters are listed in Table 10. In this stage, we adopt a constant learning rate of $1\text{e-}6$ to ensure stability. The training process involves 16 rollouts per prompt with a KL-divergence penalty controlled by $\epsilon_{\text{low}}=3\text{e-}4$ and $\epsilon_{\text{high}}=4\text{e-}4$. The input and output max lengths are set to 8,192 and 16,384 tokens, respectively.