Generalized Visual Language Models
Lilian Weng
The design choice is where image information enters a language model—and whether adapting it changes the language model at all.
Open library
The library
The essential reading of generative AI and agents, open in a clean reading view, with Meow and Standard summaries when available. No account needed to read.
560 papers · 558 with a Meow summary · 333 posts
Key publications from Anthropic, OpenAI, Google DeepMind, Meta, DeepSeek, Qwen, Z.ai, Kimi and other labs, plus the most influential essays and blogs.
Multimodality and Large Multimodal Models (LMMs)
Chip Huyen
The article’s core design pattern is to align different modalities in a shared embedding space, then let a language model condition its answers on visual tokens.
Understanding Multimodal LLMs
Sebastian Raschka, PhD · Sebastian Raschka
The article’s central point: multimodal LLMs can join image and text either by concatenating their embeddings or through cross-attention. Neither design wins every trade-off.