Michael Chang
Cohere Labs Community
Michael Chang is the research lead of Tiny Aya Vision. He is a machine learning engineer working on computer vision and fascinated by multimodal research.
Tiny Aya Vision
Extending Tiny Aya, a 3.35B multilingual model in 70+ languages, with lightweight visual capabilities through parameter-efficient fusion.
Part of Expedition Tiny Aya. Code: Cohere-Labs-Community/expedition-tayavision. Results below reflect an in-progress v0 model, not a finished system.
What’s Tiny Aya Vision?
Tiny Aya Vision is an open-weight multilingual vision-language model (VLM) with fewer than 4B parameters, and it is the first of its kind. It extends Tiny Aya
Core question: Can a small (~3B) multilingual language model gain effective visual grounding through parameter-efficient fusion without sacrificing multilingual text performance or on-device deployability?
Why This Matters
No existing sub-4B model combines vision capabilities with support for 70+ languages, particularly for low-resource languages.
| Model | Params | Languages | Vision |
|---|---|---|---|
| Tiny Aya Vision | ~4B | 70+ | Yes |
| Qwen3-VL-2GB | 2B | ~32 | Yes |
| Gemma 3-1B | 1B | 140+ | Limited |
| SmolVLM | <4B | English-centric | Yes |
| Ministral-3-3B | 3.4B | Basic multilingual | Yes |
Claim
Tiny Aya adds vision to a 3.35B multilingual (70+ languages) model using a frozen encoder, a small connector, and LoRA
Evidence
Architecture
Tiny Aya Vision uses a late-fusion design, following Aya Vision and LLaVA
| Component | Configurations | Params | Trained |
|---|---|---|---|
| Vision Encoder | SigLIP2-so400m | ~400M | Frozen |
| Connector | Pixel Shuffle (2x2) + 2-layer SwiGLU MLP | ~12M | Stage 1 |
| LLM backbone | Tiny Aya Global, 70+ languages | 3.35B | Frozen |
| LLM adaptation | LoRA, rank 256 | ~280M | Stage 2 |
An image enters SigLIP2 at 384×384 and comes out as 729 patch embeddings. Passing all of them to a 3.35B backbone is expensive relative to the model’s total budget, so Pixel Shuffle stacks each 2×2 neighbourhood along the embedding dimension, trading spatial resolution for depth and cutting the sequence to 196 tokens. The SwiGLU MLP then projects those into the backbone’s embedding space. Only the connector trains in Stage 1; LoRA is added in Stage 2 so the backbone can adapt to visual input without full fine-tuning.
Training Recipe
Training follows LLaVA’s two-stage recipe, previously validated by Aya Vision and Maya
We then add a training-free third step: cross-modal weight merging. The LLM weights are linearly interpolated between the fine-tuned multimodal checkpoint and the original text-only Tiny Aya Global,
W_merged = (1 − α) · W_text + α · W_mm
with the connector and vision encoder left unchanged. Aya Vision selected α = 0.4 at both 8B and 32B based on a sweep over interpolation weights (Fig. 9). At that setting, Aya Vision 8B limited text-only win-rate degradation on m-ArenaHard to 5.92% relative to the LLM it was initialized from, compared to 16.4% for Pangea-7B, 22.1% for Qwen2.5-VL-7B, and 44.1% for Molmo-7B (Fig. 5). Whether this behavior holds at 3.35B, where there is far less parameter redundancy to absorb the merge, remains an open question.
| Benchmark | Blind Score | Notes |
|---|---|---|
| CVQA | 25.60% | Tiny Aya Global; chance ≈ 25% for 4-choice |
| Kaleidoscope | 26.07% (avg) | Range: 19% (Nepali) to 33.6% (Portuguese) |
| MaXM | ~0.0% | Generation task, impossible without images |
| MTVQA | 0.0% | Text reading in images, impossible without vision |
Tiny Aya Global scores 25.6% on CVQA in the blind setting, with no image at all. After English-only alignment, it jumps to 45.5%, an increase of nearly 20 points, confirming real visual grounding rather than just language priors.
By language, Portuguese and Spanish lead (~50%+), reflecting the strength of high-resource, Latin-script languages. Sinhala is lowest (~21%), with Kinyarwanda and Oromo close behind (the lowest-resource languages), showing that this isn’t simply a script issue (Kinyarwanda and Oromo are Latin-script too). That gap is exactly what Milestone C’s culturally diverse visual data is designed to close.
Limitations
These results are early. The CVQA blind baseline sits at 25.6%, near chance, so it confirms the model gets real signal from images. But CVQA is partly language-solvable too. The gap is wide enough to show vision helps. It’s not wide enough to cleanly separate visual reasoning from language priors.
The per-language numbers here only cover Milestone A: English-only visual training. Milestone B (multilingual SFT) and Milestone C (culturally diverse visual data, a stretch goal) haven’t happened yet. Low-resource-language scores will likely look better once those land.
Cross-modal merging is also untested at this scale. At 8B, it recovered text performance and pushed vision win-rate up 20.5%. We don’t yet know if α=0.4 is still the right value at 3.35B, where the model has much less parameter redundancy to work with.
Two more open items: we haven’t run the MoonViT ablation, and we haven’t resolved the LoRA learning-rate discrepancy between Maya and Aya Vision at this scale.
Next Steps
A few things are still in progress. We’re running the merge ratio sweep (α = 0.3 to 0.7) at 3.35B to find whether the same Goldilocks zone from 8B holds here. We’re finishing Milestone B (multilingual SFT) and comparing it against Milestone A to isolate what explicit multilingual data adds on top of cross-lingual transfer. Milestone C, culturally diverse visual data from XM3600
We also want to settle the LoRA learning-rate question left open by the Maya and Aya Vision comparison, and finish the MoonViT ablation to see if native-resolution encoding matters for script-heavy languages like Arabic or Thai.
Once results are final, we’ll release model weights on Hugging Face, publish the code, and report on-device latency numbers.
About the authors
