Xiaomi AI Lab Introduces Xiaomi-Lens-0: 4B MLLM Built on Qwen3.5 Sets SOTA on RefCOCO with Native Pixel-Level Mask Decoding

Background and the Problem

While modern Multimodal Large Language Models (MLLMs, e.g., Qwen-VL, LLaVA) perform impressively on holistic image question answering, they struggle with fine-grained spatial grounding and pixel-level segmentation. Conventional architectures rely on text-only or coarse bounding-box supervision, leaving token embeddings largely disconnected from spatial image structures. Existing segmentation-augmented models (such as LISA and PixelLM) typically append heavy external decoders that drastically increase latency and degrade conversational abilities. Researchers at Xiaomi AI Lab unveiled Xiaomi-Lens-0 (arXiv:2610.23279), a compact 4B multimodal model solving spatial grounding natively.

Architecture and How It Works

1. Native Pixel-Level Mask Decoder:

• Built upon Qwen3.5 4B, Xiaomi-Lens-0 integrates a lightweight cross-attention mask decoder that combines three signals: vision-encoder patch embeddings, visual projection latents, and the language model's [MASK] token hidden states to directly output fine-grained pixel masks;

1. Multi-Stage Progressive Spatial Supervision:

• Pretrains mask projection layers before jointly optimizing the full backbone with a combination of autoregressive next-token prediction, Focal loss, and Dice loss, preserving general vision-language conversational fluency while acquiring pixel-exact spatial precision.

Benchmarks and Measured Results

Evaluated on Referring Expression Comprehension (REC), Referring Expression Segmentation (RES), and general perception benchmarks (Tables 1, 3, 6, 8):

• Referring Expression Comprehension (REC, [email protected]):
• Scores 94.6% on RefCOCO (beating Florence-2 at 93.4% and Grounding-DINO-L at 90.6%);
• Hits 92.1% on RefCOCO+ (outperforming Florence-2 at 88.3% and Grounding-DINO-L at 82.8%);
• Reaches 92.5% on RefCOCOg (vs Florence-2 at 91.2%);
• Referring Expression Segmentation (RES, cIoU):
• Achieves 83.4% on RefCOCO, 79.9% on RefCOCO+, and 81.1% on RefCOCOg, decisively outperforming 7B-parameter architectures like LISA LLaVA-7B (74.9% / 65.1% / 67.9%) and GSVA Vicuna-7B (77.2% / 65.9% / 72.7%);
• General Multimodal Perception:
• Achieves 69.7% on MME-RW-en, 67.5% on MME-RW-cn, and 88.1% on CV-B, surpassing larger 7B models including Qwen2.5-VL 7B (60.8% / 58.3%).

Getting Started for Developers

• Research Details: Full architectural proofs and dataset splits are available in arXiv:2610.23279.
• Edge Deployment Takeaway: Demonstrates that a 4B-parameter model with an integrated patch-and-mask-token decoder surpasses 7B–13B models in spatial grounding. Engineers building on-device photo editing, GUI element segmentation, or robotic pick-and-place systems can adopt this design to achieve sub-50ms pixel-level interactive segmentation.