Mini-VLM: a small vision-language model in your browser
A LLaVA-style model built from a SigLIP image encoder, a trained MLP projector and Qwen2.5-0.5B-Instruct with LoRA adapters. The image becomes 196 tokens that the language model reads like words. Ask about a photo and the answer is generated on your device.
This model (stage 2, LoRA instruction-tuned): 76.8% acc / 80.1 F1 on POPEStage 1 (projector only): 109.0 CIDEr on COCO captions36.7M trainable parameters