I keep hearing that small on-device models are getting good enough for real work. I wanted a concrete test, not a vibe check. Payment screenshots are a useful stress case: look at the image, then pull out amount, merchant, date, and maybe a category.

At first this looked like an OCR problem. The screenshot already has the words. OCR can read those words. But reading the words is not the same as understanding what they mean. A UPI screen can say "Paid to", "Received from", a long UTR, a bank name, and an amount with commas. I needed structured fields, not a pile of text.

So I asked a sharper question. How far can small on-device models get on this, and where do classic OCR engines still win?

Cash amount is clear. The hard part is ignoring noise and filling structured fields.

Ground truth

{
  "amount": 112,
  "merchant": "ZEPTO MARKETPLACE",
  "date": "2026-08-04",
  "spending_category": "groceries",
  "payment_method": "UPI",
  "transaction_type": "expense"
}

Model output

Gemma 3nQwen2-VL 2BSmolVLM2 2.2BSmolVLM2 500M

• amount
• merchant
• date
• category

{
  "amount": 112,
  "merchant": "ZEPTO MARKETPLACE PRIVAT...",
  "date": "2026-08-04",
  "spending_category": "shopping",
  "payment_method": null,
  "transaction_type": "expense"
}

What I expected

I expected the bigger VLMs to win end-to-end extraction. A VLM, or vision-language model, is a model that can look at an image and reason about what it sees. That sounded perfect for messy payment UIs.

I also expected something under 500 MB to be "good enough" if I kept the prompt tight. That hope did not survive contact with the benchmark.

The VLM lineup

I ran 26 real cases through Gemma 3n, Qwen2-VL 2B, SmolVLM2 2.2B, and SmolVLM2 500M. The metric that mattered most was complete transaction accuracy: did the model return a usable transaction object, not just one lucky field?

The smaller model failed

SmolVLM2 500M was the romantic pick. About 361 MB on the scorecard. Easy to imagine on a phone. In this benchmark it scored 0% on complete transactions, safe auto-add, amount, merchant, date, and category.

It was also the slowest. About 16.8 seconds on average. So it was not just wrong. It made me wait to be wrong.

That changed the question for me. The useful filter is not "what is the smallest model?" It is "how small can the whole system get before it stops being useful?"

Amount is the easy win. Gemma and Qwen both hit 92%. Merchant is messier. Qwen leads at 64%. Category is where Gemma pulls away.

Latency sat around 5 seconds for the useful models. Five seconds feels like waiting for AI. That is fine for an offline import flow. It is not fine for a snappy camera experience.

Then OCR got interesting

OCR does a different job. It is about glyphs: reading characters from the image. I ran Apple Vision, PaddleOCR, and Tesseract across 128 probes.

Paddle won raw accuracy. Apple Vision won the speed tradeoff.

Vision is surprisingly good at merchant names and still strong on transaction IDs. Amount is where it slips. That matches the feeling I care about in a UI: merchant text can look native, while money values need tighter handling.

Preprocess mattered too. Cropped regions helped everyone. Tesseract jumped from almost useless on full images to somewhat usable after upscale and contrast. I still would not build the first path on it.

The mobile OCR rematch

The first OCR ranking used Paddle medium. That is useful lab data. It is less realistic if you care about a small mobile package. So I ran a later pass with Paddle mobile v4 and v5 against Apple Vision.

What I learned

• A tiny VLM can be small and still useless. Size is not useful by itself.
• Gemma is my current pick for full transaction quality. Qwen is the better pick when I only need amount and merchant.
• OCR is not obsolete. Vision gives me speed. Paddle gives me accuracy. They solve different moments in the flow.
• Five seconds of VLM latency changes the UX. Fast OCR can stay in the foreground. Heavy reasoning belongs in a deliberate import step.

What's next

I started by asking which model was best. I ended up asking a different question. How small can the entire system become before it stops being useful?

Next I want to test cropped regions before the VLM sees the image, a hybrid path where Vision extracts glyphs and a smaller language model turns them into JSON, and whether category prediction should live outside the vision model entirely.