VLM (Vision-Language Model)

An AI model that reads and understands the layout and meaning of a document, not just the individual characters on it, the way traditional OCR does.

Instead of only spotting letters, it can tell that a block of text is a name, that a number is a date of birth, and that a stamp is a security feature.

A machine learning model trained jointly on images and text, enabling it to interpret a document's structure and content in context, rather than only extracting characters field by field against a fixed template. A template is a fixed map that says where each field sits on a particular document design.

A vision-language model works on a document in a few steps:

1. It takes in the whole image, including layout, labels, and text.
2. It uses what it learned about documents to work out what each region means.
3. It outputs structured fields, often with a confidence level.

The approach has clear strengths and limits:

• Flexibility, since it can handle layouts it has never seen.
• Context, since it understands labels, such as which date is the birth date and which is the expiry date.
• Risk of errors, since large models can produce confident but wrong answers.

Because of that last point, strong systems check a model's output against other sources, such as check digits and the machine-readable zone.

A vision-language model sits next to terms that are easy to mix up:

• Optical character recognition reads characters, while a vision-language model also interprets layout and meaning.
• Data extraction is the step of producing structured, scored fields, which a vision-language model can perform.
• Document coverage is how many document types a system can read, which this approach can widen.