# Visual Retrieval: Cut Document AI Costs by 90% with ColQwen2

> Learn how visual retrieval with ColPali/ColQwen2 can drastically reduce document AI costs by replacing expensive parsing with GPU-time, saving up to 90%.

Published: 2026-09-09

Canonical: https://sygnet.ai/blog/visual-retrieval-cut-document-ai-costs-by-90-percent-with-colqwen2

---

## Key takeaways

- Parsing every page upfront is the largest cost in document-AI budgets, with services like Azure Layout costing $10,000 per million pages before indexing.
- Visual retrieval methods like ColPali/ColQwen2 replace this ingest cost with GPU time, reducing processing from 7.22 s/page to 0.39 s/page, or 108 GPU-hours per million pages instead of ~2,000.
- This approach trades ingest cost for storage, as a full ColPali index requires about 250 GB per million pages, making quantisation essential.
- Parse-on-retrieval significantly reduces costs for corpora where only a small fraction of pages are queried, potentially costing $80 per million pages instead of $4,000 for deferred parsing at a 2% touch rate.

## Should you stop parsing every page?

Yes, for retrieval-driven corpora where most pages will never be read, but no for anything feeding a transactional workflow. The rule that holds up in cost modelling: if a page's extracted fields will be written into a system of record (invoice lines, claim amounts, KYC identifiers), parse it at ingest. If the page only exists to be *findable*, index it visually and parse it on the way out.

The reason this matters at all is corpus asymmetry. Analyst estimates have long put unstructured content at roughly 80 to 90 percent of all enterprise data, and Splunk research found 55 percent of an organisation's data is dark on average, collected, stored, never used. A pipeline that runs full [document parsing](https://sygnet.ai/glossary/document-parsing) on every page pays 100% of the extraction cost to serve a query pattern that touches single-digit percentages of the archive. That's the arbitrage visual retrieval exploits.

## What is visual retrieval (ColPali/ColQwen2) and why does it skip parsing?

Visual retrieval embeds a rendered page image directly, with no OCR, layout detection or text chunking in between. ColPali is built on a SigLIP-So400m vision encoder producing 1,024 patch embeddings per page (a 32×32 grid over 448×448 input), each projected to 128 dimensions, and it computes relevance through MaxSim, summing the maximum similarity between each query token and all document patches. That late-interaction mechanism, inherited from ColBERT, allows fine-grained matching while staying tractable at scale.

ColQwen2 is the practical successor: a Qwen2-VL-2B extension that is more accurate than ColPali on ViDoRe (+5 nDCG@5 points), permissively licensed, and uses 768 patch embeddings instead of 1,024, cutting both compute and storage. On benchmark quality, ColPali outperformed all other evaluated systems on ViDoRe, including baselines that used a strong proprietary vision model (Claude Sonnet) to caption every visual element. Crucially, the approach removes the need for brittle layout recognition and OCR pipelines, using one model that accounts for both textual and visual content.

## What does parse-on-retrieval mean in practice?

Parse-on-retrieval means the visual index answers "which page?", and only those returned pages get sent to a parser or [vision-language model](https://sygnet.ai/glossary/vision-language-model) for structured extraction. A typical flow: render pages → embed with ColQwen2 → store multi-vectors → at query time, retrieve top-k pages via MaxSim → parse those k pages to markdown/JSON → generate the answer with citations back to the page image.

Two properties make this work. First, retrieval latency is not the bottleneck: query encoding with ColPali's language model takes about 30 ms, and the late-interaction operation adds roughly 1 ms per 1,000 pages in the corpus. Second, parsing k pages instead of N pages moves the expensive step from a fixed capital cost to a marginal cost that scales with actual usage. The catch: your first answer now carries parse latency inline, so cache aggressively, a page parsed once should never be parsed twice. Skipping ingest-time [chunking](https://sygnet.ai/glossary/chunking) also removes a whole class of tuning work, since the retrieval unit is simply the page.

## How much does 1M pages actually cost under each strategy?

Here is the model, per 1,000,000 pages, using published list rates. Indexing compute is derived from the ColPali paper's measured 0.39 s/page on an NVIDIA L4 (≈108 GPU-hours per million pages) priced against a cited GPU rate; treat it as an order of magnitude, not a quote.

| Strategy | Ingest cost per 1M pages | Index storage per 1M pages | Structured fields at ingest? |
|---|---|---|---|
| Azure Layout, parse everything | $10,000 ($10/1k) | Text + single vectors (GB scale) | Yes |
| Mistral OCR 4, batch, parse everything | $2,000 ($2/1k batched) | Text + single vectors | Yes |
| Textract Forms + Tables | $65,000+ ($15/1k tables, $50/1k forms) | Text + single vectors | Yes, key-value |
| ColQwen2 visual index only, self-hosted | ~108 GPU-hours (≈$110 at $1.04/hr A100 on-demand) | ~192 GB at fp16 (768 × 128-dim vectors/page) | No |
| Visual index + parse-on-retrieval (2% touched) | ≈$110 GPU + $80 parsing at $4/1k | Same ~192 GB (or ~6 GB binarised) | On demand |

The spread between the top and bottom rows is roughly two orders of magnitude. Note also that Azure gives you almost no cost levers on the parse-everything path: batch Read is $1.50/1k, batch Layout $10, batch custom extraction $30, there is no batch discount, unlike Gemini and Mistral which halve batch inference. If you want to sanity-check these numbers against your own volumes and error rates, Sygnet publishes an [ROI calculator](https://sygnet.ai/roi-calculator) for per-document processing economics.

## Where does the money actually go in a full-parse pipeline?

Preprocessing, not embedding. The ColPali authors measured the breakdown per page: layout detection 0.81 s, OCR 2.67 s, captioning 3.71 s, page encoding 0.03 s, 7.22 s total, against 0.39 s for ColPali end-to-end. Independent benchmarking reaches the same conclusion from a different angle: for PDF/image pipelines with OCR tools, format conversion accounts for 98.2% of indexing duration on average, with average GPU utilisation around 10%.

That's the structural insight. In a text-first [RAG](https://sygnet.ai/glossary/retrieval-augmented-generation) architecture, you are paying, in API fees, GPU-hours and wall-clock time, mostly to convert pixels into strings, then throwing away the layout you just spent money detecting. OCR-based pipelines break on real enterprise documents because they discard spatial relationships between tables, charts and text; a multi-column financial report loses most of its information when flattened to a string.

## What are the hidden costs of visual retrieval?

Storage and granularity, and both are quantifiable. Multi-vector storage runs 10× to 100× a dense single-vector index, because you keep a vector per token, and at 10M pages the difference between single-vector and ColPali-class indexes is tens of GB versus several TB, before replication and indexing overhead. Compression is well-trodden: int8 quantisation cuts storage 4×, Vespa stores the vectors as int8 with binary compression for a 32× reduction versus float, and Matryoshka truncation from 1024 to 256 dims typically costs under 2% recall while cutting storage four-fold.

The second cost is architectural: ColPali-family retrieval is page-level. ViDoRe measures NDCG@5 for page-level retrieval, and region-level retrieval remains largely unexplored. You get "this page is relevant", not a bounding box, not a validated field, not a confidence score you can route on. For anything requiring [table extraction](https://sygnet.ai/glossary/table-extraction) or auditable field-level provenance, a parse step still has to happen, the question is only *when*.

## When is full VLM ingestion still the right call?

When the touch rate is high, the output must be structured, or the workflow is transactional. The breakeven is arithmetic: deferred parsing wins while (pages retrieved × on-demand parse price) < (all pages × batch parse price). With Mistral OCR 4 at $4/1,000 pages standard and $2/1,000 batched, that breakeven sits at a 50% touch rate. Above it, parse everything up front and take the batch discount.

Three other cases force full ingest regardless of maths: (1) regulated workflows where every page must be classified and retained in extracted form; (2) straight-through processing, where fields are written to an ERP or claims system without a human in the loop; (3) corpora where the same pages are read constantly, cached parses amortise instantly. Sygnet's write-up on [OCR vs VLM extraction](https://sygnet.ai/ocr-vs-vlm) covers where each engine class actually breaks. In practice, most enterprises land on a hybrid: full extraction on the transactional 5% (invoices, claims, onboarding files), visual index plus parse-on-retrieval on the archival 95%.

## FAQ

### Is ColPali cheaper than OCR at scale?

At ingest, yes, substantially. Skipping layout detection, OCR and captioning takes per-page indexing from 7.22 s to 0.39 s on the same NVIDIA L4 hardware, roughly an 18× compute reduction, and one GPU-cloud analysis claims self-hosting ColQwen2.5 for indexing runs 37× cheaper per page than Azure Document Intelligence. But you then pay in index storage, about 250 GB per million pages unpooled, so total cost of ownership depends on your vector-database bill, not just GPU-hours.

### How much storage does a ColQwen2 index need per million pages?

Roughly 190 GB at 16-bit precision. ColQwen2 divides each page into 768 tokens of 128 dimensions; stored as 16-bit floats, a 50-page document consumes 9.6 MB, about 192 KB per page. Int8 quantisation brings that near 48 GB, and binary compression as used in Vespa reduces the footprint 32× versus float, landing around 6 GB per million pages with a modest recall cost.

### Does parse-on-retrieval hurt answer quality?

Not for retrieval accuracy, and it can help extraction quality. Visual retrieval finds the right page without an OCR failure mode in the loop, the model reads the page as a human would, with no OCR step to fail and no layout detection confusing a multi-column abstract with a sidebar. Quality risk shifts to the parse step, which now runs on far fewer pages and can therefore use a more expensive, more accurate model than you could afford across the whole corpus.

### What corpus size justifies visual retrieval?

Below a few thousand pages, it barely matters. Under 1,000 pages, memory usage is measured in megabytes and indexing finishes in minutes, so the choice should prioritise retrieval quality and available infrastructure over efficiency. The economics become decisive in the hundreds of thousands to millions of pages, where index size determines whether it fits on a single node or requires distributed storage, and where a $10/1,000-page ingest decision turns into a five- or six-figure commitment.