The output looked readable. Retrieval told a different story
Korean public documents combine scans, multi-column layouts, forms, charts, stamps and dense tables. We learned quickly that readable text alone was not the useful output. The structure had to survive long enough for retrieval to recover it.
A parser can look visually clean and still remove the value that answers a user’s question. That gap changed the evaluation plan: WigtnOCR separates intrinsic parsing quality from downstream retrieval quality instead of compressing both into one headline score.
OmniDocBench measures text, tables, formulas and reading order. KoGovDoc then holds the retriever and chunking policy fixed and measures how much answer-bearing content remains recoverable after parsing.
Four kinds of parser, and what each one loses on a Korean government PDF
The model exists because the four things you would reach for first each fail differently on this corpus.
It recovers text and drops the structure that says which value belongs to which field. PaddleOCR, the one measured here, returned between a third and a thirtieth of the text the deployed model did, losing most tables, forms and multi-column layouts.
PyMuPDF4LLM extracts quickly and recognises almost no structure: the article-clause-item hierarchy in a statute, and any page mixing a table with a diagram and prose, come out flat. Not the whole family, though. Marker is rule-based and leads text accuracy and reading order in the comparison two sections down.
The current open VLM parsers are trained mostly on English and Chinese material. Korean government documents bring complex tables, forms, official seals, scanned pages mixed with digital ones, and multi-column layouts they were not tuned for.
Parsing quality is high, but it needs two GPUs and is slow to serve. The project's constraint was the GPU budget, and a 2B model meets it: one GPU to serve, and an edge deployment that is actually realistic.
Use the 30B teacher offline and serve the 2B student
The practical decision was to spend the large-model budget during data creation, not on every document served. The 30B model creates structured supervision offline; production inference runs through the 2B student.
Qwen3-VL-30B-Instruct converts 4,501 page images from 49 documents into structured Markdown.
Qwen3.5-122B scores structure, table quality, completeness, hallucination and consistency.
Low-quality pages are removed, document imbalance is corrected, and 294 government pages are held out.
Qwen3-VL-2B-Instruct is fine-tuned with LoRA rank 8 / alpha 32 for three epochs using ms-swift and ZeRO-2.
Released training corpus
| Source | Documents | Pages | Role |
|---|---|---|---|
| KoGovDoc | 10 | 3,637 | Domain adaptation |
| ArXiv | 39 | 864 | Layout diversity |
| Total | 49 | 4,501 | Teacher generation |
The judge reads text only, and that is the point
A 122B text-only model scored every generated page on five dimensions: structure, table quality, completeness, hallucination and consistency. It never sees the source image, and that is deliberate rather than a shortcut.
A vision model grading a vision model's output shares its visual interpretation bias, so the two agree on the same misreading and the evaluation closes a loop instead of testing anything. Separating the judge into a text-only model asks a different question: not whether this matches the image, but whether this output is usable as training data at all. Repetition loops, truncated text and leaked reasoning are all detectable from the text alone.
Scores run one to five and anything below three was dropped. 75.1% of the Korean government pages cleared that bar, and 73.8% of the arXiv pages.
Two things wrong with the corpus before any training ran
- 01One document accounted for 53% of the pages. A model trained on that learns that document rather than the domain, so a per-document ratio cap of 0.25 was applied.
- 02The reasoning teacher had left English thinking traces inside some of the generated Markdown. Twenty pages were deleted outright and 257 were repaired.
- 03What survived: 2,667 training pages and 294 held out and excluded from training. The split is at page level; the source does not establish that a held-out page never shares a document with a trained one.
The student improves tables without winning every metric
WigtnOCR matches the teacher on text NED and substantially improves table TEDS, while the teacher remains stronger on formula CDM.
OmniDocBench selected metrics
| Parser | Text NED ↓ | Table TEDS ↑ | Formula CDM ↑ | Order NED ↓ | Skip ↓ |
|---|---|---|---|---|---|
| Qwen3-VL-30B | 0.289 | 0.523 | 0.939 | 0.227 | 5.5% |
| Qwen3-VL-2B | 0.364 | 0.561 | 0.865 | 0.300 | 18.8% |
| Marker | 0.218 | 0.586 | 0.863 | 0.165 | 0.4% |
| WigtnOCR-2B | 0.288 | 0.649 | 0.884 | 0.211 | 5.8% |
A bigger adapter made the tables worse
At this data size, rank 8 beat rank 32. The larger adapter improved formula CDM slightly and regressed the structure preservation the model exists for.
Rank 32 costs 4.9 points of Table TEDS and 2.1 points of text NED, which is an error metric, so that is a regression too. Five epochs overfits: validation loss turns up, and the table metric does not return to rank 8's level.
The tempting row is v2 at five epochs, which reaches a 0% skip rate. It gets there by producing something for every page rather than by parsing better, and the parsing metrics say so. The deployed model keeps a 5.8% skip rate and the table quality, which is the trade this corpus rewards.
LoRA configuration against OmniDocBench
| Config | Rank | Epochs | Text NED ↓ | Table TEDS ↑ | TEDS-S ↑ | CDM F1 ↑ | RO NED ↓ | Skip % ↓ |
|---|---|---|---|---|---|---|---|---|
| v1, deployed | 8 | 3 | 0.288 | 0.649 | 0.732 | 0.884 | 0.211 | 5.8% |
| v2, best | 32 | 3 | 0.309 | 0.600 | 0.697 | not run | 0.215 | 0.7% |
| v2, last | 32 | 5 | 0.306 | 0.610 | 0.695 | 0.892 | 0.214 | 0.0% |
Cleaner chunks do not automatically retrieve better
MinerU produces the strongest boundary metrics but ranks fifth in retrieval. WigtnOCR preserves more answer-bearing structure and leads Hit@1, Hit@5 and MRR@10.
KoGovDoc retrieval, 564 queries
| Parser | Hit@1 ↑ | Hit@5 ↑ | MRR@10 ↑ | nDCG@10 ↑ |
|---|---|---|---|---|
| WigtnOCR-2B | 0.739 | 0.855 | 0.788 | 0.437 |
| Qwen3-VL-30B | 0.716 | 0.839 | 0.771 | 0.411 |
| Marker | 0.711 | 0.853 | 0.771 | 0.412 |
| Qwen3-VL-2B | 0.709 | 0.814 | 0.756 | 0.444 |
| MinerU | 0.608 | 0.789 | 0.682 | 0.384 |
| PaddleOCR | 0.512 | 0.693 | 0.592 | 0.293 |
What did not transfer cleanly
- 01Formula CDM remains below the 30B teacher, so the compact model should not be presented as uniformly better.
- 02Five of 294 KoGovDoc validation pages failed to produce evaluable output.
- 03Qualitative examples still contain character-level OCR errors even when chart and table structure improves.
- 04The retrieval result is specific to Korean government documents, BGE-M3 and the released chunking policy.
- 05No controlled throughput, energy or serving-cost comparison has been released.
Where the claim stops
- L01KoGovDoc represents one Korean government-document distribution and one retrieval stack.
- L02Pseudo-label filtering reduces weak supervision but does not turn generated labels into human ground truth.
- L03The release supports a parameter-count claim; it does not yet support a precise speed or cost-reduction claim.