What fine-tuning is: a complete guide to when additional training is actually needed

Fine-tuning takes a model already trained on large-scale data and trains it further on material matched to a specific document set and set of business rules, raising performance within that domain.

Problems arise at the exceptions, not the average

Problems arise at the exceptions, not the average.

A general-purpose model performs well on ordinary documents. But the point where an incident occurs in a real operation is not the average document; it is the exceptional one. Stamps intruding into tables, strikethroughs crossing rows, fax headers stacked four deep.

Measurement makes the difference plain. Drawing 1,000 pages at random from 900,000 public administrative documents, accuracy at finding field names separated at 97.3 percent against 80.4 percent, while accuracy at retrieving the value bound to that name separated at 96.0 percent against 48.7 percent, nearly a factor of two. That is the gap between a model trained on the specific document set and one that was not.

What fine-tuning actually is: how it differs from adjusting criteria

Criteria adjustment and fine-tuning across three axes

First, what changes differs. Adjusting extraction criteria changes the prose telling the model what to look for. Fine-tuning changes the model's internal values.

Second, what is required differs. Criteria adjustment needs business knowledge and a handful of examples. Fine-tuning needs a substantial volume of labelled training data, plus compute and time.

Third, reversal differs. Criteria adjustment is undone by reverting the text. Fine-tuning is only reversible if the previous model version was retained.

The two are not alternatives. As a matter of sequence, trying criteria adjustment first and reserving fine-tuning for what it cannot reach gives the better return.

Four criteria for the decision

Four criteria to weigh

First, confirm whether criteria adjustment resolves it. Refining field definitions and changing extraction hints solves a substantial share of problems. Skipping this step produces unnecessary training cost.

Second, confirm that training data can be produced. Enough labelled documents have to exist, and producing those labels takes considerable time.

Third, confirm that the document set is stable. In an area where forms change monthly, the target will have moved by the time training finishes.

Fourth, confirm the personal data policy. Using real business documents for training requires de-identification first, and the training environment has to be a controlled one.

How fine-tuning is carried out

Collect data around the hard cases

Adding more documents that already process well does not raise performance. Collecting the failures and the cases routed to exceptions is far more effective.

Review history is what helps here. Fields an operator corrected repeatedly, and cases separated as exceptions for low confidence, become training data candidates directly. Adding well-handled documents only has the model relearn what it already knows.

Separate the evaluation set first

Training data and evaluation data must not mix. Measuring performance on documents used in training reports a figure higher than reality.

Separate the evaluation documents before training begins and keep them out of training entirely. Without that separation there is no way to distinguish improved performance from memorisation.

Fine-tuning in the Korean environment

Real business documents in Korean finance and the public sector contain personal data, so de-identification has to precede any training use. On top of that, network separation requires the training process itself to run inside the internal network.

When considering fine-tuning, therefore, confirm not only model performance but whether the training environment can be built internally. Configurations that send documents out for training drop out at this point.

Frequently asked questions