View all glossary terms

Data and quality

What Is Exact Match: The Strict Standard for Honestly Measuring Document AI Accuracy, A Complete Guide

Korea Deep LearningPublished

Exact Match (exact match rate) is a metric that counts a value as correct only when the extracted value is identical to the answer set by a person, down to every letter and every symbol. It is the strictest standard for judging document AI performance.

Why accuracy does not mean the same thing everywhere

The same product gets reported with figures that differ by double digits.

When accuracy is discussed during a solution review, different numbers sometimes come up together. Neither is false, yet they differ. The former figure is a similarity standard that allows for differences in spacing and symbols, while the latter is an exact-match standard that counts every single character.

The unit of calculation also differs. If you count by character, a single wrong letter in a long address still counts as mostly correct. If you count by item, that same address counts as entirely wrong. From a business standpoint, the latter calculation is the right one, because a value with one letter wrong cannot be entered into a system.

So what matters in an adoption review is not what percentage it is, but what was counted and how.

The precise definition of Exact Match: how it differs from similarity-based accuracy

Three axes that separate Exact Match and Fuzzy Similarity

First, the way they judge differs. Exact Match judges each value as either right or wrong. Fuzzy Similarity calculates how similar two strings are as a ratio, giving partial credit.

Second, what they reflect differs. Exact Match treats even minor differences in notation as errors. A business registration number missing its hyphen, or a company name with different spacing, gets caught here. Fuzzy Similarity reflects cases that a person reading the value would judge to be the same, so it is closer to perceived accuracy.

Third, the place they are used differs. Exact Match is the standard in work where the extracted value is entered into the system as is. If the value differs even slightly, registration fails or incorrect data accumulates. Fuzzy Similarity is useful for comparing recognition quality or estimating the review burden. In one actual validation, a financial document set showed high similarity-based accuracy for printed and handwritten text, yet its exact-match target was set lower.

The two metrics are not alternatives. Looking only at similarity overlooks the re-entry burden that will arise in real work. Looking only at exact match mistakes notation issues that normalization could solve for a performance shortfall. It is more accurate to look at both values together.

Five criteria for using Exact Match properly

Five criteria for measuring exact-match rate

First, take the item as the unit of calculation. Calculating by character produces numbers that are far from the actual impact on work. The right unit is one item, the value staff enter into the system.

Second, set the normalization rules in advance. If you do not decide how to treat hyphens, spaces, parentheses, and unit notation, the same result will produce a different score each time it is measured. It is practical to normalize the notation differences that the work allows before comparing against the answer.

Third, break the results down by item. With only one overall average, you cannot tell which item is the problem. Amounts, dates, names, and account numbers each have different types of errors and different impacts on the work.

Fourth, set the target by business standards. Demanding 100% is not realistic, but the target cannot be set arbitrarily either. Calculate it together with what one error leads to and whether validation rules and human review come after it.

Fifth, count undeterminable items separately. Some items have no value in the original, or the scan has lost information so that even a person cannot judge them. Counting these items as wrong turns the evaluation into a measure of document quality rather than product performance.

How to apply exact-match rate in practice

Build the answer set and the normalization rules together

The first output of measurement is the answer set. This is a job of having a person confirm and record the correct answer for each item in each document, and it is better to record the normalization rules alongside.

For example, a business registration number is compared as ten digits with the hyphens removed, an amount is compared as a number with commas and currency symbols removed, and a date is converted to a set format before comparison. If these rules are written down, the same standard applies when you measure again later.

Manage it separately from operating metrics

The exact-match rate is a metric for adoption decisions and performance comparison. Once in operation, other metrics become necessary: the share of cases processed to completion without human intervention, the share of items that staff actually corrected during review, and the share of cases that fell out as exceptions.

Managing the two kinds separately makes the purpose of each clear. If the exact-match rate drops, the problem lies with the model or the form. If the exact-match rate stays the same but the correction rate rises, look at the review standards or the business rules.

Exact Match in the Korean context

Korean documents often contain notation differences in specific places. The word "Co., Ltd." in company names may appear at the front, at the back, or in parentheses. Addresses mix road-name and lot-number formats, and floor and unit notation varies. Bank names are used in both their formal and short forms, so one document may show the formal name while another shows the short one.

If all these differences are counted as errors, the exact-match rate comes out lower than the real performance. Conversely, if all of them are allowed, differences that matter for the work get through. It is necessary to work with business staff to decide which notation differences count as the same value in the work.

Frequently asked questions

Should I look at exact match or similarity-based accuracy?

It is better to look at both. For work where values are entered into the system as they are, use the exact-match rate as the primary standard, and refer to similarity-based accuracy when comparing recognition quality.

What percentage should the target be?

It depends on the work. In one financial document validation, the exact-match target was set lower than the similarity standard. Calculate the impact of one error together with the validation rules that follow, and set the target accordingly.

Are hyphen or spacing differences counted as errors?

It depends on the normalization rules. Notation differences that the work treats as the same value should be normalized before comparison.

How are items with no value in the original counted?

Classify them separately as undeterminable. Counting items that a person also cannot judge as wrong measures document quality, not product performance.

If the exact-match rate is high, does the work go down automatically?

Not necessarily. The share of cases processed to completion without human intervention must also be checked to see the actual savings.

We will send you a measurement sheet and a sample answer set. If you send one document type, we will also run the actual measurement.

Contact us