View all glossary terms

Data and quality

Fuzzy Similarity Explained: A Similarity Metric Close to Perceived Accuracy, Complete Guide

Korea Deep LearningPublished

Fuzzy Similarity is a metric that calculates as a ratio how similar two strings are. It expresses accuracy that takes into account differences a person would consider minor, such as spacing or symbols.

Why exact match alone can hide performance

One character difference scores zero.

If document AI performance is measured only on an exact-match basis, the real recognition quality can be distorted. If one space differs in a long address, or the position of parentheses in a company name differs, that item is counted as entirely wrong.

To a person it is the same value, yet its score is zero. When many such items pile up, a situation arises where recognition is working well but the performance figure comes out low. Then you cannot tell whether what needs improvement is the model or the notation normalization. Fuzzy Similarity is the metric that makes this distinction possible.

The exact definition of Fuzzy Similarity: how it differs from exact match rate

Three axes of difference between exact match rate and similarity accuracy

First, the calculation method differs. The exact match rate only checks whether an item was right or wrong. Similarity accuracy counts the difference between two strings and converts it to a ratio, so partial scores appear.

Second, what is reflected differs. The exact match rate counts even minor notation differences as errors. Similarity accuracy reflects cases where a person would treat the values as the same, producing a figure closer to perceived accuracy.

Third, where each is used differs. The exact match rate is used to judge whether a value can be entered into a system. Similarity accuracy is used to compare recognition quality itself or to compare difficulty across document groups. In actual validation, there is a financial document group where both printed and handwritten text were confirmed high on the similarity basis, and text similarity was also measured high in one public document group.

The two metrics are not substitutes. Similarity alone misses the re-entry burden, and exact match alone mistakes a problem that normalization would solve for a lack of performance.

Four cautions when reading similarity accuracy

Four points of caution in interpretation

First, the calculation method can differ by product. The value changes depending on whether characters or words are counted and which distance calculation is used. To compare, you must measure in the same way.

Second, long strings yield high values. For long items such as addresses, even one or two wrong characters keep the similarity high. Reviewing separately by item length is more accurate.

Third, it is not tied directly to business impact. An account number is unusable with even one wrong character. Apply an exact match standard together for numeric items.

Fourth, it does not show the direction of improvement. The fact that similarity is low alone does not reveal whether the problem is recognition or the state of the document. You must look directly at the low-scoring items.

Practical use of similarity accuracy

Place it side by side with the exact match rate

Looking at both values together reveals the direction of improvement. If similarity is high but the exact match rate is low, notation normalization will solve much of the gap. If both are low, the recognition quality itself needs to be examined.

When you request validation data, asking for both metrics together lets you grasp the product's condition much more accurately.

Use it to compare document group difficulty

Measuring several document groups with the same product reveals which documents are hard. Measuring printed and handwritten text, and scanned and faxed copies, separately gives a basis for predicting the exception rate later.

Measuring similarity in a domestic environment

In Korean documents there are particular points where similarity is especially useful. Typical examples are the Korean corporate designator "Co., Ltd." (jusikhoesa) attached to company names, the mix of road-name and lot-number addresses, and the formal versus abbreviated names of banks.

These items score high on similarity but low on exact match. Organizing which notation differences the business treats as the same value, and moving them into normalization rules, narrows the gap between the two values.

Use together with normalization rules

The similarity metric is convenient in that it accounts for notation differences, but which values count as the same is decided by the business, not by the metric. So it is more accurate to set normalization rules before measuring.

Several points often trip in Korean documents. The corporate designator on a company name may come before or after the name, or be enclosed in parentheses. Addresses mix road names and lot numbers, and the notation for floor and unit numbers varies. Bank names have formal and abbreviated forms used together within one business.

If the business treats these differences as the same value, normalize them before comparison. If they must be treated as different, leave them as they are and measure. Because the rules must remain documented, the same standard applies when measuring again.

Frequently asked questions

Which metric should be the standard?

For work where values go directly into a system, use the exact match rate as the standard, and refer to similarity as a supplementary comparison of recognition quality.

Is 98% a good figure?

It depends on the document group. There is a financial document group case where both printed and handwritten text were confirmed high, but judging by values measured on your own documents is more accurate.

Can it be used for numeric items?

It is not recommended. For account numbers and amounts, one character difference is an error, so apply the exact match standard.

Are there cautions when comparing products?

The calculation method must be aligned. Unless they are measured on the same documents with the same calculation method, the comparison does not hold.

Which should we look at, similarity accuracy or exact match rate?

Looking at both is better. Similarity is the basis for comparing recognition quality, and the exact match rate is the basis for work whose purpose is system entry.

How many percent is enough?

It depends on the business. There are cases in financial document validation where both printed and handwritten text were confirmed high, but it is best to use values measured on your own documents as the basis.

Send one type of document and we will measure both the exact match rate and the similarity accuracy together.

Contact us