What Fuzzy Similarity is: a complete guide to measuring accuracy as people perceive it

Fuzzy Similarity calculates how closely two strings resemble each other as a ratio, expressing accuracy in a way that allows for the differences in spacing and punctuation people treat as immaterial.

One character off scores zero

One character off scores zero.

Measuring document AI performance on exact match alone can distort the picture of recognition quality. A single spacing difference in a long address, or a bracket in a different position in a company name, makes the whole field wrong.

To a person the value is identical, and the score is zero. Accumulate enough such fields and you arrive at a situation where recognition is working well and the performance figure reads low. At that point there is no telling whether the thing to improve is the model or the notation normalisation. Fuzzy Similarity is the metric that makes that distinction possible.

What Fuzzy Similarity actually is: how it differs from exact match

Exact match and fuzzy similarity across three axes

First, the calculation differs. Exact match reads only correct or incorrect. Fuzzy similarity counts the differences between two strings and converts them to a ratio, so partial credit exists.

Second, what they reflect differs. Exact match treats even trivial notation differences as errors. Fuzzy similarity reflects cases a person reads as the same value and produces a figure closer to perception.

Third, where they apply differs. Exact match is used to judge whether a value can go into a system. Fuzzy similarity is used to compare recognition quality itself or to compare difficulty between document sets. In real validations, one financial document set confirmed 98 percent by fuzzy similarity on both printed and handwritten fields, and one public document set measured 98.80 percent text similarity.

The two are not alternatives. Reading only similarity hides the re-entry burden; reading only exact match misreads a normalisation problem as insufficient performance.

Four cautions in interpretation

Four cautions when reading the figure

First, the calculation can differ between products. Counting by character or by word, and which distance measure is used, changes the value. Comparison requires measurement by the same method.

Second, long strings score higher. A long field such as an address holds a high similarity even with one or two characters wrong. Reading it broken out by field length is more accurate.

Third, it does not translate directly into business impact. An account number is unusable with a single character wrong. Apply an exact match criterion alongside for numeric fields.

Fourth, it does not tell you the direction of improvement. A low similarity on its own does not say whether the problem is recognition or document condition. The fields scoring low have to be inspected directly.

How fuzzy similarity is used in practice

Place it alongside exact match

Reading the two together reveals the direction of improvement. High similarity with low exact match means notation normalisation resolves much of it. Both low means recognition quality itself needs attention.

Asking for both metrics when requesting validation material gives you a far more accurate picture of the product's state.

Use it to compare difficulty across document sets

Measuring several document sets with the same product shows which are hard. Print-heavy sets, handwriting-heavy sets, and fax-heavy sets separate.

That comparison helps when scoping adoption. A document set scoring clearly lower is grounds for deferring it to a later phase or considering a different processing approach.

Fuzzy similarity in the Korean environment

Korean documents produce notation differences at predictable points: the corporate designation in a company name, the mixture of road-name and lot-number address systems, and full legal versus abbreviated bank names.

These are differences a person reads as the same value, so they score high on similarity and low on exact match. Collecting the fields where the gap between the two figures is widest shows where normalisation rules belong.