What TEDS is: a complete guide to the structure similarity metric for table recognition

TEDS (Tree Edit Distance based Similarity) converts an extracted table and the original into tree structures, calculates the amount of editing needed to turn one into the other, and expresses how closely structure and values agree as a single ratio.

Why tables need a metric of their own

Character accuracy tells you nothing about table performance.

Most material presented when evaluating document AI quotes text accuracy, and it is easy to assume that a high figure there means tables will come out well too. Actual measurement says otherwise.

In one public-sector evaluation, text similarity stood at 98.80 percent while table structure similarity came in at 79.18 percent — a gap of almost twenty points. The characters were nearly all read; the divergence appeared in restoring which cell they had occupied.

That gap matters because of what a table carries. A value in a table means nothing on its own. It acquires meaning only once its row and column are established. When the structure collapses, an accurate value is still unusable.

What TEDS actually is: how it differs from text accuracy

Text accuracy and TEDS across three axes

First, the target differs. Text accuracy works on strings. TEDS works on the table converted into a tree, which carries the row and column arrangement, the span of merged cells, and the division between header and data regions.

Second, the calculation differs. Text accuracy matches characters. TEDS compares two trees, counts the edits needed to turn one into the other, and normalises that edit distance by table size to express a similarity. Every cell value can be correct and the score still falls if a merge span is wrong.

Third, what they expose differs. Text accuracy exposes reading failures. TEDS exposes structure restoration failures. Looking at how the two diverge in real evaluations: on one public document set, element detection and positional accuracy both reached 100 percent with text at 98.80 percent while TEDS stood at 79.18 percent. On another, body text reached 99.2 percent with table structure restoration at 86 percent, and validation against filing forms confirmed table structure restoration above 99 percent.

The two are not alternatives. Without reading the characters, a restored structure is empty; without restoring the structure, accurately read values lose their place. Ask for both side by side.

Five cautions when reading TEDS

Five cautions in interpretation

First, it is not an absolute standard. There is no figure above which the result is simply good. The same product moves between 79 percent and above 99 percent depending on the character of the documents. Only a figure measured on your own documents is a basis for judgment.

Second, read the difficulty of the documents alongside. A set full of simple grid tables produces high figures easily. Administrative documents where multi-level headers and merges overlap pull the same product down sharply. Comparing scores without comparing documents leads to a wrong conclusion.

Third, structure and value scores can be separated. TEDS reflects structure and cell content together, so a low score does not say which caused it. To find the cause, measure alongside a method that compares structure only.

Fourth, how the table was cut out changes the result. A table spanning a page recognised as two produces high scores for each while the reality is one table split in half. Confirm it against the table region detection result.

Fifth, it does not translate directly into business impact. A figure of 79 percent does not mean 21 percent of the work fails. Which fields diverged, and whether those fields are used in the business, has to be checked separately before the real impact is known.

How TEDS is used in practice

Ask for it when requesting validation

Table metrics are frequently missing from solution evaluation material. If your documents are ones where the table is the substance, ask explicitly for table structure similarity alongside text accuracy.

Specify the documents to be measured as well. A figure measured on published sample documents differs from one measured on your own intake queue. The comparison is only meaningful when real forms with merged cells and multi-level headers are included.

Narrow down the cause of a low figure

When TEDS comes in lower than expected, look in three directions. Was the table region itself misidentified, were row and column boundaries misjudged, or was a merge span missed?

Each calls for a different response. Region detection problems trace back to layout analysis, boundary problems to preprocessing of documents with faint or absent ruling lines, and merge problems often to the output format. If results are being received in a format that cannot express merges, the information disappears from the deliverable even when the model identified the structure correctly.

TEDS in the Korean environment

Korean government forms and tax returns have particularly complex table structures. Headers stack two or three deep, one column value represents several rows below it, and completion instructions sit inside the table. Line numbers appear as circled figures and mix with values.

The condition of real intake documents comes on top. Stamps intrude into tables, strikethroughs cross rows, and fax retransmission thins the ruling lines. A figure measured under those conditions is normally lower than one measured on clean documents. Rather than hiding a low figure, stating which documents it was measured on is more useful for the decision.

Frequently asked questions