View all glossary terms

Data and quality

What Is TEDS: The Structure Similarity Metric for Table Recognition Performance, Explained

Korea Deep LearningPublished

TEDS (Tree Edit Distance based Similarity) is a metric that converts the extracted table and the original table into tree structures, calculates how much editing is needed to turn one into the other, and expresses how closely the structure and values of the table match as a single ratio.

Why Tables Need a Separate Metric

Text accuracy does not tell you how well a table performs.

Most materials on document AI performance report text accuracy. When that number is high, it is easy to assume tables will come out well too. Actual measurement turns out differently.

In one public-document evaluation, text similarity was high but table structure similarity was much lower. Nearly all the characters were read, but the gap was in restoring which cell each character had been in.

This gap matters because of the nature of the information a table holds. A value in a table has no meaning by itself. Only once you know which row and which column it was in does it take on meaning. When the structure collapses, the values can be accurate and still be unusable.

The Exact Definition of TEDS: How It Differs from Text Accuracy

Three Axes That Separate Text Accuracy and TEDS

First, the target differs. Text accuracy targets character strings. TEDS targets the structure obtained by turning the table into a tree. The arrangement of rows and columns, the range of merged cells, and the division between header and data areas are all captured in this tree.

Second, the calculation differs. Text accuracy compares characters. TEDS compares the two trees, counts how many edits are needed to change one into the other, and converts that edit amount into a similarity score relative to the table size. So even if every cell value is correct, the score drops if the merge ranges are wrong.

Third, the problem each one exposes differs. Text accuracy reveals reading failures. TEDS reveals structure restoration failures. Looking at how the two diverge in actual evaluations, in one set of public documents, element detection and position accuracy were all high, and text was high, but TEDS alone was low. In another set of public documents, text was high while table structure restoration was at a middle level, and in a validation on tax return forms, table structure restoration was confirmed to be high.

The two metrics are not substitutes. If characters are not read, the content is empty even if the structure is restored. If the structure is not restored, correctly read values have no place to go. It is right to require both values side by side.

Five Things to Watch When Reading TEDS

Five Cautions for Interpreting TEDS

First, it is not an absolute standard. It is not a value that can be called good once it passes a certain percentage. The same product moves widely depending on the nature of the document. Only values measured on your own documents are a basis for judgment.

Second, the difficulty of the documents must be considered too. Document sets with many simple grid-type tables produce high values easily. Administrative documents where multi-level headers and merges overlap bring down even the same product substantially. Comparing scores without comparing the documents leads to wrong conclusions.

Third, the structure score and the value score can be separated. TEDS reflects both structure and cell content together, so it is not clear whether a low score comes from structure or from values. If you want to know the cause, measure together with a method that compares only the structure.

Fourth, how the table was cut determines the result. If a table that spans a page is split into two for recognition, each score can be high, yet in reality one table has been broken apart. Check this together with the table region detection results.

Fifth, it does not map directly to business impact. A low score does not mean the work fails in that proportion. To understand the real impact, check which items were misaligned and whether those items are used in the work.

How to Apply TEDS in Practice

Require It When Requesting Validation

Solution review materials often leave out table metrics. For work that handles documents where tables are the main content, explicitly require table structure similarity alongside text accuracy.

When you request it, also specify the documents to be measured. Values measured on public sample documents and values measured on your own intake documents come out differently. To make a meaningful comparison, include real forms with merged cells and multi-level headers.

Narrow Down the Cause of a Low Value

If TEDS comes out lower than expected, split the cause into three directions: whether the table region itself was detected wrongly, whether the boundaries between rows and columns were misjudged, or whether merge ranges were missed.

Each calls for a different response. Region detection problems are often traced to layout analysis at the front end, boundary misjudgments to preprocessing for documents with faint or missing lines, and merge problems to the output format. If results are received in a format that cannot express merge information, the model may identify the structure correctly, yet that information still disappears from the output.

TEDS in the Korean Environment

Korean government forms and tax returns have particularly complex table structures. Headers stack two or three levels deep, a value in one column represents several rows below it, and guidance sentences on how to fill in the form sit inside the table. Circled-number item numbers are sometimes mixed in with the values.

On top of this, the condition of real received documents adds to the difficulty. Stamps invade tables, strikethroughs cut across rows, and resent faxes blur the lines. Values measured under these conditions normally come out lower than values measured on clean documents. Rather than hiding a low value, it is more helpful for judgment to state which documents it was measured on.

Frequently asked questions

What percentage of TEDS is enough?

It depends on the document. There are cases of high values on tax return forms, and cases of much lower values on administrative documents with overlapping multi-level headers. Using the value measured on your own documents as the standard is advisable.

Why is text accuracy high but TEDS low?

The two metrics measure different things. Even if characters are read correctly, if you cannot restore which cell they were in, TEDS drops.

Can all values be correct and still get a low score?

Yes. TEDS reflects structure too, so if merge ranges or the arrangement of rows and columns are off, the score drops even when every cell value is correct.

Should a product be ruled out if its score is low?

It is hard to conclude that. You need to check which items were misaligned and whether those items are used in the work to understand the actual impact. The difficulty of the documents should also be considered.

How should a product that does not provide a table metric be judged?

Request direct measurement on your own documents. In work where the documents are mainly tables, if this value is missing, the performance has not been confirmed.

Send us one document type with many tables, and we will return the structure reconstruction results together with the measured values.

Contact us