Technology and architecture
What Is Table Recognition: How to Turn the Numbers in a Table into Business Data
Korea Deep LearningPublished
Table Recognition is processing that, for a table drawn in a document, determines the boundaries of its rows and columns and the range of its merged cells, then restores which item each value belongs to. The result is structured data that a machine can use as is.
Why Table Recognition Is Needed
The text reads correctly, but the numbers do not line up.
Technology for extracting text from documents has existed for a long time. For printed documents, character-level accuracy is not hard to achieve, and there is a public-document evaluation that recorded high text similarity. In the same evaluation, however, table structure similarity was much lower. Nearly all the characters were read, but the gap was large in restoring which cell each character had been in.
The problem is what this gap does to the work. A value-added tax return puts major category, middle category, classification, item number, amount, tax rate and tax amount into one table. Even if the figure 5,000,000 is read exactly, it cannot be used unless it is linked to whether it is the tax amount for tax invoices issued or for tax invoices issued by the buyer. Worse, the person in charge must reopen the original to check, which adds work. Table recognition is not a problem of reading values. It is a problem of restoring where each value belongs.
The Exact Definition of Table Recognition: How It Differs from OCR
Three Axes That Separate OCR and Table Recognition
First, the target differs. OCR targets the characters in an image. Table recognition targets, in addition to characters, the cell boundaries, the merge ranges, the division between header and data areas, and the coordinate relationships of each value. Many tables have no drawn lines, so you must determine boundaries implied by margins and alignment, not only the visible rules.
Second, the goal differs. The goal of OCR is to transcribe the written characters completely. The goal of table recognition is to produce data with the same structure as the original table. Given the same table, OCR lines up the text inside each cell in sequence, while table recognition restores a grid with row and column coordinates and fills each cell with its value.
Third, the performance metrics differ. OCR is measured by character-level accuracy or similarity-based accuracy. Table recognition is measured by table structure similarity (TEDS). This metric converts an extracted table into a tree and measures how closely it matches the tree of the original table. So even if every cell value is correct, the score drops if the merge ranges are wrong. This is why deciding adoption on text accuracy alone can produce unexpected results for table-heavy documents.
The two technologies are not substitutes. Characters that OCR fails to read cannot be filled in by table recognition, and when table recognition fails, even correctly read characters have no place to go. The right order is to layer structure restoration on top of character recognition.
What Table Recognition Requires for Business Use
Six Conditions for Running Table Recognition in Production
First, the range of merged cells must be identified accurately. A cell merged in both the row and column directions makes one value represent several rows. Take a local government document on the return procedure for expenditure funds as an example. The value "direct" in the classification column covers all three rows below it, national funds and provincial funds each split into balance and interest beneath them, and beneath those, revenue accounts and expenditure accounts split again. If the merge range is wrong, the return account for a commissioned project gets attached to an account of a direct provincial project.
Second, the item path that each value belongs to must be restored under multi-level headers. Government forms and tax returns often stack their headers two or three levels deep. To extract a value, you need the entire path from the top header down, not just one column name. In a comprehensive income tax return, the value 39,460,000 only has a definite meaning after passing through three levels: determined tax, comprehensive taxation, and comprehensive income tax.
Third, the elements that cover the table must be handled. Real received documents have stamps placed over tables, strikethrough lines across rows, and watermarks overlapping the content. A person reads the number under a stamp naturally, but for a model that judges structure, the stamp is an obstacle that erases cell boundaries. If one stamp shifts the row divisions of the whole table, the item links for every value below it slide out of place.
Fourth, the table must be connected to information outside it. Amount units are often written in the table title as thousand won, exceptions are attached by footnote symbols, and verification formulas for totals are described in sentences below the table. If you cut out the table alone, this information is lost, and values can end up off by a factor of a thousand.
Fifth, performance must be measured with structure metrics. If the only figure you receive at the adoption review stage is character accuracy, table performance has not been confirmed. Request table structure similarity separately, and obtain values measured on your own documents. Even in public-document evaluations, the level of table structure restoration varies by case, which shows that this metric swings widely depending on document type.
Sixth, each value's source position must be returned with it. If the position a value came from in the original cannot be confirmed, a reviewer must reread the entire table just to check one suspicious value. When position information comes along, the reviewer can check just the relevant cell on screen.
How to Apply Table Recognition in Practice
Decide the Output Format First
The first decision in a project is not the choice of model but the output format. Whether tables are received as HTML, Markdown, or a JSON schema determines everything that follows.
For tables with merged cells, HTML is the safer choice. Row and column merges can be expressed as attributes, so almost no structure is lost. Markdown is easy for people to read and easy to feed into a language model, but it has no way to express merges, so you must either expand merged cells into repeated values or give up the structure. If the documents are for search and generation, creating both formats is a practical choice. If the goal is entering values into a business system, JSON that uses the item path as the key is the better option from the start.
Build on Layout Analysis, Not Character Recognition Alone
Table recognition does not work as an independent function. It operates on the results of layout analysis. The order is to first detect the table region on the page, then determine the row and column structure inside it, and then recognize the characters in each cell. So the accuracy of the earlier element detection step sets the ceiling on table performance. In one case, every element was detected and its position was accurate, yet table structure similarity was still low. This shows that finding a table region and restoring the inside of the table are problems of different difficulty.
From a technical standpoint, methods that specify coordinates by rule work only for tables with fixed layouts. Models that interpret the layout and context of a document together make a difference on tables without rules or with complex merges. This is why table item scores differ so widely even among top models on international document parsing benchmarks.
Check the Structure on the Review Screen
Table recognition is hard to judge without looking at the results. On a screen that shows only a list of cell values, you cannot tell whether merges were detected correctly or whether header paths are right.
So the review environment needs the original image and the restored table side by side, with the corresponding source cell shown when a value is selected. Showing the header path alongside lets you confirm at a glance which item a value belongs to. Once reviewers can point at only the suspicious cells instead of rereading the whole table, exception-driven operation becomes possible even for documents with many tables.
During operation, collect the corrections that reviewers make. If the same cell in the same form is corrected repeatedly, that is the basis for adding a processing rule for that form.
Domestic Conditions for Table Recognition
Domestic documents have several conditions that overseas solutions do not prepare for.
First is the document format. Public institutions and many companies use Hangul word processor documents, and tables in this format behave differently in cell properties and margin handling than in other formats. If the merge information is lost during format conversion, the later steps cannot recover it, so you must check whether the format is supported from the document conversion step onward.
Next is the structure of the forms themselves. Government forms and tax returns contain item numbers, circled-number symbols, and calculation guidance sentences within a single table. It is common for a cell to be guidance on how to fill in the form rather than data. If such a cell is extracted as a value, the business data becomes contaminated, so a judgment that separates guidance text from input values is also required.
The last is the condition of the documents. Low-resolution fax scans, images taken on the floor that show wrinkles and shadows, and documents that were resent several times, so their headers overlap, all arrive in real intake trays. Under these conditions, rules are broken or smeared, so a method that only follows lines cannot restore the table.
On top of this, finance and public organizations are often subject to network separation rules that restrict external API calls. Whether the entire document process, including table recognition, can run inside the internal network becomes the first gate in the technical review.
Frequently asked questions
If text recognition accuracy is high, do tables come out well too?
No. In the same evaluation, there was a case where text similarity was high but table structure similarity was much lower. The two metrics measure different things, so table performance must be checked separately.
What percentage of table structure similarity is enough?
It depends on the document. There are cases where the level of table structure restoration differed greatly between simple public documents, administrative documents with multi-level headers and merges, and tax return forms. Using the value measured on your own documents as the standard is more accurate.
Can tables with many merged cells be processed?
Yes, but the output format must be decided together. If the results are not received in a format that can express merge information, the structure the model identifies correctly will still disappear from the output.
What happens if a stamp is placed over a table?
It can be processed. A stamp is an element that covers cell boundaries, so it affects structure recognition more than character recognition. When verifying with real received documents, it is important to include such cases.
Are tables in Hangul word processor documents supported?
Yes. This format handles table properties differently, and its structure is easily lost during the conversion step. When reviewing adoption, check whether it can be processed in the original format.
Are tables without drawn lines recognized?
Yes. Even without visible lines, the boundaries implied by margins and alignment are determined. However, these tables are harder, so be sure to include them in the validation documents.
Can it run in a network-separated environment?
Yes. With an on-premise deployment installed on a server inside the customer's network, it runs without any external communication.
Send us a single document type with many tables, and we will return the structure reconstruction results as they are.
Contact us



