What layout analysis is: a complete guide to the processing that decides reading order

Layout analysis identifies where headings, body text, tables, figures, and headers sit on a page of a document, and orders those regions the way a person would read them.

Why the sentences come out scrambled

The sentences come out scrambled.

You will have seen text extracted from a document arrive in disarray. Sentences from the left and right columns of a two-column report alternating, or values from inside a table wedged between sentences of body text.

The cause is usually reading order. Scanning a page top to bottom and left to right collapses immediately on a document with columns or a table. A person sees the headings, recognises the columns, and treats the table as separate. Without that judgment, however accurately the characters are read, the result is unusable. Layout analysis is the processing that makes that judgment.

What layout analysis actually is: how it differs from character recognition

Character recognition and layout analysis across three axes

First, the target differs. Character recognition works on characters. Layout analysis works on the regions those characters form and the arrangement of those regions.

Second, the objective differs. Character recognition aims to transcribe accurately. Layout analysis aims to establish what sits where and in what order it should be read.

Third, the metrics differ. Character recognition is judged on character accuracy. Layout analysis is judged on whether every region was found and whether each region's position is correct. In one public-sector evaluation, all 302 assessed elements were detected for 100 percent detection accuracy with 100 percent positional accuracy.

The two are not alternatives. Order has to be established for recognition output to carry meaning, and recognition has to happen for the regions to be filled. In a working pipeline, layout analysis comes first and character recognition sits on top of it.

Four judgments in the processing

Four judgments made during processing

First, identify the type of each region. Heading, body text, table, figure, or page header. Characters of the same size take on different roles depending on position and surrounding relationships.

Second, establish region boundaries. Where one table ends and the next paragraph begins. Difficulty rises on tables without ruling lines and paragraphs separated only by whitespace.

Third, establish reading order. Multi-column layouts, inserted figures, and footnotes together produce an order that even a person has to pause over.

Fourth, establish hierarchy. Determining which body text belongs under which section heading raises the usefulness of the output considerably for later retrieval and structuring.

How layout analysis is applied in practice

Place it at the front and measure it separately

Layout analysis sits early in the pipeline, and a region missed here is not recovered downstream. Its performance therefore has to be measured separately from character recognition.

When measuring, separate how many regions were found from how accurate their positions are. Strong detection with weak restoration inside tables appears often in practice, so do not collapse the two into one number.

Set different expectations by document type

Region identification is straightforward on simple single-column documents and produces high figures. Multi-column reports and forms with tables inside tables pull the same product down.

Compose the validation set with documents of varying difficulty. Figures measured on simple documents alone will not be reproduced after adoption.

Layout analysis in the Korean environment

Korean government forms and filing documents carry distinctive structures: tables with multi-level headers, line numbers written as circled figures, and completion instructions printed inside the table. Failing to distinguish instructional text from body content contaminates the structured output.

The condition of real intake documents is a factor too. A black band from a page folded during scanning, margins lost to repeated photocopying, and headers stacked by fax retransmission all make region identification harder. Real performance is only confirmed once such documents are in the validation set.

Frequently asked questions