What unstructured data is: a complete guide to the data you have and cannot use

Unstructured data is data a person can read but a system cannot handle directly — documents, images, scans, and faxes that are not organised into rows and columns and cannot be loaded into a database as they are.

The data exists and cannot be used

The data exists and cannot be used.

Most of what companies and institutions hold is unstructured. Contracts, applications, supporting certificates, official correspondence, manuals, and minutes accumulate on file servers and in document management systems. By volume, no organization is short of data.

The problem is that this data cannot be queried. Checking which contract carries which condition means a person opening files and reading. Adopting generative AI does not change the situation much, because the material is not in a form the model can be given. Whether the goal is automation or retrieval, the starting point sits at reading this data.

What unstructured data actually is: how it differs from structured data

Structured and unstructured data across three axes

First, the shape differs. Structured data holds fields and values inside a predefined structure. Unstructured data has no such structure. The same business registration number appears inside a table in one document, inside a sentence in another, and beside a stamp in a third.

Second, the handling differs. Structured data is queried. Unstructured data becomes queryable only after a stage of reading, identifying, and imposing structure. That stage is the territory of document AI.

Third, how quality is measured differs. Structured data is read on omissions and duplicates. Unstructured data has to be read on recognition accuracy and structure restoration accuracy together, because accurately read characters bound to the wrong field are still unusable.

The two are not in opposition. The purpose of processing unstructured data is ultimately to turn it into structured data and put it into existing systems.

Five characteristics of real intake documents

Five characteristics that appear in real intake

First, formats vary. Not only PDFs and images but Korean word processor files, office documents, and spreadsheets arrive mixed within one process.

Second, condition is uneven. Low-quality scans received by fax, images photographed on a desk with creases and shadows, and documents with headers stacked by repeated retransmission all coexist.

Third, obstructions abound. Stamps intrude into tables, strikethroughs cross characters, watermarks overlap. Natural to a person, these are a source of misreading for a machine.

Fourth, forms keep changing. The same application is laid out differently by branch and by year, and government forms are revised on a cycle.

Fifth, one case is made of several documents. An average case arrives with six document types, and processing finishes only once the information across them has been reconciled.

How unstructured data is handled in practice

Separate reading from structuring in the design

Combining the work of reading characters and the work of imposing structure into one process means the cause cannot be narrowed when something goes wrong. Identify regions on the page and establish reading order first, then recognise the content of each region.

That split also clarifies the direction of improvement. A missing table is a structure identification problem; a table found with wrong values is a recognition problem.

Organise the document inventory with its intake paths

Evaluating adoption requires more than a list of forms. Record which path each form arrives by. A form arriving by fax and the same form scanned at a branch differ in processing difficulty.

That inventory also makes it easier to decide which document set to automate first. Starting with the paths in better condition and widening the scope is the realistic approach.

Unstructured data in the Korean environment

Korean business documents stack several distinctive conditions. Korean word processor format is widely used, government forms carry multi-level tables and circled line numbers, and fax intake persists.

When composing a validation set, therefore, use documents from your actual intake queue rather than clean samples. Figures measured on published sample documents differ from figures measured on real ones.

Frequently asked questions