View all glossary terms

Adoption and operations

What Is Chunking? The Criteria for Cutting Documents into Search Units, Explained Completely

Korea Deep LearningPublished

Chunking is the work of splitting a long document into pieces of a size suited for search and generation. Where the cuts fall determines whether the passage needed for a later question can be found.

Why Where You Cut Changes Answer Quality

If a sentence is cut in the middle, its meaning is lost.

To use a document for search, you cannot feed it in whole, so it has to be split. The simplest method is to count characters and cut at a fixed length. It is easy to implement, and the pieces come out evenly sized.

The problem is that documents are not structured by character count. When a table breaks in the middle, one piece has the header and another has only values. When a clause splits across two pieces, the condition and the exception get separated. If search retrieves only one of those pieces, the model builds its answer from half the information. Often the cause of poor answer quality is not the model but this cutting step.

The Precise Definition of Chunking: How It Differs from Simple Splitting

Three Axes Along Which Length Splitting and Structure-Based Chunking Differ

First, the basis differs. Length splitting cuts by character count. Structure-based chunking cuts at logical units of the document, such as articles, clauses and tables, and treats length as a secondary condition.

Second, the results differ. Length splitting gives pieces of even size but breaks up the content. Structure-based chunking gives pieces of uneven size, but each piece carries one complete meaning.

Third, the prerequisite work differs. Length splitting needs only text. Structure-based chunking requires the document's structure to be analyzed first. That means the quality of chunking depends on the quality of the structuring step that comes before it.

The two methods are not alternatives to each other. A practical combination is to split by structural unit first, then split only the overly long pieces by length.

Five Decisions to Make When Building Chunks

Five Decisions Built into the Design

First, the boundary criterion. Decide by document type whether the unit is the section or the clause, and whether a table is kept whole.

Second, the size range. If pieces are too small they lack context, and if they are too large they mix in unrelated content. In practical setups, splitting at a few thousand characters is commonly used.

Third, whether to overlap. Letting neighboring pieces overlap slightly helps restore the meaning of a sentence cut at a boundary.

Fourth, what information to carry. Attaching the document title, section title, page number and effective date to each piece improves both search accuracy and the ability to show sources.

Fifth, how to handle tables. A table differs in nature from running text, and if left as is it sometimes cannot be found by search. Storing a summary sentence alongside it is an effective approach.

How to Apply Chunking in Practice

Set Rules by Document Type

Clause-structured documents such as statutes and notices, manuals whose sections and subsections form a hierarchy, short standalone memos, and forms in which the table is the main content each call for a different basis.

Applying a single rule to everything causes problems somewhere. It is better to classify the document type first and then apply rules for each type.

Keep the Structuring Result and Redo Only the Chunks

Chunking rules keep being adjusted during operation. If structuring and splitting are bundled into one process, the document must be analyzed again from the beginning each time a rule changes.

Separating the two steps and keeping the structuring result lets you experiment quickly by changing only the chunking rules.

Chunking in the Domestic Setting

Domestic public documents often have multi-level tables, circled-number item markers, and schedules and forms inserted between the body text. Treating these elements as sentences and cutting them produces messy chunks.

Frequent revisions are also a constraint. Attaching an effective date to each chunk makes it possible to remove earlier chunks from search automatically when a revised document comes in.

Align Chunk Boundaries with the Document Structure

If character count is set first as the basis for splitting, tables and clauses break in the middle. The safer order is to set logical units such as sections and subsections as the boundaries first, then adjust the length within them.

In practice, three things are decided together. First, decide which units must not be cut; one table and one clause are better kept whole. Next, decide how much neighboring pieces overlap, to reduce the problem of a sentence cut at a boundary and losing its meaning. Last, decide what information to attach to each piece. If the title of the section it belongs to and the original page number are attached, the retrieved piece alone can show its context.

One public-institution build divided text into fixed-length units and then sorted the results. But that length must be adjusted to the nature of the document, and for material with many tables, prioritizing structure over length gives better results.

Frequently asked questions

How many characters should a chunk be?

It depends on the nature of the document. Boundaries matter more than length, so decide the logical units first and adjust the length within them.

Should chunks overlap?

Overlap reduces sentences being cut at boundaries. However, large overlaps increase storage and duplicate search results, so find a reasonable middle ground.

How should tables be split?

Keep the table as one chunk and attach the information about the section it belongs to. Add a summary sentence if needed.

Do I have to reprocess everything when the rules change?

If you kept the structuring result, you only need to rebuild the chunks. It is best to design the two steps separately.

How can I tell whether chunking is wrong?

Read only the chunks that come back from a search and check whether a person could answer the question from them. If many chunks cannot be understood without looking at the surrounding context, the criteria need to be reconsidered.

How should a document containing a table be split?

Because the value of a table is lost once the relationship between rows and columns is broken, keeping the whole table as a single chunk is the safer choice. If the table is too long and must be divided, repeat the header row in each chunk.

Send us one document you plan to search, and we will send back the chunks it produces as they are.

Contact us