What RAG preprocessing is: a complete guide to the document preparation that decides answer quality

RAG preprocessing analyses and organises the documents used for retrieval-augmented generation so that structure and meaning survive, building them into searchable chunks, and it is where the quality of the material the model will ground on is decided.

Why RAG preprocessing is needed

Changing the model often does not improve the answers.

A recurring account from institutions that have adopted generative AI: a good model was connected, yet business questions come back with answers that miss. Trace the cause and it frequently sits not with the model but with the material given to it. Administrative documents, manuals, and statutes had accumulated as PDFs and Korean word processor files, and those files were pulled straight into text and loaded in.

The trouble is that structure vanishes the moment only text is extracted. Unfold a multi-level table splitting national and provincial funds into balance and interest into a run of sentences, and there is no telling which account applies under which condition. The model then produces plausible prose on top of that disordered material. That is usually where poor answer quality originates.

What RAG preprocessing actually is: how it differs from text extraction

Text extraction and RAG preprocessing across three axes

First, the target differs. Text extraction works on the characters written in a document. RAG preprocessing works on those characters together with the hierarchy of headings and body, the rows and columns of tables, the relationship between figures and their captions, and the link between footnotes and the text. It is not reading the document but carrying its frame across as well.

Second, the objective differs. Text extraction aims to pull out the content completely. RAG preprocessing aims to arrange the material so that exactly the right passage is retrieved when a question arrives later. The same document is therefore divided differently depending on what questions it will face.

Third, the metrics differ. Text extraction is measured on character accuracy. RAG preprocessing is measured on the accuracy of the final answer. One tax administration validation built an evaluation set of 33 documents and 361 questions to confirm structuring and retrieval quality together. Because the preprocessing stage cannot easily be scored in isolation, preparing question and answer pairs in advance is effectively mandatory.

The two are not alternatives. Inaccurate character recognition means the wrong content gets retrieved however well it was divided. Structure restoration goes on top of recognition, and chunking on top of that.

Six conditions that protect answer quality

Six conditions that hold in a real build

First, convert into a form that preserves structure. Retrieval can only use headings, sections, tables, lists, and figure captions if a marker survives saying what each one is. Tagging structural elements and producing both a markup form and structured data widens the options downstream.

Second, align chunk boundaries with the document's logical units. Cutting by character count alone breaks tables mid-way and splits clauses in two. Take sections or tables as the primary boundary and treat length as a secondary condition.

Third, handle tables separately. A table behaves differently from prose, and slicing it the same way makes it hard to retrieve. Keeping a whole table as one chunk while attaching the section it belongs to is the practical arrangement.

Fourth, carry metadata alongside. Document title, issuing body, effective date, revision history, and source page number have to travel with each chunk for an answer to cite its grounding. An answer that cannot cite grounding does not get used in the business.

Fifth, de-identify personal data. Moving public data into retrieval use makes masking or replacing identifying fields close to a precondition. This has to happen before chunks are built, or the original values remain in the search index.

Sixth, build the evaluation set first. Judging whether a change to the preprocessing helped or hurt requires a fixed set of questions to compare on. Constructing an evaluation set of a few hundred questions early makes every subsequent improvement measurable.

How RAG preprocessing is applied in practice

Classify the document estate first

The opening task is not conversion but understanding the character of the target documents. Statutes and notices with a clear clause structure, manuals where sections and subsections form a hierarchy, official correspondence that is short and self-contained, and filing forms where the table is the substance all need dividing differently.

One municipal deployment structured its administrative documents, separated text, tables, and images, and produced markup, structured data, chunks, and keywords together for loading into a knowledge store. Converting once and producing several artefacts means the source never has to be reprocessed when the retrieval approach changes.

Separate structuring from chunking

Combining structuring and chunking into one process means reanalysing the document every time chunk size is adjusted. Split into two stages, the structured result stays put while only the chunking rules change.

Real configurations convert the document into structured data, split text at a set length, and order the results. Overlapping consecutive chunks slightly reduces the problem of sentences being cut at a boundary and losing their meaning.

Keep the source and the chunk connected

Building an answer from retrieved chunks alone leaves the user no way to verify the grounding. Storing the source file, page number, and in-document position with each chunk allows an answer to carry its citation.

This connection matters particularly in the business. An administrator looking up policy information needs not only the answer text but which page of which guideline it came from. Only when the source can be opened and checked does an answer become something to attach to an approval.

RAG preprocessing in the Korean environment

A large share of Korean public document estates sits in Korean word processor format. Whether that format can be processed natively is the first gate; merge information in tables lost during conversion cannot be recovered later.

The documents themselves carry distinctive conditions. Official correspondence, manuals, statutes, and reports coexist in one store, with the multi-level tables, circled line numbers, and stamps intruding into tables that are peculiar to government forms. Frequent revision is another characteristic, so managing the current version and tracking change history belong in the preprocessing requirements.

On top of this, network separation restricts calls to external embedding services. Confirm that conversion, chunk generation, and embedding can all run inside the internal network.

Frequently asked questions