Chunking divides documents into smaller pieces that a search system can retrieve. The purchasing decision is not which chunk size sounds most advanced. It is whether the chosen boundaries let the assistant find a complete, attributable answer without combining unrelated instructions.
Related service: Custom AI chatbot development- 01Inspect document structures
- 02Create candidate chunks
- 03Run representative retrieval questions
- 04Review missing context
SECTION 01
Inspect what the documents actually contain
Bring examples of policy pages, product manuals, tables and scanned files to the scope discussion. A page break may cut a sentence or separate a table from its unit labels. Conversely, a long page can contain several unrelated procedures. Ask the agency to show the extracted text before deciding where it should be divided.
Microsoft's chunking documentation describes fixed, structure-based and combined approaches. Use these as options to compare, not a prescription to copy a platform default. The source title and location should travel with each piece. Restricted sections must not inherit broader access merely because they share a document with public material.
SECTION 02
Illustrative example: an installation manual
Suppose a manual has separate procedures for indoor and outdoor equipment. A fixed split leaves the heading in one piece and the mounting instructions in another. A question about outdoor placement retrieves the instruction paragraph but loses the warning that applies to that model.
A revised approach keeps the section heading with the relevant instructions and preserves the nearby warning reference. Test a second question about the indoor model to ensure the two procedures remain distinguishable. Do not solve the problem by sending the complete manual for every request; that can add irrelevant context and obscure the source of an answer.
SECTION 03
Accept retrieval behavior before judging the prose
For each test, inspect which pieces were returned and whether they contain the answer. If the correct evidence never arrives, rewriting the response prompt cannot reliably repair the missing input. Microsoft's RAG guidance treats content preparation and retrieval as important parts of grounding; evaluate those stages separately from the final wording.
Request examples covering tables, adjacent warnings, repeated headings, changed revisions and questions whose answers are absent. Compare a simple approach with a more elaborate one using the same cases. At handoff, receive chunk identifiers, source references, extraction rules and a reprocessing procedure. Agree how a document update replaces all affected pieces rather than leaving conflicting versions searchable.
SECTION 04
Frequently asked questions
Are smaller chunks always more accurate?
No. Small pieces may lose conditions or definitions, while large pieces can include distracting material. Test the question and document types you actually need.
Should chunks overlap?
Overlap can preserve context at a boundary, but too much can repeat evidence and increase processing. Compare a limited overlap with structure-aware splitting.
What should the agency demonstrate?
Show the original document, extracted pieces, retrieved results and resulting answer for the same test question. The links between those stages should be understandable.
PRIMARY REFERENCES
Sources and further reading
These references cover the standards, platforms, or published prices discussed in the guide. Worked examples and checklists are our editorial guidance.
- Chunk documents for searchMicrosoft Learn
- Retrieval-augmented generationMicrosoft Learn
EDITORIAL METHOD
About this guide
We use AI to assist with drafting and editing. Catapult AI Work is responsible for the published content. Examples illustrate possible approaches; they are not client case studies unless identified as such.
Budget examples are not Catapult package prices. Check linked provider pages for current fees and plan limits before making a purchase.
Read the editorial policy