Chunking splits documents into smaller passages before embedding and indexing, so retrieval systems can pass the most relevant piece of a page to an AI model.
What is Chunking?
Chunking is the process of splitting documents into smaller passages, called chunks, before they are embedded and stored for retrieval. When a user asks a question, the system searches these chunks and passes the most relevant ones to a language model. Chunk size and boundaries determine whether a retrieved passage contains a complete, usable answer.
The term comes from retrieval-augmented generation (RAG) engineering, where chunking is one of the first design decisions in a pipeline. Vector database vendors such as Pinecone and research teams at AI companies have published extensive guidance on it, because poor chunking is a common cause of irrelevant or incomplete answers.
How it works
flowchart TD
A[Web page or document] --> B[Extract main text]
B --> C{Split strategy}
C --> D[Fixed size with overlap]
C --> E[By headings and sections]
C --> F[By sentences or meaning]
D --> G[Embed each chunk]
E --> G
F --> G
G --> H[Index for retrieval]
Common strategies:
- Fixed size. Split every N tokens, often with some overlap so sentences at the edges are not lost. Simple but can cut ideas in half.
- Sentence or paragraph. Split at natural language boundaries.
- Structure-aware. Split at headings, list items or HTML sections, keeping each section together.
- Semantic. Split where the topic shifts, detected with embeddings.
- Contextual enrichment. Add the page title or a short summary to each chunk so it carries its own context.
From SEO to AEO and GEO
Classic SEO evaluated pages. Google did add passage ranking in 2021, which lets a specific section of a page rank for a narrow query, but the unit users saw was still a link to the page. AI answer engines work with passages more directly: a retrieved chunk may be quoted, paraphrased or used as evidence, with the page credited as a citation.
What carries over: heading hierarchy, scannable structure and answer-first sections, all good SEO practice already. What changed: every section now needs to stand on its own. A paragraph that opens with “This approach also…” loses its meaning when retrieved alone, and a comparison split across two sections may be retrieved half-complete.
What to do about it
- Use descriptive H2s and H3s. Each heading should say what the section answers; headings are natural chunk boundaries.
- Keep sections focused and fairly short. One question, one answer, supporting detail.
- Open each section with the answer, restating the subject by name.
- Avoid cross-references like “see above” for essential facts; repeat the key fact briefly.
- Keep tables compact with clear header rows, so a chunk containing part of the table still makes sense.
- Render content as text in HTML. Text inside images, tabs that load later or PDFs may never be chunked at all.
- For your own RAG or docs bot, test chunk strategies against a set of real questions and measure answer quality.
Common mistakes
- Writing for scroll, not extraction. Long narrative build-ups bury the answer mid-chunk.
- Pronoun-heavy paragraphs that lose meaning out of context.
- Assuming one fixed chunk size suits every page. Docs, FAQs and long guides behave differently.
Related terms
- Embeddings
- Vector Search
- Context Window
- Passage Ranking
- Retrieval-Augmented Generation
- Answer engine optimization guide
Sources
- Chunking strategies for LLM applications, Pinecone
- Contextual retrieval in AI systems, Anthropic
- Retrieval, OpenAI developer docs
Frequently asked questions
Why do AI systems chunk content?
Embedding a whole long page into one vector blurs its meaning, and passing whole pages to a model wastes context window space. Chunking lets the system retrieve just the passage that answers the question, which improves relevance and leaves room for several sources in one answer.
How do AI search engines chunk web pages?
They do not publish their methods. Common approaches in retrieval systems include fixed token windows with overlap, sentence or paragraph splits and structure-aware splits at headings. Because the method is unknown, the safest strategy is content that reads well when cut at any heading or paragraph boundary.
What is contextual chunking?
It means adding context to each chunk so it makes sense in isolation, for example prepending the document title or a short summary before embedding. Anthropic described a version of this, called contextual retrieval, in 2024. Writers can do the human equivalent by restating the subject in each section.
