Chunking: how AI splits your page into usable pieces

Before an AI assistant can quote your page, a retrieval system cuts it into chunks: blocks of a few hundred tokens that are stored and scored one by one. The model never reads your page as a whole; it reads a shortlist of chunks that matched the query. If the winning fact sits in a chunk that never names your brand or its subject, the fact gets used and the credit does not.

What chunking is and why every retrieval system does it

Chunking is the step where a retrieval system splits a document into smaller passages before indexing them. It exists for a mechanical reason: language models have finite context windows, and retrieval has to compare a query against millions of candidate passages at speed. So the practical unit of AI search is the passage, not the page.

That unit choice runs through the whole stack. When an assistant answers, it first breaks the question into sub-queries, the mechanism we covered in query fan-out, then retrieves the best-scoring chunks from many pages and writes one answer out of the pieces. Your page competes chunk by chunk, never as a whole.

Nor is this exclusive to chatbots. Google announced passage ranking for classic search in 2020, saying it would affect about 7 percent of queries when fully rolled out, because the sentence that answers a question "might be buried deep in a web page". Google's AI Mode goes further and retrieves at passage level by design. We cover the search side separately in passage ranking.

The three ways a page gets cut

Different systems cut differently, and you never learn which chunker read your page. Three families cover most of what runs in production:

• Heading-based splitting. The page is cut at H2 and H3 boundaries, so your heading hierarchy literally decides where the knife falls. Clean, descriptive headings produce clean, self-labelled chunks.
• Fixed windows. The text is cut every few hundred tokens, often with an overlap, regardless of structure. A window can open mid-argument and close mid-sentence, which punishes long unbroken passages.
• Semantic breaks. Embeddings track where the meaning shifts, and the cut lands at the topic change. Sections that mix three ideas produce muddled boundaries and muddled chunks.

The practical conclusion: since you cannot choose your chunker, write pages that survive all three. A section that starts at a descriptive heading, runs 100 to 200 words and handles exactly one idea comes out nearly identical under every method.

A chunk must carry its own context

Here is the most common failure we see in audits at Neurise. The introduction names the company and the product; every later section says "it", "the platform" or "our approach". Read top to bottom by a human, the page flows. Cut into chunks, section five contains no entity at all: no brand, no product, no subject. The retriever cannot match that chunk to a branded query, and a model that borrows the fact has nothing to attribute it to.

The size of this effect has been measured. Anthropic's engineering post on contextual retrieval, published in September 2024, showed that prepending a short context sentence to every chunk before indexing cut top-20 retrieval failures by 49 percent. That is how much performance was recovered purely by restoring the context that chunking had removed. The systems reading your site will mostly not do that repair work for you, so the fix belongs in the copy itself.

The writing rule that follows is cheap: every H2 section should name its subject, and where relevant the brand, once, in a natural sentence. Not a keyword forced into every line; one honest restatement, the way this section keeps saying "chunk" instead of "it".

A chunk that needs the rest of the page to make sense will never be quoted on its own merits.

Tables and lists are chunk-friendly formats

Some formats survive cutting better than prose, because they bind labels to values inside the block itself:

• Lists keep facts atomic. Each bullet is a self-contained claim, so wherever the cut lands, whole facts survive rather than halves of an argument.
• Tables pair every value with a header. A row reading "Starter plan, 49 euros, 5 users" still means something outside the page; a paragraph where the price arrives three sentences after the plan name does not.
• Question-shaped headings with the answer in the first two sentences hand heading-based chunkers a perfect unit: the question and its answer inside one block.
• Definition blocks, spec sheets and numbered steps do the same job. The relation lives inside the chunk, not in the surrounding text.

Markup can reinforce what the copy does. Structured data makes entities and their relations machine-readable at page level, and it works alongside chunk-level clarity, not instead of it.

The chunk-killers

Four habits reliably destroy otherwise good content at the chunking stage:

1. Pronouns pointing at earlier sections. "As mentioned above", "this approach", "it also": after the cut there is no above. Every backwards reference becomes a dangling pointer.
2. Walls of text. A 300-word paragraph mixing three ideas gets sliced mid-idea by fixed windows and gets a muddy embedding under semantic splitting. Short paragraphs, one idea each.
3. Teaser headings. A heading like "The plot thickens" carries no query terms, so the chunk beneath it inherits a label that matches nothing anyone asks.
4. Split facts. The number in one paragraph, its subject in another. If a figure and what it measures end up in different chunks, both chunks lose.

How to check your own pages

The test is simple and slightly brutal. Take any H2 section of a page that matters commercially, cut it out and hand it to someone who has never seen your site. Can they tell who it is about, which product, and what exactly is being claimed? If not, that section is invisible to retrieval, whatever position the page holds. This section-level discipline is a core part of Generative Engine Optimization, the practice we describe across our GEO service.

Start with the pages already earning you visibility in AI answers; where AI citations come from explains how to find them. Then check that what each chunk says is what you want repeated, because assistants verify claims against the retrieved text, a process we unpack in grounding in AI search. If you would rather have the whole pass done for you, chunk-level review is part of our free SEO and GEO audit.

Common questions

What is a chunk in RAG?

A chunk is the block of text, usually a few hundred tokens, that a retrieval system stores and scores instead of the whole page. AI assistants assemble answers from chunks, not from documents.

How do retrieval systems decide where to cut a page?

Three common methods: at heading boundaries, at fixed token windows with overlap, or at semantic breaks where the topic shifts. You cannot control which method reads your page, so structure sections to survive all three.

Why should every section repeat the brand and subject?

Because after chunking, a section that only says "it" or "the platform" carries no entity for the retriever to match or for the model to credit. One natural mention per section keeps each fact attached to your name.

Does chunking matter for Google search or only for chatbots?

Both. Google has ranked individual passages since 2021 and its AI Mode retrieves at passage level, so section-level clarity pays off in classic results and AI answers alike.

Neurise editorial team · published 1 September 2026