RAG Text Chunker

Split text into overlapping chunks by characters, tokens (o200k_base) or sentences, see each chunk’s size and export the result as JSON. Runs in your browser.

Private by design: everything you enter is processed in your browser. It is not sent to our servers.

    How to use

    1. Paste the text you want to split.
    2. Choose the unit: characters, tokens (exact o200k_base count) or sentences, then set the chunk size and the overlap in that unit.
    3. Review the chunk list: each shows its index, character count and token count (exact in token mode, otherwise an estimate).
    4. Download or copy the result as JSON, or copy a share link to keep the settings. Long texts are not put in share links.

    Worked example

    To see the overlap clearly, split the ten letters “abcdefghij” into chunks of 4 characters with an overlap of 1.

    Modecharacters
    Chunk size / overlap4 / 1

    Text

    abcdefghij

    Chunks

    #0  "abcd"  (characters 0–4)
#1  "defg"  (characters 3–7)
#2  "ghij"  (characters 6–10)

    Each new chunk starts size − overlap = 3 characters after the previous one, so every pair of neighbours shares one character (d, then g).

    How it works

    Characters mode counts Unicode code points, so an emoji or accented letter is never cut in half. Chunk k starts at k × (size − overlap) and the final chunk may be shorter.

    Tokens mode encodes the whole text with the o200k_base tokenizer (js-tiktoken, running in your browser), slices the token list into windows of the chosen size with the chosen overlap, and decodes each window back to text. Token counts per chunk are therefore exact for that tokenizer. A window can start or end inside a multi-byte character; when that happens the decoded text may show a replacement character.

    Sentences mode splits the text with the browser’s Intl.Segmenter (English rules) and groups the chosen number of sentences per chunk, overlapping by the chosen number of sentences. Sentence boundaries depend on the browser’s Unicode data, so edge cases such as abbreviations can be split differently across browsers.

    Token counts shown in characters and sentences modes are the characters ÷ 4 estimate, labelled as such, and the JSON export records which kind of count it contains. There is no guidance here on the “best” size: it depends on your embedding model, documents and questions, and should be tested.

    Common errors

    Enter some text to split.
    Paste text into the box first.
    Overlap must be smaller than the chunk size.
    An overlap equal to or larger than the chunk size would never move forward. Lower the overlap.
    Chunk size must be a whole number of at least 1.
    Use a whole number such as 800; sizes are in the unit you chose.
    This would create more than 5,000 chunks. Use a larger size.
    Increase the chunk size or split the document in parts.
    Text is too long (limit 2,000,000 characters).
    Process the document in sections.

    Limits

    • Splitting is purely mechanical. It does not understand headings, code blocks, tables or paragraphs; structure-aware splitting needs a different approach.
    • Token mode uses o200k_base only. If your embedding model uses another tokenizer, its token counts will differ.
    • Sentence detection is heuristic. Text without punctuation becomes a single sentence.
    • Offsets (start and end) are provided in characters and sentences modes only; token mode returns the decoded text and token count.
    • The tool runs in your browser; very large inputs may be slow on older devices.

    Privacy

    The text is split in your browser and never sent anywhere. Share links include the text only if it is short; otherwise they carry just the settings. No account is needed, no cookies are set and no analytics run. See thePrivacy Policy. The “Copy share link” button puts your settings after the “#” in the address; browsers do not send that part to servers, and the page’s canonical address never includes it.

    FAQ

    What chunk size should I use?

    There is no universal value. The guide “Choosing chunk size and overlap for RAG” explains the trade-offs and how to test sizes on your own questions.

    Why use overlap at all?

    So that a fact straddling a boundary appears whole in at least one chunk. The cost is more chunks and some duplicated text to embed and store.

    Are the token counts exact?

    In token mode, yes, for the o200k_base encoding. In the other modes they are estimates (characters ÷ 4).

    Why does a decoded chunk show a strange character?

    A token window can cut through a multi-byte character. Use characters or sentences mode if that matters.

    Embed this tool

    Paste this snippet into your own page or docs to show the tool inline. The embedded view works the same way (everything runs in the visitor’s browser) and carries a small link back to MakeTools.

    Sources

    Page last updated 2026-10-14. Written by Prajyot Mulik; see the editorial policy.