Skip to content

Step: Chunk Content

  • Key: chunk-content
  • Category: Utilities
  • Description: Split content into chunks based on parameters.

Inputs

  • content (long-text, required): Content to chunk
  • chunkType (select, required): sentenceSplitter | tokenizer | recursiveCharacterWithSpecialSeparators | paragraphSplitter | regexSplitter
  • chunkSize (number): Max size
  • chunkOverlap (number): Overlap
  • maxTokens (number): Max tokens (tokenizer)
  • maxChunkSize (number): For paragraph/regex splitters
  • separators (long-text): JSON array of separators
  • regexPattern (text): Pattern for regex splitter

Outputs

  • chunks (array): Content chunks
  • chunkCount (number): Total chunks

Notes

  • Use before embedding or LLM summarization to manage large texts.

Built with VitePress