React Native RAG - v0.2.0
    Preparing search index...

    Class TokenTextSplitter

    A text splitter that splits text based on a token count rather than character count. This is a wrapper around Langchain's TokenTextSplitter.

    Implements

    Index

    Constructors

    Methods

    Constructors

    • Creates an instance of TokenTextSplitter.

      Parameters

      • params: TextSplitterParams

        Parameters for the text splitter.

        • chunkSize

          The maximum size of each text chunk (in tokens).

        • chunkOverlap

          The amount of overlap between consecutive text chunks (in tokens).

      Returns TokenTextSplitter

    Methods

    • Splits the given text into chunks based on token count.

      Parameters

      • text: string

        The input text to be split.

      Returns Promise<string[]>

      A promise that resolves to an array of string chunks.