Defines the core functionality for splitting text into smaller chunks.
This is essential for NLP and LLM applications, helping to manage
text size for model token limits, improve RAG, and enable focused
text analysis. Implementations will define specific splitting strategies
(e.g., by character, word, or semantic boundaries) while aiming to
preserve context.
Defines the core functionality for splitting text into smaller chunks. This is essential for NLP and LLM applications, helping to manage text size for model token limits, improve RAG, and enable focused text analysis. Implementations will define specific splitting strategies (e.g., by character, word, or semantic boundaries) while aiming to preserve context.