org.noear.solon.ai.rag.splitter

Class TextSplitter

java.lang.Object

org.noear.solon.ai.rag.splitter.TextSplitter

All Implemented Interfaces:

DocumentSplitter

Direct Known Subclasses:

RegexTextSplitter, TokenSizeTextSplitter

public abstract class TextSplitter
extends Object
implements DocumentSplitter

文本分割器

Since:

3.1

Author:

noear

Constructor Summary

Constructors

Constructor and Description
TextSplitter()

Method Summary

Modifier and Type	Method and Description
List<Document>	split(List<Document> documents) 分割
protected List<Document>	splitDocument(Document in, List<Document> outs)
protected abstract List<String>	splitText(String text)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.noear.solon.ai.rag.DocumentSplitter

split

Constructor Detail

TextSplitter

public TextSplitter()

Method Detail

split

public List<Document> split(List<Document> documents)

Description copied from interface: DocumentSplitter

分割

Specified by:

split in interface DocumentSplitter

Parameters:

documents - 文档

splitDocument

protected List<Document> splitDocument(Document in,
                                       List<Document> outs)

splitText

protected abstract List<String> splitText(String text)