Module org.nasdanika.ai

Package org.nasdanika.ai

Class TextFloatVectorCharChunkingEmbeddings

java.lang.Object

org.nasdanika.ai.TextFloatVectorChunkingEmbeddingModel<char[]>

org.nasdanika.ai.TextFloatVectorCharChunkingEmbeddings

All Implemented Interfaces:

Coordinates, EmbeddingGenerator<String,List<List<Float>>>, EmbeddingModel<String,List<List<Float>>>, FloatVectorEmbeddingGenerator<String>, Model, TextEmbeddingGenerator<List<List<Float>>>, TextFloatVectorEmbeddingModel, VectorEmbeddingGenerator<String,Float>, Composable<EmbeddingGenerator<String,List<List<Float>>>>

public class TextFloatVectorCharChunkingEmbeddings
extends TextFloatVectorChunkingEmbeddingModel<char[]>

A simple implementation which treats a character as a token. Can be used for testing and when an encoder is unknown or not available.

Nested Class Summary

Nested classes/interfaces inherited from interface org.nasdanika.ai.EmbeddingGenerator

EmbeddingGenerator.Requirement

Constructor Summary

Constructors

Constructor	Description
TextFloatVectorCharChunkingEmbeddings(TextFloatVectorEmbeddingModel target, int chunkSize, int overlap)

Method Summary

Modifier and Type	Method	Description
protected String	decode(char[] tokens)	Decodes a string from an array of tokens
protected char[]	encode(String input)	Encodes a string into tokens
protected int	size(char[] tokens)
protected char[]	slice(char[] tokens, int offset, int length)

Methods inherited from class org.nasdanika.ai.TextFloatVectorChunkingEmbeddingModel

chunk, generate, generateAsync, getDimensions, getMaxInputTokens, getName, getProvider, getVersion, isTooLong

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.nasdanika.ai.EmbeddingGenerator

adapt, compose, compose, composeAsync, generate, generateAsync, then

Constructor Details

TextFloatVectorCharChunkingEmbeddings

public TextFloatVectorCharChunkingEmbeddings(TextFloatVectorEmbeddingModel target,
 int chunkSize,
 int overlap)

Method Details

encode

protected char[] encode(String input)

Description copied from class: TextFloatVectorChunkingEmbeddingModel

Encodes a string into tokens

Specified by:

encode in class TextFloatVectorChunkingEmbeddingModel<char[]>

Parameters:

input -

Returns:

decode

protected String decode(char[] tokens)

Description copied from class: TextFloatVectorChunkingEmbeddingModel

Decodes a string from an array of tokens

Specified by:

decode in class TextFloatVectorChunkingEmbeddingModel<char[]>

Parameters:

tokens -

Returns:

size

protected int size(char[] tokens)

Specified by:

size in class TextFloatVectorChunkingEmbeddingModel<char[]>

slice

protected char[] slice(char[] tokens,
 int offset,
 int length)

Specified by:

slice in class TextFloatVectorChunkingEmbeddingModel<char[]>