ktoken/com.aallam.ktoken.encoding/R50KBase

R50KBase

data class R50KBase(val file: String = "r50k_base.tiktoken") : Encoding

Default configuration of r50k_base encoding.

Constructors

constructor(file: String = "r50k_base.tiktoken")

Properties

open override val file: String

The encoding file path.

Functions

open override fun encodingConfig(ranks: Map<ByteString, Int>): EncodingConfig

Retrieve token encoding utilizing the provided BpeLoader.