avail/avail.persistence.cache.record/PhrasePathRecord/PhraseNode/PhraseNodeToken

PhraseNodeToken

data class PhraseNodeToken(val start: Int, val pastEnd: Int, val line: Int, val tokenIndexInName: Int, val tokenContent: String?)

An entry in the tokenSpans of a PhraseNode. The start and pastEnd identify where the token occurs in the UCS-2 source String, using zero-based indices. The tokenIndexInName is either zero or a one-based index into the atom's MessageSplitter's MessageSplitter.messageParts, indicating the part of the message that this token matched during parsing.

Constructors

PhraseNodeToken

constructor(start: Int, pastEnd: Int, line: Int, tokenIndexInName: Int, tokenContent: String?)

Properties

line

val line: Int

The one-based line number containing the token. If the token spans multiple lines (say a string literal), this is the line number of the start of the token.

pastEnd

val pastEnd: Int

The zero-based index into the UCS-2 String just past the token.

start

val start: Int

The zero-based index into the UCS-2 String at which the token begins.

tokenContent

val tokenContent: String?

If the tokenIndexInName is zero, this is the text of a token that occurred. This is generally not a token of any method, and is useful for capturing tokens produced by a lexer and consumed as a literal token by an ellipsis ("…") within a method name.

tokenIndexInName

val tokenIndexInName: Int

Either zero to indicate this was not a token that occurred in the actual method name, or the one-based index into the PhraseNode's MessageSplitter's tuple of tokenized parts.