Phrase
data class PhraseNodeToken( val start: Int, val pastEnd: Int, val tokenIndexInName: Int)
Content copied to clipboard
An entry in the tokenSpans of a PhraseNode. The start and pastEnd identify where the token occurs in the UCS-2 source String, but using one-based indices. The tokenIndexInName is either zero or a one-based index into the atom's MessageSplitter's MessageSplitter.messageParts, indicating the part of the message that this token matched during parsing.
TODO 2022.12.19 - This just indicates the first equivalent token in the tokenized message, since we don't preserve which tokens matched what during parsing. So when we parse "a<<_", we can't (yet) tell afterward that the "<" between a and b was the first "<" in the method name, and the "<" between b and c was the second "<" in the method name.