Companion
Functions
Link copied to clipboard
fun newToken( string: A_String, start: Int, lineNumber: Int, tokenType: TokenDescriptor.TokenType, generatingLexer: A_Lexer): A_Token
Content copied to clipboard
Create and initialize a new A_Token. The NEXT_LEXING_STATE_POJO is initially set to nil. For a token constructed by a lexer body, this pojo is updated automatically by the lexical scanning machinery to wrap a new LexingState. That machinery also sets up the new scanning position, the new line number, and the list of LexingState.allTokens.