Class Token

Hierarchy (view full)

Token
- LiteralToken

Index

Constructors

Properties

extraTokens lexeme span startWithWhitespace type

Methods

hasNewlineInExtraTokens toString

Constructors

constructor

new Token(type, span, lexeme): Token
Parameters
- type: TokenType
- span: CodeSpan
- lexeme: string
Returns Token
- Defined in src/Token.ts:19

Properties

extraTokens

extraTokens: ExtraToken[] = []

All the newlines and comments that appear before this token and should be preserved when printing the AST.

lexeme

lexeme: string

span

span: CodeSpan

startWithWhitespace

startWithWhitespace: CodeLocation

Start of this token, including all the whitespace before it.

Set externally in the lexer.

type

type: TokenType

Methods

hasNewlineInExtraTokens

hasNewlineInExtraTokens(): boolean
Returns boolean
- Defined in src/Token.ts:29

toString

toString(): string
Returns string
- Defined in src/Token.ts:25