LexerParser

I keep needing a lexer/parser. I can't find one that's quite what I want, so I'm going to write one.

Lexer Regex

To start use regexp.FindIndex As long as the index is 0, it's a match.

Later I'd like to write a stripped down regex library, but it's not necessary at first.

Lexer Interface

• Kind() string // what kind of lexeme it is
• Value() string // the value of the lexeme
• Pos() (int,int) // the line and col from the original source, for debugging messages

Lexer

Takes a mapping of strings to regular expressions. The string is the type, the regex is the match.

Lexer Errors

When the lexer hits a state where it can't lex the input, it pulls off one character and tries again. When it succeeds, it concats all the bad characters and passes them as an Error. This means the string "Error" is reserved.

Lexer Helpers

Define a list of helpful regex strings - variable names, numbers (can get tricky with hex, octal, decimals, E notation).

Parser Interface

A Parser takes a string of Lexemes and returns a parse tree.

Parse Tree

A Parse Node has a parent (which is nil if if the node is root), a Lexeme and an ordered set of children. Note that the Lexeme does not need to be the same one received, they can be created and injected. Also, some Lexemes can be discarded (whitespace, commas).

ParserNode

• Parse(Lexeme) parses the lexeme into the parse tree
• AddChild: Adds a child node to the parse tree
• SetChild: Adds a child to the parse tree and sets it to the active node
• Pop: Sets parent to the current node
• Lexeme: direct access to the Lexeme
• Children: direct access to []ParserNode