Lines Matching defs:Lexer
30 /** A lexer is recognizer that draws input symbols from a character stream.
31 * lexer grammars result in a subclass of this object. A Lexer object
35 public class Lexer extends BaseRecognizer implements TokenSource {
36 /** Where is the lexer drawing characters from? */
39 public function Lexer(input:CharStream = null, state:RecognizerSharedState = null) {
46 // wack Lexer state variables
99 /** Instruct the lexer to skip creating a token for current lexer rule
101 * a lexer rule finishes with token set to SKIP_TOKEN. Recall that
109 /** This is the lexer entry point that sets instance var 'token' */
115 /** Set the char stream and reset the lexer */
185 recover(mte); // don't really recover; just consume in lexer
