Lines Matching refs:lexer
77 holds newly constructed tokens for lexer rules
180 For all effective purposes, the class and its immediate subclasses Lexer,
682 # If you change what tokens must be created by the lexer,
911 =begin rdoc ANTLR3::Lexer
913 = Lexer
915 Lexer is the default superclass of all lexers generated by ANTLR. The class
921 A lexer's job is to take input text and break it up into _tokens_ -- objects
923 position of the text with respect to the input. Thus, a lexer is essentially a
927 the lexer converts text into tokens for use by a parser, which recognizes larger
938 == ANTLR Lexers and the Lexer API
942 each lexer rule as a method of the same name. The subclass will also provide an
945 to fetch a token. The primary method in the lexer API, #next_token, uses
948 If the lexer is preparing tokens for use by an ANTLR generated parser, the lexer
954 class Lexer < ANTLR3::Lexer
975 lexer = Hypothetical::Lexer.new( input )
976 tokens = ANTLR3::CommonTokenStream.new( lexer )
983 lexer = Hypothetical::Lexer.new("some hypothetical source code", :file => 'blah-de-blah.hyp')
984 parser = Hypothetical::Parser.new( lexer )
987 class Lexer < Recognizer
1234 a lexer concerns the basic vocabulary symbols of the language, a parser's
1238 a corresponding lexer. Then, the user requests a specific sentence-structure
1265 @grammar_home and @grammar_home::Lexer
1269 require "#{ grammar_name }Lexer"
1270 @grammar_home::Lexer
1334 | does not appear to have an associated lexer class
