Lines Matching full:lexer
71 The job of a TokenStream is to read lexer output and then provide ANTLR
280 usually a lexer. They provide the operations required by higher-level
290 to them for some other purpose, you have the lexer assign the comments and
311 # expected to return the token source object (such as a lexer) from which
346 feed text input into an ANTLR Lexer for token processing.
532 # This is an extra utility method for use inside lexer actions if needed.
540 # This is an extra utility method for use inside lexer actions if needed.
548 # This is an extra utility method for use inside lexer actions if needed.
556 # This is an extra utility method for use inside lexer actions if needed.
726 Using some TokenSource (such as a lexer), the stream collects a token sequence,
735 lexer = Calculator::Lexer.new(source_input)
736 tokens = ANTLR3::CommonTokenStream.new(lexer)
748 lexer.reset
750 ANTLR3::CommonTokenStream.new(lexer, :channel => ANTLR3::HIDDEN)
761 # usually a lexer, but can be any object that implements +next_token+ and includes
776 # ANTLR3::CommonTokenStream.new(lexer, :channel => :comment) do |token|
