Lines Matching full:lexer
43 /// to the lexer. When you free the lexer, the allocated memory is returned to the pool. The same applies
44 /// to 'strings' that contain the token text and various other text elements accessed within the lexer.
58 /// concerned with your lexer, parser, or tree parser. The is is the context pointer for your generated
67 /// function for your parser/lexer/tree parser (see code example in "How to build Generated Code" .)
82 /// \section lexermacros Lexer Macros
84 /// There are a number of macros that are useful exclusively within lexer rules. There are additional
87 /// \subsection lexer LEXER
89 /// The <code>LEXER</code> macro returns a pointer to the base lexer object, which is of type #pANTLR3_LEXER. This is
90 /// not the pointer to your generated lexer, which is supplied by the CTX macro,
91 /// but to the common implementation of a lexer interface,
96 /// Provides a pointer to the lexer shared state structure, which is where the tokens for a
97 /// rule are constructed and the status elements of the lexer are kept. This pointer is of type
106 /// and so on. The <code>LA(n)</code> macro is useful for constructing semantic predicates in lexer rules. The
107 /// reference <code>LA(0)</code> is undefined and will cause an error in your lexer.
120 /// stream before invoking the lexer and after creating the input stream.
124 /// The <code>GETTEXT</code> macro returns the text currently matched by the lexer rule. In general you should use the
131 /// the structure that holds the string. The #pANTLR3_STRING_FACTORY associated with the lexer handles this
132 /// and when you close the lexer, it will automatically free any space allocated for strings and their structures.
142 /// The macro <code>EMIT</code> causes the text range currently matched to the lexer rule to be emitted
144 /// the token is the name of the lexer rule or, if you have change this by using $type = XXX;, the type
162 /// can be used to stack many input streams to the lexer, and implement constructs such as the C pre-processor
165 /// An input stream that is pushed on to the stack becomes the current input stream for the lexer and
167 /// popped from the stack when it is exhausted by the lexer. You may use the macro <code>POPSTREAM</code>
170 /// Here is an example of using the macro in a lexer to implement the C \#include pre-processor directive:
218 /// to pop the stream at an appropriate point in your lexer action, so you do not match characters
223 /// A token manufactured by the lexer does not actually physically store the text from the
232 /// While you can create your own custom token class and have the lexer deal with this, this
241 /// but when you are building the token in the lexer, you must assign to the fields using the
296 /// Returns a reference to the input stream of the appropriate type for the recognizer. In a lexer
326 /// input stream types, both lexer, parser and tree parser.
