Lines Matching full:lexer
2 Kaleidoscope: Tutorial Introduction and the Lexer
39 language, and the definition of its Lexer - This shows where we are
42 to implement everything in Objective Caml instead of using lexer and
46 AST - With the lexer in place, we can talk about parsing techniques
86 language including a hand-written lexer, parser, AST, as well as code
146 The Lexer
152 "`lexer <http://en.wikipedia.org/wiki/Lexical_analysis>`_" (aka
154 the lexer includes a token code and potentially some metadata (e.g. the
159 (* The lexer returns these 'Kwd' if it is an unknown character, otherwise one of
171 Each token returned by our lexer will be one of the token variant
177 The actual implementation of the lexer is a collection of functions
178 driven by a function named ``Lexer.lex``. The ``Lexer.lex`` function is
187 * Lexer
194 ``Lexer.lex`` works by recursing over a ``char Stream.t`` to read
200 The next thing ``Lexer.lex`` needs to do is recognize identifiers and
277 With this, we have the complete lexer for the basic Kaleidoscope
279 Lexer is available in the `next chapter <OCamlLangImpl2.html>`_ of the
282 include a driver so that you can use the lexer and parser together.
