Lines Matching full:flex
1 This is Info file flex.info, produced by Makeinfo-1.55 from the input
2 file flex.texi.
5 * Flex: (flex). A fast scanner generator.
8 This file documents Flex.
37 File: flex.info, Node: Top, Next: Name, Prev: (dir), Up: (dir)
39 flex
42 This manual documents `flex'. It covers release 2.5.
73 File: flex.info, Node: Name, Next: Synopsis, Prev: Top, Up: Top
78 flex - fast lexical analyzer generator
81 File: flex.info, Node: Synopsis, Next: Overview, Prev: Name, Up: Top
86 flex [-bcdfhilnpstvwBFILTV78+? -C[aefFmr] -ooutput -Pprefix -Sskeleton]
90 File: flex.info, Node: Overview, Next: Description, Prev: Synopsis, Up: Top
95 This manual describes `flex', a tool for generating programs that
105 the extended regular expressions used by flex
114 details regarding the scanner that flex produces; how to control
135 connecting flex scanners together with yacc parsers
138 flex command-line options, and the "%option" directive
147 how flex differs from AT&T lex and the POSIX lex standard
150 those error messages produced by flex (or scanners it generates)
154 files used by flex
157 known problems with flex
166 File: flex.info, Node: Description, Next: Examples, Prev: Overview, Up: Top
171 `flex' is a tool for generating "scanners": programs which
172 recognized lexical patterns in text. `flex' reads the given input
175 of pairs of regular expressions and C code, called "rules". `flex'
183 File: flex.info, Node: Examples, Next: Format, Prev: Description, Up: Top
188 First some simple examples to get the flavor of how one uses `flex'.
189 The following `flex' input specifies a scanner which whenever it
196 By default, any text not matched by a `flex' scanner is copied to
288 File: flex.info, Node: Format, Next: Patterns, Prev: Examples, Up: Top
293 The `flex' input file consists of three sections, separated by a
332 The RULES section of the `flex' input contains a series of rules of
366 File: flex.info, Node: Patterns, Next: Matching, Prev: Format, Up: Top
449 combinations of `R/S' that `flex' cannot match correctly; see
461 Note that flex's notion of "newline" is exactly whatever the C
462 compiler used to compile flex interprets '\n' as; in particular,
520 `isblank()', so flex defines `[:blank:]' as a blank or a tab.
575 File: flex.info, Node: Matching, Next: Actions, Prev: Patterns, Up: Top
585 of the same length, the rule listed first in the `flex' input file is
597 output. Thus, the simplest legal `flex' input is:
606 definition `flex' uses by including one of the special directives
607 `%pointer' or `%array' in the first (definitions) section of your flex
629 of your `flex' input. As mentioned above, with `%pointer' yytext grows
642 File: flex.info, Node: Actions, Next: Generated scanner, Prev: Matching, Up: Top
669 `flex' knows about C strings and comments and won't be fooled by braces
704 later in the `flex' input file, or one which matched less text.
776 Note that `yyless' is a macro and can only be used in the flex
856 File: flex.info, Node: Generated scanner, Next: Start conditions, Prev: Actions, Up: Top
861 The output of `flex' is the file `lex.yy.c', which contains the
895 with previous versions of `flex', and because it can be used to switch
953 File: flex.info, Node: Start conditions, Next: Multiple buffers, Prev: Generated scanner, Up: Top
958 `flex' provides a mechanism for conditionally activating rules. Any
987 any of the other rules in the `flex' input. Because of this, exclusive
1146 Flex provides `YYSTATE' as an alias for `YY_START' (since that is
1211 whole bunch of rules all preceded by the same start condition(s). Flex
1260 File: flex.info, Node: Multiple buffers, Next: End-of-file rules, Prev: Start conditions, Up: Top
1266 reading from several input streams. As `flex' scanners do a large
1274 To negotiate these sorts of problems, `flex' provides a mechanism
1411 File: flex.info, Node: End-of-file rules, Next: Miscellaneous, Prev: Multiple buffers, Up: Top
1422 flex, after doing the assignment you had to call the special
1460 File: flex.info, Node: Miscellaneous, Next: User variables, Prev: End-of-file rules, Up: Top
1517 File: flex.info, Node: User variables, Next: YACC interface, Prev: Miscellaneous, Up: Top
1544 - `FILE *yyin' is the file which by default `flex' reads from. It
1547 midst of scanning will have unexpected results since `flex'
1571 File: flex.info, Node: YACC interface, Next: Options, Prev: User variables, Up: Top
1576 One of the main uses of `flex' is as a companion to the `yacc'
1580 value in the global `yylval'. To use `flex' with `yacc', one specifies
1583 input. This file is then included in the `flex' scanner. For example,
1596 File: flex.info, Node: Options, Next: Performance, Prev: YACC interface, Up: Top
1601 `flex' has the following options:
1625 defining the scanner (i.e., the file that was fed to flex).
1637 generates a "help" summary of `flex's' options to `stdout' and
1641 instructs `flex' to generate a *case-insensitive* scanner. The
1642 case of letters given in the `flex' input patterns will be
1663 comments regarding features of the `flex' input file which will
1680 instructs `flex' to write the scanner it generates to standard
1684 specifies that `flex' should write to `stderr' a summary of
1686 statistics are meaningless to the casual `flex' user, but the
1687 first line identifies the version of `flex' (same as reported by
1695 instructs `flex' to generate a *batch* scanner, the opposite of
1726 instructs `flex' to generate an *interactive* scanner. An
1737 `Flex' scanners default to *interactive* unless you use the `-Cf'
1740 these options, so if you didn't, `flex' assumes you'd rather trade
1750 instructs `flex' not to generate `#line' directives. Without this
1751 option, `flex' peppers the generated scanner with #line directives
1753 respect to either the original `flex' input file (if the errors
1755 are `flex's' fault - you should report these sorts of errors to
1759 makes `flex' run in `trace' mode. It will generate a lot of
1762 This option is mostly for use in maintaining `flex'.
1769 instructs `flex' to generate a 7-bit scanner, i.e., one which can
1779 considerably less portable. `Flex's' default behavior is to
1781 which case `flex' defaults to generating 7-bit scanners unless
1784 flex generated a 7-bit or an 8-bit scanner by inspecting the flag
1789 below), flex still defaults to generating an 8-bit scanner, since
1794 instructs `flex' to generate an 8-bit scanner, i.e., one which can
1796 generated using `-Cf' or `-CF', as otherwise flex defaults to
1799 See the discussion of `-7' above for flex's default behavior and
1803 specifies that you want flex to generate a C++ scanner class. See
1810 `-Ca' ("align") instructs flex to trade off larger tables in the
1818 `-Ce' directs `flex' to construct "equivalence classes", i.e.,
1820 example, if the only appearance of digits in the `flex' input is
1828 - `flex' should not compress the tables by taking advantages of
1835 `-Cm' directs `flex' to construct "meta-equivalence classes",
1865 The default setting is `-Cem', which specifies that `flex' should
1889 directs flex to write the scanner to the file `out-' `put' instead
1895 changes the default `yy' prefix used by `flex' for all
1922 This option lets you easily link together multiple `flex' programs
1930 overrides the default skeleton file from which `flex' constructs
1932 `flex' maintenance or development.
1934 `flex' also provides a mechanism for controlling options within the
1935 scanner specification itself, rather than from the flex command-line.
1939 section of your flex input file. Most options are given simply as
1941 whitespace) to negate their meaning. A number are equivalent to flex
1978 instructs flex to generate a scanner which always considers its
1986 directs flex to provide a default `main()' program for the
1991 instructs flex to generate a scanner which never considers its
2004 directs `flex' to generate a scanner that maintains the number of
2014 `flex' scans your rule actions to determine whether you use the
2036 `flex' that you have derived `foo' as a subclass of `yyFlexLexer' so
2037 `flex' will place your actions in the member function `foo::yylex()'
2056 File: flex.info, Node: Performance, Next: C++, Prev: Options, Up: Top
2061 The main design goal of `flex' is that it generate high-performance
2171 `flex' feature will be to automatically add rules to eliminate backing
2287 `flex' can't figure this out, and it will plan for possibly needing to
2309 Compiled with `-Cf', this is about as fast as one can get a `flex'
2312 A final note: `flex' is slow when matching NUL's, particularly when
2326 File: flex.info, Node: C++, Next: Incompatibilities, Prev: Performance, Up: Top
2331 `flex' provides two different ways to generate scanners for use with
2332 C++. The first way is to simply compile a scanner generated by `flex'
2340 You can also use `flex' to generate a C++ scanner class, using the
2342 specified if the name of the flex executable ends in a `+', such as
2343 `flex++'. When using this option, flex defaults to generating the
2389 performs the same role is `yylex()' does for ordinary flex
2394 yyclass="S"' to inform `flex' that you will be using that subclass
2396 `yyFlexLexer::yylex()', `flex' generates `S::yylex()' (and also
2441 // An example of using the flex C++ scanner class.
2521 File: flex.info, Node: Incompatibilities, Next: Diagnostics, Prev: C++, Up: Top
2526 `flex' is a rewrite of the AT&T Unix `lex' tool (the two
2529 to write scanners acceptable to either implementation. Flex is fully
2535 between flex, AT&T lex, and the POSIX specification.
2537 `flex's' `-l' option turns on maximum compatibility with the
2542 `flex' is fully compatible with `lex' with the following exceptions:
2558 The `flex' restriction that `input()' cannot be redefined is in
2566 - `flex' scanners are not as reentrant as `lex' scanners. In
2571 fatal flex scanner internal error--end of buffer missed
2580 Also note that flex C++ scanner classes *are* reentrant, so if
2592 - When definitions are expanded, `flex' encloses them in
2602 is such that the '?' is associated with "[A-Z0-9]*". With `flex',
2609 the `<s>, /', and `<<EOF>>' operators cannot be used in a `flex'
2625 `flex' does not support this feature.
2637 occurrences of 'abc'", whereas `flex' interprets it as "match 'ab'
2643 'bar' anywhere", whereas `flex' interprets it as "match either
2648 `lex' are not required by `flex' scanners; `flex' ignores them.
2651 use with either `flex' or `lex'. Scanners also include
2653 which version of `flex' generated the scanner (for example, for the
2656 The following `flex' features are not included in `lex' or the POSIX
2679 plus almost all of the flex flags. The last feature in the list refers
2680 to the fact that with `flex' you can put multiple actions on the same
2689 `flex' does not truncate the action. Actions that are not enclosed
2693 File: flex.info, Node: Diagnostics, Next: Files, Prev: Incompatibilities, Up: Top
2718 scanner uses `REJECT' or `yymore()' but that `flex' failed to
2719 notice the fact, meaning that `flex' scanned the first two sections
2722 Use `%option reject' or `%option yymore' to indicate to flex that
2725 `flex scanner jammed'
2734 section of your `flex' input.
2743 `flex scanner push-back overflow'
2754 `fatal flex scanner internal error--end of buffer missed'
2768 File: flex.info, Node: Files, Next: Deficiencies, Prev: Diagnostics, Up: Top
2786 `flex.skl'
2787 skeleton scanner. This file is only used when building flex, not
2788 when flex executes.
2795 File: flex.info, Node: Deficiencies, Next: See also, Prev: Files, Up: Top
2832 `flex' rules and expect it to work. Call `input()' instead.
2842 The `flex' internal algorithms need documentation.
2845 File: flex.info, Node: See also, Next: Author, Prev: Deficiencies, Up: Top
2859 pattern-matching techniques used by `flex' (deterministic finite
2863 File: flex.info, Node: Author, Prev: See also, Up: Top
2873 Thanks to the many `flex' beta-testers, feedbackers, and
