Lines Matching full:token
566 .I token)
662 it also must rescan the entire token from the beginning, so matching such
681 action is empty, then when the pattern is matched the input token
699 [ \\t]+$ /* ignore this token */
778 scanner normally executes only one action per token.
782 active rule. For example, when the following scanner scans the token
820 token should be
847 correctly reflecting the size of the current token, so you must not
860 characters of the current token back to the input stream, where they
894 The following action will take the current token and cause it
983 so that the next time the scanner attempts to match a token, it will
1333 it will treat it as a single token, the floating-point number
1453 /* return string constant token type and
1654 so the next time the scanner attempts to match a token from the
1916 context for the next token match is done as though at the
1922 returns true if the next token scanned from the current buffer
1944 holds the text of the current token. It may be modified but not lengthened
1980 holds the length of the current token.
2032 to find the next input token. The routine is supposed to
2033 return the type of the next token as well as putting any associated
2280 what token has been matched if it absolutely must. It turns out that
2282 seen enough text to disambiguate the current token, is a bit faster than
2285 a newline, it is not recognized as a newline token until they enter
2287 token, which often means typing in another whole line.
2972 a valid token. A possible future
3120 when it has scanned a token like "auto" and then the next character
3151 is slow when matching NUL's, particularly when a token contains
3161 token be rescanned from the beginning. Thus if performance is
3164 characters/token.
3206 returns the text of the most recently matched token, the equivalent of
3210 returns the length of the most recently matched token, the equivalent of
3662 is undefined until the next token is matched, unless the scanner
3822 .I token too large, exceeds YYLMAX -
3851 both the pushed-back text and the current token in
3858 the scanner was working on matching an extremely large token and needed
3950 all the text matched so far by the current (generally huge) token.
