Home | History | Annotate | Download | only in js

Lines Matching full:tokenizer

816     var tokenizer;
837       tokenizer = combinePrefixPatterns(allRegexs);
863       var tokens = sourceCode.match(tokenizer) || [];