Lines Matching full:tokenizer
28 * A simple tokenizer for loading and parsing ASCII text files line by line.
30 class Tokenizer {
31 Tokenizer(const String8& filename, FileMap* fileMap, char* buffer,
35 ~Tokenizer();
40 * Returns NO_ERROR and a tokenizer for the file, if successful.
43 static status_t open(const String8& filename, Tokenizer** outTokenizer);
48 * Returns NO_ERROR and a tokenizer for the string, if successful.
52 const char* contents, Tokenizer** outTokenizer);
119 Tokenizer(const Tokenizer& other); // not copyable
