Lines Matching refs:readline
3 generate_tokens(readline) is a generator that breaks a stream of
4 text into Python tokens. It accepts a readline-like method which is called
19 tokenize_loop(readline, tokeneater)
20 tokenize(readline, tokeneater=printtoken)
155 def tokenize(readline, tokeneater=printtoken):
160 The first parameter, readline, must be a callable object which provides
161 the same interface as the readline() method of built-in file objects.
169 tokenize_loop(readline, tokeneater)
174 def tokenize_loop(readline, tokeneater):
175 for token_info in generate_tokens(readline):
255 t1 = [tok[:2] for tok in generate_tokens(f.readline)]
257 readline = iter(newcode.splitlines(1)).next
258 t2 = [tok[:2] for tok in generate_tokens(readline)]
264 def generate_tokens(readline):
266 The generate_tokens() generator requires one argment, readline, which
268 readline() method of built-in file objects. Each call to the function
269 should return one line of input as a string. Alternately, readline
271 readline = open(myfile).next # Example of alternate readline
288 line = readline()
424 tokenize(open(sys.argv[1]).readline)
426 tokenize(sys.stdin.readline)
