1 .. _2to3-reference: 2 3 2to3 - Automated Python 2 to 3 code translation 4 =============================================== 5 6 .. sectionauthor:: Benjamin Peterson <benjamin (a] python.org> 7 8 2to3 is a Python program that reads Python 2.x source code and applies a series 9 of *fixers* to transform it into valid Python 3.x code. The standard library 10 contains a rich set of fixers that will handle almost all code. 2to3 supporting 11 library :mod:`lib2to3` is, however, a flexible and generic library, so it is 12 possible to write your own fixers for 2to3. :mod:`lib2to3` could also be 13 adapted to custom applications in which Python code needs to be edited 14 automatically. 15 16 17 .. _2to3-using: 18 19 Using 2to3 20 ---------- 21 22 2to3 will usually be installed with the Python interpreter as a script. It is 23 also located in the :file:`Tools/scripts` directory of the Python root. 24 25 2to3's basic arguments are a list of files or directories to transform. The 26 directories are recursively traversed for Python sources. 27 28 Here is a sample Python 2.x source file, :file:`example.py`:: 29 30 def greet(name): 31 print "Hello, {0}!".format(name) 32 print "What's your name?" 33 name = raw_input() 34 greet(name) 35 36 It can be converted to Python 3.x code via 2to3 on the command line: 37 38 .. code-block:: shell-session 39 40 $ 2to3 example.py 41 42 A diff against the original source file is printed. 2to3 can also write the 43 needed modifications right back to the source file. (A backup of the original 44 file is made unless :option:`!-n` is also given.) Writing the changes back is 45 enabled with the :option:`!-w` flag: 46 47 .. code-block:: shell-session 48 49 $ 2to3 -w example.py 50 51 After transformation, :file:`example.py` looks like this:: 52 53 def greet(name): 54 print("Hello, {0}!".format(name)) 55 print("What's your name?") 56 name = input() 57 greet(name) 58 59 Comments and exact indentation are preserved throughout the translation process. 60 61 By default, 2to3 runs a set of :ref:`predefined fixers <2to3-fixers>`. The 62 :option:`!-l` flag lists all available fixers. An explicit set of fixers to run 63 can be given with :option:`!-f`. Likewise the :option:`!-x` explicitly disables a 64 fixer. The following example runs only the ``imports`` and ``has_key`` fixers: 65 66 .. code-block:: shell-session 67 68 $ 2to3 -f imports -f has_key example.py 69 70 This command runs every fixer except the ``apply`` fixer: 71 72 .. code-block:: shell-session 73 74 $ 2to3 -x apply example.py 75 76 Some fixers are *explicit*, meaning they aren't run by default and must be 77 listed on the command line to be run. Here, in addition to the default fixers, 78 the ``idioms`` fixer is run: 79 80 .. code-block:: shell-session 81 82 $ 2to3 -f all -f idioms example.py 83 84 Notice how passing ``all`` enables all default fixers. 85 86 Sometimes 2to3 will find a place in your source code that needs to be changed, 87 but 2to3 cannot fix automatically. In this case, 2to3 will print a warning 88 beneath the diff for a file. You should address the warning in order to have 89 compliant 3.x code. 90 91 2to3 can also refactor doctests. To enable this mode, use the :option:`!-d` 92 flag. Note that *only* doctests will be refactored. This also doesn't require 93 the module to be valid Python. For example, doctest like examples in a reST 94 document could also be refactored with this option. 95 96 The :option:`!-v` option enables output of more information on the translation 97 process. 98 99 Since some print statements can be parsed as function calls or statements, 2to3 100 cannot always read files containing the print function. When 2to3 detects the 101 presence of the ``from __future__ import print_function`` compiler directive, it 102 modifies its internal grammar to interpret :func:`print` as a function. This 103 change can also be enabled manually with the :option:`!-p` flag. Use 104 :option:`!-p` to run fixers on code that already has had its print statements 105 converted. 106 107 The :option:`!-o` or :option:`!--output-dir` option allows specification of an 108 alternate directory for processed output files to be written to. The 109 :option:`!-n` flag is required when using this as backup files do not make sense 110 when not overwriting the input files. 111 112 .. versionadded:: 2.7.3 113 The :option:`!-o` option was added. 114 115 The :option:`!-W` or :option:`!--write-unchanged-files` flag tells 2to3 to always 116 write output files even if no changes were required to the file. This is most 117 useful with :option:`!-o` so that an entire Python source tree is copied with 118 translation from one directory to another. 119 This option implies the :option:`!-w` flag as it would not make sense otherwise. 120 121 .. versionadded:: 2.7.3 122 The :option:`!-W` flag was added. 123 124 The :option:`!--add-suffix` option specifies a string to append to all output 125 filenames. The :option:`!-n` flag is required when specifying this as backups 126 are not necessary when writing to different filenames. Example: 127 128 .. code-block:: shell-session 129 130 $ 2to3 -n -W --add-suffix=3 example.py 131 132 Will cause a converted file named ``example.py3`` to be written. 133 134 .. versionadded:: 2.7.3 135 The :option:`!--add-suffix` option was added. 136 137 To translate an entire project from one directory tree to another use: 138 139 .. code-block:: shell-session 140 141 $ 2to3 --output-dir=python3-version/mycode -W -n python2-version/mycode 142 143 144 .. _2to3-fixers: 145 146 Fixers 147 ------ 148 149 Each step of transforming code is encapsulated in a fixer. The command ``2to3 150 -l`` lists them. As :ref:`documented above <2to3-using>`, each can be turned on 151 and off individually. They are described here in more detail. 152 153 154 .. 2to3fixer:: apply 155 156 Removes usage of :func:`apply`. For example ``apply(function, *args, 157 **kwargs)`` is converted to ``function(*args, **kwargs)``. 158 159 .. 2to3fixer:: asserts 160 161 Replaces deprecated :mod:`unittest` method names with the correct ones. 162 163 ================================ ========================================== 164 From To 165 ================================ ========================================== 166 ``failUnlessEqual(a, b)`` :meth:`assertEqual(a, b) 167 <unittest.TestCase.assertEqual>` 168 ``assertEquals(a, b)`` :meth:`assertEqual(a, b) 169 <unittest.TestCase.assertEqual>` 170 ``failIfEqual(a, b)`` :meth:`assertNotEqual(a, b) 171 <unittest.TestCase.assertNotEqual>` 172 ``assertNotEquals(a, b)`` :meth:`assertNotEqual(a, b) 173 <unittest.TestCase.assertNotEqual>` 174 ``failUnless(a)`` :meth:`assertTrue(a) 175 <unittest.TestCase.assertTrue>` 176 ``assert_(a)`` :meth:`assertTrue(a) 177 <unittest.TestCase.assertTrue>` 178 ``failIf(a)`` :meth:`assertFalse(a) 179 <unittest.TestCase.assertFalse>` 180 ``failUnlessRaises(exc, cal)`` :meth:`assertRaises(exc, cal) 181 <unittest.TestCase.assertRaises>` 182 ``failUnlessAlmostEqual(a, b)`` :meth:`assertAlmostEqual(a, b) 183 <unittest.TestCase.assertAlmostEqual>` 184 ``assertAlmostEquals(a, b)`` :meth:`assertAlmostEqual(a, b) 185 <unittest.TestCase.assertAlmostEqual>` 186 ``failIfAlmostEqual(a, b)`` :meth:`assertNotAlmostEqual(a, b) 187 <unittest.TestCase.assertNotAlmostEqual>` 188 ``assertNotAlmostEquals(a, b)`` :meth:`assertNotAlmostEqual(a, b) 189 <unittest.TestCase.assertNotAlmostEqual>` 190 ================================ ========================================== 191 192 .. 2to3fixer:: basestring 193 194 Converts :class:`basestring` to :class:`str`. 195 196 .. 2to3fixer:: buffer 197 198 Converts :class:`buffer` to :class:`memoryview`. This fixer is optional 199 because the :class:`memoryview` API is similar but not exactly the same as 200 that of :class:`buffer`. 201 202 .. 2to3fixer:: callable 203 204 Converts ``callable(x)`` to ``isinstance(x, collections.Callable)``, adding 205 an import to :mod:`collections` if needed. Note ``callable(x)`` has returned 206 in Python 3.2, so if you do not intend to support Python 3.1, you can disable 207 this fixer. 208 209 .. 2to3fixer:: dict 210 211 Fixes dictionary iteration methods. :meth:`dict.iteritems` is converted to 212 :meth:`dict.items`, :meth:`dict.iterkeys` to :meth:`dict.keys`, and 213 :meth:`dict.itervalues` to :meth:`dict.values`. Similarly, 214 :meth:`dict.viewitems`, :meth:`dict.viewkeys` and :meth:`dict.viewvalues` are 215 converted respectively to :meth:`dict.items`, :meth:`dict.keys` and 216 :meth:`dict.values`. It also wraps existing usages of :meth:`dict.items`, 217 :meth:`dict.keys`, and :meth:`dict.values` in a call to :class:`list`. 218 219 .. 2to3fixer:: except 220 221 Converts ``except X, T`` to ``except X as T``. 222 223 .. 2to3fixer:: exec 224 225 Converts the :keyword:`exec` statement to the :func:`exec` function. 226 227 .. 2to3fixer:: execfile 228 229 Removes usage of :func:`execfile`. The argument to :func:`execfile` is 230 wrapped in calls to :func:`open`, :func:`compile`, and :func:`exec`. 231 232 .. 2to3fixer:: exitfunc 233 234 Changes assignment of :attr:`sys.exitfunc` to use of the :mod:`atexit` 235 module. 236 237 .. 2to3fixer:: filter 238 239 Wraps :func:`filter` usage in a :class:`list` call. 240 241 .. 2to3fixer:: funcattrs 242 243 Fixes function attributes that have been renamed. For example, 244 ``my_function.func_closure`` is converted to ``my_function.__closure__``. 245 246 .. 2to3fixer:: future 247 248 Removes ``from __future__ import new_feature`` statements. 249 250 .. 2to3fixer:: getcwdu 251 252 Renames :func:`os.getcwdu` to :func:`os.getcwd`. 253 254 .. 2to3fixer:: has_key 255 256 Changes ``dict.has_key(key)`` to ``key in dict``. 257 258 .. 2to3fixer:: idioms 259 260 This optional fixer performs several transformations that make Python code 261 more idiomatic. Type comparisons like ``type(x) is SomeClass`` and 262 ``type(x) == SomeClass`` are converted to ``isinstance(x, SomeClass)``. 263 ``while 1`` becomes ``while True``. This fixer also tries to make use of 264 :func:`sorted` in appropriate places. For example, this block :: 265 266 L = list(some_iterable) 267 L.sort() 268 269 is changed to :: 270 271 L = sorted(some_iterable) 272 273 .. 2to3fixer:: import 274 275 Detects sibling imports and converts them to relative imports. 276 277 .. 2to3fixer:: imports 278 279 Handles module renames in the standard library. 280 281 .. 2to3fixer:: imports2 282 283 Handles other modules renames in the standard library. It is separate from 284 the :2to3fixer:`imports` fixer only because of technical limitations. 285 286 .. 2to3fixer:: input 287 288 Converts ``input(prompt)`` to ``eval(input(prompt))``. 289 290 .. 2to3fixer:: intern 291 292 Converts :func:`intern` to :func:`sys.intern`. 293 294 .. 2to3fixer:: isinstance 295 296 Fixes duplicate types in the second argument of :func:`isinstance`. For 297 example, ``isinstance(x, (int, int))`` is converted to ``isinstance(x, 298 (int))``. 299 300 .. 2to3fixer:: itertools_imports 301 302 Removes imports of :func:`itertools.ifilter`, :func:`itertools.izip`, and 303 :func:`itertools.imap`. Imports of :func:`itertools.ifilterfalse` are also 304 changed to :func:`itertools.filterfalse`. 305 306 .. 2to3fixer:: itertools 307 308 Changes usage of :func:`itertools.ifilter`, :func:`itertools.izip`, and 309 :func:`itertools.imap` to their built-in equivalents. 310 :func:`itertools.ifilterfalse` is changed to :func:`itertools.filterfalse`. 311 312 .. 2to3fixer:: long 313 314 Renames :class:`long` to :class:`int`. 315 316 .. 2to3fixer:: map 317 318 Wraps :func:`map` in a :class:`list` call. It also changes ``map(None, x)`` 319 to ``list(x)``. Using ``from future_builtins import map`` disables this 320 fixer. 321 322 .. 2to3fixer:: metaclass 323 324 Converts the old metaclass syntax (``__metaclass__ = Meta`` in the class 325 body) to the new (``class X(metaclass=Meta)``). 326 327 .. 2to3fixer:: methodattrs 328 329 Fixes old method attribute names. For example, ``meth.im_func`` is converted 330 to ``meth.__func__``. 331 332 .. 2to3fixer:: ne 333 334 Converts the old not-equal syntax, ``<>``, to ``!=``. 335 336 .. 2to3fixer:: next 337 338 Converts the use of iterator's :meth:`~iterator.next` methods to the 339 :func:`next` function. It also renames :meth:`~iterator.next` methods to 340 :meth:`~iterator.__next__`. 341 342 .. 2to3fixer:: nonzero 343 344 Renames :meth:`__nonzero__` to :meth:`~object.__bool__`. 345 346 .. 2to3fixer:: numliterals 347 348 Converts octal literals into the new syntax. 349 350 .. 2to3fixer:: paren 351 352 Add extra parenthesis where they are required in list comprehensions. For 353 example, ``[x for x in 1, 2]`` becomes ``[x for x in (1, 2)]``. 354 355 .. 2to3fixer:: print 356 357 Converts the :keyword:`print` statement to the :func:`print` function. 358 359 .. 2to3fixer:: raise 360 361 Converts ``raise E, V`` to ``raise E(V)``, and ``raise E, V, T`` to ``raise 362 E(V).with_traceback(T)``. If ``E`` is a tuple, the translation will be 363 incorrect because substituting tuples for exceptions has been removed in Python 3. 364 365 .. 2to3fixer:: raw_input 366 367 Converts :func:`raw_input` to :func:`input`. 368 369 .. 2to3fixer:: reduce 370 371 Handles the move of :func:`reduce` to :func:`functools.reduce`. 372 373 .. 2to3fixer:: renames 374 375 Changes :data:`sys.maxint` to :data:`sys.maxsize`. 376 377 .. 2to3fixer:: repr 378 379 Replaces backtick repr with the :func:`repr` function. 380 381 .. 2to3fixer:: set_literal 382 383 Replaces use of the :class:`set` constructor with set literals. This fixer 384 is optional. 385 386 .. 2to3fixer:: standarderror 387 388 Renames :exc:`StandardError` to :exc:`Exception`. 389 390 .. 2to3fixer:: sys_exc 391 392 Changes the deprecated :data:`sys.exc_value`, :data:`sys.exc_type`, 393 :data:`sys.exc_traceback` to use :func:`sys.exc_info`. 394 395 .. 2to3fixer:: throw 396 397 Fixes the API change in generator's :meth:`throw` method. 398 399 .. 2to3fixer:: tuple_params 400 401 Removes implicit tuple parameter unpacking. This fixer inserts temporary 402 variables. 403 404 .. 2to3fixer:: types 405 406 Fixes code broken from the removal of some members in the :mod:`types` 407 module. 408 409 .. 2to3fixer:: unicode 410 411 Renames :class:`unicode` to :class:`str`. 412 413 .. 2to3fixer:: urllib 414 415 Handles the rename of :mod:`urllib` and :mod:`urllib2` to the :mod:`urllib` 416 package. 417 418 .. 2to3fixer:: ws_comma 419 420 Removes excess whitespace from comma separated items. This fixer is 421 optional. 422 423 .. 2to3fixer:: xrange 424 425 Renames :func:`xrange` to :func:`range` and wraps existing :func:`range` 426 calls with :class:`list`. 427 428 .. 2to3fixer:: xreadlines 429 430 Changes ``for x in file.xreadlines()`` to ``for x in file``. 431 432 .. 2to3fixer:: zip 433 434 Wraps :func:`zip` usage in a :class:`list` call. This is disabled when 435 ``from future_builtins import zip`` appears. 436 437 438 :mod:`lib2to3` - 2to3's library 439 ------------------------------- 440 441 .. module:: lib2to3 442 :synopsis: the 2to3 library 443 .. moduleauthor:: Guido van Rossum 444 .. moduleauthor:: Collin Winter 445 .. moduleauthor:: Benjamin Peterson <benjamin (a] python.org> 446 447 448 .. note:: 449 450 The :mod:`lib2to3` API should be considered unstable and may change 451 drastically in the future. 452 453 .. XXX What is the public interface anyway? 454
