1 **************************** 2 What's New In Python 3.6 3 **************************** 4 5 :Editors: Elvis Pranskevichus <elvis (a] magic.io>, Yury Selivanov <yury (a] magic.io> 6 7 .. Rules for maintenance: 8 9 * Anyone can add text to this document. Do not spend very much time 10 on the wording of your changes, because your text will probably 11 get rewritten to some degree. 12 13 * The maintainer will go through Misc/NEWS periodically and add 14 changes; it's therefore more important to add your changes to 15 Misc/NEWS than to this file. 16 17 * This is not a complete list of every single change; completeness 18 is the purpose of Misc/NEWS. Some changes I consider too small 19 or esoteric to include. If such a change is added to the text, 20 I'll just remove it. (This is another reason you shouldn't spend 21 too much time on writing your addition.) 22 23 * If you want to draw your new text to the attention of the 24 maintainer, add 'XXX' to the beginning of the paragraph or 25 section. 26 27 * It's OK to just add a fragmentary note about a change. For 28 example: "XXX Describe the transmogrify() function added to the 29 socket module." The maintainer will research the change and 30 write the necessary text. 31 32 * You can comment out your additions if you like, but it's not 33 necessary (especially when a final release is some months away). 34 35 * Credit the author of a patch or bugfix. Just the name is 36 sufficient; the e-mail address isn't necessary. 37 38 * It's helpful to add the bug/patch number as a comment: 39 40 XXX Describe the transmogrify() function added to the socket 41 module. 42 (Contributed by P.Y. Developer in :issue:`12345`.) 43 44 This saves the maintainer the effort of going through the Mercurial log 45 when researching a change. 46 47 This article explains the new features in Python 3.6, compared to 3.5. 48 Python 3.6 was released on December 23, 2016. See the 49 `changelog <https://docs.python.org/3.6/whatsnew/changelog.html>`_ for a full 50 list of changes. 51 52 .. seealso:: 53 54 :pep:`494` - Python 3.6 Release Schedule 55 56 57 Summary -- Release highlights 58 ============================= 59 60 New syntax features: 61 62 * :ref:`PEP 498 <whatsnew36-pep498>`, formatted string literals. 63 64 * :ref:`PEP 515 <whatsnew36-pep515>`, underscores in numeric literals. 65 66 * :ref:`PEP 526 <whatsnew36-pep526>`, syntax for variable annotations. 67 68 * :ref:`PEP 525 <whatsnew36-pep525>`, asynchronous generators. 69 70 * :ref:`PEP 530 <whatsnew36-pep530>`: asynchronous comprehensions. 71 72 73 New library modules: 74 75 * :mod:`secrets`: :ref:`PEP 506 -- Adding A Secrets Module To The Standard Library <whatsnew36-pep506>`. 76 77 78 CPython implementation improvements: 79 80 * The :ref:`dict <typesmapping>` type has been reimplemented to use 81 a :ref:`more compact representation <whatsnew36-compactdict>` 82 based on `a proposal by Raymond Hettinger 83 <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_ 84 and similar to the `PyPy dict implementation`_. This resulted in dictionaries 85 using 20% to 25% less memory when compared to Python 3.5. 86 87 * Customization of class creation has been simplified with the 88 :ref:`new protocol <whatsnew36-pep487>`. 89 90 * The class attribute definition order is 91 :ref:`now preserved <whatsnew36-pep520>`. 92 93 * The order of elements in ``**kwargs`` now 94 :ref:`corresponds to the order <whatsnew36-pep468>` in which keyword 95 arguments were passed to the function. 96 97 * DTrace and SystemTap :ref:`probing support <whatsnew36-tracing>` has 98 been added. 99 100 * The new :ref:`PYTHONMALLOC <whatsnew36-pythonmalloc>` environment variable 101 can now be used to debug the interpreter memory allocation and access 102 errors. 103 104 105 Significant improvements in the standard library: 106 107 * The :mod:`asyncio` module has received new features, significant 108 usability and performance improvements, and a fair amount of bug fixes. 109 Starting with Python 3.6 the ``asyncio`` module is no longer provisional 110 and its API is considered stable. 111 112 * A new :ref:`file system path protocol <whatsnew36-pep519>` has been 113 implemented to support :term:`path-like objects <path-like object>`. 114 All standard library functions operating on paths have been updated to 115 work with the new protocol. 116 117 * The :mod:`datetime` module has gained support for 118 :ref:`Local Time Disambiguation <whatsnew36-pep495>`. 119 120 * The :mod:`typing` module received a number of 121 :ref:`improvements <whatsnew36-typing>`. 122 123 * The :mod:`tracemalloc` module has been significantly reworked 124 and is now used to provide better output for :exc:`ResourceWarning` 125 as well as provide better diagnostics for memory allocation errors. 126 See the :ref:`PYTHONMALLOC section <whatsnew36-pythonmalloc>` for more 127 information. 128 129 130 Security improvements: 131 132 * The new :mod:`secrets` module has been added to simplify the generation of 133 cryptographically strong pseudo-random numbers suitable for 134 managing secrets such as account authentication, tokens, and similar. 135 136 * On Linux, :func:`os.urandom` now blocks until the system urandom entropy 137 pool is initialized to increase the security. See the :pep:`524` for the 138 rationale. 139 140 * The :mod:`hashlib` and :mod:`ssl` modules now support OpenSSL 1.1.0. 141 142 * The default settings and feature set of the :mod:`ssl` module have been 143 improved. 144 145 * The :mod:`hashlib` module received support for the BLAKE2, SHA-3 and SHAKE 146 hash algorithms and the :func:`~hashlib.scrypt` key derivation function. 147 148 149 Windows improvements: 150 151 * :ref:`PEP 528 <whatsnew36-pep528>` and :ref:`PEP 529 <whatsnew36-pep529>`, 152 Windows filesystem and console encoding changed to UTF-8. 153 154 * The ``py.exe`` launcher, when used interactively, no longer prefers 155 Python 2 over Python 3 when the user doesn't specify a version (via 156 command line arguments or a config file). Handling of shebang lines 157 remains unchanged - "python" refers to Python 2 in that case. 158 159 * ``python.exe`` and ``pythonw.exe`` have been marked as long-path aware, 160 which means that the 260 character path limit may no longer apply. 161 See :ref:`removing the MAX_PATH limitation <max-path>` for details. 162 163 * A ``._pth`` file can be added to force isolated mode and fully specify 164 all search paths to avoid registry and environment lookup. See 165 :ref:`the documentation <finding_modules>` for more information. 166 167 * A ``python36.zip`` file now works as a landmark to infer 168 :envvar:`PYTHONHOME`. See :ref:`the documentation <finding_modules>` for 169 more information. 170 171 172 .. _PyPy dict implementation: https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html 173 174 175 New Features 176 ============ 177 178 .. _whatsnew36-pep498: 179 180 PEP 498: Formatted string literals 181 ---------------------------------- 182 183 :pep:`498` introduces a new kind of string literals: *f-strings*, or 184 :ref:`formatted string literals <f-strings>`. 185 186 Formatted string literals are prefixed with ``'f'`` and are similar to 187 the format strings accepted by :meth:`str.format`. They contain replacement 188 fields surrounded by curly braces. The replacement fields are expressions, 189 which are evaluated at run time, and then formatted using the 190 :func:`format` protocol:: 191 192 >>> name = "Fred" 193 >>> f"He said his name is {name}." 194 'He said his name is Fred.' 195 >>> width = 10 196 >>> precision = 4 197 >>> value = decimal.Decimal("12.34567") 198 >>> f"result: {value:{width}.{precision}}" # nested fields 199 'result: 12.35' 200 201 .. seealso:: 202 203 :pep:`498` -- Literal String Interpolation. 204 PEP written and implemented by Eric V. Smith. 205 206 :ref:`Feature documentation <f-strings>`. 207 208 209 .. _whatsnew36-pep526: 210 211 PEP 526: Syntax for variable annotations 212 ---------------------------------------- 213 214 :pep:`484` introduced the standard for type annotations of function 215 parameters, a.k.a. type hints. This PEP adds syntax to Python for annotating 216 the types of variables including class variables and instance variables:: 217 218 primes: List[int] = [] 219 220 captain: str # Note: no initial value! 221 222 class Starship: 223 stats: Dict[str, int] = {} 224 225 Just as for function annotations, the Python interpreter does not attach any 226 particular meaning to variable annotations and only stores them in the 227 ``__annotations__`` attribute of a class or module. 228 229 In contrast to variable declarations in statically typed languages, 230 the goal of annotation syntax is to provide an easy way to specify structured 231 type metadata for third party tools and libraries via the abstract syntax tree 232 and the ``__annotations__`` attribute. 233 234 .. seealso:: 235 236 :pep:`526` -- Syntax for variable annotations. 237 PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach, 238 and Guido van Rossum. Implemented by Ivan Levkivskyi. 239 240 Tools that use or will use the new syntax: 241 `mypy <http://www.mypy-lang.org/>`_, 242 `pytype <https://github.com/google/pytype>`_, PyCharm, etc. 243 244 245 .. _whatsnew36-pep515: 246 247 PEP 515: Underscores in Numeric Literals 248 ---------------------------------------- 249 250 :pep:`515` adds the ability to use underscores in numeric literals for 251 improved readability. For example:: 252 253 >>> 1_000_000_000_000_000 254 1000000000000000 255 >>> 0x_FF_FF_FF_FF 256 4294967295 257 258 Single underscores are allowed between digits and after any base 259 specifier. Leading, trailing, or multiple underscores in a row are not 260 allowed. 261 262 The :ref:`string formatting <formatspec>` language also now has support 263 for the ``'_'`` option to signal the use of an underscore for a thousands 264 separator for floating point presentation types and for integer 265 presentation type ``'d'``. For integer presentation types ``'b'``, 266 ``'o'``, ``'x'``, and ``'X'``, underscores will be inserted every 4 267 digits:: 268 269 >>> '{:_}'.format(1000000) 270 '1_000_000' 271 >>> '{:_x}'.format(0xFFFFFFFF) 272 'ffff_ffff' 273 274 .. seealso:: 275 276 :pep:`515` -- Underscores in Numeric Literals 277 PEP written by Georg Brandl and Serhiy Storchaka. 278 279 280 .. _whatsnew36-pep525: 281 282 PEP 525: Asynchronous Generators 283 -------------------------------- 284 285 :pep:`492` introduced support for native coroutines and ``async`` / ``await`` 286 syntax to Python 3.5. A notable limitation of the Python 3.5 implementation 287 is that it was not possible to use ``await`` and ``yield`` in the same 288 function body. In Python 3.6 this restriction has been lifted, making it 289 possible to define *asynchronous generators*:: 290 291 async def ticker(delay, to): 292 """Yield numbers from 0 to *to* every *delay* seconds.""" 293 for i in range(to): 294 yield i 295 await asyncio.sleep(delay) 296 297 The new syntax allows for faster and more concise code. 298 299 .. seealso:: 300 301 :pep:`525` -- Asynchronous Generators 302 PEP written and implemented by Yury Selivanov. 303 304 305 .. _whatsnew36-pep530: 306 307 PEP 530: Asynchronous Comprehensions 308 ------------------------------------ 309 310 :pep:`530` adds support for using ``async for`` in list, set, dict 311 comprehensions and generator expressions:: 312 313 result = [i async for i in aiter() if i % 2] 314 315 Additionally, ``await`` expressions are supported in all kinds 316 of comprehensions:: 317 318 result = [await fun() for fun in funcs if await condition()] 319 320 .. seealso:: 321 322 :pep:`530` -- Asynchronous Comprehensions 323 PEP written and implemented by Yury Selivanov. 324 325 326 .. _whatsnew36-pep487: 327 328 PEP 487: Simpler customization of class creation 329 ------------------------------------------------ 330 331 It is now possible to customize subclass creation without using a metaclass. 332 The new ``__init_subclass__`` classmethod will be called on the base class 333 whenever a new subclass is created:: 334 335 class PluginBase: 336 subclasses = [] 337 338 def __init_subclass__(cls, **kwargs): 339 super().__init_subclass__(**kwargs) 340 cls.subclasses.append(cls) 341 342 class Plugin1(PluginBase): 343 pass 344 345 class Plugin2(PluginBase): 346 pass 347 348 In order to allow zero-argument :func:`super` calls to work correctly from 349 :meth:`~object.__init_subclass__` implementations, custom metaclasses must 350 ensure that the new ``__classcell__`` namespace entry is propagated to 351 ``type.__new__`` (as described in :ref:`class-object-creation`). 352 353 .. seealso:: 354 355 :pep:`487` -- Simpler customization of class creation 356 PEP written and implemented by Martin Teichmann. 357 358 :ref:`Feature documentation <class-customization>` 359 360 361 .. _whatsnew36-pep487-descriptors: 362 363 PEP 487: Descriptor Protocol Enhancements 364 ----------------------------------------- 365 366 :pep:`487` extends the descriptor protocol to include the new optional 367 :meth:`~object.__set_name__` method. Whenever a new class is defined, the new 368 method will be called on all descriptors included in the definition, providing 369 them with a reference to the class being defined and the name given to the 370 descriptor within the class namespace. In other words, instances of 371 descriptors can now know the attribute name of the descriptor in the 372 owner class:: 373 374 class IntField: 375 def __get__(self, instance, owner): 376 return instance.__dict__[self.name] 377 378 def __set__(self, instance, value): 379 if not isinstance(value, int): 380 raise ValueError(f'expecting integer in {self.name}') 381 instance.__dict__[self.name] = value 382 383 # this is the new initializer: 384 def __set_name__(self, owner, name): 385 self.name = name 386 387 class Model: 388 int_field = IntField() 389 390 391 .. seealso:: 392 393 :pep:`487` -- Simpler customization of class creation 394 PEP written and implemented by Martin Teichmann. 395 396 :ref:`Feature documentation <descriptors>` 397 398 399 .. _whatsnew36-pep519: 400 401 PEP 519: Adding a file system path protocol 402 ------------------------------------------- 403 404 File system paths have historically been represented as :class:`str` 405 or :class:`bytes` objects. This has led to people who write code which 406 operate on file system paths to assume that such objects are only one 407 of those two types (an :class:`int` representing a file descriptor 408 does not count as that is not a file path). Unfortunately that 409 assumption prevents alternative object representations of file system 410 paths like :mod:`pathlib` from working with pre-existing code, 411 including Python's standard library. 412 413 To fix this situation, a new interface represented by 414 :class:`os.PathLike` has been defined. By implementing the 415 :meth:`~os.PathLike.__fspath__` method, an object signals that it 416 represents a path. An object can then provide a low-level 417 representation of a file system path as a :class:`str` or 418 :class:`bytes` object. This means an object is considered 419 :term:`path-like <path-like object>` if it implements 420 :class:`os.PathLike` or is a :class:`str` or :class:`bytes` object 421 which represents a file system path. Code can use :func:`os.fspath`, 422 :func:`os.fsdecode`, or :func:`os.fsencode` to explicitly get a 423 :class:`str` and/or :class:`bytes` representation of a path-like 424 object. 425 426 The built-in :func:`open` function has been updated to accept 427 :class:`os.PathLike` objects, as have all relevant functions in the 428 :mod:`os` and :mod:`os.path` modules, and most other functions and 429 classes in the standard library. The :class:`os.DirEntry` class 430 and relevant classes in :mod:`pathlib` have also been updated to 431 implement :class:`os.PathLike`. 432 433 The hope is that updating the fundamental functions for operating 434 on file system paths will lead to third-party code to implicitly 435 support all :term:`path-like objects <path-like object>` without any 436 code changes, or at least very minimal ones (e.g. calling 437 :func:`os.fspath` at the beginning of code before operating on a 438 path-like object). 439 440 Here are some examples of how the new interface allows for 441 :class:`pathlib.Path` to be used more easily and transparently with 442 pre-existing code:: 443 444 >>> import pathlib 445 >>> with open(pathlib.Path("README")) as f: 446 ... contents = f.read() 447 ... 448 >>> import os.path 449 >>> os.path.splitext(pathlib.Path("some_file.txt")) 450 ('some_file', '.txt') 451 >>> os.path.join("/a/b", pathlib.Path("c")) 452 '/a/b/c' 453 >>> import os 454 >>> os.fspath(pathlib.Path("some_file.txt")) 455 'some_file.txt' 456 457 (Implemented by Brett Cannon, Ethan Furman, Dusty Phillips, and Jelle Zijlstra.) 458 459 .. seealso:: 460 461 :pep:`519` -- Adding a file system path protocol 462 PEP written by Brett Cannon and Koos Zevenhoven. 463 464 465 .. _whatsnew36-pep495: 466 467 PEP 495: Local Time Disambiguation 468 ---------------------------------- 469 470 In most world locations, there have been and will be times when local clocks 471 are moved back. In those times, intervals are introduced in which local 472 clocks show the same time twice in the same day. In these situations, the 473 information displayed on a local clock (or stored in a Python datetime 474 instance) is insufficient to identify a particular moment in time. 475 476 :pep:`495` adds the new *fold* attribute to instances of 477 :class:`datetime.datetime` and :class:`datetime.time` classes to differentiate 478 between two moments in time for which local times are the same:: 479 480 >>> u0 = datetime(2016, 11, 6, 4, tzinfo=timezone.utc) 481 >>> for i in range(4): 482 ... u = u0 + i*HOUR 483 ... t = u.astimezone(Eastern) 484 ... print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold) 485 ... 486 04:00:00 UTC = 00:00:00 EDT 0 487 05:00:00 UTC = 01:00:00 EDT 0 488 06:00:00 UTC = 01:00:00 EST 1 489 07:00:00 UTC = 02:00:00 EST 0 490 491 The values of the :attr:`fold <datetime.datetime.fold>` attribute have the 492 value ``0`` for all instances except those that represent the second 493 (chronologically) moment in time in an ambiguous case. 494 495 .. seealso:: 496 497 :pep:`495` -- Local Time Disambiguation 498 PEP written by Alexander Belopolsky and Tim Peters, implementation 499 by Alexander Belopolsky. 500 501 502 .. _whatsnew36-pep529: 503 504 PEP 529: Change Windows filesystem encoding to UTF-8 505 ---------------------------------------------------- 506 507 Representing filesystem paths is best performed with str (Unicode) rather than 508 bytes. However, there are some situations where using bytes is sufficient and 509 correct. 510 511 Prior to Python 3.6, data loss could result when using bytes paths on Windows. 512 With this change, using bytes to represent paths is now supported on Windows, 513 provided those bytes are encoded with the encoding returned by 514 :func:`sys.getfilesystemencoding()`, which now defaults to ``'utf-8'``. 515 516 Applications that do not use str to represent paths should use 517 :func:`os.fsencode()` and :func:`os.fsdecode()` to ensure their bytes are 518 correctly encoded. To revert to the previous behaviour, set 519 :envvar:`PYTHONLEGACYWINDOWSFSENCODING` or call 520 :func:`sys._enablelegacywindowsfsencoding`. 521 522 See :pep:`529` for more information and discussion of code modifications that 523 may be required. 524 525 526 .. _whatsnew36-pep528: 527 528 PEP 528: Change Windows console encoding to UTF-8 529 ------------------------------------------------- 530 531 The default console on Windows will now accept all Unicode characters and 532 provide correctly read str objects to Python code. ``sys.stdin``, 533 ``sys.stdout`` and ``sys.stderr`` now default to utf-8 encoding. 534 535 This change only applies when using an interactive console, and not when 536 redirecting files or pipes. To revert to the previous behaviour for interactive 537 console use, set :envvar:`PYTHONLEGACYWINDOWSSTDIO`. 538 539 .. seealso:: 540 541 :pep:`528` -- Change Windows console encoding to UTF-8 542 PEP written and implemented by Steve Dower. 543 544 545 .. _whatsnew36-pep520: 546 547 PEP 520: Preserving Class Attribute Definition Order 548 ---------------------------------------------------- 549 550 Attributes in a class definition body have a natural ordering: the same 551 order in which the names appear in the source. This order is now 552 preserved in the new class's :attr:`~object.__dict__` attribute. 553 554 Also, the effective default class *execution* namespace (returned from 555 :ref:`type.__prepare__() <prepare>`) is now an insertion-order-preserving 556 mapping. 557 558 .. seealso:: 559 560 :pep:`520` -- Preserving Class Attribute Definition Order 561 PEP written and implemented by Eric Snow. 562 563 564 .. _whatsnew36-pep468: 565 566 PEP 468: Preserving Keyword Argument Order 567 ------------------------------------------ 568 569 ``**kwargs`` in a function signature is now guaranteed to be an 570 insertion-order-preserving mapping. 571 572 .. seealso:: 573 574 :pep:`468` -- Preserving Keyword Argument Order 575 PEP written and implemented by Eric Snow. 576 577 578 .. _whatsnew36-compactdict: 579 580 New :ref:`dict <typesmapping>` implementation 581 --------------------------------------------- 582 583 The :ref:`dict <typesmapping>` type now uses a "compact" representation 584 based on `a proposal by Raymond Hettinger 585 <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_ 586 which was `first implemented by PyPy 587 <https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_. 588 The memory usage of the new :func:`dict` is between 20% and 25% smaller 589 compared to Python 3.5. 590 591 The order-preserving aspect of this new implementation is considered an 592 implementation detail and should not be relied upon (this may change in 593 the future, but it is desired to have this new dict implementation in 594 the language for a few releases before changing the language spec to mandate 595 order-preserving semantics for all current and future Python 596 implementations; this also helps preserve backwards-compatibility 597 with older versions of the language where random iteration order is 598 still in effect, e.g. Python 3.5). 599 600 (Contributed by INADA Naoki in :issue:`27350`. Idea 601 `originally suggested by Raymond Hettinger 602 <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.) 603 604 605 .. _whatsnew36-pep523: 606 607 PEP 523: Adding a frame evaluation API to CPython 608 ------------------------------------------------- 609 610 While Python provides extensive support to customize how code 611 executes, one place it has not done so is in the evaluation of frame 612 objects. If you wanted some way to intercept frame evaluation in 613 Python there really wasn't any way without directly manipulating 614 function pointers for defined functions. 615 616 :pep:`523` changes this by providing an API to make frame 617 evaluation pluggable at the C level. This will allow for tools such 618 as debuggers and JITs to intercept frame evaluation before the 619 execution of Python code begins. This enables the use of alternative 620 evaluation implementations for Python code, tracking frame 621 evaluation, etc. 622 623 This API is not part of the limited C API and is marked as private to 624 signal that usage of this API is expected to be limited and only 625 applicable to very select, low-level use-cases. Semantics of the 626 API will change with Python as necessary. 627 628 .. seealso:: 629 630 :pep:`523` -- Adding a frame evaluation API to CPython 631 PEP written by Brett Cannon and Dino Viehland. 632 633 634 .. _whatsnew36-pythonmalloc: 635 636 PYTHONMALLOC environment variable 637 --------------------------------- 638 639 The new :envvar:`PYTHONMALLOC` environment variable allows setting the Python 640 memory allocators and installing debug hooks. 641 642 It is now possible to install debug hooks on Python memory allocators on Python 643 compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks: 644 645 * Newly allocated memory is filled with the byte ``0xCB`` 646 * Freed memory is filled with the byte ``0xDB`` 647 * Detect violations of the Python memory allocator API. For example, 648 :c:func:`PyObject_Free` called on a memory block allocated by 649 :c:func:`PyMem_Malloc`. 650 * Detect writes before the start of a buffer (buffer underflows) 651 * Detect writes after the end of a buffer (buffer overflows) 652 * Check that the :term:`GIL <global interpreter lock>` is held when allocator 653 functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and 654 :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. 655 656 Checking if the GIL is held is also a new feature of Python 3.6. 657 658 See the :c:func:`PyMem_SetupDebugHooks` function for debug hooks on Python 659 memory allocators. 660 661 It is now also possible to force the usage of the :c:func:`malloc` allocator of 662 the C library for all Python memory allocations using ``PYTHONMALLOC=malloc``. 663 This is helpful when using external memory debuggers like Valgrind on 664 a Python compiled in release mode. 665 666 On error, the debug hooks on Python memory allocators now use the 667 :mod:`tracemalloc` module to get the traceback where a memory block was 668 allocated. 669 670 Example of fatal error on buffer overflow using 671 ``python3.6 -X tracemalloc=5`` (store 5 frames in traces):: 672 673 Debug memory block at address p=0x7fbcd41666f8: API 'o' 674 4 bytes originally requested 675 The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected. 676 The 8 pad bytes at tail=0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb): 677 at tail+0: 0x02 *** OUCH 678 at tail+1: 0xfb 679 at tail+2: 0xfb 680 at tail+3: 0xfb 681 at tail+4: 0xfb 682 at tail+5: 0xfb 683 at tail+6: 0xfb 684 at tail+7: 0xfb 685 The block was made by call #1233329 to debug malloc/realloc. 686 Data at p: 1a 2b 30 00 687 688 Memory block allocated at (most recent call first): 689 File "test/test_bytes.py", line 323 690 File "unittest/case.py", line 600 691 File "unittest/case.py", line 648 692 File "unittest/suite.py", line 122 693 File "unittest/suite.py", line 84 694 695 Fatal Python error: bad trailing pad byte 696 697 Current thread 0x00007fbcdbd32700 (most recent call first): 698 File "test/test_bytes.py", line 323 in test_hex 699 File "unittest/case.py", line 600 in run 700 File "unittest/case.py", line 648 in __call__ 701 File "unittest/suite.py", line 122 in run 702 File "unittest/suite.py", line 84 in __call__ 703 File "unittest/suite.py", line 122 in run 704 File "unittest/suite.py", line 84 in __call__ 705 ... 706 707 (Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.) 708 709 710 .. _whatsnew36-tracing: 711 712 DTrace and SystemTap probing support 713 ------------------------------------ 714 715 Python can now be built ``--with-dtrace`` which enables static markers 716 for the following events in the interpreter: 717 718 * function call/return 719 720 * garbage collection started/finished 721 722 * line of code executed. 723 724 This can be used to instrument running interpreters in production, 725 without the need to recompile specific debug builds or providing 726 application-specific profiling/debugging code. 727 728 More details in :ref:`instrumentation`. 729 730 The current implementation is tested on Linux and macOS. Additional 731 markers may be added in the future. 732 733 (Contributed by ukasz Langa in :issue:`21590`, based on patches by 734 Jess Cea Avin, David Malcolm, and Nikhil Benesch.) 735 736 737 Other Language Changes 738 ====================== 739 740 Some smaller changes made to the core Python language are: 741 742 * A ``global`` or ``nonlocal`` statement must now textually appear 743 before the first use of the affected name in the same scope. 744 Previously this was a :exc:`SyntaxWarning`. 745 746 * It is now possible to set a :ref:`special method <specialnames>` to 747 ``None`` to indicate that the corresponding operation is not available. 748 For example, if a class sets :meth:`__iter__` to ``None``, the class 749 is not iterable. 750 (Contributed by Andrew Barnert and Ivan Levkivskyi in :issue:`25958`.) 751 752 * Long sequences of repeated traceback lines are now abbreviated as 753 ``"[Previous line repeated {count} more times]"`` (see 754 :ref:`whatsnew36-traceback` for an example). 755 (Contributed by Emanuel Barry in :issue:`26823`.) 756 757 * Import now raises the new exception :exc:`ModuleNotFoundError` 758 (subclass of :exc:`ImportError`) when it cannot find a module. Code 759 that currently checks for ImportError (in try-except) will still work. 760 (Contributed by Eric Snow in :issue:`15767`.) 761 762 * Class methods relying on zero-argument ``super()`` will now work correctly 763 when called from metaclass methods during class creation. 764 (Contributed by Martin Teichmann in :issue:`23722`.) 765 766 767 New Modules 768 =========== 769 770 .. _whatsnew36-pep506: 771 772 secrets 773 ------- 774 775 The main purpose of the new :mod:`secrets` module is to provide an obvious way 776 to reliably generate cryptographically strong pseudo-random values suitable 777 for managing secrets, such as account authentication, tokens, and similar. 778 779 .. warning:: 780 781 Note that the pseudo-random generators in the :mod:`random` module 782 should *NOT* be used for security purposes. Use :mod:`secrets` 783 on Python 3.6+ and :func:`os.urandom()` on Python 3.5 and earlier. 784 785 .. seealso:: 786 787 :pep:`506` -- Adding A Secrets Module To The Standard Library 788 PEP written and implemented by Steven D'Aprano. 789 790 791 Improved Modules 792 ================ 793 794 array 795 ----- 796 797 Exhausted iterators of :class:`array.array` will now stay exhausted even 798 if the iterated array is extended. This is consistent with the behavior 799 of other mutable sequences. 800 801 Contributed by Serhiy Storchaka in :issue:`26492`. 802 803 ast 804 --- 805 806 The new :class:`ast.Constant` AST node has been added. It can be used 807 by external AST optimizers for the purposes of constant folding. 808 809 Contributed by Victor Stinner in :issue:`26146`. 810 811 812 asyncio 813 ------- 814 815 Starting with Python 3.6 the ``asyncio`` module is no longer provisional and its 816 API is considered stable. 817 818 Notable changes in the :mod:`asyncio` module since Python 3.5.0 819 (all backported to 3.5.x due to the provisional status): 820 821 * The :func:`~asyncio.get_event_loop` function has been changed to 822 always return the currently running loop when called from coroutines 823 and callbacks. 824 (Contributed by Yury Selivanov in :issue:`28613`.) 825 826 * The :func:`~asyncio.ensure_future` function and all functions that 827 use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`, 828 now accept all kinds of :term:`awaitable objects <awaitable>`. 829 (Contributed by Yury Selivanov.) 830 831 * New :func:`~asyncio.run_coroutine_threadsafe` function to submit 832 coroutines to event loops from other threads. 833 (Contributed by Vincent Michel.) 834 835 * New :meth:`Transport.is_closing() <asyncio.BaseTransport.is_closing>` 836 method to check if the transport is closing or closed. 837 (Contributed by Yury Selivanov.) 838 839 * The :meth:`loop.create_server() <asyncio.loop.create_server>` 840 method can now accept a list of hosts. 841 (Contributed by Yann Sionneau.) 842 843 * New :meth:`loop.create_future() <asyncio.loop.create_future>` 844 method to create Future objects. This allows alternative event 845 loop implementations, such as 846 `uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster 847 :class:`asyncio.Future` implementation. 848 (Contributed by Yury Selivanov in :issue:`27041`.) 849 850 * New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>` 851 method to get the current exception handler. 852 (Contributed by Yury Selivanov in :issue:`27040`.) 853 854 * New :meth:`StreamReader.readuntil() <asyncio.StreamReader.readuntil>` 855 method to read data from the stream until a separator bytes 856 sequence appears. 857 (Contributed by Mark Korenberg.) 858 859 * The performance of :meth:`StreamReader.readexactly() <asyncio.StreamReader.readexactly>` 860 has been improved. 861 (Contributed by Mark Korenberg in :issue:`28370`.) 862 863 * The :meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>` 864 method is optimized to avoid calling the system ``getaddrinfo`` 865 function if the address is already resolved. 866 (Contributed by A. Jesse Jiryu Davis.) 867 868 * The :meth:`loop.stop() <asyncio.loop.stop>` 869 method has been changed to stop the loop immediately after 870 the current iteration. Any new callbacks scheduled as a result 871 of the last iteration will be discarded. 872 (Contributed by Guido van Rossum in :issue:`25593`.) 873 874 * :meth:`Future.set_exception <asyncio.futures.Future.set_exception>` 875 will now raise :exc:`TypeError` when passed an instance of 876 the :exc:`StopIteration` exception. 877 (Contributed by Chris Angelico in :issue:`26221`.) 878 879 * New :meth:`loop.connect_accepted_socket() <asyncio.loop.connect_accepted_socket>` 880 method to be used by servers that accept connections outside of asyncio, 881 but that use asyncio to handle them. 882 (Contributed by Jim Fulton in :issue:`27392`.) 883 884 * ``TCP_NODELAY`` flag is now set for all TCP transports by default. 885 (Contributed by Yury Selivanov in :issue:`27456`.) 886 887 * New :meth:`loop.shutdown_asyncgens() <asyncio.loop.shutdown_asyncgens>` 888 to properly close pending asynchronous generators before closing the 889 loop. 890 (Contributed by Yury Selivanov in :issue:`28003`.) 891 892 * :class:`Future <asyncio.Future>` and :class:`Task <asyncio.Task>` 893 classes now have an optimized C implementation which makes asyncio 894 code up to 30% faster. 895 (Contributed by Yury Selivanov and INADA Naoki in :issue:`26081` 896 and :issue:`28544`.) 897 898 899 binascii 900 -------- 901 902 The :func:`~binascii.b2a_base64` function now accepts an optional *newline* 903 keyword argument to control whether the newline character is appended to the 904 return value. 905 (Contributed by Victor Stinner in :issue:`25357`.) 906 907 908 cmath 909 ----- 910 911 The new :const:`cmath.tau` (**) constant has been added. 912 (Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.) 913 914 New constants: :const:`cmath.inf` and :const:`cmath.nan` to 915 match :const:`math.inf` and :const:`math.nan`, and also :const:`cmath.infj` 916 and :const:`cmath.nanj` to match the format used by complex repr. 917 (Contributed by Mark Dickinson in :issue:`23229`.) 918 919 920 collections 921 ----------- 922 923 The new :class:`~collections.abc.Collection` abstract base class has been 924 added to represent sized iterable container classes. 925 (Contributed by Ivan Levkivskyi, docs by Neil Girdhar in :issue:`27598`.) 926 927 The new :class:`~collections.abc.Reversible` abstract base class represents 928 iterable classes that also provide the :meth:`__reversed__` method. 929 (Contributed by Ivan Levkivskyi in :issue:`25987`.) 930 931 The new :class:`~collections.abc.AsyncGenerator` abstract base class represents 932 asynchronous generators. 933 (Contributed by Yury Selivanov in :issue:`28720`.) 934 935 The :func:`~collections.namedtuple` function now accepts an optional 936 keyword argument *module*, which, when specified, is used for 937 the ``__module__`` attribute of the returned named tuple class. 938 (Contributed by Raymond Hettinger in :issue:`17941`.) 939 940 The *verbose* and *rename* arguments for 941 :func:`~collections.namedtuple` are now keyword-only. 942 (Contributed by Raymond Hettinger in :issue:`25628`.) 943 944 Recursive :class:`collections.deque` instances can now be pickled. 945 (Contributed by Serhiy Storchaka in :issue:`26482`.) 946 947 948 concurrent.futures 949 ------------------ 950 951 The :class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>` 952 class constructor now accepts an optional *thread_name_prefix* argument 953 to make it possible to customize the names of the threads created by the 954 pool. 955 (Contributed by Gregory P. Smith in :issue:`27664`.) 956 957 958 contextlib 959 ---------- 960 961 The :class:`contextlib.AbstractContextManager` class has been added to 962 provide an abstract base class for context managers. It provides a 963 sensible default implementation for `__enter__()` which returns 964 ``self`` and leaves `__exit__()` an abstract method. A matching 965 class has been added to the :mod:`typing` module as 966 :class:`typing.ContextManager`. 967 (Contributed by Brett Cannon in :issue:`25609`.) 968 969 970 datetime 971 -------- 972 973 The :class:`~datetime.datetime` and :class:`~datetime.time` classes have 974 the new :attr:`~time.fold` attribute used to disambiguate local time 975 when necessary. Many functions in the :mod:`datetime` have been 976 updated to support local time disambiguation. 977 See :ref:`Local Time Disambiguation <whatsnew36-pep495>` section for more 978 information. 979 (Contributed by Alexander Belopolsky in :issue:`24773`.) 980 981 The :meth:`datetime.strftime() <datetime.datetime.strftime>` and 982 :meth:`date.strftime() <datetime.date.strftime>` methods now support 983 ISO 8601 date directives ``%G``, ``%u`` and ``%V``. 984 (Contributed by Ashley Anderson in :issue:`12006`.) 985 986 The :func:`datetime.isoformat() <datetime.datetime.isoformat>` function 987 now accepts an optional *timespec* argument that specifies the number 988 of additional components of the time value to include. 989 (Contributed by Alessandro Cucci and Alexander Belopolsky in :issue:`19475`.) 990 991 The :meth:`datetime.combine() <datetime.datetime.combine>` now 992 accepts an optional *tzinfo* argument. 993 (Contributed by Alexander Belopolsky in :issue:`27661`.) 994 995 996 decimal 997 ------- 998 999 New :meth:`Decimal.as_integer_ratio() <decimal.Decimal.as_integer_ratio>` 1000 method that returns a pair ``(n, d)`` of integers that represent the given 1001 :class:`~decimal.Decimal` instance as a fraction, in lowest terms and 1002 with a positive denominator:: 1003 1004 >>> Decimal('-3.14').as_integer_ratio() 1005 (-157, 50) 1006 1007 (Contributed by Stefan Krah amd Mark Dickinson in :issue:`25928`.) 1008 1009 1010 1011 distutils 1012 --------- 1013 1014 The ``default_format`` attribute has been removed from 1015 :class:`distutils.command.sdist.sdist` and the ``formats`` 1016 attribute defaults to ``['gztar']``. Although not anticipated, 1017 any code relying on the presence of ``default_format`` may 1018 need to be adapted. See :issue:`27819` for more details. 1019 1020 1021 email 1022 ----- 1023 1024 The new email API, enabled via the *policy* keyword to various constructors, is 1025 no longer provisional. The :mod:`email` documentation has been reorganized and 1026 rewritten to focus on the new API, while retaining the old documentation for 1027 the legacy API. (Contributed by R. David Murray in :issue:`24277`.) 1028 1029 The :mod:`email.mime` classes now all accept an optional *policy* keyword. 1030 (Contributed by Berker Peksag in :issue:`27331`.) 1031 1032 The :class:`~email.generator.DecodedGenerator` now supports the *policy* 1033 keyword. 1034 1035 There is a new :mod:`~email.policy` attribute, 1036 :attr:`~email.policy.Policy.message_factory`, that controls what class is used 1037 by default when the parser creates new message objects. For the 1038 :attr:`email.policy.compat32` policy this is :class:`~email.message.Message`, 1039 for the new policies it is :class:`~email.message.EmailMessage`. 1040 (Contributed by R. David Murray in :issue:`20476`.) 1041 1042 1043 encodings 1044 --------- 1045 1046 On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP``, and the ``'ansi'`` 1047 alias for the existing ``'mbcs'`` encoding, which uses the ``CP_ACP`` code page. 1048 (Contributed by Steve Dower in :issue:`27959`.) 1049 1050 1051 enum 1052 ---- 1053 1054 Two new enumeration base classes have been added to the :mod:`enum` module: 1055 :class:`~enum.Flag` and :class:`~enum.IntFlags`. Both are used to define 1056 constants that can be combined using the bitwise operators. 1057 (Contributed by Ethan Furman in :issue:`23591`.) 1058 1059 Many standard library modules have been updated to use the 1060 :class:`~enum.IntFlags` class for their constants. 1061 1062 The new :class:`enum.auto` value can be used to assign values to enum 1063 members automatically:: 1064 1065 >>> from enum import Enum, auto 1066 >>> class Color(Enum): 1067 ... red = auto() 1068 ... blue = auto() 1069 ... green = auto() 1070 ... 1071 >>> list(Color) 1072 [<Color.red: 1>, <Color.blue: 2>, <Color.green: 3>] 1073 1074 1075 faulthandler 1076 ------------ 1077 1078 On Windows, the :mod:`faulthandler` module now installs a handler for Windows 1079 exceptions: see :func:`faulthandler.enable`. (Contributed by Victor Stinner in 1080 :issue:`23848`.) 1081 1082 1083 fileinput 1084 --------- 1085 1086 :func:`~fileinput.hook_encoded` now supports the *errors* argument. 1087 (Contributed by Joseph Hackman in :issue:`25788`.) 1088 1089 1090 hashlib 1091 ------- 1092 1093 :mod:`hashlib` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. 1094 (Contributed by Christian Heimes in :issue:`26470`.) 1095 1096 BLAKE2 hash functions were added to the module. :func:`~hashlib.blake2b` 1097 and :func:`~hashlib.blake2s` are always available and support the full 1098 feature set of BLAKE2. 1099 (Contributed by Christian Heimes in :issue:`26798` based on code by 1100 Dmitry Chestnykh and Samuel Neves. Documentation written by Dmitry Chestnykh.) 1101 1102 The SHA-3 hash functions :func:`~hashlib.sha3_224`, :func:`~hashlib.sha3_256`, 1103 :func:`~hashlib.sha3_384`, :func:`~hashlib.sha3_512`, and SHAKE hash functions 1104 :func:`~hashlib.shake_128` and :func:`~hashlib.shake_256` were added. 1105 (Contributed by Christian Heimes in :issue:`16113`. Keccak Code Package 1106 by Guido Bertoni, Joan Daemen, Michal Peeters, Gilles Van Assche, and 1107 Ronny Van Keer.) 1108 1109 The password-based key derivation function :func:`~hashlib.scrypt` is now 1110 available with OpenSSL 1.1.0 and newer. 1111 (Contributed by Christian Heimes in :issue:`27928`.) 1112 1113 http.client 1114 ----------- 1115 1116 :meth:`HTTPConnection.request() <http.client.HTTPConnection.request>` and 1117 :meth:`~http.client.HTTPConnection.endheaders` both now support 1118 chunked encoding request bodies. 1119 (Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.) 1120 1121 1122 idlelib and IDLE 1123 ---------------- 1124 1125 The idlelib package is being modernized and refactored to make IDLE look and 1126 work better and to make the code easier to understand, test, and improve. Part 1127 of making IDLE look better, especially on Linux and Mac, is using ttk widgets, 1128 mostly in the dialogs. As a result, IDLE no longer runs with tcl/tk 8.4. It 1129 now requires tcl/tk 8.5 or 8.6. We recommend running the latest release of 1130 either. 1131 1132 'Modernizing' includes renaming and consolidation of idlelib modules. The 1133 renaming of files with partial uppercase names is similar to the renaming of, 1134 for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0. As a 1135 result, imports of idlelib files that worked in 3.5 will usually not work in 1136 3.6. At least a module name change will be needed (see idlelib/README.txt), 1137 sometimes more. (Name changes contributed by Al Swiegart and Terry Reedy in 1138 :issue:`24225`. Most idlelib patches since have been and will be part of the 1139 process.) 1140 1141 In compensation, the eventual result with be that some idlelib classes will be 1142 easier to use, with better APIs and docstrings explaining them. Additional 1143 useful information will be added to idlelib when available. 1144 1145 New in 3.6.2: 1146 1147 Multiple fixes for autocompletion. (Contributed by Louie Lu in :issue:`15786`.) 1148 1149 New in 3.6.3: 1150 1151 Module Browser (on the File menu, formerly called Class Browser), 1152 now displays nested functions and classes in addition to top-level 1153 functions and classes. 1154 (Contributed by Guilherme Polo, Cheryl Sabella, and Terry Jan Reedy 1155 in :issue:`1612262`.) 1156 1157 The IDLE features formerly implemented as extensions have been reimplemented 1158 as normal features. Their settings have been moved from the Extensions tab 1159 to other dialog tabs. 1160 (Contributed by Charles Wohlganger and Terry Jan Reedy in :issue:`27099`.) 1161 1162 The Settings dialog (Options, Configure IDLE) has been partly rewritten 1163 to improve both appearance and function. 1164 (Contributed by Cheryl Sabella and Terry Jan Reedy in multiple issues.) 1165 1166 New in 3.6.4: 1167 1168 The font sample now includes a selection of non-Latin characters so that 1169 users can better see the effect of selecting a particular font. 1170 (Contributed by Terry Jan Reedy in :issue:`13802`.) 1171 The sample can be edited to include other characters. 1172 (Contributed by Serhiy Storchaka in :issue:`31860`.) 1173 1174 New in 3.6.6: 1175 1176 Editor code context option revised. Box displays all context lines up to 1177 maxlines. Clicking on a context line jumps the editor to that line. Context 1178 colors for custom themes is added to Highlights tab of Settings dialog. 1179 (Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`, 1180 :issue:`33768`, and :issue:`33679`.) 1181 1182 On Windows, a new API call tells Windows that tk scales for DPI. On Windows 1183 8.1+ or 10, with DPI compatibility properties of the Python binary 1184 unchanged, and a monitor resolution greater than 96 DPI, this should 1185 make text and lines sharper. It should otherwise have no effect. 1186 (Contributed by Terry Jan Reedy in :issue:`33656`.) 1187 1188 New in 3.6.7: 1189 1190 Output over N lines (50 by default) is squeezed down to a button. 1191 N can be changed in the PyShell section of the General page of the 1192 Settings dialog. Fewer, but possibly extra long, lines can be squeezed by 1193 right clicking on the output. Squeezed output can be expanded in place 1194 by double-clicking the button or into the clipboard or a separate window 1195 by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) 1196 1197 1198 importlib 1199 --------- 1200 1201 Import now raises the new exception :exc:`ModuleNotFoundError` 1202 (subclass of :exc:`ImportError`) when it cannot find a module. Code 1203 that current checks for ``ImportError`` (in try-except) will still work. 1204 (Contributed by Eric Snow in :issue:`15767`.) 1205 1206 :class:`importlib.util.LazyLoader` now calls 1207 :meth:`~importlib.abc.Loader.create_module` on the wrapped loader, removing the 1208 restriction that :class:`importlib.machinery.BuiltinImporter` and 1209 :class:`importlib.machinery.ExtensionFileLoader` couldn't be used with 1210 :class:`importlib.util.LazyLoader`. 1211 1212 :func:`importlib.util.cache_from_source`, 1213 :func:`importlib.util.source_from_cache`, and 1214 :func:`importlib.util.spec_from_file_location` now accept a 1215 :term:`path-like object`. 1216 1217 1218 inspect 1219 ------- 1220 1221 The :func:`inspect.signature() <inspect.signature>` function now reports the 1222 implicit ``.0`` parameters generated by the compiler for comprehension and 1223 generator expression scopes as if they were positional-only parameters called 1224 ``implicit0``. (Contributed by Jelle Zijlstra in :issue:`19611`.) 1225 1226 To reduce code churn when upgrading from Python 2.7 and the legacy 1227 :func:`inspect.getargspec` API, the previously documented deprecation of 1228 :func:`inspect.getfullargspec` has been reversed. While this function is 1229 convenient for single/source Python 2/3 code bases, the richer 1230 :func:`inspect.signature` interface remains the recommended approach for new 1231 code. (Contributed by Nick Coghlan in :issue:`27172`) 1232 1233 1234 json 1235 ---- 1236 1237 :func:`json.load` and :func:`json.loads` now support binary input. Encoded 1238 JSON should be represented using either UTF-8, UTF-16, or UTF-32. 1239 (Contributed by Serhiy Storchaka in :issue:`17909`.) 1240 1241 1242 logging 1243 ------- 1244 1245 The new :meth:`WatchedFileHandler.reopenIfNeeded() <logging.handlers.WatchedFileHandler.reopenIfNeeded>` 1246 method has been added to add the ability to check if the log file needs to 1247 be reopened. 1248 (Contributed by Marian Horban in :issue:`24884`.) 1249 1250 1251 math 1252 ---- 1253 1254 The tau (**) constant has been added to the :mod:`math` and :mod:`cmath` 1255 modules. 1256 (Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.) 1257 1258 1259 multiprocessing 1260 --------------- 1261 1262 :ref:`Proxy Objects <multiprocessing-proxy_objects>` returned by 1263 :func:`multiprocessing.Manager` can now be nested. 1264 (Contributed by Davin Potts in :issue:`6766`.) 1265 1266 1267 os 1268 -- 1269 1270 See the summary of :ref:`PEP 519 <whatsnew36-pep519>` for details on how the 1271 :mod:`os` and :mod:`os.path` modules now support 1272 :term:`path-like objects <path-like object>`. 1273 1274 :func:`~os.scandir` now supports :class:`bytes` paths on Windows. 1275 1276 A new :meth:`~os.scandir.close` method allows explicitly closing a 1277 :func:`~os.scandir` iterator. The :func:`~os.scandir` iterator now 1278 supports the :term:`context manager` protocol. If a :func:`scandir` 1279 iterator is neither exhausted nor explicitly closed a :exc:`ResourceWarning` 1280 will be emitted in its destructor. 1281 (Contributed by Serhiy Storchaka in :issue:`25994`.) 1282 1283 On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool 1284 is initialized to increase the security. See the :pep:`524` for the rationale. 1285 1286 The Linux ``getrandom()`` syscall (get random bytes) is now exposed as the new 1287 :func:`os.getrandom` function. 1288 (Contributed by Victor Stinner, part of the :pep:`524`) 1289 1290 1291 pathlib 1292 ------- 1293 1294 :mod:`pathlib` now supports :term:`path-like objects <path-like object>`. 1295 (Contributed by Brett Cannon in :issue:`27186`.) 1296 1297 See the summary of :ref:`PEP 519 <whatsnew36-pep519>` for details. 1298 1299 1300 pdb 1301 --- 1302 1303 The :class:`~pdb.Pdb` class constructor has a new optional *readrc* argument 1304 to control whether ``.pdbrc`` files should be read. 1305 1306 1307 pickle 1308 ------ 1309 1310 Objects that need ``__new__`` called with keyword arguments can now be pickled 1311 using :ref:`pickle protocols <pickle-protocols>` older than protocol version 4. 1312 Protocol version 4 already supports this case. (Contributed by Serhiy 1313 Storchaka in :issue:`24164`.) 1314 1315 1316 pickletools 1317 ----------- 1318 1319 :func:`pickletools.dis()` now outputs the implicit memo index for the 1320 ``MEMOIZE`` opcode. 1321 (Contributed by Serhiy Storchaka in :issue:`25382`.) 1322 1323 1324 pydoc 1325 ----- 1326 1327 The :mod:`pydoc` module has learned to respect the ``MANPAGER`` 1328 environment variable. 1329 (Contributed by Matthias Klose in :issue:`8637`.) 1330 1331 :func:`help` and :mod:`pydoc` can now list named tuple fields in the 1332 order they were defined rather than alphabetically. 1333 (Contributed by Raymond Hettinger in :issue:`24879`.) 1334 1335 1336 random 1337 ------- 1338 1339 The new :func:`~random.choices` function returns a list of elements of 1340 specified size from the given population with optional weights. 1341 (Contributed by Raymond Hettinger in :issue:`18844`.) 1342 1343 1344 re 1345 -- 1346 1347 Added support of modifier spans in regular expressions. Examples: 1348 ``'(?i:p)ython'`` matches ``'python'`` and ``'Python'``, but not ``'PYTHON'``; 1349 ``'(?i)g(?-i:v)r'`` matches ``'GvR'`` and ``'gvr'``, but not ``'GVR'``. 1350 (Contributed by Serhiy Storchaka in :issue:`433028`.) 1351 1352 Match object groups can be accessed by ``__getitem__``, which is 1353 equivalent to ``group()``. So ``mo['name']`` is now equivalent to 1354 ``mo.group('name')``. (Contributed by Eric Smith in :issue:`24454`.) 1355 1356 :class:`~re.Match` objects now support 1357 :meth:`index-like objects <object.__index__>` as group 1358 indices. 1359 (Contributed by Jeroen Demeyer and Xiang Zhang in :issue:`27177`.) 1360 1361 1362 readline 1363 -------- 1364 1365 Added :func:`~readline.set_auto_history` to enable or disable 1366 automatic addition of input to the history list. (Contributed by 1367 Tyler Crompton in :issue:`26870`.) 1368 1369 1370 rlcompleter 1371 ----------- 1372 1373 Private and special attribute names now are omitted unless the prefix starts 1374 with underscores. A space or a colon is added after some completed keywords. 1375 (Contributed by Serhiy Storchaka in :issue:`25011` and :issue:`25209`.) 1376 1377 1378 shlex 1379 ----- 1380 1381 The :class:`~shlex.shlex` has much 1382 :ref:`improved shell compatibility <improved-shell-compatibility>` 1383 through the new *punctuation_chars* argument to control which characters 1384 are treated as punctuation. 1385 (Contributed by Vinay Sajip in :issue:`1521950`.) 1386 1387 1388 site 1389 ---- 1390 1391 When specifying paths to add to :attr:`sys.path` in a `.pth` file, 1392 you may now specify file paths on top of directories (e.g. zip files). 1393 (Contributed by Wolfgang Langner in :issue:`26587`). 1394 1395 1396 sqlite3 1397 ------- 1398 1399 :attr:`sqlite3.Cursor.lastrowid` now supports the ``REPLACE`` statement. 1400 (Contributed by Alex LordThorsen in :issue:`16864`.) 1401 1402 1403 socket 1404 ------ 1405 1406 The :func:`~socket.socket.ioctl` function now supports the 1407 :data:`~socket.SIO_LOOPBACK_FAST_PATH` control code. 1408 (Contributed by Daniel Stokes in :issue:`26536`.) 1409 1410 The :meth:`~socket.socket.getsockopt` constants ``SO_DOMAIN``, 1411 ``SO_PROTOCOL``, ``SO_PEERSEC``, and ``SO_PASSSEC`` are now supported. 1412 (Contributed by Christian Heimes in :issue:`26907`.) 1413 1414 The :meth:`~socket.socket.setsockopt` now supports the 1415 ``setsockopt(level, optname, None, optlen: int)`` form. 1416 (Contributed by Christian Heimes in :issue:`27744`.) 1417 1418 The socket module now supports the address family 1419 :data:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``, 1420 ``SOL_ALG`` and :meth:`~socket.socket.sendmsg_afalg` were added. 1421 (Contributed by Christian Heimes in :issue:`27744` with support from 1422 Victor Stinner.) 1423 1424 New Linux constants ``TCP_USER_TIMEOUT`` and ``TCP_CONGESTION`` were added. 1425 (Contributed by Omar Sandoval, issue:`26273`). 1426 1427 1428 socketserver 1429 ------------ 1430 1431 Servers based on the :mod:`socketserver` module, including those 1432 defined in :mod:`http.server`, :mod:`xmlrpc.server` and 1433 :mod:`wsgiref.simple_server`, now support the :term:`context manager` 1434 protocol. 1435 (Contributed by Aviv Palivoda in :issue:`26404`.) 1436 1437 The :attr:`~socketserver.StreamRequestHandler.wfile` attribute of 1438 :class:`~socketserver.StreamRequestHandler` classes now implements 1439 the :class:`io.BufferedIOBase` writable interface. In particular, 1440 calling :meth:`~io.BufferedIOBase.write` is now guaranteed to send the 1441 data in full. (Contributed by Martin Panter in :issue:`26721`.) 1442 1443 1444 ssl 1445 --- 1446 1447 :mod:`ssl` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. 1448 (Contributed by Christian Heimes in :issue:`26470`.) 1449 1450 3DES has been removed from the default cipher suites and ChaCha20 Poly1305 1451 cipher suites have been added. 1452 (Contributed by Christian Heimes in :issue:`27850` and :issue:`27766`.) 1453 1454 :class:`~ssl.SSLContext` has better default configuration for options 1455 and ciphers. 1456 (Contributed by Christian Heimes in :issue:`28043`.) 1457 1458 SSL session can be copied from one client-side connection to another 1459 with the new :class:`~ssl.SSLSession` class. TLS session resumption can 1460 speed up the initial handshake, reduce latency and improve performance 1461 (Contributed by Christian Heimes in :issue:`19500` based on a draft by 1462 Alex Warhawk.) 1463 1464 The new :meth:`~ssl.SSLContext.get_ciphers` method can be used to 1465 get a list of enabled ciphers in order of cipher priority. 1466 1467 All constants and flags have been converted to :class:`~enum.IntEnum` and 1468 :class:`~enum.IntFlags`. 1469 (Contributed by Christian Heimes in :issue:`28025`.) 1470 1471 Server and client-side specific TLS protocols for :class:`~ssl.SSLContext` 1472 were added. 1473 (Contributed by Christian Heimes in :issue:`28085`.) 1474 1475 1476 statistics 1477 ---------- 1478 1479 A new :func:`~statistics.harmonic_mean` function has been added. 1480 (Contributed by Steven D'Aprano in :issue:`27181`.) 1481 1482 1483 struct 1484 ------ 1485 1486 :mod:`struct` now supports IEEE 754 half-precision floats via the ``'e'`` 1487 format specifier. 1488 (Contributed by Eli Stevens, Mark Dickinson in :issue:`11734`.) 1489 1490 1491 subprocess 1492 ---------- 1493 1494 :class:`subprocess.Popen` destructor now emits a :exc:`ResourceWarning` warning 1495 if the child process is still running. Use the context manager protocol (``with 1496 proc: ...``) or explicitly call the :meth:`~subprocess.Popen.wait` method to 1497 read the exit status of the child process. (Contributed by Victor Stinner in 1498 :issue:`26741`.) 1499 1500 The :class:`subprocess.Popen` constructor and all functions that pass arguments 1501 through to it now accept *encoding* and *errors* arguments. Specifying either 1502 of these will enable text mode for the *stdin*, *stdout* and *stderr* streams. 1503 (Contributed by Steve Dower in :issue:`6135`.) 1504 1505 1506 sys 1507 --- 1508 1509 The new :func:`~sys.getfilesystemencodeerrors` function returns the name of 1510 the error mode used to convert between Unicode filenames and bytes filenames. 1511 (Contributed by Steve Dower in :issue:`27781`.) 1512 1513 On Windows the return value of the :func:`~sys.getwindowsversion` function 1514 now includes the *platform_version* field which contains the accurate major 1515 version, minor version and build number of the current operating system, 1516 rather than the version that is being emulated for the process 1517 (Contributed by Steve Dower in :issue:`27932`.) 1518 1519 1520 telnetlib 1521 --------- 1522 1523 :class:`~telnetlib.Telnet` is now a context manager (contributed by 1524 Stphane Wirtel in :issue:`25485`). 1525 1526 1527 time 1528 ---- 1529 1530 The :class:`~time.struct_time` attributes :attr:`tm_gmtoff` and 1531 :attr:`tm_zone` are now available on all platforms. 1532 1533 1534 timeit 1535 ------ 1536 1537 The new :meth:`Timer.autorange() <timeit.Timer.autorange>` convenience 1538 method has been added to call :meth:`Timer.timeit() <timeit.Timer.timeit>` 1539 repeatedly so that the total run time is greater or equal to 200 milliseconds. 1540 (Contributed by Steven D'Aprano in :issue:`6422`.) 1541 1542 :mod:`timeit` now warns when there is substantial (4x) variance 1543 between best and worst times. 1544 (Contributed by Serhiy Storchaka in :issue:`23552`.) 1545 1546 1547 tkinter 1548 ------- 1549 1550 Added methods :meth:`~tkinter.Variable.trace_add`, 1551 :meth:`~tkinter.Variable.trace_remove` and :meth:`~tkinter.Variable.trace_info` 1552 in the :class:`tkinter.Variable` class. They replace old methods 1553 :meth:`~tkinter.Variable.trace_variable`, :meth:`~tkinter.Variable.trace`, 1554 :meth:`~tkinter.Variable.trace_vdelete` and 1555 :meth:`~tkinter.Variable.trace_vinfo` that use obsolete Tcl commands and might 1556 not work in future versions of Tcl. 1557 (Contributed by Serhiy Storchaka in :issue:`22115`). 1558 1559 1560 .. _whatsnew36-traceback: 1561 1562 traceback 1563 --------- 1564 1565 Both the traceback module and the interpreter's builtin exception display now 1566 abbreviate long sequences of repeated lines in tracebacks as shown in the 1567 following example:: 1568 1569 >>> def f(): f() 1570 ... 1571 >>> f() 1572 Traceback (most recent call last): 1573 File "<stdin>", line 1, in <module> 1574 File "<stdin>", line 1, in f 1575 File "<stdin>", line 1, in f 1576 File "<stdin>", line 1, in f 1577 [Previous line repeated 995 more times] 1578 RecursionError: maximum recursion depth exceeded 1579 1580 (Contributed by Emanuel Barry in :issue:`26823`.) 1581 1582 1583 tracemalloc 1584 ----------- 1585 1586 The :mod:`tracemalloc` module now supports tracing memory allocations in 1587 multiple different address spaces. 1588 1589 The new :class:`~tracemalloc.DomainFilter` filter class has been added 1590 to filter block traces by their address space (domain). 1591 1592 (Contributed by Victor Stinner in :issue:`26588`.) 1593 1594 1595 .. _whatsnew36-typing: 1596 1597 typing 1598 ------ 1599 1600 Since the :mod:`typing` module is :term:`provisional <provisional api>`, 1601 all changes introduced in Python 3.6 have also been 1602 backported to Python 3.5.x. 1603 1604 The :mod:`typing` module has a much improved support for generic type 1605 aliases. For example ``Dict[str, Tuple[S, T]]`` is now a valid 1606 type annotation. 1607 (Contributed by Guido van Rossum in `Github #195 1608 <https://github.com/python/typing/pull/195>`_.) 1609 1610 The :class:`typing.ContextManager` class has been added for 1611 representing :class:`contextlib.AbstractContextManager`. 1612 (Contributed by Brett Cannon in :issue:`25609`.) 1613 1614 The :class:`typing.Collection` class has been added for 1615 representing :class:`collections.abc.Collection`. 1616 (Contributed by Ivan Levkivskyi in :issue:`27598`.) 1617 1618 The :const:`typing.ClassVar` type construct has been added to 1619 mark class variables. As introduced in :pep:`526`, a variable annotation 1620 wrapped in ClassVar indicates that a given attribute is intended to be used as 1621 a class variable and should not be set on instances of that class. 1622 (Contributed by Ivan Levkivskyi in `Github #280 1623 <https://github.com/python/typing/pull/280>`_.) 1624 1625 A new :const:`~typing.TYPE_CHECKING` constant that is assumed to be 1626 ``True`` by the static type checkers, but is ``False`` at runtime. 1627 (Contributed by Guido van Rossum in `Github #230 1628 <https://github.com/python/typing/issues/230>`_.) 1629 1630 A new :func:`~typing.NewType` helper function has been added to create 1631 lightweight distinct types for annotations:: 1632 1633 from typing import NewType 1634 1635 UserId = NewType('UserId', int) 1636 some_id = UserId(524313) 1637 1638 The static type checker will treat the new type as if it were a subclass 1639 of the original type. (Contributed by Ivan Levkivskyi in `Github #189 1640 <https://github.com/python/typing/issues/189>`_.) 1641 1642 1643 unicodedata 1644 ----------- 1645 1646 The :mod:`unicodedata` module now uses data from `Unicode 9.0.0 1647 <http://unicode.org/versions/Unicode9.0.0/>`_. 1648 (Contributed by Benjamin Peterson.) 1649 1650 1651 unittest.mock 1652 ------------- 1653 1654 The :class:`~unittest.mock.Mock` class has the following improvements: 1655 1656 * Two new methods, :meth:`Mock.assert_called() 1657 <unittest.mock.Mock.assert_called>` and :meth:`Mock.assert_called_once() 1658 <unittest.mock.Mock.assert_called_once>` to check if the mock object 1659 was called. 1660 (Contributed by Amit Saha in :issue:`26323`.) 1661 1662 * The :meth:`Mock.reset_mock() <unittest.mock.Mock.reset_mock>` method 1663 now has two optional keyword only arguments: *return_value* and 1664 *side_effect*. 1665 (Contributed by Kushal Das in :issue:`21271`.) 1666 1667 1668 urllib.request 1669 -------------- 1670 1671 If a HTTP request has a file or iterable body (other than a 1672 bytes object) but no ``Content-Length`` header, rather than 1673 throwing an error, :class:`~urllib.request.AbstractHTTPHandler` now 1674 falls back to use chunked transfer encoding. 1675 (Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.) 1676 1677 1678 urllib.robotparser 1679 ------------------ 1680 1681 :class:`~urllib.robotparser.RobotFileParser` now supports the ``Crawl-delay`` and 1682 ``Request-rate`` extensions. 1683 (Contributed by Nikolay Bogoychev in :issue:`16099`.) 1684 1685 1686 venv 1687 ---- 1688 1689 :mod:`venv` accepts a new parameter ``--prompt``. This parameter provides an 1690 alternative prefix for the virtual environment. (Proposed by ukasz Balcerzak 1691 and ported to 3.6 by Stphane Wirtel in :issue:`22829`.) 1692 1693 1694 warnings 1695 -------- 1696 1697 A new optional *source* parameter has been added to the 1698 :func:`warnings.warn_explicit` function: the destroyed object which emitted a 1699 :exc:`ResourceWarning`. A *source* attribute has also been added to 1700 :class:`warnings.WarningMessage` (contributed by Victor Stinner in 1701 :issue:`26568` and :issue:`26567`). 1702 1703 When a :exc:`ResourceWarning` warning is logged, the :mod:`tracemalloc` module is now 1704 used to try to retrieve the traceback where the destroyed object was allocated. 1705 1706 Example with the script ``example.py``:: 1707 1708 import warnings 1709 1710 def func(): 1711 return open(__file__) 1712 1713 f = func() 1714 f = None 1715 1716 Output of the command ``python3.6 -Wd -X tracemalloc=5 example.py``:: 1717 1718 example.py:7: ResourceWarning: unclosed file <_io.TextIOWrapper name='example.py' mode='r' encoding='UTF-8'> 1719 f = None 1720 Object allocated at (most recent call first): 1721 File "example.py", lineno 4 1722 return open(__file__) 1723 File "example.py", lineno 6 1724 f = func() 1725 1726 The "Object allocated at" traceback is new and is only displayed if 1727 :mod:`tracemalloc` is tracing Python memory allocations and if the 1728 :mod:`warnings` module was already imported. 1729 1730 1731 winreg 1732 ------ 1733 1734 Added the 64-bit integer type :data:`REG_QWORD <winreg.REG_QWORD>`. 1735 (Contributed by Clement Rouault in :issue:`23026`.) 1736 1737 1738 winsound 1739 -------- 1740 1741 Allowed keyword arguments to be passed to :func:`Beep <winsound.Beep>`, 1742 :func:`MessageBeep <winsound.MessageBeep>`, and :func:`PlaySound 1743 <winsound.PlaySound>` (:issue:`27982`). 1744 1745 1746 xmlrpc.client 1747 ------------- 1748 1749 The :mod:`xmlrpc.client` module now supports unmarshalling 1750 additional data types used by the Apache XML-RPC implementation 1751 for numerics and ``None``. 1752 (Contributed by Serhiy Storchaka in :issue:`26885`.) 1753 1754 1755 zipfile 1756 ------- 1757 1758 A new :meth:`ZipInfo.from_file() <zipfile.ZipInfo.from_file>` class method 1759 allows making a :class:`~zipfile.ZipInfo` instance from a filesystem file. 1760 A new :meth:`ZipInfo.is_dir() <zipfile.ZipInfo.is_dir>` method can be used 1761 to check if the :class:`~zipfile.ZipInfo` instance represents a directory. 1762 (Contributed by Thomas Kluyver in :issue:`26039`.) 1763 1764 The :meth:`ZipFile.open() <zipfile.ZipFile.open>` method can now be used to 1765 write data into a ZIP file, as well as for extracting data. 1766 (Contributed by Thomas Kluyver in :issue:`26039`.) 1767 1768 1769 zlib 1770 ---- 1771 1772 The :func:`~zlib.compress` and :func:`~zlib.decompress` functions now accept 1773 keyword arguments. 1774 (Contributed by Aviv Palivoda in :issue:`26243` and 1775 Xiang Zhang in :issue:`16764` respectively.) 1776 1777 1778 Optimizations 1779 ============= 1780 1781 * The Python interpreter now uses a 16-bit wordcode instead of bytecode which 1782 made a number of opcode optimizations possible. 1783 (Contributed by Demur Rumed with input and reviews from 1784 Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.) 1785 1786 * The :class:`asyncio.Future` class now has an optimized C implementation. 1787 (Contributed by Yury Selivanov and INADA Naoki in :issue:`26081`.) 1788 1789 * The :class:`asyncio.Task` class now has an optimized 1790 C implementation. (Contributed by Yury Selivanov in :issue:`28544`.) 1791 1792 * Various implementation improvements in the :mod:`typing` module 1793 (such as caching of generic types) allow up to 30 times performance 1794 improvements and reduced memory footprint. 1795 1796 * The ASCII decoder is now up to 60 times as fast for error handlers 1797 ``surrogateescape``, ``ignore`` and ``replace`` (Contributed 1798 by Victor Stinner in :issue:`24870`). 1799 1800 * The ASCII and the Latin1 encoders are now up to 3 times as fast for the 1801 error handler ``surrogateescape`` 1802 (Contributed by Victor Stinner in :issue:`25227`). 1803 1804 * The UTF-8 encoder is now up to 75 times as fast for error handlers 1805 ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass`` (Contributed 1806 by Victor Stinner in :issue:`25267`). 1807 1808 * The UTF-8 decoder is now up to 15 times as fast for error handlers 1809 ``ignore``, ``replace`` and ``surrogateescape`` (Contributed 1810 by Victor Stinner in :issue:`25301`). 1811 1812 * ``bytes % args`` is now up to 2 times faster. (Contributed by Victor Stinner 1813 in :issue:`25349`). 1814 1815 * ``bytearray % args`` is now between 2.5 and 5 times faster. (Contributed by 1816 Victor Stinner in :issue:`25399`). 1817 1818 * Optimize :meth:`bytes.fromhex` and :meth:`bytearray.fromhex`: they are now 1819 between 2x and 3.5x faster. (Contributed by Victor Stinner in :issue:`25401`). 1820 1821 * Optimize ``bytes.replace(b'', b'.')`` and ``bytearray.replace(b'', b'.')``: 1822 up to 80% faster. (Contributed by Josh Snider in :issue:`26574`). 1823 1824 * Allocator functions of the :c:func:`PyMem_Malloc` domain 1825 (:c:data:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc memory allocator 1826 <pymalloc>` instead of :c:func:`malloc` function of the C library. The 1827 pymalloc allocator is optimized for objects smaller or equal to 512 bytes 1828 with a short lifetime, and use :c:func:`malloc` for larger memory blocks. 1829 (Contributed by Victor Stinner in :issue:`26249`). 1830 1831 * :func:`pickle.load` and :func:`pickle.loads` are now up to 10% faster when 1832 deserializing many small objects (Contributed by Victor Stinner in 1833 :issue:`27056`). 1834 1835 * Passing :term:`keyword arguments <keyword argument>` to a function has an 1836 overhead in comparison with passing :term:`positional arguments 1837 <positional argument>`. Now in extension functions implemented with using 1838 Argument Clinic this overhead is significantly decreased. 1839 (Contributed by Serhiy Storchaka in :issue:`27574`). 1840 1841 * Optimized :func:`~glob.glob` and :func:`~glob.iglob` functions in the 1842 :mod:`glob` module; they are now about 3--6 times faster. 1843 (Contributed by Serhiy Storchaka in :issue:`25596`). 1844 1845 * Optimized globbing in :mod:`pathlib` by using :func:`os.scandir`; 1846 it is now about 1.5--4 times faster. 1847 (Contributed by Serhiy Storchaka in :issue:`26032`). 1848 1849 * :class:`xml.etree.ElementTree` parsing, iteration and deepcopy performance 1850 has been significantly improved. 1851 (Contributed by Serhiy Storchaka in :issue:`25638`, :issue:`25873`, 1852 and :issue:`25869`.) 1853 1854 * Creation of :class:`fractions.Fraction` instances from floats and 1855 decimals is now 2 to 3 times faster. 1856 (Contributed by Serhiy Storchaka in :issue:`25971`.) 1857 1858 1859 Build and C API Changes 1860 ======================= 1861 1862 * Python now requires some C99 support in the toolchain to build. 1863 Most notably, Python now uses standard integer types and macros in 1864 place of custom macros like ``PY_LONG_LONG``. 1865 For more information, see :pep:`7` and :issue:`17884`. 1866 1867 * Cross-compiling CPython with the Android NDK and the Android API level set to 1868 21 (Android 5.0 Lollipop) or greater runs successfully. While Android is not 1869 yet a supported platform, the Python test suite runs on the Android emulator 1870 with only about 16 tests failures. See the Android meta-issue :issue:`26865`. 1871 1872 * The ``--enable-optimizations`` configure flag has been added. Turning it on 1873 will activate expensive optimizations like PGO. 1874 (Original patch by Alecsandru Patrascu of Intel in :issue:`26359`.) 1875 1876 * The :term:`GIL <global interpreter lock>` must now be held when allocator 1877 functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and 1878 :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. 1879 1880 * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data 1881 failed. 1882 (Contributed by Martin Panter in :issue:`5319`.) 1883 1884 * :c:func:`PyArg_ParseTupleAndKeywords` now supports :ref:`positional-only 1885 parameters <positional-only_parameter>`. Positional-only parameters are 1886 defined by empty names. 1887 (Contributed by Serhiy Storchaka in :issue:`26282`). 1888 1889 * ``PyTraceback_Print`` method now abbreviates long sequences of repeated lines 1890 as ``"[Previous line repeated {count} more times]"``. 1891 (Contributed by Emanuel Barry in :issue:`26823`.) 1892 1893 * The new :c:func:`PyErr_SetImportErrorSubclass` function allows for 1894 specifying a subclass of :exc:`ImportError` to raise. 1895 (Contributed by Eric Snow in :issue:`15767`.) 1896 1897 * The new :c:func:`PyErr_ResourceWarning` function can be used to generate 1898 a :exc:`ResourceWarning` providing the source of the resource allocation. 1899 (Contributed by Victor Stinner in :issue:`26567`.) 1900 1901 * The new :c:func:`PyOS_FSPath` function returns the file system 1902 representation of a :term:`path-like object`. 1903 (Contributed by Brett Cannon in :issue:`27186`.) 1904 1905 * The :c:func:`PyUnicode_FSConverter` and :c:func:`PyUnicode_FSDecoder` 1906 functions will now accept :term:`path-like objects <path-like object>`. 1907 1908 1909 Other Improvements 1910 ================== 1911 1912 * When :option:`--version` (short form: :option:`-V`) is supplied twice, 1913 Python prints :data:`sys.version` for detailed information. 1914 1915 .. code-block:: shell-session 1916 1917 $ ./python -VV 1918 Python 3.6.0b4+ (3.6:223967b49e49+, Nov 21 2016, 20:55:04) 1919 [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] 1920 1921 1922 Deprecated 1923 ========== 1924 1925 New Keywords 1926 ------------ 1927 1928 ``async`` and ``await`` are not recommended to be used as variable, class, 1929 function or module names. Introduced by :pep:`492` in Python 3.5, they will 1930 become proper keywords in Python 3.7. Starting in Python 3.6, the use of 1931 ``async`` or ``await`` as names will generate a :exc:`DeprecationWarning`. 1932 1933 1934 Deprecated Python behavior 1935 -------------------------- 1936 1937 Raising the :exc:`StopIteration` exception inside a generator will now 1938 generate a :exc:`DeprecationWarning`, and will trigger a :exc:`RuntimeError` 1939 in Python 3.7. See :ref:`whatsnew-pep-479` for details. 1940 1941 The :meth:`__aiter__` method is now expected to return an asynchronous 1942 iterator directly instead of returning an awaitable as previously. 1943 Doing the former will trigger a :exc:`DeprecationWarning`. Backward 1944 compatibility will be removed in Python 3.7. 1945 (Contributed by Yury Selivanov in :issue:`27243`.) 1946 1947 A backslash-character pair that is not a valid escape sequence now generates 1948 a :exc:`DeprecationWarning`. Although this will eventually become a 1949 :exc:`SyntaxError`, that will not be for several Python releases. 1950 (Contributed by Emanuel Barry in :issue:`27364`.) 1951 1952 When performing a relative import, falling back on ``__name__`` and 1953 ``__path__`` from the calling module when ``__spec__`` or 1954 ``__package__`` are not defined now raises an :exc:`ImportWarning`. 1955 (Contributed by Rose Ames in :issue:`25791`.) 1956 1957 1958 Deprecated Python modules, functions and methods 1959 ------------------------------------------------ 1960 1961 asynchat 1962 ~~~~~~~~ 1963 1964 The :mod:`asynchat` has been deprecated in favor of :mod:`asyncio`. 1965 (Contributed by Mariatta in :issue:`25002`.) 1966 1967 1968 asyncore 1969 ~~~~~~~~ 1970 1971 The :mod:`asyncore` has been deprecated in favor of :mod:`asyncio`. 1972 (Contributed by Mariatta in :issue:`25002`.) 1973 1974 1975 dbm 1976 ~~~ 1977 1978 Unlike other :mod:`dbm` implementations, the :mod:`dbm.dumb` module 1979 creates databases with the ``'rw'`` mode and allows modifying the database 1980 opened with the ``'r'`` mode. This behavior is now deprecated and will 1981 be removed in 3.8. 1982 (Contributed by Serhiy Storchaka in :issue:`21708`.) 1983 1984 1985 distutils 1986 ~~~~~~~~~ 1987 1988 The undocumented ``extra_path`` argument to the 1989 :class:`~distutils.Distribution` constructor is now considered deprecated 1990 and will raise a warning if set. Support for this parameter will be 1991 removed in a future Python release. See :issue:`27919` for details. 1992 1993 1994 grp 1995 ~~~ 1996 1997 The support of non-integer arguments in :func:`~grp.getgrgid` has been 1998 deprecated. 1999 (Contributed by Serhiy Storchaka in :issue:`26129`.) 2000 2001 2002 importlib 2003 ~~~~~~~~~ 2004 2005 The :meth:`importlib.machinery.SourceFileLoader.load_module` and 2006 :meth:`importlib.machinery.SourcelessFileLoader.load_module` methods 2007 are now deprecated. They were the only remaining implementations of 2008 :meth:`importlib.abc.Loader.load_module` in :mod:`importlib` that had not 2009 been deprecated in previous versions of Python in favour of 2010 :meth:`importlib.abc.Loader.exec_module`. 2011 2012 The :class:`importlib.machinery.WindowsRegistryFinder` class is now 2013 deprecated. As of 3.6.0, it is still added to :attr:`sys.meta_path` by 2014 default (on Windows), but this may change in future releases. 2015 2016 os 2017 ~~ 2018 2019 Undocumented support of general :term:`bytes-like objects <bytes-like object>` 2020 as paths in :mod:`os` functions, :func:`compile` and similar functions is 2021 now deprecated. 2022 (Contributed by Serhiy Storchaka in :issue:`25791` and :issue:`26754`.) 2023 2024 re 2025 ~~ 2026 2027 Support for inline flags ``(?letters)`` in the middle of the regular 2028 expression has been deprecated and will be removed in a future Python 2029 version. Flags at the start of a regular expression are still allowed. 2030 (Contributed by Serhiy Storchaka in :issue:`22493`.) 2031 2032 ssl 2033 ~~~ 2034 2035 OpenSSL 0.9.8, 1.0.0 and 1.0.1 are deprecated and no longer supported. 2036 In the future the :mod:`ssl` module will require at least OpenSSL 1.0.2 or 2037 1.1.0. 2038 2039 SSL-related arguments like ``certfile``, ``keyfile`` and ``check_hostname`` 2040 in :mod:`ftplib`, :mod:`http.client`, :mod:`imaplib`, :mod:`poplib`, 2041 and :mod:`smtplib` have been deprecated in favor of ``context``. 2042 (Contributed by Christian Heimes in :issue:`28022`.) 2043 2044 A couple of protocols and functions of the :mod:`ssl` module are now 2045 deprecated. Some features will no longer be available in future versions 2046 of OpenSSL. Other features are deprecated in favor of a different API. 2047 (Contributed by Christian Heimes in :issue:`28022` and :issue:`26470`.) 2048 2049 tkinter 2050 ~~~~~~~ 2051 2052 The :mod:`tkinter.tix` module is now deprecated. :mod:`tkinter` users 2053 should use :mod:`tkinter.ttk` instead. 2054 2055 venv 2056 ~~~~ 2057 2058 The ``pyvenv`` script has been deprecated in favour of ``python3 -m venv``. 2059 This prevents confusion as to what Python interpreter ``pyvenv`` is 2060 connected to and thus what Python interpreter will be used by the virtual 2061 environment. (Contributed by Brett Cannon in :issue:`25154`.) 2062 2063 2064 Deprecated functions and types of the C API 2065 ------------------------------------------- 2066 2067 Undocumented functions :c:func:`PyUnicode_AsEncodedObject`, 2068 :c:func:`PyUnicode_AsDecodedObject`, :c:func:`PyUnicode_AsEncodedUnicode` 2069 and :c:func:`PyUnicode_AsDecodedUnicode` are deprecated now. 2070 Use the :ref:`generic codec based API <codec-registry>` instead. 2071 2072 2073 Deprecated Build Options 2074 ------------------------ 2075 2076 The ``--with-system-ffi`` configure flag is now on by default on non-macOS 2077 UNIX platforms. It may be disabled by using ``--without-system-ffi``, but 2078 using the flag is deprecated and will not be accepted in Python 3.7. 2079 macOS is unaffected by this change. Note that many OS distributors already 2080 use the ``--with-system-ffi`` flag when building their system Python. 2081 2082 2083 Removed 2084 ======= 2085 2086 API and Feature Removals 2087 ------------------------ 2088 2089 * Unknown escapes consisting of ``'\'`` and an ASCII letter in 2090 regular expressions will now cause an error. In replacement templates for 2091 :func:`re.sub` they are still allowed, but deprecated. 2092 The :const:`re.LOCALE` flag can now only be used with binary patterns. 2093 2094 * ``inspect.getmoduleinfo()`` was removed (was deprecated since CPython 3.3). 2095 :func:`inspect.getmodulename` should be used for obtaining the module 2096 name for a given path. 2097 (Contributed by Yury Selivanov in :issue:`13248`.) 2098 2099 * ``traceback.Ignore`` class and ``traceback.usage``, ``traceback.modname``, 2100 ``traceback.fullmodname``, ``traceback.find_lines_from_code``, 2101 ``traceback.find_lines``, ``traceback.find_strings``, 2102 ``traceback.find_executable_lines`` methods were removed from the 2103 :mod:`traceback` module. They were undocumented methods deprecated since 2104 Python 3.2 and equivalent functionality is available from private methods. 2105 2106 * The ``tk_menuBar()`` and ``tk_bindForTraversal()`` dummy methods in 2107 :mod:`tkinter` widget classes were removed (corresponding Tk commands 2108 were obsolete since Tk 4.0). 2109 2110 * The :meth:`~zipfile.ZipFile.open` method of the :class:`zipfile.ZipFile` 2111 class no longer supports the ``'U'`` mode (was deprecated since Python 3.4). 2112 Use :class:`io.TextIOWrapper` for reading compressed text files in 2113 :term:`universal newlines` mode. 2114 2115 * The undocumented ``IN``, ``CDROM``, ``DLFCN``, ``TYPES``, ``CDIO``, and 2116 ``STROPTS`` modules have been removed. They had been available in the 2117 platform specific ``Lib/plat-*/`` directories, but were chronically out of 2118 date, inconsistently available across platforms, and unmaintained. The 2119 script that created these modules is still available in the source 2120 distribution at :source:`Tools/scripts/h2py.py`. 2121 2122 * The deprecated ``asynchat.fifo`` class has been removed. 2123 2124 2125 Porting to Python 3.6 2126 ===================== 2127 2128 This section lists previously described changes and other bugfixes 2129 that may require changes to your code. 2130 2131 Changes in 'python' Command Behavior 2132 ------------------------------------ 2133 2134 * The output of a special Python build with defined ``COUNT_ALLOCS``, 2135 ``SHOW_ALLOC_COUNT`` or ``SHOW_TRACK_COUNT`` macros is now off by 2136 default. It can be re-enabled using the ``-X showalloccount`` option. 2137 It now outputs to ``stderr`` instead of ``stdout``. 2138 (Contributed by Serhiy Storchaka in :issue:`23034`.) 2139 2140 2141 Changes in the Python API 2142 ------------------------- 2143 2144 * :func:`open() <open>` will no longer allow combining the ``'U'`` mode flag 2145 with ``'+'``. 2146 (Contributed by Jeff Balogh and John O'Connor in :issue:`2091`.) 2147 2148 * :mod:`sqlite3` no longer implicitly commits an open transaction before DDL 2149 statements. 2150 2151 * On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool 2152 is initialized to increase the security. 2153 2154 * When :meth:`importlib.abc.Loader.exec_module` is defined, 2155 :meth:`importlib.abc.Loader.create_module` must also be defined. 2156 2157 * :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg** 2158 argument is not set. Previously only ``NULL`` was returned. 2159 2160 * The format of the ``co_lnotab`` attribute of code objects changed to support 2161 a negative line number delta. By default, Python does not emit bytecode with 2162 a negative line number delta. Functions using ``frame.f_lineno``, 2163 ``PyFrame_GetLineNumber()`` or ``PyCode_Addr2Line()`` are not affected. 2164 Functions directly decoding ``co_lnotab`` should be updated to use a signed 2165 8-bit integer type for the line number delta, but this is only required to 2166 support applications using a negative line number delta. See 2167 ``Objects/lnotab_notes.txt`` for the ``co_lnotab`` format and how to decode 2168 it, and see the :pep:`511` for the rationale. 2169 2170 * The functions in the :mod:`compileall` module now return booleans instead 2171 of ``1`` or ``0`` to represent success or failure, respectively. Thanks to 2172 booleans being a subclass of integers, this should only be an issue if you 2173 were doing identity checks for ``1`` or ``0``. See :issue:`25768`. 2174 2175 * Reading the :attr:`~urllib.parse.SplitResult.port` attribute of 2176 :func:`urllib.parse.urlsplit` and :func:`~urllib.parse.urlparse` results 2177 now raises :exc:`ValueError` for out-of-range values, rather than 2178 returning :const:`None`. See :issue:`20059`. 2179 2180 * The :mod:`imp` module now raises a :exc:`DeprecationWarning` instead of 2181 :exc:`PendingDeprecationWarning`. 2182 2183 * The following modules have had missing APIs added to their :attr:`__all__` 2184 attributes to match the documented APIs: 2185 :mod:`calendar`, :mod:`cgi`, :mod:`csv`, 2186 :mod:`~xml.etree.ElementTree`, :mod:`enum`, 2187 :mod:`fileinput`, :mod:`ftplib`, :mod:`logging`, :mod:`mailbox`, 2188 :mod:`mimetypes`, :mod:`optparse`, :mod:`plistlib`, :mod:`smtpd`, 2189 :mod:`subprocess`, :mod:`tarfile`, :mod:`threading` and 2190 :mod:`wave`. This means they will export new symbols when ``import *`` 2191 is used. 2192 (Contributed by Joel Taddei and Jacek Koodziej in :issue:`23883`.) 2193 2194 * When performing a relative import, if ``__package__`` does not compare equal 2195 to ``__spec__.parent`` then :exc:`ImportWarning` is raised. 2196 (Contributed by Brett Cannon in :issue:`25791`.) 2197 2198 * When a relative import is performed and no parent package is known, then 2199 :exc:`ImportError` will be raised. Previously, :exc:`SystemError` could be 2200 raised. (Contributed by Brett Cannon in :issue:`18018`.) 2201 2202 * Servers based on the :mod:`socketserver` module, including those 2203 defined in :mod:`http.server`, :mod:`xmlrpc.server` and 2204 :mod:`wsgiref.simple_server`, now only catch exceptions derived 2205 from :exc:`Exception`. Therefore if a request handler raises 2206 an exception like :exc:`SystemExit` or :exc:`KeyboardInterrupt`, 2207 :meth:`~socketserver.BaseServer.handle_error` is no longer called, and 2208 the exception will stop a single-threaded server. (Contributed by 2209 Martin Panter in :issue:`23430`.) 2210 2211 * :func:`spwd.getspnam` now raises a :exc:`PermissionError` instead of 2212 :exc:`KeyError` if the user doesn't have privileges. 2213 2214 * The :meth:`socket.socket.close` method now raises an exception if 2215 an error (e.g. ``EBADF``) was reported by the underlying system call. 2216 (Contributed by Martin Panter in :issue:`26685`.) 2217 2218 * The *decode_data* argument for the :class:`smtpd.SMTPChannel` and 2219 :class:`smtpd.SMTPServer` constructors is now ``False`` by default. 2220 This means that the argument passed to 2221 :meth:`~smtpd.SMTPServer.process_message` is now a bytes object by 2222 default, and ``process_message()`` will be passed keyword arguments. 2223 Code that has already been updated in accordance with the deprecation 2224 warning generated by 3.5 will not be affected. 2225 2226 * All optional arguments of the :func:`~json.dump`, :func:`~json.dumps`, 2227 :func:`~json.load` and :func:`~json.loads` functions and 2228 :class:`~json.JSONEncoder` and :class:`~json.JSONDecoder` class 2229 constructors in the :mod:`json` module are now :ref:`keyword-only 2230 <keyword-only_parameter>`. 2231 (Contributed by Serhiy Storchaka in :issue:`18726`.) 2232 2233 * Subclasses of :class:`type` which don't override ``type.__new__`` may no 2234 longer use the one-argument form to get the type of an object. 2235 2236 * As part of :pep:`487`, the handling of keyword arguments passed to 2237 :class:`type` (other than the metaclass hint, ``metaclass``) is now 2238 consistently delegated to :meth:`object.__init_subclass__`. This means that 2239 :meth:`type.__new__` and :meth:`type.__init__` both now accept arbitrary 2240 keyword arguments, but :meth:`object.__init_subclass__` (which is called from 2241 :meth:`type.__new__`) will reject them by default. Custom metaclasses 2242 accepting additional keyword arguments will need to adjust their calls to 2243 :meth:`type.__new__` (whether direct or via :class:`super`) accordingly. 2244 2245 * In :class:`distutils.command.sdist.sdist`, the ``default_format`` 2246 attribute has been removed and is no longer honored. Instead, the 2247 gzipped tarfile format is the default on all platforms and no 2248 platform-specific selection is made. 2249 In environments where distributions are 2250 built on Windows and zip distributions are required, configure 2251 the project with a ``setup.cfg`` file containing the following: 2252 2253 .. code-block:: ini 2254 2255 [sdist] 2256 formats=zip 2257 2258 This behavior has also been backported to earlier Python versions 2259 by Setuptools 26.0.0. 2260 2261 * In the :mod:`urllib.request` module and the 2262 :meth:`http.client.HTTPConnection.request` method, if no Content-Length 2263 header field has been specified and the request body is a file object, 2264 it is now sent with HTTP 1.1 chunked encoding. If a file object has to 2265 be sent to a HTTP 1.0 server, the Content-Length value now has to be 2266 specified by the caller. 2267 (Contributed by Demian Brecht and Rolf Krahl with tweaks from 2268 Martin Panter in :issue:`12319`.) 2269 2270 * The :class:`~csv.DictReader` now returns rows of type 2271 :class:`~collections.OrderedDict`. 2272 (Contributed by Steve Holden in :issue:`27842`.) 2273 2274 * The :const:`crypt.METHOD_CRYPT` will no longer be added to ``crypt.methods`` 2275 if unsupported by the platform. 2276 (Contributed by Victor Stinner in :issue:`25287`.) 2277 2278 * The *verbose* and *rename* arguments for 2279 :func:`~collections.namedtuple` are now keyword-only. 2280 (Contributed by Raymond Hettinger in :issue:`25628`.) 2281 2282 * On Linux, :func:`ctypes.util.find_library` now looks in 2283 ``LD_LIBRARY_PATH`` for shared libraries. 2284 (Contributed by Vinay Sajip in :issue:`9998`.) 2285 2286 * The :class:`imaplib.IMAP4` class now handles flags containing the 2287 ``']'`` character in messages sent from the server to improve 2288 real-world compatibility. 2289 (Contributed by Lita Cho in :issue:`21815`.) 2290 2291 * The :func:`mmap.write() <mmap.write>` function now returns the number 2292 of bytes written like other write methods. 2293 (Contributed by Jakub Stasiak in :issue:`26335`.) 2294 2295 * The :func:`pkgutil.iter_modules` and :func:`pkgutil.walk_packages` 2296 functions now return :class:`~pkgutil.ModuleInfo` named tuples. 2297 (Contributed by Ramchandra Apte in :issue:`17211`.) 2298 2299 * :func:`re.sub` now raises an error for invalid numerical group 2300 references in replacement templates even if the pattern is not 2301 found in the string. The error message for invalid group references 2302 now includes the group index and the position of the reference. 2303 (Contributed by SilentGhost, Serhiy Storchaka in :issue:`25953`.) 2304 2305 * :class:`zipfile.ZipFile` will now raise :exc:`NotImplementedError` for 2306 unrecognized compression values. Previously a plain :exc:`RuntimeError` 2307 was raised. Additionally, calling :class:`~zipfile.ZipFile` methods 2308 on a closed ZipFile or calling the :meth:`~zipfile.ZipFile.write` method 2309 on a ZipFile created with mode ``'r'`` will raise a :exc:`ValueError`. 2310 Previously, a :exc:`RuntimeError` was raised in those scenarios. 2311 2312 * when custom metaclasses are combined with zero-argument :func:`super` or 2313 direct references from methods to the implicit ``__class__`` closure 2314 variable, the implicit ``__classcell__`` namespace entry must now be passed 2315 up to ``type.__new__`` for initialisation. Failing to do so will result in 2316 a :exc:`DeprecationWarning` in Python 3.6 and a :exc:`RuntimeError` in 2317 Python 3.8. 2318 2319 * With the introduction of :exc:`ModuleNotFoundError`, import system consumers 2320 may start expecting import system replacements to raise that more specific 2321 exception when appropriate, rather than the less-specific :exc:`ImportError`. 2322 To provide future compatibility with such consumers, implementors of 2323 alternative import systems that completely replace :func:`__import__` will 2324 need to update their implementations to raise the new subclass when a module 2325 can't be found at all. Implementors of compliant plugins to the default 2326 import system shouldn't need to make any changes, as the default import 2327 system will raise the new subclass when appropriate. 2328 2329 2330 Changes in the C API 2331 -------------------- 2332 2333 * The :c:func:`PyMem_Malloc` allocator family now uses the :ref:`pymalloc allocator 2334 <pymalloc>` rather than the system :c:func:`malloc`. Applications calling 2335 :c:func:`PyMem_Malloc` without holding the GIL can now crash. Set the 2336 :envvar:`PYTHONMALLOC` environment variable to ``debug`` to validate the 2337 usage of memory allocators in your application. See :issue:`26249`. 2338 2339 * :c:func:`Py_Exit` (and the main interpreter) now override the exit status 2340 with 120 if flushing buffered data failed. See :issue:`5319`. 2341 2342 2343 CPython bytecode changes 2344 ------------------------ 2345 2346 There have been several major changes to the :term:`bytecode` in Python 3.6. 2347 2348 * The Python interpreter now uses a 16-bit wordcode instead of bytecode. 2349 (Contributed by Demur Rumed with input and reviews from 2350 Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.) 2351 2352 * The new :opcode:`FORMAT_VALUE` and :opcode:`BUILD_STRING` opcodes as part 2353 of the :ref:`formatted string literal <whatsnew36-pep498>` implementation. 2354 (Contributed by Eric Smith in :issue:`25483` and 2355 Serhiy Storchaka in :issue:`27078`.) 2356 2357 * The new :opcode:`BUILD_CONST_KEY_MAP` opcode to optimize the creation 2358 of dictionaries with constant keys. 2359 (Contributed by Serhiy Storchaka in :issue:`27140`.) 2360 2361 * The function call opcodes have been heavily reworked for better performance 2362 and simpler implementation. 2363 The :opcode:`MAKE_FUNCTION`, :opcode:`CALL_FUNCTION`, 2364 :opcode:`CALL_FUNCTION_KW` and :opcode:`BUILD_MAP_UNPACK_WITH_CALL` opcodes 2365 have been modified, the new :opcode:`CALL_FUNCTION_EX` and 2366 :opcode:`BUILD_TUPLE_UNPACK_WITH_CALL` have been added, and 2367 ``CALL_FUNCTION_VAR``, ``CALL_FUNCTION_VAR_KW`` and ``MAKE_CLOSURE`` opcodes 2368 have been removed. 2369 (Contributed by Demur Rumed in :issue:`27095`, and Serhiy Storchaka in 2370 :issue:`27213`, :issue:`28257`.) 2371 2372 * The new :opcode:`SETUP_ANNOTATIONS` and :opcode:`STORE_ANNOTATION` opcodes 2373 have been added to support the new :term:`variable annotation` syntax. 2374 (Contributed by Ivan Levkivskyi in :issue:`27985`.) 2375 2376 2377 Notable changes in Python 3.6.2 2378 =============================== 2379 2380 New ``make regen-all`` build target 2381 ----------------------------------- 2382 2383 To simplify cross-compilation, and to ensure that CPython can reliably be 2384 compiled without requiring an existing version of Python to already be 2385 available, the autotools-based build system no longer attempts to implicitly 2386 recompile generated files based on file modification times. 2387 2388 Instead, a new ``make regen-all`` command has been added to force regeneration 2389 of these files when desired (e.g. after an initial version of Python has 2390 already been built based on the pregenerated versions). 2391 2392 More selective regeneration targets are also defined - see 2393 :source:`Makefile.pre.in` for details. 2394 2395 (Contributed by Victor Stinner in :issue:`23404`.) 2396 2397 .. versionadded:: 3.6.2 2398 2399 2400 Removal of ``make touch`` build target 2401 -------------------------------------- 2402 2403 The ``make touch`` build target previously used to request implicit regeneration 2404 of generated files by updating their modification times has been removed. 2405 2406 It has been replaced by the new ``make regen-all`` target. 2407 2408 (Contributed by Victor Stinner in :issue:`23404`.) 2409 2410 .. versionchanged:: 3.6.2 2411 2412 2413 Notable changes in Python 3.6.4 2414 =============================== 2415 2416 The ``PyExc_RecursionErrorInst`` singleton that was part of the public API 2417 has been removed as its members being never cleared may cause a segfault 2418 during finalization of the interpreter. 2419 (Contributed by Xavier de Gaye in :issue:`22898` and :issue:`30697`.) 2420 2421 2422 Notable changes in Python 3.6.5 2423 =============================== 2424 2425 The :func:`locale.localeconv` function now sets temporarily the ``LC_CTYPE`` 2426 locale to the ``LC_NUMERIC`` locale in some cases. 2427 (Contributed by Victor Stinner in :issue:`31900`.) 2428 2429 Notable changes in Python 3.6.7 2430 =============================== 2431 2432 In 3.6.7 the :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token 2433 when provided with input that does not have a trailing new line. This behavior 2434 now matches what the C tokenizer does internally. 2435 (Contributed by Ammar Askar in :issue:`33899`.) 2436
