Home | History | Annotate | Download | only in whatsnew
      1 ****************************
      2   What's New in Python 2.2
      3 ****************************
      4 
      5 :Author: A.M. Kuchling
      6 
      7 .. |release| replace:: 1.02
      8 
      9 .. $Id: whatsnew22.tex 37315 2004-09-10 19:33:00Z akuchling $
     10 
     11 
     12 Introduction
     13 ============
     14 
     15 This article explains the new features in Python 2.2.2, released on October 14,
     16 2002.  Python 2.2.2 is a bugfix release of Python 2.2, originally released on
     17 December 21, 2001.
     18 
     19 Python 2.2 can be thought of as the "cleanup release".  There are some features
     20 such as generators and iterators that are completely new, but most of the
     21 changes, significant and far-reaching though they may be, are aimed at cleaning
     22 up irregularities and dark corners of the language design.
     23 
     24 This article doesn't attempt to provide a complete specification of the new
     25 features, but instead provides a convenient overview.  For full details, you
     26 should refer to the documentation for Python 2.2, such as the `Python Library
     27 Reference <https://docs.python.org/2.2/lib/lib.html>`_ and the `Python
     28 Reference Manual <https://docs.python.org/2.2/ref/ref.html>`_.  If you want to
     29 understand the complete implementation and design rationale for a change, refer
     30 to the PEP for a particular new feature.
     31 
     32 
     33 .. see also, now defunct
     34 
     35    http://www.unixreview.com/documents/s=1356/urm0109h/0109h.htm
     36       "What's So Special About Python 2.2?" is also about the new 2.2 features, and
     37       was written by Cameron Laird and Kathryn Soraiz.
     38 
     39 .. ======================================================================
     40 
     41 
     42 PEPs 252 and 253: Type and Class Changes
     43 ========================================
     44 
     45 The largest and most far-reaching changes in Python 2.2 are to Python's model of
     46 objects and classes.  The changes should be backward compatible, so it's likely
     47 that your code will continue to run unchanged, but the changes provide some
     48 amazing new capabilities. Before beginning this, the longest and most
     49 complicated section of this article, I'll provide an overview of the changes and
     50 offer some comments.
     51 
     52 A long time ago I wrote a Web page listing flaws in Python's design.  One of the
     53 most significant flaws was that it's impossible to subclass Python types
     54 implemented in C.  In particular, it's not possible to subclass built-in types,
     55 so you can't just subclass, say, lists in order to add a single useful method to
     56 them. The :mod:`UserList` module provides a class that supports all of the
     57 methods of lists and that can be subclassed further, but there's lots of C code
     58 that expects a regular Python list and won't accept a :class:`~UserList.UserList`
     59 instance.
     60 
     61 Python 2.2 fixes this, and in the process adds some exciting new capabilities.
     62 A brief summary:
     63 
     64 * You can subclass built-in types such as lists and even integers, and your
     65   subclasses should work in every place that requires the original type.
     66 
     67 * It's now possible to define static and class methods, in addition to the
     68   instance methods available in previous versions of Python.
     69 
     70 * It's also possible to automatically call methods on accessing or setting an
     71   instance attribute by using a new mechanism called :dfn:`properties`.  Many uses
     72   of :meth:`__getattr__` can be rewritten to use properties instead, making the
     73   resulting code simpler and faster.  As a small side benefit, attributes can now
     74   have docstrings, too.
     75 
     76 * The list of legal attributes for an instance can be limited to a particular
     77   set using :dfn:`slots`, making it possible to safeguard against typos and
     78   perhaps make more optimizations possible in future versions of Python.
     79 
     80 Some users have voiced concern about all these changes.  Sure, they say, the new
     81 features are neat and lend themselves to all sorts of tricks that weren't
     82 possible in previous versions of Python, but they also make the language more
     83 complicated.  Some people have said that they've always recommended Python for
     84 its simplicity, and feel that its simplicity is being lost.
     85 
     86 Personally, I think there's no need to worry.  Many of the new features are
     87 quite esoteric, and you can write a lot of Python code without ever needed to be
     88 aware of them.  Writing a simple class is no more difficult than it ever was, so
     89 you don't need to bother learning or teaching them unless they're actually
     90 needed.  Some very complicated tasks that were previously only possible from C
     91 will now be possible in pure Python, and to my mind that's all for the better.
     92 
     93 I'm not going to attempt to cover every single corner case and small change that
     94 were required to make the new features work.  Instead this section will paint
     95 only the broad strokes.  See section :ref:`sect-rellinks`, "Related Links", for
     96 further sources of information about Python 2.2's new object model.
     97 
     98 
     99 Old and New Classes
    100 -------------------
    101 
    102 First, you should know that Python 2.2 really has two kinds of classes: classic
    103 or old-style classes, and new-style classes.  The old-style class model is
    104 exactly the same as the class model in earlier versions of Python.  All the new
    105 features described in this section apply only to new-style classes. This
    106 divergence isn't intended to last forever; eventually old-style classes will be
    107 dropped, possibly in Python 3.0.
    108 
    109 So how do you define a new-style class?  You do it by subclassing an existing
    110 new-style class.  Most of Python's built-in types, such as integers, lists,
    111 dictionaries, and even files, are new-style classes now.  A new-style class
    112 named :class:`object`, the base class for all built-in types, has also been
    113 added so if no built-in type is suitable, you can just subclass
    114 :class:`object`::
    115 
    116    class C(object):
    117        def __init__ (self):
    118            ...
    119        ...
    120 
    121 This means that :keyword:`class` statements that don't have any base classes are
    122 always classic classes in Python 2.2.  (Actually you can also change this by
    123 setting a module-level variable named :attr:`__metaclass__` --- see :pep:`253`
    124 for the details --- but it's easier to just subclass :keyword:`object`.)
    125 
    126 The type objects for the built-in types are available as built-ins, named using
    127 a clever trick.  Python has always had built-in functions named :func:`int`,
    128 :func:`float`, and :func:`str`.  In 2.2, they aren't functions any more, but
    129 type objects that behave as factories when called. ::
    130 
    131    >>> int
    132    <type 'int'>
    133    >>> int('123')
    134    123
    135 
    136 To make the set of types complete, new type objects such as :func:`dict` and
    137 :func:`file` have been added.  Here's a more interesting example, adding a
    138 :meth:`lock` method to file objects::
    139 
    140    class LockableFile(file):
    141        def lock (self, operation, length=0, start=0, whence=0):
    142            import fcntl
    143            return fcntl.lockf(self.fileno(), operation,
    144                               length, start, whence)
    145 
    146 The now-obsolete :mod:`posixfile` module contained a class that emulated all of
    147 a file object's methods and also added a :meth:`lock` method, but this class
    148 couldn't be passed to internal functions that expected a built-in file,
    149 something which is possible with our new :class:`LockableFile`.
    150 
    151 
    152 Descriptors
    153 -----------
    154 
    155 In previous versions of Python, there was no consistent way to discover what
    156 attributes and methods were supported by an object. There were some informal
    157 conventions, such as defining :attr:`__members__` and :attr:`__methods__`
    158 attributes that were lists of names, but often the author of an extension type
    159 or a class wouldn't bother to define them.  You could fall back on inspecting
    160 the :attr:`~object.__dict__` of an object, but when class inheritance or an arbitrary
    161 :meth:`__getattr__` hook were in use this could still be inaccurate.
    162 
    163 The one big idea underlying the new class model is that an API for describing
    164 the attributes of an object using :dfn:`descriptors` has been formalized.
    165 Descriptors specify the value of an attribute, stating whether it's a method or
    166 a field.  With the descriptor API, static methods and class methods become
    167 possible, as well as more exotic constructs.
    168 
    169 Attribute descriptors are objects that live inside class objects, and have a few
    170 attributes of their own:
    171 
    172 * :attr:`~definition.__name__` is the attribute's name.
    173 
    174 * :attr:`__doc__` is the attribute's docstring.
    175 
    176 * ``__get__(object)`` is a method that retrieves the attribute value from
    177   *object*.
    178 
    179 * ``__set__(object, value)`` sets the attribute on *object* to *value*.
    180 
    181 * ``__delete__(object, value)`` deletes the *value*  attribute of *object*.
    182 
    183 For example, when you write ``obj.x``, the steps that Python actually performs
    184 are::
    185 
    186    descriptor = obj.__class__.x
    187    descriptor.__get__(obj)
    188 
    189 For methods, :meth:`descriptor.__get__` returns a temporary object that's
    190 callable, and wraps up the instance and the method to be called on it. This is
    191 also why static methods and class methods are now possible; they have
    192 descriptors that wrap up just the method, or the method and the class.  As a
    193 brief explanation of these new kinds of methods, static methods aren't passed
    194 the instance, and therefore resemble regular functions.  Class methods are
    195 passed the class of the object, but not the object itself.  Static and class
    196 methods are defined like this::
    197 
    198    class C(object):
    199        def f(arg1, arg2):
    200            ...
    201        f = staticmethod(f)
    202 
    203        def g(cls, arg1, arg2):
    204            ...
    205        g = classmethod(g)
    206 
    207 The :func:`staticmethod` function takes the function :func:`f`, and returns it
    208 wrapped up in a descriptor so it can be stored in the class object.  You might
    209 expect there to be special syntax for creating such methods (``def static f``,
    210 ``defstatic f()``, or something like that) but no such syntax has been defined
    211 yet; that's been left for future versions of Python.
    212 
    213 More new features, such as slots and properties, are also implemented as new
    214 kinds of descriptors, and it's not difficult to write a descriptor class that
    215 does something novel.  For example, it would be possible to write a descriptor
    216 class that made it possible to write Eiffel-style preconditions and
    217 postconditions for a method.  A class that used this feature might be defined
    218 like this::
    219 
    220    from eiffel import eiffelmethod
    221 
    222    class C(object):
    223        def f(self, arg1, arg2):
    224            # The actual function
    225            ...
    226        def pre_f(self):
    227            # Check preconditions
    228            ...
    229        def post_f(self):
    230            # Check postconditions
    231            ...
    232 
    233        f = eiffelmethod(f, pre_f, post_f)
    234 
    235 Note that a person using the new :func:`eiffelmethod` doesn't have to understand
    236 anything about descriptors.  This is why I think the new features don't increase
    237 the basic complexity of the language. There will be a few wizards who need to
    238 know about it in order to write :func:`eiffelmethod` or the ZODB or whatever,
    239 but most users will just write code on top of the resulting libraries and ignore
    240 the implementation details.
    241 
    242 
    243 Multiple Inheritance: The Diamond Rule
    244 --------------------------------------
    245 
    246 Multiple inheritance has also been made more useful through changing the rules
    247 under which names are resolved.  Consider this set of classes (diagram taken
    248 from :pep:`253` by Guido van Rossum)::
    249 
    250          class A:
    251            ^ ^  def save(self): ...
    252           /   \
    253          /     \
    254         /       \
    255        /         \
    256    class B     class C:
    257        ^         ^  def save(self): ...
    258         \       /
    259          \     /
    260           \   /
    261            \ /
    262          class D
    263 
    264 The lookup rule for classic classes is simple but not very smart; the base
    265 classes are searched depth-first, going from left to right.  A reference to
    266 :meth:`D.save` will search the classes :class:`D`, :class:`B`, and then
    267 :class:`A`, where :meth:`save` would be found and returned.  :meth:`C.save`
    268 would never be found at all.  This is bad, because if :class:`C`'s :meth:`save`
    269 method is saving some internal state specific to :class:`C`, not calling it will
    270 result in that state never getting saved.
    271 
    272 New-style classes follow a different algorithm that's a bit more complicated to
    273 explain, but does the right thing in this situation. (Note that Python 2.3
    274 changes this algorithm to one that produces the same results in most cases, but
    275 produces more useful results for really complicated inheritance graphs.)
    276 
    277 #. List all the base classes, following the classic lookup rule and include a
    278    class multiple times if it's visited repeatedly.  In the above example, the list
    279    of visited classes is [:class:`D`, :class:`B`, :class:`A`, :class:`C`,
    280    :class:`A`].
    281 
    282 #. Scan the list for duplicated classes.  If any are found, remove all but one
    283    occurrence, leaving the *last* one in the list.  In the above example, the list
    284    becomes [:class:`D`, :class:`B`, :class:`C`, :class:`A`] after dropping
    285    duplicates.
    286 
    287 Following this rule, referring to :meth:`D.save` will return :meth:`C.save`,
    288 which is the behaviour we're after.  This lookup rule is the same as the one
    289 followed by Common Lisp.  A new built-in function, :func:`super`, provides a way
    290 to get at a class's superclasses without having to reimplement Python's
    291 algorithm. The most commonly used form will be  ``super(class, obj)``, which
    292 returns  a bound superclass object (not the actual class object).  This form
    293 will be used in methods to call a method in the superclass; for example,
    294 :class:`D`'s :meth:`save` method would look like this::
    295 
    296    class D (B,C):
    297        def save (self):
    298            # Call superclass .save()
    299            super(D, self).save()
    300            # Save D's private information here
    301            ...
    302 
    303 :func:`super` can also return unbound superclass objects when called as
    304 ``super(class)`` or ``super(class1, class2)``, but this probably won't
    305 often be useful.
    306 
    307 
    308 Attribute Access
    309 ----------------
    310 
    311 A fair number of sophisticated Python classes define hooks for attribute access
    312 using :meth:`__getattr__`; most commonly this is done for convenience, to make
    313 code more readable by automatically mapping an attribute access such as
    314 ``obj.parent`` into a method call such as ``obj.get_parent``.  Python 2.2 adds
    315 some new ways of controlling attribute access.
    316 
    317 First, ``__getattr__(attr_name)`` is still supported by new-style classes,
    318 and nothing about it has changed.  As before, it will be called when an attempt
    319 is made to access ``obj.foo`` and no attribute named ``foo`` is found in the
    320 instance's dictionary.
    321 
    322 New-style classes also support a new method,
    323 ``__getattribute__(attr_name)``.  The difference between the two methods is
    324 that :meth:`__getattribute__` is *always* called whenever any attribute is
    325 accessed, while the old :meth:`__getattr__` is only called if ``foo`` isn't
    326 found in the instance's dictionary.
    327 
    328 However, Python 2.2's support for :dfn:`properties` will often be a simpler way
    329 to trap attribute references.  Writing a :meth:`__getattr__` method is
    330 complicated because to avoid recursion you can't use regular attribute accesses
    331 inside them, and instead have to mess around with the contents of
    332 :attr:`~object.__dict__`. :meth:`__getattr__` methods also end up being called by Python
    333 when it checks for other methods such as :meth:`__repr__` or :meth:`__coerce__`,
    334 and so have to be written with this in mind. Finally, calling a function on
    335 every attribute access results in a sizable performance loss.
    336 
    337 :class:`property` is a new built-in type that packages up three functions that
    338 get, set, or delete an attribute, and a docstring.  For example, if you want to
    339 define a :attr:`size` attribute that's computed, but also settable, you could
    340 write::
    341 
    342    class C(object):
    343        def get_size (self):
    344            result = ... computation ...
    345            return result
    346        def set_size (self, size):
    347            ... compute something based on the size
    348            and set internal state appropriately ...
    349 
    350        # Define a property.  The 'delete this attribute'
    351        # method is defined as None, so the attribute
    352        # can't be deleted.
    353        size = property(get_size, set_size,
    354                        None,
    355                        "Storage size of this instance")
    356 
    357 That is certainly clearer and easier to write than a pair of
    358 :meth:`__getattr__`/:meth:`__setattr__` methods that check for the :attr:`size`
    359 attribute and handle it specially while retrieving all other attributes from the
    360 instance's :attr:`~object.__dict__`.  Accesses to :attr:`size` are also the only ones
    361 which have to perform the work of calling a function, so references to other
    362 attributes run at their usual speed.
    363 
    364 Finally, it's possible to constrain the list of attributes that can be
    365 referenced on an object using the new :attr:`~object.__slots__` class attribute. Python
    366 objects are usually very dynamic; at any time it's possible to define a new
    367 attribute on an instance by just doing ``obj.new_attr=1``.   A new-style class
    368 can define a class attribute named :attr:`~object.__slots__` to limit the legal
    369 attributes  to a particular set of names.  An example will make this clear::
    370 
    371    >>> class C(object):
    372    ...     __slots__ = ('template', 'name')
    373    ...
    374    >>> obj = C()
    375    >>> print obj.template
    376    None
    377    >>> obj.template = 'Test'
    378    >>> print obj.template
    379    Test
    380    >>> obj.newattr = None
    381    Traceback (most recent call last):
    382      File "<stdin>", line 1, in ?
    383    AttributeError: 'C' object has no attribute 'newattr'
    384 
    385 Note how you get an :exc:`AttributeError` on the attempt to assign to an
    386 attribute not listed in :attr:`~object.__slots__`.
    387 
    388 
    389 .. _sect-rellinks:
    390 
    391 Related Links
    392 -------------
    393 
    394 This section has just been a quick overview of the new features, giving enough
    395 of an explanation to start you programming, but many details have been
    396 simplified or ignored.  Where should you go to get a more complete picture?
    397 
    398 https://docs.python.org/dev/howto/descriptor.html is a lengthy tutorial introduction to
    399 the descriptor features, written by Guido van Rossum. If my description has
    400 whetted your appetite, go read this tutorial next, because it goes into much
    401 more detail about the new features while still remaining quite easy to read.
    402 
    403 Next, there are two relevant PEPs, :pep:`252` and :pep:`253`.  :pep:`252` is
    404 titled "Making Types Look More Like Classes", and covers the descriptor API.
    405 :pep:`253` is titled "Subtyping Built-in Types", and describes the changes to
    406 type objects that make it possible to subtype built-in objects.  :pep:`253` is
    407 the more complicated PEP of the two, and at a few points the necessary
    408 explanations of types and meta-types may cause your head to explode.  Both PEPs
    409 were written and implemented by Guido van Rossum, with substantial assistance
    410 from the rest of the Zope Corp. team.
    411 
    412 Finally, there's the ultimate authority: the source code.  Most of the machinery
    413 for the type handling is in :file:`Objects/typeobject.c`, but you should only
    414 resort to it after all other avenues have been exhausted, including posting a
    415 question to python-list or python-dev.
    416 
    417 .. ======================================================================
    418 
    419 
    420 PEP 234: Iterators
    421 ==================
    422 
    423 Another significant addition to 2.2 is an iteration interface at both the C and
    424 Python levels.  Objects can define how they can be looped over by callers.
    425 
    426 In Python versions up to 2.1, the usual way to make ``for item in obj`` work is
    427 to define a :meth:`__getitem__` method that looks something like this::
    428 
    429    def __getitem__(self, index):
    430        return <next item>
    431 
    432 :meth:`__getitem__` is more properly used to define an indexing operation on an
    433 object so that you can write ``obj[5]`` to retrieve the sixth element.  It's a
    434 bit misleading when you're using this only to support :keyword:`for` loops.
    435 Consider some file-like object that wants to be looped over; the *index*
    436 parameter is essentially meaningless, as the class probably assumes that a
    437 series of :meth:`__getitem__` calls will be made with *index* incrementing by
    438 one each time.  In other words, the presence of the :meth:`__getitem__` method
    439 doesn't mean that using ``file[5]``  to randomly access the sixth element will
    440 work, though it really should.
    441 
    442 In Python 2.2, iteration can be implemented separately, and :meth:`__getitem__`
    443 methods can be limited to classes that really do support random access.  The
    444 basic idea of iterators is  simple.  A new built-in function, ``iter(obj)``
    445 or ``iter(C, sentinel)``, is used to get an iterator. ``iter(obj)`` returns
    446 an iterator for the object *obj*, while ``iter(C, sentinel)`` returns an
    447 iterator that will invoke the callable object *C* until it returns *sentinel* to
    448 signal that the iterator is done.
    449 
    450 Python classes can define an :meth:`__iter__` method, which should create and
    451 return a new iterator for the object; if the object is its own iterator, this
    452 method can just return ``self``.  In particular, iterators will usually be their
    453 own iterators.  Extension types implemented in C can implement a :c:member:`~PyTypeObject.tp_iter`
    454 function in order to return an iterator, and extension types that want to behave
    455 as iterators can define a :c:member:`~PyTypeObject.tp_iternext` function.
    456 
    457 So, after all this, what do iterators actually do?  They have one required
    458 method, :meth:`next`, which takes no arguments and returns the next value.  When
    459 there are no more values to be returned, calling :meth:`next` should raise the
    460 :exc:`StopIteration` exception. ::
    461 
    462    >>> L = [1,2,3]
    463    >>> i = iter(L)
    464    >>> print i
    465    <iterator object at 0x8116870>
    466    >>> i.next()
    467    1
    468    >>> i.next()
    469    2
    470    >>> i.next()
    471    3
    472    >>> i.next()
    473    Traceback (most recent call last):
    474      File "<stdin>", line 1, in ?
    475    StopIteration
    476    >>>
    477 
    478 In 2.2, Python's :keyword:`for` statement no longer expects a sequence; it
    479 expects something for which :func:`iter` will return an iterator. For backward
    480 compatibility and convenience, an iterator is automatically constructed for
    481 sequences that don't implement :meth:`__iter__` or a :c:member:`~PyTypeObject.tp_iter` slot, so
    482 ``for i in [1,2,3]`` will still work.  Wherever the Python interpreter loops
    483 over a sequence, it's been changed to use the iterator protocol.  This means you
    484 can do things like this::
    485 
    486    >>> L = [1,2,3]
    487    >>> i = iter(L)
    488    >>> a,b,c = i
    489    >>> a,b,c
    490    (1, 2, 3)
    491 
    492 Iterator support has been added to some of Python's basic types.   Calling
    493 :func:`iter` on a dictionary will return an iterator which loops over its keys::
    494 
    495    >>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6,
    496    ...      'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12}
    497    >>> for key in m: print key, m[key]
    498    ...
    499    Mar 3
    500    Feb 2
    501    Aug 8
    502    Sep 9
    503    May 5
    504    Jun 6
    505    Jul 7
    506    Jan 1
    507    Apr 4
    508    Nov 11
    509    Dec 12
    510    Oct 10
    511 
    512 That's just the default behaviour.  If you want to iterate over keys, values, or
    513 key/value pairs, you can explicitly call the :meth:`iterkeys`,
    514 :meth:`itervalues`, or :meth:`iteritems` methods to get an appropriate iterator.
    515 In a minor related change, the :keyword:`in` operator now works on dictionaries,
    516 so ``key in dict`` is now equivalent to ``dict.has_key(key)``.
    517 
    518 Files also provide an iterator, which calls the :meth:`readline` method until
    519 there are no more lines in the file.  This means you can now read each line of a
    520 file using code like this::
    521 
    522    for line in file:
    523        # do something for each line
    524        ...
    525 
    526 Note that you can only go forward in an iterator; there's no way to get the
    527 previous element, reset the iterator, or make a copy of it. An iterator object
    528 could provide such additional capabilities, but the iterator protocol only
    529 requires a :meth:`next` method.
    530 
    531 
    532 .. seealso::
    533 
    534    :pep:`234` - Iterators
    535       Written by Ka-Ping Yee and GvR; implemented  by the Python Labs crew, mostly by
    536       GvR and Tim Peters.
    537 
    538 .. ======================================================================
    539 
    540 
    541 PEP 255: Simple Generators
    542 ==========================
    543 
    544 Generators are another new feature, one that interacts with the introduction of
    545 iterators.
    546 
    547 You're doubtless familiar with how function calls work in Python or C.  When you
    548 call a function, it gets a private namespace where its local variables are
    549 created.  When the function reaches a :keyword:`return` statement, the local
    550 variables are destroyed and the resulting value is returned to the caller.  A
    551 later call to the same function will get a fresh new set of local variables.
    552 But, what if the local variables weren't thrown away on exiting a function?
    553 What if you could later resume the function where it left off?  This is what
    554 generators provide; they can be thought of as resumable functions.
    555 
    556 Here's the simplest example of a generator function::
    557 
    558    def generate_ints(N):
    559        for i in range(N):
    560            yield i
    561 
    562 A new keyword, :keyword:`yield`, was introduced for generators.  Any function
    563 containing a :keyword:`yield` statement is a generator function; this is
    564 detected by Python's bytecode compiler which compiles the function specially as
    565 a result.  Because a new keyword was introduced, generators must be explicitly
    566 enabled in a module by including a ``from __future__ import generators``
    567 statement near the top of the module's source code.  In Python 2.3 this
    568 statement will become unnecessary.
    569 
    570 When you call a generator function, it doesn't return a single value; instead it
    571 returns a generator object that supports the iterator protocol.  On executing
    572 the :keyword:`yield` statement, the generator outputs the value of ``i``,
    573 similar to a :keyword:`return` statement.  The big difference between
    574 :keyword:`yield` and a :keyword:`return` statement is that on reaching a
    575 :keyword:`yield` the generator's state of execution is suspended and local
    576 variables are preserved.  On the next call to the generator's ``next()`` method,
    577 the function will resume executing immediately after the :keyword:`yield`
    578 statement.  (For complicated reasons, the :keyword:`yield` statement isn't
    579 allowed inside the :keyword:`try` block of a :keyword:`try`...\
    580 :keyword:`finally` statement; read :pep:`255` for a full explanation of the
    581 interaction between :keyword:`yield` and exceptions.)
    582 
    583 Here's a sample usage of the :func:`generate_ints` generator::
    584 
    585    >>> gen = generate_ints(3)
    586    >>> gen
    587    <generator object at 0x8117f90>
    588    >>> gen.next()
    589    0
    590    >>> gen.next()
    591    1
    592    >>> gen.next()
    593    2
    594    >>> gen.next()
    595    Traceback (most recent call last):
    596      File "<stdin>", line 1, in ?
    597      File "<stdin>", line 2, in generate_ints
    598    StopIteration
    599 
    600 You could equally write ``for i in generate_ints(5)``, or ``a,b,c =
    601 generate_ints(3)``.
    602 
    603 Inside a generator function, the :keyword:`return` statement can only be used
    604 without a value, and signals the end of the procession of values; afterwards the
    605 generator cannot return any further values. :keyword:`return` with a value, such
    606 as ``return 5``, is a syntax error inside a generator function.  The end of the
    607 generator's results can also be indicated by raising :exc:`StopIteration`
    608 manually, or by just letting the flow of execution fall off the bottom of the
    609 function.
    610 
    611 You could achieve the effect of generators manually by writing your own class
    612 and storing all the local variables of the generator as instance variables.  For
    613 example, returning a list of integers could be done by setting ``self.count`` to
    614 0, and having the :meth:`next` method increment ``self.count`` and return it.
    615 However, for a moderately complicated generator, writing a corresponding class
    616 would be much messier. :file:`Lib/test/test_generators.py` contains a number of
    617 more interesting examples.  The simplest one implements an in-order traversal of
    618 a tree using generators recursively. ::
    619 
    620    # A recursive generator that generates Tree leaves in in-order.
    621    def inorder(t):
    622        if t:
    623            for x in inorder(t.left):
    624                yield x
    625            yield t.label
    626            for x in inorder(t.right):
    627                yield x
    628 
    629 Two other examples in :file:`Lib/test/test_generators.py` produce solutions for
    630 the N-Queens problem (placing $N$ queens on an $NxN$ chess board so that no
    631 queen threatens another) and the Knight's Tour (a route that takes a knight to
    632 every square of an $NxN$ chessboard without visiting any square twice).
    633 
    634 The idea of generators comes from other programming languages, especially Icon
    635 (https://www.cs.arizona.edu/icon/), where the idea of generators is central.  In
    636 Icon, every expression and function call behaves like a generator.  One example
    637 from "An Overview of the Icon Programming Language" at
    638 https://www.cs.arizona.edu/icon/docs/ipd266.htm gives an idea of what this looks
    639 like::
    640 
    641    sentence := "Store it in the neighboring harbor"
    642    if (i := find("or", sentence)) > 5 then write(i)
    643 
    644 In Icon the :func:`find` function returns the indexes at which the substring
    645 "or" is found: 3, 23, 33.  In the :keyword:`if` statement, ``i`` is first
    646 assigned a value of 3, but 3 is less than 5, so the comparison fails, and Icon
    647 retries it with the second value of 23.  23 is greater than 5, so the comparison
    648 now succeeds, and the code prints the value 23 to the screen.
    649 
    650 Python doesn't go nearly as far as Icon in adopting generators as a central
    651 concept.  Generators are considered a new part of the core Python language, but
    652 learning or using them isn't compulsory; if they don't solve any problems that
    653 you have, feel free to ignore them. One novel feature of Python's interface as
    654 compared to Icon's is that a generator's state is represented as a concrete
    655 object (the iterator) that can be passed around to other functions or stored in
    656 a data structure.
    657 
    658 
    659 .. seealso::
    660 
    661    :pep:`255` - Simple Generators
    662       Written by Neil Schemenauer, Tim Peters, Magnus Lie Hetland.  Implemented mostly
    663       by Neil Schemenauer and Tim Peters, with other fixes from the Python Labs crew.
    664 
    665 .. ======================================================================
    666 
    667 
    668 PEP 237: Unifying Long Integers and Integers
    669 ============================================
    670 
    671 In recent versions, the distinction between regular integers, which are 32-bit
    672 values on most machines, and long integers, which can be of arbitrary size, was
    673 becoming an annoyance.  For example, on platforms that support files larger than
    674 ``2**32`` bytes, the :meth:`tell` method of file objects has to return a long
    675 integer. However, there were various bits of Python that expected plain integers
    676 and would raise an error if a long integer was provided instead.  For example,
    677 in Python 1.5, only regular integers could be used as a slice index, and
    678 ``'abc'[1L:]`` would raise a :exc:`TypeError` exception with the message 'slice
    679 index must be int'.
    680 
    681 Python 2.2 will shift values from short to long integers as required. The 'L'
    682 suffix is no longer needed to indicate a long integer literal, as now the
    683 compiler will choose the appropriate type.  (Using the 'L' suffix will be
    684 discouraged in future 2.x versions of Python, triggering a warning in Python
    685 2.4, and probably dropped in Python 3.0.)  Many operations that used to raise an
    686 :exc:`OverflowError` will now return a long integer as their result.  For
    687 example::
    688 
    689    >>> 1234567890123
    690    1234567890123L
    691    >>> 2 ** 64
    692    18446744073709551616L
    693 
    694 In most cases, integers and long integers will now be treated identically.  You
    695 can still distinguish them with the :func:`type` built-in function, but that's
    696 rarely needed.
    697 
    698 
    699 .. seealso::
    700 
    701    :pep:`237` - Unifying Long Integers and Integers
    702       Written by Moshe Zadka and Guido van Rossum.  Implemented mostly by Guido van
    703       Rossum.
    704 
    705 .. ======================================================================
    706 
    707 
    708 PEP 238: Changing the Division Operator
    709 =======================================
    710 
    711 The most controversial change in Python 2.2 heralds the start of an effort to
    712 fix an old design flaw that's been in Python from the beginning. Currently
    713 Python's division operator, ``/``, behaves like C's division operator when
    714 presented with two integer arguments: it returns an integer result that's
    715 truncated down when there would be a fractional part.  For example, ``3/2`` is
    716 1, not 1.5, and ``(-1)/2`` is -1, not -0.5.  This means that the results of
    717 division can vary unexpectedly depending on the type of the two operands and
    718 because Python is dynamically typed, it can be difficult to determine the
    719 possible types of the operands.
    720 
    721 (The controversy is over whether this is *really* a design flaw, and whether
    722 it's worth breaking existing code to fix this.  It's caused endless discussions
    723 on python-dev, and in July 2001 erupted into a storm of acidly sarcastic
    724 postings on :newsgroup:`comp.lang.python`. I won't argue for either side here
    725 and will stick to describing what's  implemented in 2.2.  Read :pep:`238` for a
    726 summary of arguments and counter-arguments.)
    727 
    728 Because this change might break code, it's being introduced very gradually.
    729 Python 2.2 begins the transition, but the switch won't be complete until Python
    730 3.0.
    731 
    732 First, I'll borrow some terminology from :pep:`238`.  "True division" is the
    733 division that most non-programmers are familiar with: 3/2 is 1.5, 1/4 is 0.25,
    734 and so forth.  "Floor division" is what Python's ``/`` operator currently does
    735 when given integer operands; the result is the floor of the value returned by
    736 true division.  "Classic division" is the current mixed behaviour of ``/``; it
    737 returns the result of floor division when the operands are integers, and returns
    738 the result of true division when one of the operands is a floating-point number.
    739 
    740 Here are the changes 2.2 introduces:
    741 
    742 * A new operator, ``//``, is the floor division operator. (Yes, we know it looks
    743   like C++'s comment symbol.)  ``//`` *always* performs floor division no matter
    744   what the types of its operands are, so ``1 // 2`` is 0 and ``1.0 // 2.0`` is
    745   also 0.0.
    746 
    747   ``//`` is always available in Python 2.2; you don't need to enable it using a
    748   ``__future__`` statement.
    749 
    750 * By including a ``from __future__ import division`` in a module, the ``/``
    751   operator will be changed to return the result of true division, so ``1/2`` is
    752   0.5.  Without the ``__future__`` statement, ``/`` still means classic division.
    753   The default meaning of ``/`` will not change until Python 3.0.
    754 
    755 * Classes can define methods called :meth:`__truediv__` and :meth:`__floordiv__`
    756   to overload the two division operators.  At the C level, there are also slots in
    757   the :c:type:`PyNumberMethods` structure so extension types can define the two
    758   operators.
    759 
    760 * Python 2.2 supports some command-line arguments for testing whether code will
    761   work with the changed division semantics.  Running python with :option:`-Q
    762   warn <-Q>` will cause a warning to be issued whenever division is applied to two
    763   integers.  You can use this to find code that's affected by the change and fix
    764   it.  By default, Python 2.2 will simply perform classic division without a
    765   warning; the warning will be turned on by default in Python 2.3.
    766 
    767 
    768 .. seealso::
    769 
    770    :pep:`238` - Changing the Division Operator
    771       Written by Moshe Zadka and  Guido van Rossum.  Implemented by Guido van Rossum..
    772 
    773 .. ======================================================================
    774 
    775 
    776 Unicode Changes
    777 ===============
    778 
    779 Python's Unicode support has been enhanced a bit in 2.2.  Unicode strings are
    780 usually stored as UCS-2, as 16-bit unsigned integers. Python 2.2 can also be
    781 compiled to use UCS-4, 32-bit unsigned integers, as its internal encoding by
    782 supplying :option:`!--enable-unicode=ucs4` to the configure script.   (It's also
    783 possible to specify :option:`!--disable-unicode` to completely disable Unicode
    784 support.)
    785 
    786 When built to use UCS-4 (a "wide Python"), the interpreter can natively handle
    787 Unicode characters from U+000000 to U+110000, so the range of legal values for
    788 the :func:`unichr` function is expanded accordingly.  Using an interpreter
    789 compiled to use UCS-2 (a "narrow Python"), values greater than 65535 will still
    790 cause :func:`unichr` to raise a :exc:`ValueError` exception. This is all
    791 described in :pep:`261`, "Support for 'wide' Unicode characters"; consult it for
    792 further details.
    793 
    794 Another change is simpler to explain. Since their introduction, Unicode strings
    795 have supported an :meth:`encode` method to convert the string to a selected
    796 encoding such as UTF-8 or Latin-1.  A symmetric ``decode([*encoding*])``
    797 method has been added to 8-bit strings (though not to Unicode strings) in 2.2.
    798 :meth:`decode` assumes that the string is in the specified encoding and decodes
    799 it, returning whatever is returned by the codec.
    800 
    801 Using this new feature, codecs have been added for tasks not directly related to
    802 Unicode.  For example, codecs have been added for uu-encoding, MIME's base64
    803 encoding, and compression with the :mod:`zlib` module::
    804 
    805    >>> s = """Here is a lengthy piece of redundant, overly verbose,
    806    ... and repetitive text.
    807    ... """
    808    >>> data = s.encode('zlib')
    809    >>> data
    810    'x\x9c\r\xc9\xc1\r\x80 \x10\x04\xc0?Ul...'
    811    >>> data.decode('zlib')
    812    'Here is a lengthy piece of redundant, overly verbose,\nand repetitive text.\n'
    813    >>> print s.encode('uu')
    814    begin 666 <data>
    815    M2&5R92!I<R!A(&QE;F=T:'D@<&EE8V4@;V8@<F5D=6YD86YT+"!O=F5R;'D@
    816    >=F5R8F]S92P*86YD(')E<&5T:71I=F4@=&5X="X*
    817 
    818    end
    819    >>> "sheesh".encode('rot-13')
    820    'furrfu'
    821 
    822 To convert a class instance to Unicode, a :meth:`__unicode__` method can be
    823 defined by a class, analogous to :meth:`__str__`.
    824 
    825 :meth:`encode`, :meth:`decode`, and :meth:`__unicode__` were implemented by
    826 Marc-Andr Lemburg.  The changes to support using UCS-4 internally were
    827 implemented by Fredrik Lundh and Martin von Lwis.
    828 
    829 
    830 .. seealso::
    831 
    832    :pep:`261` - Support for 'wide' Unicode characters
    833       Written by Paul Prescod.
    834 
    835 .. ======================================================================
    836 
    837 
    838 PEP 227: Nested Scopes
    839 ======================
    840 
    841 In Python 2.1, statically nested scopes were added as an optional feature, to be
    842 enabled by a ``from __future__ import nested_scopes`` directive.  In 2.2 nested
    843 scopes no longer need to be specially enabled, and are now always present.  The
    844 rest of this section is a copy of the description of nested scopes from my
    845 "What's New in Python 2.1" document; if you read it when 2.1 came out, you can
    846 skip the rest of this section.
    847 
    848 The largest change introduced in Python 2.1, and made complete in 2.2, is to
    849 Python's scoping rules.  In Python 2.0, at any given time there are at most
    850 three namespaces used to look up variable names: local, module-level, and the
    851 built-in namespace.  This often surprised people because it didn't match their
    852 intuitive expectations.  For example, a nested recursive function definition
    853 doesn't work::
    854 
    855    def f():
    856        ...
    857        def g(value):
    858            ...
    859            return g(value-1) + 1
    860        ...
    861 
    862 The function :func:`g` will always raise a :exc:`NameError` exception, because
    863 the binding of the name ``g`` isn't in either its local namespace or in the
    864 module-level namespace.  This isn't much of a problem in practice (how often do
    865 you recursively define interior functions like this?), but this also made using
    866 the :keyword:`lambda` statement clumsier, and this was a problem in practice.
    867 In code which uses :keyword:`lambda` you can often find local variables being
    868 copied by passing them as the default values of arguments. ::
    869 
    870    def find(self, name):
    871        "Return list of any entries equal to 'name'"
    872        L = filter(lambda x, name=name: x == name,
    873                   self.list_attribute)
    874        return L
    875 
    876 The readability of Python code written in a strongly functional style suffers
    877 greatly as a result.
    878 
    879 The most significant change to Python 2.2 is that static scoping has been added
    880 to the language to fix this problem.  As a first effect, the ``name=name``
    881 default argument is now unnecessary in the above example.  Put simply, when a
    882 given variable name is not assigned a value within a function (by an assignment,
    883 or the :keyword:`def`, :keyword:`class`, or :keyword:`import` statements),
    884 references to the variable will be looked up in the local namespace of the
    885 enclosing scope.  A more detailed explanation of the rules, and a dissection of
    886 the implementation, can be found in the PEP.
    887 
    888 This change may cause some compatibility problems for code where the same
    889 variable name is used both at the module level and as a local variable within a
    890 function that contains further function definitions. This seems rather unlikely
    891 though, since such code would have been pretty confusing to read in the first
    892 place.
    893 
    894 One side effect of the change is that the ``from module import *`` and
    895 :keyword:`exec` statements have been made illegal inside a function scope under
    896 certain conditions.  The Python reference manual has said all along that ``from
    897 module import *`` is only legal at the top level of a module, but the CPython
    898 interpreter has never enforced this before.  As part of the implementation of
    899 nested scopes, the compiler which turns Python source into bytecodes has to
    900 generate different code to access variables in a containing scope.  ``from
    901 module import *`` and :keyword:`exec` make it impossible for the compiler to
    902 figure this out, because they add names to the local namespace that are
    903 unknowable at compile time. Therefore, if a function contains function
    904 definitions or :keyword:`lambda` expressions with free variables, the compiler
    905 will flag this by raising a :exc:`SyntaxError` exception.
    906 
    907 To make the preceding explanation a bit clearer, here's an example::
    908 
    909    x = 1
    910    def f():
    911        # The next line is a syntax error
    912        exec 'x=2'
    913        def g():
    914            return x
    915 
    916 Line 4 containing the :keyword:`exec` statement is a syntax error, since
    917 :keyword:`exec` would define a new local variable named ``x`` whose value should
    918 be accessed by :func:`g`.
    919 
    920 This shouldn't be much of a limitation, since :keyword:`exec` is rarely used in
    921 most Python code (and when it is used, it's often a sign of a poor design
    922 anyway).
    923 
    924 
    925 .. seealso::
    926 
    927    :pep:`227` - Statically Nested Scopes
    928       Written and implemented by Jeremy Hylton.
    929 
    930 .. ======================================================================
    931 
    932 
    933 New and Improved Modules
    934 ========================
    935 
    936 * The :mod:`xmlrpclib` module was contributed to the standard library by Fredrik
    937   Lundh, providing support for writing XML-RPC clients.  XML-RPC is a simple
    938   remote procedure call protocol built on top of HTTP and XML. For example, the
    939   following snippet retrieves a list of RSS channels from the O'Reilly Network,
    940   and then  lists the recent headlines for one channel::
    941 
    942      import xmlrpclib
    943      s = xmlrpclib.Server(
    944            'http://www.oreillynet.com/meerkat/xml-rpc/server.php')
    945      channels = s.meerkat.getChannels()
    946      # channels is a list of dictionaries, like this:
    947      # [{'id': 4, 'title': 'Freshmeat Daily News'}
    948      #  {'id': 190, 'title': '32Bits Online'},
    949      #  {'id': 4549, 'title': '3DGamers'}, ... ]
    950 
    951      # Get the items for one channel
    952      items = s.meerkat.getItems( {'channel': 4} )
    953 
    954      # 'items' is another list of dictionaries, like this:
    955      # [{'link': 'http://freshmeat.net/releases/52719/',
    956      #   'description': 'A utility which converts HTML to XSL FO.',
    957      #   'title': 'html2fo 0.3 (Default)'}, ... ]
    958 
    959   The :mod:`SimpleXMLRPCServer` module makes it easy to create straightforward
    960   XML-RPC servers.  See http://www.xmlrpc.com/ for more information about XML-RPC.
    961 
    962 * The new :mod:`hmac` module implements the HMAC algorithm described by
    963   :rfc:`2104`. (Contributed by Gerhard Hring.)
    964 
    965 * Several functions that originally returned lengthy tuples now return
    966   pseudo-sequences that still behave like tuples but also have mnemonic attributes such
    967   as memberst_mtime or :attr:`tm_year`. The enhanced functions include
    968   :func:`stat`, :func:`fstat`, :func:`statvfs`, and :func:`fstatvfs` in the
    969   :mod:`os` module, and :func:`localtime`, :func:`gmtime`, and :func:`strptime` in
    970   the :mod:`time` module.
    971 
    972   For example, to obtain a file's size using the old tuples, you'd end up writing
    973   something like ``file_size = os.stat(filename)[stat.ST_SIZE]``, but now this can
    974   be written more clearly as ``file_size = os.stat(filename).st_size``.
    975 
    976   The original patch for this feature was contributed by Nick Mathewson.
    977 
    978 * The Python profiler has been extensively reworked and various errors in its
    979   output have been corrected.  (Contributed by Fred L. Drake, Jr. and Tim Peters.)
    980 
    981 * The :mod:`socket` module can be compiled to support IPv6; specify the
    982   :option:`!--enable-ipv6` option to Python's configure script.  (Contributed by
    983   Jun-ichiro "itojun" Hagino.)
    984 
    985 * Two new format characters were added to the :mod:`struct` module for 64-bit
    986   integers on platforms that support the C :c:type:`long long` type.  ``q`` is for
    987   a signed 64-bit integer, and ``Q`` is for an unsigned one.  The value is
    988   returned in Python's long integer type.  (Contributed by Tim Peters.)
    989 
    990 * In the interpreter's interactive mode, there's a new built-in function
    991   :func:`help` that uses the :mod:`pydoc` module introduced in Python 2.1 to
    992   provide interactive help. ``help(object)`` displays any available help text
    993   about *object*.  :func:`help` with no argument puts you in an online help
    994   utility, where you can enter the names of functions, classes, or modules to read
    995   their help text. (Contributed by Guido van Rossum, using Ka-Ping Yee's
    996   :mod:`pydoc` module.)
    997 
    998 * Various bugfixes and performance improvements have been made to the SRE engine
    999   underlying the :mod:`re` module.  For example, the :func:`re.sub` and
   1000   :func:`re.split` functions have been rewritten in C.  Another contributed patch
   1001   speeds up certain Unicode character ranges by a factor of two, and a new
   1002   :meth:`finditer`  method that returns an iterator over all the non-overlapping
   1003   matches in  a given string.  (SRE is maintained by Fredrik Lundh.  The
   1004   BIGCHARSET patch was contributed by Martin von Lwis.)
   1005 
   1006 * The :mod:`smtplib` module now supports :rfc:`2487`, "Secure SMTP over TLS", so
   1007   it's now possible to encrypt the SMTP traffic between a Python program and the
   1008   mail transport agent being handed a message.  :mod:`smtplib` also supports SMTP
   1009   authentication.  (Contributed by Gerhard Hring.)
   1010 
   1011 * The :mod:`imaplib` module, maintained by Piers Lauder, has support for several
   1012   new extensions: the NAMESPACE extension defined in :rfc:`2342`, SORT, GETACL and
   1013   SETACL.  (Contributed by Anthony Baxter and Michel Pelletier.)
   1014 
   1015 * The :mod:`rfc822` module's parsing of email addresses is now compliant with
   1016   :rfc:`2822`, an update to :rfc:`822`.  (The module's name is *not* going to be
   1017   changed to ``rfc2822``.)  A new package, :mod:`email`, has also been added for
   1018   parsing and generating e-mail messages.  (Contributed by Barry Warsaw, and
   1019   arising out of his work on Mailman.)
   1020 
   1021 * The :mod:`difflib` module now contains a new :class:`Differ` class for
   1022   producing human-readable lists of changes (a "delta") between two sequences of
   1023   lines of text.  There are also two generator functions, :func:`ndiff` and
   1024   :func:`restore`, which respectively return a delta from two sequences, or one of
   1025   the original sequences from a delta. (Grunt work contributed by David Goodger,
   1026   from ndiff.py code by Tim Peters who then did the generatorization.)
   1027 
   1028 * New constants :const:`ascii_letters`, :const:`ascii_lowercase`, and
   1029   :const:`ascii_uppercase` were added to the :mod:`string` module.  There were
   1030   several modules in the standard library that used :const:`string.letters` to
   1031   mean the ranges A-Za-z, but that assumption is incorrect when locales are in
   1032   use, because :const:`string.letters` varies depending on the set of legal
   1033   characters defined by the current locale.  The buggy modules have all been fixed
   1034   to use :const:`ascii_letters` instead. (Reported by an unknown person; fixed by
   1035   Fred L. Drake, Jr.)
   1036 
   1037 * The :mod:`mimetypes` module now makes it easier to use alternative MIME-type
   1038   databases by the addition of a :class:`MimeTypes` class, which takes a list of
   1039   filenames to be parsed.  (Contributed by Fred L. Drake, Jr.)
   1040 
   1041 * A :class:`Timer` class was added to the :mod:`threading` module that allows
   1042   scheduling an activity to happen at some future time.  (Contributed by Itamar
   1043   Shtull-Trauring.)
   1044 
   1045 .. ======================================================================
   1046 
   1047 
   1048 Interpreter Changes and Fixes
   1049 =============================
   1050 
   1051 Some of the changes only affect people who deal with the Python interpreter at
   1052 the C level because they're writing Python extension modules, embedding the
   1053 interpreter, or just hacking on the interpreter itself. If you only write Python
   1054 code, none of the changes described here will affect you very much.
   1055 
   1056 * Profiling and tracing functions can now be implemented in C, which can operate
   1057   at much higher speeds than Python-based functions and should reduce the overhead
   1058   of profiling and tracing.  This  will be of interest to authors of development
   1059   environments for Python.  Two new C functions were added to Python's API,
   1060   :c:func:`PyEval_SetProfile` and :c:func:`PyEval_SetTrace`. The existing
   1061   :func:`sys.setprofile` and :func:`sys.settrace` functions still exist, and have
   1062   simply been changed to use the new C-level interface.  (Contributed by Fred L.
   1063   Drake, Jr.)
   1064 
   1065 * Another low-level API, primarily of interest to implementors of Python
   1066   debuggers and development tools, was added. :c:func:`PyInterpreterState_Head` and
   1067   :c:func:`PyInterpreterState_Next` let a caller walk through all the existing
   1068   interpreter objects; :c:func:`PyInterpreterState_ThreadHead` and
   1069   :c:func:`PyThreadState_Next` allow looping over all the thread states for a given
   1070   interpreter.  (Contributed by David Beazley.)
   1071 
   1072 * The C-level interface to the garbage collector has been changed to make it
   1073   easier to write extension types that support garbage collection and to debug
   1074   misuses of the functions. Various functions have slightly different semantics,
   1075   so a bunch of functions had to be renamed.  Extensions that use the old API will
   1076   still compile but will *not* participate in garbage collection, so updating them
   1077   for 2.2 should be considered fairly high priority.
   1078 
   1079   To upgrade an extension module to the new API, perform the following steps:
   1080 
   1081 * Rename :c:func:`Py_TPFLAGS_GC` to :c:func:`PyTPFLAGS_HAVE_GC`.
   1082 
   1083 * Use :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar` to allocate
   1084     objects, and :c:func:`PyObject_GC_Del` to deallocate them.
   1085 
   1086 * Rename :c:func:`PyObject_GC_Init` to :c:func:`PyObject_GC_Track` and
   1087     :c:func:`PyObject_GC_Fini` to :c:func:`PyObject_GC_UnTrack`.
   1088 
   1089 * Remove :c:func:`PyGC_HEAD_SIZE` from object size calculations.
   1090 
   1091 * Remove calls to :c:func:`PyObject_AS_GC` and :c:func:`PyObject_FROM_GC`.
   1092 
   1093 * A new ``et`` format sequence was added to :c:func:`PyArg_ParseTuple`; ``et``
   1094   takes both a parameter and an encoding name, and converts the parameter to the
   1095   given encoding if the parameter turns out to be a Unicode string, or leaves it
   1096   alone if it's an 8-bit string, assuming it to already be in the desired
   1097   encoding.  This differs from the ``es`` format character, which assumes that
   1098   8-bit strings are in Python's default ASCII encoding and converts them to the
   1099   specified new encoding. (Contributed by M.-A. Lemburg, and used for the MBCS
   1100   support on Windows described in the following section.)
   1101 
   1102 * A different argument parsing function, :c:func:`PyArg_UnpackTuple`, has been
   1103   added that's simpler and presumably faster.  Instead of specifying a format
   1104   string, the caller simply gives the minimum and maximum number of arguments
   1105   expected, and a set of pointers to :c:type:`PyObject\*` variables that will be
   1106   filled in with argument values.
   1107 
   1108 * Two new flags :const:`METH_NOARGS` and :const:`METH_O` are available in method
   1109   definition tables to simplify implementation of methods with no arguments or a
   1110   single untyped argument. Calling such methods is more efficient than calling a
   1111   corresponding method that uses :const:`METH_VARARGS`.  Also, the old
   1112   :const:`METH_OLDARGS` style of writing C methods is  now officially deprecated.
   1113 
   1114 * Two new wrapper functions, :c:func:`PyOS_snprintf` and :c:func:`PyOS_vsnprintf`
   1115   were added to provide  cross-platform implementations for the relatively new
   1116   :c:func:`snprintf` and :c:func:`vsnprintf` C lib APIs. In contrast to the standard
   1117   :c:func:`sprintf` and :c:func:`vsprintf` functions, the Python versions check the
   1118   bounds of the buffer used to protect against buffer overruns. (Contributed by
   1119   M.-A. Lemburg.)
   1120 
   1121 * The :c:func:`_PyTuple_Resize` function has lost an unused parameter, so now it
   1122   takes 2 parameters instead of 3.  The third argument was never used, and can
   1123   simply be discarded when porting code from earlier versions to Python 2.2.
   1124 
   1125 .. ======================================================================
   1126 
   1127 
   1128 Other Changes and Fixes
   1129 =======================
   1130 
   1131 As usual there were a bunch of other improvements and bugfixes scattered
   1132 throughout the source tree.  A search through the CVS change logs finds there
   1133 were 527 patches applied and 683 bugs fixed between Python 2.1 and 2.2; 2.2.1
   1134 applied 139 patches and fixed 143 bugs; 2.2.2 applied 106 patches and fixed 82
   1135 bugs.  These figures are likely to be underestimates.
   1136 
   1137 Some of the more notable changes are:
   1138 
   1139 * The code for the MacOS port for Python, maintained by Jack Jansen, is now kept
   1140   in the main Python CVS tree, and many changes have been made to support MacOS X.
   1141 
   1142   The most significant change is the ability to build Python as a framework,
   1143   enabled by supplying the :option:`!--enable-framework` option to the configure
   1144   script when compiling Python.  According to Jack Jansen, "This installs a
   1145   self-contained Python installation plus the OS X framework "glue" into
   1146   :file:`/Library/Frameworks/Python.framework` (or another location of choice).
   1147   For now there is little immediate added benefit to this (actually, there is the
   1148   disadvantage that you have to change your PATH to be able to find Python), but
   1149   it is the basis for creating a full-blown Python application, porting the
   1150   MacPython IDE, possibly using Python as a standard OSA scripting language and
   1151   much more."
   1152 
   1153   Most of the MacPython toolbox modules, which interface to MacOS APIs such as
   1154   windowing, QuickTime, scripting, etc. have been ported to OS X, but they've been
   1155   left commented out in :file:`setup.py`.  People who want to experiment with
   1156   these modules can uncomment them manually.
   1157 
   1158   .. Jack's original comments:
   1159      The main change is the possibility to build Python as a
   1160      framework. This installs a self-contained Python installation plus the
   1161      OSX framework "glue" into /Library/Frameworks/Python.framework (or
   1162      another location of choice). For now there is little immediate added
   1163      benefit to this (actually, there is the disadvantage that you have to
   1164      change your PATH to be able to find Python), but it is the basis for
   1165      creating a fullblown Python application, porting the MacPython IDE,
   1166      possibly using Python as a standard OSA scripting language and much
   1167      more. You enable this with "configure --enable-framework".
   1168      The other change is that most MacPython toolbox modules, which
   1169      interface to all the MacOS APIs such as windowing, quicktime,
   1170      scripting, etc. have been ported. Again, most of these are not of
   1171      immediate use, as they need a full application to be really useful, so
   1172      they have been commented out in setup.py. People wanting to experiment
   1173      can uncomment them. Gestalt and Internet Config modules are enabled by
   1174      default.
   1175 
   1176 * Keyword arguments passed to built-in functions that don't take them now cause a
   1177   :exc:`TypeError` exception to be raised, with the message "*function* takes no
   1178   keyword arguments".
   1179 
   1180 * Weak references, added in Python 2.1 as an extension module, are now part of
   1181   the core because they're used in the implementation of new-style classes.  The
   1182   :exc:`ReferenceError` exception has therefore moved from the :mod:`weakref`
   1183   module to become a built-in exception.
   1184 
   1185 * A new script, :file:`Tools/scripts/cleanfuture.py` by Tim Peters,
   1186   automatically removes obsolete ``__future__`` statements from Python source
   1187   code.
   1188 
   1189 * An additional *flags* argument has been added to the built-in function
   1190   :func:`compile`, so the behaviour of ``__future__`` statements can now be
   1191   correctly observed in simulated shells, such as those presented by IDLE and
   1192   other development environments.  This is described in :pep:`264`. (Contributed
   1193   by Michael Hudson.)
   1194 
   1195 * The new license introduced with Python 1.6 wasn't GPL-compatible.  This is
   1196   fixed by some minor textual changes to the 2.2 license, so it's now legal to
   1197   embed Python inside a GPLed program again.  Note that Python itself is not
   1198   GPLed, but instead is under a license that's essentially equivalent to the BSD
   1199   license, same as it always was.  The license changes were also applied to the
   1200   Python 2.0.1 and 2.1.1 releases.
   1201 
   1202 * When presented with a Unicode filename on Windows, Python will now convert it
   1203   to an MBCS encoded string, as used by the Microsoft file APIs.  As MBCS is
   1204   explicitly used by the file APIs, Python's choice of ASCII as the default
   1205   encoding turns out to be an annoyance.  On Unix, the locale's character set is
   1206   used if ``locale.nl_langinfo(CODESET)`` is available.  (Windows support was
   1207   contributed by Mark Hammond with assistance from Marc-Andr Lemburg. Unix
   1208   support was added by Martin von Lwis.)
   1209 
   1210 * Large file support is now enabled on Windows.  (Contributed by Tim Peters.)
   1211 
   1212 * The :file:`Tools/scripts/ftpmirror.py` script now parses a :file:`.netrc`
   1213   file, if you have one. (Contributed by Mike Romberg.)
   1214 
   1215 * Some features of the object returned by the :func:`xrange` function are now
   1216   deprecated, and trigger warnings when they're accessed; they'll disappear in
   1217   Python 2.3. :class:`xrange` objects tried to pretend they were full sequence
   1218   types by supporting slicing, sequence multiplication, and the :keyword:`in`
   1219   operator, but these features were rarely used and therefore buggy.  The
   1220   :meth:`tolist` method and the :attr:`start`, :attr:`stop`, and :attr:`step`
   1221   attributes are also being deprecated.  At the C level, the fourth argument to
   1222   the :c:func:`PyRange_New` function, ``repeat``, has also been deprecated.
   1223 
   1224 * There were a bunch of patches to the dictionary implementation, mostly to fix
   1225   potential core dumps if a dictionary contains objects that sneakily changed
   1226   their hash value, or mutated the dictionary they were contained in. For a while
   1227   python-dev fell into a gentle rhythm of Michael Hudson finding a case that
   1228   dumped core, Tim Peters fixing the bug, Michael finding another case, and round
   1229   and round it went.
   1230 
   1231 * On Windows, Python can now be compiled with Borland C thanks to a number of
   1232   patches contributed by Stephen Hansen, though the result isn't fully functional
   1233   yet.  (But this *is* progress...)
   1234 
   1235 * Another Windows enhancement: Wise Solutions generously offered PythonLabs use
   1236   of their InstallerMaster 8.1 system.  Earlier PythonLabs Windows installers used
   1237   Wise 5.0a, which was beginning to show its age.  (Packaged up by Tim Peters.)
   1238 
   1239 * Files ending in ``.pyw`` can now be imported on Windows. ``.pyw`` is a
   1240   Windows-only thing, used to indicate that a script needs to be run using
   1241   PYTHONW.EXE instead of PYTHON.EXE in order to prevent a DOS console from popping
   1242   up to display the output.  This patch makes it possible to import such scripts,
   1243   in case they're also usable as modules.  (Implemented by David Bolen.)
   1244 
   1245 * On platforms where Python uses the C :c:func:`dlopen` function  to load
   1246   extension modules, it's now possible to set the flags used  by :c:func:`dlopen`
   1247   using the :func:`sys.getdlopenflags` and :func:`sys.setdlopenflags` functions.
   1248   (Contributed by Bram Stolk.)
   1249 
   1250 * The :func:`pow` built-in function no longer supports 3 arguments when
   1251   floating-point numbers are supplied. ``pow(x, y, z)`` returns ``(x**y) % z``,
   1252   but this is never useful for floating point numbers, and the final result varies
   1253   unpredictably depending on the platform.  A call such as ``pow(2.0, 8.0, 7.0)``
   1254   will now raise a :exc:`TypeError` exception.
   1255 
   1256 .. ======================================================================
   1257 
   1258 
   1259 Acknowledgements
   1260 ================
   1261 
   1262 The author would like to thank the following people for offering suggestions,
   1263 corrections and assistance with various drafts of this article: Fred Bremmer,
   1264 Keith Briggs, Andrew Dalke, Fred L. Drake, Jr., Carel Fellinger, David Goodger,
   1265 Mark Hammond, Stephen Hansen, Michael Hudson, Jack Jansen, Marc-Andr Lemburg,
   1266 Martin von Lwis, Fredrik Lundh, Michael McLay, Nick Mathewson, Paul Moore,
   1267 Gustavo Niemeyer, Don O'Donnell, Joonas Paalasma, Tim Peters, Jens Quade, Tom
   1268 Reinhardt, Neil Schemenauer, Guido van Rossum, Greg Ward, Edward Welbourne.
   1269 
   1270