1 :mod:`gettext` --- Multilingual internationalization services 2 ============================================================= 3 4 .. module:: gettext 5 :synopsis: Multilingual internationalization services. 6 7 .. moduleauthor:: Barry A. Warsaw <barry (a] python.org> 8 .. sectionauthor:: Barry A. Warsaw <barry (a] python.org> 9 10 **Source code:** :source:`Lib/gettext.py` 11 12 -------------- 13 14 The :mod:`gettext` module provides internationalization (I18N) and localization 15 (L10N) services for your Python modules and applications. It supports both the 16 GNU ``gettext`` message catalog API and a higher level, class-based API that may 17 be more appropriate for Python files. The interface described below allows you 18 to write your module and application messages in one natural language, and 19 provide a catalog of translated messages for running under different natural 20 languages. 21 22 Some hints on localizing your Python modules and applications are also given. 23 24 25 GNU :program:`gettext` API 26 -------------------------- 27 28 The :mod:`gettext` module defines the following API, which is very similar to 29 the GNU :program:`gettext` API. If you use this API you will affect the 30 translation of your entire application globally. Often this is what you want if 31 your application is monolingual, with the choice of language dependent on the 32 locale of your user. If you are localizing a Python module, or if your 33 application needs to switch languages on the fly, you probably want to use the 34 class-based API instead. 35 36 37 .. function:: bindtextdomain(domain, localedir=None) 38 39 Bind the *domain* to the locale directory *localedir*. More concretely, 40 :mod:`gettext` will look for binary :file:`.mo` files for the given domain using 41 the path (on Unix): :file:`localedir/language/LC_MESSAGES/domain.mo`, where 42 *languages* is searched for in the environment variables :envvar:`LANGUAGE`, 43 :envvar:`LC_ALL`, :envvar:`LC_MESSAGES`, and :envvar:`LANG` respectively. 44 45 If *localedir* is omitted or ``None``, then the current binding for *domain* is 46 returned. [#]_ 47 48 49 .. function:: bind_textdomain_codeset(domain, codeset=None) 50 51 Bind the *domain* to *codeset*, changing the encoding of strings returned by the 52 :func:`gettext` family of functions. If *codeset* is omitted, then the current 53 binding is returned. 54 55 56 .. function:: textdomain(domain=None) 57 58 Change or query the current global domain. If *domain* is ``None``, then the 59 current global domain is returned, otherwise the global domain is set to 60 *domain*, which is returned. 61 62 63 .. function:: gettext(message) 64 65 Return the localized translation of *message*, based on the current global 66 domain, language, and locale directory. This function is usually aliased as 67 :func:`_` in the local namespace (see examples below). 68 69 70 .. function:: lgettext(message) 71 72 Equivalent to :func:`gettext`, but the translation is returned in the 73 preferred system encoding, if no other encoding was explicitly set with 74 :func:`bind_textdomain_codeset`. 75 76 77 .. function:: dgettext(domain, message) 78 79 Like :func:`gettext`, but look the message up in the specified *domain*. 80 81 82 .. function:: ldgettext(domain, message) 83 84 Equivalent to :func:`dgettext`, but the translation is returned in the 85 preferred system encoding, if no other encoding was explicitly set with 86 :func:`bind_textdomain_codeset`. 87 88 89 .. function:: ngettext(singular, plural, n) 90 91 Like :func:`gettext`, but consider plural forms. If a translation is found, 92 apply the plural formula to *n*, and return the resulting message (some 93 languages have more than two plural forms). If no translation is found, return 94 *singular* if *n* is 1; return *plural* otherwise. 95 96 The Plural formula is taken from the catalog header. It is a C or Python 97 expression that has a free variable *n*; the expression evaluates to the index 98 of the plural in the catalog. See 99 `the GNU gettext documentation <https://www.gnu.org/software/gettext/manual/gettext.html>`__ 100 for the precise syntax to be used in :file:`.po` files and the 101 formulas for a variety of languages. 102 103 104 .. function:: lngettext(singular, plural, n) 105 106 Equivalent to :func:`ngettext`, but the translation is returned in the 107 preferred system encoding, if no other encoding was explicitly set with 108 :func:`bind_textdomain_codeset`. 109 110 111 .. function:: dngettext(domain, singular, plural, n) 112 113 Like :func:`ngettext`, but look the message up in the specified *domain*. 114 115 116 .. function:: ldngettext(domain, singular, plural, n) 117 118 Equivalent to :func:`dngettext`, but the translation is returned in the 119 preferred system encoding, if no other encoding was explicitly set with 120 :func:`bind_textdomain_codeset`. 121 122 123 Note that GNU :program:`gettext` also defines a :func:`dcgettext` method, but 124 this was deemed not useful and so it is currently unimplemented. 125 126 Here's an example of typical usage for this API:: 127 128 import gettext 129 gettext.bindtextdomain('myapplication', '/path/to/my/language/directory') 130 gettext.textdomain('myapplication') 131 _ = gettext.gettext 132 # ... 133 print(_('This is a translatable string.')) 134 135 136 Class-based API 137 --------------- 138 139 The class-based API of the :mod:`gettext` module gives you more flexibility and 140 greater convenience than the GNU :program:`gettext` API. It is the recommended 141 way of localizing your Python applications and modules. :mod:`gettext` defines 142 a "translations" class which implements the parsing of GNU :file:`.mo` format 143 files, and has methods for returning strings. Instances of this "translations" 144 class can also install themselves in the built-in namespace as the function 145 :func:`_`. 146 147 148 .. function:: find(domain, localedir=None, languages=None, all=False) 149 150 This function implements the standard :file:`.mo` file search algorithm. It 151 takes a *domain*, identical to what :func:`textdomain` takes. Optional 152 *localedir* is as in :func:`bindtextdomain` Optional *languages* is a list of 153 strings, where each string is a language code. 154 155 If *localedir* is not given, then the default system locale directory is used. 156 [#]_ If *languages* is not given, then the following environment variables are 157 searched: :envvar:`LANGUAGE`, :envvar:`LC_ALL`, :envvar:`LC_MESSAGES`, and 158 :envvar:`LANG`. The first one returning a non-empty value is used for the 159 *languages* variable. The environment variables should contain a colon separated 160 list of languages, which will be split on the colon to produce the expected list 161 of language code strings. 162 163 :func:`find` then expands and normalizes the languages, and then iterates 164 through them, searching for an existing file built of these components: 165 166 :file:`{localedir}/{language}/LC_MESSAGES/{domain}.mo` 167 168 The first such file name that exists is returned by :func:`find`. If no such 169 file is found, then ``None`` is returned. If *all* is given, it returns a list 170 of all file names, in the order in which they appear in the languages list or 171 the environment variables. 172 173 174 .. function:: translation(domain, localedir=None, languages=None, class_=None, fallback=False, codeset=None) 175 176 Return a :class:`Translations` instance based on the *domain*, *localedir*, 177 and *languages*, which are first passed to :func:`find` to get a list of the 178 associated :file:`.mo` file paths. Instances with identical :file:`.mo` file 179 names are cached. The actual class instantiated is either *class_* if 180 provided, otherwise :class:`GNUTranslations`. The class's constructor must 181 take a single :term:`file object` argument. If provided, *codeset* will change 182 the charset used to encode translated strings in the :meth:`lgettext` and 183 :meth:`lngettext` methods. 184 185 If multiple files are found, later files are used as fallbacks for earlier ones. 186 To allow setting the fallback, :func:`copy.copy` is used to clone each 187 translation object from the cache; the actual instance data is still shared with 188 the cache. 189 190 If no :file:`.mo` file is found, this function raises :exc:`OSError` if 191 *fallback* is false (which is the default), and returns a 192 :class:`NullTranslations` instance if *fallback* is true. 193 194 .. versionchanged:: 3.3 195 :exc:`IOError` used to be raised instead of :exc:`OSError`. 196 197 198 .. function:: install(domain, localedir=None, codeset=None, names=None) 199 200 This installs the function :func:`_` in Python's builtins namespace, based on 201 *domain*, *localedir*, and *codeset* which are passed to the function 202 :func:`translation`. 203 204 For the *names* parameter, please see the description of the translation 205 object's :meth:`~NullTranslations.install` method. 206 207 As seen below, you usually mark the strings in your application that are 208 candidates for translation, by wrapping them in a call to the :func:`_` 209 function, like this:: 210 211 print(_('This string will be translated.')) 212 213 For convenience, you want the :func:`_` function to be installed in Python's 214 builtins namespace, so it is easily accessible in all modules of your 215 application. 216 217 218 The :class:`NullTranslations` class 219 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 220 221 Translation classes are what actually implement the translation of original 222 source file message strings to translated message strings. The base class used 223 by all translation classes is :class:`NullTranslations`; this provides the basic 224 interface you can use to write your own specialized translation classes. Here 225 are the methods of :class:`NullTranslations`: 226 227 228 .. class:: NullTranslations(fp=None) 229 230 Takes an optional :term:`file object` *fp*, which is ignored by the base class. 231 Initializes "protected" instance variables *_info* and *_charset* which are set 232 by derived classes, as well as *_fallback*, which is set through 233 :meth:`add_fallback`. It then calls ``self._parse(fp)`` if *fp* is not 234 ``None``. 235 236 .. method:: _parse(fp) 237 238 No-op'd in the base class, this method takes file object *fp*, and reads 239 the data from the file, initializing its message catalog. If you have an 240 unsupported message catalog file format, you should override this method 241 to parse your format. 242 243 244 .. method:: add_fallback(fallback) 245 246 Add *fallback* as the fallback object for the current translation object. 247 A translation object should consult the fallback if it cannot provide a 248 translation for a given message. 249 250 251 .. method:: gettext(message) 252 253 If a fallback has been set, forward :meth:`gettext` to the fallback. 254 Otherwise, return the translated message. Overridden in derived classes. 255 256 257 .. method:: lgettext(message) 258 259 If a fallback has been set, forward :meth:`lgettext` to the fallback. 260 Otherwise, return the translated message. Overridden in derived classes. 261 262 263 .. method:: ngettext(singular, plural, n) 264 265 If a fallback has been set, forward :meth:`ngettext` to the fallback. 266 Otherwise, return the translated message. Overridden in derived classes. 267 268 269 .. method:: lngettext(singular, plural, n) 270 271 If a fallback has been set, forward :meth:`lngettext` to the fallback. 272 Otherwise, return the translated message. Overridden in derived classes. 273 274 275 .. method:: info() 276 277 Return the "protected" :attr:`_info` variable. 278 279 280 .. method:: charset() 281 282 Return the "protected" :attr:`_charset` variable, which is the encoding of 283 the message catalog file. 284 285 286 .. method:: output_charset() 287 288 Return the "protected" :attr:`_output_charset` variable, which defines the 289 encoding used to return translated messages in :meth:`lgettext` and 290 :meth:`lngettext`. 291 292 293 .. method:: set_output_charset(charset) 294 295 Change the "protected" :attr:`_output_charset` variable, which defines the 296 encoding used to return translated messages. 297 298 299 .. method:: install(names=None) 300 301 This method installs :meth:`self.gettext` into the built-in namespace, 302 binding it to ``_``. 303 304 If the *names* parameter is given, it must be a sequence containing the 305 names of functions you want to install in the builtins namespace in 306 addition to :func:`_`. Supported names are ``'gettext'`` (bound to 307 :meth:`self.gettext`), ``'ngettext'`` (bound to :meth:`self.ngettext`), 308 ``'lgettext'`` and ``'lngettext'``. 309 310 Note that this is only one way, albeit the most convenient way, to make 311 the :func:`_` function available to your application. Because it affects 312 the entire application globally, and specifically the built-in namespace, 313 localized modules should never install :func:`_`. Instead, they should use 314 this code to make :func:`_` available to their module:: 315 316 import gettext 317 t = gettext.translation('mymodule', ...) 318 _ = t.gettext 319 320 This puts :func:`_` only in the module's global namespace and so only 321 affects calls within this module. 322 323 324 The :class:`GNUTranslations` class 325 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 326 327 The :mod:`gettext` module provides one additional class derived from 328 :class:`NullTranslations`: :class:`GNUTranslations`. This class overrides 329 :meth:`_parse` to enable reading GNU :program:`gettext` format :file:`.mo` files 330 in both big-endian and little-endian format. 331 332 :class:`GNUTranslations` parses optional meta-data out of the translation 333 catalog. It is convention with GNU :program:`gettext` to include meta-data as 334 the translation for the empty string. This meta-data is in :rfc:`822`\ -style 335 ``key: value`` pairs, and should contain the ``Project-Id-Version`` key. If the 336 key ``Content-Type`` is found, then the ``charset`` property is used to 337 initialize the "protected" :attr:`_charset` instance variable, defaulting to 338 ``None`` if not found. If the charset encoding is specified, then all message 339 ids and message strings read from the catalog are converted to Unicode using 340 this encoding, else ASCII encoding is assumed. 341 342 Since message ids are read as Unicode strings too, all :meth:`*gettext` methods 343 will assume message ids as Unicode strings, not byte strings. 344 345 The entire set of key/value pairs are placed into a dictionary and set as the 346 "protected" :attr:`_info` instance variable. 347 348 If the :file:`.mo` file's magic number is invalid, the major version number is 349 unexpected, or if other problems occur while reading the file, instantiating a 350 :class:`GNUTranslations` class can raise :exc:`OSError`. 351 352 The following methods are overridden from the base class implementation: 353 354 355 .. method:: GNUTranslations.gettext(message) 356 357 Look up the *message* id in the catalog and return the corresponding message 358 string, as a Unicode string. If there is no entry in the catalog for the 359 *message* id, and a fallback has been set, the look up is forwarded to the 360 fallback's :meth:`gettext` method. Otherwise, the *message* id is returned. 361 362 363 .. method:: GNUTranslations.lgettext(message) 364 365 Equivalent to :meth:`gettext`, but the translation is returned as a 366 bytestring encoded in the selected output charset, or in the preferred system 367 encoding if no encoding was explicitly set with :meth:`set_output_charset`. 368 369 370 .. method:: GNUTranslations.ngettext(singular, plural, n) 371 372 Do a plural-forms lookup of a message id. *singular* is used as the message id 373 for purposes of lookup in the catalog, while *n* is used to determine which 374 plural form to use. The returned message string is a Unicode string. 375 376 If the message id is not found in the catalog, and a fallback is specified, the 377 request is forwarded to the fallback's :meth:`ngettext` method. Otherwise, when 378 *n* is 1 *singular* is returned, and *plural* is returned in all other cases. 379 380 Here is an example:: 381 382 n = len(os.listdir('.')) 383 cat = GNUTranslations(somefile) 384 message = cat.ngettext( 385 'There is %(num)d file in this directory', 386 'There are %(num)d files in this directory', 387 n) % {'num': n} 388 389 390 .. method:: GNUTranslations.lngettext(singular, plural, n) 391 392 Equivalent to :meth:`gettext`, but the translation is returned as a 393 bytestring encoded in the selected output charset, or in the preferred system 394 encoding if no encoding was explicitly set with :meth:`set_output_charset`. 395 396 397 Solaris message catalog support 398 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 399 400 The Solaris operating system defines its own binary :file:`.mo` file format, but 401 since no documentation can be found on this format, it is not supported at this 402 time. 403 404 405 The Catalog constructor 406 ^^^^^^^^^^^^^^^^^^^^^^^ 407 408 .. index:: single: GNOME 409 410 GNOME uses a version of the :mod:`gettext` module by James Henstridge, but this 411 version has a slightly different API. Its documented usage was:: 412 413 import gettext 414 cat = gettext.Catalog(domain, localedir) 415 _ = cat.gettext 416 print(_('hello world')) 417 418 For compatibility with this older module, the function :func:`Catalog` is an 419 alias for the :func:`translation` function described above. 420 421 One difference between this module and Henstridge's: his catalog objects 422 supported access through a mapping API, but this appears to be unused and so is 423 not currently supported. 424 425 426 Internationalizing your programs and modules 427 -------------------------------------------- 428 429 Internationalization (I18N) refers to the operation by which a program is made 430 aware of multiple languages. Localization (L10N) refers to the adaptation of 431 your program, once internationalized, to the local language and cultural habits. 432 In order to provide multilingual messages for your Python programs, you need to 433 take the following steps: 434 435 #. prepare your program or module by specially marking translatable strings 436 437 #. run a suite of tools over your marked files to generate raw messages catalogs 438 439 #. create language specific translations of the message catalogs 440 441 #. use the :mod:`gettext` module so that message strings are properly translated 442 443 In order to prepare your code for I18N, you need to look at all the strings in 444 your files. Any string that needs to be translated should be marked by wrapping 445 it in ``_('...')`` --- that is, a call to the function :func:`_`. For example:: 446 447 filename = 'mylog.txt' 448 message = _('writing a log message') 449 fp = open(filename, 'w') 450 fp.write(message) 451 fp.close() 452 453 In this example, the string ``'writing a log message'`` is marked as a candidate 454 for translation, while the strings ``'mylog.txt'`` and ``'w'`` are not. 455 456 There are a few tools to extract the strings meant for translation. 457 The original GNU :program:`gettext` only supported C or C++ source 458 code but its extended version :program:`xgettext` scans code written 459 in a number of languages, including Python, to find strings marked as 460 translatable. `Babel <http://babel.pocoo.org/>`__ is a Python 461 internationalization library that includes a :file:`pybabel` script to 462 extract and compile message catalogs. Franois Pinard's program 463 called :program:`xpot` does a similar job and is available as part of 464 his `po-utils package <https://github.com/pinard/po-utils>`__. 465 466 (Python also includes pure-Python versions of these programs, called 467 :program:`pygettext.py` and :program:`msgfmt.py`; some Python distributions 468 will install them for you. :program:`pygettext.py` is similar to 469 :program:`xgettext`, but only understands Python source code and 470 cannot handle other programming languages such as C or C++. 471 :program:`pygettext.py` supports a command-line interface similar to 472 :program:`xgettext`; for details on its use, run ``pygettext.py 473 --help``. :program:`msgfmt.py` is binary compatible with GNU 474 :program:`msgfmt`. With these two programs, you may not need the GNU 475 :program:`gettext` package to internationalize your Python 476 applications.) 477 478 :program:`xgettext`, :program:`pygettext`, and similar tools generate 479 :file:`.po` files that are message catalogs. They are structured 480 human-readable files that contain every marked string in the source 481 code, along with a placeholder for the translated versions of these 482 strings. 483 484 Copies of these :file:`.po` files are then handed over to the 485 individual human translators who write translations for every 486 supported natural language. They send back the completed 487 language-specific versions as a :file:`<language-name>.po` file that's 488 compiled into a machine-readable :file:`.mo` binary catalog file using 489 the :program:`msgfmt` program. The :file:`.mo` files are used by the 490 :mod:`gettext` module for the actual translation processing at 491 run-time. 492 493 How you use the :mod:`gettext` module in your code depends on whether you are 494 internationalizing a single module or your entire application. The next two 495 sections will discuss each case. 496 497 498 Localizing your module 499 ^^^^^^^^^^^^^^^^^^^^^^ 500 501 If you are localizing your module, you must take care not to make global 502 changes, e.g. to the built-in namespace. You should not use the GNU ``gettext`` 503 API but instead the class-based API. 504 505 Let's say your module is called "spam" and the module's various natural language 506 translation :file:`.mo` files reside in :file:`/usr/share/locale` in GNU 507 :program:`gettext` format. Here's what you would put at the top of your 508 module:: 509 510 import gettext 511 t = gettext.translation('spam', '/usr/share/locale') 512 _ = t.lgettext 513 514 515 Localizing your application 516 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 517 518 If you are localizing your application, you can install the :func:`_` function 519 globally into the built-in namespace, usually in the main driver file of your 520 application. This will let all your application-specific files just use 521 ``_('...')`` without having to explicitly install it in each file. 522 523 In the simple case then, you need only add the following bit of code to the main 524 driver file of your application:: 525 526 import gettext 527 gettext.install('myapplication') 528 529 If you need to set the locale directory, you can pass it into the 530 :func:`install` function:: 531 532 import gettext 533 gettext.install('myapplication', '/usr/share/locale') 534 535 536 Changing languages on the fly 537 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 538 539 If your program needs to support many languages at the same time, you may want 540 to create multiple translation instances and then switch between them 541 explicitly, like so:: 542 543 import gettext 544 545 lang1 = gettext.translation('myapplication', languages=['en']) 546 lang2 = gettext.translation('myapplication', languages=['fr']) 547 lang3 = gettext.translation('myapplication', languages=['de']) 548 549 # start by using language1 550 lang1.install() 551 552 # ... time goes by, user selects language 2 553 lang2.install() 554 555 # ... more time goes by, user selects language 3 556 lang3.install() 557 558 559 Deferred translations 560 ^^^^^^^^^^^^^^^^^^^^^ 561 562 In most coding situations, strings are translated where they are coded. 563 Occasionally however, you need to mark strings for translation, but defer actual 564 translation until later. A classic example is:: 565 566 animals = ['mollusk', 567 'albatross', 568 'rat', 569 'penguin', 570 'python', ] 571 # ... 572 for a in animals: 573 print(a) 574 575 Here, you want to mark the strings in the ``animals`` list as being 576 translatable, but you don't actually want to translate them until they are 577 printed. 578 579 Here is one way you can handle this situation:: 580 581 def _(message): return message 582 583 animals = [_('mollusk'), 584 _('albatross'), 585 _('rat'), 586 _('penguin'), 587 _('python'), ] 588 589 del _ 590 591 # ... 592 for a in animals: 593 print(_(a)) 594 595 This works because the dummy definition of :func:`_` simply returns the string 596 unchanged. And this dummy definition will temporarily override any definition 597 of :func:`_` in the built-in namespace (until the :keyword:`del` command). Take 598 care, though if you have a previous definition of :func:`_` in the local 599 namespace. 600 601 Note that the second use of :func:`_` will not identify "a" as being 602 translatable to the :program:`gettext` program, because the parameter 603 is not a string literal. 604 605 Another way to handle this is with the following example:: 606 607 def N_(message): return message 608 609 animals = [N_('mollusk'), 610 N_('albatross'), 611 N_('rat'), 612 N_('penguin'), 613 N_('python'), ] 614 615 # ... 616 for a in animals: 617 print(_(a)) 618 619 In this case, you are marking translatable strings with the function 620 :func:`N_`, which won't conflict with any definition of :func:`_`. 621 However, you will need to teach your message extraction program to 622 look for translatable strings marked with :func:`N_`. :program:`xgettext`, 623 :program:`pygettext`, ``pybabel extract``, and :program:`xpot` all 624 support this through the use of the :option:`!-k` command-line switch. 625 The choice of :func:`N_` here is totally arbitrary; it could have just 626 as easily been :func:`MarkThisStringForTranslation`. 627 628 629 Acknowledgements 630 ---------------- 631 632 The following people contributed code, feedback, design suggestions, previous 633 implementations, and valuable experience to the creation of this module: 634 635 * Peter Funk 636 637 * James Henstridge 638 639 * Juan David Ibez Palomar 640 641 * Marc-Andr Lemburg 642 643 * Martin von Lwis 644 645 * Franois Pinard 646 647 * Barry Warsaw 648 649 * Gustavo Niemeyer 650 651 .. rubric:: Footnotes 652 653 .. [#] The default locale directory is system dependent; for example, on RedHat Linux 654 it is :file:`/usr/share/locale`, but on Solaris it is :file:`/usr/lib/locale`. 655 The :mod:`gettext` module does not try to support these system dependent 656 defaults; instead its default is :file:`sys.prefix/share/locale`. For this 657 reason, it is always best to call :func:`bindtextdomain` with an explicit 658 absolute path at the start of your application. 659 660 .. [#] See the footnote for :func:`bindtextdomain` above. 661