1 :mod:`zipapp` --- Manage executable Python zip archives 2 ======================================================= 3 4 .. module:: zipapp 5 :synopsis: Manage executable Python zip archives 6 7 .. versionadded:: 3.5 8 9 **Source code:** :source:`Lib/zipapp.py` 10 11 .. index:: 12 single: Executable Zip Files 13 14 -------------- 15 16 This module provides tools to manage the creation of zip files containing 17 Python code, which can be :ref:`executed directly by the Python interpreter 18 <using-on-interface-options>`. The module provides both a 19 :ref:`zipapp-command-line-interface` and a :ref:`zipapp-python-api`. 20 21 22 Basic Example 23 ------------- 24 25 The following example shows how the :ref:`zipapp-command-line-interface` 26 can be used to create an executable archive from a directory containing 27 Python code. When run, the archive will execute the ``main`` function from 28 the module ``myapp`` in the archive. 29 30 .. code-block:: shell-session 31 32 $ python -m zipapp myapp -m "myapp:main" 33 $ python myapp.pyz 34 <output from myapp> 35 36 37 .. _zipapp-command-line-interface: 38 39 Command-Line Interface 40 ---------------------- 41 42 When called as a program from the command line, the following form is used: 43 44 .. code-block:: shell-session 45 46 $ python -m zipapp source [options] 47 48 If *source* is a directory, this will create an archive from the contents of 49 *source*. If *source* is a file, it should be an archive, and it will be 50 copied to the target archive (or the contents of its shebang line will be 51 displayed if the --info option is specified). 52 53 The following options are understood: 54 55 .. program:: zipapp 56 57 .. cmdoption:: -o <output>, --output=<output> 58 59 Write the output to a file named *output*. If this option is not specified, 60 the output filename will be the same as the input *source*, with the 61 extension ``.pyz`` added. If an explicit filename is given, it is used as 62 is (so a ``.pyz`` extension should be included if required). 63 64 An output filename must be specified if the *source* is an archive (and in 65 that case, *output* must not be the same as *source*). 66 67 .. cmdoption:: -p <interpreter>, --python=<interpreter> 68 69 Add a ``#!`` line to the archive specifying *interpreter* as the command 70 to run. Also, on POSIX, make the archive executable. The default is to 71 write no ``#!`` line, and not make the file executable. 72 73 .. cmdoption:: -m <mainfn>, --main=<mainfn> 74 75 Write a ``__main__.py`` file to the archive that executes *mainfn*. The 76 *mainfn* argument should have the form "pkg.mod:fn", where "pkg.mod" is a 77 package/module in the archive, and "fn" is a callable in the given module. 78 The ``__main__.py`` file will execute that callable. 79 80 :option:`--main` cannot be specified when copying an archive. 81 82 .. cmdoption:: -c, --compress 83 84 Compress files with the deflate method, reducing the size of the output 85 file. By default, files are stored uncompressed in the archive. 86 87 :option:`--compress` has no effect when copying an archive. 88 89 .. versionadded:: 3.7 90 91 .. cmdoption:: --info 92 93 Display the interpreter embedded in the archive, for diagnostic purposes. In 94 this case, any other options are ignored and SOURCE must be an archive, not a 95 directory. 96 97 .. cmdoption:: -h, --help 98 99 Print a short usage message and exit. 100 101 102 .. _zipapp-python-api: 103 104 Python API 105 ---------- 106 107 The module defines two convenience functions: 108 109 110 .. function:: create_archive(source, target=None, interpreter=None, main=None, filter=None, compressed=False) 111 112 Create an application archive from *source*. The source can be any 113 of the following: 114 115 * The name of a directory, or a :term:`path-like object` referring 116 to a directory, in which case a new application archive will be 117 created from the content of that directory. 118 * The name of an existing application archive file, or a :term:`path-like object` 119 referring to such a file, in which case the file is copied to 120 the target (modifying it to reflect the value given for the *interpreter* 121 argument). The file name should include the ``.pyz`` extension, if required. 122 * A file object open for reading in bytes mode. The content of the 123 file should be an application archive, and the file object is 124 assumed to be positioned at the start of the archive. 125 126 The *target* argument determines where the resulting archive will be 127 written: 128 129 * If it is the name of a file, or a :term:`path-like object`, 130 the archive will be written to that file. 131 * If it is an open file object, the archive will be written to that 132 file object, which must be open for writing in bytes mode. 133 * If the target is omitted (or ``None``), the source must be a directory 134 and the target will be a file with the same name as the source, with 135 a ``.pyz`` extension added. 136 137 The *interpreter* argument specifies the name of the Python 138 interpreter with which the archive will be executed. It is written as 139 a "shebang" line at the start of the archive. On POSIX, this will be 140 interpreted by the OS, and on Windows it will be handled by the Python 141 launcher. Omitting the *interpreter* results in no shebang line being 142 written. If an interpreter is specified, and the target is a 143 filename, the executable bit of the target file will be set. 144 145 The *main* argument specifies the name of a callable which will be 146 used as the main program for the archive. It can only be specified if 147 the source is a directory, and the source does not already contain a 148 ``__main__.py`` file. The *main* argument should take the form 149 "pkg.module:callable" and the archive will be run by importing 150 "pkg.module" and executing the given callable with no arguments. It 151 is an error to omit *main* if the source is a directory and does not 152 contain a ``__main__.py`` file, as otherwise the resulting archive 153 would not be executable. 154 155 The optional *filter* argument specifies a callback function that 156 is passed a Path object representing the path to the file being added 157 (relative to the source directory). It should return ``True`` if the 158 file is to be added. 159 160 The optional *compressed* argument determines whether files are 161 compressed. If set to ``True``, files in the archive are compressed 162 with the deflate method; otherwise, files are stored uncompressed. 163 This argument has no effect when copying an existing archive. 164 165 If a file object is specified for *source* or *target*, it is the 166 caller's responsibility to close it after calling create_archive. 167 168 When copying an existing archive, file objects supplied only need 169 ``read`` and ``readline``, or ``write`` methods. When creating an 170 archive from a directory, if the target is a file object it will be 171 passed to the ``zipfile.ZipFile`` class, and must supply the methods 172 needed by that class. 173 174 .. versionadded:: 3.7 175 Added the *filter* and *compressed* arguments. 176 177 .. function:: get_interpreter(archive) 178 179 Return the interpreter specified in the ``#!`` line at the start of the 180 archive. If there is no ``#!`` line, return :const:`None`. 181 The *archive* argument can be a filename or a file-like object open 182 for reading in bytes mode. It is assumed to be at the start of the archive. 183 184 185 .. _zipapp-examples: 186 187 Examples 188 -------- 189 190 Pack up a directory into an archive, and run it. 191 192 .. code-block:: shell-session 193 194 $ python -m zipapp myapp 195 $ python myapp.pyz 196 <output from myapp> 197 198 The same can be done using the :func:`create_archive` function:: 199 200 >>> import zipapp 201 >>> zipapp.create_archive('myapp.pyz', 'myapp') 202 203 To make the application directly executable on POSIX, specify an interpreter 204 to use. 205 206 .. code-block:: shell-session 207 208 $ python -m zipapp myapp -p "/usr/bin/env python" 209 $ ./myapp.pyz 210 <output from myapp> 211 212 To replace the shebang line on an existing archive, create a modified archive 213 using the :func:`create_archive` function:: 214 215 >>> import zipapp 216 >>> zipapp.create_archive('old_archive.pyz', 'new_archive.pyz', '/usr/bin/python3') 217 218 To update the file in place, do the replacement in memory using a :class:`BytesIO` 219 object, and then overwrite the source afterwards. Note that there is a risk 220 when overwriting a file in place that an error will result in the loss of 221 the original file. This code does not protect against such errors, but 222 production code should do so. Also, this method will only work if the archive 223 fits in memory:: 224 225 >>> import zipapp 226 >>> import io 227 >>> temp = io.BytesIO() 228 >>> zipapp.create_archive('myapp.pyz', temp, '/usr/bin/python2') 229 >>> with open('myapp.pyz', 'wb') as f: 230 >>> f.write(temp.getvalue()) 231 232 233 .. _zipapp-specifying-the-interpreter: 234 235 Specifying the Interpreter 236 -------------------------- 237 238 Note that if you specify an interpreter and then distribute your application 239 archive, you need to ensure that the interpreter used is portable. The Python 240 launcher for Windows supports most common forms of POSIX ``#!`` line, but there 241 are other issues to consider: 242 243 * If you use "/usr/bin/env python" (or other forms of the "python" command, 244 such as "/usr/bin/python"), you need to consider that your users may have 245 either Python 2 or Python 3 as their default, and write your code to work 246 under both versions. 247 * If you use an explicit version, for example "/usr/bin/env python3" your 248 application will not work for users who do not have that version. (This 249 may be what you want if you have not made your code Python 2 compatible). 250 * There is no way to say "python X.Y or later", so be careful of using an 251 exact version like "/usr/bin/env python3.4" as you will need to change your 252 shebang line for users of Python 3.5, for example. 253 254 Typically, you should use an "/usr/bin/env python2" or "/usr/bin/env python3", 255 depending on whether your code is written for Python 2 or 3. 256 257 258 Creating Standalone Applications with zipapp 259 -------------------------------------------- 260 261 Using the :mod:`zipapp` module, it is possible to create self-contained Python 262 programs, which can be distributed to end users who only need to have a 263 suitable version of Python installed on their system. The key to doing this 264 is to bundle all of the application's dependencies into the archive, along 265 with the application code. 266 267 The steps to create a standalone archive are as follows: 268 269 1. Create your application in a directory as normal, so you have a ``myapp`` 270 directory containing a ``__main__.py`` file, and any supporting application 271 code. 272 273 2. Install all of your application's dependencies into the ``myapp`` directory, 274 using pip: 275 276 .. code-block:: shell-session 277 278 $ python -m pip install -r requirements.txt --target myapp 279 280 (this assumes you have your project requirements in a ``requirements.txt`` 281 file - if not, you can just list the dependencies manually on the pip command 282 line). 283 284 3. Optionally, delete the ``.dist-info`` directories created by pip in the 285 ``myapp`` directory. These hold metadata for pip to manage the packages, and 286 as you won't be making any further use of pip they aren't required - 287 although it won't do any harm if you leave them. 288 289 4. Package the application using: 290 291 .. code-block:: shell-session 292 293 $ python -m zipapp -p "interpreter" myapp 294 295 This will produce a standalone executable, which can be run on any machine with 296 the appropriate interpreter available. See :ref:`zipapp-specifying-the-interpreter` 297 for details. It can be shipped to users as a single file. 298 299 On Unix, the ``myapp.pyz`` file is executable as it stands. You can rename the 300 file to remove the ``.pyz`` extension if you prefer a "plain" command name. On 301 Windows, the ``myapp.pyz[w]`` file is executable by virtue of the fact that 302 the Python interpreter registers the ``.pyz`` and ``.pyzw`` file extensions 303 when installed. 304 305 306 Making a Windows executable 307 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 308 309 On Windows, registration of the ``.pyz`` extension is optional, and 310 furthermore, there are certain places that don't recognise registered 311 extensions "transparently" (the simplest example is that 312 ``subprocess.run(['myapp'])`` won't find your application - you need to 313 explicitly specify the extension). 314 315 On Windows, therefore, it is often preferable to create an executable from the 316 zipapp. This is relatively easy, although it does require a C compiler. The 317 basic approach relies on the fact that zipfiles can have arbitrary data 318 prepended, and Windows exe files can have arbitrary data appended. So by 319 creating a suitable launcher and tacking the ``.pyz`` file onto the end of it, 320 you end up with a single-file executable that runs your application. 321 322 A suitable launcher can be as simple as the following:: 323 324 #define Py_LIMITED_API 1 325 #include "Python.h" 326 327 #define WIN32_LEAN_AND_MEAN 328 #include <windows.h> 329 330 #ifdef WINDOWS 331 int WINAPI wWinMain( 332 HINSTANCE hInstance, /* handle to current instance */ 333 HINSTANCE hPrevInstance, /* handle to previous instance */ 334 LPWSTR lpCmdLine, /* pointer to command line */ 335 int nCmdShow /* show state of window */ 336 ) 337 #else 338 int wmain() 339 #endif 340 { 341 wchar_t **myargv = _alloca((__argc + 1) * sizeof(wchar_t*)); 342 myargv[0] = __wargv[0]; 343 memcpy(myargv + 1, __wargv, __argc * sizeof(wchar_t *)); 344 return Py_Main(__argc+1, myargv); 345 } 346 347 If you define the ``WINDOWS`` preprocessor symbol, this will generate a 348 GUI executable, and without it, a console executable. 349 350 To compile the executable, you can either just use the standard MSVC 351 command line tools, or you can take advantage of the fact that distutils 352 knows how to compile Python source:: 353 354 >>> from distutils.ccompiler import new_compiler 355 >>> import distutils.sysconfig 356 >>> import sys 357 >>> import os 358 >>> from pathlib import Path 359 360 >>> def compile(src): 361 >>> src = Path(src) 362 >>> cc = new_compiler() 363 >>> exe = src.stem 364 >>> cc.add_include_dir(distutils.sysconfig.get_python_inc()) 365 >>> cc.add_library_dir(os.path.join(sys.base_exec_prefix, 'libs')) 366 >>> # First the CLI executable 367 >>> objs = cc.compile([str(src)]) 368 >>> cc.link_executable(objs, exe) 369 >>> # Now the GUI executable 370 >>> cc.define_macro('WINDOWS') 371 >>> objs = cc.compile([str(src)]) 372 >>> cc.link_executable(objs, exe + 'w') 373 374 >>> if __name__ == "__main__": 375 >>> compile("zastub.c") 376 377 The resulting launcher uses the "Limited ABI", so it will run unchanged with 378 any version of Python 3.x. All it needs is for Python (``python3.dll``) to be 379 on the user's ``PATH``. 380 381 For a fully standalone distribution, you can distribute the launcher with your 382 application appended, bundled with the Python "embedded" distribution. This 383 will run on any PC with the appropriate architecture (32 bit or 64 bit). 384 385 386 Caveats 387 ~~~~~~~ 388 389 There are some limitations to the process of bundling your application into 390 a single file. In most, if not all, cases they can be addressed without 391 needing major changes to your application. 392 393 1. If your application depends on a package that includes a C extension, that 394 package cannot be run from a zip file (this is an OS limitation, as executable 395 code must be present in the filesystem for the OS loader to load it). In this 396 case, you can exclude that dependency from the zipfile, and either require 397 your users to have it installed, or ship it alongside your zipfile and add code 398 to your ``__main__.py`` to include the directory containing the unzipped 399 module in ``sys.path``. In this case, you will need to make sure to ship 400 appropriate binaries for your target architecture(s) (and potentially pick the 401 correct version to add to ``sys.path`` at runtime, based on the user's machine). 402 403 2. If you are shipping a Windows executable as described above, you either need to 404 ensure that your users have ``python3.dll`` on their PATH (which is not the 405 default behaviour of the installer) or you should bundle your application with 406 the embedded distribution. 407 408 3. The suggested launcher above uses the Python embedding API. This means that in 409 your application, ``sys.executable`` will be your application, and *not* a 410 conventional Python interpreter. Your code and its dependencies need to be 411 prepared for this possibility. For example, if your application uses the 412 :mod:`multiprocessing` module, it will need to call 413 :func:`multiprocessing.set_executable` to let the module know where to find the 414 standard Python interpreter. 415 416 417 The Python Zip Application Archive Format 418 ----------------------------------------- 419 420 Python has been able to execute zip files which contain a ``__main__.py`` file 421 since version 2.6. In order to be executed by Python, an application archive 422 simply has to be a standard zip file containing a ``__main__.py`` file which 423 will be run as the entry point for the application. As usual for any Python 424 script, the parent of the script (in this case the zip file) will be placed on 425 :data:`sys.path` and thus further modules can be imported from the zip file. 426 427 The zip file format allows arbitrary data to be prepended to a zip file. The 428 zip application format uses this ability to prepend a standard POSIX "shebang" 429 line to the file (``#!/path/to/interpreter``). 430 431 Formally, the Python zip application format is therefore: 432 433 1. An optional shebang line, containing the characters ``b'#!'`` followed by an 434 interpreter name, and then a newline (``b'\n'``) character. The interpreter 435 name can be anything acceptable to the OS "shebang" processing, or the Python 436 launcher on Windows. The interpreter should be encoded in UTF-8 on Windows, 437 and in :func:`sys.getfilesystemencoding()` on POSIX. 438 2. Standard zipfile data, as generated by the :mod:`zipfile` module. The 439 zipfile content *must* include a file called ``__main__.py`` (which must be 440 in the "root" of the zipfile - i.e., it cannot be in a subdirectory). The 441 zipfile data can be compressed or uncompressed. 442 443 If an application archive has a shebang line, it may have the executable bit set 444 on POSIX systems, to allow it to be executed directly. 445 446 There is no requirement that the tools in this module are used to create 447 application archives - the module is a convenience, but archives in the above 448 format created by any means are acceptable to Python. 449 450
