1 :mod:`subprocess` --- Subprocess management 2 =========================================== 3 4 .. module:: subprocess 5 :synopsis: Subprocess management. 6 7 .. moduleauthor:: Peter strand <astrand (a] lysator.liu.se> 8 .. sectionauthor:: Peter strand <astrand (a] lysator.liu.se> 9 10 **Source code:** :source:`Lib/subprocess.py` 11 12 -------------- 13 14 The :mod:`subprocess` module allows you to spawn new processes, connect to their 15 input/output/error pipes, and obtain their return codes. This module intends to 16 replace several older modules and functions:: 17 18 os.system 19 os.spawn* 20 21 Information about how the :mod:`subprocess` module can be used to replace these 22 modules and functions can be found in the following sections. 23 24 .. seealso:: 25 26 :pep:`324` -- PEP proposing the subprocess module 27 28 29 Using the :mod:`subprocess` Module 30 ---------------------------------- 31 32 The recommended approach to invoking subprocesses is to use the :func:`run` 33 function for all use cases it can handle. For more advanced use cases, the 34 underlying :class:`Popen` interface can be used directly. 35 36 The :func:`run` function was added in Python 3.5; if you need to retain 37 compatibility with older versions, see the :ref:`call-function-trio` section. 38 39 40 .. function:: run(args, *, stdin=None, input=None, stdout=None, stderr=None,\ 41 shell=False, timeout=None, check=False, \ 42 encoding=None, errors=None) 43 44 Run the command described by *args*. Wait for command to complete, then 45 return a :class:`CompletedProcess` instance. 46 47 The arguments shown above are merely the most common ones, described below 48 in :ref:`frequently-used-arguments` (hence the use of keyword-only notation 49 in the abbreviated signature). The full function signature is largely the 50 same as that of the :class:`Popen` constructor - apart from *timeout*, 51 *input* and *check*, all the arguments to this function are passed through to 52 that interface. 53 54 This does not capture stdout or stderr by default. To do so, pass 55 :data:`PIPE` for the *stdout* and/or *stderr* arguments. 56 57 The *timeout* argument is passed to :meth:`Popen.communicate`. If the timeout 58 expires, the child process will be killed and waited for. The 59 :exc:`TimeoutExpired` exception will be re-raised after the child process 60 has terminated. 61 62 The *input* argument is passed to :meth:`Popen.communicate` and thus to the 63 subprocess's stdin. If used it must be a byte sequence, or a string if 64 *encoding* or *errors* is specified or *universal_newlines* is true. When 65 used, the internal :class:`Popen` object is automatically created with 66 ``stdin=PIPE``, and the *stdin* argument may not be used as well. 67 68 If *check* is true, and the process exits with a non-zero exit code, a 69 :exc:`CalledProcessError` exception will be raised. Attributes of that 70 exception hold the arguments, the exit code, and stdout and stderr if they 71 were captured. 72 73 If *encoding* or *errors* are specified, or *universal_newlines* is true, 74 file objects for stdin, stdout and stderr are opened in text mode using the 75 specified *encoding* and *errors* or the :class:`io.TextIOWrapper` default. 76 Otherwise, file objects are opened in binary mode. 77 78 Examples:: 79 80 >>> subprocess.run(["ls", "-l"]) # doesn't capture output 81 CompletedProcess(args=['ls', '-l'], returncode=0) 82 83 >>> subprocess.run("exit 1", shell=True, check=True) 84 Traceback (most recent call last): 85 ... 86 subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1 87 88 >>> subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE) 89 CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0, 90 stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n') 91 92 .. versionadded:: 3.5 93 94 .. versionchanged:: 3.6 95 96 Added *encoding* and *errors* parameters 97 98 .. class:: CompletedProcess 99 100 The return value from :func:`run`, representing a process that has finished. 101 102 .. attribute:: args 103 104 The arguments used to launch the process. This may be a list or a string. 105 106 .. attribute:: returncode 107 108 Exit status of the child process. Typically, an exit status of 0 indicates 109 that it ran successfully. 110 111 A negative value ``-N`` indicates that the child was terminated by signal 112 ``N`` (POSIX only). 113 114 .. attribute:: stdout 115 116 Captured stdout from the child process. A bytes sequence, or a string if 117 :func:`run` was called with an encoding or errors. ``None`` if stdout was not 118 captured. 119 120 If you ran the process with ``stderr=subprocess.STDOUT``, stdout and 121 stderr will be combined in this attribute, and :attr:`stderr` will be 122 ``None``. 123 124 .. attribute:: stderr 125 126 Captured stderr from the child process. A bytes sequence, or a string if 127 :func:`run` was called with an encoding or errors. ``None`` if stderr was not 128 captured. 129 130 .. method:: check_returncode() 131 132 If :attr:`returncode` is non-zero, raise a :exc:`CalledProcessError`. 133 134 .. versionadded:: 3.5 135 136 .. data:: DEVNULL 137 138 Special value that can be used as the *stdin*, *stdout* or *stderr* argument 139 to :class:`Popen` and indicates that the special file :data:`os.devnull` 140 will be used. 141 142 .. versionadded:: 3.3 143 144 145 .. data:: PIPE 146 147 Special value that can be used as the *stdin*, *stdout* or *stderr* argument 148 to :class:`Popen` and indicates that a pipe to the standard stream should be 149 opened. Most useful with :meth:`Popen.communicate`. 150 151 152 .. data:: STDOUT 153 154 Special value that can be used as the *stderr* argument to :class:`Popen` and 155 indicates that standard error should go into the same handle as standard 156 output. 157 158 159 .. exception:: SubprocessError 160 161 Base class for all other exceptions from this module. 162 163 .. versionadded:: 3.3 164 165 166 .. exception:: TimeoutExpired 167 168 Subclass of :exc:`SubprocessError`, raised when a timeout expires 169 while waiting for a child process. 170 171 .. attribute:: cmd 172 173 Command that was used to spawn the child process. 174 175 .. attribute:: timeout 176 177 Timeout in seconds. 178 179 .. attribute:: output 180 181 Output of the child process if it was captured by :func:`run` or 182 :func:`check_output`. Otherwise, ``None``. 183 184 .. attribute:: stdout 185 186 Alias for output, for symmetry with :attr:`stderr`. 187 188 .. attribute:: stderr 189 190 Stderr output of the child process if it was captured by :func:`run`. 191 Otherwise, ``None``. 192 193 .. versionadded:: 3.3 194 195 .. versionchanged:: 3.5 196 *stdout* and *stderr* attributes added 197 198 .. exception:: CalledProcessError 199 200 Subclass of :exc:`SubprocessError`, raised when a process run by 201 :func:`check_call` or :func:`check_output` returns a non-zero exit status. 202 203 .. attribute:: returncode 204 205 Exit status of the child process. If the process exited due to a 206 signal, this will be the negative signal number. 207 208 .. attribute:: cmd 209 210 Command that was used to spawn the child process. 211 212 .. attribute:: output 213 214 Output of the child process if it was captured by :func:`run` or 215 :func:`check_output`. Otherwise, ``None``. 216 217 .. attribute:: stdout 218 219 Alias for output, for symmetry with :attr:`stderr`. 220 221 .. attribute:: stderr 222 223 Stderr output of the child process if it was captured by :func:`run`. 224 Otherwise, ``None``. 225 226 .. versionchanged:: 3.5 227 *stdout* and *stderr* attributes added 228 229 230 .. _frequently-used-arguments: 231 232 Frequently Used Arguments 233 ^^^^^^^^^^^^^^^^^^^^^^^^^ 234 235 To support a wide variety of use cases, the :class:`Popen` constructor (and 236 the convenience functions) accept a large number of optional arguments. For 237 most typical use cases, many of these arguments can be safely left at their 238 default values. The arguments that are most commonly needed are: 239 240 *args* is required for all calls and should be a string, or a sequence of 241 program arguments. Providing a sequence of arguments is generally 242 preferred, as it allows the module to take care of any required escaping 243 and quoting of arguments (e.g. to permit spaces in file names). If passing 244 a single string, either *shell* must be :const:`True` (see below) or else 245 the string must simply name the program to be executed without specifying 246 any arguments. 247 248 *stdin*, *stdout* and *stderr* specify the executed program's standard input, 249 standard output and standard error file handles, respectively. Valid values 250 are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive 251 integer), an existing file object, and ``None``. :data:`PIPE` indicates 252 that a new pipe to the child should be created. :data:`DEVNULL` indicates 253 that the special file :data:`os.devnull` will be used. With the default 254 settings of ``None``, no redirection will occur; the child's file handles 255 will be inherited from the parent. Additionally, *stderr* can be 256 :data:`STDOUT`, which indicates that the stderr data from the child 257 process should be captured into the same file handle as for *stdout*. 258 259 .. index:: 260 single: universal newlines; subprocess module 261 262 If *encoding* or *errors* are specified, or *universal_newlines* is true, 263 the file objects *stdin*, *stdout* and *stderr* will be opened in text 264 mode using the *encoding* and *errors* specified in the call or the 265 defaults for :class:`io.TextIOWrapper`. 266 267 For *stdin*, line ending characters ``'\n'`` in the input will be converted 268 to the default line separator :data:`os.linesep`. For *stdout* and *stderr*, 269 all line endings in the output will be converted to ``'\n'``. For more 270 information see the documentation of the :class:`io.TextIOWrapper` class 271 when the *newline* argument to its constructor is ``None``. 272 273 If text mode is not used, *stdin*, *stdout* and *stderr* will be opened as 274 binary streams. No encoding or line ending conversion is performed. 275 276 .. versionadded:: 3.6 277 Added *encoding* and *errors* parameters. 278 279 .. note:: 280 281 The newlines attribute of the file objects :attr:`Popen.stdin`, 282 :attr:`Popen.stdout` and :attr:`Popen.stderr` are not updated by 283 the :meth:`Popen.communicate` method. 284 285 If *shell* is ``True``, the specified command will be executed through 286 the shell. This can be useful if you are using Python primarily for the 287 enhanced control flow it offers over most system shells and still want 288 convenient access to other shell features such as shell pipes, filename 289 wildcards, environment variable expansion, and expansion of ``~`` to a 290 user's home directory. However, note that Python itself offers 291 implementations of many shell-like features (in particular, :mod:`glob`, 292 :mod:`fnmatch`, :func:`os.walk`, :func:`os.path.expandvars`, 293 :func:`os.path.expanduser`, and :mod:`shutil`). 294 295 .. versionchanged:: 3.3 296 When *universal_newlines* is ``True``, the class uses the encoding 297 :func:`locale.getpreferredencoding(False) <locale.getpreferredencoding>` 298 instead of ``locale.getpreferredencoding()``. See the 299 :class:`io.TextIOWrapper` class for more information on this change. 300 301 .. note:: 302 303 Read the `Security Considerations`_ section before using ``shell=True``. 304 305 These options, along with all of the other options, are described in more 306 detail in the :class:`Popen` constructor documentation. 307 308 309 Popen Constructor 310 ^^^^^^^^^^^^^^^^^ 311 312 The underlying process creation and management in this module is handled by 313 the :class:`Popen` class. It offers a lot of flexibility so that developers 314 are able to handle the less common cases not covered by the convenience 315 functions. 316 317 318 .. class:: Popen(args, bufsize=-1, executable=None, stdin=None, stdout=None, \ 319 stderr=None, preexec_fn=None, close_fds=True, shell=False, \ 320 cwd=None, env=None, universal_newlines=False, \ 321 startupinfo=None, creationflags=0, restore_signals=True, \ 322 start_new_session=False, pass_fds=(), *, \ 323 encoding=None, errors=None) 324 325 Execute a child program in a new process. On POSIX, the class uses 326 :meth:`os.execvp`-like behavior to execute the child program. On Windows, 327 the class uses the Windows ``CreateProcess()`` function. The arguments to 328 :class:`Popen` are as follows. 329 330 *args* should be a sequence of program arguments or else a single string. 331 By default, the program to execute is the first item in *args* if *args* is 332 a sequence. If *args* is a string, the interpretation is 333 platform-dependent and described below. See the *shell* and *executable* 334 arguments for additional differences from the default behavior. Unless 335 otherwise stated, it is recommended to pass *args* as a sequence. 336 337 On POSIX, if *args* is a string, the string is interpreted as the name or 338 path of the program to execute. However, this can only be done if not 339 passing arguments to the program. 340 341 .. note:: 342 343 :meth:`shlex.split` can be useful when determining the correct 344 tokenization for *args*, especially in complex cases:: 345 346 >>> import shlex, subprocess 347 >>> command_line = input() 348 /bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'" 349 >>> args = shlex.split(command_line) 350 >>> print(args) 351 ['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"] 352 >>> p = subprocess.Popen(args) # Success! 353 354 Note in particular that options (such as *-input*) and arguments (such 355 as *eggs.txt*) that are separated by whitespace in the shell go in separate 356 list elements, while arguments that need quoting or backslash escaping when 357 used in the shell (such as filenames containing spaces or the *echo* command 358 shown above) are single list elements. 359 360 On Windows, if *args* is a sequence, it will be converted to a string in a 361 manner described in :ref:`converting-argument-sequence`. This is because 362 the underlying ``CreateProcess()`` operates on strings. 363 364 The *shell* argument (which defaults to ``False``) specifies whether to use 365 the shell as the program to execute. If *shell* is ``True``, it is 366 recommended to pass *args* as a string rather than as a sequence. 367 368 On POSIX with ``shell=True``, the shell defaults to :file:`/bin/sh`. If 369 *args* is a string, the string specifies the command 370 to execute through the shell. This means that the string must be 371 formatted exactly as it would be when typed at the shell prompt. This 372 includes, for example, quoting or backslash escaping filenames with spaces in 373 them. If *args* is a sequence, the first item specifies the command string, and 374 any additional items will be treated as additional arguments to the shell 375 itself. That is to say, :class:`Popen` does the equivalent of:: 376 377 Popen(['/bin/sh', '-c', args[0], args[1], ...]) 378 379 On Windows with ``shell=True``, the :envvar:`COMSPEC` environment variable 380 specifies the default shell. The only time you need to specify 381 ``shell=True`` on Windows is when the command you wish to execute is built 382 into the shell (e.g. :command:`dir` or :command:`copy`). You do not need 383 ``shell=True`` to run a batch file or console-based executable. 384 385 .. note:: 386 387 Read the `Security Considerations`_ section before using ``shell=True``. 388 389 *bufsize* will be supplied as the corresponding argument to the 390 :func:`open` function when creating the stdin/stdout/stderr pipe 391 file objects: 392 393 - :const:`0` means unbuffered (read and write are one 394 system call and can return short) 395 - :const:`1` means line buffered 396 (only usable if ``universal_newlines=True`` i.e., in a text mode) 397 - any other positive value means use a buffer of approximately that 398 size 399 - negative bufsize (the default) means the system default of 400 io.DEFAULT_BUFFER_SIZE will be used. 401 402 .. versionchanged:: 3.3.1 403 *bufsize* now defaults to -1 to enable buffering by default to match the 404 behavior that most code expects. In versions prior to Python 3.2.4 and 405 3.3.1 it incorrectly defaulted to :const:`0` which was unbuffered 406 and allowed short reads. This was unintentional and did not match the 407 behavior of Python 2 as most code expected. 408 409 The *executable* argument specifies a replacement program to execute. It 410 is very seldom needed. When ``shell=False``, *executable* replaces the 411 program to execute specified by *args*. However, the original *args* is 412 still passed to the program. Most programs treat the program specified 413 by *args* as the command name, which can then be different from the program 414 actually executed. On POSIX, the *args* name 415 becomes the display name for the executable in utilities such as 416 :program:`ps`. If ``shell=True``, on POSIX the *executable* argument 417 specifies a replacement shell for the default :file:`/bin/sh`. 418 419 *stdin*, *stdout* and *stderr* specify the executed program's standard input, 420 standard output and standard error file handles, respectively. Valid values 421 are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive 422 integer), an existing :term:`file object`, and ``None``. :data:`PIPE` 423 indicates that a new pipe to the child should be created. :data:`DEVNULL` 424 indicates that the special file :data:`os.devnull` will be used. With the 425 default settings of ``None``, no redirection will occur; the child's file 426 handles will be inherited from the parent. Additionally, *stderr* can be 427 :data:`STDOUT`, which indicates that the stderr data from the applications 428 should be captured into the same file handle as for stdout. 429 430 If *preexec_fn* is set to a callable object, this object will be called in the 431 child process just before the child is executed. 432 (POSIX only) 433 434 .. warning:: 435 436 The *preexec_fn* parameter is not safe to use in the presence of threads 437 in your application. The child process could deadlock before exec is 438 called. 439 If you must use it, keep it trivial! Minimize the number of libraries 440 you call into. 441 442 .. note:: 443 444 If you need to modify the environment for the child use the *env* 445 parameter rather than doing it in a *preexec_fn*. 446 The *start_new_session* parameter can take the place of a previously 447 common use of *preexec_fn* to call os.setsid() in the child. 448 449 If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and 450 :const:`2` will be closed before the child process is executed. (POSIX only). 451 The default varies by platform: Always true on POSIX. On Windows it is 452 true when *stdin*/*stdout*/*stderr* are :const:`None`, false otherwise. 453 On Windows, if *close_fds* is true then no handles will be inherited by the 454 child process. Note that on Windows, you cannot set *close_fds* to true and 455 also redirect the standard handles by setting *stdin*, *stdout* or *stderr*. 456 457 .. versionchanged:: 3.2 458 The default for *close_fds* was changed from :const:`False` to 459 what is described above. 460 461 *pass_fds* is an optional sequence of file descriptors to keep open 462 between the parent and child. Providing any *pass_fds* forces 463 *close_fds* to be :const:`True`. (POSIX only) 464 465 .. versionadded:: 3.2 466 The *pass_fds* parameter was added. 467 468 If *cwd* is not ``None``, the function changes the working directory to 469 *cwd* before executing the child. *cwd* can be a :class:`str` and 470 :term:`path-like <path-like object>` object. In particular, the function 471 looks for *executable* (or for the first item in *args*) relative to *cwd* 472 if the executable path is a relative path. 473 474 .. versionchanged:: 3.6 475 *cwd* parameter accepts a :term:`path-like object`. 476 477 If *restore_signals* is true (the default) all signals that Python has set to 478 SIG_IGN are restored to SIG_DFL in the child process before the exec. 479 Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. 480 (POSIX only) 481 482 .. versionchanged:: 3.2 483 *restore_signals* was added. 484 485 If *start_new_session* is true the setsid() system call will be made in the 486 child process prior to the execution of the subprocess. (POSIX only) 487 488 .. versionchanged:: 3.2 489 *start_new_session* was added. 490 491 If *env* is not ``None``, it must be a mapping that defines the environment 492 variables for the new process; these are used instead of the default 493 behavior of inheriting the current process' environment. 494 495 .. note:: 496 497 If specified, *env* must provide any variables required for the program to 498 execute. On Windows, in order to run a `side-by-side assembly`_ the 499 specified *env* **must** include a valid :envvar:`SystemRoot`. 500 501 .. _side-by-side assembly: https://en.wikipedia.org/wiki/Side-by-Side_Assembly 502 503 If *encoding* or *errors* are specified, the file objects *stdin*, *stdout* 504 and *stderr* are opened in text mode with the specified encoding and 505 *errors*, as described above in :ref:`frequently-used-arguments`. If 506 *universal_newlines* is ``True``, they are opened in text mode with default 507 encoding. Otherwise, they are opened as binary streams. 508 509 .. versionadded:: 3.6 510 *encoding* and *errors* were added. 511 512 If given, *startupinfo* will be a :class:`STARTUPINFO` object, which is 513 passed to the underlying ``CreateProcess`` function. 514 *creationflags*, if given, can be :data:`CREATE_NEW_CONSOLE` or 515 :data:`CREATE_NEW_PROCESS_GROUP`. (Windows only) 516 517 Popen objects are supported as context managers via the :keyword:`with` statement: 518 on exit, standard file descriptors are closed, and the process is waited for. 519 :: 520 521 with Popen(["ifconfig"], stdout=PIPE) as proc: 522 log.write(proc.stdout.read()) 523 524 .. versionchanged:: 3.2 525 Added context manager support. 526 527 .. versionchanged:: 3.6 528 Popen destructor now emits a :exc:`ResourceWarning` warning if the child 529 process is still running. 530 531 532 Exceptions 533 ^^^^^^^^^^ 534 535 Exceptions raised in the child process, before the new program has started to 536 execute, will be re-raised in the parent. Additionally, the exception object 537 will have one extra attribute called :attr:`child_traceback`, which is a string 538 containing traceback information from the child's point of view. 539 540 The most common exception raised is :exc:`OSError`. This occurs, for example, 541 when trying to execute a non-existent file. Applications should prepare for 542 :exc:`OSError` exceptions. 543 544 A :exc:`ValueError` will be raised if :class:`Popen` is called with invalid 545 arguments. 546 547 :func:`check_call` and :func:`check_output` will raise 548 :exc:`CalledProcessError` if the called process returns a non-zero return 549 code. 550 551 All of the functions and methods that accept a *timeout* parameter, such as 552 :func:`call` and :meth:`Popen.communicate` will raise :exc:`TimeoutExpired` if 553 the timeout expires before the process exits. 554 555 Exceptions defined in this module all inherit from :exc:`SubprocessError`. 556 557 .. versionadded:: 3.3 558 The :exc:`SubprocessError` base class was added. 559 560 561 Security Considerations 562 ----------------------- 563 564 Unlike some other popen functions, this implementation will never 565 implicitly call a system shell. This means that all characters, 566 including shell metacharacters, can safely be passed to child processes. 567 If the shell is invoked explicitly, via ``shell=True``, it is the application's 568 responsibility to ensure that all whitespace and metacharacters are 569 quoted appropriately to avoid 570 `shell injection <https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ 571 vulnerabilities. 572 573 When using ``shell=True``, the :func:`shlex.quote` function can be 574 used to properly escape whitespace and shell metacharacters in strings 575 that are going to be used to construct shell commands. 576 577 578 Popen Objects 579 ------------- 580 581 Instances of the :class:`Popen` class have the following methods: 582 583 584 .. method:: Popen.poll() 585 586 Check if child process has terminated. Set and return 587 :attr:`~Popen.returncode` attribute. 588 589 590 .. method:: Popen.wait(timeout=None) 591 592 Wait for child process to terminate. Set and return 593 :attr:`~Popen.returncode` attribute. 594 595 If the process does not terminate after *timeout* seconds, raise a 596 :exc:`TimeoutExpired` exception. It is safe to catch this exception and 597 retry the wait. 598 599 .. note:: 600 601 This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE`` 602 and the child process generates enough output to a pipe such that 603 it blocks waiting for the OS pipe buffer to accept more data. 604 Use :meth:`Popen.communicate` when using pipes to avoid that. 605 606 .. note:: 607 608 The function is implemented using a busy loop (non-blocking call and 609 short sleeps). Use the :mod:`asyncio` module for an asynchronous wait: 610 see :class:`asyncio.create_subprocess_exec`. 611 612 .. versionchanged:: 3.3 613 *timeout* was added. 614 615 .. deprecated:: 3.4 616 617 Do not use the *endtime* parameter. It is was unintentionally 618 exposed in 3.3 but was left undocumented as it was intended to be 619 private for internal use. Use *timeout* instead. 620 621 .. method:: Popen.communicate(input=None, timeout=None) 622 623 Interact with process: Send data to stdin. Read data from stdout and stderr, 624 until end-of-file is reached. Wait for process to terminate. The optional 625 *input* argument should be data to be sent to the child process, or 626 ``None``, if no data should be sent to the child. If streams were opened in 627 text mode, *input* must be a string. Otherwise, it must be bytes. 628 629 :meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``. 630 The data will be strings if streams were opened in text mode; otherwise, 631 bytes. 632 633 Note that if you want to send data to the process's stdin, you need to create 634 the Popen object with ``stdin=PIPE``. Similarly, to get anything other than 635 ``None`` in the result tuple, you need to give ``stdout=PIPE`` and/or 636 ``stderr=PIPE`` too. 637 638 If the process does not terminate after *timeout* seconds, a 639 :exc:`TimeoutExpired` exception will be raised. Catching this exception and 640 retrying communication will not lose any output. 641 642 The child process is not killed if the timeout expires, so in order to 643 cleanup properly a well-behaved application should kill the child process and 644 finish communication:: 645 646 proc = subprocess.Popen(...) 647 try: 648 outs, errs = proc.communicate(timeout=15) 649 except TimeoutExpired: 650 proc.kill() 651 outs, errs = proc.communicate() 652 653 .. note:: 654 655 The data read is buffered in memory, so do not use this method if the data 656 size is large or unlimited. 657 658 .. versionchanged:: 3.3 659 *timeout* was added. 660 661 662 .. method:: Popen.send_signal(signal) 663 664 Sends the signal *signal* to the child. 665 666 .. note:: 667 668 On Windows, SIGTERM is an alias for :meth:`terminate`. CTRL_C_EVENT and 669 CTRL_BREAK_EVENT can be sent to processes started with a *creationflags* 670 parameter which includes `CREATE_NEW_PROCESS_GROUP`. 671 672 673 .. method:: Popen.terminate() 674 675 Stop the child. On Posix OSs the method sends SIGTERM to the 676 child. On Windows the Win32 API function :c:func:`TerminateProcess` is called 677 to stop the child. 678 679 680 .. method:: Popen.kill() 681 682 Kills the child. On Posix OSs the function sends SIGKILL to the child. 683 On Windows :meth:`kill` is an alias for :meth:`terminate`. 684 685 686 The following attributes are also available: 687 688 .. attribute:: Popen.args 689 690 The *args* argument as it was passed to :class:`Popen` -- a 691 sequence of program arguments or else a single string. 692 693 .. versionadded:: 3.3 694 695 .. attribute:: Popen.stdin 696 697 If the *stdin* argument was :data:`PIPE`, this attribute is a writeable 698 stream object as returned by :func:`open`. If the *encoding* or *errors* 699 arguments were specified or the *universal_newlines* argument was ``True``, 700 the stream is a text stream, otherwise it is a byte stream. If the *stdin* 701 argument was not :data:`PIPE`, this attribute is ``None``. 702 703 704 .. attribute:: Popen.stdout 705 706 If the *stdout* argument was :data:`PIPE`, this attribute is a readable 707 stream object as returned by :func:`open`. Reading from the stream provides 708 output from the child process. If the *encoding* or *errors* arguments were 709 specified or the *universal_newlines* argument was ``True``, the stream is a 710 text stream, otherwise it is a byte stream. If the *stdout* argument was not 711 :data:`PIPE`, this attribute is ``None``. 712 713 714 .. attribute:: Popen.stderr 715 716 If the *stderr* argument was :data:`PIPE`, this attribute is a readable 717 stream object as returned by :func:`open`. Reading from the stream provides 718 error output from the child process. If the *encoding* or *errors* arguments 719 were specified or the *universal_newlines* argument was ``True``, the stream 720 is a text stream, otherwise it is a byte stream. If the *stderr* argument was 721 not :data:`PIPE`, this attribute is ``None``. 722 723 .. warning:: 724 725 Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`, 726 :attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid 727 deadlocks due to any of the other OS pipe buffers filling up and blocking the 728 child process. 729 730 731 .. attribute:: Popen.pid 732 733 The process ID of the child process. 734 735 Note that if you set the *shell* argument to ``True``, this is the process ID 736 of the spawned shell. 737 738 739 .. attribute:: Popen.returncode 740 741 The child return code, set by :meth:`poll` and :meth:`wait` (and indirectly 742 by :meth:`communicate`). A ``None`` value indicates that the process 743 hasn't terminated yet. 744 745 A negative value ``-N`` indicates that the child was terminated by signal 746 ``N`` (POSIX only). 747 748 749 Windows Popen Helpers 750 --------------------- 751 752 The :class:`STARTUPINFO` class and following constants are only available 753 on Windows. 754 755 .. class:: STARTUPINFO() 756 757 Partial support of the Windows 758 `STARTUPINFO <https://msdn.microsoft.com/en-us/library/ms686331(v=vs.85).aspx>`__ 759 structure is used for :class:`Popen` creation. 760 761 .. attribute:: dwFlags 762 763 A bit field that determines whether certain :class:`STARTUPINFO` 764 attributes are used when the process creates a window. :: 765 766 si = subprocess.STARTUPINFO() 767 si.dwFlags = subprocess.STARTF_USESTDHANDLES | subprocess.STARTF_USESHOWWINDOW 768 769 .. attribute:: hStdInput 770 771 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute 772 is the standard input handle for the process. If 773 :data:`STARTF_USESTDHANDLES` is not specified, the default for standard 774 input is the keyboard buffer. 775 776 .. attribute:: hStdOutput 777 778 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute 779 is the standard output handle for the process. Otherwise, this attribute 780 is ignored and the default for standard output is the console window's 781 buffer. 782 783 .. attribute:: hStdError 784 785 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute 786 is the standard error handle for the process. Otherwise, this attribute is 787 ignored and the default for standard error is the console window's buffer. 788 789 .. attribute:: wShowWindow 790 791 If :attr:`dwFlags` specifies :data:`STARTF_USESHOWWINDOW`, this attribute 792 can be any of the values that can be specified in the ``nCmdShow`` 793 parameter for the 794 `ShowWindow <https://msdn.microsoft.com/en-us/library/ms633548(v=vs.85).aspx>`__ 795 function, except for ``SW_SHOWDEFAULT``. Otherwise, this attribute is 796 ignored. 797 798 :data:`SW_HIDE` is provided for this attribute. It is used when 799 :class:`Popen` is called with ``shell=True``. 800 801 802 Constants 803 ^^^^^^^^^ 804 805 The :mod:`subprocess` module exposes the following constants. 806 807 .. data:: STD_INPUT_HANDLE 808 809 The standard input device. Initially, this is the console input buffer, 810 ``CONIN$``. 811 812 .. data:: STD_OUTPUT_HANDLE 813 814 The standard output device. Initially, this is the active console screen 815 buffer, ``CONOUT$``. 816 817 .. data:: STD_ERROR_HANDLE 818 819 The standard error device. Initially, this is the active console screen 820 buffer, ``CONOUT$``. 821 822 .. data:: SW_HIDE 823 824 Hides the window. Another window will be activated. 825 826 .. data:: STARTF_USESTDHANDLES 827 828 Specifies that the :attr:`STARTUPINFO.hStdInput`, 829 :attr:`STARTUPINFO.hStdOutput`, and :attr:`STARTUPINFO.hStdError` attributes 830 contain additional information. 831 832 .. data:: STARTF_USESHOWWINDOW 833 834 Specifies that the :attr:`STARTUPINFO.wShowWindow` attribute contains 835 additional information. 836 837 .. data:: CREATE_NEW_CONSOLE 838 839 The new process has a new console, instead of inheriting its parent's 840 console (the default). 841 842 .. data:: CREATE_NEW_PROCESS_GROUP 843 844 A :class:`Popen` ``creationflags`` parameter to specify that a new process 845 group will be created. This flag is necessary for using :func:`os.kill` 846 on the subprocess. 847 848 This flag is ignored if :data:`CREATE_NEW_CONSOLE` is specified. 849 850 .. _call-function-trio: 851 852 Older high-level API 853 -------------------- 854 855 Prior to Python 3.5, these three functions comprised the high level API to 856 subprocess. You can now use :func:`run` in many cases, but lots of existing code 857 calls these functions. 858 859 .. function:: call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None) 860 861 Run the command described by *args*. Wait for command to complete, then 862 return the :attr:`~Popen.returncode` attribute. 863 864 This is equivalent to:: 865 866 run(...).returncode 867 868 (except that the *input* and *check* parameters are not supported) 869 870 The arguments shown above are merely the most 871 common ones. The full function signature is largely the 872 same as that of the :class:`Popen` constructor - this function passes all 873 supplied arguments other than *timeout* directly through to that interface. 874 875 .. note:: 876 877 Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this 878 function. The child process will block if it generates enough 879 output to a pipe to fill up the OS pipe buffer as the pipes are 880 not being read from. 881 882 .. versionchanged:: 3.3 883 *timeout* was added. 884 885 .. function:: check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None) 886 887 Run command with arguments. Wait for command to complete. If the return 888 code was zero then return, otherwise raise :exc:`CalledProcessError`. The 889 :exc:`CalledProcessError` object will have the return code in the 890 :attr:`~CalledProcessError.returncode` attribute. 891 892 This is equivalent to:: 893 894 run(..., check=True) 895 896 (except that the *input* parameter is not supported) 897 898 The arguments shown above are merely the most 899 common ones. The full function signature is largely the 900 same as that of the :class:`Popen` constructor - this function passes all 901 supplied arguments other than *timeout* directly through to that interface. 902 903 .. note:: 904 905 Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this 906 function. The child process will block if it generates enough 907 output to a pipe to fill up the OS pipe buffer as the pipes are 908 not being read from. 909 910 .. versionchanged:: 3.3 911 *timeout* was added. 912 913 914 .. function:: check_output(args, *, stdin=None, stderr=None, shell=False, \ 915 encoding=None, errors=None, \ 916 universal_newlines=False, timeout=None) 917 918 Run command with arguments and return its output. 919 920 If the return code was non-zero it raises a :exc:`CalledProcessError`. The 921 :exc:`CalledProcessError` object will have the return code in the 922 :attr:`~CalledProcessError.returncode` attribute and any output in the 923 :attr:`~CalledProcessError.output` attribute. 924 925 This is equivalent to:: 926 927 run(..., check=True, stdout=PIPE).stdout 928 929 The arguments shown above are merely the most common ones. 930 The full function signature is largely the same as that of :func:`run` - 931 most arguments are passed directly through to that interface. 932 However, explicitly passing ``input=None`` to inherit the parent's 933 standard input file handle is not supported. 934 935 By default, this function will return the data as encoded bytes. The actual 936 encoding of the output data may depend on the command being invoked, so the 937 decoding to text will often need to be handled at the application level. 938 939 This behaviour may be overridden by setting *universal_newlines* to 940 ``True`` as described above in :ref:`frequently-used-arguments`. 941 942 To also capture standard error in the result, use 943 ``stderr=subprocess.STDOUT``:: 944 945 >>> subprocess.check_output( 946 ... "ls non_existent_file; exit 0", 947 ... stderr=subprocess.STDOUT, 948 ... shell=True) 949 'ls: non_existent_file: No such file or directory\n' 950 951 .. versionadded:: 3.1 952 953 .. versionchanged:: 3.3 954 *timeout* was added. 955 956 .. versionchanged:: 3.4 957 Support for the *input* keyword argument was added. 958 959 .. _subprocess-replacements: 960 961 Replacing Older Functions with the :mod:`subprocess` Module 962 ----------------------------------------------------------- 963 964 In this section, "a becomes b" means that b can be used as a replacement for a. 965 966 .. note:: 967 968 All "a" functions in this section fail (more or less) silently if the 969 executed program cannot be found; the "b" replacements raise :exc:`OSError` 970 instead. 971 972 In addition, the replacements using :func:`check_output` will fail with a 973 :exc:`CalledProcessError` if the requested operation produces a non-zero 974 return code. The output is still available as the 975 :attr:`~CalledProcessError.output` attribute of the raised exception. 976 977 In the following examples, we assume that the relevant functions have already 978 been imported from the :mod:`subprocess` module. 979 980 981 Replacing /bin/sh shell backquote 982 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 983 984 .. code-block:: bash 985 986 output=`mycmd myarg` 987 988 becomes:: 989 990 output = check_output(["mycmd", "myarg"]) 991 992 Replacing shell pipeline 993 ^^^^^^^^^^^^^^^^^^^^^^^^ 994 995 .. code-block:: bash 996 997 output=`dmesg | grep hda` 998 999 becomes:: 1000 1001 p1 = Popen(["dmesg"], stdout=PIPE) 1002 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) 1003 p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. 1004 output = p2.communicate()[0] 1005 1006 The p1.stdout.close() call after starting the p2 is important in order for p1 1007 to receive a SIGPIPE if p2 exits before p1. 1008 1009 Alternatively, for trusted input, the shell's own pipeline support may still 1010 be used directly: 1011 1012 .. code-block:: bash 1013 1014 output=`dmesg | grep hda` 1015 1016 becomes:: 1017 1018 output=check_output("dmesg | grep hda", shell=True) 1019 1020 1021 Replacing :func:`os.system` 1022 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1023 1024 :: 1025 1026 sts = os.system("mycmd" + " myarg") 1027 # becomes 1028 sts = call("mycmd" + " myarg", shell=True) 1029 1030 Notes: 1031 1032 * Calling the program through the shell is usually not required. 1033 1034 A more realistic example would look like this:: 1035 1036 try: 1037 retcode = call("mycmd" + " myarg", shell=True) 1038 if retcode < 0: 1039 print("Child was terminated by signal", -retcode, file=sys.stderr) 1040 else: 1041 print("Child returned", retcode, file=sys.stderr) 1042 except OSError as e: 1043 print("Execution failed:", e, file=sys.stderr) 1044 1045 1046 Replacing the :func:`os.spawn <os.spawnl>` family 1047 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1048 1049 P_NOWAIT example:: 1050 1051 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg") 1052 ==> 1053 pid = Popen(["/bin/mycmd", "myarg"]).pid 1054 1055 P_WAIT example:: 1056 1057 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg") 1058 ==> 1059 retcode = call(["/bin/mycmd", "myarg"]) 1060 1061 Vector example:: 1062 1063 os.spawnvp(os.P_NOWAIT, path, args) 1064 ==> 1065 Popen([path] + args[1:]) 1066 1067 Environment example:: 1068 1069 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env) 1070 ==> 1071 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"}) 1072 1073 1074 1075 Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3` 1076 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1077 1078 :: 1079 1080 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize) 1081 ==> 1082 p = Popen(cmd, shell=True, bufsize=bufsize, 1083 stdin=PIPE, stdout=PIPE, close_fds=True) 1084 (child_stdin, child_stdout) = (p.stdin, p.stdout) 1085 1086 :: 1087 1088 (child_stdin, 1089 child_stdout, 1090 child_stderr) = os.popen3(cmd, mode, bufsize) 1091 ==> 1092 p = Popen(cmd, shell=True, bufsize=bufsize, 1093 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True) 1094 (child_stdin, 1095 child_stdout, 1096 child_stderr) = (p.stdin, p.stdout, p.stderr) 1097 1098 :: 1099 1100 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize) 1101 ==> 1102 p = Popen(cmd, shell=True, bufsize=bufsize, 1103 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) 1104 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout) 1105 1106 Return code handling translates as follows:: 1107 1108 pipe = os.popen(cmd, 'w') 1109 ... 1110 rc = pipe.close() 1111 if rc is not None and rc >> 8: 1112 print("There were some errors") 1113 ==> 1114 process = Popen(cmd, stdin=PIPE) 1115 ... 1116 process.stdin.close() 1117 if process.wait() != 0: 1118 print("There were some errors") 1119 1120 1121 Replacing functions from the :mod:`popen2` module 1122 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1123 1124 .. note:: 1125 1126 If the cmd argument to popen2 functions is a string, the command is executed 1127 through /bin/sh. If it is a list, the command is directly executed. 1128 1129 :: 1130 1131 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode) 1132 ==> 1133 p = Popen("somestring", shell=True, bufsize=bufsize, 1134 stdin=PIPE, stdout=PIPE, close_fds=True) 1135 (child_stdout, child_stdin) = (p.stdout, p.stdin) 1136 1137 :: 1138 1139 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode) 1140 ==> 1141 p = Popen(["mycmd", "myarg"], bufsize=bufsize, 1142 stdin=PIPE, stdout=PIPE, close_fds=True) 1143 (child_stdout, child_stdin) = (p.stdout, p.stdin) 1144 1145 :class:`popen2.Popen3` and :class:`popen2.Popen4` basically work as 1146 :class:`subprocess.Popen`, except that: 1147 1148 * :class:`Popen` raises an exception if the execution fails. 1149 1150 * the *capturestderr* argument is replaced with the *stderr* argument. 1151 1152 * ``stdin=PIPE`` and ``stdout=PIPE`` must be specified. 1153 1154 * popen2 closes all file descriptors by default, but you have to specify 1155 ``close_fds=True`` with :class:`Popen` to guarantee this behavior on 1156 all platforms or past Python versions. 1157 1158 1159 Legacy Shell Invocation Functions 1160 --------------------------------- 1161 1162 This module also provides the following legacy functions from the 2.x 1163 ``commands`` module. These operations implicitly invoke the system shell and 1164 none of the guarantees described above regarding security and exception 1165 handling consistency are valid for these functions. 1166 1167 .. function:: getstatusoutput(cmd) 1168 1169 Return ``(status, output)`` of executing *cmd* in a shell. 1170 1171 Execute the string *cmd* in a shell with :meth:`Popen.check_output` and 1172 return a 2-tuple ``(status, output)``. The locale encoding is used; 1173 see the notes on :ref:`frequently-used-arguments` for more details. 1174 1175 A trailing newline is stripped from the output. 1176 The exit status for the command can be interpreted 1177 according to the rules for the C function :c:func:`wait`. Example:: 1178 1179 >>> subprocess.getstatusoutput('ls /bin/ls') 1180 (0, '/bin/ls') 1181 >>> subprocess.getstatusoutput('cat /bin/junk') 1182 (256, 'cat: /bin/junk: No such file or directory') 1183 >>> subprocess.getstatusoutput('/bin/junk') 1184 (256, 'sh: /bin/junk: not found') 1185 1186 Availability: POSIX & Windows 1187 1188 .. versionchanged:: 3.3.4 1189 Windows support added 1190 1191 1192 .. function:: getoutput(cmd) 1193 1194 Return output (stdout and stderr) of executing *cmd* in a shell. 1195 1196 Like :func:`getstatusoutput`, except the exit status is ignored and the return 1197 value is a string containing the command's output. Example:: 1198 1199 >>> subprocess.getoutput('ls /bin/ls') 1200 '/bin/ls' 1201 1202 Availability: POSIX & Windows 1203 1204 .. versionchanged:: 3.3.4 1205 Windows support added 1206 1207 1208 Notes 1209 ----- 1210 1211 .. _converting-argument-sequence: 1212 1213 Converting an argument sequence to a string on Windows 1214 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1215 1216 On Windows, an *args* sequence is converted to a string that can be parsed 1217 using the following rules (which correspond to the rules used by the MS C 1218 runtime): 1219 1220 1. Arguments are delimited by white space, which is either a 1221 space or a tab. 1222 1223 2. A string surrounded by double quotation marks is 1224 interpreted as a single argument, regardless of white space 1225 contained within. A quoted string can be embedded in an 1226 argument. 1227 1228 3. A double quotation mark preceded by a backslash is 1229 interpreted as a literal double quotation mark. 1230 1231 4. Backslashes are interpreted literally, unless they 1232 immediately precede a double quotation mark. 1233 1234 5. If backslashes immediately precede a double quotation mark, 1235 every pair of backslashes is interpreted as a literal 1236 backslash. If the number of backslashes is odd, the last 1237 backslash escapes the next double quotation mark as 1238 described in rule 3. 1239 1240 1241 .. seealso:: 1242 1243 :mod:`shlex` 1244 Module which provides function to parse and escape command lines. 1245