1 .. currentmodule:: asyncio 2 3 .. _asyncio-dev: 4 5 ======================= 6 Developing with asyncio 7 ======================= 8 9 Asynchronous programming is different from classic "sequential" 10 programming. 11 12 This page lists common mistakes and traps and explains how 13 to avoid them. 14 15 16 .. _asyncio-debug-mode: 17 18 Debug Mode 19 ========== 20 21 By default asyncio runs in production mode. In order to ease 22 the development asyncio has a *debug mode*. 23 24 There are several ways to enable asyncio debug mode: 25 26 * Setting the :envvar:`PYTHONASYNCIODEBUG` environment variable to ``1``. 27 28 * Using the :option:`-X` ``dev`` Python command line option. 29 30 * Passing ``debug=True`` to :func:`asyncio.run`. 31 32 * Calling :meth:`loop.set_debug`. 33 34 In addition to enabling the debug mode, consider also: 35 36 * setting the log level of the :ref:`asyncio logger <asyncio-logger>` to 37 :py:data:`logging.DEBUG`, for example the following snippet of code 38 can be run at startup of the application:: 39 40 logging.basicConfig(level=logging.DEBUG) 41 42 * configuring the :mod:`warnings` module to display 43 :exc:`ResourceWarning` warnings. One way of doing that is by 44 using the :option:`-W` ``default`` command line option. 45 46 47 When the debug mode is enabled: 48 49 * asyncio checks for :ref:`coroutines that were not awaited 50 <asyncio-coroutine-not-scheduled>` and logs them; this mitigates 51 the "forgotten await" pitfall. 52 53 * Many non-threadsafe asyncio APIs (such as :meth:`loop.call_soon` and 54 :meth:`loop.call_at` methods) raise an exception if they are called 55 from a wrong thread. 56 57 * The execution time of the I/O selector is logged if it takes too long to 58 perform an I/O operation. 59 60 * Callbacks taking longer than 100ms are logged. The 61 :attr:`loop.slow_callback_duration` attribute can be used to set the 62 minimum execution duration in seconds that is considered "slow". 63 64 65 .. _asyncio-multithreading: 66 67 Concurrency and Multithreading 68 ============================== 69 70 An event loop runs in a thread (typically the main thread) and executes 71 all callbacks and Tasks in its thread. While a Task is running in the 72 event loop, no other Tasks can run in the same thread. When a Task 73 executes an ``await`` expression, the running Task gets suspended, and 74 the event loop executes the next Task. 75 76 To schedule a callback from a different OS thread, the 77 :meth:`loop.call_soon_threadsafe` method should be used. Example:: 78 79 loop.call_soon_threadsafe(callback, *args) 80 81 Almost all asyncio objects are not thread safe, which is typically 82 not a problem unless there is code that works with them from outside 83 of a Task or a callback. If there's a need for such code to call a 84 low-level asyncio API, the :meth:`loop.call_soon_threadsafe` method 85 should be used, e.g.:: 86 87 loop.call_soon_threadsafe(fut.cancel) 88 89 To schedule a coroutine object from a different OS thread, the 90 :func:`run_coroutine_threadsafe` function should be used. It returns a 91 :class:`concurrent.futures.Future` to access the result:: 92 93 async def coro_func(): 94 return await asyncio.sleep(1, 42) 95 96 # Later in another OS thread: 97 98 future = asyncio.run_coroutine_threadsafe(coro_func(), loop) 99 # Wait for the result: 100 result = future.result() 101 102 To handle signals and to execute subprocesses, the event loop must be 103 run in the main thread. 104 105 The :meth:`loop.run_in_executor` method can be used with a 106 :class:`concurrent.futures.ThreadPoolExecutor` to execute 107 blocking code in a different OS thread without blocking the OS thread 108 that the event loop runs in. 109 110 111 .. _asyncio-handle-blocking: 112 113 Running Blocking Code 114 ===================== 115 116 Blocking (CPU-bound) code should not be called directly. For example, 117 if a function performs a CPU-intensive calculation for 1 second, 118 all concurrent asyncio Tasks and IO operations would be delayed 119 by 1 second. 120 121 An executor can be used to run a task in a different thread or even in 122 a different process to avoid blocking block the OS thread with the 123 event loop. See the :meth:`loop.run_in_executor` method for more 124 details. 125 126 127 .. _asyncio-logger: 128 129 Logging 130 ======= 131 132 asyncio uses the :mod:`logging` module and all logging is performed 133 via the ``"asyncio"`` logger. 134 135 The default log level is :py:data:`logging.INFO`, which can be easily 136 adjusted:: 137 138 logging.getLogger("asyncio").setLevel(logging.WARNING) 139 140 141 .. _asyncio-coroutine-not-scheduled: 142 143 Detect never-awaited coroutines 144 =============================== 145 146 When a coroutine function is called, but not awaited 147 (e.g. ``coro()`` instead of ``await coro()``) 148 or the coroutine is not scheduled with :meth:`asyncio.create_task`, asyncio 149 will emit a :exc:`RuntimeWarning`:: 150 151 import asyncio 152 153 async def test(): 154 print("never scheduled") 155 156 async def main(): 157 test() 158 159 asyncio.run(main()) 160 161 Output:: 162 163 test.py:7: RuntimeWarning: coroutine 'test' was never awaited 164 test() 165 166 Output in debug mode:: 167 168 test.py:7: RuntimeWarning: coroutine 'test' was never awaited 169 Coroutine created at (most recent call last) 170 File "../t.py", line 9, in <module> 171 asyncio.run(main(), debug=True) 172 173 < .. > 174 175 File "../t.py", line 7, in main 176 test() 177 test() 178 179 The usual fix is to either await the coroutine or call the 180 :meth:`asyncio.create_task` function:: 181 182 async def main(): 183 await test() 184 185 186 Detect never-retrieved exceptions 187 ================================= 188 189 If a :meth:`Future.set_exception` is called but the Future object is 190 never awaited on, the exception would never be propagated to the 191 user code. In this case, asyncio would emit a log message when the 192 Future object is garbage collected. 193 194 Example of an unhandled exception:: 195 196 import asyncio 197 198 async def bug(): 199 raise Exception("not consumed") 200 201 async def main(): 202 asyncio.create_task(bug()) 203 204 asyncio.run(main()) 205 206 Output:: 207 208 Task exception was never retrieved 209 future: <Task finished coro=<bug() done, defined at test.py:3> 210 exception=Exception('not consumed')> 211 212 Traceback (most recent call last): 213 File "test.py", line 4, in bug 214 raise Exception("not consumed") 215 Exception: not consumed 216 217 :ref:`Enable the debug mode <asyncio-debug-mode>` to get the 218 traceback where the task was created:: 219 220 asyncio.run(main(), debug=True) 221 222 Output in debug mode:: 223 224 Task exception was never retrieved 225 future: <Task finished coro=<bug() done, defined at test.py:3> 226 exception=Exception('not consumed') created at asyncio/tasks.py:321> 227 228 source_traceback: Object created at (most recent call last): 229 File "../t.py", line 9, in <module> 230 asyncio.run(main(), debug=True) 231 232 < .. > 233 234 Traceback (most recent call last): 235 File "../t.py", line 4, in bug 236 raise Exception("not consumed") 237 Exception: not consumed 238
