1 :mod:`queue` --- A synchronized queue class 2 =========================================== 3 4 .. module:: queue 5 :synopsis: A synchronized queue class. 6 7 **Source code:** :source:`Lib/queue.py` 8 9 -------------- 10 11 The :mod:`queue` module implements multi-producer, multi-consumer queues. 12 It is especially useful in threaded programming when information must be 13 exchanged safely between multiple threads. The :class:`Queue` class in this 14 module implements all the required locking semantics. It depends on the 15 availability of thread support in Python; see the :mod:`threading` 16 module. 17 18 The module implements three types of queue, which differ only in the order in 19 which the entries are retrieved. In a :abbr:`FIFO (first-in, first-out)` 20 queue, the first tasks added are the first retrieved. In a 21 :abbr:`LIFO (last-in, first-out)` queue, the most recently added entry is 22 the first retrieved (operating like a stack). With a priority queue, 23 the entries are kept sorted (using the :mod:`heapq` module) and the 24 lowest valued entry is retrieved first. 25 26 Internally, those three types of queues use locks to temporarily block 27 competing threads; however, they are not designed to handle reentrancy 28 within a thread. 29 30 In addition, the module implements a "simple" 31 :abbr:`FIFO (first-in, first-out)` queue type, :class:`SimpleQueue`, whose 32 specific implementation provides additional guarantees 33 in exchange for the smaller functionality. 34 35 The :mod:`queue` module defines the following classes and exceptions: 36 37 .. class:: Queue(maxsize=0) 38 39 Constructor for a :abbr:`FIFO (first-in, first-out)` queue. *maxsize* is 40 an integer that sets the upperbound 41 limit on the number of items that can be placed in the queue. Insertion will 42 block once this size has been reached, until queue items are consumed. If 43 *maxsize* is less than or equal to zero, the queue size is infinite. 44 45 .. class:: LifoQueue(maxsize=0) 46 47 Constructor for a :abbr:`LIFO (last-in, first-out)` queue. *maxsize* is 48 an integer that sets the upperbound 49 limit on the number of items that can be placed in the queue. Insertion will 50 block once this size has been reached, until queue items are consumed. If 51 *maxsize* is less than or equal to zero, the queue size is infinite. 52 53 54 .. class:: PriorityQueue(maxsize=0) 55 56 Constructor for a priority queue. *maxsize* is an integer that sets the upperbound 57 limit on the number of items that can be placed in the queue. Insertion will 58 block once this size has been reached, until queue items are consumed. If 59 *maxsize* is less than or equal to zero, the queue size is infinite. 60 61 The lowest valued entries are retrieved first (the lowest valued entry is the 62 one returned by ``sorted(list(entries))[0]``). A typical pattern for entries 63 is a tuple in the form: ``(priority_number, data)``. 64 65 If the *data* elements are not comparable, the data can be wrapped in a class 66 that ignores the data item and only compares the priority number:: 67 68 from dataclasses import dataclass, field 69 from typing import Any 70 71 @dataclass(order=True) 72 class PrioritizedItem: 73 priority: int 74 item: Any=field(compare=False) 75 76 .. class:: SimpleQueue() 77 78 Constructor for an unbounded :abbr:`FIFO (first-in, first-out)` queue. 79 Simple queues lack advanced functionality such as task tracking. 80 81 .. versionadded:: 3.7 82 83 84 .. exception:: Empty 85 86 Exception raised when non-blocking :meth:`~Queue.get` (or 87 :meth:`~Queue.get_nowait`) is called 88 on a :class:`Queue` object which is empty. 89 90 91 .. exception:: Full 92 93 Exception raised when non-blocking :meth:`~Queue.put` (or 94 :meth:`~Queue.put_nowait`) is called 95 on a :class:`Queue` object which is full. 96 97 98 .. _queueobjects: 99 100 Queue Objects 101 ------------- 102 103 Queue objects (:class:`Queue`, :class:`LifoQueue`, or :class:`PriorityQueue`) 104 provide the public methods described below. 105 106 107 .. method:: Queue.qsize() 108 109 Return the approximate size of the queue. Note, qsize() > 0 doesn't 110 guarantee that a subsequent get() will not block, nor will qsize() < maxsize 111 guarantee that put() will not block. 112 113 114 .. method:: Queue.empty() 115 116 Return ``True`` if the queue is empty, ``False`` otherwise. If empty() 117 returns ``True`` it doesn't guarantee that a subsequent call to put() 118 will not block. Similarly, if empty() returns ``False`` it doesn't 119 guarantee that a subsequent call to get() will not block. 120 121 122 .. method:: Queue.full() 123 124 Return ``True`` if the queue is full, ``False`` otherwise. If full() 125 returns ``True`` it doesn't guarantee that a subsequent call to get() 126 will not block. Similarly, if full() returns ``False`` it doesn't 127 guarantee that a subsequent call to put() will not block. 128 129 130 .. method:: Queue.put(item, block=True, timeout=None) 131 132 Put *item* into the queue. If optional args *block* is true and *timeout* is 133 ``None`` (the default), block if necessary until a free slot is available. If 134 *timeout* is a positive number, it blocks at most *timeout* seconds and raises 135 the :exc:`Full` exception if no free slot was available within that time. 136 Otherwise (*block* is false), put an item on the queue if a free slot is 137 immediately available, else raise the :exc:`Full` exception (*timeout* is 138 ignored in that case). 139 140 141 .. method:: Queue.put_nowait(item) 142 143 Equivalent to ``put(item, False)``. 144 145 146 .. method:: Queue.get(block=True, timeout=None) 147 148 Remove and return an item from the queue. If optional args *block* is true and 149 *timeout* is ``None`` (the default), block if necessary until an item is available. 150 If *timeout* is a positive number, it blocks at most *timeout* seconds and 151 raises the :exc:`Empty` exception if no item was available within that time. 152 Otherwise (*block* is false), return an item if one is immediately available, 153 else raise the :exc:`Empty` exception (*timeout* is ignored in that case). 154 155 156 .. method:: Queue.get_nowait() 157 158 Equivalent to ``get(False)``. 159 160 Two methods are offered to support tracking whether enqueued tasks have been 161 fully processed by daemon consumer threads. 162 163 164 .. method:: Queue.task_done() 165 166 Indicate that a formerly enqueued task is complete. Used by queue consumer 167 threads. For each :meth:`get` used to fetch a task, a subsequent call to 168 :meth:`task_done` tells the queue that the processing on the task is complete. 169 170 If a :meth:`join` is currently blocking, it will resume when all items have been 171 processed (meaning that a :meth:`task_done` call was received for every item 172 that had been :meth:`put` into the queue). 173 174 Raises a :exc:`ValueError` if called more times than there were items placed in 175 the queue. 176 177 178 .. method:: Queue.join() 179 180 Blocks until all items in the queue have been gotten and processed. 181 182 The count of unfinished tasks goes up whenever an item is added to the queue. 183 The count goes down whenever a consumer thread calls :meth:`task_done` to 184 indicate that the item was retrieved and all work on it is complete. When the 185 count of unfinished tasks drops to zero, :meth:`join` unblocks. 186 187 188 Example of how to wait for enqueued tasks to be completed:: 189 190 def worker(): 191 while True: 192 item = q.get() 193 if item is None: 194 break 195 do_work(item) 196 q.task_done() 197 198 q = queue.Queue() 199 threads = [] 200 for i in range(num_worker_threads): 201 t = threading.Thread(target=worker) 202 t.start() 203 threads.append(t) 204 205 for item in source(): 206 q.put(item) 207 208 # block until all tasks are done 209 q.join() 210 211 # stop workers 212 for i in range(num_worker_threads): 213 q.put(None) 214 for t in threads: 215 t.join() 216 217 218 SimpleQueue Objects 219 ------------------- 220 221 :class:`SimpleQueue` objects provide the public methods described below. 222 223 .. method:: SimpleQueue.qsize() 224 225 Return the approximate size of the queue. Note, qsize() > 0 doesn't 226 guarantee that a subsequent get() will not block. 227 228 229 .. method:: SimpleQueue.empty() 230 231 Return ``True`` if the queue is empty, ``False`` otherwise. If empty() 232 returns ``False`` it doesn't guarantee that a subsequent call to get() 233 will not block. 234 235 236 .. method:: SimpleQueue.put(item, block=True, timeout=None) 237 238 Put *item* into the queue. The method never blocks and always succeeds 239 (except for potential low-level errors such as failure to allocate memory). 240 The optional args *block* and *timeout* are ignored and only provided 241 for compatibility with :meth:`Queue.put`. 242 243 .. impl-detail:: 244 This method has a C implementation which is reentrant. That is, a 245 ``put()`` or ``get()`` call can be interrupted by another ``put()`` 246 call in the same thread without deadlocking or corrupting internal 247 state inside the queue. This makes it appropriate for use in 248 destructors such as ``__del__`` methods or :mod:`weakref` callbacks. 249 250 251 .. method:: SimpleQueue.put_nowait(item) 252 253 Equivalent to ``put(item)``, provided for compatibility with 254 :meth:`Queue.put_nowait`. 255 256 257 .. method:: SimpleQueue.get(block=True, timeout=None) 258 259 Remove and return an item from the queue. If optional args *block* is true and 260 *timeout* is ``None`` (the default), block if necessary until an item is available. 261 If *timeout* is a positive number, it blocks at most *timeout* seconds and 262 raises the :exc:`Empty` exception if no item was available within that time. 263 Otherwise (*block* is false), return an item if one is immediately available, 264 else raise the :exc:`Empty` exception (*timeout* is ignored in that case). 265 266 267 .. method:: SimpleQueue.get_nowait() 268 269 Equivalent to ``get(False)``. 270 271 272 .. seealso:: 273 274 Class :class:`multiprocessing.Queue` 275 A queue class for use in a multi-processing (rather than multi-threading) 276 context. 277 278 :class:`collections.deque` is an alternative implementation of unbounded 279 queues with fast atomic :meth:`~collections.deque.append` and 280 :meth:`~collections.deque.popleft` operations that do not require locking. 281
