1 :mod:`mmap` --- Memory-mapped file support 2 ========================================== 3 4 .. module:: mmap 5 :synopsis: Interface to memory-mapped files for Unix and Windows. 6 7 -------------- 8 9 Memory-mapped file objects behave like both :class:`bytearray` and like 10 :term:`file objects <file object>`. You can use mmap objects in most places 11 where :class:`bytearray` are expected; for example, you can use the :mod:`re` 12 module to search through a memory-mapped file. You can also change a single 13 byte by doing ``obj[index] = 97``, or change a subsequence by assigning to a 14 slice: ``obj[i1:i2] = b'...'``. You can also read and write data starting at 15 the current file position, and :meth:`seek` through the file to different positions. 16 17 A memory-mapped file is created by the :class:`~mmap.mmap` constructor, which is 18 different on Unix and on Windows. In either case you must provide a file 19 descriptor for a file opened for update. If you wish to map an existing Python 20 file object, use its :meth:`fileno` method to obtain the correct value for the 21 *fileno* parameter. Otherwise, you can open the file using the 22 :func:`os.open` function, which returns a file descriptor directly (the file 23 still needs to be closed when done). 24 25 .. note:: 26 If you want to create a memory-mapping for a writable, buffered file, you 27 should :func:`~io.IOBase.flush` the file first. This is necessary to ensure 28 that local modifications to the buffers are actually available to the 29 mapping. 30 31 For both the Unix and Windows versions of the constructor, *access* may be 32 specified as an optional keyword parameter. *access* accepts one of three 33 values: :const:`ACCESS_READ`, :const:`ACCESS_WRITE`, or :const:`ACCESS_COPY` 34 to specify read-only, write-through or copy-on-write memory respectively. 35 *access* can be used on both Unix and Windows. If *access* is not specified, 36 Windows mmap returns a write-through mapping. The initial memory values for 37 all three access types are taken from the specified file. Assignment to an 38 :const:`ACCESS_READ` memory map raises a :exc:`TypeError` exception. 39 Assignment to an :const:`ACCESS_WRITE` memory map affects both memory and the 40 underlying file. Assignment to an :const:`ACCESS_COPY` memory map affects 41 memory but does not update the underlying file. 42 43 To map anonymous memory, -1 should be passed as the fileno along with the length. 44 45 .. class:: mmap(fileno, length, tagname=None, access=ACCESS_DEFAULT[, offset]) 46 47 **(Windows version)** Maps *length* bytes from the file specified by the 48 file handle *fileno*, and creates a mmap object. If *length* is larger 49 than the current size of the file, the file is extended to contain *length* 50 bytes. If *length* is ``0``, the maximum length of the map is the current 51 size of the file, except that if the file is empty Windows raises an 52 exception (you cannot create an empty mapping on Windows). 53 54 *tagname*, if specified and not ``None``, is a string giving a tag name for 55 the mapping. Windows allows you to have many different mappings against 56 the same file. If you specify the name of an existing tag, that tag is 57 opened, otherwise a new tag of this name is created. If this parameter is 58 omitted or ``None``, the mapping is created without a name. Avoiding the 59 use of the tag parameter will assist in keeping your code portable between 60 Unix and Windows. 61 62 *offset* may be specified as a non-negative integer offset. mmap references 63 will be relative to the offset from the beginning of the file. *offset* 64 defaults to 0. *offset* must be a multiple of the ALLOCATIONGRANULARITY. 65 66 67 .. class:: mmap(fileno, length, flags=MAP_SHARED, prot=PROT_WRITE|PROT_READ, access=ACCESS_DEFAULT[, offset]) 68 :noindex: 69 70 **(Unix version)** Maps *length* bytes from the file specified by the file 71 descriptor *fileno*, and returns a mmap object. If *length* is ``0``, the 72 maximum length of the map will be the current size of the file when 73 :class:`~mmap.mmap` is called. 74 75 *flags* specifies the nature of the mapping. :const:`MAP_PRIVATE` creates a 76 private copy-on-write mapping, so changes to the contents of the mmap 77 object will be private to this process, and :const:`MAP_SHARED` creates a 78 mapping that's shared with all other processes mapping the same areas of 79 the file. The default value is :const:`MAP_SHARED`. 80 81 *prot*, if specified, gives the desired memory protection; the two most 82 useful values are :const:`PROT_READ` and :const:`PROT_WRITE`, to specify 83 that the pages may be read or written. *prot* defaults to 84 :const:`PROT_READ \| PROT_WRITE`. 85 86 *access* may be specified in lieu of *flags* and *prot* as an optional 87 keyword parameter. It is an error to specify both *flags*, *prot* and 88 *access*. See the description of *access* above for information on how to 89 use this parameter. 90 91 *offset* may be specified as a non-negative integer offset. mmap references 92 will be relative to the offset from the beginning of the file. *offset* 93 defaults to 0. *offset* must be a multiple of the PAGESIZE or 94 ALLOCATIONGRANULARITY. 95 96 To ensure validity of the created memory mapping the file specified 97 by the descriptor *fileno* is internally automatically synchronized 98 with physical backing store on Mac OS X and OpenVMS. 99 100 This example shows a simple way of using :class:`~mmap.mmap`:: 101 102 import mmap 103 104 # write a simple example file 105 with open("hello.txt", "wb") as f: 106 f.write(b"Hello Python!\n") 107 108 with open("hello.txt", "r+b") as f: 109 # memory-map the file, size 0 means whole file 110 mm = mmap.mmap(f.fileno(), 0) 111 # read content via standard file methods 112 print(mm.readline()) # prints b"Hello Python!\n" 113 # read content via slice notation 114 print(mm[:5]) # prints b"Hello" 115 # update content using slice notation; 116 # note that new content must have same size 117 mm[6:] = b" world!\n" 118 # ... and read again using standard file methods 119 mm.seek(0) 120 print(mm.readline()) # prints b"Hello world!\n" 121 # close the map 122 mm.close() 123 124 125 :class:`~mmap.mmap` can also be used as a context manager in a :keyword:`with` 126 statement.:: 127 128 import mmap 129 130 with mmap.mmap(-1, 13) as mm: 131 mm.write(b"Hello world!") 132 133 .. versionadded:: 3.2 134 Context manager support. 135 136 137 The next example demonstrates how to create an anonymous map and exchange 138 data between the parent and child processes:: 139 140 import mmap 141 import os 142 143 mm = mmap.mmap(-1, 13) 144 mm.write(b"Hello world!") 145 146 pid = os.fork() 147 148 if pid == 0: # In a child process 149 mm.seek(0) 150 print(mm.readline()) 151 152 mm.close() 153 154 155 Memory-mapped file objects support the following methods: 156 157 .. method:: close() 158 159 Closes the mmap. Subsequent calls to other methods of the object will 160 result in a ValueError exception being raised. This will not close 161 the open file. 162 163 164 .. attribute:: closed 165 166 ``True`` if the file is closed. 167 168 .. versionadded:: 3.2 169 170 171 .. method:: find(sub[, start[, end]]) 172 173 Returns the lowest index in the object where the subsequence *sub* is 174 found, such that *sub* is contained in the range [*start*, *end*]. 175 Optional arguments *start* and *end* are interpreted as in slice notation. 176 Returns ``-1`` on failure. 177 178 .. versionchanged:: 3.5 179 Writable :term:`bytes-like object` is now accepted. 180 181 182 .. method:: flush([offset[, size]]) 183 184 Flushes changes made to the in-memory copy of a file back to disk. Without 185 use of this call there is no guarantee that changes are written back before 186 the object is destroyed. If *offset* and *size* are specified, only 187 changes to the given range of bytes will be flushed to disk; otherwise, the 188 whole extent of the mapping is flushed. 189 190 **(Windows version)** A nonzero value returned indicates success; zero 191 indicates failure. 192 193 **(Unix version)** A zero value is returned to indicate success. An 194 exception is raised when the call failed. 195 196 197 .. method:: move(dest, src, count) 198 199 Copy the *count* bytes starting at offset *src* to the destination index 200 *dest*. If the mmap was created with :const:`ACCESS_READ`, then calls to 201 move will raise a :exc:`TypeError` exception. 202 203 204 .. method:: read([n]) 205 206 Return a :class:`bytes` containing up to *n* bytes starting from the 207 current file position. If the argument is omitted, ``None`` or negative, 208 return all bytes from the current file position to the end of the 209 mapping. The file position is updated to point after the bytes that were 210 returned. 211 212 .. versionchanged:: 3.3 213 Argument can be omitted or ``None``. 214 215 .. method:: read_byte() 216 217 Returns a byte at the current file position as an integer, and advances 218 the file position by 1. 219 220 221 .. method:: readline() 222 223 Returns a single line, starting at the current file position and up to the 224 next newline. 225 226 227 .. method:: resize(newsize) 228 229 Resizes the map and the underlying file, if any. If the mmap was created 230 with :const:`ACCESS_READ` or :const:`ACCESS_COPY`, resizing the map will 231 raise a :exc:`TypeError` exception. 232 233 234 .. method:: rfind(sub[, start[, end]]) 235 236 Returns the highest index in the object where the subsequence *sub* is 237 found, such that *sub* is contained in the range [*start*, *end*]. 238 Optional arguments *start* and *end* are interpreted as in slice notation. 239 Returns ``-1`` on failure. 240 241 .. versionchanged:: 3.5 242 Writable :term:`bytes-like object` is now accepted. 243 244 245 .. method:: seek(pos[, whence]) 246 247 Set the file's current position. *whence* argument is optional and 248 defaults to ``os.SEEK_SET`` or ``0`` (absolute file positioning); other 249 values are ``os.SEEK_CUR`` or ``1`` (seek relative to the current 250 position) and ``os.SEEK_END`` or ``2`` (seek relative to the file's end). 251 252 253 .. method:: size() 254 255 Return the length of the file, which can be larger than the size of the 256 memory-mapped area. 257 258 259 .. method:: tell() 260 261 Returns the current position of the file pointer. 262 263 264 .. method:: write(bytes) 265 266 Write the bytes in *bytes* into memory at the current position of the 267 file pointer and return the number of bytes written (never less than 268 ``len(bytes)``, since if the write fails, a :exc:`ValueError` will be 269 raised). The file position is updated to point after the bytes that 270 were written. If the mmap was created with :const:`ACCESS_READ`, then 271 writing to it will raise a :exc:`TypeError` exception. 272 273 .. versionchanged:: 3.5 274 Writable :term:`bytes-like object` is now accepted. 275 276 .. versionchanged:: 3.6 277 The number of bytes written is now returned. 278 279 280 .. method:: write_byte(byte) 281 282 Write the integer *byte* into memory at the current 283 position of the file pointer; the file position is advanced by ``1``. If 284 the mmap was created with :const:`ACCESS_READ`, then writing to it will 285 raise a :exc:`TypeError` exception. 286
