1 :mod:`shutil` --- High-level file operations 2 ============================================ 3 4 .. module:: shutil 5 :synopsis: High-level file operations, including copying. 6 .. sectionauthor:: Fred L. Drake, Jr. <fdrake (a] acm.org> 7 .. partly based on the docstrings 8 9 .. index:: 10 single: file; copying 11 single: copying files 12 13 **Source code:** :source:`Lib/shutil.py` 14 15 -------------- 16 17 The :mod:`shutil` module offers a number of high-level operations on files and 18 collections of files. In particular, functions are provided which support file 19 copying and removal. For operations on individual files, see also the 20 :mod:`os` module. 21 22 .. warning:: 23 24 Even the higher-level file copying functions (:func:`shutil.copy`, 25 :func:`shutil.copy2`) can't copy all file metadata. 26 27 On POSIX platforms, this means that file owner and group are lost as well 28 as ACLs. On Mac OS, the resource fork and other metadata are not used. 29 This means that resources will be lost and file type and creator codes will 30 not be correct. On Windows, file owners, ACLs and alternate data streams 31 are not copied. 32 33 34 .. _file-operations: 35 36 Directory and files operations 37 ------------------------------ 38 39 .. function:: copyfileobj(fsrc, fdst[, length]) 40 41 Copy the contents of the file-like object *fsrc* to the file-like object *fdst*. 42 The integer *length*, if given, is the buffer size. In particular, a negative 43 *length* value means to copy the data without looping over the source data in 44 chunks; by default the data is read in chunks to avoid uncontrolled memory 45 consumption. Note that if the current file position of the *fsrc* object is not 46 0, only the contents from the current file position to the end of the file will 47 be copied. 48 49 50 .. function:: copyfile(src, dst) 51 52 Copy the contents (no metadata) of the file named *src* to a file named 53 *dst*. *dst* must be the complete target file name; look at 54 :func:`shutil.copy` for a copy that accepts a target directory path. If 55 *src* and *dst* are the same files, :exc:`Error` is raised. 56 The destination location must be writable; otherwise, an :exc:`IOError` exception 57 will be raised. If *dst* already exists, it will be replaced. Special files 58 such as character or block devices and pipes cannot be copied with this 59 function. *src* and *dst* are path names given as strings. 60 61 62 .. function:: copymode(src, dst) 63 64 Copy the permission bits from *src* to *dst*. The file contents, owner, and 65 group are unaffected. *src* and *dst* are path names given as strings. 66 67 68 .. function:: copystat(src, dst) 69 70 Copy the permission bits, last access time, last modification time, and flags 71 from *src* to *dst*. The file contents, owner, and group are unaffected. *src* 72 and *dst* are path names given as strings. 73 74 75 .. function:: copy(src, dst) 76 77 Copy the file *src* to the file or directory *dst*. If *dst* is a directory, a 78 file with the same basename as *src* is created (or overwritten) in the 79 directory specified. Permission bits are copied. *src* and *dst* are path 80 names given as strings. 81 82 83 .. function:: copy2(src, dst) 84 85 Similar to :func:`shutil.copy`, but metadata is copied as well -- in fact, 86 this is just :func:`shutil.copy` followed by :func:`copystat`. This is 87 similar to the Unix command :program:`cp -p`. 88 89 90 .. function:: ignore_patterns(\*patterns) 91 92 This factory function creates a function that can be used as a callable for 93 :func:`copytree`\'s *ignore* argument, ignoring files and directories that 94 match one of the glob-style *patterns* provided. See the example below. 95 96 .. versionadded:: 2.6 97 98 99 .. function:: copytree(src, dst, symlinks=False, ignore=None) 100 101 Recursively copy an entire directory tree rooted at *src*. The destination 102 directory, named by *dst*, must not already exist; it will be created as 103 well as missing parent directories. Permissions and times of directories 104 are copied with :func:`copystat`, individual files are copied using 105 :func:`shutil.copy2`. 106 107 If *symlinks* is true, symbolic links in the source tree are represented as 108 symbolic links in the new tree, but the metadata of the original links is NOT 109 copied; if false or omitted, the contents and metadata of the linked files 110 are copied to the new tree. 111 112 If *ignore* is given, it must be a callable that will receive as its 113 arguments the directory being visited by :func:`copytree`, and a list of its 114 contents, as returned by :func:`os.listdir`. Since :func:`copytree` is 115 called recursively, the *ignore* callable will be called once for each 116 directory that is copied. The callable must return a sequence of directory 117 and file names relative to the current directory (i.e. a subset of the items 118 in its second argument); these names will then be ignored in the copy 119 process. :func:`ignore_patterns` can be used to create such a callable that 120 ignores names based on glob-style patterns. 121 122 If exception(s) occur, an :exc:`Error` is raised with a list of reasons. 123 124 The source code for this should be considered an example rather than the 125 ultimate tool. 126 127 .. versionchanged:: 2.3 128 :exc:`Error` is raised if any exceptions occur during copying, rather than 129 printing a message. 130 131 .. versionchanged:: 2.5 132 Create intermediate directories needed to create *dst*, rather than raising an 133 error. Copy permissions and times of directories using :func:`copystat`. 134 135 .. versionchanged:: 2.6 136 Added the *ignore* argument to be able to influence what is being copied. 137 138 139 .. function:: rmtree(path[, ignore_errors[, onerror]]) 140 141 .. index:: single: directory; deleting 142 143 Delete an entire directory tree; *path* must point to a directory (but not a 144 symbolic link to a directory). If *ignore_errors* is true, errors resulting 145 from failed removals will be ignored; if false or omitted, such errors are 146 handled by calling a handler specified by *onerror* or, if that is omitted, 147 they raise an exception. 148 149 If *onerror* is provided, it must be a callable that accepts three 150 parameters: *function*, *path*, and *excinfo*. The first parameter, 151 *function*, is the function which raised the exception; it will be 152 :func:`os.path.islink`, :func:`os.listdir`, :func:`os.remove` or 153 :func:`os.rmdir`. The second parameter, *path*, will be the path name passed 154 to *function*. The third parameter, *excinfo*, will be the exception 155 information return by :func:`sys.exc_info`. Exceptions raised by *onerror* 156 will not be caught. 157 158 .. versionchanged:: 2.6 159 Explicitly check for *path* being a symbolic link and raise :exc:`OSError` 160 in that case. 161 162 163 .. function:: move(src, dst) 164 165 Recursively move a file or directory (*src*) to another location (*dst*). 166 167 If the destination is an existing directory, then *src* is moved inside that 168 directory. If the destination already exists but is not a directory, it may 169 be overwritten depending on :func:`os.rename` semantics. 170 171 If the destination is on the current filesystem, then :func:`os.rename` is 172 used. Otherwise, *src* is copied (using :func:`shutil.copy2`) to *dst* and 173 then removed. 174 175 .. versionadded:: 2.3 176 177 178 .. exception:: Error 179 180 This exception collects exceptions that are raised during a multi-file 181 operation. For :func:`copytree`, the exception argument is a list of 3-tuples 182 (*srcname*, *dstname*, *exception*). 183 184 .. versionadded:: 2.3 185 186 187 .. _copytree-example: 188 189 copytree example 190 :::::::::::::::: 191 192 This example is the implementation of the :func:`copytree` function, described 193 above, with the docstring omitted. It demonstrates many of the other functions 194 provided by this module. :: 195 196 def copytree(src, dst, symlinks=False, ignore=None): 197 names = os.listdir(src) 198 if ignore is not None: 199 ignored_names = ignore(src, names) 200 else: 201 ignored_names = set() 202 203 os.makedirs(dst) 204 errors = [] 205 for name in names: 206 if name in ignored_names: 207 continue 208 srcname = os.path.join(src, name) 209 dstname = os.path.join(dst, name) 210 try: 211 if symlinks and os.path.islink(srcname): 212 linkto = os.readlink(srcname) 213 os.symlink(linkto, dstname) 214 elif os.path.isdir(srcname): 215 copytree(srcname, dstname, symlinks, ignore) 216 else: 217 copy2(srcname, dstname) 218 # XXX What about devices, sockets etc.? 219 except (IOError, os.error) as why: 220 errors.append((srcname, dstname, str(why))) 221 # catch the Error from the recursive copytree so that we can 222 # continue with other files 223 except Error as err: 224 errors.extend(err.args[0]) 225 try: 226 copystat(src, dst) 227 except WindowsError: 228 # can't copy file access times on Windows 229 pass 230 except OSError as why: 231 errors.extend((src, dst, str(why))) 232 if errors: 233 raise Error(errors) 234 235 Another example that uses the :func:`ignore_patterns` helper:: 236 237 from shutil import copytree, ignore_patterns 238 239 copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*')) 240 241 This will copy everything except ``.pyc`` files and files or directories whose 242 name starts with ``tmp``. 243 244 Another example that uses the *ignore* argument to add a logging call:: 245 246 from shutil import copytree 247 import logging 248 249 def _logpath(path, names): 250 logging.info('Working in %s' % path) 251 return [] # nothing will be ignored 252 253 copytree(source, destination, ignore=_logpath) 254 255 256 .. _archiving-operations: 257 258 Archiving operations 259 -------------------- 260 261 High-level utilities to create and read compressed and archived files are also 262 provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules. 263 264 .. function:: make_archive(base_name, format, [root_dir, [base_dir, [verbose, [dry_run, [owner, [group, [logger]]]]]]]) 265 266 Create an archive file (eg. zip or tar) and returns its name. 267 268 *base_name* is the name of the file to create, including the path, minus 269 any format-specific extension. *format* is the archive format: one of 270 "zip", "tar", "bztar" or "gztar". 271 272 *root_dir* is a directory that will be the root directory of the 273 archive; ie. we typically chdir into *root_dir* before creating the 274 archive. 275 276 *base_dir* is the directory where we start archiving from; 277 ie. *base_dir* will be the common prefix of all files and 278 directories in the archive. 279 280 *root_dir* and *base_dir* both default to the current directory. 281 282 *owner* and *group* are used when creating a tar archive. By default, 283 uses the current owner and group. 284 285 *logger* must be an object compatible with :pep:`282`, usually an instance of 286 :class:`logging.Logger`. 287 288 .. versionadded:: 2.7 289 290 291 .. function:: get_archive_formats() 292 293 Return a list of supported formats for archiving. 294 Each element of the returned sequence is a tuple ``(name, description)``. 295 296 By default :mod:`shutil` provides these formats: 297 298 - *gztar*: gzip'ed tar-file 299 - *bztar*: bzip2'ed tar-file 300 - *tar*: uncompressed tar file 301 - *zip*: ZIP file 302 303 You can register new formats or provide your own archiver for any existing 304 formats, by using :func:`register_archive_format`. 305 306 .. versionadded:: 2.7 307 308 309 .. function:: register_archive_format(name, function, [extra_args, [description]]) 310 311 Register an archiver for the format *name*. *function* is a callable that 312 will be used to invoke the archiver. 313 314 If given, *extra_args* is a sequence of ``(name, value)`` that will be 315 used as extra keywords arguments when the archiver callable is used. 316 317 *description* is used by :func:`get_archive_formats` which returns the 318 list of archivers. Defaults to an empty list. 319 320 .. versionadded:: 2.7 321 322 323 .. function:: unregister_archive_format(name) 324 325 Remove the archive format *name* from the list of supported formats. 326 327 .. versionadded:: 2.7 328 329 330 .. _archiving-example: 331 332 Archiving example 333 ::::::::::::::::: 334 335 In this example, we create a gzip'ed tar-file archive containing all files 336 found in the :file:`.ssh` directory of the user:: 337 338 >>> from shutil import make_archive 339 >>> import os 340 >>> archive_name = os.path.expanduser(os.path.join('~', 'myarchive')) 341 >>> root_dir = os.path.expanduser(os.path.join('~', '.ssh')) 342 >>> make_archive(archive_name, 'gztar', root_dir) 343 '/Users/tarek/myarchive.tar.gz' 344 345 The resulting archive contains: 346 347 .. code-block:: shell-session 348 349 $ tar -tzvf /Users/tarek/myarchive.tar.gz 350 drwx------ tarek/staff 0 2010-02-01 16:23:40 ./ 351 -rw-r--r-- tarek/staff 609 2008-06-09 13:26:54 ./authorized_keys 352 -rwxr-xr-x tarek/staff 65 2008-06-09 13:26:54 ./config 353 -rwx------ tarek/staff 668 2008-06-09 13:26:54 ./id_dsa 354 -rwxr-xr-x tarek/staff 609 2008-06-09 13:26:54 ./id_dsa.pub 355 -rw------- tarek/staff 1675 2008-06-09 13:26:54 ./id_rsa 356 -rw-r--r-- tarek/staff 397 2008-06-09 13:26:54 ./id_rsa.pub 357 -rw-r--r-- tarek/staff 37192 2010-02-06 18:23:10 ./known_hosts 358
