1 :mod:`shelve` --- Python object persistence 2 =========================================== 3 4 .. module:: shelve 5 :synopsis: Python object persistence. 6 7 8 .. index:: module: pickle 9 10 **Source code:** :source:`Lib/shelve.py` 11 12 -------------- 13 14 A "shelf" is a persistent, dictionary-like object. The difference with "dbm" 15 databases is that the values (not the keys!) in a shelf can be essentially 16 arbitrary Python objects --- anything that the :mod:`pickle` module can handle. 17 This includes most class instances, recursive data types, and objects containing 18 lots of shared sub-objects. The keys are ordinary strings. 19 20 21 .. function:: open(filename, flag='c', protocol=None, writeback=False) 22 23 Open a persistent dictionary. The filename specified is the base filename for 24 the underlying database. As a side-effect, an extension may be added to the 25 filename and more than one file may be created. By default, the underlying 26 database file is opened for reading and writing. The optional *flag* parameter 27 has the same interpretation as the *flag* parameter of :func:`anydbm.open`. 28 29 By default, version 0 pickles are used to serialize values. The version of the 30 pickle protocol can be specified with the *protocol* parameter. 31 32 .. versionchanged:: 2.3 33 The *protocol* parameter was added. 34 35 Because of Python semantics, a shelf cannot know when a mutable 36 persistent-dictionary entry is modified. By default modified objects are 37 written *only* when assigned to the shelf (see :ref:`shelve-example`). If the 38 optional *writeback* parameter is set to ``True``, all entries accessed are also 39 cached in memory, and written back on :meth:`~Shelf.sync` and 40 :meth:`~Shelf.close`; this can make it handier to mutate mutable entries in 41 the persistent dictionary, but, if many entries are accessed, it can consume 42 vast amounts of memory for the cache, and it can make the close operation 43 very slow since all accessed entries are written back (there is no way to 44 determine which accessed entries are mutable, nor which ones were actually 45 mutated). 46 47 Like file objects, shelve objects should be closed explicitly to ensure 48 that the persistent data is flushed to disk. 49 50 .. warning:: 51 52 Because the :mod:`shelve` module is backed by :mod:`pickle`, it is insecure 53 to load a shelf from an untrusted source. Like with pickle, loading a shelf 54 can execute arbitrary code. 55 56 Shelf objects support most of the methods supported by dictionaries. This 57 eases the transition from dictionary based scripts to those requiring 58 persistent storage. 59 60 Note, the Python 3 transition methods (:meth:`~dict.viewkeys`, 61 :meth:`~dict.viewvalues`, and :meth:`~dict.viewitems`) are not supported. 62 63 Two additional methods are supported: 64 65 .. method:: Shelf.sync() 66 67 Write back all entries in the cache if the shelf was opened with *writeback* 68 set to :const:`True`. Also empty the cache and synchronize the persistent 69 dictionary on disk, if feasible. This is called automatically when the shelf 70 is closed with :meth:`close`. 71 72 .. method:: Shelf.close() 73 74 Synchronize and close the persistent *dict* object. Operations on a closed 75 shelf will fail with a :exc:`ValueError`. 76 77 78 .. seealso:: 79 80 `Persistent dictionary recipe <https://code.activestate.com/recipes/576642/>`_ 81 with widely supported storage formats and having the speed of native 82 dictionaries. 83 84 85 Restrictions 86 ------------ 87 88 .. index:: 89 module: dbm 90 module: gdbm 91 module: bsddb 92 93 * The choice of which database package will be used (such as :mod:`dbm`, 94 :mod:`gdbm` or :mod:`bsddb`) depends on which interface is available. Therefore 95 it is not safe to open the database directly using :mod:`dbm`. The database is 96 also (unfortunately) subject to the limitations of :mod:`dbm`, if it is used --- 97 this means that (the pickled representation of) the objects stored in the 98 database should be fairly small, and in rare cases key collisions may cause the 99 database to refuse updates. 100 101 * The :mod:`shelve` module does not support *concurrent* read/write access to 102 shelved objects. (Multiple simultaneous read accesses are safe.) When a 103 program has a shelf open for writing, no other program should have it open for 104 reading or writing. Unix file locking can be used to solve this, but this 105 differs across Unix versions and requires knowledge about the database 106 implementation used. 107 108 109 .. class:: Shelf(dict, protocol=None, writeback=False) 110 111 A subclass of :class:`UserDict.DictMixin` which stores pickled values in the 112 *dict* object. 113 114 By default, version 0 pickles are used to serialize values. The version of the 115 pickle protocol can be specified with the *protocol* parameter. See the 116 :mod:`pickle` documentation for a discussion of the pickle protocols. 117 118 .. versionchanged:: 2.3 119 The *protocol* parameter was added. 120 121 If the *writeback* parameter is ``True``, the object will hold a cache of all 122 entries accessed and write them back to the *dict* at sync and close times. 123 This allows natural operations on mutable entries, but can consume much more 124 memory and make sync and close take a long time. 125 126 127 .. class:: BsdDbShelf(dict, protocol=None, writeback=False) 128 129 A subclass of :class:`Shelf` which exposes :meth:`first`, :meth:`!next`, 130 :meth:`previous`, :meth:`last` and :meth:`set_location` which are available in 131 the :mod:`bsddb` module but not in other database modules. The *dict* object 132 passed to the constructor must support those methods. This is generally 133 accomplished by calling one of :func:`bsddb.hashopen`, :func:`bsddb.btopen` or 134 :func:`bsddb.rnopen`. The optional *protocol* and *writeback* parameters have 135 the same interpretation as for the :class:`Shelf` class. 136 137 138 .. class:: DbfilenameShelf(filename, flag='c', protocol=None, writeback=False) 139 140 A subclass of :class:`Shelf` which accepts a *filename* instead of a dict-like 141 object. The underlying file will be opened using :func:`anydbm.open`. By 142 default, the file will be created and opened for both read and write. The 143 optional *flag* parameter has the same interpretation as for the :func:`.open` 144 function. The optional *protocol* and *writeback* parameters have the same 145 interpretation as for the :class:`Shelf` class. 146 147 148 .. _shelve-example: 149 150 Example 151 ------- 152 153 To summarize the interface (``key`` is a string, ``data`` is an arbitrary 154 object):: 155 156 import shelve 157 158 d = shelve.open(filename) # open -- file may get suffix added by low-level 159 # library 160 161 d[key] = data # store data at key (overwrites old data if 162 # using an existing key) 163 data = d[key] # retrieve a COPY of data at key (raise KeyError if no 164 # such key) 165 del d[key] # delete data stored at key (raises KeyError 166 # if no such key) 167 flag = d.has_key(key) # true if the key exists 168 klist = d.keys() # a list of all existing keys (slow!) 169 170 # as d was opened WITHOUT writeback=True, beware: 171 d['xx'] = range(4) # this works as expected, but... 172 d['xx'].append(5) # *this doesn't!* -- d['xx'] is STILL range(4)! 173 174 # having opened d without writeback=True, you need to code carefully: 175 temp = d['xx'] # extracts the copy 176 temp.append(5) # mutates the copy 177 d['xx'] = temp # stores the copy right back, to persist it 178 179 # or, d=shelve.open(filename,writeback=True) would let you just code 180 # d['xx'].append(5) and have it work as expected, BUT it would also 181 # consume more memory and make the d.close() operation slower. 182 183 d.close() # close it 184 185 186 .. seealso:: 187 188 Module :mod:`anydbm` 189 Generic interface to ``dbm``\ -style databases. 190 191 Module :mod:`bsddb` 192 BSD ``db`` database interface. 193 194 Module :mod:`dbhash` 195 Thin layer around the :mod:`bsddb` which provides an :func:`~dbhash.open` 196 function like the other database modules. 197 198 Module :mod:`dbm` 199 Standard Unix database interface. 200 201 Module :mod:`dumbdbm` 202 Portable implementation of the ``dbm`` interface. 203 204 Module :mod:`gdbm` 205 GNU database interface, based on the ``dbm`` interface. 206 207 Module :mod:`pickle` 208 Object serialization used by :mod:`shelve`. 209 210 Module :mod:`cPickle` 211 High-performance version of :mod:`pickle`. 212 213
