Lines Matching defs:in
10 ** This file is all you need to compile SQLite. To use SQLite in other
17 ** language. The code for the "sqlite3" command-line shell is also in a
29 ** The author disclaims copyright to this source code. In place of
46 ** compiler warnings due to subsequent content in this file and other files
49 /************** Include msvc.h in the middle of sqliteInt.h ******************/
54 ** The author disclaims copyright to this source code. In place of
89 /************** Continuing where we left off in sqliteInt.h ******************/
94 /************** Include vxworks.h in the middle of sqliteInt.h ***************/
99 ** The author disclaims copyright to this source code. In place of
127 /************** Continuing where we left off in sqliteInt.h ******************/
136 ** code in all source files.
142 ** without this option, LFS is enable. But LFS does not exist in the kernel
143 ** in Red Hat 6.0, so the code won't work. Hence, for maximum binary
146 ** The previous paragraph was written in 2005. (This paragraph is written
149 ** lack LFS in which case the SQLITE_DISABLE_LFS macro might still be useful.
200 ** library in use (e.g. for Windows XP).
210 ** first in QNX. Also, the _USE_32BIT_TIME_T macro must appear first for
213 /************** Include sqlite3.h in the middle of sqliteInt.h ***************/
218 ** The author disclaims copyright to this source code. In place of
228 ** or constant definition does not appear in this file, then it is
232 ** Some of the definitions that are in this file are marked as
236 ** if experience from use "in the wild" suggest such changes are prudent.
239 ** from comments in this file. This file is the authoritative source
242 ** The name of this file under configuration management is "sqlite.h.in".
276 ** These no-op macros are used in front of interfaces to mark those
280 ** experimental interfaces are subject to change in point releases.
304 ** ^(The [SQLITE_VERSION] C preprocessor macro in the sqlite3.h header
305 ** evaluates to a string literal that is the SQLite version in the
310 ** numbers used in [SQLITE_VERSION].)^
316 ** Since version 3.6.18, SQLite source code has been stored in the
319 ** a string which identifies a particular check-in of SQLite
321 ** string contains the date and time of the check-in (UTC) and an SHA1
339 ** programmers might include assert() statements in their application to
340 ** verify that values returned by these interfaces match the macros in
353 ** function is provided for use in DLLs since DLL users usually do not have
425 ** sqlite3_config(). In other words, the return value from sqlite3_threadsafe()
503 ** host languages that are garbage collected, and where the order in which
548 ** in the context of the [database connection] passed in as its 1st
566 ** occur, then sqlite3_exec() sets the pointer in its 5th parameter to
574 ** number of columns in the result. ^The 3rd argument to the sqlite3_exec()
612 ** here in order to indicate success or failure.
614 ** New error codes may be added in future versions of SQLite.
621 #define SQLITE_INTERNAL 2 /* Internal logic error in SQLite */
625 #define SQLITE_LOCKED 6 /* A table in the database is locked */
631 #define SQLITE_NOTFOUND 12 /* Unknown opcode in sqlite3_file_control() */
656 ** In its default configuration, SQLite API routines return one of 30 integer
659 ** much information about problems as programmers might like. In an effort to
726 ** These bit values are intended for use in the
728 ** in the 4th parameter to the [sqlite3_vfs.xOpen] method.
764 ** mean that writes of blocks that are nnn bytes in size and
770 ** information is written to disk in the same order as calls
772 ** after reboot following a crash or power loss, the only bytes in a
842 ** An [sqlite3_file] object represents an open file in the
874 ** flag may be ORed in to indicate that only the data of the file
887 ** either in this process or in some other process, is holding a RESERVED,
895 ** point to a structure that may contain arguments or space in which to
910 ** other bytes in the file. The xDeviceCharacteristics()
930 ** mean that writes of blocks that are nnn bytes in size and
936 ** information is written to disk in the same order as calls
940 ** in the unread portions of the buffer with zeros. A VFS that
969 /* Additional methods may be added in future releases */
995 ** file space based on this hint in order to help writes to the database
1000 ** extends and truncates the database file in chunks of a size specified
1003 ** for the nominated database. Allocating database file space in large
1014 ** No longer in use.
1021 ** [PRAGMA synchronous | PRAGMA synchronous=OFF] it is invoked in place
1022 ** of the xSync method. In most cases, the pointer argument passed with
1041 ** windows [VFS] in order to provide robustness in the presence of
1064 ** in order for the database to be readable. The fourth parameter to
1088 ** all [VFSes] in the VFS stack. The names are of all VFS shims and the
1090 ** [sqlite3_malloc()] and the result is stored in the char* variable
1095 ** pointer in case this file-control is not implemented. This file-control
1103 ** pointers to strings (char**) in which the second element of the array
1121 ** it is able to override built-in [PRAGMA] statements.
1126 ** shortly after it is opened in order to provide a custom VFS with access
1129 ** to a function of type (int (*)(void *)). In order to invoke the connections
1130 ** busy-handler, this function should be invoked with the second (void *) in
1148 ** is an advisory maximum number of bytes in the file to memory map. The
1151 ** can be queried by passing in a pointer to a negative number. This
1158 ** The argument is a zero-terminated string. Higher layers in the
1178 ** circumstances in order to fix a problem with priority inversion.
1240 ** in the name of the object stands for "virtual file system". See
1243 ** The value of the iVersion field is initially 1 but may be larger in
1246 ** of the sqlite3_vfs object changes in the transaction between
1252 ** a pathname in this VFS.
1257 ** in a thread-safe way. The [sqlite3_vfs_find()] interface
1261 ** The pNext field is the only field in the sqlite3_vfs
1287 ** The flags argument to xOpen() includes all bits set in
1292 ** include [SQLITE_OPEN_READONLY]. Other bits in *pOutFlags may be set.
1314 ** file will be doing page-aligned sector reads and writes in a random
1329 ** ^The [SQLITE_OPEN_EXCLUSIVE] flag is always used in conjunction
1341 ** allocate the structure; it should just fill it in. Note that
1364 ** included in the VFS structure for completeness.
1373 ** Day Number multiplied by 86400000 (the number of milliseconds in
1415 ** The methods above are in version 1 of the sqlite_vfs object
1416 ** definition. Those that follow are added in version 2 or later
1420 ** The methods above are in versions 1 and 2 of the sqlite_vfs object.
1427 ** The methods above are in versions 1 through 3 of the sqlite_vfs object.
1428 ** New fields may be appended in figure versions. The iVersion
1443 ** (in other words, if files can be added, removed, and renamed within
1446 ** [temp_store_directory pragma], though this could change in a future
1450 ** currently unused, though it might be used in a future release of
1501 ** These routines are designed to aid in process initialization and
1543 ** of SQLite may require this. In other words, the behavior exhibited
1545 ** default behavior in some future release of SQLite.
1579 ** changes to SQLite in order to tune SQLite to the specific needs of
1600 ** in the first argument.
1615 ** [database connection] (specified in the first argument).
1633 ** This object is used in only one place in the SQLite interface.
1643 ** Note that SQLite comes with several [built-in memory allocators]
1647 ** also used during testing of SQLite in order to specify an alternative
1648 ** memory allocator that simulates memory out-of-memory conditions in
1665 ** Every memory allocation request coming in through [sqlite3_malloc()]
1709 ** New configuration options may be added in future releases of SQLite.
1719 ** [threading mode] to Single-thread. In other words, it disables
1730 ** [threading mode] to Multi-thread. In other words, it disables
1734 ** are enabled so that SQLite will be safe to use in a multi-threaded
1744 ** [threading mode] to Serialized. In other words, this option enables
1747 ** In this mode (which is the default when SQLite is compiled with
1751 ** same [prepared statement] in different threads at the same time.
1762 ** alternative low-level memory allocation routines to be used in place of
1788 ** compiled with [SQLITE_DEFAULT_MEMSTATUS]=0 in which case memory
1811 ** fragmentation in low-memory embedded systems.
1849 ** the number of bytes in the memory buffer, and the minimum allocation size.
1864 ** in place the mutex routines built into SQLite.)^ ^SQLite makes a copy of
1923 ** In a multi-threaded application, the application-defined logger
1942 ** argument which is interpreted as a boolean in order to enable or disable
1943 ** the use of covering indices for full table scans in the query optimizer.
1965 ** in three separate circumstances, identified by the value passed as the
1972 ** third parameter is passed NULL In this case. An example of using this
1973 ** configuration option can be seen in the "test_sqllog.c" source file in
2001 ** bytes per page required for each page in [SQLITE_CONFIG_PAGECACHE].
2049 ** New configuration options may be added in future releases of SQLite.
2063 ** may be NULL in which case SQLite will allocate the
2066 ** slots. The size of the buffer in the first argument must be greater than
2072 ** connection is not currently using lookaside memory, or in other words
2076 ** memory is in use leaves the configuration unchanged and returns
2086 ** following this call. The second parameter may be a NULL pointer, in
2096 ** following this call. The second parameter may be a NULL pointer, in
2120 ** ^Each entry in most SQLite tables (except for [WITHOUT ROWID] tables)
2231 ** ^Changes made as part of [foreign key actions] are included in the
2251 ** called in response to a user action such as pressing "Cancel"
2269 ** ^The sqlite3_interrupt(D) call is in effect until all currently running
2296 ** independent tokens (they are part of the token in which they are
2316 ** UTF-16 string in native byte order.
2350 ** handler could result in a deadlock, it will go ahead and return [SQLITE_BUSY]
2373 ** database connection that invoked the busy handler. In other words,
2375 ** result in undefined behavior.
2422 ** There are (N+1)*M elements in the array. The first M pointers point
2425 ** in NULL pointers. All other values are in their UTF-8 zero-terminated
2445 ** in an array names azResult. Then azResult holds this content:
2459 ** semicolon-separated SQL statements in the zero-terminated UTF-8
2461 ** pointer given in its 3rd parameter.
2464 ** it must pass the result table pointer to sqlite3_free_table() in order to
2473 ** interface defined here. As a consequence, errors that occur in the
2475 ** reflected in subsequent calls to [sqlite3_errcode()] or
2528 ** All of the usual printf() formatting options apply. In addition, there
2531 ** ^(The %q option works like %s in that it substitutes a nul-terminated
2543 ** One can use this text in an SQL statement as follows:
2551 ** Because the %q format string is used, the '\'' character in zText
2569 ** the outside of the total string. Additionally, if the parameter in the
2579 ** The code above will render a correct SQL statement in the zSQL
2601 ** internal memory allocation needs. "Core" in the previous sentence
2606 ** of memory at least N bytes in length, where N is the parameter.
2622 ** memory might result in a segmentation fault or other severe error.
2636 ** of at least N bytes in size or NULL if insufficient memory is available.
2649 ** sqlite3_msize(X) returns the size of that memory allocation in bytes.
2663 ** In SQLite version 3.5.0 and 3.5.1, it was possible to define
2664 ** the SQLITE_OMIT_MEMORY_ALLOCATION which would cause the built-in
2666 ** is no longer provided. Only built-in memory allocators can be used.
2697 ** routines, which form the built-in memory allocation subsystem.
2705 ** added by SQLite in its implementation of [sqlite3_malloc()],
2724 ** the build-in random() and randomblob() SQL functions. This interface allows
2746 ** [database connection], supplied in the first argument.
2777 ** a NULL value in place of the table column that would have
2792 ** database. An authorizer could then be put in place while the
2799 ** in addition to using an authorizer.
2801 ** ^(Only a single authorizer can be in place on a database connection
2809 ** database connections for the meaning of "modify" in this paragraph.
2814 ** correct authorizer callback remains in place during the [sqlite3_step()].
2818 ** performed during statement evaluation in [sqlite3_step()], unless
2819 ** as stated in the previous paragraph, sqlite3_step() invokes
2832 ** return either [SQLITE_OK] or one of these two constants in order
2914 ** the length of [bound parameter] expansion in the output of sqlite3_trace().
2920 ** time is in units of nanoseconds, however the current implementation
2922 ** digits in the time are meaningless. Future versions of SQLite
2925 ** subject to change in future versions of SQLite.
2960 ** database connections for the meaning of "modify" in this paragraph.
2971 ** sqlite3_open() and sqlite3_open_v2() and as UTF-16 in the native byte
2973 ** returned in *ppDb, even if an error occurs. The only exception is that
2984 ** created using sqlite3_open16() will be UTF-16 in the native byte order.
3000 ** <dd>The database is opened in read-only mode. If the database does not
3005 ** only if the file is write protected by the operating system. In either
3020 ** opens in the multi-thread [threading mode] as long as the single-thread
3023 ** in the serialized [threading mode] unless single-thread was
3029 ** participate in [shared cache mode] even if it is enabled.
3036 ** ^If the filename is ":memory:", then a private, temporary in-memory database
3037 ** is created for the connection. ^This in-memory database will vanish when
3048 ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3>
3053 ** set in the fourth argument to sqlite3_open_v2(), or if it has
3078 ** SQLite and its built-in [VFSes] interpret the
3094 ** access, just as if the [SQLITE_OPEN_READONLY] flag had been set in the
3100 ** set to "memory" then a pure [in-memory database] that never reads
3103 ** the flags passed in the third parameter to sqlite3_open_v2().
3107 ** SQLITE_OPEN_SHAREDCACHE bit in the flags argument passed to
3110 ** ^If sqlite3_open_v2() is used and the "cache" parameter is present in
3119 ** which if set disables file locking in rollback journal modes. This
3131 ** property on a database file that does in fact change can result
3132 ** in incorrect query results and/or [SQLITE_CORRUPT] errors.
3137 ** ^Specifying an unknown parameter in the query component of a URI is not an
3147 ** Open the file "data.db" in the current directory.
3157 ** C:. Note that the %20 escaping in this example is not strictly
3159 ** in URI filenames.
3161 ** Open file "data.db" in the current directory for read-only access.
3166 ** that uses dot-files in place of posix advisory locking.
3227 ** value of query parameter P is one of "yes", "true", or "on" in any
3230 ** query parameter P is one of "no", "false", or "off" in any case or
3278 ** When the serialized [threading mode] is in use, it might be the
3279 ** case that a second error occurs on a separate thread in between
3289 ** was invoked incorrectly by the application. In that case, the
3340 ** (The "_LIMIT_" in the name is changed to "_MAX_".))^
3349 ** Run-time limits are intended for use in applications that manage
3362 ** New run-time limit categories may be added in future releases.
3373 ** Additional information is available at [limits | Limits in SQLite].
3377 ** <dd>The maximum size of any string or BLOB or table row, in bytes.<dd>)^
3380 ** <dd>The maximum length of an SQL statement, in bytes.</dd>)^
3383 ** <dd>The maximum number of columns in a table definition or in the
3384 ** result set of a [SELECT] or the maximum number of columns in an index
3385 ** or in an ORDER BY or GROUP BY clause.</dd>)^
3391 ** <dd>The maximum number of terms in a compound SELECT statement.</dd>)^
3394 ** <dd>The maximum number of instructions in a virtual machine program
3396 ** enforced, though that might be added in some future release of
3412 ** <dd>The maximum index number of any [parameter] in an SQL statement.)^
3459 ** is the number of bytes in the input string <i>including</i>
3463 ** past the end of the first SQL statement in zSql. These routines only
3464 ** compile the first statement in zSql, so *pzTail is left pointing to
3481 ** ^In the "v2" interfaces, the prepared statement
3484 ** behave differently in three ways:
3499 ** in order to find the underlying cause of the problem. With the "v2" prepare
3504 ** ^If the specific value bound to [parameter | host parameter] in the
3519 int nByte, /* Maximum length of zSql in bytes. */
3526 int nByte, /* Maximum length of zSql in bytes. */
3533 int nByte, /* Maximum length of zSql in bytes. */
3540 int nByte, /* Maximum length of zSql in bytes. */
3600 ** This interface can be used in combination [sqlite3_next_stmt()]
3602 ** connection that are in need of being reset. This can be used,
3603 ** for example, in diagnostic routines to search for prepared
3613 ** that can be stored in a database table. SQLite uses dynamic typing
3614 ** for the values it stores. ^Values stored in sqlite3_value objects
3630 ** or if SQLite is run in one of reduced mutex modes
3652 ** The context in which an SQL function executes is stored in an
3669 ** ^(In the SQL statement text input to [sqlite3_prepare_v2()] and its variants,
3681 ** In the templates above, NNN represents an integer literal,
3705 ** ^(In those routines that have a fourth argument, its value is the
3706 ** number of bytes in the parameter. To be clear: the value is the
3707 ** number of <u>bytes</u> in the value, not the number of characters.)^
3728 ** information is in static, unmanaged space and does not need to be freed.
3735 ** to specify the encoding of the text in the third parameter. If
3747 ** ^A negative value for the zeroblob results in a zero-length BLOB.
3790 ** in a [prepared statement]. SQL parameters are tokens of the
3798 ** there may be gaps in the list.)^
3811 ** the name of the N-th [SQL parameter] in the [prepared statement] P.
3815 ** In other words, the initial ":" or "$" or "@" or "?"
3824 ** always in UTF-8 encoding even if the named parameter was
3825 ** originally specified as UTF-16 in [sqlite3_prepare16()] or
3842 ** name must be given in UTF-8 even if the original statement
3862 ** CAPI3REF: Number Of Columns In A Result Set
3865 ** ^Return the number of columns in the result set returned by the
3874 ** CAPI3REF: Column Names In A Result Set
3878 ** in the result set of a [SELECT] statement. ^The sqlite3_column_name()
3904 ** CAPI3REF: Source Of Data In A Query Result
3908 ** table column that is the origin of a particular result column in
3918 ** again in a different encoding.
3981 ** data stored in that column is of the declared type. SQLite is
4005 ** ^In
4033 ** [prepared statement]. ^In the "v2" interface,
4041 ** more threads at the same moment in time.
4047 ** [sqlite3_reset()] would result in an [SQLITE_MISUSE] return from
4049 ** calling [sqlite3_reset()] automatically in this circumstance rather
4055 ** <b>Goofy Interface Alert:</b> In the legacy interface, the sqlite3_step()
4058 ** [sqlite3_reset()] or [sqlite3_finalize()] in order to find one of the
4070 ** CAPI3REF: Number of columns in a result set
4073 ** ^The sqlite3_data_count(P) interface returns the number of columns in the
4082 ** [SQLITE_ROW], except in the case of the [PRAGMA incremental_vacuum]
4094 ** ^(Every value in SQLite has one of five fundamental datatypes:
4106 ** Note that the SQLITE_TEXT constant was also used in SQLite version 2
4128 ** result row of a query. ^In every case the first argument is a pointer
4133 ** ^The number of columns in the result can be determined using
4159 ** routine returns the number of bytes in that BLOB or string.
4164 ** the number of bytes in that string.
4168 ** routine returns the number of bytes in that BLOB or string.
4173 ** the number of bytes in that string.
4180 ** bytes in the string, not the number of characters.
4187 ** [unprotected sqlite3_value] object. In a multithreaded environment,
4191 ** [sqlite3_column_value()] is used in any other way, including calls
4228 ** in the following cases:
4242 ** ^Conversions between UTF-16be and UTF-16le are always done in place and do
4245 ** of conversion are done in place when it is possible, but sometimes they
4246 ** are not possible and in those cases prior pointers are invalidated.
4249 ** in one of the following ways:
4257 ** In other words, you should call sqlite3_column_text(),
4309 ** The application must finalize every [prepared statement] in order to avoid
4312 ** statement after it has been finalized can result in undefined and
4365 ** redefined. ^The length of the name is limited to 255 bytes in a UTF-8
4367 ** length limit is in UTF-8 bytes, not characters nor UTF-16 bytes.
4369 ** will result in [SQLITE_MISUSE] being returned.
4395 ** deterministic. The built-in [random()] SQL function is an example of a
4425 ** the implementation that most closely matches the way in which the
4435 ** ^Built-in functions may be overloaded by new application-defined functions.
4440 ** statement in which the function is running.
4501 ** These functions are [deprecated]. In order to maintain
4535 ** object results in undefined behavior.
4542 ** in the native byte-order of the host machine. ^The
4549 ** such a conversion is possible without loss of information (in other
4626 ** In those cases, sqlite3_aggregate_context() might be called for the
4635 ** value of N in subsequent call to sqlite3_aggregate_context() within
4638 ** N=0 in calls to sqlite3_aggregate_context(C,N) so that no
4649 ** This routine must be called from the same thread in which
4664 ** This routine must be called from the same thread in which
4689 ** of where this might be useful is in a regular-expression matching
4718 ** Note the last bullet in particular. The destructor X in
4725 ** ^(In practice, metadata is preserved between function calls for
4729 ** These routines must be called from the same thread in which
4739 ** These are special values for the destructor that is passed in as the
4743 ** SQLITE_TRANSIENT value means that the content will likely change in
4747 ** The typedef is necessary to work around problems in certain
4764 ** functions used to bind values to host parameters in prepared statements.
4774 ** a BLOB containing all zero bytes and N bytes in size.
4786 ** interprets the string from sqlite3_result_error16() as UTF-16 in native
4798 ** returned by SQLite as a result of an error in a function. ^By default,
4810 ** value given in the 2nd argument.
4813 ** value given in the 2nd argument.
4824 ** application-defined function to be a text string in an encoding
4838 ** in the string at a byte offset that is less than the value of the 3rd
4847 ** assumes that the text or BLOB result is in constant space and does not
4859 ** so that the [sqlite3_value] specified in the parameter may change or
4899 ** of the subtype T are preserved in current versions of SQLite;
4902 ** in future releases of SQLite.
4915 ** and a UTF-16 string in native byte order for sqlite3_create_collation16().
4946 ** application data pointer and with two strings in the encoding specified
5020 ** encoded in UTF-8. ^If sqlite3_collation_needed16() is used,
5021 ** the names are passed as UTF-16 in machine native byte order.
5052 ** The code to implement this API is not available in the public release
5070 ** The code to implement this API is not available in the public release
5106 ** for at least a number of milliseconds specified in its parameter.
5117 ** in the previous paragraphs.
5126 ** created by SQLite when using a built-in [sqlite3_vfs | VFS]
5127 ** will be placed in that directory.)^ ^If this variable
5136 ** be avoided in new projects.
5138 ** It is not safe to read or modify this variable in more than one
5140 ** if a [database connection] is being used at the same time in a separate
5150 ** that this variable points to is held in memory obtained from
5185 ** SQLite when using a built-in windows [sqlite3_vfs | VFS] will be assumed
5193 ** open can result in a corrupt database.
5195 ** It is not safe to read or modify this variable in more than one
5197 ** if a [database connection] is being used at the same time in a separate
5207 ** that this variable points to is held in memory obtained from
5222 ** zero if the given database connection is or is not in autocommit mode,
5249 ** create the statement in the first place.
5260 ** connection D, or if database N is a temporary or in-memory database, then
5264 ** xFullPathname method of the [VFS]. ^In other words, the filename
5290 ** The [database connection] pointer D in a call to
5292 ** connection and in particular must not be a NULL pointer.
5322 ** or rollback hook in the first place.
5325 ** the database connections for the meaning of "modify" in this paragraph.
5352 ** to be invoked whenever a row is updated, inserted or deleted in
5358 ** row is updated, inserted or deleted in a rowid table.
5367 ** ^In the case of an update, this is the [rowid] after the update takes place.
5373 ** ^In the current implementation, the update hook
5377 ** The exceptions defined in this paragraph might change in a future
5385 ** database connections for the meaning of "modify" in this paragraph.
5410 ** This is a change as of SQLite version 3.5.0. In prior versions of SQLite,
5416 ** that was in effect at the time they were opened.)^
5421 ** ^Shared cache is disabled by default. But this might change in
5459 ** [sqlite3_release_memory()] interface, this interface is in effect even
5473 ** limit by reducing the number of pages held in the page cache
5477 ** an [SQLITE_NOMEM] error. In other words, the soft heap limit
5481 ** the soft heap limit prior to the call, or negative in the case of an
5489 ** ^(The soft heap limit is not enforced in the current implementation
5510 ** the page cache is the predominate memory user in SQLite, most
5515 ** changes in future releases of SQLite.
5536 ** information about column C of table T in database D
5538 ** interface returns SQLITE_OK and fills in the non-NULL pointers in
5559 ** NULL, in which case the corresponding element of metadata is omitted.
5617 ** [SQLite extension] library contained in the file zFile. If
5625 ** ^(zProc may be 0, in which case SQLite will try to come up with an
5629 ** characters in the filename from the last "/" to the first following
5635 ** fill *pzErrMsg with error message text stored in memory
5656 ** ^So as not to open security holes in older applications that are
5728 ** to be experimental. The interface might change in incompatible ways.
5751 ** ^A virtual table module is created by filling in a persistent
5787 /* The methods above are in version 1 of the sqlite_module object. Those
5810 ** stored in aConstraint[].op using one of the
5812 ** ^(The index of the column is stored in
5818 ** and makes other simplifications to the WHERE clause in an attempt to
5823 ** ^Information about the ORDER BY clause is stored in aOrderBy[].
5829 ** and becomes the argvIndex-th entry in argv. ^(If aConstraintUsage[].omit
5838 ** ^The orderByConsumed means that output from [xFilter]/[xNext] will occur in
5860 ** any database changes. In other words, if the xUpdate() returns
5879 int nConstraint; /* Number of entries in aConstraint */
5886 int nOrderBy; /* Number of terms in the ORDER BY clause */
5901 /* Fields below are only available in SQLite 3.8.2 and later */
5903 /* Fields below are only available in SQLite 3.9.0 and later */
5917 ** an operator that is part of a constraint term in the wHERE clause of
6006 ** This superclass exists in order to define fields of the cursor that
6031 ** must exist in order to be overloaded.)^
6046 ** to be experimental. The interface might change in incompatible ways.
6063 ** ^The [sqlite3_blob_bytes()] interface returns the size of the BLOB in bytes.
6073 ** in row iRow, column zColumn, table zTable in database zDb;
6074 ** in other words, the same BLOB that would be selected by:
6082 ** the name that appears after the AS keyword in the [ATTACH] statement.
6091 ** in *ppBlob. Otherwise an [error code] is returned and, unless the error
6102 ** <li> ^(Row iRow is not present in the table)^,
6134 ** and the built-in [zeroblob] SQL function may be used to create a
6162 ** it must exist and there must be either a blob or text value stored in
6163 ** the nominated column.)^ ^If the new row is not present in the table, or if
6184 ** the database is in auto-commit mode and there are no other open read-write
6190 ** open blob handle results in undefined behaviour. ^Calling this routine
6202 ** ^Returns the size in bytes of the BLOB accessible via the
6203 ** successfully opened [BLOB handle] in its only argument. ^The
6209 ** been closed by [sqlite3_blob_close()]. Passing any other pointer in
6210 ** to this routine results in undefined and probably undesirable behavior.
6236 ** been closed by [sqlite3_blob_close()]. Passing any other pointer in
6237 ** to this routine results in undefined and probably undesirable behavior.
6278 ** been closed by [sqlite3_blob_close()]. Passing any other pointer in
6279 ** to this routine results in undefined and probably undesirable behavior.
6329 ** implementations are available in the SQLite core:
6338 ** that does no real locking and is appropriate for use in
6345 ** implementation is included with the library. In this case the
6380 ** not want to. SQLite will only request a recursive mutex in
6383 ** might return such a mutex in response to SQLITE_MUTEX_FAST.
6402 ** mutex results in undefined behavior.
6410 ** In such cases, the
6447 ** does not provide a suitable implementation. In
6500 ** If xMutexInit fails in any way, it is expected to clean up after itself
6528 ** These routines should return true if the mutex in their argument
6557 ** next. Applications that override the built-in mutex logic must be
6573 #define SQLITE_MUTEX_STATIC_VFS1 11 /* For use by built-in VFS */
6582 ** serializes access to the [database connection] given in the argument
6599 ** ^A NULL pointer can be used in place of "main" to refer to the
6687 ** ^The highest recorded value is returned in *pHighwater. ^If the
6727 ** memory controlled by [SQLITE_CONFIG_PAGECACHE] is not included in
6729 ** sizes as reported by the xSize method in [sqlite3_mem_methods].</dd>)^
6734 ** internal equivalents). Only the value returned in the
6746 ** value returned is in pages, not in bytes.</dd>)^
6756 ** no space was left in the page cache.</dd>)^
6760 ** handed to [pagecache memory allocator]. Only the value returned in the
6767 ** [SQLITE_CONFIG_SCRATCH]. The value returned is in allocations, not
6768 ** in bytes. Since a single thread may only have one scratch allocation
6784 ** handed to [scratch memory allocator]. Only the value returned in the
6817 ** to grow in future releases of SQLite.
6838 ** New verbs may be added in future releases of SQLite. Existing verbs
6866 ** memory already being in use.
6906 ** wal file in wal mode databases, or the number of pages written to the
6907 ** database file in rollback mode databases. Any pages written as part of
6971 ** <dd>^This is the number of times that SQLite has stepped forward in
6978 ** A non-zero value in this counter may indicate an opportunity to
6983 ** were created automatically in order to help joins run faster.
6984 ** A non-zero value in this counter may indicate an opportunity to
7019 ** The sqlite3_pcache_page object represents a single page in the
7037 ** register an alternative page cache implementation by passing in an
7039 ** In many applications, most of the heap memory allocated by
7043 ** the amount of memory consumed by SQLite, the way in which
7050 ** The built-in page cache is recommended for most uses.
7065 ** built-in default page cache is used instead of the application defined
7078 ** in multithreaded applications.
7087 ** first parameter, szPage, is the size in bytes of the pages that must
7097 ** false if it is used for an in-memory database. The cache implementation
7101 ** ^In other words, calls to xUnpin() on a cache with bPurgeable set to
7116 ** stored in the cache, both pinned and unpinned.
7119 ** The xFetch() method locates a page in the cache and returns a pointer to
7125 ** for each entry in the page cache.
7131 ** If the requested page is already in the page cache, then the page cache
7133 ** intact. If the requested page is not already in the cache, then the
7138 ** <tr><th> createFlag <th> Behavior when page is not already in cache
7148 ** failed.)^ In between the to xFetch() calls, SQLite may
7212 ** retained in the header file for backwards compatibility only.
7247 ** for copying in-memory databases to or from persistent files.
7277 ** temporary database, or the name specified after the AS keyword in
7291 ** returned and an error code and error message are stored in the
7322 ** <li> the destination database is an in-memory database and the
7330 ** [SQLITE_BUSY] is returned to the caller. ^In this case the call to
7334 ** is called, then [SQLITE_LOCKED] is returned immediately. ^Again, in this
7338 ** there is no point in retrying the call to sqlite3_backup_step(). These
7388 ** in the source database at the conclusion of the most recent
7391 ** sqlite3_backup_step(). If the source database is modified in a way that
7393 ** those changes are not reflected in the output of sqlite3_backup_pagecount()
7412 ** backup is in progress might also also cause a mutex deadlock.
7414 ** If running in [shared cache mode], the application must
7416 ** is not accessed while the backup is running. In practice this means
7443 ** ^When running in shared-cache mode, a database operation may fail with
7468 ** ^(If sqlite3_unlock_notify() is called in a multi-threaded application,
7502 ** and the second is the number of entries in the array.
7525 ** system in a deadlocked state, then SQLITE_LOCKED is returned and no
7526 ** unlock-notify callback is registered. The system is said to be in
7543 ** returned. In this case there is no "blocking connection", so invoking
7544 ** sqlite3_unlock_notify() results in the unlock-notify callback being
7550 ** extended error code is set to SQLITE_LOCKED_SHAREDCACHE. Otherwise, in
7566 ** strings in a case-independent fashion, using the same definition of "case
7577 ** the glob pattern P. ^The definition of glob pattern matching used in
7578 ** [sqlite3_strglob(P,X)] is the same as for the "X GLOB P" operator in the
7603 ** will not use dynamically allocated memory. The log message is stored in
7615 ** is invoked each time data is committed to a database in wal mode.
7626 ** is the number of pages currently in the write-ahead log file,
7658 ** more frames in the [write-ahead log] file. ^Passing zero or
7688 ** In brief, sqlite3_wal_checkpoint(D,X) causes the content in the
7708 ** operation on database X of [database connection] D in mode M. Status
7716 ** in the log were checkpointed. ^The [busy-handler callback]
7717 ** is never invoked in the SQLITE_CHECKPOINT_PASSIVE mode.
7725 ** snapshot. ^It then checkpoints all frames in the log file and syncs the
7744 ** ^If pnLog is not NULL, then *pnLog is set to the total number of frames in
7746 ** of an error or because the database is not in [WAL mode]. ^If pnCkpt is not
7747 ** NULL,then *pnCkpt is set to the total number of checkpointed frames in the
7750 ** because the database is not in WAL mode. ^Note that upon successful
7757 ** busy-handler configured, it will not be invoked in this case.
7766 ** checkpoint operation proceeds from that point in the same way as
7768 ** without blocking any further. ^SQLITE_BUSY is returned in this case.
7772 ** [database connection] db. In this case the
7782 in WAL
7799 int *pnLog, /* OUT: Size of WAL log in frames */
7829 ** may be added in the future.
7846 ** support constraints. In this configuration (which is the default) if
7868 ** SQLITE_CONSTRAINT, in which case SQLite falls back to OR ABORT
7947 ** The select-id is the same value as is output in the first column
7982 ** ^Statistics might not be available for all loops in all statements. ^In cases
8024 ** The author disclaims copyright to this source code. In place of
8111 unsigned int *anQueue; /* Number of pending entries in the queue */
8114 int mxLevel; /* The largest iLevel value in the tree */
8120 /* The following fields are only available in 3.8.11 and later */
8141 ** The author disclaims copyright to this source code. In place of
8150 ** Interfaces to extend FTS5. Using the interfaces defined in this file,
8181 int nVal, /* Number of values in apVal[] array */
8199 ** to the total number of tokens in the FTS5 table. Or, if iCol is
8200 in the table, return
8201 ** the total number of tokens in column iCol, considering all rows in
8205 ** in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g.
8210 ** Return the number of columns in the table.
8214 ** to the total number of tokens in the current row. Or, if iCol is
8215 ** non-negative but less than the number of columns in the table, set
8216 ** *pnToken to the number of tokens in column iCol of the current row.
8219 ** in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g.
8226 ** containing the text in utf-8 encoding, (*pn) is set to the size in bytes
8232 ** Returns the number of phrases in the current query expression.
8235 ** Returns the number of tokens in phrase iPhrase of the query. Phrases
8313 ** (set to NULL) before this function returns. In this case the xDelete,
8319 ** This function is used to retrieve the total number of rows in the table.
8320 ** In other words, the same value that would be returned by:
8411 ** be returned. In this case, fts5 assumes that the final value of *ppOut
8449 ** For each token in the input string, the supplied callback xToken() must
8453 ** size of the token in bytes. The 4th and 5th arguments are the byte offsets
8459 ** synonyms. In this case see the discussion below for details.
8461 ** FTS5 assumes the xToken() callback is invoked for each token in the
8474 ** Custom tokenizers may also support synonyms. Consider a case in which a
8476 ** built-in tokenizers, the FTS5 query 'first + place' will match instances
8478 ** such as "1st place". In some applications, it would be better to match
8480 ** the user specified in the MATCH query text.
8482 ** There are several ways to approach this in FTS5:
8484 ** <ol><li> By mapping all synonyms to a single token. In this case, the
8485 ** In the above example, this means that the tokenizer returns the
8486 ** same token for inputs "first" and "1st". Say that token is in
8494 ** In this case, when tokenizing query text, the tokenizer may
8503 ** first token in the MATCH query and FTS5 effectively runs a query
8523 ** 'first + place' or '1st + place', as there are entires in the
8543 ** by making multiple calls to xToken(FTS5_TOKEN_COLOCATED) in sequence.
8547 ** In many cases, method (1) above is the best approach. It does not add
8549 in terms of disk space and query speed. However, it
8559 ** For full prefix support, method (3) may be preferred. In this case,
8592 int nToken, /* Size of token in bytes */
8660 /************** Continuing where we left off in sqliteInt.h ******************/
8670 /************** Include sqliteLimit.h in the middle of sqliteInt.h ***********/
8675 ** The author disclaims copyright to this source code. In place of
8688 ** The maximum length of a TEXT or BLOB in bytes. This also
8689 ** limits the size of a row in a table or index.
8701 ** * Columns in a table
8702 ** * Columns in an index
8703 ** * Columns in a view
8704 ** * Terms in the SET clause of an UPDATE statement
8705 ** * Terms in the result set of a SELECT statement
8706 ** * Terms in the GROUP BY or ORDER BY clauses of a SELECT statement.
8707 ** * Terms in the VALUES clause of an INSERT statement
8710 ** tell you that in a well-normalized database, you usually should
8711 ** not have more than a dozen or so columns in any table. And if
8712 ** that is the case, there is no point in having more than a few
8713 ** dozen values in any of the other situations described above.
8720 ** The maximum length of a single SQL statement in bytes.
8745 ** The maximum number of terms in a compound SELECT statement.
8748 ** if the number of terms is too large. In practice, most SQL
8750 ** any limit on the number of terms in a compount SELECT.
8757 ** The maximum number of opcodes in a VDBE program.
8772 ** The suggested maximum number of in-memory pages to use for
8785 ** The default number of frames to accumulate in the log file before
8786 ** checkpointing the database in WAL mode.
8854 ** Maximum number of pages in one database file.
8865 ** Maximum length (in bytes) of the pattern in a LIKE or GLOB
8884 /************** Continuing where we left off in sqliteInt.h ******************/
8918 ** So we have to define the macros in different ways depending on the
8994 ** default unless SQLite is compiled with SQLITE_DEFAULT_MEMSTATUS=0 in
9002 ** Exactly one of the following macros must be defined in order to
9043 ** We need to define _XOPEN_SOURCE as follows in order to enable
9058 ** assert() statements in the code. So we want the default action
9060 ** is set. Thus NDEBUG becomes an opt-in rather than an opt-out
9078 ** The testcase() macro is used to aid in coverage testing. When
9080 ** testcase() must be evaluated both true and false in order to
9082 ** to help ensure adequate test coverage in places where simple
9114 ** "Verification, Validation, and Accreditation". In other words, the
9127 ** are included in a few cases in order to enhance the resilience
9132 ** In other words, ALWAYS and NEVER are added for defensive code.
9163 ** Is the sqlite3ErrName() function needed in the build? Currently,
9177 ** to fit in 32-bits. This macro is used inside of various testcase()
9186 ** in theory, be used by the compiler to generate better code, but
9192 /************** Include hash.h in the middle of sqliteInt.h ******************/
9197 ** The author disclaims copyright to this source code. In place of
9206 ** used in SQLite.
9226 ** There are Hash.htsize buckets. Each bucket points to a spot in
9228 ** element pointed to plus the next _ht.count-1 elements in the list.
9230 ** Hash.htsize and Hash.ht may be zero. In that case lookup is done
9233 ** in the table, it is faster to do a linear search than to manage
9237 unsigned int htsize; /* Number of buckets in the hash table */
9238 unsigned int count; /* Number of entries in this table */
9246 /* Each element in the hash table is an instance of the following
9253 HashElem *next, *prev; /* Next and previous elements in the table */
9285 ** Number of entries in a hash table
9292 /************** Continuing where we left off in sqliteInt.h ******************/
9293 /************** Include parse.h in the middle of sqliteInt.h *****************/
9456 /************** Continuing where we left off in sqliteInt.h ******************/
9514 ** Provide a default value for SQLITE_TEMP_STORE in case it is not specified
9637 ** that can be stored in a u32 without loss of data. The value
9639 ** have to specify the value in the less intuitive manner shown:
9644 ** The datatype used to store estimates of the number of rows in a
9681 ** Set the SQLITE_PTRSIZE macro to the number of bytes in a pointer
9757 ** pointers. In that case, only verify 4-byte alignment.
9774 ** Default maximum size of memory used by memory-mapped I/O in the VFS
9870 ** A convenience macro that returns the number of elements in
9901 ** In the usual case where WSD is supported, the SQLITE_WSD and GLOBAL
9929 ** For example those parameters only used in assert() statements. In these
9990 /************** Include btree.h in the middle of sqliteInt.h *****************/
9995 ** The author disclaims copyright to this source code. In place of
10004 ** subsystem. See comments in the source code for a detailed description
10047 ** NOTE: These values must match the corresponding PAGER_ values in
10051 #define BTREE_MEMORY 2 /* This is an in-memory DB */
10097 ** is stored in the leaves. (BTREE_INTKEY is used for SQL tables.) With
10118 ** to constants so that the offset of the corresponding field in an
10127 ** The BTREE_DATA_VERSION value is not really a value stored in the header.
10148 ** to be filled with content that is already in sorted order.
10152 ** selected will all have the same key. In other words, the cursor will
10156 #define BTREE_BULKLOAD 0x00000001 /* Used to full index in sorted order */
10268 /************** Continuing where we left off in sqliteInt.h ******************/
10269 /************** Include vdbe.h in the middle of sqliteInt.h ******************/
10274 ** The author disclaims copyright to this source code. In place of
10294 ** in the source file sqliteVdbe.c are allowed to see the insides
10300 ** The names of the following types declared in vdbeInt.h are required
10314 u8 opflags; /* Mask of the OPFLG_* flags in opcodes.h */
10354 int nOp; /* Elements in aOp[] */
10421 ** The following macro converts a relative address in the p2 field
10432 /************** Include opcodes.h in the middle of vdbe.h ********************/
10495 #define OP_OpenPseudo 60 /* synopsis: P3 columns in r[P2] */
10566 #define OP_RowSetTest 131 /* synopsis: if r[P3] in rowset(P1) goto P2 */
10598 /* Properties such as "out2" or "jump" that are specified in
10599 ** comments following the "case" for each opcode in the vdbe.c
10632 /************** Continuing where we left off in vdbe.h ***********************/
10707 ** comments in VDBE programs that show key decision points in the code
10729 ** numbers in the sqlite3.c source file. VDBE branch coverage testing
10745 ** -DSQLITE_DEBUG then an ALWAYS() will fail in the vdbeTakeBranch()
10746 ** routine in vdbe.c, alerting the developer to the missed tag.
10772 /************** Continuing where we left off in sqliteInt.h ******************/
10773 /************** Include pager.h in the middle of sqliteInt.h *****************/
10778 ** The author disclaims copyright to this source code. In place of
10804 ** The type used to represent a page number. The first page in a file
10820 ** Page number PAGER_MJ_PGNO is never used in an SQLite database (it is
10822 ** used in the journal to signify that the remainder of the journal file
10824 ** roll back. See comments for function writeMasterJournal() in pager.c
10832 ** NOTE: These values must match the corresponding BTREE_ values in btree.h.
10835 #define PAGER_MEMORY 0x0002 /* In-memory database */
10852 #define PAGER_JOURNALMODE_MEMORY 4 /* In-memory journal file */
10995 /************** Continuing where we left off in sqliteInt.h ******************/
10996 /************** Include pcache.h in the middle of sqliteInt.h ****************/
11001 ** The author disclaims copyright to this source code. In place of
11019 ** Every page in the cache is controlled by an instance of the following
11041 PgHdr *pDirtyNext; /* Next element in list of dirty pages */
11042 PgHdr *pDirtyPrev; /* Previous element in list of dirty pages */
11080 /* Return the size in bytes of a PCache object. Used to preallocate
11104 /* Get a list of all dirty pages in the cache, sorted by page number */
11124 /* Return the total number of pages stored in the cache */
11128 /* Iterate through all dirty pages currently stored in the cache. This
11167 /************** Continuing where we left off in sqliteInt.h ******************/
11169 /************** Include os.h in the middle of sqliteInt.h ********************/
11174 ** The author disclaims copyright to this source code. In place of
11197 /************** Include os_setup.h in the middle of os.h *********************/
11202 ** The author disclaims copyright to this source code. In place of
11258 /************** Continuing where we left off in os.h *************************/
11276 ** alphanumeric characters, and no file extension. They are stored in the
11278 ** If sqlite is being embedded in another program, you may wish to change the
11284 ** Mcafee started using SQLite in their anti-virus product and it
11285 ** started putting files with the "sqlite" name in the c:/temp folder.
11334 ** An EXCLUSIVE_LOCK is obtained by locking all bytes in the range.
11343 ** for probabilistic locking in Win95/98/ME. Hence, the locking scheme
11349 ** SHARED_SIZE is the number of bytes available in the pool from which
11361 ** Locking in windows is manditory. For this reason, we cannot store
11362 ** actual data in the bytes used for locking. The pager never allocates
11363 ** the pages involved in locking therefore. SHARED_SIZE is selected so
11370 ** Changing the value of PENDING_BYTE results in a subtly incompatible
11443 /************** Continuing where we left off in sqliteInt.h ******************/
11444 /************** Include mutex.h in the middle of sqliteInt.h *****************/
11449 ** The author disclaims copyright to this source code. In place of
11460 ** to all source files. We break it out in an effort to keep the code
11518 /************** Continuing where we left off in sqliteInt.h ******************/
11524 ** in the sqlite.aDb[] array. aDb[0] is the main database file and
11540 ** In shared cache mode, a single Schema object can be shared by multiple
11547 ** A thread must be holding a mutex on the corresponding Btree in order
11563 int cache_size; /* Number of pages to use in the cache */
11567 ** These macros can be used to test, set, or clear bits in the
11604 ** lookaside malloc subsystem. Each available memory allocation in
11610 ** be stored in lookaside because in shared cache mode the schema information
11616 u16 sz; /* Size of each buffer in bytes */
11627 LookasideSlot *pNext; /* Next buffer in the list of free buffers */
11642 ** Information held in the "sqlite3" database connection object and used
11648 int nAuthPW; /* Size of the zAuthPW in bytes */
11688 int nDb; /* Number of backends currently in use */
11762 VTable *pDisconnect; /* Disconnect these in next sqlite3_prepare() */
11769 int busyTimeout; /* Busy handler timeout, in msec */
11774 int *pnBytesFreed; /* If not NULL, increment this in DbFree() */
11777 ** mutex, not by sqlite3.mutex. They are used by code in notify.c.
11790 sqlite3 *pNextBlocked; /* Next in list of all blocked connections */
11824 #define SQLITE_LegacyFileFmt 0x00008000 /* Create new databases in format 1 */
11830 #define SQLITE_PreferBuiltin 0x00200000 /* Preference to built-in funcs */
11836 #define SQLITE_Vacuum 0x08000000 /* Currently in a VACUUM */
11855 #define SQLITE_OmitNoopJoin 0x0400 /* Omit unused tables in joins */
11884 #define SQLITE_MAGIC_BUSY 0xf03b7906 /* Database currently in use */
11890 ** structure. A pointer to this structure is stored in the sqlite.aFunc
11930 ** are assert() statements in the code to verify this.
11937 #define SQLITE_FUNC_LENGTH 0x0040 /* Built-in length() function */
11938 #define SQLITE_FUNC_TYPEOF 0x0080 /* Built-in typeof() function */
11939 #define SQLITE_FUNC_COUNT 0x0100 /* Built-in count(*) aggregate */
11940 #define SQLITE_FUNC_COALESCE 0x0200 /* Built-in coalesce() or ifnull() */
11941 #define SQLITE_FUNC_UNLIKELY 0x0400 /* Built-in unlikely() function */
11970 ** are interpreted in the same way as the first 4 parameters to
12007 ** All current savepoints are stored in a linked list starting at
12008 ** sqlite3.pSavepoint. The first element in the list is the most recently
12030 ** instance of the following structure, stored in the sqlite3.aModule
12042 ** information about each column of an SQL table is held in an instance
12060 #define COLFLAG_HIDDEN 0x0002 /* A hidden column in a virtual table */
12082 #define SQLITE_SO_ASC 0 /* Sort in ascending order */
12083 #define SQLITE_SO_DESC 1 /* Sort in ascending order */
12124 #define SQLITE_STOREP2 0x20 /* Store result in reg[P2] rather than jump */
12129 ** An object of this type is created for each virtual table present in
12137 ** database connections, even when the rest of the in-memory database
12146 ** All VTable objects that correspond to a single table in a shared
12147 ** database schema are initially stored in a linked-list pointed to by
12152 ** sqlite3_vtab* handle in the compiled query.
12154 ** When an in-memory Table object is deleted (for example when the
12167 ** sqlite3DbMalloc(), using the connection handle stored in VTable.db as
12177 VTable *pNext; /* Next in linked list (see above) */
12181 ** The schema for each SQL table and view is represented in memory
12189 FKey *pFKey; /* Linked list of all foreign keys in this table */
12192 /* ... also used as column name list in a VIEW */
12195 i16 nCol; /* Number of columns in this table */
12197 LogEst nRowLogEst; /* Estimated rows in table - from sqlite_stat1 table */
12198 LogEst szTabRow; /* Estimated size of each table row in bytes */
12203 u8 keyConf; /* What to do in case of uniqueness conflict on iPKey */
12205 int addColOffset; /* Offset in CREATE TABLE stmt to add a new column */
12212 Trigger *pTrigger; /* List of triggers stored in pSchema */
12223 ** the TF_OOOHidden attribute would apply in this case. Such tables require
12258 ** key. The "to" table is the table that is named in the REFERENCES clause.
12279 ** is held in Schema.fkeyHash with a hash key of Z.
12283 FKey *pNextFrom; /* Next FKey with the same in pFrom. Next parent of pFrom */
12287 int nCol; /* Number of columns in this key */
12292 struct sColMap { /* Mapping of columns in pFrom to columns in zTo */
12293 int iFrom; /* Index of column in pFrom */
12294 char *zCol; /* Name of column in zTo. If NULL use PRIMARY KEY */
12301 ** causes the operation in process to fail and for the current transaction
12302 ** to be rolled back. ABORT processing means the operation in process
12305 ** the operation in progress stops and returns an error code. But prior
12350 u16 nField; /* Number of key columns in the index */
12376 u16 nField; /* Number of entries in apMem[] */
12386 ** Each SQL index is represented in memory by an
12396 ** In the Table structure describing Ex1, nCol==3 because there are
12397 ** three columns in the table. In the Index structure describing
12400 ** first column to be indexed (c3) has an index of 2 in Ex1.aCol[].
12401 ** The second column to be indexed (c1) has an index of 0 in
12411 ** While parsing a CREATE TABLE or CREATE INDEX statement in order to
12414 ** of this structure may be created. In this case the Index.tnum variable is
12432 LogEst szIdxRow; /* Estimated average row size in bytes */
12434 u16 nColumn; /* Number of columns stored in the index */
12437 unsigned bUnordered:1; /* Use this index for == or IN queries only */
12443 int nSample; /* Number of elements in aSample[] */
12445 tRowcnt *aAvgEq; /* Average nEq values for keys not in aSample */
12448 tRowcnt nRowEst0; /* Non-logarithmic number of rows in the index */
12472 ** Each sample stored in the sqlite_stat3 table is represented in memory
12478 int n; /* Size of record in bytes */
12494 unsigned int n; /* Number of characters in this token */
12502 ** pointer to this structure. The Expr.iColumn field is the index in
12513 u8 useSortingIdx; /* In direct mode, reference the sorting index rather
12517 int nSortingColumn; /* Number of columns in the sorting index */
12520 struct AggInfo_col { /* For each column used in source tables */
12524 int iSorterColumn; /* Column number in the sorting index */
12528 int nColumn; /* Number of used entries in aCol[] */
12538 int nFunc; /* Number of entries in aFunc[] */
12545 ** it uses less memory in the Expr object, which is a big memory user
12546 ** in systems with lots of prepared statements. And few applications
12558 ** Each node of an expression in the parse tree is an instance
12564 ** to represent the greater-than-or-equal-to operator in the expression
12577 ** a CASE expression or an IN expression of the form "<lhs> IN (<y>, <z>...)".
12579 ** the form "<lhs> IN (SELECT ...)". If the EP_xIsSelect bit is set in the
12583 ** An expression of the form ID or ID.ID refers to a column in a table.
12587 ** expression is used as a result in an aggregate SELECT, then the
12588 ** value is also stored in the Expr.iAgg column in the aggregate so that
12592 ** character '?' in the original SQL) then the Expr.iTable holds the index
12607 ** Expr objects can use a lot of memory space in database schema. To
12610 ** two or more Expr objects will be stored in a single memory allocation,
12615 ** the child Expr objects in the Expr.pLeft and Expr.pRight subtrees
12617 ** the subtrees in Expr.x.pList or Expr.x.pSelect are always separately
12629 /* If the EP_TokenOnly flag is set in the Expr.flags mask, then no
12631 ** access them will result in a segfault or malfunction.
12637 ExprList *pList; /* op = IN, EXISTS, SELECT, CASE, FUNCTION, BETWEEN */
12638 Select *pSelect; /* EP_xIsSelect and op = IN, EXISTS, SELECT */
12641 /* If the EP_Reduced flag is set in the Expr.flags mask, then no
12643 ** access them will result in a segfault or malfunction.
12655 i16 iAgg; /* Which entry in pAggInfo->aCol[] or ->aFunc[] */
12665 ** The following are the meanings of bits in the Expr.flags field.
12667 #define EP_FromJoin 0x000001 /* Originates in ON/USING clause of outer join */
12673 #define EP_DblQuoted 0x000040 /* token.z was originally in "..." */
12677 #define EP_IntValue 0x000400 /* Integer value contained in u.iValue */
12682 #define EP_Static 0x008000 /* Held in memory not obtained from malloc() */
12697 ** These macros can be used to test, set, or clear bits in the
12717 ** struct, an Expr struct with the EP_Reduced flag set in Expr.flags
12732 ** name. An expr/name combination can be used in several ways, such
12733 ** as the list of "expr AS ID" fields following a "SELECT" or in the
12734 ** list of "ID = expr" items in an UPDATE. A list of expressions can
12735 ** also be used as the argument to a function, in which case the a.zName
12739 ** the expression that is used in the generation of error messages and
12740 ** column labels. In this case, Expr.zSpan is typically the text of a
12741 ** column expression as it exists in a SELECT statement. However, if
12743 ** of the result column in the form: DATABASE.TABLE.COLUMN. This later
12748 struct ExprList_item { /* For each expression in the list */
12758 u16 iOrderByCol; /* For ORDER BY, column number in result set */
12761 int iConstExprReg; /* Register in which Expr value is cached */
12779 ** such as the list "a,b,c" in the following statements:
12786 ** column names after a table name in an INSERT statement. In the statement
12795 int idx; /* Index in some Table.aCol[] of a column named zName */
12804 ** tables in a join to 32 instead of 64. But it also reduces the size
12810 ** The number of bits in a Bitmask. "BMS" means "BitMask Size".
12815 ** A bit in a Bitmask
12822 ** Each table or subquery in the FROM clause is a separate element of
12827 ** is modified by an INSERT, DELETE, or UPDATE statement. In standard SQL,
12828 ** such a table must be a simple name: ID. But in SQLite, the table can
12836 ** In the colUsed field, the high-order bit (bit 63) is set if the table
12840 int nSrc; /* Number of tables or subqueries in the FROM clause */
12841 u32 nAlloc; /* Number of entries allocated in a[] below */
12848 Select *pSelect; /* A SELECT statement used in place of a table name */
12859 unsigned isRecursive :1; /* True for recursive reference in WITH */
12862 u8 iSelectId; /* If pSelect!=0, the id of the sub-select in EQP */
12899 #define WHERE_ONETABLE_ONLY 0x0040 /* Only code the 1st table in pTabList */
12916 ** A NameContext defines a context in which to resolve table and column
12932 ** NameContext in the parent query. Thus the process of scanning the
12956 #define NC_IsCheck 0x0004 /* True if resolving names in a CHECK constraint */
12970 ** in the VDBE that record the limit and offset counters.
12973 ** These addresses must be stored so that we can go back and fill in
12975 ** the number of columns in P2 can be computed at the same time
12997 Select *pPrior; /* Prior select in a compound select statement */
12998 Select *pNext; /* Next select to the left in a compound */
13026 ** The results of a SELECT can be distributed in several ways, as defined
13030 ** SRT_Union Store results as a key in a temporary index
13035 ** SRT_Exists Store a 1 in memory cell pDest->iSDParm if the result
13046 ** opcode) for each row in the result set.
13050 ** in register pDest->iSDParm then abandon the rest
13054 ** row of result as the key in table pDest->iSDParm.
13056 ** results. Used to implement "IN (SELECT ...)".
13066 ** of the co-routine is stored in register pDest->iSDParm
13067 ** and the result row is stored in pDest->nDest registers
13070 ** SRT_Table Store results in temporary table pDest->iSDParm.
13076 ** SRT_DistFifo Store results in a temporary table pDest->iSDParm.
13081 ** SRT_Queue Store results in priority queue pDest->iSDParm (really
13085 ** SRT_DistQueue Store results in priority queue pDest->iSDParm only if
13089 #define SRT_Union 1 /* Store result as keys in an index */
13095 #define SRT_Queue 7 /* Store result in an queue */
13102 #define SRT_Mem 10 /* Store result in a memory cell */
13103 #define SRT_Set 11 /* Store results as keys in an index */
13126 ** information in case inserts are down within triggers. Triggers do not
13131 AutoincInfo *pNext; /* Next info block in a list of them all */
13133 int iDb; /* Index in sqlite3.aDb[] of database holding pTab */
13147 ** statement. All such objects are stored in the linked list headed at
13153 ** TriggerPrg.orconf, is stored in the TriggerPrg.pProgram variable.
13164 TriggerPrg *pNext; /* Next entry in Parse.pTriggerPrg list */
13191 ** the parser and down into all the parser action routine in order to
13213 u8 nTempReg; /* Number of temporary registers in aTempReg[] */
13220 int iRangeReg; /* First register in temporary register block */
13255 int nTableLock; /* Number of locks in aTableLock */
13275 ** in the recursive region.
13278 int nVar; /* Number of '?' variables seen in the SQL so far */
13279 int nzVar; /* Number of available slots in azVar[] */
13329 ** Bitfield flags for P5 value in various opcodes.
13336 #define OPFLAG_USESEEKRESULT 0x10 /* Try to avoid a seek in BtreeInsert() */
13345 * Each trigger present in the database schema is stored as an instance of
13348 * Pointers to instances of struct Trigger are stored in two ways.
13349 * 1. In the "trigHash" hash table (part of the sqlite3* that represents the
13378 ** In that cases, the constants below can be ORed together.
13387 * Instances of struct TriggerStep are stored in a singly linked list (linked
13430 TriggerStep *pNext; /* Next in the link-list */
13431 TriggerStep *pLast; /* Last element in link-list. Valid for 1st elem only */
13444 const char *zDb; /* Make sure all objects are contained in this database */
13451 ** do not necessarily know how big the string will be in the end.
13458 int nAlloc; /* Amount of space allocated in zText */
13503 int szPage; /* Size of each page in pPage[] */
13504 int nPage; /* Number of pages in pPage[] */
13511 int inProgress; /* True while initialization in progress */
13593 int nCte; /* Number of CTEs in the WITH clause */
13595 struct Cte { /* For each CTE in the WITH clause.... */
13610 u8 bLine[100]; /* Draw vertical in column i if bLine[i] is true */
13650 ** needed by FTS3 when FTS3 is included in the amalgamation.
14110 ** be defined locally, but now we use the varint routines in the util.c
14348 /* Declarations for functions in fkey.c. All of these are replaced by
14349 ** no-op macros if OMIT_FOREIGN_KEY is defined. In this case no foreign
14351 ** OMIT_FOREIGN_KEY is not, only some of the functions are no-oped. In
14385 ** The interface to the code in fault.c used for identifying "benign"
14409 #define IN_INDEX_MEMBERSHIP 0x0002 /* IN operator used for membership test */
14410 #define IN_INDEX_LOOP 0x0004 /* IN operator used as a loop */
14476 ** sqlite3MemdebugHasType() returns true if any of the bits in its second
14480 ** sqlite3MemdebugNoType() returns true if none of the bits in its second
14528 ** The author disclaims copyright to this source code. In place of
14587 ** The following 256 byte lookup table is used to support SQLites built-in
14608 ** Bit 0x40 is set if the character non-alphanumeric and can be used in an
14614 ** locale of "C". They are implemented as macros in sqliteInt.h.
14656 /* EVIDENCE-OF: R-02982-34736 In order to maintain full backwards
14680 ** page size in bytes.
14759 ** 1-gibabyte boundary) in a compatible database. SQLite never uses
14765 ** a different position in the file. This allows code that has to
14771 ** 0x40000000 results in an incompatible database file format!
14772 ** Changing the pending byte during operation will result in undefined
14783 ** from the comments following the "case OP_xxxx:" statements in
14793 ** The author disclaims copyright to this source code. In place of
14814 ** This array looks large, but in a typical installation actually uses
14821 ** for those options in which the value is meaningful. */
15202 /* Since ArraySize(azCompileOpt) is normally in single digits, a
15232 ** The author disclaims copyright to this source code. In place of
15245 /************** Include vdbeInt.h in the middle of status.c ******************/
15250 ** The author disclaims copyright to this source code. In place of
15287 /* Opaque type used by code in vdbesort.c */
15315 i16 nField; /* Number of fields in the header */
15320 i8 iDb; /* Index of cursor database in db->aDb[] (or -1) */
15346 u32 payloadSize; /* Total number of bytes in the record */
15347 u32 szRow; /* Byte available in aRow */
15351 u32 aType[1]; /* Type values for all entries in the record */
15353 ** static element declared in the structure. nField total array slots for
15362 ** values stored in the Vdbe struct. When the sub-program is finished,
15368 ** cell in the parent (calling) frame. When the memory cell is deleted or
15371 ** list is deleted when the VM is reset in VdbeHalt(). The reason for doing
15376 ** The currently executing frame is stored in Vdbe.pFrame. Vdbe.pFrame is
15390 int nCursor; /* Number of entries in apCsr */
15391 int pc; /* Program Counter in parent (calling) frame */
15393 int nMem; /* Number of entries in aMem */
15394 int nOnceFlag; /* Number of entries in aOnceFlag */
15415 double r; /* Real value used when MEM_Real is set in flags */
15416 i64 i; /* Integer value used when MEM_Int is set in flags */
15417 int nZero; /* Used when bit MEM_Zero is set in flags */
15425 int n; /* Number of characters in string value, excluding '\0' */
15430 u32 uTemp; /* Transient storage for serial_type in OP_MakeRecord */
15446 ** representations of the value stored in the Mem struct.
15449 ** No other flags may be set in this case.
15452 ** Usually this is encoded in the same unicode encoding as the main
15502 ** implementation calling sqlite3_set_auxdata() is stored in an instance
15504 ** are stored in a linked list headed at Vdbe.pAuxData. All are destroyed
15512 AuxData *pNext; /* Next element in list */
15520 ** There is a typedef for this structure in sqlite.h. So all routines,
15543 ** in describing recursive data structures.
15548 int nIndent; /* Number of elements in aIndent */
15584 int nOp; /* Number of instructions in the program */
15585 int nCursor; /* Number of slots in apCsr[] */
15592 ynVar nVar; /* Number of entries in aVar[] */
15593 ynVar nzVar; /* Number of entries in azVar[] */
15600 u16 nResColumn; /* Number of columns in one row of the result set */
15601 u8 errorAction; /* Recovery action to do in case of an error */
15628 int nFrame; /* Number of frames in pFrame list */
15636 int nScan; /* Entries in aScan[] */
15757 /************** Continuing where we left off in status.c *********************/
15760 ** Variables in which to record status information.
15793 ** we have to locate the state vector at run-time. In the more common
16087 ** The author disclaims copyright to this source code. In place of
16098 ** There is only one exported symbol in this file - the function
16104 ** in Greenwich on November 24, 4714 B.C. according to the Gregorian
16121 ** in the following text:
16145 int tz; /* Timezone offset in minutes */
16156 ** come in groups of 5 as follows:
16158 ** N number of digits in the integer
16214 ** of change in p->tz and return 0. If a parser error occurs,
16399 ** In the first form, the +/-HH:MM is always optional. The fractional
16514 ** EVIDENCE-OF: R-62172-00036 In this implementation, the standard C
16515 ** library function localtime_r() is used to assist in the calculation of
16550 ** Compute the difference (in milliseconds) between localtime and UTC
16551 ** (a.k.a. GMT) for the time value p where p is in UTC. If no error occurs,
16555 ** is undefined in this case.
16632 ** is in a system call (i.e. localtime()), then an error message is written
16880 ** Return the julian day number of the date specified in the arguments
17195 ** functions. This should be the only routine in this file with
17229 ** The author disclaims copyright to this source code. In place of
17518 ** error in sqlite3_os_init() by the upper layers can be tested.
17626 ** The author disclaims copyright to this source code. In place of
17664 ** we have to locate the state vector at run-time. In the more common
17716 ** The author disclaims copyright to this source code. In place of
17752 ** This routine is the only routine in this file with external linkage.
17754 ** Populate the low-level memory allocation function pointers in
17755 ** sqlite3GlobalConfig.m with pointers to the routines in this file.
17778 ** The author disclaims copyright to this source code. In place of
17792 ** routines specified in the sqlite3_mem_methods object. The content of
17796 ** default configuration is to use memory allocation routines in this
18050 ** This routine is the only routine in this file with external linkage.
18052 ** Populate the low-level memory allocation function pointers in
18053 ** sqlite3GlobalConfig.m with pointers to the routines in this file.
18076 ** The author disclaims copyright to this source code. In place of
18088 ** information to each allocation in order to help detect and fix memory
18092 ** routines specified in the sqlite3_mem_methods object.
18152 ** when this module is combined with other in the amalgamation.
18168 ** The number of levels of backtrace to save in new allocations.
18174 ** Title text to insert in front of each block
18410 ** allocation into a new place in memory. In this way, if the
18433 ** Populate the low-level memory allocation function pointers in
18434 ** sqlite3GlobalConfig.m with pointers to the routines in this file.
18463 ** Return TRUE if the mask of type in eType matches the type of the
18485 ** Return TRUE if the mask of type in eType matches no bits of the type of the
18607 ** The author disclaims copyright to this source code. In place of
18627 ** in the build only if SQLITE_ENABLE_MEMSYS3 is defined.
18641 ** Maximum size (in Mem3Blocks) of a "small" chunk.
18658 ** size of the allocation in blocks if the allocation is free.
18663 ** the previous chunk in blocks if the previous chunk is on the
18668 ** We often identify a chunk by its index in mem3.aPool[]. When
18670 ** the chunk. In this way, the first chunk has an index of 1.
18676 ** Pointers to the head of the list are stored in mem3.aiSmall[]
18687 u32 prevSize; /* Size of previous chunk in Mem3Block elements */
18688 u32 size4x; /* 4x the size of current chunk in Mem3Block elements */
18691 u32 next; /* Index in mem3.aPool[] of next free chunk */
18692 u32 prev; /* Index in mem3.aPool[] of previous free chunk */
18701 ** when this module is combined with other in the amalgamation.
18706 ** (in Mem3Blocks) pointed to by aPool less 2.
18728 ** occur off of this chunk. szMaster is the size (in Mem3Blocks)
18730 ** The master chunk is not in either the aiHash[] or aiSmall[].
18822 ** will already be held (obtained by code in malloc.c) if
18904 ** or same size hash. In other words, *pRoot is an entry in either
18911 ** the current mem3.iMaster with the new larger chunk. In order for
18951 ** Return a block of memory of at least nBytes in size.
18972 ** Look for an entry of the correct size in either the small
18973 ** chunk table or in the large chunk hash table. This is
19076 ** Return the size of an outstanding allocation, in bytes. The
19161 /* Store a pointer to the memory block in global structure mem3. */
19266 ** This routine is the only routine in this file with external
19269 ** Populate the low-level memory allocation function pointers in
19270 ** sqlite3GlobalConfig.m with pointers to the routines in this file. The
19297 ** The author disclaims copyright to this source code. In place of
19317 ** in the build only if SQLITE_ENABLE_MEMSYS5 is defined.
19328 ** This algorithm is described in: J. M. Robson. "Bounds for Some Functions
19358 ** verified in memsys5Init().
19383 ** when this module is combined with other in the amalgamation.
19389 int szAtom; /* Smallest possible allocation in bytes */
19390 int nBlock; /* Number of szAtom sized blocks in zPool */
19480 ** will already be held (obtained by code in malloc.c) if
19491 ** Return the size of an outstanding allocation, in bytes. The
19506 ** Return a block of memory of at least nBytes in size.
19513 ** threads can be in this routine at the same time.
19542 ** two in order to create a new free block of size iLogsize.
19588 /* Set iBlock to the index of the block pointed to by pOld in
19747 int nMinLog; /* Log base 2 of minimum allocation size in bytes */
19764 /* boundaries on sqlite3GlobalConfig.mnReq are enforced in sqlite3_config() */
19851 ** This routine is the only routine in this file with external
19876 ** The author disclaims copyright to this source code. In place of
20048 ** The author disclaims copyright to this source code. In place of
20058 ** This implementation in this file does not provide any mutual
20059 ** exclusion and is thus suitable for use only in applications
20060 ** that use SQLite in a single thread. The routines defined
20116 ** In this implementation, error checking is provided for testing
20202 ** be entered multiple times by the same thread. In such cases the,
20266 ** The author disclaims copyright to this source code. In place of
20279 ** The code in this file is only used if we are compiling threadsafe
20324 ** deliver incorrect results. In particular, if pthread_equal() is
20347 ** and also for the implementation of xShmBarrier in the VFS in cases
20393 ** not want to. But SQLite will only request a recursive mutex in
20396 ** might return such a mutex in response to SQLITE_MUTEX_FAST.
20499 ** be entered multiple times by the same thread. In such cases the,
20530 /* Use the built-in recursive mutexes if they are available.
20576 /* Use the built-in recursive mutexes if they are available.
20654 ** The author disclaims copyright to this source code. In place of
20670 /************** Include os_common.h in the middle of mutex_w32.c *************/
20675 ** The author disclaims copyright to this source code. In place of
20695 ** At least two bugs have slipped in because we changed the MEMORY_DEBUG
20713 /************** Include hwtime.h in the middle of os_common.h ****************/
20718 ** The author disclaims copyright to this source code. In place of
20802 /************** Continuing where we left off in os_common.h ******************/
20868 /************** Continuing where we left off in mutex_w32.c ******************/
20873 /************** Include os_win.h in the middle of mutex_w32.c ****************/
20878 ** The author disclaims copyright to this source code. In place of
20965 /************** Continuing where we left off in mutex_w32.c ******************/
20969 ** The code in this file is only used if we are compiling multithreaded
21022 ** and also for the xShmBarrier method of the VFS in cases when SQLite is
21081 /* Another thread is (in the process of) initializing the static
21134 ** not want to. But SQLite will only request a recursive mutex in
21137 ** might return such a mutex in response to SQLITE_MUTEX_FAST.
21220 ** be entered multiple times by the same thread. In such cases the,
21347 ** The author disclaims copyright to this source code. In place of
21364 ** cache database pages that are not currently in use.
21513 ** Return true if the heap is currently under memory pressure - in other
21640 ** xScratchMalloc(). We verify this constraint in the single-threaded
21651 ** This routine is similar to alloca() in that it is not intended
21686 ** This can only be checked in single-threaded mode.
21699 ** is outstanding at one time. (This is only checked in the
21700 ** single-threaded case since checking in the multi-threaded case
21798 ** Add the size of memory allocation "p" to the count in
21820 /* Trash all content in the buffer being freed */
21852 /* The 0x7ffff00 limit term is explained in comments on sqlite3Malloc() */
21919 ** the mallocFailed flag in the connection pointer.
21931 ** the mallocFailed flag in the connection pointer.
21937 ** This is an important assumption. There are many places in the
21944 ** In other words, if a subsequent malloc (ex: "b") worked, it is assumed
21989 ** resize fails, set the mallocFailed flag in the connection object.
22025 ** and set the mallocFailed flag in the database connection.
22037 ** Make a copy of a string in memory obtained from sqliteMalloc(). These
22040 ** called via macros that record the current file and line number in the
22072 ** Free any prior content in *pz and replace it with a copy of zNew.
22116 ** The "printf" code that follows dates from the 1980's. It is in
22141 /* The rest are extensions, not normally found in printf() */
22143 #define etSQLESCAPE2 11 /* Strings with '\'' doubled and enclosed in '',
22160 ** Each builtin conversion character (ex: the 'd' in "%d") is described
22232 ** 16 (the number of significant digits in a 64-bit float) '0' is
22292 int c; /* Next character in the format string */
22634 /* The sign in front of the number */
22689 /* The converted number is in buf[] and zero terminated. Output it.
22690 ** Note that the number is in the usual order, not reversed as with
22759 case etSQLESCAPE2: /* Escape ' and enclose in '...' */
22799 /* The precision in %q and %Q means how many input characters to
22993 ** zBase: An initial buffer. May be NULL in which case the initial buffer
22995 ** n: Size of zBase in bytes. If total space requirements never exceed
23084 ** are not able to use a "," as the decimal point in place of "." as
23119 ** We house it in a separate routine from sqlite3_log() to avoid using
23190 ** The author disclaims copyright to this source code. In place of
23203 ** The interfaces in this file is only available when compiling
23211 ** is not the last item in the tree.
23520 sqlite3TreeViewLine(pView, "IN");
23538 ** X is stored in pExpr->pLeft.
23539 ** Y is stored in pExpr->pList->a[0].pExpr.
23540 ** Z is stored in pExpr->pList->a[1].pExpr.
23554 ** to a column in the new.* or old.* pseudo-tables available to
23555 ** trigger programs. In this case Expr.iTable is set to 1 for the
23635 ** The author disclaims copyright to this source code. In place of
23646 ** Random numbers are used by some of the database backends in order
23670 ** we have to locate the state vector at run-time. In the more common
23705 ** Nothing in this file or anywhere else in SQLite does any kind of
23772 ** The author disclaims copyright to this source code. In place of
23790 ** or sqlite3ThreadJoin() call. This is, in fact, what happens in
23791 ** single threaded systems. Nothing in SQLite requires multiple threads.
23821 void *(*xTask)(void*), /* Routine to run in a separate thread */
23829 /* This routine is never used in single-threaded mode */
23892 void *pArg /* IN: Pointer to the SQLiteThread structure */
23916 void *(*xTask)(void*), /* Routine to run in a separate thread */
23982 ** work of the thread in the main thread, when either the thread is created
23996 void *(*xTask)(void*), /* Routine to run in a separate thread */
24049 ** The author disclaims copyright to this source code. In place of
24191 ** * Bytes in the range of 0x80 through 0xbf which occur as the first
24248 int len; /* Maximum length of output string in bytes */
24293 /* Set len to the maximum number of bytes required in the output buffer. */
24305 ** character. Two bytes are required in the output buffer for the
24383 ** UTF-16 string stored in *pMem. If one is present, it is removed and
24423 ** return the number of unicode characters in pZ up to (but not including)
24425 ** number of unicode characters in the first nByte of pZ (or up to
24446 ** Hence it is only available in debug builds.
24455 ** The translation is done in-place and aborted if the output
24476 ** Convert a UTF-16 string in the native encoding into a UTF-8 string.
24500 ** Return the number of bytes in the first nChar unicode characters
24501 ** in pZ. nChar must be non-negative.
24526 ** characters in each encoding are inverses of each other.
24582 ** The author disclaims copyright to this source code. In place of
24614 ** in places where it is difficult or expensive to do so purely by means
24651 ** -O option since it can result in incorrect output for programs
24678 ** Compute a string length that is limited to what can be stored in
24704 ** error string in the style of the printf functions: The following
24714 ** encoded in UTF-8.
24771 ** the quote characters. The conversion is done in-place. If the
24824 ** the contents of two buffers containing UTF-8 strings in a
24857 ** The string z[] is length bytes in length (bytes, not characters) and
24862 ** are in one of these formats:
24891 *pResult = 0.0; /* Default return value, in case of an error */
24982 /* In the IEEE 754 standard, zero is signed.
24997 ** and store in result. */
25049 ** in the values of the last digit if the only difference is in the
25088 ** length is the number of bytes in the string (bytes, not characters).
25141 /* Less than 19 digits, so we know that it fits in 64 bits */
25164 ** Transform a UTF-8 integer literal, in either decimal or hexadecimal,
25170 ** 0 Successful transformation. Fits in a 64-bit signed integer.
25196 ** If zNum represents an integer that will fit in 32-bits, then set
25338 ** inline in order to work around bugs in the RVT compiler.
25350 ** Return the number of bytes read. The value is stored in *v.
25425 /* we can skip these cause they were (effectively) done above in calc'ing s */
25446 /* we can skip this cause it was (effectively) done above in calc'ing s */
25513 ** Return the number of bytes read. The value is stored in *v.
25515 ** If the varint stored in p[0] is larger than can fit in a 32-bit unsigned
25566 /* A 32-bit varint is used to store size information in btrees.
25625 ** file. In that case we are not in any hurry. Use the (relatively
25748 ** misuse of the interface such as passing in db pointers that are
25792 ** the other 64-bit signed integer at *pA and store the result in *pA.
25793 ** Return 0 on success. Or if the operation would have resulted in an
25869 ** filename in zBaseFilename is a URI with the "8_3_names=1" parameter and
25870 ** if filename in z[] has a suffix (a.k.a. "extension") that is longer than
25927 ** Convert an integer into a LogEst. In other words, compute an
25946 ** In other words, compute an approximation for 10*log2(x).
25981 ** The author disclaims copyright to this source code. In place of
25990 ** used in SQLite.
26050 HashElem *pHead; /* First element already in pEntry */
26115 /* This function (for internal use only) locates an element in an
26117 ** also computed and returned in the *pH parameter.
26206 ** The key is not copied in this instance. If a malloc fails, then
26317 /* 60 */ "OpenPseudo" OpHelp("P3 columns in r[P2]"),
26388 /* 131 */ "RowSetTest" OpHelp("if r[P3] in rowset(P1) goto P2"),
26428 ** The author disclaims copyright to this source code. In place of
26440 ** There are actually several different VFS implementations in this file.
26441 ** The differences are in the way that file locking is done. The default
26449 ** in the correct division and should be clearly labeled.
26577 ** kernel modules in VxWorks. */
26594 ** cannot be closed immediately. In these cases, instances of the following
26644 ** transaction counter in bytes 24-27 of database files are updated
26652 unsigned char inNormalWrite; /* True if in a normal write operation */
26657 /* In test mode, increase the size of this structure a bit so that
26658 ** it is larger than the struct CrashFile defined in test6.c.
26691 /************** Include os_common.h in the middle of os_unix.c ***************/
26696 ** The author disclaims copyright to this source code. In place of
26716 ** At least two bugs have slipped in because we changed the MEMORY_DEBUG
26734 /************** Include hwtime.h in the middle of os_common.h ****************/
26739 ** The author disclaims copyright to this source code. In place of
26823 /************** Continuing where we left off in os_common.h ******************/
26889 /************** Continuing where we left off in os_unix.c ********************/
26938 ** Different Unix systems declare open() in different ways. Same use
26950 ** On some systems, calls to fchown() will trigger a message in a security
27177 ** Do not accept any file descriptor less than this value, in order to avoid
27197 ** In that way, if a database file is -rw-rw-rw or -rw-rw-r-, and a
27379 /* At one point this code was not commented out. In theory, this branch
27400 * introspection, in which it actually means what it says */
27418 * introspection, in which it actually means what it says */
27422 /* invalid fd, unless during file system support introspection, in which
27452 ** unique file ID in VxWorks. Each instance of this structure contains
27462 struct vxworksFileId *pNext; /* Next in a list of them all */
27483 ** Changes are made in-place. Return the new name length.
27485 ** The original filename is in z[0..n-1]. Return the number of
27486 ** characters in the simplified name.
27606 ** to synchronize access for threads in separate processes, but not
27612 ** st_dev and st_ino fields of the stat structure that fstat() fills in)
27620 ** canonical filename and implemented in the previous division.)
27659 ** in thread B. But there is no way to know at compile-time which
27668 ** per database file in the same process and (2) database connections
27724 ** It is invoked after an error occurs in an OS function and errno has been
27791 ** We assume that close() almost always works, since it is only in a
27809 ** Set the pFile->lastErrno. Do this in a subroutine as that provides
27817 ** Close all file descriptors accumuated in the unixInodeInfo->pUnused list.
27872 unixFile *pFile, /* Unix file with file desc used in the key */
27899 ** around this problem (we consider it a bug in OS X, not SQLite)
27903 ** in the header of every SQLite database. In this way, if there
28022 /* Check if a thread in this process holds such a lock */
28060 ** in order to coordinate access between separate database connections
28061 ** within this process, but all of that is handled in memory and the
28069 ** to fcntl() fails. In this case, errno is set appropriately (by fcntl()).
28109 ** are inserted in between. The locking might fail on one of the later
28125 ** lock transitions in terms of the POSIX advisory shared and exclusive
28128 ** slightly in order to be compatible with windows systems simultaneously
28129 ** accessing the same database file, in case that is ever required.
28131 ** Symbols defined in os.h indentify the 'pending byte' and the 'reserved
28283 /* We are trying for an exclusive lock but another thread in this
28368 ** around a bug in BSD NFS lockd (also seen on MacOSX 10.3+) that fails to
28406 ** the lock in 2 blocks, so that part of the range will be covered by a
28467 /* In theory, the call to unixFileLock() cannot fail because another
28471 ** SQLITE_BUSY would confuse the upper layer (in practice it causes
28493 ** OS call only when all threads in this same process have released
28632 ** corruption if this locking mode is used in situations where multiple
28677 ** Dotfile locking works by creating a subdirectory in the same directory as
28684 ** The file suffix added to the data base filename in order to create the
28695 ** In dotfile locking, either a lock exists or it does not. So in this
28708 /* Check if a thread in this process holds such a lock */
28710 /* Either this connection or some other connection in the same process
28733 ** are inserted in between. The locking might fail on one of the later
28863 ** flock() locking is like dot-file locking in that the various
28865 ** a single exclusive lock. In other words, SHARED, RESERVED, and
28903 /* Check if a thread in this process holds such a lock */
28957 ** are inserted in between. The locking might fail on one of the later
28969 ** lock states in the sqlite3_file structure, but all locks SHARED or
29073 ** Semaphore locking is like dot-lock and flock in that it really only
29095 /* Check if a thread in this process holds such a lock */
29110 /* someone else has the lock when we are in NO_LOCK */
29134 ** are inserted in between. The locking might fail on one of the later
29146 ** lock states in the sqlite3_file structure, but all locks SHARED or
29298 OSTRACE(("AFPSETLOCK [%s] for %d%s in range %llx:%llx\n",
29344 /* Check if a thread in this process holds such a lock */
29355 /* if we succeeded in taking the reserved lock, unlock it to restore
29384 ** are inserted in between. The locking might fail on one of the later
29511 /* We are trying for an exclusive lock but another thread in this
29649 ** OS call only when all threads in this same process have released
29744 ** methods were defined in divisions above (one locking method per
29756 ** in any form by default, we will not attempt to define _XOPEN_SOURCE.
29828 ** file), the bytes in the locking range should never be read or written. */
29878 int nBuf, /* Size of buffer pBuf in bytes */
29914 ** Seek to the offset in id->offset then read cnt bytes into pBuf.
29941 ** file), the bytes in the locking range should never be read or written. */
30145 ** The ability to override this routine was added in support of the
30285 ** Determine the current size of a file in bytes
30300 ** writes a single byte into that file in order to work around a bug
30301 ** in the OS-X msdos filesystem. In order to avoid problems with upper
30313 ** Handler for proxy-locking file-control verbs. Defined below in the
30321 ** file-control operation. Enlarge the database to nBytes in size
30348 ** single byte to the last byte in each block that falls entirely
30504 ** Return the sector size in bytes of the underlying block device for
30509 ** if two files are created in the same file-system directory (i.e.
30605 ** file system does not always provide powersafe overwrites. (In other
30608 ** very rare. And asserting PSOW makes a large reduction in the amount
30631 ** This function should not be called directly by other code in this file.
30653 ** of this unixShmNode object. In other words, each wal-index is opened
30658 ** every open file that does not use shared memory (in other words, most
30675 ** in this structure.
30808 ** to use 64KB pages - in this case each mapping must cover at least two
30854 ** The file used to implement shared-memory is in the same directory
30860 ** Another approach to is to use files in /dev/shm or /dev/tmp or an
30861 ** some other tmpfs mount. But if a file in a different directory
30866 ** in database corruption. Nevertheless, this tmpfs file usage
30869 ** option results in an incompatible build of SQLite; builds of SQLite
30873 ** "unsupported" and may go away in a future SQLite release.
30876 ** file are currently open, in this process or in other processes, then
30881 ** that no other processes are able to read or write the database. In
30891 int nShmFilename; /* Size of the SHM filename in bytes */
31017 ** bytes in size.
31022 ** region has not been allocated (by any client, including one running in a
31091 ** last page in order to extend the file. But writing to all new
31165 ** different here than in posix. In xShmLock(), one can go from unlocked
31374 ** the three variables above are zeroed. In this case SQLite should
31385 i64 nOrig = pFd->mmapSizeActual; /* Size of pOrig region in bytes */
31445 ** in this case. */
31503 ** If such a pointer can be obtained, store it in *pp and return SQLITE_OK.
31506 ** value of *pp is undefined in this case.
31608 ** to the METHOD object in the previous bullet.
31715 ** The proxy locking method is a "super-method" in the sense that it
31719 ** proxy locking is located much further down in the file. But we need
31886 ** exception is when opening the proxy "conch" file in builds that
31896 /* No locking occurs in temporary files */
31928 /* Cache zFilename in the locking context (AFP and dotlock override) for
31943 /* If an error occurred in findInodeInfo(), close the file descriptor
31945 ** in two scenarios:
31958 ** implicit assumption here is that if fstat() fails, things are in
31969 /* AFP locking uses the file path so it needs to be included in
31996 /* Dotfile locking uses the file path so it needs to be included in
32015 ** included in the semLockingContext
32056 ** Return the name of a directory in which to put temporary files.
32087 ** Create a temporary file name in zBuf. zBuf must be allocated
32131 ** Implementation in the proxy-lock division, but used by unixOpen()
32146 ** Refer to comments in the unixClose() function and the lengthy comment
32166 ** For this reason, if an error occurs in the stat() call here, it is
32202 ** In most cases, this routine sets *pMode to 0, which will become
32229 int nDb; /* Number of valid bytes in zDb */
32274 ** Previously, the SQLite OS layer used three functions in place of this
32296 sqlite3_file *pFile, /* The file descriptor to be filled in */
32331 ** a temporary file. Use this buffer to store the file name in.
32523 /* Use unixClose to clean up the resources added in fillInUnixFile
32638 ** is stored as a nul-terminated string in the buffer pointed to by
32642 ** (in this case, MAX_PATHNAME bytes). The full-path is written to
32648 int nOut, /* Size of output buffer in bytes */
32749 ** uninitialized space in zBuf - but valgrind errors tend to worry
32752 ** in the random seed.
32816 SQLITE_API int sqlite3_current_time = 0; /* Fake system time in seconds since 1970. */
32820 ** Find the current time (in Universal Coordinated Time). Write into *piNow
32821 ** the current time and date as a Julian Day number times 86_400_000. In
32823 ** epoch of noon in Greenwich on November 24, 4714 B.C according to the
32859 ** Find the current time (in Universal Coordinated Time). Write the
32876 ** in the core. So this routine is never called. For now, it is merely
32894 ** Proxy locking is a "uber-locking-method" in this sense: It uses the
32898 ** until late in the file (here) after all of the other I/O methods have
32904 ** The default locking schemes in SQLite use byte-range locks on the
32912 ** address in the shared range is taken for a SHARED lock, the entire
32920 ** slowdown in read performance on AFP because the AFP client disables
32953 ** host ID in it, the proxy path in the conch file will be used, otherwise
32982 ** proxy path against the values stored in the conch. The conch file is
32983 ** stored in the same directory as the database file and the file name
33048 ** and local proxy files in it
33107 ** Creates the lock file and any missing directories in lockPath
33143 ** Create a new VFS file descriptor (stored in memory obtained from
33144 ** sqlite3_malloc) and open the file named "path" in the file descriptor.
33239 /* Not always defined in the headers as it ought to be */
33420 ** lockPath means that the lockPath in the conch file will be used if the
33470 /* if the host id matches and the lock path already exists in the conch
33474 do { /* in case we need to try again for an :auto: named lock file */
33482 ** use the local lock file path that's already in there
33526 /* We are trying for an exclusive lock but another thread in this
33638 } while (1); /* in case we need to retry the :auto: lock file -
33667 ** Store the conch filename in memory obtained from sqlite3_malloc64().
33679 char *conchPath; /* buffer in which to construct conch name */
33744 ** is a string buffer at least MAXPATHLEN+1 characters in size.
33752 /* afp style keeps a reference to the db path in the filePath field
33773 ** Takes an already filled in unix file and alters it so all file locking
33775 ** are preserved in the locking context so that they can be restored and
33965 ** are inserted in between. The locking might fail on one of the later
34076 ** should be the only routines in this file that are visible from other
34131 ** All default VFSes for unix are contained in the following array.
34169 /* Register all VFSes defined in the aVfs[] array */
34179 ** Some operating systems might need to do some cleanup in this routine,
34194 ** The author disclaims copyright to this source code. In place of
34211 /************** Include os_common.h in the middle of os_win.c ****************/
34216 ** The author disclaims copyright to this source code. In place of
34236 ** At least two bugs have slipped in because we changed the MEMORY_DEBUG
34254 /************** Include hwtime.h in the middle of os_common.h ****************/
34259 ** The author disclaims copyright to this source code. In place of
34343 /************** Continuing where we left off in os_common.h ******************/
34409 /************** Continuing where we left off in os_win.c *********************/
34418 ** available in Windows platforms based on the NT kernel.
34469 ** target system. GetVersionEx was first deprecated in Win8.1.
34480 ** This constant should already be defined (in the "WinDef.h" SDK file).
34487 ** Maximum pathname length (in chars) for Win32. This should normally be
34495 ** This constant should already be defined (in the "WinNT.h" SDK file).
34502 ** Maximum pathname length (in chars) for WinNT. This should normally be
34510 ** Maximum pathname length (in bytes) for Win32. The MAX_PATH macro is in
34519 ** Maximum pathname length (in bytes) for WinNT. This should normally be
34528 ** Maximum error message length (in chars) for WinRT.
34559 ** mode or memory mapped files (e.g. these APIs are available in the Windows
34560 ** CE SDK; however, they are not present in the header file)?
34711 * data will almost certainly result in an immediate access violation.
34734 * The extra flags to use in calls to the Win32 heap APIs. This value may be
34805 ** In order to facilitate testing on a WinNT system, the test fixture
35576 ** largest committed free block in the heap, in bytes.
35648 ** The Win32 native heap cannot be modified because it may be in use.
35860 ** Return the size of an outstanding allocation, in bytes.
35966 ** Populate the low-level memory allocation function pointers in
35967 ** sqlite3GlobalConfig.m with pointers to the routines in this file. The
36138 ** the provided arguments. The type argument must be 1 in order to set the
36139 ** data directory or 2 in order to set the temporary directory. The zValue
36175 ** is zero if the error message fits in the buffer, or non-zero
36258 ** This routine is invoked after an error occurs in an OS function.
36312 ** non-zero if the error code is transient in nature and the operation
36319 ** may be used to include additional error codes in the set that should
36320 ** result in the failing I/O operation being retried by the caller. If
36427 ** Create the mutex and shared memory used for locking in the file
36483 /* If we succeeded in making the shared memory handle, map it. */
36529 /* The following blocks should probably assert in debug mode, but they
36530 are to cleanup in case any locks remained open */
37223 ** Determine the current size of a file in bytes
37369 ** are inserted in between. The locking might fail on one of the later
37507 /* Update the state of the lock has held in the file descriptor then
37734 ** Return the sector size in bytes of the underlying block device for
37739 ** if two files are created in the same file-system directory (i.e.
37796 ** point to a single instance of this object. In other words, each
37812 ** in this structure.
37830 winShmNode *pNext; /* Next in list of all winShmNode objects */
37973 ** file are currently open, in this process or in other processes, then
37981 int nName; /* Size of zName in bytes */
38250 ** bytes in size.
38255 ** region has not been allocated (by any client, including one running in a
38544 ** If such a pointer can be obtained, store it in *pp and return SQLITE_OK.
38547 ** value of *pp is undefined in this case.
38688 ** operating system wants filenames in. Space to hold the result
38729 ** The pointer returned in pzBuf must be freed via sqlite3_free().
38926 ** name in the following format:
38930 ** If not, return SQLITE_ERROR. The number 17 is used here in order to
39006 void *zConverted; /* Filename in OS encoding */
39007 const char *zUtf8Name = zName; /* Filename in UTF-8 encoding */
39011 ** a temporary file. Use this buffer to store the file name in.
39273 ** will open a journal file shortly after it is created in order to do
39483 ** a legal UNC name, a volume relative path, or an absolute path name in the
39512 ** bytes in size.
39517 int nFull, /* Size of output buffer in bytes */
39839 SQLITE_API int sqlite3_current_time = 0; /* Fake system time in seconds since 1970. */
39843 ** Find the current time (in Universal Coordinated Time). Write into *piNow
39844 ** the current time and date as a Julian Day number times 86_400_000. In
39846 ** epoch of noon in Greenwich on November 24, 4714 B.C according to the
39861 /* 2^32 - to avoid use of LL and warnings in gcc */
39891 ** Find the current time (in Universal Coordinated Time). Write the
39917 ** is zero if the error message fits in the buffer, or non-zero
39920 ** in the output buffer.
40035 ** The author disclaims copyright to this source code. In place of
40051 ** or all of the pages in a database can get journalled. In those cases,
40064 ** Bitvec object is the number of pages in the database file at the
40070 /* Size of the Bitvec structure in bytes. */
40082 /* Size, in bits, of the bitmap element. */
40084 /* Number of elements in a bitmap array. */
40086 /* Number of bits in the bitmap array. */
40089 /* Number of u32 values in hash table. */
40091 /* Maximum number of entries in hash table before
40096 ** (an arbitrary prime)in the hash function provided
40230 /* in hash, if not, try to find a spot for it */
40236 /* we didn't find it in the hash. h points to the first */
40353 ** 5 N S X Set N bits from S increment X in array only, not in bitvec
40358 ** Opcode 5 is used to deliberately induce a fault in order to
40448 ** The author disclaims copyright to this source code. In place of
40464 PgHdr *pDirty, *pDirtyTail; /* List of dirty pages in LRU order */
40465 PgHdr *pSynced; /* Last synced page in dirty page list */
40468 int szPage; /* Size of every page in this cache */
40548 ** being used for an in-memory database, this function is a no-op.
40583 ** built-in default page cache is used instead of the application defined
40597 ** Return the size in bytes of a PCache object.
40603 ** has already been allocated and is passed in as the p pointer.
40653 ** such an object is already in cache, or if a new one is created.
40654 ** This routine returns a NULL pointer if the object was not in cache
40661 ** is not already in the cache. If createFlag is 1, then a new page
40753 ** In the uncommon case where the page being fetched has not been
40756 ** requires extra stack manipulation that can be avoided in the common
40782 ** must be called after sqlite3PcacheFetch() in order to get a usable
40876 ** Make every page in the cache clean.
40938 if( ALWAYS(pPage1) ){ /* Page 1 is always available in cache, because
40964 ** Merge two lists of pages connected by pDirty and in pgno order.
40992 ** Sort the list of pages in accending order by pgno. Pages are
40996 ** Since there cannot be more than 2^31 distinct pages in a database,
40998 ** One extra bucket is added to catch overflow in case something
41020 /* To get here, there need to be 2^(N_SORT_BUCKET) elements in
41034 ** Return a list of all dirty pages in the cache, sorted by page number.
41062 ** Return the total number of pages in the cache.
41098 ** in the page-cache hierarchy.
41105 ** For all dirty pages currently in the cache, invoke the specified
41122 ** The author disclaims copyright to this source code. In place of
41163 ** fragmentation" in academic literature) of allocating a few bytes more
41212 ** in memory.
41217 u8 isPinned; /* Page in use, not on the LRU list */
41220 PgHdr1 *pNext; /* Next in hash table chain */
41222 PgHdr1 *pLruNext; /* Next in LRU list of unpinned pages */
41223 PgHdr1 *pLruPrev; /* Previous in LRU list of unpinned pages */
41231 ** This page cache implementation works in one of two modes:
41241 ** and is therefore often faster. Mode 2 requires a mutex in order to be
41258 ** open database file (including each in-memory database and each
41284 unsigned int nRecyclable; /* Number of pages in the LRU list */
41285 unsigned int nPage; /* Total number of pages in apHash */
41286 unsigned int nHash; /* Number of slots in apHash[] */
41293 ** Free slots in the allocator used to divide up the global page cache
41330 ** All code in this file should access the global structure above via the
41424 ** such buffer exists or there is no space left in it, this function falls
41428 ** in pcache1 need to be protected via mutex.
41447 /* Memory is not available in the SQLITE_CONFIG_PAGECACHE pool. Get
41615 ** allocating a new page cache entry in order to avoid stressing
41699 ** (PCache1.apHash structure) that it is currently stored in.
41787 ** PGroup in pcache1.grp is used for all page caches (mode-2).
41791 ** * Use a unified cache in single-threaded applications that have
41828 ** Note that the static mutex allocated in xInit does
41940 ** in the header of the pcache1Fetch() procedure.
42025 ** For a non-purgeable cache (a cache used as the storage for an in-memory
42036 ** 2. If createFlag==0 and the page is not already in the cache, NULL is
42039 ** 3. If createFlag is 1, and the page is not already in the cache, then
42086 /* Step 2: If the page was found in the hash table, then return it.
42087 ** If the page was not in the hash table and createFlag is 0, abort.
42088 ** Otherwise (page not in hash and createFlag!=0) continue with
42222 ** Discard all unpinned pages in the cache with a page number equal to
42299 ** held by the pager system. Memory in use by any SQLite pager allocated
42360 ** The author disclaims copyright to this source code. In place of
42372 ** are inserted into the RowSet in an arbitrary order. Inserts
42377 ** elements of the RowSet in sorted order. Once this extraction
42390 ** TEST checks to see if an element is already in the RowSet. SMALLEST
42394 ** allocated in chunks so most INSERTs do no allocation. There is an
42400 ** in the batch number. In other words, if an INSERT occurs between
42411 ** batch number is O(NlogN) where N is the number of elements in the RowSet.
42434 ** Each entry in a RowSet is an instance of the following object.
42437 ** objects. In that alternative use, pRight points to the next entry
42438 ** in the list, pLeft points to the tree, and v is unused. The
42448 ** RowSetEntry objects are allocated in large chunks (instances of the
42459 ** A RowSet in an instance of the following structure.
42461 ** A typedef of this structure if found in sqliteInt.h.
42533 ** In an OOM situation, the RowSet.db->mallocFailed flag is set and this
42586 ** assumed to each already be in sorted order.
42716 ** as deep as it needs to be in order to contain the entire list.
42738 ** Take all the entries on p->pEntry and on the trees in p->pForest and
42741 ** This routine should only be called once in the life of a RowSet.
42848 /* Test to see if the iRowid value appears anywhere in the forest.
42871 ** The author disclaims copyright to this source code. In place of
42890 /************** Include wal.h in the middle of pager.c ***********************/
42895 ** The author disclaims copyright to this source code. In place of
42905 ** the implementation of each function in
42956 ** at an instant in time. sqlite3WalOpenSnapshot gets a read lock and
42979 ** position in the WAL */
42982 /* Move the write position of the WAL back to iFrame. Called in
42998 int *pnLog, /* OUT: Number of frames in WAL */
42999 int *pnCkpt /* OUT: Number of backfilled frames in WAL */
43003 ** number of frames in the WAL at the point of the last commit since
43022 ** stored in each frame (i.e. the db page-size when the WAL was created).
43031 /************** Continuing where we left off in pager.c **********************/
43054 ** (c) The page number is greater than the largest page that existed in
43067 ** both the content in the database when the rollback journal was written
43068 ** and the content in the database at the beginning of the current
43072 ** in length and are aligned on a page boundary.
43075 ** an integer multiple of the page size in length or are taken from the
43086 ** all queries. Note in particular the content of freelist leaf
43097 ** the beginning of the transaction. (In some VFSes, the xTruncate
43101 ** (9) Whenever the database file is modified, at least one bit in the range
43106 ** (10) The pattern of bits in bytes 24 through 39 shall not repeat in less
43144 ** pager may be in any one of the seven states shown in the following
43182 ** The pager starts up in this state. Nothing is guaranteed in this
43192 ** In this state all the requirements for reading the database in
43194 ** was) in exclusive-locking mode, a user-level read transaction is
43195 ** open. The database size is known in this state.
43200 ** running in locking_mode=exclusive (including temp databases) remains in
43212 ** there is no hot-journal in the file-system.
43217 ** is first opened on the database. In WRITER_LOCKED state, all locks
43221 ** In rollback mode, a RESERVED or (if the transaction was opened with
43224 ** to in this state. If the transaction is committed or rolled back while
43225 ** in WRITER_LOCKED state, all that is required is to unlock the database
43228 ** IN WAL mode, WalBeginWriteTransaction() is called to lock the log file.
43233 ** * If the connection is open in rollback-mode, a RESERVED or greater
43235 ** * If the connection is open in WAL-mode, a WAL write transaction
43246 ** first modified by the upper layer. In rollback mode the journal file
43276 ** database file. In this state the transaction may be committed simply
43277 in WRITER_FINISHED state, it is
43291 ** SQLITE_IOERR_NOMEM) occurs at a point in the code that makes it
43292 ** difficult to be sure that the in-memory pager state (cache contents,
43295 ** Temporary pager files may enter the ERROR state, but in-memory pagers
43299 ** the contents of the page-cache may be left in an inconsistent state.
43311 ** page-cache and any other in-memory state at the same time. Everything
43319 ** 1. An error occurs while attempting a rollback. This happens in
43323 ** following a commit in function sqlite3PagerCommitPhaseTwo().
43326 ** database file in function pagerStress() in order to free up
43329 ** In other cases, the error is returned to the b-tree layer. The b-tree
43334 ** statement executed within a transaction. In this case, if the error
43336 ** automatically attempt a rollback, as it assumes that an error in a
43337 ** read-only statement cannot leave the pager in an internally inconsistent
43343 ** * The pager is not an in-memory pager.
43348 ** * A pager is never in WRITER_DBMOD or WRITER_FINISHED state if the
43349 ** connection is open in WAL mode. A WAL connection is always in one
43352 ** * Normally, a connection open in exclusive mode is never in PAGER_OPEN
43376 ** the operation was successful. In these circumstances pagerLockDb() and
43389 ** in the file-system that needs to be rolled back (as part of an OPEN->SHARED
43398 ** doesn't know it because of a previous error in xUnlock). If this happens
43400 ** transaction in another process, causing SQLite to read from the database
43404 ** database in the ERROR state, Pager.eLock is set to UNKNOWN_LOCK. It
43412 ** Pager.eLock may only be set to UNKNOWN_LOCK when the pager is in
43441 ** savepoint and statement transaction in the system. All such structures
43442 ** are stored in the Pager.aSavepoint[] array, which is allocated and
43454 i64 iOffset; /* Starting offset in main journal */
43456 Bitvec *pInSavepoint; /* Set of pages in this savepoint */
43457 Pgno nOrig; /* Original number of pages in file */
43458 Pgno iSubRec; /* Index of first record in sub-journal */
43485 ** For a temporary or in-memory database (neither of which require any
43491 ** In some (obscure) circumstances, this variable may also be set to
43508 ** This mechanism means that when running in exclusive mode, a connection
43519 ** the way in which the journal file is finalized after the transaction is
43520 ** committed or rolled back when running in "journal_mode=PERSIST" mode.
43525 ** running in "journal_mode=truncate" mode.
43541 ** to the file-system in order to free up memory).
43545 ** The SPILLFLAG_ROLLBACK case is done in a very obscure case that
43548 ** while it is being traversed by code in pager_playback(). The SPILLFLAG_OFF
43554 ** is larger than the database page-size in order to prevent a journal sync
43555 ** from happening in between the journalling of two pages on the same sector.
43560 ** is opened as an in-memory journal file. If false, then in-memory
43561 ** sub-journals are only used for in-memory pager files.
43568 ** Variable dbSize is set to the number of pages in the database file.
43569 ** It is valid in PAGER_READER and higher states (all states except for
43575 ** is not an integer multiple of the page-size, the value stored in
43577 ** Except, any file that is greater than 0 bytes in size is considered
43582 ** dbSize are modified in the cache, dbSize is updated accordingly.
43586 ** Variables dbOrigSize and dbFileSize are valid in states
43593 ** the file on disk in pages. It is set to a copy of dbSize when the
43603 ** dbFileSize is not used when rolling back a transaction. In this case
43605 ** a call to xFilesize() that is not strictly required). In either case,
43622 ** The Pager.errCode variable is only ever used in PAGER_ERROR state. It
43623 ** is set to zero in all other states. In PAGER_ERROR state, Pager.errCode
43638 u8 noLock; /* Do not lock (except in WAL mode) */
43644 ** routine operation. Class members not in this block are either fixed
43656 u8 subjInMemory; /* True to use in-memory sub-journals */
43659 Pgno dbSize; /* Number of pages in the database */
43661 Pgno dbFileSize; /* Number of pages in the database file */
43667 Bitvec *pInJournal; /* One bit for each page in the database file */
43671 i64 journalOff; /* Current write offset in the journal file */
43675 int nSavepoint; /* Number of elements in aSavepoint[] */
43686 u16 nExtra; /* Add this many bytes to each in-memory page */
43690 int pageSize; /* Number of bytes in a page */
43727 ** testing purposes only. These variables do not exist in
43747 ** written, semi-random garbage data might appear in the journal
43750 ** sanity checking data is an attempt to discover the garbage in the
43756 ** This cksum is initialized to a 32-bit random value that appears in the
43759 ** data that was once in other files that have now been deleted. If the
43769 ** The size of the of each page record in the journal is given by
43781 ** The macro MEMDB is true if we are dealing with an in-memory database.
43843 ** This function runs many asserts to try to find inconsistencies in
43872 /* Check that MEMDB implies noSync. And an in-memory journal. Since
43873 ** this means an in-memory pager performs no IO at all, it cannot encounter
43875 ** a journal file. (although the in-memory journal implementation may
43877 ** is therefore not possible for an in-memory pager to enter the ERROR
43963 ** in ERROR state. Otherwise the pager should have already dropped
43977 ** Return a pointer to a human readable string in a static buffer
43980 ** to "print *pPager" in gdb:
44032 ** * The bit corresponding to the page-number is not set in
44051 ** Return true if the page is already in the journal file.
44060 ** that is read in *pRes. Return SQLITE_OK if everything worked, or an
44075 ** Write a 32-bit integer into a string buffer in big-endian byte order.
44150 ** an error to call this function if pPager is opened on an in-memory
44234 ** name in the journal is longer than nMaster bytes (including a
44236 ** were present in the journal.
44251 u32 len; /* Length in bytes of master journal name */
44252 i64 szJ; /* Total size in bytes of journal file pJrnl */
44290 ** following the value in pPager->journalOff, assuming a sector
44322 ** set to 0, then truncate the journal file to zero bytes in size. Otherwise,
44323 ** zero the 28-byte header at the start of the journal file. In either case,
44324 ** if the pager is not in no-sync mode, sync the journal file immediately
44329 ** journal file in bytes is larger than this value, then truncate the
44377 ** - 4 bytes: Number of records in journal, or -1 no-sync mode is on.
44414 ** if in full-sync mode), the zero is overwritten with the true number
44424 ** * When the pager is in no-sync mode. Corruption can follow a
44425 ** power failure in this case anyway.
44459 /* In theory, it is only necessary to write the 28 bytes that the
44462 in the file
44463 ** that will be implicitly filled in by the OS).
44471 ** The loop is required here in case the sector-size is larger than the
44473 ** bytes in size, more than one call to sqlite3OsWrite() may be required
44488 ** (JOURNAL_HDR_SZ bytes) is read from the current location in the journal
44489 ** file. The current location in the journal file is given by
44495 ** database before the transaction began, in pages. Also, pPager->cksumInit
44497 ** in this case.
44506 i64 journalSize, /* Size of the open journal file in bytes */
44526 /* Read in the first 8 bytes of the journal header. If they do not match
44564 ** journal header to zero. In this case, assume that the Pager.pageSize
44572 ** are within range. To be 'in range', both values need to be a power
44580 /* If the either the page-size or sector-size in the journal-header is
44582 ** crashed before the header was synced. In this case stop reading
44612 ** thing written to a journal file. If the pager is in full-sync mode, the
44617 ** + N bytes: Master journal filename in utf-8.
44618 ** + 4 bytes: N (length of master journal name in bytes, no nul-terminator).
44622 ** The master journal page checksum is the sum of the bytes in the master
44631 i64 iHdrOff; /* Offset of header in journal file */
44647 /* Calculate the length in bytes and the checksum of zMaster */
44652 /* If in full-sync mode, advance to the next disk sector before writing
44653 ** the master journal name. This is in case the previous page written to
44675 /* If the pager is in peristent-journal mode, then the physical
44682 ** Easiest thing to do in this scenario is to truncate the journal
44694 ** Discard the entire contents of the in-memory page-cache.
44711 ** Free all structures in the Pager.aSavepoint[] array and set both
44713 ** if it is open and the pager is not in exclusive mode.
44730 ** Set the bit number pgno in the PagerSavepoint.pInSavepoint
44750 ** This function is a no-op if the pager is in exclusive mode and not
44751 ** in the ERROR state. Otherwise, it switches the pager to PAGER_OPEN
44754 ** If the pager is not in exclusive-access mode, the database file is
44759 ** If the pager is in ERROR state when this function is called, the
44761 ** the OPEN state. Regardless of whether the pager is in exclusive-mode
44762 ** or not, any journal file left in the file-system will be treated
44802 /* If the pager is in the ERROR state and the call to unlock the database
44814 ** code is cleared and the cache reset in the block below.
44823 ** it can safely move back to PAGER_OPEN state. This happens in both
44849 ** is stored in Pager.errCode. While the pager remains in the ERROR state,
44883 ** This routine is never called in PAGER_ERROR state. If it is called
44884 ** in PAGER_NONE or PAGER_SHARED state and the lock held is less
44893 ** depends on whether or not the pager is running in exclusive mode and
44898 ** in-memory journal.
44901 ** Journal file is truncated to zero bytes in size.
44905 ** the first journal header in the file, and hence the entire journal
44911 ** If the pager is running in exclusive mode, this method of finalizing
44913 ** DELETE and the pager is in exclusive mode, the method described under
44917 ** If running in non-exclusive rollback mode, the lock on the file is
44924 ** tries to unlock the database file if not in exclusive mode. If the
44983 ** a hot-journal was just rolled back. In this case the journal
44985 ** the database file, it will do so using an in-memory journal.
45017 /* Drop the WAL write-lock, if any. Also, if the connection was in
45024 /* This branch is taken when committing a transaction in rollback-journal
45057 ** call to pager_unlock() will discard all in-memory pages, unlock
45059 ** means that there is a hot-journal left in the file-system, the next
45093 ** Changing the formula used to compute this checksum results in an
45129 ** Make sure the number of reserved bits is the same in the destination
45130 ** pager as it is in the source. This comes up when a VACUUM changes the
45145 ** value is increased to the start of the next page in the journal.
45165 ** corrupted, SQLITE_DONE is returned. Data is considered corrupted in
45186 PgHdr *pPg; /* An existing page in the cache */
45187 Pgno pgno; /* The page number of a page in journal */
45205 ** is in state OPEN and holds an EXCLUSIVE lock. Hot-journal rollback
45257 /* If the pager is in CACHEMOD state, then there must be a copy of this
45258 ** page in the pager cache. In this case just update the pager cache,
45259 ** not the database file. The page is left marked dirty in this case.
45261 ** An exception to the above rule: If the database is in no-sync mode
45263 ** not be in the pager cache. Later: if a malloc() or IO error occurs
45264 ** during a Movepage() call, then the page may not be in the cache
45265 ** either. So the condition described in the above paragraph is not
45268 ** If in WRITER_DBMOD, WRITER_FINISHED or OPEN state, then we update the
45270 ** not dirty. Since this code is only executed in PAGER_OPEN state for
45272 ** if the pager is in OPEN state.
45280 ** journal. Otherwise, a power loss might leave modified data in the
45281 ** database file without an entry in the rollback journal that can
45285 ** in the main journal either because the page is not in cache or else
45326 ** the database and the page is not in-memory, there is a potential
45332 ** obscure. When running in synchronous mode, this can only happen
45336 ** The solution is to add an in-memory page to the cache containing
45352 /* No page should ever be explicitly rolled back that is in use, except
45353 ** for page 1 which is held in use in order to keep the lock on the
45355 ** of an internal error resulting in an automatic call to
45365 ** transaction was first opened. In this case we can mark the page
45374 ** already in the journal file (recorded in Pager.pInJournal) and
45469 ** sufficient space (in zMasterPtr) to hold the names of master
45535 ** file in the file-system. This only happens when committing a transaction,
45538 ** If the main database file is not open, or the pager is not in either
45617 ** might change if a crash occurs while writing to a single byte in
45631 ** may not have been opened yet, in which case the OsSectorSize()
45641 ** the state it was in before we started making changes.
45647 ** in the journal. If this value is 0xffffffff, then compute the
45654 ** is this many bytes in size.
45663 ** Each entry in the journal is an instance of the 8th item.
45666 ** valid page entries in the journal. In most cases, you can compute the
45670 In such a case,
45672 ** that reason, we always use the nRec value in the header.
45677 ** in this case. But for things like temporary table (which will be
45698 i64 szJ; /* Size of the journal file in bytes */
45699 u32 nRec; /* Number of Records in the journal */
45701 Pgno mxPg = 0; /* Size of the original file in pages */
45708 /* Figure out how many records are in the journal. Abort early if
45746 ** not enough bytes left in the journal file for a complete header, or
45759 ** working in no-sync mode. This means that the rest of the journal
45769 ** process and if this is the final header in the journal, then it means
45777 ** when doing a ROLLBACK and the nRec==0 chunk is the last chunk in
45816 ** not completely written and synced prior to a crash. In that
45817 ** case, the database should have never been written in the
45836 /* Following a rollback, the database file should be back in its original
45850 ** modification may just have been reverted. If this happens in exclusive
45853 ** problems for other processes at some point in the future. So, just
45854 ** in case this has happened, clear the changeCountDone flag now.
45932 ** zero or the size of the database in page. Bytes 32..35 and 35..39
45959 ** Update the value of the change-counter at offsets 24 and 92 in
45973 /* Also store the SQLite version number in bytes 96..99 and in
45987 ** If page iPg is present in the cache, and has no outstanding references,
46036 /* For all pages in the cache that are currently dirty or have already
46071 int nList; /* Number of pages in pList */
46077 /* Verify that the page list is in accending order */
46085 /* If a WAL transaction is being committed, there is no point in writing
46127 ** makes a snapshot of the database at the current point in time and preserves
46128 ** that snapshot for use by the reader in spite of concurrently changes by
46139 ** transaction in locking_mode=EXCLUSIVE. So call it now. If we
46140 ** are in locking_mode=NORMAL and EndRead() was previously called,
46158 ** in pages (assuming the page size currently stored in Pager.pageSize).
46161 ** in pages is stored in *pnPage. Otherwise, an error code (perhaps
46177 /* If the number of pages in the database is not available from the
46183 i64 n = 0; /* Size of db file in bytes */
46194 /* If the current number of pages in the file is greater than the
46213 ** in WAL mode. If the database is empty or if no *-wal file exists and
46266 ** performed in the order specified:
46282 ** corresponding bit is set in a bitvec structure (variable pDone in the
46284 ** rolled back the first time it is encountered in either journal.
46287 ** journal file. There is no need for a bitvec in this case.
46289 ** In either case, before playback commences the Pager.dbSize variable
46322 ** journal. The actual file might be larger than this in
46331 ** There might be records in the main journal that have a page number
46361 ** test is related to ticket #2565. See the discussion in the
46377 ** previously rolled back out of the main journal (and are hence in pDone)
46403 ** Change the maximum number of in-memory pages that are allowed.
46440 ** Adjust settings of the pager to those specified in the pgFlags parameter.
46442 ** The "level" in pgFlags & PAGER_SYNCHRONOUS_MASK sets the robustness
46454 ** in a state which would cause damage to the database
46459 ** of the journal header - being written in between the two
46470 ** an ordinary commit in NORMAL mode with WAL. FULL means that the WAL
46471 ** file is synced following each commit operation, in addition to the
46600 ** is passed in *pPageSize.
46602 ** If the pager is in the error state when this function is called, it
46613 ** * the database is either not an in-memory database or it is
46614 ** an in-memory database that currently consists of zero pages.
46621 ** In all other cases, SQLITE_OK is returned.
46624 ** conditions above is not true, the pager was in error state when this
46635 ** At one point this function returned an error if the pager was in
46714 ** errors in places where we do not care about errors.
46740 ** opened on a file less than N bytes in size, the output buffer is
46772 ** the pager. It returns the total number of pages in the database.
46774 ** However, if the file is between 1 and <page-size> bytes in size, then
46819 ** following is true for all dirty pages currently in the page-cache:
46822 ** current database image, in pages, OR
46833 ** content of the page. However, since this content is not present in either
46852 ** Truncate the in-memory database file image to nPage pages. This
46869 ** if one or more savepoints are open, present in the savepoint
46967 ** Free all PgHdr objects stored in the Pager.pMmapFreelist list.
46982 ** If a transaction was in progress when this routine is called, that
46986 ** result in a coredump.
46990 in the filesystem but no error is returned
47063 ** Sync the journal. In other words, make sure all the pages that have
47065 ** disk and can be restored in the event of a hot-journal rollback.
47071 ** * If the journal file is an in-memory journal file, no action need
47077 ** been written following it. If the pager is operating in full-sync
47083 ** Or, in pseudo-code:
47085 ** if( NOT <in-memory journal> ){
47094 ** page currently held in memory before returning SQLITE_OK. If an IO
47117 ** that wrote to this database was operating in persistent-journal
47119 ** than Pager.journalOff bytes. If the next thing in the journal
47155 /* Write the nRec value into the journal file header. If in
47161 ** SAFE_APPEND property. Because in this case it is not possible
47198 /* Unless the pager is in noSync mode, the journal file was just
47209 ** The argument is the first in a linked list of dirty pages connected
47211 ** in-memory pages in the list to the database file. The argument may
47212 ** be NULL, representing an empty list. In this case this function is
47225 ** the pages are written out to the database file in list order. Writing
47233 ** in Pager.dbFileVers[] is updated to match the new value stored in
47243 /* This function is only called for rollback pagers in WRITER_DBMOD state. */
47273 /* If there are dirty pages in the page cache with page numbers greater
47295 ** the value now stored in the database file. If writing this
47346 ** If successful, set the bit corresponding to pPg->pgno in the bitvecs
47351 ** SQLITE_NOMEM if a malloc fails while setting a bit in a savepoint
47402 ** (cast as a void*). The pager is always 'purgeable' (not an in-memory
47434 ** Spilling is also prohibited when in an error state since that could
47435 ** lead to database corruption. In the current implementation it
47437 ** while in the error state, hence it is impossible for this routine to
47438 ** be called in the error state. Nevertheless, we include a NEVER()
47487 ** in *ppPager. The pager should eventually be freed by passing it
47494 ** all information is held in cache. It is never written to disk.
47495 ** This can be used to implement an in-memory database.
47519 int nExtra, /* Extra bytes append to each in-memory page */
47527 int tempFile = 0; /* True for temp files (incl. in-memory files) */
47528 int memDb = 0; /* True if this is an in-memory file */
47532 int nPathname = 0; /* Number of bytes in zPathname */
47541 ** is the maximum space required for an in-memory journal file handle
47544 ** file in memory to implement the atomic-write optimization (see
47553 /* Set the output variable to NULL in case an error occurs. */
47568 /* Compute and store the full pathname in an allocated buffer pointed
47592 ** bytes in length. This means the database cannot be opened,
47606 ** file name. The layout in memory is as follows:
47640 /* Fill in the Pager.zFilename and Pager.zJournal buffers, if required. */
47669 ** choose a default page size in case we have to create the
47711 ** In this case we accept the default page size and delay actually
47714 ** This branch is also run for an in-memory database. An in-memory
47716 ** disk and uses an in-memory rollback journal.
47723 pPager->eLock = EXCLUSIVE_LOCK; /* Pretend we are in EXCLUSIVE mode */
47838 ** PAGER_SHARED state. It tests if there is a hot journal present in
47843 ** * The journal file exists in the file system, and
47845 ** * The database file itself is greater than 0 bytes in size, and
47850 ** database with the same name. In this case the journal file is
47856 ** does not exist, then the journal file is not really hot. In this
47893 ** in fact there is none. This results in a false-positive which will
47898 Pgno nPage; /* Number of pages in database file */
47902 /* If the database is zero pages in size, that means that either (1) the
47906 ** In either case, the journal file can be deleted. However, take care
47939 /* If we cannot open the rollback journal file in order to see if
47941 ** it might be due to the race condition described above and in
47967 ** 1) If the pager is currently in PAGER_OPEN state (no lock held
47976 ** 2) If the pager is running in exclusive-mode, and there are currently
47977 ** no outstanding references to any pages, and is in the error state,
47991 ** be OPEN or READER. READER is only possible if the pager is or was in
48034 ** this point in the code and fail to obtain its own EXCLUSIVE lock
48037 ** Unless the pager is in locking_mode=exclusive mode, the lock is
48047 ** in exclusive-access mode the file descriptor will be kept open
48049 ** is usually required to finalize the journal in journal_mode=persist
48053 ** other connection managed to get in and roll it back before
48055 ** may mean that the pager was in the error-state when this
48103 ** In order to get pager_unlock() to do this, set Pager.eState to
48105 ** to ERROR state in the state diagram at the top of this file,
48109 ** to be in ERROR state when there are zero outstanding page
48126 ** occurring on the very first access to a file, in order to save a
48133 ** a codec is in use.
48161 ** In this case there may exist a Pager.pMap mapping that appears
48170 /* If there is a WAL file in the file-system, open this database in WAL
48204 ** Except, in locking_mode=EXCLUSIVE when there is nothing to in
48215 ** Acquire a reference to page number pgno in pager pPager (a page
48219 ** If the requested page is already in the cache, it is returned.
48221 ** read from the database file. In some cases, the pcache module may
48227 ** already in the cache when this function is called, then the extra
48232 ** requested page is not already stored in the cache, then no
48233 ** actual disk read occurs. In this case the memory image of the
48237 ** of the page. This occurs in two scenarios:
48247 ** to pgno in Pager.pInJournal (bitvec of pages already written to the
48250 ** point in the future, using a call to sqlite3PagerWrite(), its contents
48253 ** The acquisition might fail for several reasons. In all cases,
48257 ** to find a page in the in-memory cache first. If the page is not already
48258 ** in memory, this routine goes to disk to read it in whereas Lookup()
48278 ** temporary or in-memory database. */
48288 ** test in the previous statement, and avoid testing pgno==0 in the
48299 /* If the pager is in the error state, return an error immediately.
48355 ** pager was already in the error-state when this function was called.
48365 /* In this case the pcache already contains an initialized copy of
48390 /* Failure to set the bits in the InJournal bit-vectors is benign.
48394 ** a bit in a bit vector.
48436 ** Acquire a page if it is already in the in-memory cache. Do
48438 ** or 0 if the page is not in cache.
48442 ** in the page if the page is not already in cache. This routine
48443 ** returns NULL if the page is not in cache or if a disk I/O error
48491 ** If the journal file is already open (as it may be in exclusive mode),
48510 /* If already in the error state, this function is a no-op. But on
48511 ** the other hand, this routine is never called if we are already in
48584 ** within this transaction will be opened as an in-memory file. This
48586 ** running in exclusive mode) or if the transaction does not require a
48588 ** sub-journal is implemented in-memory if pPager is an in-memory database,
48636 ** This is because in those states the code to roll back savepoint
48638 ** file as well as into the page cache. Which would be incorrect in
48677 ** page in the block above, set the need-sync flag for the page.
48678 ** Otherwise, when the transaction is rolled back, the logic in
48680 ** in the database file. And if an IO error occurs while doing so,
48713 ** one of the journals, the corresponding bit is set in the
48723 ** It is never called in the ERROR state.
48740 ** an error might occur and the pager would end up in WRITER_LOCKED state
48741 ** with pages marked as dirty in the cache.
48753 /* If a rollback journal is in use, them make sure the page that is about
48754 ** to change is in the rollback journal, or if the page is a new page off
48784 /* If the statement journal is open and the page is not in it,
48802 ** a sector need to be journalled in case of a power loss in the middle of
48805 ** Usually, the sector size is less than or equal to the page size, in which
48806 ** case pages can be individually written. This routine only runs in the
48811 Pgno nPageCount; /* Total number of pages in database file */
48897 ** fit on a single disk sector. In this case all co-resident pages
48920 ** Return TRUE if the page given in the argument was previously passed
48921 ** to sqlite3PagerWrite(). In other words, return TRUE if it is ok
48968 ** page data. In this case the file will be updated when the current
48972 ** with the SQLITE_ENABLE_ATOMIC_WRITE macro defined. In this case,
48986 ** atomic-write optimization is enabled in this build, then isDirect
48992 ** 'isDirect' below, as well as the block enclosed in the
49013 ** operating in direct-mode, make page 1 writable. When not in
49014 ** direct mode, page 1 is always held in cache and hence the PagerGet()
49025 /* If running in direct mode, write the contents of page 1 to the file. */
49054 ** Sync the database file to disk. This is a no-op for in-memory databases
49076 ** This function may only be called while a write-transaction is active in
49077 ** rollback. If the connection is in WAL mode, this call is a no-op.
49082 ** successful, or the connection is in WAL mode, SQLITE_OK is returned.
49123 ** journal file in this case.
49149 /* If this is an in-memory db, or no pages have been written to, or this
49151 ** backup in progress needs to be restarted.
49182 ** * Exactly one page has been modified and store in the journal file.
49186 ** counter in 'indirect-mode'. If the optimization is compiled in but
49189 ** pager_incr_changecounter() to update the change-counter in indirect
49194 ** in 'direct' mode. In this case the journal file will never be
49209 ** following call will modify the in-memory representation of page 1
49239 ** journal requires a sync here. However, in locking_mode=exclusive
49241 ** not the case. In this case it is likely enough that the redundant
49257 ** last page in the db image moved to the free-list. In this case the
49286 ** synced to disk. The journal file still exists in the file-system
49302 ** But if (due to a coding error elsewhere in the system) it does get
49313 ** this transaction, the pager is running in exclusive-mode and is
49320 ** header. Since the pager is in exclusive mode, there is no need
49344 ** If the pager is already in PAGER_ERROR state when this function is called,
49345 ** it returns Pager.errCode immediately. No work is performed in this case.
49347 ** Otherwise, in rollback mode, this function performs two functions:
49350 ** in-memory cache pages to the state they were in when the transaction
49354 ** rollback at any point in the future.
49359 ** In WAL mode, all cache-entries containing data modified within the
49368 /* PagerRollback() is a no-op if called in READER or OPEN state. If
49369 ** the pager is already in the ERROR state, the rollback is not
49411 ** if the database is (in theory) writable.
49491 ** Return true if this is an in-memory pager.
49518 ** if the allocation fails. Otherwise, zero the new portion in case a
49519 ** malloc failure occurs while populating it in the for(...) loop below.
49586 ** In any case, all savepoints with an index greater than iSavepoint
49605 ** operation. Store this value in nNew. Then free resources associated
49615 ** the sub-journal to zero bytes in size. */
49618 /* Only truncate if it is an in-memory sub-journal. */
49628 ** not yet been opened. In this case there have been no changes to
49644 ** Except, if the pager is in-memory only, then return an empty string if
49648 ** shared cache, it uses nullIfMemDb==0 so that in-memory databases can
49649 ** participate in shared-cache.
49731 ** Move the page pPg to location pgno in the file.
49735 ** in cache. If the page previously located at pgno is not already
49736 ** in the rollback journal, it is not put there by by this routine.
49739 ** meta-data associated with pPg (i.e. data stored in the nExtra bytes
49749 ** is being committed. In this case, it is guaranteed that the database page
49767 /* In order to be able to rollback, an in-memory database must journal
49780 ** <journal page X, then modify it in memory>
49804 ** be written to, store pPg->pgno in local variable needSyncPgno.
49828 /* Do not discard pages from an in-memory database since we might
49840 /* For an in-memory database, make sure the original page continues
49841 ** to exist, in case the transaction needs to roll back. Use pPgOld
49853 ** Currently, no such page exists in the page-cache and the
49859 ** to a malloc() or IO failure), clear the bit in the pInJournal[]
49860 ** array. Otherwise, if the page is loaded and written again in
49862 ** it is synced into the journal file. This way, it may end up in
49947 ** * An in-memory database can only have its journal_mode set to _OFF
49978 /* Do allow the journalmode of an in-memory database to be set to
49995 ** mode except WAL, unless the pager is in locking_mode=exclusive mode,
50008 /* In this case we would like to delete the journal file. If it is
50014 ** while it is in use by some other client.
50057 ** Return TRUE if the pager is in a state where it is OK to change the
50084 ** in backup.c maintains the content of this variable. This module
50094 ** Unless this is an in-memory or temporary database, clear the pager cache.
50154 ** Call sqlite3WalOpen() to open the WAL handle. If the pager is in
50157 ** in. Otherwise, use the normal shared-memory.
50165 /* If the pager is already in exclusive-mode, the WAL module will use
50194 ** file (not a temp file or an in-memory database), and the WAL file
50198 ** not modified in either case.
50200 ** If the pager is open on a temp-file (or in-memory database), or if
50248 /* If the log file is not already open, but does exist in the file-system,
50285 ** the pager is in WAL mode and the WAL file currently contains one or more
50286 ** frames, return the size in bytes of the page images stored within the
50304 ** The author disclaims copyright to this source code. In place of
50313 ** This file contains the implementation of a write-ahead log (WAL) used in
50324 ** transferred back into the database file in an operation called a
50327 ** A single WAL file can be used multiple times. In other words, the
50334 ** The WAL header is 32 bytes in size and consists of the following eight
50352 ** 4: For commit records, the size of the database image in pages
50362 ** (1) The salt-1 and salt-2 values in the frame-header match
50363 ** salt values in the wal-header
50365 ** (2) The checksum values in the final 8 bytes of the frame-header
50371 ** magic number in the first 4 bytes of the WAL is 0x377f0683 and it
50373 ** The checksum values are always stored in the frame header in a
50385 ** in reverse order (the largest fibonacci weight occurs on the first element
50396 ** value is randomized. This prevents old and new frames in the WAL from
50411 ** valid frame in the WAL. The reader uses this recorded "mxFrame" value
50415 ** of the database from a single point in time. This technique allows
50419 ** The reader algorithm in the previous paragraphs works correctly, but
50436 ** be) reconstructed from the original WAL file. In fact, the VFS is required
50441 ** as big endian, the wal-index can store multi-byte values in the native
50446 ** last frame in the wal before frame M for page P in the WAL, or return
50447 ** NULL if there are no frames for page P in the WAL prior to M.
50453 ** in the mxFrame field.
50459 ** first index block are the same size as all other index blocks in the
50466 ** for the first index block) 32-bit page numbers. The first entry in the
50468 ** first frame in the WAL file. The first entry in the second index block
50469 ** in the WAL file corresponds to the (HASHTABLE_NPAGE_ONE+1)th frame in
50472 ** The last index block in a wal-index usually contains less than the full
50486 ** HASHTABLE_NSLOT = 2*HASHTABLE_NPAGE, and there is one entry in the
50487 ** hash table for each page number in the mapping section, so the hash
50490 ** 1-based index of an entry in the mapping section of the same
50491 ** index block. Let K be the 1-based index of the largest entry in
50497 ** To look for page P in the hash table, first compute a hash iKey on
50510 ** no hash slot such that aHash[i]==p) then page P is not in the
50517 ** average, only two or three slots in each index block need to be
50518 ** examined in order to either find the last entry for page P, or to
50519 ** establish that no such entry exists in the block. Each index block
50522 ** comparisons (on average) suffice to either locate a frame in the
50523 ** WAL or to establish that the frame does not exist in the WAL. This
50526 ** Note that entries are added in order of increasing K. Hence, one
50531 ** the correct result. There may be entries in the hash table with
50533 ** slots in the hash table and so the first reader will get an answer as
50535 ** in the first place - which is what reader one wants. Meanwhile, the
50562 ** values in the wal-header are correct and (b) the version field is not
50594 ** The actual header in the wal-index consists of two copies of this
50599 ** added in 3.7.1 when support for 64K pages was added.
50606 u8 bigEndCksum; /* True if checksums in WAL are big-endian */
50607 u16 szPage; /* Database page size in bytes. 1==64K */
50608 u32 mxFrame; /* Index of last valid frame in the WAL */
50609 u32 nPage; /* Size of database in pages */
50610 u32 aFrameCksum[2]; /* Checksum of last frame in log */
50616 ** A copy of the following object occurs in the wal-index immediately
50620 ** nBackfill is the number of frames in the WAL that have been written
50628 ** There is one entry in aReadMark[] for each reader lock. If a reader
50629 ** holds read-lock K, then the value in aReadMark[K] is no greater than
50644 ** in use (that is, every aReadMark[j] for which there is a corresponding
50649 ** in the WAL has been backfilled into the database) then new readers
50657 ** (in other words, if there are no WAL_READ_LOCK(i) where i>0) then
50661 ** We assume that 32-bit loads are atomic and so no locks are needed in
50680 /* Size of header before each frame in wal */
50688 ** significant bit also set (WAL_MAGIC | 0x00000001) is stored in 32-bit
50689 ** big-endian format in the first 4 bytes of a WAL file.
50699 ** Return the offset of frame iFrame in the write-ahead log file,
50719 volatile u32 **apWiData; /* Pointer to wal-index content in memory */
50723 u8 exclusiveMode; /* Non-zero if connection is in exclusive mode */
50724 u8 writeLock; /* True if in a write transaction */
50733 u32 nCkpt; /* Checkpoint sequence counter in the wal-header */
50761 ** all frames in the WAL in database page order. Where two or more frames
50763 ** frame most recently written to the WAL (in other words, the frame with
50776 int nSegment; /* Number of entries in aSegment[] */
50778 int iNext; /* Next slot in aIndex[] not yet returned */
50779 ht_slot *aIndex; /* i0, i1, i2... such that aPgno[iN] ascend */
50781 int nEntry; /* Nr. of entries in aPgno[] and aIndex[] */
50783 } aSegment[1]; /* One for every 32KB page in the wal-index */
50787 ** Define the parameters of the hash tables in the wal-index file. There
50788 ** is a hash-table following every HASHTABLE_NPAGE page numbers in the
50799 ** The block of page numbers associated with the first hash-table in a
50859 ** Return a pointer to the WalCkptInfo structure in the wal-index.
50867 ** Return a pointer to the WalIndexHdr structure in the wal-index.
50887 ** Generate or extend an 8 byte checksum based on the data in
50898 int nByte, /* Bytes of content in a[]. Must be a multiple of 8. */
50940 ** Write the header information in pWal->hdr into the wal-index.
50963 ** 4: For commit records, the size of the database image in pages
50993 ** Check to see if the frame with header in aFrame[] and content
50994 ** in aData[] is valid. If it is a valid frame, fill *piPage and
51009 /* A frame is only valid if the salt values in the frame-header
51010 ** match the salt values in the wal-header.
51025 ** and the frame-data matches the checksum in the last 8
51074 ** In locking_mode=EXCLUSIVE, all of these routines become no-ops.
51114 ** the hash to the next value in the event of a collision.
51131 ** in the wal-index file. Set *piZero to one less than the frame
51133 ** slot in the hash table is set to N, it refers to frame number
51134 ** (*piZero+N) in the log.
51189 ** Return the page number associated with frame iFrame in this WAL.
51216 int nByte; /* Number of bytes to zero in aPgno[] */
51245 /* Zero the entries in the aPgno array that correspond to frames with
51252 /* Verify that the every entry in the mapping region is still reachable
51270 ** Set an entry in the wal-index that will map database page number
51300 /* If the entry in aPgno[] is already set, then the previous writer
51301 ** must have exited unexpectedly in the middle of a transaction (after
51320 /* Verify that the number of entries in the hash table exactly equals
51321 ** the number of entries in the mapping region.
51325 int nEntry = 0; /* Number of entries in the hash table */
51330 /* Verify that the every entry in the mapping region is reachable
51369 /* Obtain an exclusive lock on all byte in the locking range not already
51397 int szFrame; /* Number of bytes in buffer aFrame[] */
51406 /* Read in the WAL header. */
51559 int bNoShm, /* True to run in heap-memory mode */
51570 /* In the amalgamation, the os_unix.c and os_win.c source files come before
51572 ** in os_unix.c and os_win.c agree with the WALINDEX_LOCK_OFFSET value.
51630 ** Find the smallest page number out of all pages held in the WAL that
51636 ** Return 0 on success. If there are no pages in the WAL with a page
51680 ** of indices such that the aRight[] contains every index that appears in
51693 const u32 *aContent, /* Pages in wal - keys for the sort */
51694 ht_slot *aLeft, /* IN: Left hand input list */
51695 int nLeft, /* IN: Elements in array *paLeft */
51696 ht_slot **paRight, /* IN/OUT: Right hand input list */
51697 int *pnRight, /* IN/OUT: Elements in *paRight */
51700 int iLeft = 0; /* Current index in
51701 int iRight = 0; /* Current index in aRight */
51702 int iOut = 0; /* Current index in output buffer */
51733 ** Sort the elements in list aList using aContent[] as the sort key.
51737 ** The aList[] entries are indices into aContent[]. The values in
51750 const u32 *aContent, /* Pages in wal */
51752 ht_slot *aList, /* IN/OUT: List to sort */
51753 int *pnList /* IN/OUT: Number of elements in aList[] */
51756 int nList; /* Number of elements in aList */
51761 int nMerge = 0; /* Number of elements in list aMerge */
51818 ** pages in the WAL in ascending order. The caller must hold the checkpoint
51831 u32 iLast; /* Last frame in log */
51838 ** it only runs if there is actually content in the log (mxFrame>0).
51873 int nEntry; /* Number of entries in this segment */
51926 ** Return the page-size in bytes used by the database.
51945 ** The value of parameter salt1 is used as the aSalt[1] value in the
51966 ** in response to an sqlite3_wal_checkpoint() request or the equivalent.
51972 ** All I/O barrier operations (a.k.a fsyncs) occur in this routine when
51973 ** SQLite is in WAL-mode in synchronous=NORMAL. That means that if
51979 ** in the WAL and can be recovered following a power-loss or hard reset.
51983 ** it safe to delete the WAL since the new content will persist in the
51992 ** checkpoint is running (in any other thread or process) at the same
52027 ** in the SQLITE_CHECKPOINT_PASSIVE mode. */
52030 /* Compute in mxSafeFrame the index of the last frame of the WAL that is
52032 ** overwrite database pages that are in use by active readers and thus
52039 ** as some other thread may be in the process of updating the value
52149 ** In
52150 ** wal-index header in shared memory, as all subsequent reader or
52153 ** as it would leave the system in a state where the contents of
52171 ** If the WAL file is currently larger than nMax bytes in size, truncate
52204 ** the database. In this case checkpoint the database and unlink both
52224 ** fsyned (rc==SQLITE_OK) and if we are not in persistent-wal
52229 ** completed and fsynced (rc==SQLITE_OK) and we are in persistent
52257 ** The wal-index is in shared memory. Another thread or process might
52259 ** read it, which might result in inconsistency. A dirty read is detected
52273 WalIndexHdr volatile *aHdr; /* Header in shared memory */
52279 ** same area of shared memory on a different CPU in a SMP,
52284 ** When reading, read [0] first then [1]. Writes are in the reverse order.
52321 ** Set *pChanged to 1 if the wal-index header value in pWal->hdr is
52426 ** in the range 0 <= pWal->readLock < WAL_NREADER. If pWal->readLock==(-1)
52441 ** update values of the aReadMark[] array in the header, but if it does
52446 volatile WalCkptInfo *pInfo; /* Checkpoint information in wal-index */
52461 ** during the few nanoseconds that it is holding the lock. In that case,
52472 int nDelay = 1; /* Pause time in microseconds */
52484 /* If there is not a recovery running in another thread or process
52496 ** modules may return SQLITE_BUSY due to a race condition in the
52533 ** it finished. Leaving a corrupt image in the database file.
52587 ** value in the aReadMark[] array or the contents of the wal-index
52594 ** that occur later in the log than pWal->hdr.mxFrame may have been
52601 ** since it was read, set Wal.minFrame to the first frame in the wal
52607 ** nBackfill and checking that the wal-header in shared-memory still
52608 ** matches the one cached in pWal->hdr, it is guaranteed that the
52610 ** header newer than that cached in pWal->hdr. If it were, that could
52614 ** page later in the wal file. But if version B happens to like past
52640 ** instant in time. The current thread will continue to use this snapshot.
52677 ** contains the page. Otherwise, if pgno is not in the wal file, set *piRead
52689 u32 iLast = pWal->hdr.mxFrame; /* Last page in WAL for this reader */
52698 ** in this case as an optimization. Likewise, if pWal->readLock==0,
52711 ** This code might run concurrently to the code in walIndexAppend()
52721 ** For the reasons above, the if(...) condition featured in the inner
52782 ** (which is nOut bytes in size). Return SQLITE_OK if successful, or an
52788 int nOut, /* Size of buffer pOut in bytes */
52803 ** Return the size of the database in pages (or zero, if unknown).
52822 ** returns SQLITE_BUSY in that case and no write transaction is started.
52891 ** was in before the client began writing to the database.
52900 ** is passed as the second argument is (a) in the cache and
52922 ** point in the event of a savepoint rollback (via WalSavepointUndo()).
52934 ** the values in the aWalData[] array. aWalData must point to an array
52987 in other words if no
52992 ** In theory it would be Ok to update the cache of the header only
53034 ** In other words, if iSyncPoint is in between iOffset and iOffset+iAmt,
53071 u8 aFrame[WAL_FRAME_HDRSIZE]; /* Buffer to assemble frame-header in */
53091 int szPage, /* Database page-size in bytes */
53100 PgHdr *pLast = 0; /* Last frame in list */
53103 i64 iOffset; /* Next byte to write in WAL file */
53133 u8 aWalHdr[WAL_HDRSIZE]; /* Buffer to assemble wal-header in */
53160 ** an out-of-order write following a WAL restart could result in
53222 /* If this frame set completes the first transaction in the WAL and
53238 ** be in use by existing readers is being overwritten.
53280 ** callback. In this case this function runs a blocking checkpoint.
53290 int *pnLog, /* OUT: Number of frames in WAL */
53291 int *pnCkpt /* OUT: Number of backfilled frames in WAL */
53302 ** in the SQLITE_CHECKPOINT_PASSIVE mode. */
53316 ** it will not be invoked in this case.
53386 ** number of frames in the WAL at the point of the last commit since
53405 ** on the pWal->readLock byte. If the WAL is already in locking_mode=NORMAL
53414 ** WAL is already in exclusive-locking mode - meaning that this
53430 ** happen if the connection is actually in exclusive mode (as no xShmLock
53431 ** locks are taken in this case). Nor should the pager attempt to
53445 /* Already in locking_mode=NORMAL */
53488 ** The author disclaims copyright to this source code. In place of
53498 ** This code really belongs in btree.c. But btree.c is getting too
53502 /************** Include btreeInt.h in the middle of btmutex.c ****************/
53507 ** The author disclaims copyright to this source code. In place of
53536 ** disk where M is the number of entries in the tree.
53538 ** In this implementation, a single file can hold one or more separate
53562 ** 16 2 Page size in bytes. (1 means 65536)
53572 ** 36 4 Number of freelist pages in the file
53594 ** space in a page that can be consumed by a single cell for standard
53605 ** except that it applies to leaf nodes in a LEAFDATA tree. The maximum
53607 ** not specified in the header.
53642 ** which is stored in the key size entry of the cell header rather than in
53647 ** offsets from the beginning of the page to the cell content in the cell
53648 ** content area. The cell pointers occur in sorted order. The system strives
53656 ** freeblocks. Each freeblock is at least 4 bytes in size. The byte offset
53657 ** to the first freeblock is given in the header. Freeblocks occur in
53658 ** increasing order. Because a freeblock must be at least 4 bytes in size,
53659 ** any group of 3 or fewer unused bytes in the cell content area cannot
53661 ** a fragment. The total number of bytes in all fragments is recorded.
53662 ** in the page header at offset 7.
53666 ** 2 Bytes in this freeblock
53668 ** Cells are of variable length. Cells are stored in the cell content area at
53669 ** the end of the page. Pointers to the cells are in the cell pointer array
53671 ** contiguous or in order, but cell pointers are contiguous and in order.
53679 ** allows a 64-bit integer to be encoded in 9 bytes.
53690 ** bytes of key and data in a btree cell.
53709 ** Freelist pages come in two subtypes: trunk pages and leaf pages. The
53710 ** file header points to the first in a linked list of trunk page. Each trunk
53729 ** plus 2 bytes for the index to the cell in the page header). Such
53741 ** SQLite database in order to identify the file as a real database.
53775 ** stored in MemPage.pBt->mutex.
53779 u8 nOverflow; /* Number of overflow cell bodies in aCell[] */
53790 in aData of first cell pointer */
53809 ** The in-memory image of a disk page has the auxiliary information appended
53826 BtLock *pNext; /* Next in BtShared.pLock list */
53842 ** shared between multiple connections. In that case, each connection
53848 ** All fields in this structure are accessed under sqlite3.mutex.
53850 ** in the referenced BtShared that point back to this Btree since those
53885 ** A single database file can be in use at the same time by two
53892 ** Fields in this structure are accessed under the BtShared.mutex
53912 ** while in the 'pending-lock' state, no connection may start a new
53934 u16 maxLocal; /* Maximum local payload in non-LEAFDATA tables */
53935 u16 minLocal; /* Minimum local payload in non-LEAFDATA tables */
53936 u16 maxLeaf; /* Maximum local payload in a LEAFDATA table */
53937 u16 minLeaf; /* Minimum local payload in a LEAFDATA table */
53941 u32 nPage; /* Number of pages in the database */
53968 ** about a cell. The parseCellPtr() function fills in this structure
53995 ** The entry is identified by its MemPage and the index in
54002 ** Fields in this structure are accessed under the BtShared.mutex
54029 i8 iPage; /* Index of current page in apPage */
54033 u16 aiIdx[BTCURSOR_MAX_DEPTH]; /* Current index in apPage[i] */
54066 ** in variables BtCursor.pKey and BtCursor.nKey. When a cursor is in
54073 ** cursor. The error has left the cache in an inconsistent state.
54075 ** should return the error code stored in BtCursor.skipNext
54092 ** page number to look up in the pointer map.
54109 ** each child page in the database file. The parent page is the page that
54110 ** contains a pointer to the child. Every page in the database contains
54111 ** 0 or 1 parent pages. (In this context 'database page' refers
54117 ** position in the file to another as part of autovacuum. When a page
54118 ** is moved, the pointer in its parent must be updated to point to the
54122 ** used in this case.
54125 ** is not used in this case.
54127 ** PTRMAP_OVERFLOW1: The database page is the first page in a list of
54131 ** PTRMAP_OVERFLOW2: The database page is the second or later page in a list of
54133 ** page in the overflow page list.
54136 ** identifies the parent page in the btree.
54168 ** in order to keep track of some global state information.
54170 ** The aRef[] array is allocated so that there is 1 bit for each page in
54171 ** the database. As the integrity-check proceeds, for each page used in
54180 u8 *aPgRef; /* 1 bit per page in the db (see above) */
54181 Pgno nPage; /* Number of pages in the database */
54186 int v1, v2; /* Values for up to two %d fields in zPfx */
54202 ** cell addresses in a btree header.
54217 /************** Continuing where we left off in btmutex.c ********************/
54259 ** But we keep a reference count in Btree.wantToLock so the behavior
54262 ** To avoid deadlocks, multiple Btrees are locked in the same order
54272 ** connected by pNext and pPrev should be in sorted order by
54302 ** in the common case.
54307 /* In most cases, we should be able to acquire the lock we
54319 ** the other BtShared locks that we used to hold in ascending
54395 ** Enter the mutexes in accending order by BtShared pointer address
54397 ** two or more btrees in common both try to lock all their btrees
54479 ** in single threaded applications that use shared cache. Except for
54480 ** these two routines, all mutex operations are no-ops in that case and
54481 ** are null #defines in btree.h.
54484 ** the ones below, are no-ops and are null #defines in btree.h.
54507 ** The author disclaims copyright to this source code. In place of
54571 ** in shared cache. This variable has file scope during normal builds,
54604 ** manipulate entries in the BtShared.pLock linked list used to store
54632 ** When writing to an index that resides in a sharable database, the
54684 ** useful in that case. */
54780 /* The condition (pIter->eLock!=eLock) in the following if(...)
54786 ** may hold a WRITE_LOCK on any table in this file (since there can
54833 ** by a connection in read-uncommitted mode is on the sqlite_master
54834 ** table, and that lock is obtained in BtreeBeginTrans(). */
54917 ** than the writer must be about to drop to zero. In this case
54921 ** be zero already. So this next line is harmless in that case.
54983 ** table is about to be deleted. In this case invalidate all incrblob
54987 ** rowid iRow is being replaced or deleted. In this case invalidate
55039 ** may be lost. In the event of a rollback, it may not be possible
55044 ** set in the bitvec. Whenever a leaf page is extracted from the free-list,
55046 ** set in BtShared.pHasContent. The contents of the bitvec are cleared
55100 ** function saves the current cursor key in variables pCur->nKey and
55105 ** (the rowid) is stored in pCur->nKey and pCur->pKey is left set to
55107 ** set to point to a malloced buffer pCur->nKey bytes in size containing
55120 ** stores the integer key in pCur->nKey. In this case this value is
55142 ** Save the current cursor position in the variables BtCursor.nKey
55177 ** the location in the btree is remembered in such a way that it can be
55180 ** table, for example in BtreeDelete() or BtreeInsert().
55184 ** routine enforces that rule. This routine only needs to be called in
55192 ** need to be saved. It calls out to saveCursorsOnList() in the (unusual)
55193 ** event that cursors are in need to being saved.
55245 ** In this version of BtreeMoveto, pKey is a packed index record
55283 ** Restore the cursor to the position it was in (or as close to as possible)
55402 int offset; /* Offset in pointer map page */
55454 int offset; /* Offset of entry in pointer map */
55512 CellInfo *pInfo /* Fill in this structure */
55518 ** in between minLocal and maxLocal.
55520 ** Warning: changing the way overflow payload is distributed in any
55521 ** way will result in an incompatible file format.
55545 ** Parse a cell content block and fill in the CellInfo structure.
55558 CellInfo *pInfo /* Fill in this structure */
55577 CellInfo *pInfo /* Fill in this structure */
55648 CellInfo *pInfo /* Fill in this structure */
55687 CellInfo *pInfo /* Fill in this structure */
55696 ** Compute the total number of bytes that a Cell needs in the cell
55806 ** big FreeBlk that occurs in between the header and cell
55811 ** unused bytes are contained in the unallocated space region, and all
55851 /* These conditions have already been verified in btreeInitPage()
55891 ** Search the free-list on page pPg for space to store a cell nByte bytes in
55915 /* EVIDENCE-OF: R-06866-39125 Freeblocks are always connected in order of
55923 ** in bytes, including the 4-byte header. */
55932 /* EVIDENCE-OF: R-11498-58022 In a well-formed b-tree page, the total
55933 ** number of bytes in fragments may not exceed 60. */
55961 ** allocation. This routine might need to defragment in order to bring
55964 ** allocation is being made in order to insert a new cell, so we will
55988 ** However, that integer is too large to be stored in a 2-byte unsigned
55989 ** integer, so a value of 0 is used in its place. */
56031 /* Allocate memory from the gap in between the cell pointer array
56061 u8 nFrag = 0; /* Reduction in fragmentation */
56081 /* The list of freeblocks must be in ascending order. Find the
56112 ** pointer in the page header) then check to see if iStart should be
56310 /* EVIDENCE-OF: R-55530-52930 In a well-formed b-tree page, there will
56320 /* Free blocks must be in ascending order. And the last byte of
56401 ** to fetch the content. Just fill in the content with zeros for now.
56402 ** If in the future we call sqlite3PagerWrite() on this page, that
56409 MemPage **ppPage, /* Return the page in this parameter */
56425 ** already in the pager cache return NULL. Initialize the MemPage.pBt and
56439 ** Return the size of the database file in pages. If there is any kind of
56455 ** call. Do additional sanity checking on the page in this case.
56535 ** * If the page is already in use for some other purpose, immediately
56542 MemPage **ppPage, /* Return the page in this parameter */
56577 ** or ptrmap page or a free page. In those cases, the following
56581 ** the call for every page that comes in for re-initing. */
56602 ** be exclusively in memory, or it might use a disk-based memory cache.
56606 ** If zFilename is ":memory:" then an in-memory database is created
56612 ** If the database is already opened in the same database connection
56613 ** and we are in shared cache mode, then the open will fail with an
56615 ** objects in the same database connection since doing so will lead
56636 /* Set the variable isMemdb to true for an in-memory database, or
56650 assert( (flags&0xff)==flags ); /* flags fit in 8 bits */
56734 /* In debug mode, we mark all persistent databases as sharable
56789 /* If the magic name ":memory:" will create an in-memory database, then
56793 ** regular file-name. In this case the auto-vacuum applies as per normal.
56843 ** The list is kept in ascending order by pBt address.
56948 ** a cell is less than 4 bytes in size, it is rounded up to 4 bytes
57038 ** Change the limit on the number of pages allowed in the cache.
57048 ** could be left in an inconsistent and unrecoverable state.
57077 ** Change the way data is synced to disk in order to increase or decrease
57099 ** Return TRUE if the given btree is set to safety level 1. In other
57175 ** This is useful in one special case in the backup API code where it is
57177 ** database handle that owns *p is not. In this case if sqlite3BtreeEnter()
57299 int nPage; /* Number of pages in the database */
57300 int nPageFile = 0; /* Number of pages in the database file */
57301 int nPageHeader; /* Number of pages in the database according to hdr */
57324 ** with the following 16 bytes (in hex): 53 51 4c 69 74 65 20 66 6f 72 6d
57346 ** in WAL mode. If the log is not already open, open it now. Then
57349 ** required as the version of page 1 currently in the page1 buffer
57350 ** may not be the latest version - there may be a newer one in the log
57388 /* EVIDENCE-OF: R-59310-51205 The "reserved space" size in the 1-byte
57416 ** be less than 480. In other words, if the page size is 512, then the
57465 ** in assert() expressions, so it is only compiled if NDEBUG is not
57487 ** If there are no outstanding cursors and we are not in the middle
57492 ** If there is a transaction in progress, this routine is a no-op.
57589 ** returned when there is already a read-lock in order to avoid a deadlock.
57607 /* If the btree is already in a write-transaction, or it
57608 ** is already in a read-transaction and a read-transaction
57660 ** file is not pBt->pageSize. In this case lockBtree() will update
57746 int nCell; /* Number of cells in page pPage */
57792 ** overflow page in the list.
57798 /* The pointer is always the first 4 bytes of the page in this case. */
57848 ** Move the open database page pDbPage to location iFreePage in the
57934 ** SQLITE_OK. If there is no work to do (and therefore no point in
57939 ** that the last page of the file currently in use is no longer in use.
57976 ** if bCommit is non-zero. In that case, the free-list will be
58042 ** nOrig pages in size containing nFree free pages. Return the expected
58043 ** size of the database in pages following an auto-vacuum operation.
58111 ** pages are in use.
58122 Pgno nFin; /* Number of pages in database after autovacuuming */
58256 ** Commit the transaction currently in progress.
58263 ** routine has to do is delete or truncate or zero the header in the
58271 ** transaction. In this case, the transaction has already been committed
58273 ** functions return code. So, even if an error occurs in the pager layer,
58338 ** tripped. In general, writeOnly is false if the transaction being
58339 ** rolled back modified the database schema. In this case b-tree root
58383 ** Rollback the transaction in progress.
58388 ** a tripped cursor will result in an error.
58488 ** SAVEPOINT_ROLLBACK, then iSavepoint may also be -1. In this case the
58528 ** are the conditions that must be met in order for writing to
58534 ** but which are not in the READ_UNCOMMITTED state may not have
58537 ** the read cursors in the other database connection.
58585 /* Now that no other errors can occur, finish filling in the BtCursor
58627 ** Return the size of a BtCursor object in bytes.
58686 ** Make sure the BtCursor* given in the argument has a valid
58688 ** btreeParseCell() to fill it in.
58690 ** BtCursor.info is a cache of the information in the current cell.
58717 ** that is currently pointing to a row in a (non-empty) table.
58731 ** itself, not the number of bytes in the key.
58746 ** Set *pSize to the number of bytes of data in the entry the
58750 ** valid entry. In other words, the calling procedure must guarantee
58769 ** Given the page number of an overflow page in the database (parameter
58770 ** ovfl), this function finds the page number of the next page in the
58776 ** The page number of the next overflow page in the linked list is
58777 ** written to *pPgnoNext. If page ovfl is the last page in its linked
58783 ** on *ppPage to free the reference. In no reference was obtained (because
58801 /* Try to find the next page in the overflow list using the
58802 ** autovacuum pointer-map pages. Guess that the next page in
58899 ** the cursor is moved to a different row. Additionally, in auto-vacuum
58903 ** * A commit in auto_vacuum="full" mode,
58966 ** in the overflow chain. The page number of the first overflow page is
58967 ** stored in aOverflow[0], etc. A value of 0 in the aOverflow[] array
59011 ** number for the next page in the overflow chain. The page
59104 ** in the table.
59198 ** in the common case where no overflow pages are used.
59239 ** cell in page pParent. Or, if iIdx is equal to the total number of
59240 ** cells in pParent, that page number iChild is the right-child of
59245 ** in a corrupt database */
59348 ** in such a way that page pRoot is linked into a second b-tree table
59378 ** in ascending order.
59403 ** key in ascending order.
59424 /* Move the cursor to the first entry in the table. Return SQLITE_OK
59447 /* Move the cursor to the last entry in the table. Return SQLITE_OK
59461 ** to the last entry in the b-tree. */
59579 u8 *pCell; /* Pointer to current cell in pPage */
59629 int nCell; /* Size of the pCell cell in bytes */
59655 /* The record flows over onto one or more overflow pages. In
59662 ** bytes of padding is allocated at the end of the buffer in
59741 ** past the last entry in the table or sqlite3BtreePrev() moves past
59745 /* TODO: What if the cursor is in CURSOR_REQUIRESEEK but all table entries
59753 ** Advance the cursor to the next entry in the database. If
59755 ** was already pointing to the last entry in the database before
59809 ** only happen if the database is corrupt in such a way as to link the
59863 ** Step the cursor to the back to the previous entry in the database. If
59865 ** was already pointing to the first entry in the database before
59963 ** The new page is marked as dirty. (In other words, sqlite3PagerWrite()
59969 ** an error. *ppPage is set to NULL in the event of an error.
59972 ** locate a page close to the page number "nearby". This can be used in an
59973 ** attempt to keep related pages close to each other in the database file,
59974 ** which in turn can make database access faster.
60051 ** is the page number of the next freelist trunk page in the list or
60120 ** page in this case.
60232 ** that is required in the event of a rollback. In this case, do
60342 /* If the database supports auto-vacuum, write an entry in the pointer-map
60352 ** trunk page in the free-list is full, then this page will become a
60354 ** first trunk page in the current free-list. This block tests if it
60373 /* In this case there is room on the trunk page to insert the page
60378 ** coded. But due to a coding error in versions of SQLite prior to
60380 ** usableSize/4 - 8 entries will be reported as corrupt. In order
60383 ** for now. At some point in the future (once everyone has upgraded
60388 ** avoid using the last six entries in the freelist trunk page array in
60407 ** the page being freed as a leaf page of the first trunk in the free-list.
60409 ** first trunk in the free-list is full. Either way, the page being freed
60410 ** will become the new first trunk page in the free-list.
60495 ** caller is iterating through or using in some other way, this
60515 ** allocated and filled in as necessary. The calling procedure
60521 ** be constructed in this temporary area then copied into pPage->aData
60551 /* Fill in the header. */
60562 /* Fill in the payload size */
60595 ** nPayload Total payload size in bytes
60638 ** then the optimistic overflow chain processing in clearCell()
60711 ** "sz" must be the number of bytes in the cell.
60762 ** in pPage->apOvfl[] and make it point to the cell content (either
60763 ** in pTemp or the original pCell) and also record its index.
60764 ** Allocating a new entry in pPage->aCell[] implies that
60771 int sz, /* Bytes of content in pCell */
60776 int idx = 0; /* Where to write new cell content in data[] */
60779 u8 *pIns; /* The point in pPage->aCellIdx[] where no cell inserted */
60793 ** the term after the || in the following assert(). */
60808 /* When multiple overflows occur, they are always sequential and in
60813 assert( j==0 || pPage->aiOvfl[j-1]<(u16)i ); /* Overflows in sorted order */
60860 int nCell; /* Number of cells in apCell[] */
60863 u16 *szCell; /* Local size of all cells in apCell[] */
60902 ** szCell[] array contains the size in bytes of each cell. This function
60906 ** Some of the cells in apCell[] may currently be stored in pPg. This
60959 ** contains the size in bytes of each such cell. This function attempts to
60960 ** add the cells stored in the array to page pPg. If it cannot (because
60965 ** Argument pCellptr points to the first entry in the cell-pointer array
60968 ** cell in the array. It is the responsibility of the caller to ensure
60979 ** area must be extended to before this point in order to accomodate all
60980 ** cells in apCell[], then the cells do not fit and non-zero is returned.
60985 u8 **ppData, /* IN/OUT: Page content -area pointer */
61021 ** contains the size in bytes of each such cell. This function adds the
61022 ** space associated with each cell in the array that is currently stored
61074 ** apCell[] and szCell[] contains pointers to and sizes of all cells in the
61181 ** in a balancing operation. NN is the number of neighbors on either side
61182 ** of the page that participate in the balancing operation. NB is the
61187 ** (to 2 or 3) gives a modest improvement in SELECT and DELETE performance
61188 ** in exchange for a larger degradation in INSERT and UPDATE performance.
61192 #define NB (NN*2+1) /* Total pages involved in the balance */
61199 ** tree, in other words, when the new entry will become the largest
61200 ** entry in the tree.
61203 ** a new page to the right-hand side and put the one new entry in
61209 ** pPage is the leaf page which is the right-most page in the tree.
61215 In other
61217 ** least 13 bytes in size.
61275 ** record-length (a variable length integer at most 32-bits in size)
61348 ** parent page stored in the pointer map is page pTo. If pFrom contained
61352 ** If pFrom is currently carrying any overflow cells (entries in the
61406 ** page are used in the balancing, though both siblings might come from one
61410 ** participate in the balancing.
61413 ** one or two in an effort to keep pages nearly full but not over full.
61416 ** might not actually be stored in MemPage.aData[]. This can happen
61420 ** In the course of balancing the page and its siblings, cells may be
61427 ** in a corrupted state. So if this routine fails, the database should
61447 int iParentIdx, /* Index of "the page" in pParent */
61454 int nNew = 0; /* Number of pages in apNew[] */
61455 int nOld; /* Number of pages in apOld[] */
61457 int nxDiv; /* Next divider slot in pParent->aCell[] */
61461 int usableSpace; /* Bytes in pPage beyond the header */
61468 u8 *pRight; /* Location in parent of right-sibling pointer */
61469 u8 *apDiv[NB-1]; /* Divider cells in pParent */
61470 int cntNew[NB+2]; /* Index in b.paCell[] of cell after i-th page */
61471 int cntOld[NB+2]; /* Old index in b.apCell[] */
61474 Pgno pgno; /* Temp var to store a page number in */
61504 /* Find the sibling pages to balance. Also locate the cells in pParent
61512 ** overflow cells in the parent page, since if any existed they will
61562 ** But not if we are in secure-delete mode. In secure-delete mode,
61564 ** In this case, temporarily copy the cell into the aOvflSpace[]
61584 /* Make nMaxCells a multiple of 4 in order to preserve 8-byte
61616 ** into aSpace1[]. In this way, all cells in b.apCell[] are without
61617 ** child pointers. If siblings are not leaves, then all cell in
61618 ** b.apCell[] include child pointers. Either way, all cells in b.apCell[]
61643 /* Load b.apCell[] with pointers to all cells in pOld. If pOld
61644 ** constains overflow cells, include them in the b.apCell[] array
61645 ** in the correct spot.
61651 ** adjacent and are inserted in order. There is an assert() tagged
61652 ** with "NOTE 1" in the overflow cell insertion loop to prove this
61655 ** This must be done in advance. Once the balance starts, the cell
61720 ** Store this number in "k". Also compute szNew[] which is the total
61722 ** in b.apCell[] of the cell that divides page i from page i+1.
61729 ** cntNew[i]: Index in b.apCell[] and b.szCell[] for the first cell to
61802 int r; /* Index of right-most cell in left sibling */
61876 ** Reassign page numbers so that the new pages are in ascending order.
61877 ** This helps to keep entries in the disk file in order so that a scan
61878 ** of the table is closer to a linear scan through the file. That in turn
61895 ** we do the detection here in order to avoid populating the pager
61938 ** originally in the same field of the right-most old sibling page. */
61950 ** entry associated with the first page in the overflow chain, and
62019 ** then there is no divider cell in b.apCell[]. Instead, the divider
62038 ** Note that this can never happen in an SQLite data file, as all
62039 ** cells are at least 4 bytes. It only happens in b-trees used
62040 ** to evaluate "IN (SELECT ...)" and similar clauses.
62055 /* Now update the actual sibling pages. The order in which they are updated
62057 ** cells may still to be read. In practice, this means:
62069 ** The iPg value in the following loop starts at nNew-1 goes down
62125 ** sub-algorithm in some documentation.
62210 ** page and SQLITE_OK is returned. In this case the caller is required
62262 ** The page that pCur currently points to has just been modified in
62286 /* The root page of the b-tree is overfull. In this case call the
62322 ** happens, the overflow cell is stored in the aBalanceQuickSpace[]
62328 ** of the aBalanceQuickSpace[] might sneak in.
62335 /* In this case, call balance_nonroot() to redistribute cells
62342 ** are stored in the pSpace buffer allocated immediately below.
62446 ** In some cases, the call to btreeMoveto() below is a no-op. For
62450 ** data into the intkey B-Tree. In this case btreeMoveto() recognizes
62534 ** is advantageous to leave the cursor pointing to the last entry in
62536 ** entry in the table, and the next row inserted has an integer key
62563 ** is left in a state such that the next call to BtreeNext() or BtreePrev()
62576 int bSkipnext = 0; /* Leaf cursor in SKIPNEXT state */
62593 ** the cursor to the largest entry in the tree that is smaller than
62621 ** key and leaving the cursor in CURSOR_REQUIRESEEK state before
62625 ** will be left in CURSOR_SKIPNEXT state pointing to the entry immediately
62626 ** before or after the deleted entry. In this case set bSkipnext to true. */
62650 ** is currently pointing to the largest entry in the sub-tree headed
62673 ** then the cursor still points to that page. In this case the first
62681 ** be either under or overfull. In this case run the balancing algorithm
62683 ** tree that we can be sure that any problem in the internal node has
62722 ** following values of flags are currently in use. Other values for
62750 ** to make room for the new tables root page. In case this page turns
62792 /* Save the positions of any open cursors. This is required in
62843 /* When the new root page was allocated, page 1 was made writable in
62938 ** Delete all information from a single table in the database. iTable is
62948 ** entries in the table.
62979 ** Erase all information in a table and add the root of the table to
62987 ** root page in the database file, then the last root page
62988 ** in the database file is moved into the slot formerly occupied by
62990 ** is added to the freelist instead of iTable. In this say, all
62993 ** page number that used to be the last root page in the file before
62995 ** The last root page is recorded in meta[3] and the value of
63007 ** database. This is because in auto-vacuum mode the backend may
63040 ** number in the database, put the root page on the free list.
63049 ** number in the database. So move the page that does into the
63073 /* Set the new 'max-root-page' value in the database header. This
63093 ** This really never should happen except in a corrupt
63115 ** is the number of free pages currently in the database. Meta[1]
63125 ** from the pager. The BTREE_DATA_VERSION value is not actually stored in the
63145 /* If auto-vacuum is disabled in this build and this is an auto-vacuum
63187 ** number of entries in the b-tree and write the result to *pnEntry.
63194 i64 nEntry = 0; /* Value to return in *pnEntry */
63204 ** page in the B-Tree structure (not including overflow pages).
63207 int iIdx; /* Index of child node in parent */
63221 ** the next page in the tree that has not yet been visited. The
63223 ** of the page, or to the number of cells in the page if the next page
63226 ** If all pages in the tree have been visited, return SQLITE_OK to the
63298 ** Return non-zero if the bit in the IntegrityCk.aPgRef[] array that
63307 ** Set the bit in the IntegrityCk.aPgRef[] array that corresponds to page iPg.
63321 ** Also check that the page number is in bounds.
63339 ** Check that the entry in the pointer-map for page iChild maps to
63375 int iPage, /* Page number for first page in the list */
63376 int N /* Expected number of pages in the list */
63423 ** page in this overflow list, check that the pointer-map entry for
63436 checkAppendMsg(pCheck, "free-page count in header is too small");
63451 ** root node aHeap[1] is always the minimum value currently in the heap.
63455 ** removes the root element from the heap (the minimum value in the heap)
63503 ** 2. Make sure integer cell keys are in order.
63554 /* Clear MemPage.isInit to make sure the corruption detection code in
63595 /* For leaf pages, the coverage check will occur in the same loop
63700 ** big-endian integer which is the offset in the b-tree page of the next
63701 ** freeblock in the chain, or zero if the freeblock is the last on the
63704 /* EVIDENCE-OF: R-06866-39125 Freeblocks are always connected in order of
63711 ** freeblocks, and counting the number of untracked bytes in nFrag.
63718 ** The loop below pulls entries from the min-heap in order and compares
63736 /* EVIDENCE-OF: R-43263-13491 The total number of bytes in all fragments
63737 ** is stored in the fifth field of the b-tree page header.
63762 ** a table. nRoot is the number of entries in aRoot.
63767 ** Write the number of error seen in *pnErr. Except for some memory
63768 ** allocation errors, an error message held in memory obtained from
63775 int nRoot, /* Number of entries in aRoot[] */
63842 /* Make sure every page in the file is referenced
63884 ** an empty string if the database is in-memory or a TEMP database.
64054 ** required in case any of them are holding references to an xFetch
64094 ** "write version" (single byte at byte offset 19) fields in the database
64163 ** The author disclaims copyright to this source code. In place of
64183 u32 iDestSchema; /* Original schema cookie in destination */
64211 ** structure, in that order.
64214 ** invoked by the pager layer to report various state changes in
64228 ** Non-sharable Btrees (in-memory databases for example), do not have
64234 ** in connection handle pDb. If such a database cannot be found, return
64288 ** message in database handle db.
64292 sqlite3ErrorWithMsg(db, SQLITE_ERROR, "destination database is in use");
64300 ** connection handle pSrcDb to zDestDb in pDestDb. If successful, return
64304 ** stored in database handle pDestDb.
64322 ** handle is not locked in this routine, but it is locked in
64326 ** database connection while a backup is in progress may cause
64420 /* Catch the case where the destination is an in-memory database and the
64429 ** and a codec is in use.
64534 int nSrcPage = -1; /* Size of source db in pages */
64537 /* If the source pager is currently in a write-transaction, return
64563 /* Do not allow backup if the destination database is in WAL mode
64573 ** source pager for the number of pages in the database.
64600 /* Update the schema version field in the destination database. This
64601 ** is to make sure that the schema-version really does change in
64623 /* Set nDestTruncate to the final number of pages in the destination
64628 ** round up. In this case the call to sqlite3OsTruncate() below will
64630 ** sqlite3PagerTruncateImage() here so that any pages in the
64655 ** pending-byte page in the source database may need to be
64673 ** database has been stored in the journal for pDestPager and the
64675 ** the database file in any way, knowing that if a power failure
64822 ** Return the total number of pages in the source database as of the most
64879 ** connection. In this case there is no way of knowing which of the
64935 /* 0x7FFFFFFF is the hard limit for the number of pages in a database
64964 ** The author disclaims copyright to this source code. In place of
64974 ** stores a single value in the VDBE. Mem is an opaque structure visible
64994 /* MEM_Dyn may only be set if Mem.szMalloc==0. In this way we
64997 ** That saves a few cycles in inner loops. */
65010 ** (1) Memory in Mem.zMalloc and managed by the Mem object
65074 ** in pMem->z is discarded.
65124 ** Any prior string or blob content in the pMem object may be discarded.
65145 ** Change pMem so that its MEM_Str or MEM_Blob value is stored in
65174 ** blob stored in dynamically allocated space.
65240 ** keys are strings. In the former case a NULL pointer is returned the
65312 ** invoking the external callback in Mem.xDel, then this routine
65343 ** by p->xDel and memory in p->zMalloc.
65345 ** This is a helper routine invoked by sqlite3VdbeMemRelease() in
65346 ** the unusual case where there really is memory in p that needs
65451 /* (double)0 In case of SQLITE_OMIT_FLOATING_POINT... */
65456 /* (double)0 In case of SQLITE_OMIT_FLOATING_POINT... */
65480 ** The second and third terms in the following conditional enforces
65542 ** Cast the datatype of the value in pMem according to the affinity
65543 ** "aff". Casting is different from applying affinity in that a cast
65544 ** is forced. In other words, the value is converted into the desired
65545 ** affinity even if that results in loss of data. This routine is
65599 ** Delete any previous value and set the value stored in *pMem to NULL.
65647 ** Delete any previous value and set the value stored in *pMem to val,
65661 ** Delete any previous value and set the value stored in *pMem to val,
65764 /* The pFrom==0 case in the following assert() is when an sqlite3_value
65783 ** Transfer the contents of pFrom to pTo. Any existing value in pTo is
65811 ** is required to store the string, then value of pMem is unchanged. In
65817 int n, /* Bytes in string, or negative */
65852 ** also sets a flag in local variable "flags" to indicate the memory
65912 ** to read from the disk) then the pMem is left in an inconsistent state.
65919 Mem *pMem /* OUT: Return data in this Mem structure. */
65945 Mem *pMem /* OUT: Return data in this Mem structure. */
66015 ** external API. It works in a similar way to sqlite3_value_text(),
66016 ** except the data returned is in the encoding specified by the second
66148 int nName; /* Size of function name in bytes */
66221 ** Extract a value from the supplied expression in the manner described
66228 ** in all cases.
66269 /* Handle negative integers in a single step. This is needed in the
66381 ** This is used to convert the value stored in the 'sample' column of the
66417 ** Register built-in functions used to help read ANALYZE data.
66438 ** A value is extracted in the following cases:
66440 ** * (pExpr==0). In this case the value is assumed to be an SQL NULL,
66496 ** in the sqlite_stat4 table.
66502 ** * (pExpr==0). In this case the value is assumed to be an SQL NULL,
66526 UnpackedRecord **ppRec, /* IN/OUT: Probe record */
66555 ** is undefined in this case.
66567 ** Extract the iCol-th column from the nRec-byte record in pRec. Write
66577 int nRec, /* Size of buffer pRec in bytes */
66582 int nHdr; /* Size of the header in the record */
66657 ** The sqlite3ValueBytes() routine returns the number of bytes in the
66686 ** The author disclaims copyright to this source code. In place of
66724 ** Change the error string stored in Vdbe.zErrMsg
66783 ** SQLITE_NOMEM. In this case Vdbe.aOp and Parse.nOpAlloc remain
66827 ** Add a new instruction to the list of instructions current in the
66916 ** One register is initialized for each characgter in zTypes[]. For each
66917 ** "s" character in zTypes[], the register is a string if the argument is
66919 ** in zTypes[], the register is initialized to an integer.
67061 ** in a Vdbe main program and each of the sub-programs (triggers) it may
67079 int nSub; /* Number of entries in apSub */
67128 ** Check if the program stored in the VM associated with pParse may
67144 ** part of an assert statement in the compiler. Similar to:
67180 ** true for this case to prevent the assert() in the callers frame
67195 ** and store that value in *pMaxFuncArgs.
67292 ** Before returning, *pnOp is set to the number of entries in the returned
67294 ** the number of entries in the Vdbe.apArg[] array required to execute the
67542 ** the Vdbe. In these cases we can just copy the pointer.
67612 ** in a production build.
67696 ** The Synopsis: field in comments in the vdbe.c source file gets converted
67697 ** to an extra string that is appended to the sqlite3OpcodeName(). In the
67710 int nTemp /* Space available in zTemp[] */
67892 ** The prepared statements need to know in advance the complete set of
67894 ** is maintained in p->btreeMask. The p->lockMask value is the subset of
67910 ** that may be accessed by the VM passed as an argument. In doing so it also
67923 ** statement p will ever use. Let N be the number of bits in p->btreeMask
68045 ** allocated by the OP_Program opcode in sqlite3VdbeExec().
68060 ** Give a listing of the program in the virtual machine.
68068 ** are shown in a different format. p->explain==2 is used to implement
68106 ** nRow is the sum of the number of rows in the main program, plus
68107 ** the sum of the number of rows in all trigger subprograms encountered
68144 /* The output line number is small enough that we are still in the
68171 ** kept in p->aMem[9].z to hold the new program - assuming this subprogram
68316 ** to allocate. If there is insufficient space in *ppFrom to satisfy the
68322 u8 **ppFrom, /* IN/OUT: Allocate from *ppFrom */
68339 ** Rewind the VDBE back to the beginning in preparation for
68404 int nArg; /* Number of arguments in subprograms */
68423 if( nOnce==0 ) nOnce = 1; /* Ensure at least one byte in p->aOnceFlag[] */
68429 ** cursor 0 is stored in memory cell nMem. Memory cell (nMem-1)
68437 ** an array to marshal SQL function arguments in.
68452 /* Memory for registers, parameters, cursor, etc, is allocated in two
68456 ** pass will fill in the rest using a fresh allocation.
68535 ** Close all cursors in the current frame.
68551 ** Copy the values stored in the VdbeFrame structure to its Vdbe. This
68578 ** Also release any dynamic memory held by the VM in the Vdbe.aMem memory
68580 ** pointers to VdbeFrame objects, which may in turn contain pointers to
68704 ** virtual module tables written in this transaction. This has to
68744 ** string, it means the main database is :memory: or a temp file. In
68761 ** but could happen. In this case abandon processing and return the error.
68827 /* Write the name of each database file in the transaction into the new
68868 /* Sync all the db files involved in the transaction. The same call
68869 ** sets the master journal pointer in each individual journal. If
68875 ** in case the master journal file name was written into the journal
68929 ** matches the number of vdbe's in the list sqlite3.pVdbe that are
68976 ** In this case (db->nStatement==0), and there is nothing to do.
69051 ** has made changes and is in autocommit mode, then commit those
69056 ** call this on a VM that is in the SQLITE_MAGIC_HALT state.
69077 ** Then the internal cache might have been left in an inconsistent
69116 ** pagerStress() in pager.c), the rollback is required to restore
69262 ** in p->rc. This routine sets that result back to SQLITE_OK.
69349 ** called), set the database error in this case as well.
69428 ** * the corresponding bit in argument mask is clear (where the first
69589 ** encapsulate the code that serializes values for storage in SQLite
69594 ** In an SQLite index record, the serial type is stored directly before
69595 ** the blob of data that it corresponds to. In a table record, all serial
69618 ** The 8 and 9 types were added in 3.3.0, file format 4. Prior versions
69623 ** Return the serial-type for the value stored in pMem.
69691 ** float in the wrong order. And that error has been propagated
69716 static u64 floatSwap(u64 in){
69723 u.r = in;
69735 ** Write the serialized data blob for the value stored in pMem into
69739 ** nBuf is the amount of space left in buf[]. The caller is responsible
69744 ** of bytes in the zero-filled tail is included in the return value only
69745 ** if those bytes were zeroed in buf[].
69794 ** and store the result in pMem. Return the number of bytes read.
69798 ** routine so that in most cases the overhead of moving the stack pointer
69892 /* These use local variables, so do them in a separate routine
69893 ** to avoid having to move the frame pointer in the common case */
69905 /* EVIDENCE-OF: R-14606-31564 Value is a BLOB that is (N-12)/2 bytes in
69907 ** EVIDENCE-OF: R-28401-00140 Value is a string in the text encoding and
69908 ** (N-13)/2 bytes in length. */
69935 int szSpace, /* Size of pSpace[] in bytes */
69965 ** Given the nKey-byte encoding of a record in pKey[], populate the
69977 u32 idx; /* Offset in aKey[] to read from */
70005 ** This function compares two index or table record keys in the same way
70009 ** in assert() statements to ensure that the optimized code in
70022 u32 szHdr1; /* Number of bytes in header */
70038 ** But in fact, mem1.u.i will never actually be used uninitialized, and doing
70055 /* Read the serial types for the next element in each key. */
70062 ** sqlite3VdbeSerialTypeLen() in the common case.
70110 ** Count the number of fields (a.k.a. columns) in the record given by
70148 ** *pMem2, respectively. Similar in spirit to "rc = (*pMem1) - (*pMem2);".
70157 /* The strings are already in the correct encoding. Call the
70336 ** fields that appear in both keys are equal, then pPKey2->default_rc is
70351 u32 szHdr1; /* Size of record header in bytes */
70352 u32 idx1; /* Offset of first type in header */
70360 ** two elements in the keys are equal. Fix the various stack variables so
70541 ** size-of-header varint at the start of (pKey1/nKey1) fits in a single
70625 ** fields. Return pPKey2->default_rc in this case. */
70637 ** at the start of (pKey1/nKey1) fits in a single byte.
70701 ** fits in a single byte (i.e. is 127 or less). varintRecordCompareInt()
70706 ** limit the size of the header to 64 bytes in cases where the first field
70738 ** Read the rowid (the last field in the record) and store it in *rowid.
70754 ** Any corruption is detected in sqlite3BtreeParseCellPtr(), though, so
70762 /* Read in the complete content of the index entry */
70813 ** the key string in pUnpacked. Write into *pRes a number
70863 ** Set a flag in the vdbe to update the change counter when it is finalised
70921 ** in a better query plan.
70935 ** in memory obtained from sqlite3_malloc) into a Vdbe.zErrMsg (text stored
70936 ** in memory obtained from sqlite3DbMalloc).
70952 ** The author disclaims copyright to this source code. In place of
70971 ** execution environment changes in a way that would alter the program
71084 ** Set all the parameters in the compiled SQL statement to NULL.
71155 /* EVIDENCE-OF: R-12793-43283 Every value in SQLite has one of five
71241 int n, /* Bytes in string, or negative */
71473 ** for version 3.6.23 or earlier do in fact depend on SQLITE_MISUSE
71564 ** be one of the values in the first assert() below. Variable p->rc
71574 ** error has occurred, then return the error code in p->rc to the
71575 ** caller. Set the error code in the database handle to the same value.
71662 ** of the amount of time that elapses between invocations. In other words,
71683 ** fails with an error message stating that the function is used in the
71698 "unable to use function %s in the requested context", zName);
71827 ** Return the number of columns in the result set for the statement pStmt.
71907 ** column value (i.e. a value returned by evaluating an SQL expression in the
71939 ** in the result set.
72151 ** Routines used to attach values to wildcards in a compiled SQL statement.
72154 ** Unbind the value bound to variable i in virtual machine p. This is the
72161 ** The error code stored in database p->db is overwritten with the return
72162 ** value in any case.
72188 /* If the bit corresponding to this variable in Vdbe.expmask is set, then
72192 ** parameter in the WHERE clause might influence the choice of query plan
72484 ** in the argument belongs. This is the same database handle that was
72486 ** the statement in the first place.
72501 ** Return true if the prepared statement is in need of being reset.
72623 ** The author disclaims copyright to this source code. In place of
72644 ** bytes in this text up to but excluding the first character in
72646 ** the total number of bytes in the text.
72668 ** This function returns a pointer to a nul-terminated string in memory
72683 ** ALGORITHM: Scan the input string looking for host parameters in any of
72688 ** parameter index is known, locate the value in p->aVar[]. Then render
72689 ** the value as a literal in place of the host parameter name.
72754 int nOut; /* Number of bytes of the string text to include in output */
72785 int nOut; /* Number of bytes of the blob to include in output */
72814 ** The author disclaims copyright to this source code. In place of
72822 ** The code in this file implements the function that runs the
72825 ** Various scripts scan this source file in order to generate HTML
72827 ** of the code in this file is, therefore, important. See other comments
72828 ** in this file for details. If in doubt, do not deviate from existing
72839 ** is changed while the copy is still in use, the string or blob might
72862 ** each instruction in the VDBE. When it reaches zero, the u1.isInterrupted
72863 ** field of the sqlite3 structure is set in order to simulate an interrupt.
72866 ** in an ordinary build.
72932 ** source code is in a single file (the amalgamation). Special values 1
72970 ** string that the register itself controls. In other words, it
72987 int nField, /* Number of fields in the table or index */
72997 ** purposes in a vdbe program. The different uses might require
73039 ** do so without loss of information. In other words, if the string
73304 /************** Include hwtime.h in the middle of vdbe.c *********************/
73309 ** The author disclaims copyright to this source code. In place of
73393 /************** Continuing where we left off in vdbe.c ***********************/
73401 ** the number of non-transaction savepoints currently in the
73591 ** separate instruction in the virtual machine. If we follow the usual
73595 ** big comment (similar to this one) will mark the point in the code where
73607 ** Other keywords in the comment that follows each case are used to
73614 ** comment lines are used in the generation of the opcode.html documentation
73620 ** Do not deviate from the formatting style currently in use.
73686 /* Most jump operations do a goto to this spot in order to update
73695 ** Jump to the next instruction after the address in register P1. After
73731 ** The instruction at the address in register P1 is a Yield.
73752 ** Swap the program counter with the value in register P1. This
73778 ** Check the value in register P3. If it is NULL then Halt using
73780 ** value in register P3 is not NULL, then this routine is a no-op.
73838 ** an IGNORE exception. In this case jump to the address specified
73864 zLogFmt = "abort at %d in [%s]: %s";
73966 ** The string value P4 of length P1 (bytes) is stored in register P2.
73995 ** NULL into register P3 and every register in between P2 and P3. If P3
74039 ** blob in register P2.
74055 ** If the parameter is named, then its name appears in P4.
74076 ** Move the P3 values in register P1..P1+P3-1 over into
74204 /* If the SQLITE_CountRows flag is set in sqlite3.flags mask, then
74209 ** In case this is such a statement, close any statement transaction
74253 ** Add the text in register P1 onto the end of the text in
74254 ** register P2 and store the result in register P3.
74255 ** If either the P1 or P2 text are NULL then store NULL in P3.
74301 ** Add the value in register P1 to the value in register P2
74302 ** and store the result in register P3.
74309 ** Multiply the value in register P1 by the value in register P2
74310 ** and store the result in register P3.
74316 ** Subtract the value in register P1 from the value in register P2
74317 ** and store the result in register P3.
74323 ** Divide the value in register P1 by the value in register P2
74324 ** and store the result in register P3 (P3=P2/P1). If the value in
74332 ** register P1 and store the result in register P3.
74333 ** If the value in register P1 is zero the result is NULL.
74390 /* (double)0 In case of SQLITE_OMIT_FLOATING_POINT... */
74429 ** be returned. This is used by the built-in min(), max() and nullif()
74438 ** publicly. Only built-in functions have access to this feature.
74453 ** successors. The result of the function is stored in register P3.
74471 ** in register P3. Register P3 must not be one of the function inputs.
74483 ** changed to this OP_Function opcode. In this way, the initialization of
74517 /* If this function is inside of a trigger, the register array in aMem[]
74563 ** Take the bit-wise AND of the values in register P1 and P2 and
74564 ** store the result in register P3.
74570 ** Take the bit-wise OR of the values in register P1 and P2 and
74571 ** store the result in register P3.
74577 ** Shift the integer value in register P2 to the left by the
74578 ** number of bits specified by the integer in register P1.
74579 ** Store the result in register P3.
74585 ** Shift the integer value in register P2 to the right by the
74586 ** number of bits specified by the integer in register P1.
74587 ** Store the result in register P3.
74616 /* If shifting by a negative amount, shift in the other direction */
74645 ** Add the constant P2 to the value in register P1.
74660 ** Force the value in register P1 to be an integer. If the value
74661 ** in P1 is not an integer and cannot be converted into an integer
74706 ** Force the value in register P1 to be the type defined by P2.
74737 ** Compare the values in register P1 and P3. If reg(P3)<reg(P1) then
74755 ** are text, then the appropriate collating function specified in
74763 ** store a boolean result (either 0, or 1, or NULL) in register P2.
74765 ** If the SQLITE_NULLEQ bit is set in P5, then NULL values are considered
74773 ** the operands in registers P1 and P3 are not equal. See the Lt opcode for
74776 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either
74786 ** the operands in registers P1 and P3 are equal.
74789 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either
74937 ** of integers in P4.
74940 ** the OPFLAG_PERMUTE bit set in P5. Typically the OP_Permutation should
74953 ** Compare two vectors of registers in reg(P1)..reg(P1+P3-1) (call this
74954 ** vector "A") and in reg(P2)..reg(P2+P3-1) ("B"). Save the result of
74959 ** OPFLAG_PERMUTE bit is clear, then register are compared in sequential
75020 ** in the most recent OP_Compare instruction the P1 vector was less than
75037 ** Take the logical AND of the values in registers P1 and P2 and
75047 ** Take the logical OR of the values in register P1 and P2 and
75048 ** store the answer in register P3.
75091 ** Interpret the value in register P1 as a boolean value. Store the
75092 ** boolean complement in register P2. If the value in register P1 is
75093 ** NULL, then a NULL is stored in P2.
75111 ** a NULL then store a NULL in P2.
75128 ** In other words, this opcode causes all following opcodes up through P2
75148 ** Jump to P2 if the value in register P1 is true. The value
75150 ** in P1 is NULL then take the jump if and only if P3 is non-zero.
75154 ** Jump to P2 if the value in register P1 is False. The value
75156 ** in P1 is NULL then take the jump if and only if P3 is non-zero.
75182 ** Jump to P2 if the value in register P1 is NULL.
75196 ** Jump to P2 if the value in register P1 is not NULL.
75214 ** values in the record, extract a NULL.
75216 ** The value extracted is stored in register P3.
75233 i64 payloadSize64; /* Number of bytes in the record */
75246 u32 szField; /* Number of bytes in the content of a field */
75349 ** parsed and valid information is in aOffset[] and pC->aType[].
75352 /* If there is more header available for parsing in the record, try
75370 /* Fill in pC->aType[i] and aOffset[i] values through the p2-th field. */
75455 ** and blobs and whatever is in the payloadSize64 variable
75472 ** that string in case the cursor moves before the column value is
75499 ** memory cell in the range.
75522 ** use as a data record in a database table or as a key
75523 ** in an index. The OP_Column opcode can decode the record later.
75530 ** macros defined in sqliteInt.h.
75541 int nVarint; /* Number of bytes in a varint */
75545 int nField; /* Number of fields in the record */
75548 int i; /* Space used in zNewRecord[] header */
75549 int j; /* Space used in zNewRecord[] content */
75617 ** which determines the total number of bytes in the header. The varint
75618 ** value is the size of the header in bytes including the size varint
75656 /* EVIDENCE-OF: R-64536-51728 The values for each column in the record
75670 pOut->enc = SQLITE_UTF8; /* In case the blob is ever converted to text */
75679 ** Store the number of entries (an integer value) in the table or index
75680 ** opened by cursor P1 in register P2
75730 sqlite3VdbeError(p, "cannot open savepoint - SQL statements in progress");
75788 "SQL statements in progress");
75848 ** constraint violations present in the database to the value stored
75899 "SQL statements in progress");
75953 ** connection is currently not in autocommit mode, or if there are other
75964 ** cookie in P3 differs from the schema cookie in the database header or
75965 ** if the schema generation counter in P4 differs from the current
76036 ** stored with the in-memory representation of the schema, do
76039 ** If virtual-tables are in use, this is not just an optimization.
76040 ** Often, v-tables store their data in other SQLite tables, which
76116 /* Record changes in the file format */
76132 ** P2 in a database file. The database file is determined by P3.
76155 ** value, it is set to the number of columns in the table.
76164 ** number of P2 and if it is this opcode becomes a no-op. In other words,
76184 ** value, it is set to the number of columns in the table, or to the
76188 ** in read/write mode. For a given table, there can be one or more read-only
76301 ** P2 is the number of columns in the ephemeral table.
76304 ** that defines the format of keys in the index.
76307 ** in btree.h. These flags control aspects of the operation of
76317 ** indices in joins.
76411 ** Synopsis: P3 columns in r[P2]
76415 ** register P2. In other words, cursor P1 becomes an alias for the
76416 ** MEM_Blob content contained in register P2.
76423 ** P3 is the number of fields in the records that will be stored by
76458 ** (P4_INT64) in which the first 63 bits are one for each of the
76476 ** use the value in register P3 as the key. If cursor P1 refers
76477 ** to an SQL index, then P3 is the first in an array of P4 registers
76484 ** This opcode leaves the cursor configured to move in forward order,
76485 ** from the beginning toward the end. In other words, the cursor is
76494 ** use the value in register P3 as a key. If cursor P1 refers
76495 ** to an SQL index, then P3 is the first in an array of P4 registers
76502 ** This opcode leaves the cursor configured to move in forward order,
76503 ** from the beginning toward the end. In other words, the cursor is
76512 ** use the value in register P3 as a key. If cursor P1 refers
76513 ** to an SQL index, then P3 is the first in an array of P4 registers
76520 ** This opcode leaves the cursor configured to move in reverse order,
76521 ** from the end toward the beginning. In other words, the cursor is
76530 ** use the value in register P3 as a key. If cursor P1 refers
76531 ** to an SQL index, then P3 is the first in an array of P4 registers
76538 ** This opcode leaves the cursor configured to move in reverse order,
76539 ** from the end toward the beginning. In other words, the cursor is
76587 /* The input value in P3 might be of any type: integer, real, string,
76732 ** is a prefix of any entry in P1 then a jump is made to P2 and
76735 ** This operation leaves the cursor in a state where it can be
76736 ** advanced in the forward direction. The Next instruction will work,
76749 ** is not the prefix of any entry in P1 then a jump is made to P2. If P1
76754 ** This operation leaves the cursor in a state where it cannot be
76755 ** advanced in either direction. In other words, the Next and Prev
76769 ** record are not-NULL then a check is done to determine if any row in the
76777 ** This operation leaves the cursor in a state where it cannot be
76778 ** advanced in either direction. In other words, the Next and Prev
76878 ** This opcode leaves the cursor in a state where it cannot be advanced
76879 ** in either direction. In other words, the Next and Prev opcodes will
76944 ** The record number is not previously used as a key in the database
76948 ** If P3>0 then P3 is a register in the root frame of this VDBE that holds
76972 ** thing) is obtained in a two-step algorithm.
76980 ** it already exists in the table. If it does not exist, we have
77050 assert( pOp->p3==0 ); /* We cannot be in random rowid mode if this is
77078 ** entry is overwritten. The data is the value MEM_Blob stored in register
77079 ** number P2. The key is stored in register P3. The key must
77117 ** integer value P3, not the value of the integer stored in register P3.
77190 ** either the next or the previous record in the table. If it is left
77192 ** no-op. As a result, in this case it is OK to delete a record from within a
77193 ** Next loop. If P5 is zero, then the cursor is left in an undefined state.
77261 ** record blob in register P3 against a prefix of the entry that
77265 ** If either P3 or the sorter contains a NULL in one of their significant
77320 ** it is found in the database file.
77331 ** it is found in the database file.
77361 ** would fail. Should this ever change (because of changes in the code
77398 pOut->enc = SQLITE_UTF8; /* In case the blob is ever cast to text */
77407 ** Store in register P2 an integer which is the key of the table entry that
77476 ** will refer to the last entry in the database table or index.
77481 ** This opcode leaves the cursor configured to move in reverse order,
77482 ** from the end toward the beginning. In other words, the cursor is
77536 ** will refer to the first entry in the database table or index.
77541 ** This opcode leaves the cursor configured to move in forward order,
77542 ** from the beginning toward the end. In other words, the cursor is
77576 ** Advance cursor P1 so that it points to the next key/data pair in its
77597 ** number P5-1 in the prepared statement is incremented.
77608 ** Back up cursor P1 so that it points to the previous key/data pair in its
77630 ** number P5-1 in the prepared statement is incremented.
77785 ** Write into register P2 an integer which is the last entry in the record at
77915 ** Delete an entire database table or index whose root page in the database
77918 ** The table being destroyed is in the main database file if P3==0. If
77919 ** P3==1 then the table to be clear is in the auxiliary database file
77923 ** might be moved into the newly deleted root page in order to keep all
77926 ** is stored in register P2. If no page
77928 ** the last one in the database) then a zero is stored in register P2.
77929 ** If AUTOVACUUM is disabled then a zero is stored in register P2.
77965 ** in the database file is given by P1. But, unlike Destroy, do not
77968 ** The table being clear is in the main database file if P2==0. If
77969 ** P2==1 then the table to be clear is in the auxiliary database file
77973 ** intkey table (an SQL table, not an index). In this case the row change
77974 ** count is incremented by the number of rows in the table being cleared.
77975 ** If P3 is greater than zero, then the value stored in register P3 is
77976 ** also incremented by the number of rows in the table being cleared.
78026 ** Allocate a new table in the main database file if P1==0 or in the
78027 ** auxiliary database file if P1==1 or in an attached database if
78040 ** Allocate a new index in the main database file if P1==0 or in the
78041 ** auxiliary database file if P1==1 or in an attached database if
78142 ** Remove the internal (in-memory) data structures that describe
78143 ** the table named P4 in database P1. This is called after a table
78144 ** is dropped from disk (using the Destroy opcode) in order to keep
78155 ** Remove the internal (in-memory) data structures that describe
78156 ** the index named P4 in database P1. This is called after an index
78158 ** in order to keep the internal representation of the
78168 ** Remove the internal (in-memory) data structures that describe
78169 ** the trigger named P4 in database P1. This is called after a trigger
78170 ** is dropped from disk (using the Destroy opcode) in order to keep
78183 ** Do an analysis of the currently open database. Store in
78185 ** If no problems are found, store a NULL in register P1.
78189 ** In other words, the analysis stops as soon as reg(P1) errors are
78192 ** The root page numbers of all tables in the database are integer
78193 ** stored in reg(P1), reg(P1+1), reg(P1+2), .... There are P2 tables
78247 ** held in register P1.
78290 ** Synopsis: if r[P3] in rowset(P1) goto P2
78294 ** the value held in P3, jump to register P2. Otherwise, insert the
78295 ** integer in P3 into the RowSet and continue on to the
78322 /* If there is anything other than a rowset object in memory cell P1,
78351 ** cell in an array of values used as arguments to the sub-program. P2
78354 ** of a memory cell in this (the parent) VM that is used to allocate the
78366 Mem *pEnd; /* Last memory cell in new array */
78367 VdbeFrame *pFrame; /* New vdbe frame to execute in */
78381 In some cases a single trigger may generate more than one
78404 ** program stored in SubProgram.aOp. As well as these, one memory
78476 ** This opcode is only ever present in sub-programs called via the
78477 ** OP_Program instruction. Copy a value currently stored in a memory
78478 ** cell of the calling (parent) frame to cell P2 in the current frames
78482 ** The address of the cell in the parent frame is determined by adding
78546 ** P1 is a register in the root frame of this VM (the root frame is
78549 ** its current value and the value in register P2.
78578 ** value in P1 and jump to P2.
78615 ** initially nonzero, then subtract P3 from the value in register P1 and
78633 ** Register P1 must hold an integer. Decrement the value in register P1
78684 ** the FuncDef stored in P4 is converted into an sqlite3_context and
78685 ** the opcode is changed. In this way, the initialization of the
78720 /* If this function is inside of a trigger, the register array in aMem[]
78792 ** Checkpoint database P1. This is a no-op if P1 is not currently in
78795 ** SQLITE_BUSY or not, respectively. Write the number of pages in the
78797 ** in the WAL that have been checkpointed after the checkpoint
78870 ** in temporary storage or if the VFS does not support shared memory
78996 ** P1 is the index of the database in sqlite3.aDb[] of the database
79044 ** P2 is a register that holds the name of a virtual table in database
79072 ** P4 is the name of a virtual table in database P1. Call the xDestroy method
79088 ** table and stores that cursor in P1.
79138 ** by P1. The integer query plan parameter to xFilter is stored in register
79240 ** Advance virtual table P1 to the next row in its result set and
79285 ** in register P1 is passed as the zName argument to the xRename method.
79318 ** invocation. The value in register (P3+P2-1) corresponds to the
79326 ** rowid for itself. The subsequent elements in the array are
79327 ** the values of columns in the new row.
79337 ** apply in the case of a constraint failure on an insert or update.
79395 ** Write the current number of pages in database P1 to memory cell P2.
79408 ** Try to set the maximum page count for database P1 to the value in P3.
79412 ** Store the maximum page count after the change in register P2.
79438 ** the UTF-8 string contained in P4 is emitted on the trace callback.
79488 ** the same as a no-op. This opcodesnever appears in a real VM program.
79603 ** The author disclaims copyright to this source code. In place of
79626 int nByte; /* Size of open blob, in bytes */
79627 int iOffset; /* Byte offset of blob in cursor data */
79642 ** contain a value of type TEXT or BLOB in the column nominated when the
79716 int iCol; /* Index of zColumn in row-record */
79900 ** we can invoke OP_Column to fill in the vdbe cursors type
79986 ** already been invalidated. Return SQLITE_ABORT in this case.
80056 ** already been invalidated. Return SQLITE_ABORT in this case.
80082 ** The author disclaims copyright to this source code. In place of
80090 ** This file contains code for the VdbeSorter object, used in concert with
80105 ** object. The row is a binary blob in the
80108 ** in the case of a SELECT w/ ORDER BY, or
80110 ** in the case of a CREATE INDEX.
80126 ** Used to enforce uniqueness in a
80136 ** The interfaces above must be called in a particular order. Write() can
80137 ** only occur in between Init()/Reset() and Rewind(). Next(), Rowkey(), and
80138 ** Compare() can only occur in between Rewind() and Close()/Reset(). i.e.
80150 ** unsorted in main memory. Assuming the amount of memory used never exceeds
80152 ** an in-memory merge sort. In this case, no temporary files are required
80156 ** If the amount of space used to store records in main memory exceeds the
80157 ** threshold, then the set of records currently in memory are sorted and
80158 ** written to a temporary file in "Packed Memory Array" (PMA) format.
80167 ** that returned by "PRAGMA main.cache_size", in bytes.
80169 ** If the sorter is running in single-threaded mode, then all PMAs generated
80170 ** are appended to a single temporary file. Or, if the sorter is running in
80172 ** N is the configured number of worker threads. In this case, instead of
80178 ** The sorter is running in multi-threaded mode if (a) the library was built
80183 ** When Rewind() is called, any data remaining in memory is flushed to a
80184 ** final PMA. So at this point the data is stored in some number of sorted
80187 ** If there are fewer than SORTER_MAX_MERGE_COUNT PMAs in total and the
80188 ** sorter is running in single-threaded mode, then these PMAs are merged
80190 ** MergeEngine object, described in further detail below, performs this
80193 ** Or, if running in multi-threaded mode, then a background thread is
80202 ** If there are more than SORTER_MAX_MERGE_COUNT PMAs in total when
80209 ** If running in multi-threaded mode and there are more than
80221 ** messages to stderr that may be helpful in understanding the performance
80222 ** characteristics of the sorter in multi-threaded mode.
80229 ** Hard-coded maximum amount of data to accumulate in memory before flushing
80242 typedef struct SortSubtask SortSubtask; /* A sub-task in the sort process */
80244 typedef struct SorterList SorterList; /* In-memory list of records */
80249 ** stored in the file.
80253 i64 iEof; /* Bytes of data stored in pFd */
80257 ** An in-memory list of objects to be sorted.
80261 ** are stored in the aMemory[] bulk memory, one right after the other, and
80267 int szPMA; /* Size of pList as PMA in bytes */
80273 ** combined into one big PMA in order to be able to step through the sorted
80274 ** records in order.
80280 ** actually N elements in size, where N is the smallest power of 2 greater
80284 ** The aTree[] array is also N elements in size. The value of N is stored in
80316 ** be advanced to the next key in its segment. Say the next key is
80330 ** In other words, each time we advance to the next sorter element, log2(N)
80342 ** This object represents a single thread of control in a sort operation.
80352 in
80368 ** In both cases, the effects of the main thread seeing (bDone==0) even
80379 int nPMA; /* Number of PMAs currently in file */
80393 ** largest record in the sorter.
80396 int mnPmaSize; /* Minimum PMA size, in bytes */
80397 int mxPmaSize; /* Maximum PMA size, in bytes. 0==no limit */
80405 SorterList list; /* List of in-memory records */
80406 int iMemory; /* Offset of free space in list.aMemory */
80407 int nMemory; /* Size of list.aMemory allocation in bytes */
80421 ** PMA, in sorted order. The next key to be read is cached in nKey/aKey.
80432 int nKey; /* Number of bytes in key */
80437 int nBuffer; /* Size of read buffer in bytes */
80452 ** and aFile[1]. Neither file is allowed to grow to more than mxSz bytes in
80488 ** size. But I/O is more efficient if it occurs in page-sized blocks where
80493 int eFWErr; /* Non-zero if in an error state */
80495 int nBuffer; /* Size of write buffer in bytes */
80498 i64 iWriteOff; /* Offset of start of buffer in file */
80504 ** held in memory and prior to being written out as part of a PMA.
80507 ** by this module. If using a separate allocation for each in-memory record
80515 ** has finished passing records to the sorter, or when the in-memory buffer
80521 int nVal; /* Size of the record in bytes */
80523 SorterRecord *pNext; /* Pointer to next record in list */
80561 ** The buffer returned in *ppOut is only valid until the
80570 int nAvail; /* Bytes of data available in buffer */
80582 ** than p->nBuffer bytes remaining in the PMA, read all remaining data. */
80604 /* The requested data is available in the in-memory buffer. In this
80610 /* The requested data is not all available in the in-memory buffer.
80611 ** In this case, allocate space at p->aAlloc[] to copy the requested
80626 /* Copy as much data as is available in the buffer into the start of
80689 ** *pp is undefined in this case.
80712 i64 iOff /* Offset in pFile */
80752 ** Advance PmaReader pReadr to the next key in its PMA. Return SQLITE_OK if
80757 u64 nRec = 0; /* Size of record in bytes */
80794 ** Initialize PmaReader pReadr to scan through the PMA stored in file pFile
80796 ** leaves the PmaReader pointing to the first key in the PMA (or EOF if the
80805 i64 iStart, /* Start offset in pFile */
80807 i64 *pnByte /* IN/OUT: Increment this value by PMA size */
80818 u64 nByte; /* Size of PMA in bytes */
80854 ** If IN/OUT parameter *pbKey2Cached is true when this function is called,
80994 ** statement. In this case, keys are always delivered to the sorter in
81000 ** The sorter can guarantee a stable sort when running in single-threaded
81001 ** mode, but not in multi-threaded mode.
81007 int nField, /* Number of key fields in each record */
81015 int szKeyInfo; /* Size of pCsr->pKeyInfo in bytes */
81016 int sz; /* Size of pSorter in bytes */
81201 void *(*xTask)(void*), /* Routine to run in a separate thread */
81342 ** is guaranteed to be nByte bytes or smaller in size. This function
81343 ** attempts to extend the file to nByte bytes in size and to ensure that
81600 ** Write the current contents of in-memory linked-list pList to a level-0
81601 ** PMA in the temp file belonging to sub-task pTask. Return SQLITE_OK if
81607 ** in the PMA (not including the varint itself).
81609 ** * One or more records packed end-to-end in order of ascending keys.
81611 ** key). The varint is the number of bytes in the blob of data.
81696 ** advanced (iPrev) and the one next to it in the array. */
81701 /* Compare pReadr1 and pReadr2. Store the result in variable iRes. */
81714 ** Then set pReadr2 to the next PmaReader to compare to pReadr1. In this
81715 ** case there is no cache of pReadr2 in pTask->pUnpacked, so set
81776 /* Select a sub-task to sort and flush the current list of in-memory
81777 ** records to disk. If the sorter is running in multi-threaded mode,
81858 ** already in memory and (b) the new value will not fit in memory.
81863 ** * The total memory allocated for the in-memory list is greater
81866 ** * The total memory allocated for the in-memory list is greater
81932 ** of the data stored in aFile[1] is the same as that used by regular PMAs,
81955 ** In either case exit the loop. */
82005 ** aFile[0] and aFile[1] in place. If the contents of pMerger have not
82047 ** If an OOM condition is encountered, return NULL. In this case free the
82089 int iOut /* Store the result in pMerger->aTree[iOut] */
82136 ** Only INCRINIT_NORMAL is valid in single-threaded builds (when
82154 ** MergeEngine object in the usual fashion.
82159 ** set the PmaReader objects up to read from it. In this case all that is
82178 /* eMode is always INCRINIT_NORMAL in single-threaded mode */
82187 /* PmaReaders should be normally initialized in order, as if they are
82189 ** However, in the INCRINIT_ROOT case, if PmaReader aReadr[nTask-1] is
82190 ** in use it will block the vdbePmaReaderNext() call while it uses
82214 ** in the sub-tree headed by pReadr are also initialized. Data is then
82216 ** the first key in its range.
82219 ** to be a multi-threaded PmaReader and this function is being called in a
82220 ** background thread. In this case all PmaReaders in the sub-tree are
82227 ** in the INCRINIT_TASK case is that vdbePmaReaderNext() assumes that it has
82235 ** In this case vdbePmaReaderNext() is called on all child PmaReaders and
82236 ** the current PmaReader set to point to the first key in its range.
82246 /* eMode is always INCRINIT_NORMAL in single-threaded mode */
82282 ** then this function is already running in background thread
82285 ** If this is the INCRINIT_ROOT object, then it is running in the
82304 ** The main routine for vdbePmaReaderIncrMergeInit() operations run in
82361 i64 *piOffset, /* IN/OUT: Readr offset in pTask->file */
82546 ** all records stored in the sorter.
82594 ** in single-threaded mode. This is important, as the
82630 ** in sorted order.
82640 ** from the in-memory list. */
82651 /* Write the current in-memory list to a PMA. When the VdbeSorterWrite()
82676 ** Advance to the next element in the sorter.
82715 int *pnKey /* OUT: Size of current key in bytes */
82756 ** Compare the key in memory cell pVal with the key that the sorter cursor
82765 ** key in pVal is smaller than, equal to or larger than the current sorter
82768 ** This routine forms the core of the OP_SorterCompare opcode, which in
82810 ** The author disclaims copyright to this source code. In place of
82825 ** the caller specifies a buffer size for an in-memory buffer to
82829 ** 1) The in-memory representation grows too large for the allocated
82843 int nBuf; /* Size of zBuf[] in bytes */
82870 ** returning. This way, SQLite uses the in-memory journal data to
82970 ** Query the size of the file in bytes.
83069 ** The author disclaims copyright to this source code. In place of
83078 ** This file contains code use to implement an in-memory rollback journal.
83079 ** The in-memory rollback journal is used to journal transactions for
83089 /* Space to hold the rollback journal is allocated in increments of
83094 ** a power-of-two allocation. This minimizes wasted space in power-of-two
83103 FileChunk *pNext; /* Next chunk in the journal */
83122 FileChunk *pFirst; /* Head of in-memory chunk-list */
83128 ** Read data from the in-memory journal file. This is the implementation
83186 /* An in-memory journal file should only ever be appended to. Random
83253 ** Syncing an in-memory journal is a no-op. And, in fact, this routine
83254 ** is never called in a working implementation. This implementation
83255 ** exists purely as a contingency, in case some malfunction in some other
83264 ** Query the size of the file in bytes.
83309 ** an in-memory journal
83327 ** The author disclaims copyright to this source code. In place of
83345 ** of the expression, while descending. (In other words, the callback
83382 ** Call sqlite3WalkExpr() for every expression in list p or until
83414 ** Walk the parse trees associated with all subqueries in the
83417 ** and on any subqueries further down in the tree. Return
83442 ** Call sqlite3WalkExpr() for every expression in Select statement p.
83443 ** Invoke sqlite3WalkSelect() for subqueries in the FROM clause and
83489 ** The author disclaims copyright to this source code. In place of
83531 ** result set in pEList.
83551 int iCol, /* A column in the result set. 0..pEList->nExpr-1 */
83592 ** Return TRUE if the name zCol occurs anywhere in the USING clause.
83609 ** result sets in ExprList.a[].zSpan, in the form "DATABASE.TABLE.COLUMN".
83639 ** that name in the set of source tables in pSrcList and make the pExpr
83643 ** pExpr->iDb Set the index in db->aDb[] of the database X
83662 ** in pParse and return WRC_Abort. Return WRC_Prune on success.
83679 NameContext *pTopNC = pNC; /* First namecontext in the list */
83686 assert( zCol ); /* The Z in X.Y.Z cannot be NULL */
83694 /* Translate the schema name in zDb into a pointer to the corresponding
83696 ** resulting in an appropriate error message toward the end of this routine
83758 ** is for the right-hand table of a NATURAL JOIN or is in a
83858 ** we are resolving names in the WHERE clause of the following command:
83862 ** In cases like this, replace pExpr with a copy of the expression that
83863 ** forms the result set entry ("a+b" in the example) and return immediately.
83864 ** Note that the expression in the result set should have already been
83867 ** The ability to use an output result-set column in the WHERE, GROUP BY,
83868 ** or HAVING clauses, or as part of a larger expression in the ORDER BY
83908 ** If X and Y are NULL (in other words if only the column name Z is
83909 ** supplied) and the value of Z is enclosed in double-quotes, then
83910 ** Z is a string literal if it doesn't match any column names. In that
83915 ** fields are not changed in any context.
83941 /* If a column from a table in pSrcList is referenced, then record
83942 ** this fact in the pSrcList.a[].colUsed bitmask. Column 0 causes
83944 ** column number is greater than the number of bits in the bitmask
83986 ** from datasource iSrc in SrcList pSrc.
84024 sqlite3ErrorMsg(pParse, "%s prohibited in %s", zMsg, zIn);
84046 ** node in the expression tree. Return 0 to continue the search down
84077 ** column in the FROM clause. This is used by the LIMIT and ORDER BY
84136 int nId; /* Number of characters in function name */
84199 ** in an index. */
84266 ** a SELECT statement. pE is a term in an ORDER BY or GROUP BY clause.
84270 ** elements in pEList, corresponding to the matching entry. If there is
84298 ** pE is a pointer to an expression which is a single term in the
84306 ** Attempt to match pE against result set columns in the left-most
84309 ** The left-most column is 1. In other words, the value returned is the
84310 ** same integer value that would be used in the SQL statement to indicate
84330 /* Resolve all names in the ORDER BY term expression
84346 ** in the result set. Return an 1-based index of the matching
84374 ** Analyze the ORDER BY clause in a compound SELECT statement. Modify
84376 ** and N where N is the number of columns in the compound SELECT.
84403 sqlite3ErrorMsg(pParse, "too many terms in ORDER BY clause");
84469 "column in the result set", i+1);
84477 ** Check every term in the ORDER BY or GROUP BY clause pOrderBy of
84500 sqlite3ErrorMsg(pParse, "too many terms in %s BY clause", zType);
84520 ** pOrderBy is an ORDER BY or GROUP BY clause in SELECT statement pSelect.
84526 ** number of columns in the result set of the SELECT) then the expression
84527 ** in the resolution is a copy of the I-th result-set expression. If
84530 ** result-set expression. Otherwise, the expression is resolved in
84534 ** an appropriate error message might be left in pParse. (OOM errors
84547 int nResult; /* Number of terms in the result set */
84593 ** Resolve names in the SELECT statement p and all of its descendants.
84621 ** this routine in the correct order.
84636 /* Resolve the expressions in the LIMIT and OFFSET clauses. These
84648 ** In this case the ORDER BY clause (p->pOrderBy) should be resolved
84660 /* Recursively resolve names in all subqueries
84670 ** parent contexts. After resolving references to expressions in
84694 /* Resolve names in the result set. */
84697 /* If there are no aggregate functions in the result-set, and no GROUP BY
84698 ** expression, do not allow aggregates in any of the other expressions.
84717 ** other expressions in
84718 ** expressions in the WHERE clause (etc.) can refer to expressions by
84719 ** aliases in the result set.
84728 /* Resolve names in table-valued-function arguments */
84738 /* The ORDER BY and GROUP BY clauses may not refer to terms in
84785 sqlite3ErrorMsg(pParse, "aggregate functions are not allowed in "
84793 ** number of expressions in the select list. */
84827 ** Y: The name of a table in a FROM clause. Or in a trigger
84830 ** Z: The name of a column in table Y.
84836 ** Expr.iColumn The column index in X.Y. -1 for the rowid.
84842 ** size of an AS clause in the result-set of a SELECT. The Z expression
84845 ** tree. For example, in:
84860 ** An error message is left in pParse if anything is amiss. The number
84864 NameContext *pNC, /* Namespace to resolve expressions in. */
84902 ** Resolve all names for all expression in an expression list. This is
84907 NameContext *pNC, /* Namespace to resolve expressions in. */
84919 ** Resolve all names in all expressions of a SELECT and in all
84921 ** subqueries in expressions, and subqueries used as FROM clause
84947 ** Resolve names in expressions that can only reference a single table:
84955 ** Any errors cause an error message to be set in pParse.
84986 ** The author disclaims copyright to this source code. In place of
84995 ** for generating VDBE code that evaluates expressions in SQLite.
85007 ** i.e. the WHERE clause expressions in the following statements all
85034 ** a TK_COLUMN but was previously evaluated and cached in a register */
85048 ** If a memory allocation error occurs, that fact is recorded in pParse->db
85123 ** a TK_COLUMN but was previously evaluated and cached in a register */
85212 ** pExpr is a comparison expression, eg. '=', '<', IN(...) etc.
85215 ** the comparison in pExpr.
85248 ** Argument pRight (but not pLeft) may be a null pointer. In this case,
85298 ** expression depth allowed. If it is not, leave an error message in
85351 ** Set the Expr.nHeight variable in the structure passed as an
85376 ** leave an error in pParse.
85425 ** stored in u.zToken. Instead, the integer values is written
85493 ** In that case, delete the subtrees pLeft and pRight.
85554 ** boolean values in different circumstances (a false positive.)
85618 ** in the original SQL statement.
85737 ** to store a copy of an expression or expression tree. They differ in
85792 ** This function returns the space in bytes required to store the copy
85813 ** space to duplicate all Expr nodes in the tree formed by Expr.pLeft
85889 /* Fill in the pNew->x.pSelect or pNew->x.pList member. */
85897 /* Fill in pNew->pLeft and pNew->pRight. */
85961 ** part of the in-memory representation of the database schema.
86211 ** leave an error message in pParse.
86222 sqlite3ErrorMsg(pParse, "too many columns in %s", zObject);
86244 ** Return the bitwise-OR of all Expr.flags fields in the given
86272 ** In all cases, the callbacks set Walker.eCode=0 and abort if the expression
86276 ** in a CREATE TABLE statement. The Walker.eCode value is 5 when parsing
86326 /* A bound parameter in a CREATE statement that originates from
86378 ** for any single row of the table with cursor iCur. In other words, the
86402 ** to fit in a 32-bit integer, return 1 and put the value of the integer
86403 ** in *pValue. If the expression is not an integer or if it is too big
86404 ** to fit in a signed 32-bit integer, return 0 and leave *pValue unchanged.
86409 /* If an expression is an integer literal that fits in a signed 32-bit
86445 ** (returning TRUE when in fact the expression can never be NULL) might
86448 ** will likely result in an incorrect answer. So when in doubt, return
86473 ** unchanged by OP_Affinity with the affinity given in the second
86477 ** can be omitted. When in doubt return FALSE. A false negative
86478 ** is harmless. A false positive, however, can result in the wrong
86522 ** Return true if we are able to the IN operator optimization on a
86525 ** x IN (SELECT ...)
86530 ** The Select object passed in has already been preprocessed and no
86538 if( p==0 ) return 0; /* right-hand side of IN is SELECT */
86551 if( pSrc->nSrc!=1 ) return 0; /* Single term in FROM clause */
86558 if( pEList->nExpr!=1 ) return 0; /* One column in the result set */
86592 ** The argument is an IN operator with a list (not a subquery) on the
86608 ** This function is used by the implementation of the IN (...) operator.
86609 ** The pX parameter is the expression on the RHS of the IN operator, which
86613 ** be used either to test for membership in the RHS set or to iterate through
86616 ** A cursor is opened on the b-tree object that is the RHS of the IN operator
86626 ** IN_INDEX_NOOP - No cursor was allocated. The IN operator must be
86634 ** If the RHS of the IN operator is a list or a more complex subquery, then
86643 ** IN index will be used to loop over all values of the RHS of the
86644 ** IN operator.
86658 ** if the RHS of the IN operator is a list (not a subquery) then this
86660 ** testing is too expensive and return IN_INDEX_NOOP. In that case, the
86661 ** calling routine should implement the IN operator using a sequence
86665 ** might need to know whether or not the RHS side of the IN operator
86672 ** If a register is allocated and its location stored in *prRhsHasNull, then
86673 ** the value in that register will be NULL if the b-tree contains one or more
86679 Select *p; /* SELECT to the right of IN operator */
86713 /* This function is only called from two places. In both cases the vdbe
86730 ** be used in place of a temp-table, it must be ordered according
86762 /* If no preexisting index is available for the IN clause
86764 ** and the RHS of the IN operator is a list, not a subquery
86767 ** the IN operator so return IN_INDEX_NOOP.
86804 ** or IN operators. Examples:
86808 ** x IN (4,5,11) -- IN operator with list on right-hand side
86809 ** x IN (SELECT a FROM b) -- IN operator with subquery on the right
86811 ** The pExpr parameter describes the expression that contains the IN
86815 ** to be of the form "<rowid> IN (?, ?, ?)", where <rowid> is a reference
86816 ** to some integer key column of a table B-Tree. In this case, use an
86817 ** intkey B-Tree to store the set of IN(...) values instead of the usual
86820 ** If rMayHaveNull is non-zero, that means that the operation is an IN
86827 ** result. For IN operators or if an error occurs, the return value is 0.
86832 Expr *pExpr, /* The IN, SELECT, or EXISTS operator */
86833 int rHasNullFlag, /* Register that records whether NULLs exist in RHS */
86834 int isRowid /* If true, LHS of IN operator is a rowid */
86842 /* This code must be run in its entirety every time it is encountered
86868 char affinity; /* Affinity of the LHS of the IN */
86870 Expr *pLeft = pExpr->pLeft; /* the LHS of the IN operator */
86875 /* Whether this is an 'x IN(SELECT...)' or an 'x IN(<exprlist>)'
86893 /* Case 1: expr IN (SELECT ...)
86908 testcase( pKeyInfo==0 ); /* Caused by OOM in sqlite3KeyInfoAlloc() */
86921 /* Case 2: expr IN (exprlist)
86924 ** store it in the temporary table. If <expr> is a column, then use
86941 /* Loop through each expression in <exprlist>. */
86989 ** value of this select in a memory cell and record the number
86990 ** of the memory cell in iColumn. If this is an EXISTS, write
86992 ** and record that memory cell in iColumn.
87043 ** Generate code for an IN expression.
87045 ** x IN (SELECT ...)
87046 ** x IN
87056 ** is contained in the RHS then jump to destIfNull. If the LHS is contained
87061 Expr *pExpr, /* The IN expression */
87062 int destIfFalse, /* Jump here if LHS is not contained in the RHS */
87076 VdbeNoopComment((v, "begin IN expr"));
87087 /* Code the LHS, the <expr> from "<expr> IN (...)".
87094 ** suitable for evaluating the IN operator, then evaluate using a
87154 /* In this case, the RHS is the ROWID of table b-tree
87160 /* In this case, the RHS is an index b-tree.
87165 ** "x IN (...)" expression must be either 0 or NULL. If the set
87174 ** of a "NOT NULL" constraint in the database schema.
87177 ** for this particular IN operator.
87182 /* In this branch, the RHS of the IN might contain a NULL and
87183 ** the presence of a NULL on the RHS makes a difference in the
87188 /* First check to see if the LHS is contained in the RHS. If so,
87189 ** then the answer is TRUE the presence of NULLs in the RHS does
87190 ** not matter. If the LHS is not contained in the RHS, then the
87205 VdbeComment((v, "end IN expr"));
87283 ** Record in the column cache that a particular column from a
87284 ** particular table is stored in a particular register.
87305 ** that the object will never already be in cache. Verify this guarantee.
87380 ** the previous sqlite3ExprCachePush operation. In other words, restore
87381 ** the cache to the state it was in prior the most recent Push.
87404 ** register might be in the cache in multiple places, so be sure to
87425 int regOut /* Store the index column value in this register */
87466 ** table pTab and store the column value in a register. An effort
87467 ** is made to store the column value in register iReg, but this is
87470 ** There must be an open cursor to pTab in iTable when this routine
87542 ** Return true if any register in the range iFrom..iTo (inclusive)
87546 ** and does not appear in a normal build.
87571 ** expression. Attempt to store the results in register "target".
87575 ** be stored in target. The result might be stored in some other
87583 int inReg = target; /* Results stored in register inReg */
87625 ** in the index refer to the table to which the index belongs */
87827 int nId; /* Length of the function name in bytes */
87850 /* Attempt a direct implementation of the built-in COALESCE() and
87928 ** see if it is a column in a virtual table. This is done because
87932 ** "glob(B,A). We want to use the A in "A glob B" to test
87933 ** for function overloading. But we use the B term in "glob(B,A)".
87982 ** X is stored in pExpr->pLeft.
87983 ** Y is stored in pExpr->pList->a[0].pExpr.
87984 in pExpr->pList->a[1].pExpr.
88019 ** to a column in the new.* or old.* pseudo-tables available to
88020 ** trigger programs. In this case Expr.iTable is set to 1 for the
88029 ** set to (n+1), where n is the number of columns in each pseudo-table.
88030 ** For a reference to any other column in the new.* pseudo-table, p1
88085 ** X (if it exists) is in pExpr->pLeft.
88086 ** Y is in the last element of pExpr->x.pList if pExpr->x.pList->nExpr is
88087 ** odd. The Y is also optional. If the number of elements in x.pList
88089 ** Ei is in pExpr->pList->a[i*2] and Ri is pExpr->pList->a[i*2+1].
88122 ** The value in regFree1 might get SCopy-ed into the file result.
88196 int regDest, /* Store the value in this register */
88222 ** code to fill the register in the initialization section of the
88223 ** VDBE program, in order to factor it out of the evaluation loop.
88260 ** results in register target. The results are guaranteed to appear
88261 ** in register target.
88280 ** results in register target. The results are guaranteed to appear
88281 ** in register target. If the expression is constant, then this routine
88294 ** in register target.
88388 Expr exprAnd; /* The AND operator in x>=y AND x<=z */
88434 in vdbe.c and the mkopcodeh.awk script in
88435 ** the make process cause these values to align. Assert()s in the code
88732 ** to compare equal to an equivalent element in pA with Expr.iTable==iTab.
88742 ** can be sure the expressions are the same. In the places where
88744 ** just might result in some slightly slower code. But returning
88792 ** non-zero if they differ in any way.
88795 ** to compare equal to an equivalent element in pA with Expr.iTable==iTab.
88835 ** When in doubt, return false. Returning true might give a performance
88860 ** to count references to table columns in the arguments of an
88861 ** aggregate function, in order to implement the
88865 SrcList *pSrc; /* One particular FROM clause in a nested query */
88866 int nThis; /* Number of references to columns in pSrcList */
88867 int nOther; /* Number of references to columns in other FROM clauses */
88877 ** sqlite3FunctionUsesThisSrc() is used differently in the future, the
88900 ** references columns but not columns of tables found in pSrcList.
88965 /* Check to see if the column is in one of the tables in the FROM
88974 ** that is in the FROM clause of the aggregate query.
88976 ** Make an entry for the column in pAggInfo->aCol[] if there
89015 /* There is now an entry for pExpr in pAggInfo->aCol[] (either
89035 ** function that is already in the pAggInfo structure
89044 /* pExpr is original. Make a new entry in pAggInfo->aFunc[]
89104 ** Call sqlite3ExprAnalyzeAggregates() for every expression in an
89189 ** The author disclaims copyright to this source code. In place of
89203 ** The code in this file only exists if we are not omitting the
89212 ** CREATE INDEX command. The second is a table name. The table name in
89240 /* The principle used to locate the table name in the CREATE TABLE
89251 /* Store the token that zCsr points to in tname. */
89255 /* Advance zCsr to the next token. Store that token type in 'token',
89256 ** and its length in 'len' (to be used next iteration of this loop).
89339 ** statement. The second is a table name. The table name in the CREATE
89362 /* The principle used to locate the table name in the CREATE TRIGGER
89375 /* Store the token that zCsr points to in tname. */
89379 /* Advance zCsr to the next token. Store that token type in 'token',
89380 ** and its length in 'len' (to be used next iteration of this loop).
89404 ** in the CREATE TRIGGER statement.
89414 ** Register built-in functions used to help implement ALTER TABLE
89448 ** In this case zWhere is passed to sqlite3DbFree() before returning.
89482 ** table pTab has no temporary triggers, or is itself stored in the
89490 /* If the table is not located in the temp-db (in which case NULL is
89493 ** expression being built up in zWhere.
89514 ** Argument zName is the name of the table in the database schema at
89551 /* Now, if the table is not stored in the temp database, reload any temp
89552 ** triggers. Don't use IN(...) in case SQLITE_OMIT_SUBQUERY is defined.
89564 ** in pParse->zErr (system tables may not be altered) and returns non-zero.
89590 int nTabName; /* Number of UTF-8 characters in zTabName */
89615 ** in database iDb. If so, this is an error.
89685 /* figure out how many UTF-8 characters are in zName */
89730 /* If the sqlite_sequence table exists in this database, then update
89742 ** table. Don't do this if the table being ALTERed is itself located in
89924 ** This function is called by the parser after the table-name in
89973 /* Put a copy of the Table struct in Parse.pNewTable for the
90024 ** The author disclaims copyright to this source code. In place of
90045 ** Additional tables might be added in future releases of SQLite.
90054 ** SQLITE_ENABLE_STAT4 and in SQLite versions 3.8.1 and later. It is
90064 ** name in the idx column. The tbl column is the name of the table to
90065 ** which the index belongs. In each such row, the stat column will be
90066 ** a string consisting of a list of integers. The first integer in this
90067 ** list is the number of rows in the index. (This is the same as the
90068 ** number of rows in the table, except for partial indices.) The second
90069 ** integer is the average number of rows in the index that have the same
90070 ** value in the first column of the index. The third integer is the average
90071 ** number of rows in the index that have the same value for the first two
90072 ** columns. The N-th integer (for N>1) is the average number of rows in
90074 ** a K-column index, there will be K+1 integers in the stat column. If
90077 ** The list of integers in the stat column can optionally be followed
90085 ** rows in the table identified by sqlite_stat1.tbl.
90094 ** the index belongs. There are usually 10 rows in the sqlite_stat2
90098 ** inclusive are samples of the left-most key value in the index taken at
90100 ** (10 in the standard build) and let C be the number of rows in the index.
90122 ** to aid the query planner in choosing good indices based on the values
90123 ** that indexed columns are compared against in the WHERE clauses of
90132 ** of entries in the index whose left-most column exactly matches
90133 ** the left-most column of the sample. The second integer in nEq
90134 ** is the approximate number of entries in the index where the
90138 ** integer in nLt contains the number of entries in the index where the
90140 ** The K-th integer in the nLt entry is the number of index entries
90143 ** number of distinct entries in the index that are less than the
90157 ** of a blob encoding of the complete index key as is found in
90160 ** integer in the equivalent columns in sqlite_stat4.
90188 ** or it may be a NULL pointer. If it is not NULL, then all entries in
90194 int iDb, /* The database we are looking in */
90237 ** of the new table in register pParse->regRoot. This is important
90293 i64 iRowid; /* Rowid in main table of the key */
90303 tRowcnt nRow; /* Number of rows in the entire table */
90305 int nCol; /* Number of columns in index + pk/rowid */
90311 int iMin; /* Index in a[] of entry with minimum score */
90394 ** N: The number of columns in the index including the rowid/pk (note 1)
90395 ** K: The number of columns in the index excluding the rowid/pk.
90396 ** C: The number of rows in the index (note 2)
90398 ** Note 1: In the special case of the covering index that implements a
90400 ** total number of columns in the table.
90405 ** WITHOUT ROWID tables, N=K+P where P is the number of columns in the
90409 ** This routine allocates the Stat4Accum object in heap memory. The return
90420 int nCol; /* Number of columns in index being sampled */
90515 ** In other words, if we assume that the cardinalities of the selected
90583 /* This sample is being added because the prefix that ends in column
90584 ** iCol occurs many times in the table. However, if we have already
90624 ** for the last sample in the p->a[] array. Otherwise, the samples would
90636 /* Zero the first nEqZero entries in the anEq[] array. */
90697 /* Or if it is a non-periodic sample. Add it in this case too. */
90722 ** statistical data and/or samples in the Stat4Accum object about the
90848 /* Return the value to store in the "stat" column of the sqlite_stat1
90852 ** the index. The first integer in the list is the total number of
90853 ** entries in the index. There is one additional integer in the list
90856 ** a key with the corresponding number of fields. In other words,
91040 int nCol; /* Number of columns in pIdx. "N" */
91108 ** (1) the number of columns in the index including the rowid
91110 ** (2) the number of columns in the key without the rowid/pk
91111 ** (3) the number of rows in the index,
91334 ** Generate code that will do an analysis of a single table in
91336 ** in pTab that should be analyzed.
91365 ** Form 1 causes all indices in all attached databases to be analyzed.
91380 ** and code in pParse and return NULL. */
91446 int nOut, /* Number of slots in aOut[] */
91512 ** Entries for which argv[1]==NULL simply record the number of rows in
91545 ** sqlite_stat1 entries for this index. In that case just clobber
91596 ** stored in pIdx->aSample[].
91617 tRowcnt nRow; /* Number of rows in index */
91619 i64 nDist100; /* Number of distinct values in index */
91632 ** occur in the stat4 table for this index. Set sumEq to the sum of
91693 Index *pPrevIdx = 0; /* Previous index in the loop */
91694 IndexSample *pSample; /* A slot in pIdx->aSample[] */
91706 int nIdxCol = 1; /* Number of columns in stat4 records */
91721 ** loaded from the stat4 table. In this case ignore stat3 data. */
91764 int nCol = 1; /* Number of columns in index */
91771 ** the sqlite_stat4 table. In this case ignore stat3 data. */
91784 ** This is in case the sample record is corrupted. In that case, the
91837 ** If the sqlite_stat1 table is not present in the database, SQLITE_ERROR
91838 ** is returned. In this case, even if SQLITE_ENABLE_STAT3/4 was defined
91843 ** sqlite_stat4 table is not present in the database, SQLITE_ERROR is
91844 ** returned. However, in this case, data is read from the sqlite_stat1
91918 ** The author disclaims copyright to this source code. In place of
92017 zErrDyn = sqlite3MPrintf(db, "database %s is already in use", zName);
92022 /* Allocate the new entry in the db->aDb[] array and initialize the schema
92363 ** was left unspecified in the original SQL statement. The pFix structure
92367 ** view in one database does not refer to objects in a different database.
92368 ** (Exception: indices, triggers, and views in the TEMP database are
92370 ** to an object in a different database, an error message is added to
92388 "%s %T cannot reference objects in database %s",
92505 ** The author disclaims copyright to this source code. In place of
92521 ** All of the code in this file may be omitted by defining a single
92598 ** table zTab in database zDb. This function assumes that an authorization
92603 ** is treated as SQLITE_DENY. In this case an error is left in pParse.
92635 ** is in pTabList or else it is the NEW or OLD table of a trigger.
92651 int iSrc; /* Index in pTabList->a[] of table being read */
92653 int iCol; /* Index of column in table */
92696 ** is returned, then the error count and error message in pParse are
92768 ** The author disclaims copyright to this source code. In place of
92777 ** when syntax rules are reduced. The routines in this file handle the
92815 ** The table to be locked has root page iTab and is found in database iDb.
93016 ** Run the parser and code generator recursively in order to generate
93064 ** Locate the in-memory structure that describes a particular database
93099 ** Locate the in-memory structure that describes a particular database
93102 ** error message in pParse->zErrMsg.
93105 ** routine leaves an error message in pParse->zErrMsg where
93109 Parse *pParse, /* context in which to report errors */
93117 ** and code in pParse and return NULL. */
93127 /* If zName is the not the name of a table in the schema created using
93178 ** Locate the in-memory structure that describes
93224 ** For the index called zIdxName which is found in the database iDb,
93255 ** Look through the list of open database files in db->aDb[] and if
93300 ** since TEMP might be holding triggers that reference tables in the
93367 ** contains lookaside memory. (Table objects in the schema do not use
93382 /* Record the number of outstanding lookaside allocations in schema Tables
93466 ** Open the sqlite_master table stored in database number iDb for
93481 ** function returns the index of the named database in db->aDb[], or
93502 ** index of the named database in db->aDb[], or -1 if the named db
93532 Token *pName1, /* The "xxx" in the name "xxx.yyy" or "xxx" */
93533 Token *pName2, /* The "yyy" in the name "xxx.yyy" */
93562 ** "sqlite_" (in upper, lower or mixed case). This portion of the namespace
93597 ** Begin constructing a new table representation in memory. This is
93598 ** the first of several action routines that get called in response
93599 ** to a CREATE TABLE statement. In particular, this routine is called
93601 ** flag is true if the table should be stored in the auxiliary database
93602 ** file instead of in the main database file. This is normally the case
93603 ** when the "TEMP" or "TEMPORARY" keyword occurs in between
93606 ** The new table record is initialized and put in pParse->pNewTable.
93625 int iDb; /* Database number to create the table in */
93643 ** created in.
93690 ** index or table name in the same database. Issue an error message if
93692 ** to an sqlite3_declare_vtab() call. In that case only the column names
93733 ** then record a pointer to this table in the main database structure
93744 ** the SQLITE_MASTER table. Note in particular that we must go ahead
93765 /* If the file format and encoding in the database have not been set,
93782 /* This just creates a place-holder record in the sqlite_master table.
93784 ** by the real entry in code generated at sqlite3EndTable().
93786 ** The rowid for the new entry is left in register pParse->regRowid.
93787 ** The root page number of the new table is left in reg pParse->regRoot.
93817 ** This macro is used to compare two strings in a case-insensitive manner.
93833 ** in a CREATE TABLE statement. sqlite3StartTable() gets called
93882 ** This routine is called by the parser while in the middle of
93899 ** substrings in the following table. If one of the substrings is
93916 ** If none of the substrings in the above table are found,
93982 ** This routine is called by the parser while in the middle of
93984 ** token in the sequence of tokens that describe the type of the
93986 ** in the sequence. Use this information to construct a string
93988 ** in zType.
94010 ** This routine is called by the parser while in the middle of
94041 ** Historical versions of SQLite accepted strings as column names in
94042 ** indexes and PRIMARY KEY constraints and in UNIQUE constraints. Example:
94049 ** the expression given in its argument from a TK_STRING into a TK_ID
94215 ** in the database native encoding, the collation factory is invoked to
94217 ** and the sequence is available in another text encoding, then that is
94259 ** 1 chance in 2^32. So we're safe enough.
94292 ** to the specified offset in the buffer and updates *pIdx to refer
94466 ** (3) Set the Index.tnum of the PRIMARY KEY Index object in the
94552 /* Update the in-memory representation of all UNIQUE indices by converting
94605 ** An entry for the table is made in the master table on disk, unless
94609 ** recently changed, so the entry for this table already exists in
94620 Token *pEnd, /* The ')' before options in the CREATE TABLE */
94626 int iDb; /* Database in which the table lives */
94666 /* Resolve names in all CHECK constraint expressions.
94680 ** in the SQLITE_MASTER table of the database.
94714 ** new table is in register pParse->regRoot.
94716 ** Once the SELECT has been coded by sqlite3Select(), it is in a
94780 /* A slot for the record has already been allocated in the
94821 /* Add the table to the in-memory representation of the database.
94853 ** The parser calls this routine in order to create a new VIEW
94875 sqlite3ErrorMsg(pParse, "parameters are not allowed in views");
94923 ** The Table structure pTable is really a VIEW. Fill in the names of
94924 ** the columns of the view in the pTable structure. Return the number
94925 ** of errors. If an error is seen leave an error message in pParse->zErrMsg.
94960 ** in the following:
94974 ** "*" elements in the results set of the view and will assign cursors
95027 ** Clear the column names from every VIEW in database idx.
95050 ** root-page of a table or index in database iDb has changed from iFrom
95062 ** in order to be certain that we got the right one.
95101 /* OP_Destroy stores an in integer r1. If this integer
95106 ** The "#NNN" in the SQL is a special constant that means whatever value
95107 ** is in register NNN. See grammar rules associated with the TK_REGISTER
95120 ** in case a root-page belonging to another table is moved by the btree layer
95134 ** table and index root-pages in order, starting with the numerically
95143 ** and root page 5 happened to be the largest root-page number in the
95178 ** Remove entries from the sqlite_statN tables (for N in (1,2,3))
95235 ** at the btree level, in case the sqlite_sequence table needs to
95236 ** move as a result of the drop (can happen in auto-vacuum mode).
95250 ** created in the temp database that refers to a table in another
95377 ** in the current table point to the foreign key. If pFromCol==0 then
95380 ** of tables in the parent pTo table. flags contains all
95382 ** in the ON DELETE, ON UPDATE and ON INSERT clauses.
95385 ** under construction in the pParse->pNewTable field.
95392 ExprList *pFromCol, /* Columns in this table that point to other table */
95394 ExprList *pToCol, /* Columns in the other table */
95421 "number of columns in foreign key does not match the number of "
95422 "columns in the referenced table");
95459 "unknown column \"%s\" in foreign key definition",
95524 ** content of an index in response to a REINDEX command.
95536 int iSorter; /* Cursor opened by OpenSorter (if in use) */
95617 ** pointer to this extra space in *ppExtra.
95621 i16 nCol, /* Total number of columns in the index */
95679 int nName; /* Number of characters in zName */
95682 int sortOrderMask; /* 1 to honor DESC in index. 0 to ignore. */
95902 ** report any errors. In the common case where the expression is exactly
95903 ** a table column, store that column in aiColumn[]. For general expressions,
95904 ** populate pIndex->aColExpr and store XN_EXPR (-2) in aiColumn[].
95921 sqlite3ErrorMsg(pParse, "expressions prohibited in PRIMARY KEY and "
96012 ** the constraint occur in different orders, then the constraints are
96013 ** considered distinct and both result in separate indices.
96035 ** constraint specified somewhere in the CREATE TABLE statement.
96056 ** in-memory database structures.
96078 ** the index in the sqlite_master table and populate the index with
96084 ** or UNIQUE index in a CREATE TABLE statement. Since the table
96099 ** doing so, code a Noop instruction and store its address in
96100 ** Index.tnum. This is required in case this index is actually a
96101 ** PRIMARY KEY and the table is actually a WITHOUT ROWID table. In
96122 /* Add an entry in sqlite_master for this index
96151 ** processing (in sqlite3GenerateConstraintChecks()) as part of
96185 in the index.
96187 ** number of rows in the table that match any particular value of the
96197 ** are based on typical values found in actual indices.
96206 /* Set the first entry (number of rows in the index) to the estimated
96207 ** number of rows in the table. Or 10, if the estimated number of rows
96208 ** in the table is less than that. */
96292 ** pArray is a pointer to an array of objects. Each object in the
96293 ** array is szEntry bytes in size. This routine uses sqlite3DbRealloc()
96297 ** the array (in entries - so the allocation is ((*pnEntry) * szEntry) bytes
96298 ** in total).
96303 ** returned. *pIdx is set to the index of the new array entry in this case.
96311 int szEntry, /* Size of each object in the array */
96312 int *pnEntry, /* Number of objects currently in use */
96374 ** Return the index in pList of the identifier named zId. Return -1
96409 int iStart /* Index in pSrc->a[] of first new slot */
96455 ** need be. A new entry is created in the SrcList even if pTable is NULL.
96470 ** In other words, if call like this:
96480 ** then so is B. In other words, we never have a case where:
96520 ** Assign VdbeCursor index numbers to all tables in a SrcList
96563 ** are the name of the table and database named in the FROM clause term.
96580 Select *pSubquery, /* A subquery used in place of a table name */
96654 ** When building up a FROM clause in the parser, the join operator
96665 ** in p->a[0] and p->a[1], respectively. The parser initially stores the
96760 ** the number of errors. Leave any error messages in the pParse structure.
96852 ** inserting multiple rows in a table, or inserting a row and index entries.)
96863 ** possible to abort a statement prior to completion. In order to
96872 ** makes it more difficult to prove that the code is correct (in
96997 ** Recompute all indices of all tables in all databases where the
97006 HashElem *k; /* For looping over tables in pDb */
97007 Table *pTab; /* A table in the database */
97028 ** Form 1 causes all indices in all attached databases to be rebuilt.
97029 ** Form 2 rebuilds all indices in all databases that use the named
97038 Table *pTab; /* A table in the database */
97045 ** and code in pParse and return NULL. */
97091 ** The KeyInfo structure for an index is cached in the Index object.
97143 ** not, store an error in the Parse structure. */
97201 ** The author disclaims copyright to this source code. In place of
97218 ** in the encoding enc of name zName, length nName.
97244 ** collation function in the best encoding but there may be other versions
97268 ** requested collation sequence is not available in the desired encoding.
97273 ** The return value is either the collation sequence to be used in database
97340 ** Each pointer stored in the sqlite3.aCollSeq hash table contains an
97345 ** the collation sequence name. A pointer to this string is stored in
97371 /* If a malloc() failure occurred in sqlite3HashInsert(), it will
97421 ** matches the request for a function with nArg arguments in a system
97426 ** is also -1. In other words, we are searching for a function that
97487 int nFunc /* Number of bytes in zFunc */
97545 int nName, /* Number of characters in the name */
97571 /* If no match is found, search the built-in functions.
97573 ** If the SQLITE_PreferBuiltin flag is set, then search the built-in
97575 ** priority to built-in functions.
97580 ** new function. But the FuncDefs for built-in functions are read-only.
97681 ** The author disclaims copyright to this source code. In place of
97690 ** in order to generate code for DELETE FROM statements.
97695 ** While a SrcList can in general represent multiple tables and subqueries
97696 ** (as in the FROM clause of a SELECT statement) in this case it contains
97697 ** the name of a single table, as one might find in an INSERT, DELETE,
97698 ** or UPDATE statement. Look up that table in the symbol table and
97702 ** The following fields are initialized appropriate in pSrc:
97738 ** In either case leave an error message in pParse and return non-zero.
97762 ** Evaluate a view and store its result in an ephemeral table. The
97764 ** set of rows in the view that are to be added to the ephemeral table.
97812 Expr *pInClause = NULL; /* WHERE rowid IN ( select ) */
97838 ** DELETE FROM table_a WHERE rowid IN (
97861 /* now generate the new WHERE rowid IN clause for the DELETE/UDPATE */
97911 NameContext sNC; /* Name context to resolve expressions in */
97920 i16 nPk = 1; /* Number of columns in the PRIMARY KEY */
97922 i16 nKey; /* Number of memory cells in the row key */
97943 ** put in an SrcList structure because some of the subroutines we
98019 /* Resolve the column names in the WHERE clause.
98115 ** one, so just keep it in its register(s) and fall through to the
98169 /* Set up a loop over the rowids/primary-keys that were found in the
98260 ** they may interfere with compilation of other functions in this file
98261 ** (or in another file, if this file becomes part of the amalgamation). */
98278 ** in the case of a rowid table, or the PRIMARY KEY index in the case
98284 ** 3. The primary key for the row to be deleted must be stored in a
98286 ** that a search record formed from OP_MakeRecord is contained in the
98297 ** of a ONEPASS delete that affects multiple rows. In this case, if
98322 int iOld = 0; /* First register in OLD.* array */
98343 ** use for the old.* references in the triggers. */
98345 u32 mask; /* Mask of OLD.* columns in use */
98393 ** a view (in which case the only effect of the DELETE statement is to
98408 ** handle rows (possibly in other tables) that refer via a foreign key
98476 ** Generate code that will assemble an index key and stores it in register
98482 ** Return a register number which is the first in a block of
98487 ** If *piPartIdxLabel is not NULL, fill it in with a label and jump
98500 ** a column in common, then the register corresponding to that column already
98545 ** might be stored in the table as an integer (using a compact
98576 ** The author disclaims copyright to this source code. In place of
98585 ** functions of SQLite. (Some function, and in particular the date and
98749 ** in haystack and returns the number of previous characters plus 1,
98753 ** the number of bytes in haystack prior to the first occurrence of needle,
98754 ** or 0 if needle never occurs in haystack.
98871 ** is essentially a back-out of the bug-fix in check-in [5fc125d362df4b8]
98941 /* If Y==0 and X will fit in a 64-bit int,
99033 ** routines in the function table. The noopFunc macro provides this.
99054 in a way that is testable, mask the sign bit off of negative
99055 ** values, resulting in a positive value. Then take the
99150 ** character is exactly one byte in size. Also, provde the Utf8Read()
99151 ** macro for fast reading of the next character in the common case where
99183 ** [^...] Matches one character not in the enclosed list.
99186 ** in the list by making it the first character after '[' or '^'. A
99189 ** it the last character in the list.
99202 ** This routine is usually quick, but can be N**2 in the worst case.
99226 /* Skip over multiple "*" characters in the pattern. If there
99242 ** recursive search in this case, but it is an unusual case. */
99253 ** pattern string past the "*". Search in the input string for the
99258 ** c but in the other case and search the input string for either
99350 ** the build-in LIKE operator. The first argument to the function is the
99374 ** of deep recursion and N*N behavior in patternCompare().
99528 ** "NULL". Otherwise, the argument is enclosed in single quotes with
99805 int nIn; /* Number of bytes in input */
99808 unsigned char *aLen = 0; /* Length of each character in zCharSet */
99809 unsigned char **azChar = 0; /* Individual characters in zCharSet */
99810 int nChar; /* Number of characters in zCharSet */
99980 ** 0.0 in that case. In addition, TOTAL always returns a float where
100029 /* (double)0 In case of SQLITE_OMIT_FLOATING_POINT... */
100171 ** of the built-in functions above are part of the global function set.
100195 ** Register the built-in LIKE and GLOB functions. The caseSensitive
100244 ** the first three statements in the compareInfo structure. The
100256 ** All of the FuncDef structures in the aBuiltinFunc[] array above
100265 ** defined in this file.
100374 ** The author disclaims copyright to this source code. In place of
100394 ** Foreign keys in SQLite come in two flavours: deferred and immediate.
100429 ** row in the parent table. Decrement the counter for each row
100436 ** deleted row in the child table. If such a row is not found,
100441 ** in the parent table. For each found increment the counter.
100464 ** statement that inserts a single row only (no triggers). In this case,
100515 ** A foreign key constraint requires that the key columns in the parent
100527 ** N is the number of columns in the parent key. The first element of the
100529 ** constraint to the parent table column stored in the left-most column
100547 ** consists of a different number of columns to the child key in
100555 Parse *pParse, /* Parse context to store any error in */
100559 int **paiCol /* OUT: Map of index columns in pFKey */
100563 int nCol = pFKey->nCol; /* Number of columns in parent key */
100617 ** map to an explicit list of columns in table pParent. Check if this
100622 i16 iCol = pIdx->aiColumn[i]; /* Index of column in parent tbl */
100630 ** unusable. Bail out early in this case. */
100673 ** row in the parent table that corresponds to the row being inserted into
100676 ** found in the parent table:
100688 ** These operations are identified in the comment at the top of this file
100695 Index *pIdx, /* Unique index on parent key columns in pTab */
100711 ** Check if any of the key columns in the child table row are NULL. If
100713 ** search for a matching row in the parent table. */
100887 ** the number of FK violations in the db) or +1 when deleting one (as this
100890 ** The code generated by this function scans through the rows in the child
100908 ** These operations are identified in the comment at the top of this file
100951 i16 iCol; /* Index of column in child table */
100952 const char *zCol; /* Name of column in child table */
100972 ** for WITHOUT ROWID tables. In the second form, the primary key is
101000 /* Resolve the references in the WHERE clause. */
101006 /* Create VDBE to loop through the entries in pSrc that match the WHERE
101070 ** constraint violations in the database,
101129 ** actually updated, the corresponding element in the aChange[] array
101156 ** actually updated, the corresponding element in the aChange[] array
101212 ** first register in an array of (pTab->nCol+1) registers containing the
101215 ** zero in this case.
101253 Index *pIdx = 0; /* Index on key columns in pTo */
101268 ** on the parent key columns in the parent table. If either of these
101269 ** schema items cannot be located, set an error in pParse and return
101280 /* If isIgnoreErrors is true, then a table is being dropped. In this
101282 ** before actually dropping it in order to check FK constraints.
101325 ** in the parent table. */
101366 ** an immediate foreign key violation. So do nothing in this case. */
101395 ** So do not set the "may-abort" flag in this case.
101406 ** child table to fire. In these cases the fk constraint counters
101424 ** row contained in table pTab.
101454 ** row contained in table pTab. If the operation is a DELETE, then
101456 ** to an array of size N, where N is the number of columns in table pTab.
101458 ** entry in the aChange[] array is set to -1. If the column is modified,
101475 ** table in question is either the child or parent table for any
101510 ** For example, if pFKey is the foreign key and pTab is table "p" in
101543 int nFrom; /* Length in bytes of zFrom */
101559 Token tFromCol; /* Name of column in child table */
101560 Token tToCol; /* Name of column in parent table */
101561 int iFromCol; /* Idx of column in child table */
101657 sizeof(TriggerStep) + /* Single step in trigger program */
101766 /* EV: R-30323-21917 Each foreign key constraint in SQLite is
101788 ** The author disclaims copyright to this source code. In place of
101797 ** to handle INSERT statements in SQLite.
101813 int iDb, /* The database index in sqlite3.aDb[] */
101838 ** pIdx. A column affinity string has one character for each column in
101850 ** rowid that appears as the last column in every index.
101944 ** Return non-zero if the table pTab in database iDb or any of its indices
101945 ** have been opened at any point in the VDBE program. This is used to see if
101986 ** which is in database iDb. Return the register number for the register
101999 ** (3) Register to hold the rowid in sqlite_sequence of pTab
102025 pToplevel->nMem++; /* Rowid in sqlite_sequence */
102076 ** larger than the maximum rowid in the memId memory cell, then the
102145 ** The IDLIST appears in the pColumn parameter. pColumn is NULL if IDLIST
102169 ** in other words if the SELECT pulls all columns from a single table
102179 ** transfer all records in <table2> over to <table>
102195 ** loop over the rows in the SELECT
102209 ** that is also read as part of the SELECT. In the third form,
102216 ** loop over the tables in the SELECT
102247 int nColumn; /* Number of columns in the data */
102255 int addrCont = 0; /* Top of insert loop. Label "C" in templates 3 and 4 */
102259 u8 useTempTable = 0; /* Store SELECT results in intermediate table */
102262 u8 bIdListInOrder; /* True if IDLIST is in table order */
102369 /* If this is an AUTOINCREMENT table, look up the sequence number in the
102370 ** sqlite_sequence table and store it in memory cell regAutoinc.
102390 ** is named in the IDLIST, then record in the ipkColumn variable
102392 ** the index of the primary key as it appears in IDLIST, not as
102393 ** is appears in the original table. (The index of the INTEGER
102394 ** PRIMARY KEY in the original table is pTab->iPKey.)
102459 ** temp table in the case of row triggers.
102514 ** column index in the original table definition.
102520 /* Make sure the number of columns in the source data matches the number
102713 ** in its place. Hence, fill this column with a NULL to avoid
102830 ** they may interfere with compilation of other functions in this file
102831 ** (or in another file, if this file becomes part of the amalgamation). */
102846 ** The regNewData parameter is the first register in a range that contains
102848 ** pTab->nCol+1 registers in this range. The first register (the one
102849 ** that regNewData points to) will contain the new rowid, or NULL in the
102850 ** case of a WITHOUT ROWID table. The second register in the range will
102873 ** indices where aRegIdx[i]==0. The order of indices in aRegIdx[] is
102882 ** for the first index in the pTab->pIndex list. Cursors for other indices
102906 ** any IGNORE The attempt in insert or update the current
102918 ** CHECK REPLACE Illegal. The results in an exception.
102931 int regNewData, /* First register in a range holding values to insert */
102948 int nPkField; /* Number of fields in PRIMARY KEY. 1 for ROWID tables */
102963 ** normal rowid tables. nPkField is the number of key fields in the
102964 ** pPk index or 1 for a rowid table. In other words, nPkField is the
102965 ** number of fields in the true primary key of the table. */
103052 ** exist in the table.
103057 /* Figure out what action to take in
103088 /* Check to see if the new rowid already exists in the table. Skip
103190 ** the insert or update. Store that record in the aRegIdx[ix] register
103203 if( regRowid==regIdx+i ) continue; /* ROWID already in regIdx+i */
103217 /* In an UPDATE operation, if this index is the PRIMARY KEY index
103226 /* Find out what action to take in case there is a uniqueness conflict */
103258 ** store it in registers regR..regR+nPk-1 */
103417 ** the table itself, but the PRIMARY KEY index in the case of a WITHOUT
103418 ** ROWID table) is returned in *piDataCur. The first index cursor is
103419 ** returned in *piIdxCur. The number of indices is returned.
103426 ** For a WITHOUT ROWID table, *piDataCur will be somewhere in the range
103452 ** can detect if they are used by mistake in the caller. */
103512 ** for index pDest in an insert transfer optimization. The rules
103539 return 0; /* Different expressions in the index */
103564 ** performance. Raw index records are transferred in the same way.
103568 ** embedded in the code for details.
103572 ** is empty - a factor that can only be determined at run-time. In that
103575 ** xfer optimization code if the test fails. In that case, this routine
103591 Table *pSrc; /* The table in the FROM clause of SELECT */
103665 ** correct syntactic form to participate in this optimization. Now
103688 return 0; /* Number of columns must be the same in tab1 and tab2 */
103721 return 0; /* pDestIdx has no corresponding index in pSrc */
103767 /* In some circumstances, we are able to run the xfer optimization
103834 ** index will be populated by inserting keys in strictly sorted
103835 ** order. In this case, instead of seeking within the b-tree as part
103843 ** a VACUUM command. In that case keys may not be written in strictly
103884 ** The author disclaims copyright to this source code. In place of
103892 ** Main file for the SQLite library. The routines in this file
103893 in
103905 ** If the SQL is a query, then for each row in the query result
104031 ** The author disclaims copyright to this source code. In place of
104044 #define SQLITE_CORE 1 /* Disable the API redefinition in sqlite3ext.h */
104046 /************** Include sqlite3ext.h in the middle of loadext.c **************/
104051 ** The author disclaims copyright to this source code. In place of
104075 ** WARNING: In order to maintain backwards compatibility, add new
104077 ** interfaces in the middle of this structure, then older different
104584 /************** Continuing where we left off in loadext.c ********************/
104686 ** in order to preserve backwards compatibility.
104982 ** Attempt to load an SQLite extension library contained in the file
104983 ** zFile. The entry point is zProc. zProc may be 0 in which case a
105065 ** character in the filename after the last "/" upto the first ".",
105100 "no entry point [%s] in shared library [%s]", zEntry, zFile);
105147 ** Call this routine when the database connection is closing in order
105161 ** default so as not to open security holes in older applications.
105196 u32 nExt; /* Number of entries in aExt[] */
105202 ** we have to locate the state vector at run-time. In the more common
105308 ** If anything goes wrong, set an error in the database connection.
105350 ** The author disclaims copyright to this source code. In place of
105373 ** object. This ensures that the aPragmaName[] table is arranged in
105375 ** Do not edit pragma.h directly. Edit and rerun the script in at
105377 /************** Include pragma.h in the middle of pragma.c *******************/
105844 /************** Continuing where we left off in pragma.c *********************/
105920 ** backed temporary databases, 2 for the Red-Black tree in memory database
105976 ** Set the names of the first N columns to the values in azCol[]
106068 ** defined in pager.h. This function returns the associated lowercase
106130 ** index of the database this pragma is being applied to in db.aDb[]. */
106166 ** file control is an array of pointers to strings (char**) in which the
106192 /* Locate the pragma in the lookup table */
106223 ** pages in the page cache. The second form sets both the current
106225 ** stored in the database file.
106274 ** database page size in bytes. The second form sets the
106327 ** maximum number of pages in the database file. The
106337 ** Return the number of pages in the specified database.
106373 ** of the PRAGMA command. In this case the locking-mode must be
106480 ** incr-vacuum flags. This is required in case this connection
106487 ** "incremental", write the value of meta[6] in the database
106544 ** number of pages in the cache. If N is negative, then the
106570 ** The parameter N is measured in bytes.
106777 ** in auto-commit mode. */
107029 Index *pIdx; /* Index in the parent table */
107032 HashElem *k; /* Loop counter: Next table in schema */
107187 ** In this case, the integrity of database iDb only is verified by
107191 ** "PRAGMA quick_check"), then iDb is set to 0. In this case, set iDb
107230 ** for all tables and indices in the database.
107257 sqlite3MPrintf(db, "*** in database %s ***\n", db->aDb[i].zName),
107300 zErr = sqlite3MPrintf(db, "NULL value in %s.%s", pTab->zName,
107352 sqlite3VdbeLoadString(v, 3, "non-unique entry in index ");
107362 sqlite3VdbeLoadString(v, 2, "wrong # of entries in index ");
107392 ** In its first form, this pragma returns the encoding of the main
107396 ** has not already been initialized. In this case it sets the default
107401 ** In all cases new databases created using the ATTACH command are
107406 ** In the second form this pragma sets the text encoding to be used in
107474 ** stored in the database header.
107526 ** Return the names of all compile-time options used in this build,
107573 ** after accumulating N frames in the log. Or query for the current value
107745 ** The author disclaims copyright to this source code. In place of
107988 ** Note: The #defined SQLITE_UTF* symbols in sqliteInt.h correspond to
108053 /* Ticket #2804: When we open a database in the newer file format,
108095 ** the schema loaded, even if errors occurred. In this situation the
108130 ** bit is set in the flags field of the Db structure. If the database
108153 ** schema may contain references to objects in other databases.
108193 ** Check schema cookies in all databases. If any cookie is out
108223 ** value stored as part of the in-memory schema representation,
108241 ** which database file in db->aDb[] the schema refers to.
108249 /* If pSchema is NULL, then return -1000000. This happens when code in
108251 ** created by a sub-select). In this case the return value of this
108272 ** Free all memory allocations in the pParse object
108288 int nBytes, /* Length of zSql in bytes. */
108312 ** some other database connection is holding a write-lock, which in
108318 ** back and different changes are made in their place, then when this
108323 ** of the sqlite3BtreeEnterAll() in sqlite3LockAndPrepare()) so it
108448 int nBytes, /* Length of zSql in bytes. */
108514 ** Two versions of the official API. Legacy and new use. In the legacy
108515 ** version, the original SQL text is not saved in the prepared statement
108517 ** sqlite3_step(). In the new version, the original SQL text is retained
108524 int nBytes, /* Length of zSql in bytes. */
108536 int nBytes, /* Length of zSql in bytes. */
108554 int nBytes, /* Length of zSql in bytes. */
108561 ** tricky bit is figuring out the pointer to return in *pzTail.
108602 ** Two versions of the official API. Legacy and new use. In the legacy
108603 ** version, the original SQL text is not saved in the prepared statement
108605 ** sqlite3_step(). In the new version, the original SQL text is retained
108612 int nBytes, /* Length of zSql in bytes. */
108624 int nBytes, /* Length of zSql in bytes. */
108641 ** The author disclaims copyright to this source code. In place of
108650 ** to handle SELECT statements in SQLite.
108738 ExprList *pEList, /* which columns to include in the result */
108804 ** Return a pointer to the right-most SELECT statement in a compound.
108814 ** in terms of the following bit values:
108826 ** a join type, but put an error in the pParse structure.
108835 u8 i; /* Beginning of keyword text in zKeyText[] */
108836 u8 nChar; /* Length of the keyword in characters */
108886 ** Return the index of a column in a table. Return -1 if the column
108887 ** is not contained in the table.
108898 ** Search the first N tables in pSrc, from left to right, looking for a
108908 int N, /* Number of tables in pSrc->a[] to search */
108913 int i; /* For looping over tables in pSrc */
108937 ** where tab1 is the iSrc'th table in SrcList pSrc and tab2 is the
108943 SrcList *pSrc, /* List of tables in FROM clause */
108944 int iLeft, /* Index of first table to join in pSrc */
108945 int iColLeft, /* Index of column in first table */
108946 int iRight, /* Index of second table in pSrc */
108947 int iColRight, /* Index of column in second table */
108949 Expr **ppWhere /* IN/OUT: The WHERE clause to add to */
108976 ** And set the Expr.iRightJoinTable to iTable for every term in the
108981 ** join restriction specified in the ON or USING clause and not a part
108984 ** originated in the ON or USING clause.
108988 ** explicitly mentioned in the expression. That information is needed
108994 ** term until after the t2 loop of the join. In that way, a
108997 ** after the t1 loop and rows with t1.x!=5 will never appear in
109022 ** The terms of a FROM clause are contained in the Select.pSrc structure.
109023 ** The left most table is the first entry in Select.pSrc. The right-most
109024 in the entry to
109032 SrcList *pSrc; /* All tables in the FROM clause */
109049 ** every column that the two tables have in common.
109058 char *zName; /* Name of column in the right table */
109060 int iLeftCol; /* Matching column in the left table */
109070 /* Disallow both ON and USING clauses in the same join
109074 "clauses in the same join");
109088 ** in the USING clause. Example: If the two tables to be joined are
109091 ** Report an error if any column mentioned in the USING clause is
109092 ** not contained in both tables to be joined.
109097 char *zName; /* Name of the term in the USING clause */
109108 "not present in both tables", zName);
109128 ** Generate code that will push the record in registers regData
109137 int nData, /* Number of elements in the data array */
109143 int nBase = nExpr + bSeq + nData; /* Fields in sorter record */
109243 ** seen combinations of the N values. A new entry is made in iTab
109270 ** (example: "a IN (SELECT * FROM table)") but it has more than 1 result
109271 ** column. We do this in a subroutine because the error used to occur
109272 ** in multiple places. (The error only occurs in one place now, but we
109296 ** are evaluated in order to get the data for this row. If srcTab is
109344 ** there are columns in the table on the left. The error will be caught
109346 ** to avoid other spurious errors in the meantime. */
109432 /* In this mode, write each query result to the key of the temporary
109471 ** in the index, do not write it to the output. If not, add the
109495 /* If we are creating a set for an "expr IN (SELECT ...)" construct,
109505 ** ORDER BY in this case since the order of entries in the set
109506 ** does not matter. But there might be a LIMIT clause, in which
109519 /* If any row exist in the result set, record that fact and abort.
109528 ** store the results in the appropriate memory cell and break out
109561 ** pDest->pOrderBy (in pSO). pDest->iSDParm (in iParm) is the cursor for an
109622 ** there is a sorter, in which case the sorter has already limited
109684 ** the collating sequence for each expression in that expression list.
109759 ** in sqlite3Select() to assign values to structure member variables that
109810 ** then the results were placed in a sorter. After the loop is terminated
109937 ** result in *pEstWidth.
109942 ** is considered a column can be complex in the presence of subqueries. The
109943 ** result-set expression in all of the following SELECT statements is
109985 ** extracted from in NameContext.pSrcList. This table may be real
109990 int iCol = pExpr->iColumn; /* Index of column in pTab */
110014 ** when columnType() is called on the expression "t1.col" in the
110015 ** sub-select. In this case, set the column type to NULL, even
110027 /* The "table" is actually a sub-select or a view in the FROM clause
110079 ** origin info for the single column in the result set of the SELECT
110109 ** in the result set.
110132 ** column specific strings, in case the schema is reset before this
110148 ** in the result set. This information is used to provide the
110149 ** azCol[] values in the callback.
110226 ** store NULL in *paCol and 0 in *pnCol and return SQLITE_NOMEM.
110238 int nCol; /* Number of columns in the result set */
110241 int nName; /* Size of name in zName[] */
110328 ** This routine requires that all identifiers in the SELECT
110407 in pParse.
110428 ** that appear in the original SQL statement after the LIMIT and OFFSET
110569 ** There is exactly one reference to the recursive-table in the FROM clause
110575 ** extracted row (now in the iCurrent table) becomes the content of the
110585 ** If the query has an ORDER BY, then entries in the Queue table are kept in
110602 int nCol = p->pEList->nExpr; /* Number of columns in the recursive table */
110642 ** the Distinct table must be exactly one greater than Queue in order
110673 /* Store the results of the setup-query in Queue. */
110679 /* Find the next row in the Queue and output that row */
110682 /* Transfer the next row in Queue over to Current */
110691 /* Output the single row in Current */
110702 /* Execute the recursive SELECT taking the single row in Current as
110703 ** the value for the recursive-table. Store the results in the Queue.
110784 ** in which case this routine will be called recursively.
110801 ** The arrows in the diagram above represent the Select.pPrior pointer.
110803 ** pPrior will be the t2 query. p->op will be TK_UNION in this case.
110825 ** the last (right-most) SELECT in the series may have an ORDER BY or LIMIT.
110864 /* Make sure all SELECTs in the statement have the same number of elements
110865 ** in their result sets.
110988 /* Query flattening in sqlite3Select() might refill p->pOrderBy.
111001 /* Convert the data in the temporary table into whatever form
111123 int nCol; /* Number of columns in result set */
111182 ** The data to be output is contained in pIn->iSdst. There are
111189 ** If regPrev>0 then it is the first register in a vector that
111195 ** If the LIMIT found in p->iLimit is reached, jump immediately to
111251 /* If we are creating a set for an "expr IN (SELECT ...)" construct,
111269 ** store the results in the appropriate memory cell and break out
111280 /* The results are stored in a sequence of registers
111297 ** For SRT_Output, results are stored in a sequence of registers.
111333 ** co-routines. Then run the co-routines in parallel and merge the results
111334 ** into the output. In addition to the two coroutines (called selectA and
111343 ** appears only in B.)
111371 ** In the AltB, AeqB, and AgtB subroutines, an EOF on A following nextA
111376 ** Duplicate removal in the UNION, EXCEPT, and INTERSECT cases is handled
111446 int nOrderBy; /* Number of terms in the ORDER BY clause */
111620 ** are exhausted and only data in select B remains.
111634 ** are exhausted and only data in select A remains.
111692 /* Jump to the this point in order to terminate the query.
111726 ** a column in table number iTable with a copy of the iColumn-th
111727 ** entry in pEList. (But leave references to the ROWID column
111731 ** whose result set is defined by pEList appears as entry in the
111739 Expr *pExpr, /* Expr in which substitution occurs */
111768 ExprList *pList, /* List to scan and in which to make substitutes */
111780 Select *p, /* SELECT statement in which to make substitutions */
111818 ** subquery first and store the results in a temporary table, then
111918 ** appear as unmodified result columns in the outer query. But we
111919 ** have other optimizations in mind to deal with that case.
111929 ** recursive queries in multiSelect().
111931 ** (24) The subquery is not an aggregate that uses the built-in min() or
111937 ** In this routine, the "p" parameter is a pointer to the outer query.
111950 int iFrom, /* Index in p->pSrc->a[] of the inner subquery */
111957 Select *pSub1; /* Pointer to the rightmost select in sub-query */
112116 ** using UNION ALL operators. In this case N is the number of simple
112117 ** select statements in the compound sub-query.
112173 ** in the outer query.
112207 /* The following loop runs once for each term in a compound-subquery
112214 ** the cursor number for the original outer query FROM element in
112218 ** elements we are now copying in.
112224 nSubSrc = pSubSrc->nSrc; /* Number of terms in subquery FROM clause */
112240 ** query. If the subquery has more than one element in its FROM clause,
112248 ** The outer query has 3 slots in its FROM clause. One slot of the
112251 ** The middle slot is expanded to two slots in order to make space
112252 ** for the two elements in the FROM clause of the subquery.
112272 ** references to the iParent in the outer query.
112280 ** We look at every expression in the outer query and every place we see
112295 ** do not necessarily correspond to columns in SELECT statement pParent,
112377 ** (1) The inner query is an aggregate. (In that case, we'd really want
112390 ** (5) The WHERE clause expression originates in the ON or USING clause
112508 ** SQLITE_ERROR and leave an error in pParse. Otherwise, populate
112548 ** there are COLLATE terms in the ORDER BY.
112637 ** WITH clause will never be popped from the stack. In this case it
112638 ** should be freed along with the Parse object. In other cases, when
112663 ** occurs. If an error does occur, an error message is stored in the
112687 ** recursive reference to CTE pCte. Leave an error in pParse and return
112689 ** In this case, proceed. */
112759 pCte->zCteErr = "recursive reference in a subquery: %s";
112799 ** (2) Fill in the pTabList->a[].pTab fields in the SrcList that
112800 ** defines FROM clause. When views appear in the FROM clause,
112810 ** (4) Scan the list of columns in the result set (pEList) looking
112812 ** If found, expand each "*" to be every column in every table
112813 ** and TABLE.* to be every column in TABLE.
112839 /* Make sure cursor numbers have been assigned to all entries in
112844 /* Look up every table named in the FROM clause of the select. If
112860 /* A sub-query in the FROM clause of a SELECT */
112875 /* An ordinary table or view name in the FROM clause */
112917 /* For every "*" that occurs in the column list, insert the names of
112918 ** all columns in all tables. And for every TABLE.* insert the names
112919 ** of all columns in TABLE. The parser inserted a special expression
112920 ** with the TK_ALL operator for each "*" that it found in the column list.
112922 ** each one to the list of all columns in all tables.
112938 ** in the result set and expand them one by one.
113003 ** for virtual tables), do not include it in the expanded
113016 /* In a NATURAL join, omit the join columns from the
113021 /* In a join with a USING clause, omit columns in the
113077 sqlite3ErrorMsg(pParse, "too many columns in result set");
113090 ** subquery in the parser tree.
113102 ** Expanding a SELECT statement is the first step in processing a
113156 /* A sub-query in the FROM clause of a SELECT */
113170 ** the Table structures of all FROM-clause subqueries in a
113194 ** * Wildcards "*" and "TABLE.*" in result sets are expanded.
113195 ** * Identifiers in expression are matched to tables.
113221 ** routine generates code that stores NULLs in all of those memory
113263 ** in the AggInfo structure.
113337 ** in registers, sqlite3ExprCode() may use OP_SCopy to copy the value
113386 ** Generate code for the SELECT statement given in the p argument.
113389 ** See comments in sqliteInt.h for further information.
113392 ** encountered, then an appropriate error message is left in
113395 ** This routine does NOT free the Select structure passed in. The
113447 /* If ORDER BY makes no difference in the output then neither does
113479 /* Try to flatten subqueries in the FROM clause up into the main query
113489 /* Catch mismatch in the declared columns of a view and the number of
113490 ** columns in the SELECT on the RHS */
113534 /* Generate code for all sub-queries in the FROM clause
113544 ** once, if the subquery is part of the WHERE clause in a LEFT JOIN,
113545 ** for example. In that case, do not regenerate the code to manifest
113565 /* Make copies of constant WHERE-clause terms in the outer query down
113656 ** can be rewritten as a GROUP BY. In other words, this:
113682 ** being unused if the data can be extracted in pre-sorted order.
113778 int groupBySort; /* Rows come from source in GROUP BY order */
113789 struct ExprList_item *pItem; /* For looping over expression in a list */
113805 ** in the correct order. It also may not - the GROUP BY might use a
113817 /* Convert TK_COLUMN nodes into TK_AGG_COLUMN and make entries in
113818 ** sAggInfo for all TK_AGG_FUNCTION nodes in expressions of the
113886 /* Begin a loop that will extract all source rows in GROUP BY order.
113887 ** This might involve two separate loops with an OP_Sort in between, or
113889 ** in the right order to begin with.
113897 /* The optimizer is able to deliver rows in group by order so
113903 /* Rows are coming out in undetermined order. We have to push
113905 ** then loop over the sorting index in order to get the output
113906 ** in sorted order
113962 ** will naturally deliver rows in the order required by the ORDER BY
113975 /* Evaluate the current GROUP BY terms and store in b0, b1, b2...
113978 ** from the previous row currently stored in a0, a1, a2...
114000 ** Changes in the GROUP BY are detected by the previous code
114003 ** This code copies current group by terms in b0,b1,b2,...
114005 ** and resets the aggregate accumulator registers in preparation
114022 VdbeComment((v, "indicate data in accumulator"));
114047 ** increments the iAbortFlag memory location before returning in
114107 ** (2013-10-03) Do not count the entries in a partial index.
114109 ** In practice the KeyInfo structure will not be used. It is only
114143 ** If it is, then ask the code in where.c to attempt to sort results
114145 ** If where.c is able to produce results sorted in this order, then
114158 ** + The optimizer code in where.c (the thing that decides which
114160 ** satisfying the 'ORDER BY' clause than it does in other cases.
114161 ** Refer to code and comments in where.c for details.
114230 /* The SELECT has been coded. If there is an error in the Parse structure,
114260 ** The author disclaims copyright to this source code. In place of
114272 ** These routines are in a separate files so that they will not be linked
114289 u32 nRow; /* Number of rows in the result */
114290 u32 nColumn; /* Number of columns in the result */
114291 u32 nData; /* Slots used in azResult[]. (nRow+1)*nColumn */
114296 ** This routine is called once for each row in the result table. Its job
114297 ** is to fill in the TabResult structure appropriately, allocating new
114302 int need; /* Slots needed in p->azResult[] */
114306 /* Make sure there is enough space in p->azResult to hold everything
114369 ** The result that is written to ***pazResult is held in memory obtained
114378 int *pnRow, /* Write the number of rows in the result here */
114462 ** The author disclaims copyright to this source code. In place of
114496 ** All of the triggers on pTab that are in the same database as pTab
114498 ** triggers on pTab in the TEMP schema. This routine prepends all
114504 ** pTab as well as the triggers lised in pTab->pTrigger.
114535 ** in pParse->pNewTrigger. After the trigger actions have been parsed, the
114555 int iDb; /* The database to store the trigger in */
114573 /* Figure out the db that the trigger will be created in */
114597 ** then set iDb to 1 to create the trigger in the temporary database.
114623 ** drop so the trigger cannot be dropped. This results in an
114726 ** in order to complete the process of building the trigger.
114765 /* Make an entry in the sqlite_master table */
114806 ** The parser calls this routine when it finds a SELECT statement in
114855 IdList *pColumn, /* List of columns in pTableName to insert into */
115073 ** in pEList is of the format <id>=<expr>. If any of the entries
115074 ** in pEList have an <id> which matches an identifier in pIdList,
115093 ** least one of the columns in pChanges is being modified.
115099 ExprList *pChanges, /* Columns that change in an UPDATE statement */
115126 ** forming the SrcList. This prevents a trigger in one database from
115127 ** referring to a target in another database. An exception is when the
115128 ** trigger is in TEMP in which case it can refer to any other database it
115231 ** program. It is not used in production code, only for debugging.
115392 /* It may be that this trigger has already been coded (or is in the
115395 ** in the Parse.pTriggerPrg list. Search for such an entry. */
115412 ** function are the same as those described in the header function for
115428 /* Code the OP_Program opcode in the parent VDBE. P4 of the OP_Program
115457 ** The reg argument is the address of the first in an array of registers
115459 ** in the trigger program. If N is the number of columns in table pTab
115476 ** Similarly, for ON INSERT triggers the values stored in the OLD.* registers
115494 int reg, /* The first in an array of registers (see above) */
115507 ** always defined. The trigger must be in the same schema as the table
115525 ** Triggers may access values stored in the old.* or new.* pseudo-table.
115534 ** are more than 32 columns in the table, and at least one of the columns
115545 ** included in the returned mask if the TRIGGER_BEFORE bit is set in the
115547 ** included in the returned mask if the TRIGGER_AFTER bit is set in tr_tm.
115585 ** The author disclaims copyright to this source code. In place of
115604 ExprList *pChanges, /* The columns to change in the UPDATE statement */
115606 int *aXRef, /* Mapping from columns of pTab to entries in pChanges */
115617 ** The default value of a column is specified by a DEFAULT clause in the
115620 ** command. If the latter, then the row-records in the table btree on disk
115639 ** stored in place of an 8-byte floating point value in order to save
115672 SrcList *pTabList, /* The table in which we should change things */
115690 int *aXRef = 0; /* aXRef[i] is the index in pChanges->a[] of the
115694 u8 chngPk; /* PRIMARY KEY changed in a WITHOUT ROWID table */
115695 u8 chngRowid; /* Rowid changed in a normal table */
115699 NameContext sNC; /* The name-context to resolve expressions in */
115713 int nKey = 0; /* Number of elements in regKey for WITHOUT ROWID */
115720 int regNew = 0; /* Content of the NEW.* table in triggers */
115721 int regOld = 0; /* Content of OLD.* table in
115765 ** allocate enough space, just in case.
115794 /* Resolve the column names in all the expressions of the
115796 ** for each column to be updated in the pChanges array. For each
115854 /* There is one entry in the aRegIdx[] array for each index on the table
115855 ** being updated. Fill in aRegIdx[] with a register number that will hold
115913 /* Resolve the column names in all the expressions in the
116102 ** a new.* reference in a trigger program.
116121 /* The row-trigger may have deleted the row being updated. In this
116124 ** is deleted or renamed by a BEFORE trigger - is left undefined in the
116138 ** registers in case this has happened.
116189 ** handle rows (possibly in other tables) that refer via a foreign key
116255 ** they may interfere with compilation of other functions in this file
116256 ** (or in another file, if this file becomes part of the amalgamation). */
116277 ** (C) The content of every column in the row.
116283 ** stores the same values (A, B and C above) in a register array and
116290 ExprList *pChanges, /* The columns to change in the UPDATE statement */
116292 int *aXRef, /* Mapping from columns of pTab to entries in pChanges */
116303 int regArg; /* First register in VUpdate arg array */
116304 int regRec; /* Register in which to assemble record */
116312 ** create and open the ephemeral table in which the records created from
116395 ** The author disclaims copyright to this source code. In place of
116405 ** Most of the code in this file may be omitted by defining the
116414 ** text of the error message in *pzErrMsg. Return the result code.
116468 ** in PostgreSQL. The VACUUM command works as follows:
116483 ** Two writes per page are required in step (3) because the original
116491 ** And a power loss in between deleting the original and renaming the
116526 sqlite3SetString(pzErrMsg, db,"cannot VACUUM - SQL statements in progress");
116623 ** in the temporary database.
116640 /* Loop through the tables in the main database. For each, do
116696 /* This array determines which meta meta values are preserved in the
116715 /* GetMeta() and UpdateMeta() cannot fail in this context because
116771 ** The author disclaims copyright to this source code. In place of
116804 sqlite3 *db, /* Database in which module is registered */
116849 sqlite3 *db, /* Database in which module is registered */
116864 sqlite3 *db, /* Database in which module is registered */
116923 ** Table p is a virtual table. This function moves all elements in the
116927 ** connection db is left in the p->pVTable list.
116938 ** database connection that may have an entry in the p->pVTable list.
116964 ** list in p->pVTab. It also decrements the VTable ref count. This is
116988 ** Disconnect all the virtual table objects in the sqlite3.pDisconnect list.
116995 ** 1) By this function. In this case, all BtShared mutexes and the mutex
116999 ** the sqlite3.pDisconnect list. In this case either the BtShared mutex
117000 ** associated with the database the virtual table is stored in is held
117001 ** or, if the virtual table is stored in a non-sharable database, then
117035 ** in the list are moved to the sqlite3.pDisconnect list of the associated
117081 int iDb; /* The database the table is being created in */
117121 ** in pParse->zArg[] and appends it to the list of arguments on the
117122 ** virtual table currently under construction in pParse->pTable.
117147 ** first time (in other words if the virtual table is actually being
117150 ** in the sqlite_master table.
117165 /* A slot for the record has already been allocated in the
117170 ** entry in the sqlite_master table tht was created for this vtab
117197 /* If we are rereading the sqlite_master table create the in-memory
117199 ** the first time the virtual table is used in an SQL statement. This
117219 ** of an argument to the module name in a CREATE VIRTUAL TABLE statement.
117229 ** in an argument to the module name in a CREATE VIRTUAL TABLE statement.
117378 ** and an error left in pParse.
117446 ** of the virtual table named zTab in database iDb.
117450 ** In this case the caller must call sqlite3DbFree(db, ) on *pzErr.
117559 ** of the virtual table named zTab in database iDb. This occurs
117596 ** of each of the virtual tables in the sqlite3.aVTrans array. The method
117598 ** the offset of the method to call in the sqlite3_module structure.
117624 ** Invoke the xSync method of all virtual tables in the sqlite3.aVTrans
117628 ** If an error message is available, leave it in p->zErrMsg.
117649 ** Invoke the xRollback method of all virtual tables in the
117658 ** Invoke the xCommit method of all virtual tables in the
117672 ** in the sqlite3.aVTrans array.
117681 ** virtual module tables in this case, so return SQLITE_LOCKED.
117694 /* If pVtab is already in the aVTrans array, return early */
117767 ** If pExpr is a column in a virtual table, then let the virtual
117794 /* Check to see the left operand is a column in a virtual table */
117838 ** Make sure virtual table pTab is contained in the pParse->apVirtualLock[]
117840 ** array if it is missing. If pTab is already in the array, this routine
117870 ** statement in order to come into existance. Eponymous virtual table
117924 ** Return the ON CONFLICT resolution mode in effect for the virtual
117925 ** table update operation currently in progress.
117986 ** The author disclaims copyright to this source code. In place of
117997 ** This file was split off from where.c on 2015-06-06 in order to reduce the
118003 /************** Include whereInt.h in the middle of wherecode.c **************/
118008 ** The author disclaims copyright to this source code. In place of
118018 ** planner logic in "where.c". These definitions are broken out into
118053 ** loop in WHERE clause.
118061 ** each term in the FROM clause (which is to say, for each of the
118071 int addrNxt; /* Jump here to start the next IN combination */
118078 u8 iFrom; /* Which entry in the FROM clause */
118083 int nIn; /* Number of entries in aInLoop[] */
118085 int iCur; /* The VDBE cursor used by this IN operator */
118086 int addrInTop; /* Top of the IN loop */
118087 u8 eEndLoopOp; /* IN Loop terminator. OP_Next or OP_Prev */
118088 } *aInLoop; /* Information about each nested IN operator */
118089 } in; /* Used when pWLoop->wsFlags&WHERE_IN_ABLE */
118119 u8 iTab; /* Position in FROM clause of table for this loop */
118138 u16 nLTerm; /* Number of entries in aLTerm[] */
118144 WhereLoop *pNextLoop; /* Next WhereLoop object in the WhereClause */
118149 ** subquery on one operand of an OR operator in the WHERE clause.
118172 ** Think of each WhereLoop object as a node in a graph with arcs
118183 ** number of nodes in the FROM clause. The best (lowest cost) WherePath
118187 Bitmask maskLoop; /* Bitmask of all WhereLoop objects in this path */
118222 ** In this second case, wtFlag has the TERM_ORINFO bit set and eOperator==WO_OR
118226 ** If a term in the WHERE clause does not match either of the two previous
118229 ** but no other fields in the WhereTerm object are meaningful.
118233 ** cursor number into bits and the translated bit is stored in the prereq
118234 ** fields. The translation is used in order to maximize the number of
118235 ** bits that will fit in a Bitmask. The VDBE cursor numbers might be
118239 ** beginning with 0 in order to make the best possible use of the available
118240 ** bits in the Bitmask. So, in the example above, the cursor numbers
118243 ** The number of terms in a join is limited by the number of bits
118244 ** in prereqRight and prereqAll. The default is 64 bits, hence SQLite
118250 int leftCursor; /* Cursor number of X in "X <op> <expr>" */
118252 int leftColumn; /* Column number of X in "X <op> <expr>" */
118287 ** terms in the WHERE clause that are useful to the query planner.
118295 unsigned char nEquiv; /* Number of entries in aEquiv[] */
118296 unsigned char iEquiv; /* Next unused slot in aEquiv[] */
118299 int aiCur[11]; /* Cursors in the equivalence class */
118300 i16 aiColumn[11]; /* Corresponding column number in the eq-class */
118320 int nSlot; /* Number of entries in a[] */
118335 Bitmask indexable; /* Bitmask of all indexable tables in the clause */
118348 ** between VDBE cursor numbers and bits of the bitmasks in WhereTerm.
118350 ** The VDBE cursor numbers are small integers contained in
118353 ** contain gaps in the numbering sequence. But we want to make maximum
118354 ** use of the bits in our bitmasks. This structure provides a mapping
118365 ** Note that the mapping is not necessarily ordered. In the example
118394 int nRecValid; /* Number of valid fields currently in pRec */
118410 SrcList *pTabList; /* List of tables in the join */
118430 WhereLevel a[1]; /* Information about each nest loop in WHERE */
118514 ** These are definitions of bits in the WhereLoop.wsFlags field.
118515 ** The particular combination of bits in each WhereLoop help to
118520 #define WHERE_COLUMN_IN 0x00000004 /* x IN (...) */
118530 #define WHERE_IN_ABLE 0x00000800 /* Able to support an IN operator */
118539 /************** Continuing where we left off in wherecode.c ******************/
118575 ** rows scanned by the strategy in the form of an SQL expression.
118616 ** is added to the output to describe the table scan strategy in pLevel.
118754 ** Disable a term in the WHERE clause. Except, do not disable the term
118755 ** if it controls a LEFT OUTER JOIN and it did not originate in the ON
118758 ** Consider the term t2.z='ok' in the following queries:
118764 ** The t2.z='ok' is disabled in the in (2) because it originates
118765 ** in the ON clause. The term is disabled in (3) because it is not part
118766 ** of a LEFT OUTER JOIN. In (1), the term is not disabled.
118768 ** Disabling a term causes that term to not be tested in the inner loop
118770 ** by indices, we disable them to prevent redundant tests in the inner
118773 ** as we can without disabling too much. If we disabled in (1), we'd get
118777 ** automatically disabled. In this way, terms get disabled if derived
118784 ** Only the parent term was in the original WHERE clause. The child1
118820 ** beginning and end of zAff are ignored. If all entries in zAff are
118857 ** term can be either X=expr or X IN (...). pTerm is the term to be
118860 ** The current value for the constraint is left in register iReg.
118863 ** result is left on the stack. For constraints of the form X IN (...)
118871 int bRev, /* True for reverse-order IN operations */
118872 int iTarget /* Attempt to leave results in this register */
118912 if( pLevel->u.in.nIn==0 ){
118915 pLevel->u.in.nIn++;
118916 pLevel->u.in.aInLoop =
118917 sqlite3DbReallocOrFree(pParse->db, pLevel->u.in.aInLoop,
118918 sizeof(pLevel->u.in.aInLoop[0])*pLevel->u.in.nIn);
118919 pIn = pLevel->u.in.aInLoop;
118921 pIn += pLevel->u.in.nIn - 1;
118931 pLevel->u.in.nIn = 0;
118940 ** Generate code that will evaluate all == and IN constraints for an
118944 ** Suppose the WHERE clause is this: a==5 AND b IN (1,2,3) AND c>5 AND c<10
118945 ** The index has as many as three equality constraints, but in this
118948 ** a==5 and b IN (1,2,3). The current values for a and b will be stored
118949 ** in consecutive registers and the index of the first register is returned.
118951 ** In the example above nEq==2. But this subroutine works for any value
118957 ** are == or IN and are covered by the nEq. nExtraReg is 1 if there is
118958 ** an inequality constraint (such as the "c>=5 AND c<10" in the example) that
118962 ** the index of the first memory cell in that range. The code that
118965 ** key value of the loop. If one or more IN operators appear, then
118971 ** sqlite3DbMalloc(). Except, entries in the copy of the string associated
118978 ** In the example above, the index on t1(a) has TEXT affinity. But since
118981 ** a key to search the index. Hence the first byte in the returned affinity
118982 ** string in this example would be set to SQLITE_AFF_BLOB.
118987 int bRev, /* Reverse the order of IN operators */
118991 u16 nEq; /* The number of == or IN constraints to code */
119113 ** Generate code for the start of the iLevel-th loop in the WHERE clause
119123 int addrNxt; /* Where to jump to continue with the next IN case */
119125 int bRev; /* True if we need to scan in reverse order */
119136 int iRowidReg = 0; /* Rowid is stored in this register, if not zero */
119158 ** When there is an IN operator, we also have a "addrNxt" label that
119159 ** means to continue with the next IN value combination. When
119160 ** there are no IN operators in the constraints, the "addrNxt" label
119234 ** we reference multiple rows using a "rowid IN (...)"
119348 ** terms ("==" or "IN" operators) that refer to the N
119353 ** use the "==" and "IN" operators. For example, if the
119373 ** constraints but an index is selected anyway, in order
119392 u16 nEq = pLoop->u.btree.nEq; /* Number of == or IN terms */
119417 ** should not have a NULL value stored in 'x'. If column 'x' is
119418 ** the first one after the nEq equality constraints in the index,
119440 /* Like optimization range constraints always occur in pairs */
119449 assert( pRangeStart->wtFlags & TERM_LIKEOPT ); /* occur in pairs */
119468 /* Generate code to evaluate all constraint terms using == or IN
119469 ** and store the values of those terms in an array of registers
119639 ** In the example, there are three indexed terms connected by OR.
119642 ** Null 1 # Zero the rowset in reg 1
119657 ** Null 1 # Zero the rowset in reg 1
119695 /* Set up a new SrcList in pOrTab containing the table being scanned
119696 ** by this loop in the a[0] slot and all notReady tables in a[1..] slots.
119697 ** This becomes the SrcList in the recursive call to sqlite3WhereBegin().
119719 ** capable of holding primary keys in the case of a WITHOUT ROWID.
119723 ** is required in a few obscure LEFT JOIN cases where control jumps
119724 ** over the top of the loop into the body of it. In this case the
119745 ** That way, terms in y that are factored into the disjunction will
119749 ** the "interesting" terms of z - terms that did not originate in the
119754 ** is not contained in the ON clause of a LEFT JOIN.
119807 ** row will be skipped in subsequent sub-WHERE clauses.
119834 ** the row has already been included in the result set and
119840 ** is zero, assume that the key cannot already be present in
119878 ** If the call to sqlite3WhereBegin() above resulted in a scan that
119926 /* Tables marked isRecursive have only a single row that is stored in
120039 ** The author disclaims copyright to this source code. In place of
120052 ** analyzing Expr objects in the WHERE clause.
120079 ** The index in pWC->a[] of the new WhereTerm is returned on success.
120081 ** allocation error. The memory allocation failure will be recorded in
120133 ** "=", "<", ">", "<=", ">=", "IN", and "IS NULL"
120217 ** In order for the operator to be optimizible, the RHS must be a string
120220 ** that virtual tables cannot participate in the LIKE optimization.) The
120228 int *pisComplete, /* True if the only wildcard is % in the last character */
120234 int c; /* One character in z[] */
120342 ** If the pBase expression originated in the ON or USING clause of
120378 ** two subterms are in disjunction - they are OR-ed together.
120406 int idxNew; /* Index in pWC of the next virtual term */
120437 ** subterms. So in:
120442 ** This routine analyzes terms such as the middle term in the above example.
120463 ** a single table T (as shown in example B above) then create a new virtual
120464 ** term that is an equivalent IN expression. In other words, if the term
120471 ** x IN (expr1,expr2,expr3)
120481 ** The added conjunct can sometimes be helpful in query planning.
120492 ** <op> is one of "=", "<", "<=", ">", ">=", "IS NULL", or "IN".
120506 ** always prefer case 1, so in that case we pretend that case 3 is not
120515 ** to "bitmap indices" in other database engines.
120541 ** stored in a WhereClause structure containing within the WhereOrInfo
120638 ** IN operator because one or more terms in the OR clause contain
120639 ** something other than == on a column in the single table. The 1-bit
120645 ** might be possible to form an IN operator with either table1.column
120653 int okToChngToIN = 0; /* True if the conversion to IN is valid */
120654 int iColumn = -1; /* Column index on lhs of IN operator */
120659 ** other of the == operator in every subterm. That table and column
120660 ** will be recorded in iCursor and iColumn. There might not be any
120677 /* This term must be of the form t1.a==t2.b where t2 is in the
120701 ** table and column is common to every term in the OR clause */
120727 ** case 1. In that case, construct a new virtual term that is
120728 ** pTerm converted into an IN operator.
120732 ExprList *pList = 0; /* The RHS of the IN operator */
120733 Expr *pLeft = 0; /* The LHS of the IN operator */
120734 Expr *pNew; /* The complete IN operator */
120773 ** 3. Not originating in the ON clause of an OUTER JOIN
120778 ** for the LHS anyplace else in the WHERE clause where the LHS column occurs.
120809 ** a bitmask indicating which tables are used in that expression
120836 ** in any index. Return TRUE (1) if pExpr is an indexed term and return
120880 ** "pExpr" field filled in. The job of this routine is to analyze the
121076 ** lower-case (upper-case is less than lower-case in ASCII) so that
121216 ** This routine identifies subexpressions in the WHERE clause where
121218 ** operator specified in the op parameter. The WhereClause structure
121225 ** The original WHERE clause in pExpr is unaltered. All this routine
121228 ** In the previous sentence and in the diagram, "slot[]" refers to
121284 ** a bitmask indicating which tables are used in that expression
121316 ** Call exprAnalyze on all terms in a WHERE clause.
121338 ** a HIDDEN column in the table.
121378 ** The author disclaims copyright to this source code. In place of
121421 ** Return TRUE if the WHERE clause returns rows in ORDER BY order.
121429 ** Return the VDBE address or label to jump to in order to continue
121438 ** Return the VDBE address or label to jump to in order to break
121524 ** iCursor is not in the set.
121540 ** There is one cursor per table in the FROM clause. The number of
121541 ** tables in the FROM clause is limited by a test early in the
121684 ** Search for a term in the WHERE clause that is of the form "X <op> <expr>"
121692 ** The term returned might by Y=<expr> if there is another constraint in
121694 ** identified by the WO_EQUIV bit in the pTerm->eOperator field. The
121696 ** slots in aiCur[]/aiColumn[] so that means we can look for X plus up to 10
121700 ** If there are multiple terms in the WHERE clause of the form "X <op> <expr>"
121701 ** then try for the one with no dependencies on <expr> - in other words where
121703 ** the form "X <op> Y" where Y is a column in another table if no terms of
121738 ** If such an expression is found, its index in pList->a[] is returned. If
121790 ** A DISTINCT list is redundant if any subset of the columns in the
121804 /* If there is more than one table or sub-select in the FROM clause of
121825 ** 2. All of the columns in the index are either part of the pDistinct
121859 ** Convert OP_Column opcodes to OP_Copy in previously generated code.
121868 ** value stored in its output register.
121874 int iRegister, /* The first column is in this register */
121888 /* Increment the value stored in the P2 operand of the OP_Rowid. */
121955 Bitmask notReady /* Tables in outer loops of the join */
121983 int nKeyCol; /* Number of columns in the constructed index */
121994 int mxBitCol; /* Maximum column in pSrc->colUsed */
122213 /* If the ORDER BY clause contains only columns in the current
122241 ** changing them. We have to do some funky casting in order to
122271 /* The direct assignment in the previous line is possible only because
122296 ** comes in as the 3rd argument to this function.
122340 ** Estimate the location of a particular key among all keys in an
122341 ** index. Store the results in aStat as follows:
122349 ** based on the contents of aSample[] and the number of fields in record
122360 int iCol; /* Index of required stats in anEq[] etc. */
122366 int nField; /* Number of fields in pRec */
122378 ** is simply the aSample[] array. If the samples in aSample[] contain more
122382 ** samples in aSample[] (truncated to N fields), the search also has to
122384 ** in aSample is:
122407 ** For each sample in the aSample[] array, N samples are present in the
122408 ** effective sample array. In the above, samples 0 and 1 are based on
122413 ** equal to the previous sample in the array. For example, in the above,
122415 ** appears that it should be 1 field in size. However, that would make it
122424 int iSamp; /* Index in aSample[] of test sample */
122425 int n; /* Number of fields in test sample */
122471 ** all samples in the aSample[] array, pRec must be smaller than the
122480 ** in the aSample[] array. Otherwise, if (iCol>0) then pRec must
122505 ** is larger than all samples in the array. */
122587 ** extracted from pLower and pUpper with the corresponding column in each
122658 ** using the method described in the header comment for this function. */
122691 ** If either of the upper or lower bound is not present, then NULL is passed in
122694 ** The value in (pBuilder->pNew->u.btree.nEq) is the number of the index
122711 ** rows in the index. Assuming no error occurs, *pnOut is adjusted (reduced)
122714 ** In the absence of sqlite_stat4 ANALYZE data, or if such data cannot be
122740 /* Variable iLower will be set to the estimate of the number of rows in
122744 ** columns of the index, and $L is the value in pLower.
122748 ** range is $P. Due to a quirk in the way whereKeyStats() works, even
122776 ** have been requested when testing key $P in whereEqualScanEst(). */
122826 ** the estimated selectivity more in line with what it would be
122856 ** range query (e.g. col > ?) is assumed to match 1/4 of the rows in the
122879 ** an equality constraint x=VALUE and where that VALUE occurs in
122892 ** in the pParse structure.
122897 Expr *pExpr, /* Expression for VALUE in the x=VALUE constraint */
122945 ** an IN constraint where the right-hand side of the IN operator
122948 ** WHERE x IN (1,2,3,4)
122957 ** in the pParse structure.
122962 ExprList *pList, /* The value list on the RHS of "x IN (v1,v2,v3,...)" */
122984 WHERETRACE(0x10,("IN row estimate: est=%d\n", nRowEst));
123154 sqlite3DbFree(db, pLevel->u.in.aInLoop);
123245 ** Search the list of WhereLoops in *ppPrev looking for one that can be
123249 ** pTemplate, in other words if pTemplate does not belong on the list.
123269 /* In the current implementation, the rSetup value is either zero
123332 ** information is gathered in the pBuilder->pOrSet object. This special
123458 ** value corresponds to -1 in LogEst notation, so this means decrement
123464 ** of rows in the table. In other words, assume that x==EXPR will filter
123467 ** on the "x" column and so in that case only cap the output row estimate
123473 LogEst nRow /* Number of rows in the entire table */
123497 /* In the absence of explicit truth probabilities, use heuristics to
123542 LogEst nInMul /* log(Number of iterations due to IN) */
123558 LogEst rSize; /* Number of rows in the table */
123592 LogEst nOutUnadjusted; /* nOut before IN() and WHERE adjustments */
123625 /* "x IN (SELECT ...)": TUNING: the SELECT returns 25 rows */
123628 /* "x IN (value, value, ...)" */
123632 ** changes "x IN (?)" into "x=?". */
123657 ** always used in pairs. */
123678 ** values of nIn and nInMul. In other words, assuming that all
123679 ** "x IN(...)" terms are replaced with "x = ?". This block updates
123735 /* Set rCostIdx to the cost of visiting selected rows in index. Add
123738 ** visiting the rows in the main table. */
123777 ** number of repeats in the left-most terms is at least 18.
123781 ** contains fewer than 2^17 rows we assume otherwise in other parts of
123799 /* TUNING: Because uncertainties in the estimates for skip-scan queries,
123813 ** Return True if it is possible that pIndex might be useful in
123814 ** implementing the ORDER BY clause in pBuilder.
123817 ** if there is no way for pIndex to be useful in implementing that
123869 ** in the current query. Return true if it can be and false if not.
123914 ** WHERE clause includes "x IN (....)" terms used in place of "x=?". Or when
123915 ** implicit "x IN (SELECT x FROM tbl)" terms are added for skip-scans.
123920 ** log(nRow) factor is omitted from a non-covering index scan in order to
123921 ** bias the scoring in favor of using an index, since the worst-case
123940 LogEst rSize; /* number of rows in the table */
123941 LogEst rLogSize; /* Logarithm of the number of rows in the table */
123959 /* There is no INDEXED BY clause. Create a fake Index object in local
123961 ** fake index the first in a chain of Index objects with all of the real
124008 ** estimated to be X*N*log2(N) where N is the number of rows in
124020 /* TUNING: Each index lookup yields 20 rows in the table. This
124125 ** If there are no LEFT or CROSS JOIN joins in the query, both mExtra and
124127 ** entries that occur before the virtual table in the FROM clause and are
124139 ** All the tables in mExtra must be scanned before the current virtual
124141 ** mExtra may be specified as "usable" in all calls to xBestIndex.
124142 ** Conversely, all tables in mUnusable must be scanned after the current
124164 int seenIn = 0; /* True if an IN operator is seen */
124166 int iPhase; /* 0: const w/o IN, 1: const, 2: no IN, 2: IN */
124204 case 0: /* Constants without IN operator */
124215 case 1: /* Constants with IN operators */
124219 case 2: /* Variables without IN */
124223 default: /* Variables with IN */
124271 /* Do not attempt to use an IN constraint if the virtual table
124274 ** repeated in the output. */
124277 /* A virtual table that is constrained by an IN clause may not
124278 ** consume the ORDER BY clause because (1) the order of IN terms
124280 ** (2) Multiple outputs from a single IN value will not merge
124464 /* Loop over the tables in the join, from left to right */
124501 ** parameters) to see if it outputs rows in the requested ORDER BY
124511 ** and DISTINCT do not require rows to appear in any particular order as long
124513 ** the pOrderBy terms can be matched in any order. With ORDER BY, the
124514 ** pOrderBy terms must be matched in strict left-to-right order.
124521 u16 nLoop, /* Number of entries in pPath->aLoop[] */
124523 Bitmask *pRevMask /* OUT: Mask of WhereLoops to run in reverse order */
124531 u16 nKeyCol; /* Number of key columns in pIndex */
124532 u16 nColumn; /* Total number of ordered columns in the index */
124533 u16 nOrderBy; /* Number terms in the ORDER BY clause */
124534 int iLoop; /* Index of WhereLoop in pPath being processed */
124555 ** Every one-row WhereLoop will have the WHERE_ONEROW bit set in wsFlags.
124558 ** that WhereLoop that are in the ORDER BY clause are different for every
124560 ** order-distinct. A WhereLoop that has no columns in the ORDER BY clause
124567 ** rowid appears in the ORDER BY clause, the corresponding WhereLoop is
124590 /* Mark off any ORDER BY term X that is a column in the table of
124591 ** the current loop for which there is term in the WHERE
124634 ** that are not constrained by == or IN.
124653 /* Get the column number in the table (iColumn) and sort order
124705 /* Make sure the sort order is compatible in an ORDER BY clause.
124765 ** If the WHERE_GROUPBY flag is set in the mask passed to sqlite3WhereBegin(),
124770 ** Normally, in this case it is not possible for the caller to determine
124771 ** whether or not the rows are really being delivered in sorted order, or
124772 ** just in some other order that provides the required grouping. However,
124775 ** true if the rows really will be sorted in the specified order, or false
124807 ** nOrderby columns and that the first nSorted columns are already in
124850 ** will be nRowEst (in the 10*log2 representation). Or, ignore sorting
124858 int nLoop; /* Number of terms in the join */
124867 int nTo, nFrom; /* Number of valid entries in aTo[] and aFrom[] */
124888 /* If nRowEst is zero and there is an ORDER BY clause, ignore it. In this
124916 ** the ORDER BY clause are already in order, where X is the array
124933 /* If nLoop is zero, then there are no FROM terms in the query. Since
124934 ** in this case the query may return a maximum of one row, the results
124935 ** are already in the requested order. Set isOrdered to nOrderBy to
125012 ** paths currently in the best-so-far buffer. So discard
125283 ** in order to complete the WHERE clause processing.
125287 ** The basic idea is to do a nested loop, one loop for each table in
125289 ** same as a SELECT with only a single table in the FROM clause.) For
125296 ** foreach row1 in t1 do \ Code generated
125297 ** foreach row2 in t2 do |-- by sqlite3WhereBegin()
125298 ** foreach row3 in t3 do /
125304 ** Note that the loops might not be nested in the order in which they
125305 ** appear in the FROM clause if a different order is better able to make
125306 ** use of indices. Note also that when the IN operator appears in
125307 ** the WHERE clause, it might result in additional nested loops for
125308 ** scanning through all values on the right-hand side of the IN.
125316 ** in pTabList pointing at their appropriate entries. The [...] code
125322 ** the tables have indices and there are terms in the WHERE clause that
125328 ** make it to the "..." in the middle of the loop. After each "foreach",
125329 ** terms of the WHERE clause that use only terms in that loop and outer
125338 ** foreach row1 in t1 do
125340 ** foreach row2 in t2 do
125354 ** if the WHERE_GROUPBY flag is set in wctrlFlags) of a SELECT statement
125362 ** the first cursor in an array of cursors for all indices. iIdxCur should
125372 u16 wctrlFlags, /* One of the WHERE_* flags defined in sqliteInt.h */
125376 int nTabList; /* Number of elements in pTabList */
125382 WhereLevel *pLevel; /* A single level in pWInfo->a[] */
125408 /* The number of tables in the FROM clause is limited by the number of
125409 ** bits in a Bitmask
125413 sqlite3ErrorMsg(pParse, "at most %d tables in a join", BMS);
125417 /* This function normally generates a nested loop for all tables in
125419 ** only generate code for the first table in pTabList and assume that
125485 /* Assign a bit from the bitmask to every term in the FROM clause.
125494 ** Note that bitmasks are created for all pTabList->nSrc tables in
125648 /* Open all tables in the pTabList and any indices selected for
125833 if( pLoop->wsFlags & WHERE_IN_ABLE && pLevel->u.in.nIn>0 ){
125837 for(j=pLevel->u.in.nIn, pIn=&pLevel->u.in.aInLoop[j-1]; j>0; j--, pIn--){
125900 ** the co-routine into OP_Copy of result contained in a register.
125931 ** from the index instead of from the table where possible. In some cases
125935 ** Calls to the code generator in between sqlite3WhereBegin and
125991 ** macros to disable tests that are needed in the case of a general
125992 ** LALR(1) grammar but which are always false in the
125996 ** in the input grammar file. */
126002 ** Disable all error recovery processing in the parser push-down
126049 ** all elements in the list. And make sure list length does not exceed
126064 sqlite3ErrorMsg(pParse, "too many terms in compound SELECT");
126164 /* Next is all token values, in a form suitable for use by makeheaders.
126172 ** Each symbol here is a terminal symbol in the grammar.
126187 ** number is used to fill in empty slots of the hash
126200 ** which is sqlite3ParserTOKENTYPE. The entry in the union
126211 ** YYNRULE the number of rules in the grammar
126270 ** Applications can choose to define yytestcase() in the %include section
126271 ** to a macro that can assist in verifying code coverage. For production
126302 ** slots in the yy_action[] table.
126311 ** is equal to YY_SHIFT_USE_DFLT, it means that the action is not in the table
126316 ** a reduce action) then the yy_reduce_ofst[] array is used in place of
126317 ** the yy_shift_ofst[] array and YY_REDUCE_USE_DFLT is used in place of
126320 ** The following are the tables generated in this section:
126323 ** yy_lookahead[] A table containing the lookahead for each entry in
126778 ** appears in the grammar, then ID becomes a fallback token for X, Y,
126864 ** (In other words, the "major" token.)
126867 ** the information used by the action routines in the grammar.
126875 YYACTIONTYPE stateno; /* The state-number, or reduce action in SHIFTREDUCE */
126883 /* The state of the parser is completely contained in an instance of
126886 int yyidx; /* Index of top element in stack */
126955 "AND", "IS", "BETWEEN", "IN",
127230 /* 222 */ "in_op ::= IN",
127231 /* 223 */ "in_op ::= NOT IN",
127372 ** A pointer to a parser. This pointer is used in subsequent calls
127503 ** empty in SQLite. */
127535 /* In SQLite, we never try to destroy a parser that was not successfully
127536 ** created in the first place. */
127702 int yyNewState, /* The new state to shift in */
127703 int yyMajor, /* The major token to shift in */
127704 YYMINORTYPE *yypMinor /* Pointer to the minor token to shift in */
127739 unsigned char nrhs; /* Number of right-hand side symbols in the rule */
128098 ** in some cases when it is copied into the stack after the following
128108 ** from wireshark this week. Clearly they are stressing Lemon in ways
128191 case 222: /* in_op ::= IN */ yytestcase(yyruleno==222);
128201 case 223: /* in_op ::= NOT IN */ yytestcase(yyruleno==223);
128475 ** objects in a complex query.
128790 /* When doing a nested parse, one can include terms in an expression
128792 ** in the virtual machine. #N is the N-th register. */
128933 ** expr1 IN ()
128934 ** expr1 NOT IN ()
128944 ** expr1 IN (?1)
128945 ** expr1 NOT IN (?2)
128956 ** the semantics would be subtly different from IN or NOT IN.
129359 ** onto the stack here, and skip the stack overflow test in yy_shift().
129437 ** user wants (and specified in the grammar) and is available for
129616 ** The author disclaims copyright to this source code. In place of
129672 ** mkkeywordhash.h, located in the tool subdirectory of the distribution.
129677 /************** Include keywordhash.h in the middle of tokenize.c ************/
129681 ** The code in this file has been automatically generated by
129685 ** The code in this file implements a function that determines whether
129694 /* zText[] encodes 834 bytes of keywords in 554 bytes */
129913 testcase( i==93 ); /* IN */
129955 /************** Continuing where we left off in tokenize.c *******************/
129959 ** If X is a character that can be used in an identifier then
129963 ** allowed in an identifier. For 7-bit characters,
129969 ** Ticket #1066. the SQL standard does not allow '$' in the
129971 ** SQLite will allow '$' in identifiers for compatibility.
130004 ** Store the token type in *tokenType before returning.
130272 ** passed in. An SQLITE_ status code is returned. If an error occurs
130397 ** structure built up in pParse->pNewTable. The calling code (see vtab.c)
130426 ** The author disclaims copyright to this source code. In place of
130445 ** This is defined in tokenize.c. We just have to import the definition.
130474 ** Return TRUE if the given SQL string ends in a semicolon.
130485 ** returns 1 if it ends in the START state and 0 if it ends
130486 ** in any other state.
130488 ** (2) NORMAL We are in the middle of statement which ends with a single
130498 ** (5) TRIGGER We are in the middle of a trigger definition that must be
130501 ** (6) SEMI We've seen the first semicolon in the ";END;" that occurs at
130527 u8 state = 0; /* Current state, using numbers defined in header comment */
130602 case '[': { /* Microsoft-style identifiers in [...] */
130719 In place of
130727 ** Main file for the SQLite library. The routines in this file
130728 ** implement the programmer interface to the library. Routines in
130735 /************** Include fts3.h in the middle of main.c ***********************/
130740 ** The author disclaims copyright to this source code. In place of
130765 /************** Continuing where we left off in main.c ***********************/
130768 /************** Include rtree.h in the middle of main.c **********************/
130773 ** The author disclaims copyright to this source code. In place of
130798 /************** Continuing where we left off in main.c ***********************/
130801 /************** Include sqliteicu.h in the middle of main.c ******************/
130806 ** The author disclaims copyright to this source code. In place of
130832 /************** Continuing where we left off in main.c ***********************/
131007 ** the pInitMutex mutex. Return an error in either case. */
131022 ** methods. The sqlite3_pcache_methods.xInit() all is embedded in the
131097 ** while any part of SQLite is otherwise in use in any thread. This
131133 ** this function resulted in the heap subsystem actually being shutdown.
131161 ** the SQLite library is in use. */
131167 /* Mutex configuration options are only available in a threadsafe
131216 ** low-level memory allocation routines to be used in place of the memory
131260 ** in SQLITE_CONFIG_PAGECACHE. */
131305 ** number of bytes in the memory buffer, and the minimum allocation size.
131381 ** boolean in order to enable or disable the use of covering indices for
131382 ** full table scans in the query optimizer. */
131450 ** The sz parameter is the number of bytes in each lookaside slot.
131567 u32 mask; /* Mask of the bit in sqlite3.flags to set/clear */
131624 /* EVIDENCE-OF: R-65033-28449 The built-in BINARY collation compares
131634 ** spaces at the end of either string do not change the result. In other
131636 ** differ only in the number of spaces at the end.
131646 ** Another built-in collating sequence: NOCASE.
131650 ** extends only to the 26 characters used in the English language.
131682 ** Return the number of changes in the most recent call to sqlite3_exec().
131805 ** tables in the db->aVTrans[] array. The following sqlite3VtabRollback()
131907 /* Tell the code in notify.c that the connection no longer holds any
131976 ** any write cursors are invalidated ("tripped" - as in "tripping a circuit
131979 ** but are "saved" in case the table pages are moved around.
131989 ** This is important in case the transaction being rolled back has
131991 ** here, then another shared-cache connection might sneak in between
131993 ** corruption reports in some cases. */
132028 ** specified in the argument.
132136 ** Return a static string that describes the kind of error specified in the
132189 ** an integer number of milliseconds passed in as the first
132338 ** that if a malloc() fails in sqlite3_create_function(), an error code
132535 ** A global function must exist in order for name resolution to work
132695 ** Invoke sqlite3_wal_checkpoint if the number of frames in the log file
132717 ** more frames in the log file. Passing zero or a negative value as the
132777 int *pnLog, /* OUT: Size of WAL log in frames */
132790 /* Initialize the output variables to -1 in case an error occurs. */
132837 ** not currently open in WAL mode.
133012 ** Return a string that describes the kind of error specified in the
133098 ** initializer must be kept in sync with the SQLITE_LIMIT_*
133099 ** #defines in sqlite3.h.
133176 ** macro called SQLITE_MAX_NAME. (The "_LIMIT_" in the name is changed to
133217 ** the default flags to open the database handle with. The value stored in
133221 ** If successful, SQLITE_OK is returned. In this case *ppVfs is set to point to
133235 unsigned int *pFlags, /* IN/OUT: SQLITE_OPEN_XXX flags */
133316 /* This branch is taken when "%00" appears within the URI. In this
133317 ** case we ignore all text in the remainder of the path, name or
133480 /* Only allow sensible combinations of bits in the flags argument.
133483 ** assert() statements in deeper layers. Sensible combinations
133518 ** dealt with in the previous code block. Besides these, the only
133598 ** EVIDENCE-OF: R-52786-44878 SQLite defines three built-in collating
133654 /* Register all built-in functions, but do not attempt to read the
133794 char const *zFilename8; /* zFilename encoded in UTF-8 instead of UTF-16 */
133946 ** Test to see whether or not the database connection is in autocommit
133966 ** 1. Serve as a convenient place to set a breakpoint in a debugger
134009 ** See comment in sqlite3.h (sqlite.h.in) for details.
134048 /* Locate the table in question */
134077 ** to the caller in local variables zDataType, zCollSeq, notnull, primarykey
134135 /* This function works in milliseconds, but the underlying OsSleep()
134285 ** Set the PENDING byte to the value in the argument, if X>0.
134289 ** IMPORTANT: Changing the PENDING byte from 0x40000000 results in
134291 ** while any database connection is open results in undefined and
134394 ** is obtained in every case.
134408 ** This test feature is only available in the amalgamation since
134409 ** the SQLITE_N_KEYWORD macro is not defined in this file if SQLite
134452 ** testing causes certain assert() statements in the code to be activated
134501 ** run a single CREATE TABLE statement to construct the imposter table in
134626 ** The author disclaims copyright to this source code. In place of
134669 ** 1) Each entry in the list has a non-NULL value for either
134672 ** 2) All entries in the list that share a common value for
134675 ** 3) If the argument db is not NULL, then none of the entries in the
134754 ** (call it pOther) in the same process was busy using the same shared
134788 ** blocking transaction. In either case, invoke the notify callback
134834 ** This function loops through each entry in the blocked connections
134850 int nArg = 0; /* Number of entries in aArg[] */
134859 /* This loop runs once for each entry in the blocked-connections list. */
134961 ** The author disclaims copyright to this source code. In place of
134974 ** The code in this file is only compiled if:
134977 ** (in which case SQLITE_CORE is not defined), or
134980 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
134983 /* The full-text index is stored in a series of b+tree (-like)
134985 ** structures are like b+trees in layout, but are constructed from the
134986 ** bottom up in optimal fashion and are not updatable. Since trees
134993 ** varint. We encode variable-length integers in little-endian order
135005 ** This is similar in concept to how sqlite encodes "varints" but
135007 ** are are limited to 9 bytes in length whereas FTS3 varints are
135008 ** little-endian and can be up to 10 bytes in length (in theory).
135044 ** Here, array { X } means zero or more occurrences of X, adjacent in
135045 ** memory. A "position" is an index of a token in the token stream
135047 ** in the same logical place as the position element, and act as sentinals
135056 ** The 123 value is the first docid. For column zero in this document
135094 ** Here, array { X } means zero or more occurrences of X, adjacent in
135097 ** Leaf nodes are broken into blocks which are stored contiguously in
135098 ** the %_segments table in sorted order. This means that when the end
135099 ** of a node is reached, the next term is in the node with the next
135104 ** larger than STANDALONE_MIN (default 1024) is placed in a standalone
135140 ** means zero or more occurrences of X, adjacent in memory.
135162 ** The segment directory in table %_segdir stores meta-information for
135174 ** The meta-information in the segment directory is:
135189 ** merged in batches. Each increase in level represents exponentially
135200 ** A segment merge traverses all segments at a given level in
135202 ** leaf nodes are written in to the %_segments table in order, this
135210 ** a tiny bit slower (perhaps due to more overhead in merge-time
135248 /************** Include fts3Int.h in the middle of fts3.c ********************/
135253 ** The author disclaims copyright to this source code. In place of
135288 /************** Include fts3_tokenizer.h in the middle of fts3Int.h **********/
135327 ** implementation. The xCreate() function in turn returns an
135349 ** Create a new tokenizer. The values in the argv[] array are the
135403 ** of this buffer in bytes. The input text that generated the token is
135404 ** identified by the byte offsets returned in *piStartOffset and
135406 ** byte of the token in the input buffer. *piEndOffset should be set
135407 ** to the index of the first byte just past the end of the token in
135421 int *piStartOffset, /* OUT: Byte offset of token in input buffer */
135422 int *piEndOffset, /* OUT: Byte offset of end of token in input buffer */
135453 /************** Continuing where we left off in fts3Int.h ********************/
135454 /************** Include fts3_hash.h in the middle of fts3Int.h ***************/
135459 ** The author disclaims copyright to this source code. In place of
135468 ** used in SQLite. We've modified it slightly to serve as a standalone
135490 int count; /* Number of entries in this table */
135492 int htsize; /* Number of buckets in the hash table */
135499 /* Each element in the hash table is an instance of the following
135506 Fts3HashElem *next, *prev; /* Next and previous elements in the table */
135516 ** is respected in comparisons.
135563 ** Number of entries in a hash table
135570 /************** Continuing where we left off in fts3Int.h ********************/
135592 ** This is the maximum amount of data (in bytes) to store in the
135594 ** populated as documents are inserted/updated/deleted in a transaction
135602 ** Macro to return the number of elements in an array. SQLite has a
135604 ** a collision when building an amalgamation with built-in FTS3.
135624 ** in the document set and zero or more prefix indexes. All indexes are stored
135625 ** as one or more b+-trees in the %_segments and %_segdir tables.
135628 ** value stored in the "%_segdir.level" column. Given this value L, the index
135629 ** that the b+-tree belongs to is (L<<10). In other words, all b+-trees with
135633 ** It is considered impossible for an index to use more than 1024 levels. In
135744 int nColumn; /* number of named columns in virtual table */
135765 u8 bDescIdx; /* True if doclists are in reverse order */
135775 ** The current language id is stored in variable iPrevLangid.
135778 ** there is an entry in the aIndex[] array. Index 0 is an index of all the
135779 ** terms that appear in the document set. Each subsequent index in aIndex[]
135826 sqlite3_stmt *pStmt; /* Prepared statement in use by the cursor */
135829 int nPhrase; /* Number of matchable phrases in query */
135835 u8 bDesc; /* True to sort in descending order */
135837 int nRowAvg; /* Average size of database rows, in pages */
135838 sqlite3_int64 nDoc; /* Documents in table */
135841 int isMatchinfoNeeded; /* True when aMatchinfo[] needs filling in */
135853 ** of the column to be searched. For example, in
135880 int nAll; /* Size of a[] in bytes */
135890 ** A "phrase" is a sequence of one or more tokens that must match in
135892 ** For a sequence of tokens contained in double-quotes (i.e. "one two three")
135893 ** nToken will be the number of tokens in the string.
135897 int n; /* Number of bytes in buffer z */
135902 ** parsed (by code in fts3_expr.c). Below this point the variables are
135922 int nToken; /* Number of tokens in the phrase */
135924 Fts3PhraseToken aToken[1]; /* One entry for each token in the phrase */
135932 ** of this phrase query in FTS3 doclist format. As usual, the initial
135933 ** "Length" field found in doclists stored on disk is omitted from this
135938 ** where nCol is the number of columns in the queried FTS table. The array
135963 int iPhrase; /* Index of this phrase in matchinfo() results */
135969 ** four values is in order of precedence when parsing expressions. For
136051 char *aBuffer; /* Buffer to merge doclists in */
136052 int nBuffer; /* Allocated size of aBuffer[] in bytes */
136063 int nTerm; /* Size of zTerm in bytes */
136065 int nDoclist; /* Size of aDoclist[] in bytes */
136144 /************** Continuing where we left off in fts3.c ***********************/
136188 q[-1] &= 0x7f; /* turn off high bit in final byte */
136203 ** The value is stored in *v.
136262 ** the quote characters. The conversion is done in-place. If the
136327 ** interested in. So, unless the doclist is corrupt, the 0x80 bit is
136384 sqlite3 *db, /* Database in which to run SQL */
136434 ** Otherwise, if an error occurs, an SQLite error code is stored in *pRc
136551 ** Store the current database page-size in bytes in p->nPgsz.
136554 ** Otherwise, if an error occurs, an SQLite error code is stored in *pRc
136617 int *pRc, /* IN/OUT: Error code */
136618 char **pz, /* IN/OUT: Pointer to string buffer */
136640 ** Return a copy of input string zInput enclosed in double-quotes (") and
136670 ** could be used in a SELECT statement such as the following:
136674 ** to return the docid, followed by each column of text data in order
136730 ** of columns in the %_content table (one for the docid plus one for each
136812 ** the allocated array. *pnIndex is set to the number of elements in the
136819 const char *zParam, /* ABC in prefix=ABC parameter to parse */
136824 int nIndex = 1; /* Number of entries in array */
136878 ** the name of the corresponding column in table xxx. The array
136966 int argc, /* Number of elements in argv array */
136978 int nCol = 0; /* Number of columns in the FTS table */
137030 ** + If there is a tokenizer specification included in the arguments,
137238 /* Fill in the zName and zDb fields of the vtab structure. */
137247 /* Fill in the azColumn array */
137260 /* Fill in the abNotindexed array */
137284 sqlite3Fts3ErrMsg(pzErr, "missing %s parameter in fts4 constructor", zMiss);
137290 /* If this is an xCreate call, create the underlying tables in the
137304 /* Figure out the page-size for the database. This is required in order to
137337 ** work is done in function fts3InitVtab().
137342 int argc, /* Number of elements in argv array */
137352 int argc, /* Number of elements in argv array */
137363 ** support estimatedRows. In that case this function is a no-op.
137374 ** Set the SQLITE_INDEX_SCAN_UNIQUE flag in pIdxInfo->flags. Unless this
137376 ** support index-info flags. In
137388 ** are three possible strategies, in order of preference:
137418 ** function MATCH in the requested context" error. To discourage
137443 ** it would lead to an "unable to use function MATCH in the requested
137497 /* Regardless of the strategy selected, FTS can deliver rows in rowid (or
137598 ** table is missing a row that is present in the full-text index.
137617 ** passed in zTerm/nTerm.
137630 int nTerm, /* Size of term zTerm in bytes */
137651 ** not make this guarantee explicitly, but in practice there are always
137736 ** left-most leaf node in the tree that may contain the specified term.
137752 int nTerm, /* Size of term zTerm in bytes */
137759 int iHeight; /* Height of this node in tree */
137769 int nBlob = 0; /* Size of zBlob in bytes */
137798 char **pp, /* IN/OUT: Output pointer */
137799 sqlite3_int64 *piPrev, /* IN/OUT: Previous value written to list */
137813 ** a single document record of a doclist. So, in other words, this
137814 ** routine advances *ppPoslist so that it points to the next docid in
137833 ** last byte in the position-list.
137897 ** called, *pp may point to the start of the next varint in the position-list
137899 ** (in which case **pp will be a terminator bytes POS_END (0) or
137909 ** in the position-list. Because positions are delta-encoded, the value
137910 ** of the previous position is needed in order to compute the value of
137914 char **pp, /* IN/OUT: Pointer into position-list buffer */
137915 sqlite3_int64 *pi /* IN/OUT: Value read from position-list */
137947 ** into *pp contains all positions of both *pp1 and *pp2 in sorted
137950 ** that there is enough space in *pp to hold the complete output.
137962 int iCol1; /* The current column index in pp1 */
137963 int iCol2; /* The current column index in pp2 */
138035 ** each position in *pp2 for which there exists one or more positions in
138043 char **pp, /* IN/OUT: Preallocated output buffer */
138044 int nToken, /* Maximum difference in token positions */
138047 char **pp1, /* IN/OUT: Left input list */
138048 char **pp2 /* IN/OUT: Right input list */
138124 ** column-number in the position list.
138159 ** expression and *pp2 to the right. As usual, the indexes in the position
138160 ** lists are the offsets of the last token in each phrase (tokens "1" and "2"
138161 ** in the example above).
138164 ** entries that are not sufficiently NEAR entries in *pp1 removed.
138169 int nRight, /* Maximum difference in token positions */
138170 int nLeft, /* Maximum difference in token positions */
138171 char **pp1, /* IN/OUT: Left input list */
138172 char **pp2 /* IN/OUT: Right input list */
138208 int anOutput[16]; /* Size each output buffer in bytes */
138215 ** has been reached. In this case *pp is set to 0 and the function returns.
138225 char **pp, /* IN/OUT: Point to read varint from */
138228 sqlite3_int64 *pVal /* IN/OUT: Integer value */
138259 char **pp, /* IN/OUT: Output pointer */
138261 sqlite3_int64 *piPrev, /* IN/OUT: Previous value written to list */
138262 int *pbFirst, /* IN/OUT: True after first int written */
138286 ** sorted in either ascending or descending order.
138292 ** positions contained in either argument doclist). If the docids in the
138293 ** input doclists are sorted in ascending order, parameter bDescDoclist
138294 ** should be false. If they are sorted in ascending order, it should be
138298 ** containing the output doclist and SQLITE_OK is returned. In this case
138299 ** *pnOut is set to the number of bytes in the output doclist.
138302 ** are undefined in this case.
138325 ** are delta encoded. If they are in
138326 ** then the first docid in each list is simply encoded as a varint. For
138328 ** current and previous docid (a positive number - since the list is in
138332 ** same number of bytes as it is in whichever of the input lists it is
138334 ** consumes either the same or less bytes as it did in the input (since
138335 ** the difference between it and the previous value in the output must
138343 ** be larger in the output than it was in the input (since the delta value
138350 ** A symetric argument may be made if the doclists are in descending
138385 ** This function does a "phrase" merge of two doclists. In a phrase merge,
138387 ** doclist for which there is a position in the left-hand input doclist
138390 ** If the docids in the input doclists are sorted in ascending order,
138391 ** parameter bDescDoclist should be false. If they are sorted in ascending
138400 char **paRight, int *pnRight /* IN/OUT: Right/output doclist */
138460 ** Argument pList points to a position list nList bytes in size. This
138462 ** a token in position 0 (of any column). If so, it writes argument iDelta
138470 int nList, /* Size of pList in bytes */
138511 ** Merge all doclists in the TermSelect.aaOutput[] array into a single
138512 ** doclist stored in TermSelect.aaOutput[0]. If successful, delete all
138515 ** If an OOM error occurs, return SQLITE_NOMEM. In this case it is
138516 ** the responsibility of the caller to free any doclists left in the
138524 /* Loop through the doclists in the aaOutput[] array. Merge them all
138577 int nDoclist /* Size of aDoclist in bytes */
138586 ** doclists are stored in order=ASC order, this padding would not be
138588 ** than or equal to the size of [doclistA] in that case). But this is
138590 ** may be smaller than (-1), as in the first example the -1 may be stored
138591 ** as a single-byte delta, whereas in the second it must be stored as a
138594 ** Similar padding is added in the fts3DoclistOrMerge() function.
138676 int nTerm, /* Size of zTerm in bytes */
138687 ** made by an fts4aux module, not an FTS table. In this case calling
138750 int nTerm, /* Size of zTerm in bytes */
138771 ** In addition to its current configuration, have the Fts3MultiSegReader
138780 int nTerm, /* Number of bytes in zTerm */
138804 int nTerm, /* Size of zTerm in bytes */
138916 ** This function counts the total number of docids in the doclist stored
138917 ** in buffer aList[], size nList bytes.
138940 ** Advance the cursor to the next row in the %_content table that
138943 ** simply the next row in the %_content table. For a docid lookup,
139004 ** in the %_content table.
139015 int nVal, /* Number of elements in apVal */
139045 /* In case the cursor has been used before, clear it now. */
139164 /* The column value supplied by SQLite must be in range. */
139222 ** in place to remove the entries that have already been merged. This
139248 int mxLevel = 0; /* Maximum relative level value in db */
139358 ** is the first in the doclist. In this case do not skip forward 2 bytes.
139414 int nToken = 15; /* Default number of tokens in snippet */
139466 ** function merges all segments in the database to a single segment.
139579 ** PendingTermsFlush() in in case that changes.
139701 ** The fts3 built-in tokenizers - "simple", "porter" and "icu"- are
139702 ** implemented in files fts3_tokenizer1.c, fts3_porter.c and fts3_icu.c
139704 ** declared in these files used to retrieve the respective implementations.
139762 /* Load the built-in tokenizers into the hash table */
139821 ** Allocate an Fts3MultiSegReader for each token in the expression headed
139829 ** If the allocated Fts3MultiSegReader just seeks to a single entry in a
139832 ** and merged incrementally. Otherwise, it has to be merged into an in-memory
139838 int *pnToken, /* OUT: Total number of tokens in phrase. */
139839 int *pnOr, /* OUT: Total number of OR nodes in expr. */
139840 int *pRc /* IN/OUT: Error code */
139869 ** It is merged into the main doclist stored in p->doclist.aAll/nAll.
139882 int nList /* Number of bytes in pList */
139972 ** means that the phrase does not appear in the current row, doclist.pList
139980 int nPoslist = 0; /* Number of bytes in aPoslist */
140078 ** This function is called for each Fts3Phrase in a full-text query
140096 ** scanned in forward order, and the phrase consists of
140139 ** doclists in reverse and by the fts3_write.c module to iterate through
140142 ** The doclist may be sorted in ascending (parameter bDescIdx==0) or
140149 int nDoclist, /* Length of aDoclist in bytes */
140150 char **ppIter, /* IN/OUT: Iterator pointer */
140151 sqlite3_int64 *piDocid, /* IN/OUT: Docid pointer */
140205 int nDoclist, /* Length of aDoclist in bytes */
140206 char **ppIter, /* IN/OUT: Iterator pointer */
140207 sqlite3_int64 *piDocid, /* IN/OUT: Docid pointer */
140236 ** Advance the iterator pDL to the next entry in pDL->aAll/nAll. Set *pbEof
140270 ** edited in place by fts3EvalNearTrim(), then pIter may not actually
140295 ** multi-token phrase. Advance it to the next matching document in the
140344 ** Advance it to the next matching documnent in the database and populate
140365 ** one incremental token. In which case the bIncr flag is set. */
140386 /* Advance the iterator for each token in the phrase once. */
140486 ** each phrase in the expression (subject to deferred token processing).
140495 Fts3Expr *pExpr, /* Expression to initialize phrases in */
140496 int *pRc /* IN/OUT: Error code */
140520 ** of the xFilter() method). There is one element in the array for each
140521 ** token in the FTS expression.
140523 ** Tokens are divided into AND/NEAR clusters. All tokens in a cluster belong
140526 ** separately. The root of a tokens AND/NEAR cluster is stored in
140532 int iToken; /* Position of token in phrase */
140552 int *pRc /* IN/OUT: Error code */
140590 ** Determine the average document (row) size in pages. If successful,
140594 ** The average document size in pages is calculated by first calculating
140595 ** determining the average size in bytes, B. If B is less than the amount
140596 ** of data that will fit on a single leaf page of an intkey table in
140598 ** the number of overflow pages consumed by a record B bytes in size.
140607 ** varints, where nCol is the number of columns in the FTS3 table.
140608 ** The first varint is the number of documents currently stored in
140610 ** data stored in all rows of each column of the table, from left
140652 ** This function is called once for each AND/NEAR cluster in the
140665 int nTC /* Number of entries in aTC[] */
140672 int nToken = 0; /* Total number of tokens in cluster */
140686 /* Count the tokens in this AND/NEAR cluster. If none of the doclists
140688 ** only 1 token, exit early. No tokens to defer in this case. */
140697 /* Obtain the average docsize (in pages). */
140702 /* Iterate through all tokens in this AND/NEAR cluster, in ascending order
140707 ** a. The cheapest token in the entire query (i.e. the one visited by the
140714 ** contains. Set variable "nMinEst" to the smallest number of documents in
140719 ** Then, for each token, defer it if loading the doclist would result in
140750 ** overflowing the 32-bit integer it is stored in. */
140754 /* Either this is the cheapest token in the entire query, or it is
140784 ** the full-text query currently stored in pCsr->pExpr. To iterate through
140800 /* Allocate a MultiSegReader for each token in the expression. */
140803 /* Determine which, if any, tokens in the expression should be deferred. */
140860 ** Parameter nNear is passed the NEAR distance of the expression (5 in
140862 ** the position list, and *pnToken is the number of phrase tokens in, the
140867 ** All positions in the pPhrase position list that are not sufficiently
140868 ** close to a position in the *paPoslist position list are removed. If this
140872 ** associated with pPhrase. And *pnToken is set to the number of tokens in
140876 int nNear, /* NEAR distance. As in "NEAR/nNear". */
140878 char **paPoslist, /* IN/OUT: Position list */
140879 int *pnToken, /* IN/OUT: Tokens in phrase of *paPoslist */
140911 ** point to the next matching row in the database. Expressions iterate through
140912 ** matching rows in docid order. Ascending order if Fts3Cursor.bDesc is zero,
140916 ** successful, the following variables in pExpr are set:
140925 ** FTs3Expr.pPhrase->doclist.nList (length of pList in bytes)
140932 ** entirely of deferred tokens, it is assumed to match every row in
140933 ** the db. In this case the position-list is not populated at all.
140937 ** next possible match, considering only non-deferred tokens. In other
140940 ** where "*" may match any single token. The position list in this case
140952 int *pRc /* IN/OUT: Error code */
141080 ** expression, using the data currently stored in the position lists
141081 ** (Fts3Expr->pPhrase.doclist.pList/nList) for each phrase in the expression.
141084 ** phrase in the NEAR expression is edited in place to contain only those
141086 ** constraints. In this case it returns 1. If the NEAR expression does not
141098 ** which is represented in tree form as:
141111 ** no exceptions to this - it's the way the parser in fts3_expr.c works.
141173 ** the appropriate SQLite error code. In this case the returned value is
141179 int *pRc /* IN/OUT: Error code */
141193 ** all phrases involved in the NEAR. This is because the snippet(),
141269 ** * Up until this point, "NEAR" operators in the expression have been
141314 ** Advance to the next document that matches the FTS expression in
141394 ** After allocating the Fts3Expr.aMI[] array for each phrase in the
141397 ** the values in Fts3Expr.aMI[] according to the position-list currently
141398 ** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase
141509 /* Caution: pRoot may iterate through docids in ascending or descending
141529 ** 1. The total number of occurrences of the phrase in each column of
141532 ** 2. For each column, the number of rows in the table for which the
141544 ** values are set to the number of documents in the table. In other
141545 ** words we assume that very common tokens occur exactly once in each
141553 ** that meet the NEAR constraint are included in the counts.
141589 ** a position-list indicating the occurrences of the phrase in column iCol
141593 ** occurrence of the phrase in the column, stored using the normal (delta+2)
141636 ** pCsr->iPrevId may lie earlier in the doclist buffer. Or, if the
141647 /* This is the descendent of an OR node. In this case we cannot use
141649 ** into memory in this case. */
141784 ** The author disclaims copyright to this source code. In place of
141840 int argc, /* Number of elements in argv array */
141855 /* The user should invoke this in one of two forms:
141942 /* This vtab delivers always results in "ORDER BY term ASC" order. */
142086 /* State 0. In this state the integer just read was a docid. */
142093 /* State 1. In this state we are expecting either a 1, indicating
142098 ** integer encountered in state 1 is not 0 or 1, then we need to
142145 int nVal, /* Number of elements in apVal */
142154 int iEq = -1; /* Index of term=? value in apVal */
142155 int iGe = -1; /* Index of term>=? value in apVal */
142156 int iLe = -1; /* Index of term<=? value in apVal */
142157 int iLangid = -1; /* Index of languageid=? value in apVal */
142184 /* In case this cursor is being reused, close and zero it. */
142337 ** The author disclaims copyright to this source code. In place of
142378 ** operator in a similar format to that used by the lemon parser
142421 ** FTSQUERY_PHRASE with a unary "-" attached to it. i.e. "mysql" in the
142431 int nCol; /* Number of entries in azCol[] */
142499 ** and other information (column names etc.) in pParse. Create an Fts3Expr
142599 ** is included in the buffer. This function attempts to tokenize the entire
142629 ** The first pass, in the block below, uses a tokenizer cursor to iterate
142630 ** through the tokens in the expression. This pass uses fts3ReallocOrFree()
142631 ** to assemble data in two dynamic buffers:
142639 ** The second pass, in the block that begins "if( rc==SQLITE_DONE )" below,
142640 ** appends buffer zTemp to buffer p, and fills in the Fts3Expr and Fts3Phrase
142735 unsigned char parenOnly; /* Only valid in paren mode */
142817 ** a quote character embedded in a string.
142877 ** group more tightly). For example, in the C language, the == operator
142881 ** is defined), the order of the operators in precedence from highest to
142910 ** of pNew and the existing nodes of the tree. This may result in the head
142911 ** of the tree changing, in which case *ppHead is set to the new root node.
142935 ** Parse the fts3 query expression found in buffer z, length n. This function
142952 Fts3Expr *pNotBranch = 0; /* Only used in legacy parse mode */
142992 ** an expression contained in parenthesis is required. If a
143024 ** Return an error in either case.
143110 ** an equivalent but more balanced form. The tree is modified in place.
143123 int eType = pRoot->eType; /* Type of node in this tree */
143143 /* Set $p to point to the left-most leaf in the tree of eType nodes. */
143149 /* This loop runs once for each leaf in the tree of eType nodes. */
143190 /* Set $p to point to the next leaf in the tree of eType nodes */
143291 ** in this case.
143298 int nCol, /* Number of entries in azCol[] */
143347 ** of each column in the target fts3 table, in order from left to right.
143361 int nCol, /* Number of entries in azCol[] */
143649 ** The author disclaims copyright to this source code. In place of
143657 ** This is the implementation of generic hash-tables used in SQLite.
143663 ** The code in this file is only compiled if:
143666 ** (in which case SQLITE_CORE is not defined), or
143669 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
143776 ** The C syntax in this function definition may be unfamilar to some
143797 ** For help in interpreted the obscure C code in the function definition,
143816 Fts3HashElem *pHead; /* First element already in pEntry */
143861 /* This function (for internal use only) locates an element in an
143964 ** The key is not copied in this instance. If a malloc fails, then
143973 int nKey, /* Number of bytes in the key */
144035 ** The author disclaims copyright to this source code. In place of
144048 ** The code in this file is only compiled if:
144051 ** (in which case SQLITE_CORE is not defined), or
144054 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
144080 int iOffset; /* current position in zInput */
144117 ** used to incrementally tokenize this string is returned in
144168 ** isConsonant() and isVowel() determine if their first character in
144174 ** in which case it is a vowel.
144176 ** In these routine, the letters are in reverse order. So the 'y' rule
144207 ** In prose: A word is an optional consonant followed by zero or
144212 ** Return true if the m-value for z is 1 or more. In other words,
144216 ** In this routine z[] is in reverse order. So we are really looking
144263 ** Return TRUE if the word ends in a double consonant.
144278 ** first three letters and the first one cannot be in [wxy].
144293 ** The input word *pz and zFrom are both in reverse order. zTo
144294 ** is in normal order.
144350 ** Stem the input word zIn[0..nIn-1]. Store the output in zOut.
144354 ** Any upper-case characters in the US-ASCII character set ([A-Z])
144365 ** in [a-zA-Z] then no stemming is attempted and this routine just
144389 /* The use of a character not in [a-zA-Z] means that we fallback
144413 /* Do nothing. The work was all in the test */
144421 /* Do nothing. The work was all in the test */
144595 /* z[] is now the stemmed word in reverse order. Flip it back
144608 ** part of a token. In other words, delimiters all must have
144628 int *pnBytes, /* OUT: Number of bytes in token */
144685 ** tokenizer in *ppModule
144700 ** The author disclaims copyright to this source code. In place of
144714 ** The code in this file is only compiled if:
144717 ** (in which case SQLITE_CORE is not defined), or
144720 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
144741 ** the string <key-name> must already exist in the has table. Otherwise,
144913 ** designed to be used in concert with the Tcl testing framework. This
144923 ** list. For each token in the <input-string>, three elements are
144927 in "simple" tokenizer:
144992 zErr = "error in xCreate()";
144997 zErr = "error in xOpen()";
145010 zErr = "error in xClose()";
145014 zErr = "error in xDestroy()";
145079 ** This function is used for testing only, it is not included in the
145083 ** can be used as designed by the C-code in the queryTokenizer and
145085 ** in the README.tokenizer file as an example, so it is important to
145131 ** Set up SQL objects in database db used to access the contents of
145139 ** scalarFunc() in this file for details) and, if ENABLE_TABLE is
145197 ** The author disclaims copyright to this source code. In place of
145210 ** The code in this file is only compiled if:
145213 ** (in which case SQLITE_CORE is not defined), or
145216 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
145237 int iOffset; /* current position in pInput */
145266 ** track such information in the database, then we'd only want this
145303 ** used to incrementally tokenize this string is returned in
145353 int *pnBytes, /* OUT: Number of bytes in token */
145419 ** tokenizer in *ppModule
145434 ** The author disclaims copyright to this source code. In place of
145460 ** statement and returns one row for each token in the result. With
145498 int nToken; /* Size of zToken in bytes */
145535 ** in this case. Or, if an error occurs, an SQLite error code is returned.
145536 ** The final value of *pazDequote is undefined in this case.
145539 int argc, /* Number of elements in argv[] */
145591 int argc, /* Number of elements in argv array */
145766 int nVal, /* Number of elements in apVal */
145891 ** The author disclaims copyright to this source code. In place of
145904 ** code in fts3.c.
145920 ** of it. i.e. if a full-text index node is 900 bytes in size, then a buffer
145937 ** bytes and larger. Nodes are loaded in chunks of FTS3_NODE_CHUNKSIZE bytes.
145957 ** The two values that may be meaningfully bound to the :1 parameter in
145967 ** debugging FTS only, it should not usually be turned on in production
146002 int iCol; /* Column token must occur in */
146003 Fts3DeferredToken *pNext; /* Next in list of deferred tokens */
146010 ** this structure are only manipulated by code in this file, opaque handles
146011 ** of type Fts3SegReader* are also used by code in fts3.c to iterate through
146031 sqlite3_int64 iEndBlock; /* Rowid of final block in segment (or 0) */
146046 int nTerm; /* Number of bytes in current term */
146050 int nDoclist; /* Size of doclist in current entry */
146064 ** An instance of this structure is used to create a segment b-tree in the
146074 sqlite3_int64 iFirst; /* First slot in %_segments written */
146075 sqlite3_int64 iFree; /* Next free slot in %_segments */
146077 int nTerm; /* Number of bytes in zTerm */
146081 int nData; /* Bytes of data in aData */
146099 ** the tree is assembled in memory and written out only once all leaves have
146110 int nTerm; /* Number of bytes in zTerm */
146194 /* Return segments in order from oldest to newest.*/
146220 ** of the oldest level in the db that contains at least ? segments. Or,
146221 ** if no level in the FTS index contains more than ? segments, the statement
146226 /* Estimate the upper limit on the number of leaf nodes in a new segment
146263 ** Return the largest relative level in the FTS index or indexes. */
146266 /* Return segments in order from oldest to newest.*/
146369 ** Similar to fts3SqlStmt(). Except, after binding the parameters in
146432 ** level within the index) are encoded in 64-bit integer values stored
146433 ** in the %_segdir table on disk. This function is used to convert three
146450 int iIndex, /* Index in p->aIndex[] */
146464 ** all rows in the %_segdir table, from oldest to newest. If successful,
146527 PendingList **pp, /* IN/OUT: Pointer to PendingList struct */
146567 ** Zero is always returned in this case. Otherwise, if no OOM error occurs,
146571 PendingList **pp, /* IN/OUT: PendingList structure */
146665 ** pending-terms hash-table. The docid used is that currently stored in
146675 u32 *pnWord /* IN/OUT: Incr. by number tokens inserted */
146695 ** zText==0. In this case, add zero token entries to the hash table and
146796 ** operation. It adds entries for each term in the new record to the
146863 ** defined columns in the FTS3 table, plus one for the docid field.
146881 ** In FTS3, this is an error. It is an error to specify non-NULL values
146942 ** The first element in the apVal[] array is assumed to contain the docid
146994 ** This function allocates a new level iLevel index in the segdir table.
147036 ** full, merge all segments in level iLevel into a single iLevel+1
147060 ** with the contents of the blob stored in the "block" column of the
147062 ** to the size of the blob in bytes before returning.
147069 ** This function may leave an open sqlite3_blob* handle in the
147082 char **paBlob, /* OUT: Blob data in malloc'd buffer */
147188 ** Move the iterator passed as the first argument to the next term in the
147199 int nPrefix; /* Number of bytes in term prefix */
147200 int nSuffix; /* Number of bytes in term suffix */
147301 ** Set the SegReader to point to the first docid in the doclist associated
147327 ** Advance the SegReader to point to the next docid in the doclist
147332 ** in the doclist entry (i.e. immediately past the docid varint).
147340 int *pnOffsetList /* OUT: Length of *ppOffsetList in bytes */
147350 ** Pending-terms doclists are always built up in ascending order, so
147397 /* List may have been edited in place by fts3EvalNearTrim() */
147400 /* If there are no more entries in the doclist, set pOffsetList to
147509 /* The entire segment is stored in the root node. */
147546 ** a subset of the terms stored in the Fts3Table.pendingTerms array.
147549 ** through each term in the pending-terms table. Or, if isPrefixIter is
147552 ** "firebird", then the iterator visits the following 'terms' (in the order
147605 ** objects in term order using qsort(). This uses the same comparison
147613 /* The query is a simple term lookup that matches at most one term in
147682 ** A different comparison function for SegReader structures. In this
147683 ** version, it is assumed that each SegReader points to an entry in
147686 ** 1) EOF (end of doclist in this case) is greater than not EOF.
147728 int nTerm /* Size of term zTerm in bytes */
147746 ** the final (nSegment-nSuspect) members are already in sorted order
147748 ** the array around until all entries are in sorted order.
147787 int n /* Size of buffer z in bytes */
147801 ** Find the largest relative level number in the table. If successful, set
147831 sqlite3_int64 nLeafData, /* Bytes of leaf data in segment */
147833 int nRoot /* Number of bytes in buffer zRoot */
147858 ** zNext, in bytes. For example,
147866 int nPrev, /* Size of buffer zPrev in bytes */
147868 int nNext /* Size of buffer zNext in bytes */
147882 SegmentNode **ppTree, /* IN/OUT: SegmentNode handle */
147885 int nTerm /* Size of term in bytes */
147895 int nData = pTree->nData; /* Current size of node in bytes */
147922 /* There is no prefix-length field for first term in a node */
147953 ** If this is the first node in the tree, the term is added to it.
148020 int iHeight, /* Height of this node in tree */
148022 sqlite3_int64 iFree, /* Block id of next free slot in %_segments */
148025 int *pnRoot /* OUT: Size of root node in bytes */
148082 ** and return it via the input/output variable *ppWriter in this case.
148088 SegmentWriter **ppWriter, /* IN/OUT: SegmentWriter handle */
148091 int nTerm, /* Size of term in bytes */
148093 int nDoclist /* Size of doclist in bytes */
148095 int nPrefix; /* Size of term prefix in bytes */
148096 int nSuffix; /* Size of term suffix in bytes */
148111 /* Allocate a buffer in which to accumulate data */
148116 /* Find the next free blockid in the %_segments table */
148150 ** to the database (still available in pWriter->zTerm), and
148155 ** In other words, it must be the prefix of zTerm 1 byte longer than
148274 ** The first value in the apVal[] array is assumed to contain an integer.
148303 ** Set *pnMax to the largest segment level in the database for the index
148306 ** Segment levels are stored in the 'level' column of the %_segdir table.
148370 ** Delete all entries in the %_segments table associated with the segment
148398 ** each of the SegReader objects in the array passed as the third
148457 ** If there are no entries in the input position list for column iCol, then
148466 char **ppList, /* IN/OUT: Pointer to position list */
148467 int *pnList /* IN/OUT: Size of buffer *ppList in bytes */
148502 ** Cache data in the Fts3MultiSegReader.aBuffer[] buffer (overwriting any
148530 int *pnPoslist /* OUT: Size of position list in bytes */
148597 int nTerm /* Length of zTerm in bytes */
148639 int nTerm /* Number of bytes in zTerm */
148680 ** have been made. Calling this function puts the MultiSegReader in such
148686 ** then the entire doclist for the term is available in
148735 /* Advance the first pCsr->nAdvance entries in the apSegment[] array
148736 ** forward. Then sort the list in order of current term again.
148759 ** required callbacks have been made. In this case exit early.
148798 sqlite3_int64 iPrev = 0; /* Previous docid stored in doclist */
148800 /* The current term of the first nMerge entries in the array
148910 ** by one or more space (0x20) characters. In the first case, set *piEndBlock
148911 ** to the integer value and *pnByte to zero before returning. In the second,
148962 /* Loop through all entries in the %_segdir table corresponding to
148963 ** segments in this index on levels greater than iAbsLevel. If there is
148965 ** such segments are smaller than nLimit bytes in size, they will be
148975 ** old version of FTS. In this case it is not possible to determine
148999 /* Loop through all %_segdir entries for segments in this index with
149002 ** oldest segment in the range, 1 for the next oldest, and so on.
149004 ** In other words, move all segments being promoted to level -1,
149005 ** setting the "idx" fields as appropriate to keep them in the same
149039 ** Merge all level iLevel segments in the database into a single
149042 ** currently present in the database.
149045 ** segment in the database, SQLITE_DONE is returned immediately.
149052 int iIndex, /* Index in p->aIndex[] to merge */
149080 /* This call is to merge all segments in the database to a single
149082 ** greatest segment level currently present in the database for this
149215 ** Insert the sizes (in tokens) for each column of the document
149222 u32 *aSz /* Sizes of each column, in tokens */
149225 int nBlob; /* Number of bytes in the BLOB */
149250 ** where N is the number of user defined columns in the fts3 table plus
149254 ** Varint 0: Total number of rows in the table.
149256 ** Varint 1..nCol: For each column, the total number of tokens stored in
149259 ** Varint 1+nCol: The total size, in bytes, of all text values in all
149268 int nChng /* Change in the number of documents */
149456 ** the oldest nSeg segments (idx=0 through idx=(nSeg-1)) in absolute
149510 ** to build up nodes or other blobs of data in.
149516 int n; /* Number of valid bytes of data in a[] */
149538 int iIdx; /* Index of *output* segment in iAbsLevel+1 */
149563 int nDoclist; /* Size of doclist in bytes */
149569 ** bytes in size, extend (realloc) it to be so.
149641 ** Initialize a node-reader object to read the node in buffer aNode/nNode.
149670 ** The block id of the leaf node just written to disk may be found in
149703 ** bytes in size, write the key here. */
149752 ** stored in blob *pNode. The node need not contain any terms, but the
149762 ** If the size of the value in blob pPrev is zero, then this is the first
149778 int nTerm, /* Size of zTerm in bytes */
149780 int nDoclist /* Size of aDoclist in bytes */
149784 int nPrefix; /* Size of term prefix in bytes */
149785 int nSuffix; /* Size of term suffix in bytes */
149834 int nSpace; /* Total space in bytes required on leaf */
149858 ** to the database (still available in pLeaf->key), and
149863 ** In other words, it must be the prefix of zTerm 1 byte longer than
149912 int *pRc /* IN/OUT: Error code */
149915 int iRoot; /* Index of root in pWriter->aNodeWriter */
149919 /* Set iRoot to the index in pWriter->aNodeWriter[] of the output segment
149936 ** the node would be stored as a blob in the "root" column of the %_segdir
149937 ** table. However, this is not permitted in this case. The problem is that
149938 ** space has already been reserved in the %_segments table, and so the
149947 ** Todo: Better might be to defer allocating space in the %_segments
149992 ** Compare the term in buffer zLhs (size in bytes nLhs) with that in
149993 ** zRhs (size in bytes nRhs) using memcmp. If one term is a prefix of
150014 ** Query to see if the entry in the %_segments table with blockid iEnd is
150019 ** error code. The final value of *pbRes is undefined in this case.
150053 ** greater than the largest key currently stored in the potential
150061 int nKey, /* Number of bytes in nKey */
150073 int nRoot = 0; /* Size of aRoot[] in bytes */
150094 /* Check for the zero-length marker in the %_segments table */
150186 ** *piIdx is undefined in this case.
150211 ** In the %_segdir table, a segment is defined by the values in three
150221 ** of input segments, multiplied by two. This value is stored in stack
150230 ** In the actual code below, the value "16" is replaced with the
150258 /* Calculate the first block to use in the output segment */
150270 /* Insert the marker in the %_segments table to make sure nobody tries
150319 ** Update the 'idx' values of the remaining segments in the level so that
150328 int nIdx = 0; /* Valid entries in aIdx[] */
150393 ** This function creates a new node image in blob object *pNew by copying
150399 int nNode, /* Size of aNode in bytes */
150402 int nTerm, /* Size of zTerm in bytes */
150403 sqlite3_int64 *piBlock /* OUT: Block number in next layer down */
150444 ** Remove all terms smaller than zTerm/nTerm from segment iIdx in absolute
150446 ** table, and modifying existing entries in both the %_segments and %_segdir
150457 int nTerm /* Number of bytes in buffer zTerm */
150534 ** or modified in place so that it no longer contains those entries that
150535 ** have been duplicated in the output segment.
150552 ** somewhere in the pCsr->apSegment[] array. */
150568 ** segment to the upper level. The segment is modified in place
150586 ** Store an incr-merge hint in the database.
150605 ** exists, is stored in the rowid==1 row of the %_stat table.
150640 ** Otherwise, append an entry to the hint stored in blob *pHint. Each entry
150649 i64 iAbsLevel, /* First varint to store in hint */
150650 int nInput, /* Second varint to store in hint */
150651 int *pRc /* IN/OUT: Error code */
150665 ** If no error occurs, return SQLITE_OK. If the hint blob in *pHint does
150690 ** values for the _segdir.idx field) in the highest level that contains
150691 ** at least nMin segments. Multiple merges might occur in an attempt to
150717 int iIdx = 0; /* Largest idx in level (iAbsLevel+1) */
150736 ** last entry in it specifies a relative level smaller than or equal
150759 ** nMin segments and no hint in the %_stat table. No work to do.
150760 ** Exit early in this case. */
150766 ** segments available in level iAbsLevel. In this case, no work is
150935 int nTerm, /* Size of zTerm in bytes */
150955 ** Return a checksum of all entries in the FTS index that correspond to
150961 ** return value is undefined in this case.
151035 ** code. The final value of *pbOk is undefined in this case.
151088 int nToken = 0; /* Number of bytes in token */
151090 int iPos = 0; /* Position of token in zText */
151129 ** prefix in the document set, a 64-bit checksum is calculated (by code
151130 ** in fts3ChecksumEntry()) based on the following:
151135 ** + The language-id of the row it appears in,
151136 ** + The docid of the row it appears in,
151137 ** + The column it appears in, and
151140 ** The checksums for all entries in the index are XORed together to create
151143 ** The integrity-check code calculates the same checksum in two ways:
151218 ** Free all entries in the pCsr->pDeffered list. Entries are added to
151233 ** Generate deferred-doclists for all tokens in the pCsr->pDeferred list
151262 int nToken = 0; /* Number of bytes in token */
151264 int iPos = 0; /* Position of token in zText */
151326 int iCol /* Column that token must appear in (or -1) */
151348 ** present in the FTS3 table. If it is, delete it and adjust the contents
151354 int *pnChng, /* IN/OUT: Decrement if row is deleted */
151358 int bFound = 0; /* True if *pRowid really is in the table */
151366 /* Deleting this row means the whole table is empty. In this case
151368 ** data in the pendingTerms hash table. */
151411 int nChng = 0; /* Net change in number of documents */
151441 /* Allocate space to hold the change in document sizes */
151473 /* The new rowid is not NULL (in this case the rowid will be
151484 ** be losslessly converted to an integer. In this case, the eventual
151485 ** call to fts3InsertData() (either just below or further on in this
151542 ** Flush any data in the pending-terms hash table to disk. If successful,
151543 ** merge all segments in the database (including the new segment, if
151570 ** The author disclaims copyright to this source code. In place of
151587 ** Characters that may appear in the second argument to matchinfo().
151627 int nSnippet; /* Requested snippet length (in tokens) */
151628 int nPhrase; /* Number of phrases in query */
151634 int nToken; /* Number of tokens in phrase */
151636 int iHead; /* Next value in position list */
151638 int iTail; /* Next value in trailing position list */
151644 int iPos; /* Index of first token in snippet */
151656 int nCol; /* Number of columns in table */
151657 int nPhrase; /* Number of matchable phrases in query */
151658 sqlite3_int64 nDoc; /* Number of docs in database */
151665 ** (nElem * sizeof(u32)) bytes in size. See the MatchinfoBuffer code below
151685 int n; /* Length of z in bytes (excl. nul-term) */
151686 int nAlloc; /* Allocated size of buffer z in bytes */
151797 ** the list. *piPos contains the value of the previous entry in the list.
151833 ** Iterate through all phrase nodes in an FTS3 query, except those that
151871 ** Load the doclists for each phrase in the query associated with FTS3 cursor
151875 ** phrases in the expression (all phrases except those directly or
151877 ** pnToken is not NULL, then it is set to the number of tokens in all
151882 int *pnPhrase, /* OUT: Number of phrases in query */
151883 int *pnToken /* OUT: Number of tokens in query */
151990 u64 mHighlight = 0; /* Mask of tokens to highlight in snippet */
152064 ** (a) +1 point for each occurrence of a matchable phrase in the snippet.
152066 ** (b) +1000 points for the first occurrence of each matchable phrase in
152069 ** The selected snippet parameters are stored in structure *pFragment before
152070 ** returning. The score of the selected snippet is stored in *piScore
152078 u64 *pmSeen, /* IN/OUT: Mask of phrases seen */
152083 int nList; /* Number of phrases in expression */
152091 /* Iterate through the phrases in the expression to count them. The same
152110 ** the set of phrases in the expression to populate the aPhrase[] array.
152127 /* Loop through all candidate snippets. Store the best snippet in
152128 ** *pFragment. Store its associated 'score' in iBestScore.
152162 int nAppend /* Size of zAppend in bytes (or -1) */
152199 ** This function "shifts" the beginning of the snippet forward in the
152213 int iLangid, /* Language id to use in tokenizing */
152216 int nDoc, /* Size of buffer zDoc in bytes */
152217 int *piPos, /* IN/OUT: First token of snippet */
152218 u64 *pHlmask /* IN/OUT: Mask of tokens to highlight */
152231 /* Ideally, the start of the snippet should be pushed forward in the
152247 ** or more tokens in zDoc/nDoc.
152279 int isLast, /* True for final fragment in snippet */
152280 int nSnippet, /* Number of tokens in extracted snippet */
152289 int nDoc; /* Size of zDoc in bytes */
152318 int iBegin = 0; /* Offset in zDoc of start of token */
152319 int iFin = 0; /* Offset in zDoc of end of token */
152323 ** in the FTS code the variable that the third argument to xNext points to
152391 ** This function is used to count the entries in a column-list (a
152394 ** beginning of the first varint in the column-list (the varint that
152395 ** contains the position of the first matching term in the column data).
152397 ** the last varint in the column-list (either the 0x00 signifying the end
152399 ** the next column in the position-list).
152401 ** The number of elements in the column-list is returned.
152494 ** Y values are set to nDoc, where nDoc is the number of documents in the
152621 ** hits for a single phrase on a single row in order to calculate the
152675 ** array before returning. SQLITE_OK is returned in this case.
152688 ** contains one element for each matchable phrase in the query.
152703 int nLive = 0; /* Number of iterators in aIter not at EOF */
152763 ** pInfo->aMatchinfo[] buffer is left in is undefined.
152797 sqlite3_int64 nDoc; /* Number of rows in table */
152903 ** matchinfo function has been called for this query. In this case
152908 int nMatchinfo = 0; /* Number of u32 elements in match-info */
152911 /* Determine the number of phrases in the query */
152915 /* Determine the number of integers in the buffer returned by this call. */
152969 int nToken /* Approximate number of tokens in snippet */
152977 ** the data in the current row. The first iteration of the for(...) loop
152978 ** below attempts to locate a single fragment of text nToken tokens in
152979 ** size that contains at least one instance of all phrases in the query
152980 ** expression that appear in the current row. If such a fragment of text
152984 int nSnippet = 0; /* Number of fragments in this snippet */
152986 int nFToken = -1; /* Number of tokens in each fragment */
153021 /* Find the best snippet of nFToken tokens in column iRead. */
153035 /* If all query phrases seen by fts3BestSnippet() are present in at least
153083 int nTerm; /* Number of tokens in phrase */
153086 int iPos = 0; /* First position in position-list */
153117 int nToken; /* Number of tokens in query */
153130 /* Count the number of terms in the query */
153164 /* Retreive the text stored in column iCol. If an SQL NULL is stored
153165 ** in column iCol, jump immediately to the next iteration of the loop.
153279 ** The author disclaims copyright to this source code. In place of
153368 int nInput; /* Size of aInput[] in bytes */
153391 ** all characters in string zIn/nIn to be separators (if bAlnum==0) or
153394 ** For each codepoint in the zIn/nIn string, this function checks if the
153399 ** codepoints in the aiException[] array.
153402 ** identifies as a diacritic) occurs in the zIn/nIn string it is ignored.
153410 int nIn /* Length of z in bytes */
153431 int nNew; /* Number of valid entries in array aNew[] */
153540 ** used to incrementally tokenize this string is returned in
153546 int nInput, /* Size of string aInput in bytes */
153675 ** The author disclaims copyright to this source code. In place of
153702 /* Each unsigned integer in the following array corresponds to a contiguous
153706 ** The most significant 22 bits in each 32-bit value contain the first
153707 ** codepoint in the range. The least significant 10 bits are used to store
153708 ** the size of the range (always at least 1). In other words, the value
153827 ** in the ASCII range with a diacritic added, return the codepoint
153903 /* Each entry in the following array defines a rule for folding a range
153907 ** If the least significant bit in flags is clear, then the rule applies
153910 ** every second codepoint in the range, starting with codepoint C.
153912 ** The 7 most significant bits in flags are an index into the aiOff[]
154043 ** The author disclaims copyright to this source code. In place of
154059 ** The data structure for a single virtual r-tree table is stored in three
154060 ** native SQLite tables declared as follows. In each case, the '%' character
154061 ** in the table name is replaced with the user-supplied name of the r-tree
154068 ** The data for each node of the r-tree structure is stored in the %_node
154070 ** an entry in the %_parent table associating the node with its parent.
154071 ** And for each row of data in the table, there is an entry in the %_rowid
154076 ** empty. The nodeno of the root node is always 1. All other nodes in the
154085 ** stored in the node.
154141 ** the number of rows in the virtual table to calculate the costs of
154156 int iNodeSize; /* Size in bytes of each node in the node table */
154164 i64 nRowEst; /* Estimated number of rows in this table */
154189 RtreeNode *aHash[HASHSIZE]; /* Hash table of in-memory nodes. */
154218 ** represents is the iCell-th entry in the parent node.
154230 ** maximum. In Gutman's notation:
154252 ** Number of entries in the cursor RtreeNode cache. The first entry is
154266 int nConstraint; /* Number of entries in aConstraint */
154269 int nPoint; /* Number of slots used in aPoint[] */
154339 RtreeNode *pNext; /* Next node in this hash collision chain */
154342 /* Return the number of cells in a node */
154358 ** operators in order to constrain a search.
154365 ** This object is deleted automatically by the destructor mechanism in
154384 ** An instance of this structure (in the form of a BLOB) is returned by
154484 ** in the Rtree.aHash table.
154557 /* Check if the requested node is already in the hash table. If so,
154666 ** If there is not enough free space in pNode, return SQLITE_FULL.
154673 int nCell; /* Current number of cells in pNode */
154754 ** Return coordinate iCoord from cell iCell in node pNode.
154951 ** Store the results in "r".
155045 ** in a coordinate pair. But make pCellData point to the lower bound.
155077 ** The op is given by p->op. The xN is p->iCoord-th coordinate in
155103 ** One of the cells in node pNode is guaranteed to have a 64-bit
155126 ** in its parent. If pNode is the root node, return -1.
155143 ** iLevel values coming first. In this way, if rScore is the same for all
155159 ** Interchange to search points in a cursor.
155484 ** return SQLITE_OK. If there is no such record in the table, set
155574 /* Reset the cursor to the same state as rtreeOpen() leaves it in. */
155664 ** support estimatedRows. In that case this function is a no-op.
155676 ** table scan strategies to choose from (in order from most to
155688 ** the constraint in sqlite3_index_info.aConstraintUsage[] with
155691 ** The first of each pair of bytes in idxStr identifies the constraint
155791 ** Return the N-dimensional volumn of the cell stored in *p.
155804 ** of the objects size in each dimension.
155816 ** Store the union of cells p1 and p2 in p1.
155894 ** ChooseSubTree in r*tree terminology.
155953 ** the node pNode. This function updates the bounding box cells in
156008 ** (nIdx-1) in no particular order. This function sorts the values
156009 ** in aIdx according to the indexed values in aDistance. For
156080 ** (nIdx-1) in no particular order. This function sorts the values
156081 ** in aIdx according to dimension iDim of the cells in aCell. The
156330 ** In this case avoid the all to nodeWrite().
156451 /* Remove the entry in the parent cell. */
156480 /* Remove the node from the in-memory hash table and link it into
156528 ** the in-memory node image, so it cannot fail.
156534 ** cell in the parent node so that it tightly contains the updated
156631 /* Find a node to store this cell in. pNode->iNode currently contains
156700 /* Find a node to store this cell in. pNode->iNode currently contains
156735 int iCell; /* Index of iDelete cell in pLeaf */
156749 /* Delete the cell in question from the leaf node. */
156762 /* Delete the corresponding entry in the <rtree>_rowid table. */
156775 ** in this scenario).
156868 ** In the first case, if the conflict-handling mode is REPLACE, then
156869 ** the conflicting row can be removed before proceeding. In the second
156909 /* If a rowid value was supplied, check if it is already present in
156997 ** of the number of rows in the virtual table. If possible, this is based
157161 ** table already exists. In this case the node-size is determined by inspecting
157167 ** would fit in a single node, use a smaller node-size.
157316 ** entry for each cell in the r-tree node. Each entry is itself a
157418 ** or sqlite3_rtree_query_callback(). In other words, this routine is the
157558 ** The author disclaims copyright to this source code. In place of
157603 ** Maximum length (in bytes) of the pattern in a LIKE or GLOB
157630 int iPattern = 0; /* Current byte index in zPattern */
157631 int iString = 0; /* Current byte index in zString */
157653 ** MATCH_ALL. For each MATCH_ONE, skip one character in the
157701 ** the build-in LIKE operator. The first argument to the function is the
157722 ** of deep recursion and N*N behavior in patternCompare().
158065 ** The author disclaims copyright to this source code. In place of
158100 int nChar; /* Number of UChar elements in pInput */
158102 int *aOffset; /* Offsets of each character in utf-8 input */
158114 int argc, /* Number of entries in argv[] */
158152 ** used to incrementally tokenize this string is returned in
158158 int nInput, /* Length of zInput in bytes */
158241 int *pnBytes, /* OUT: Number of bytes in token */
158330 ** The author disclaims copyright to this source code. In place of
158343 ** SQLite database. The tables it expects to find are described in
158344 ** sqlite3rbu.h. Essentially, for each table xyz in the target database
158346 ** created in the RBU database and populated with one row for each row to
158349 ** The update proceeds in three stages:
158355 ** "<database>-oal", they go on using the original database in
158363 ** updates are done in sorted key order.
158371 ** detect the new *-wal file and read the database in wal mode. At
158375 ** 3) The new *-wal file is checkpointed. This proceeds in the same way
158379 ** the checkpoint progress is saved in the RBU database and the
158380 ** checkpoint can be resumed by another RBU client at some point in
158392 ** pass unnoticed, or result in SQLITE_CONSTRAINT errors (due to UNIQUE
158402 ** collected in the rbu_xyz table. And for both UPDATEs and DELETEs all
158416 /************** Include sqlite3rbu.h in the middle of sqlite3rbu.c ***********/
158421 ** The author disclaims copyright to this source code. In place of
158440 ** The problem is that in order to update a b-tree, the leaf page (at least)
158445 ** Additionally, because the index updates are likely to be applied in
158446 ** random order, access to pages within the database is also likely to be in
158457 ** Additionally, this extension allows the work involved in writing the
158464 ** to read from the original database snapshot. In other words, partially
158467 ** "RBU" stands for "Resumable Bulk Update". As in a large database update
158482 ** non-NULL PRIMARY KEY values. Rows with NULL values stored in PRIMARY
158500 ** containing an RBU update is an "RBU database". For each table in the
158505 ** the same type as the corresponding column in the target database.
158515 ** The order of the columns in the data_% table does not matter.
158523 ** tables within the RBU database are always processed in order sorted by
158525 ** of the RBU tables the user can therefore control the order in which they
158544 ** target table must be present in the input table. For virtual tables,
158545 ** hidden columns are optional - they are updated by RBU if present in
158564 ** do so results in an SQLITE_MISMATCH error.
158569 ** real primary key values of the row to delete should be stored in the
158570 ** corresponding columns of the data_% table. The values stored in the
158577 ** stored in the corresponding columns of the data_% table row, as should
158578 ** the new values of all columns being update. The text value in the
158580 ** there are columns in the target database table, and must consist entirely
158581 ** of 'x' and '.' characters (or in some special cases 'd' - see below). For
158594 ** for UPDATEs may also be set to 'd'. In this case, instead of updating the
158595 ** target table with the value stored in the corresponding data_% column, the
158596 ** user-defined SQL function "rbu_delta()" is invoked and the result stored in
158598 ** original value currently stored in the target table column and the
158599 ** value specified in the data_xxx table.
158609 ** Finally, if an 'f' character appears in place of a 'd' or 's' in an
158611 ** to be a "fossil delta" - a patch to be applied to a blob value in the
158612 ** format used by the fossil source-code management system. In this case
158629 ** However, RBU is more efficient if reading the rows in from each data_xxx
158630 ** table in "rowid" order is roughly the same as reading them sorted by
158631 ** the PRIMARY KEY of the corresponding target database table. In other
158656 ** update application is saved in the RBU database for later
158672 ** An RBU update may not be applied to a database in WAL mode. Attempting
158710 ** Or, if zState is non-NULL, it must be a path to a database file in which
158724 ** SQLite's built-in VFSs, including the multiplexor VFS. However it does
158742 ** Accessing the underlying database handles may be useful in the
158749 ** * If the data_xxx tables in the RBU source database are virtual
158783 ** was last saved. In other words, from the most recent successful call to
158827 ** The exception is for applications that also use zipvfs. In this case,
158830 ** VFS uses the RBU VFS, which in turn uses any other VFS layers in use
158852 ** that does not include the RBU layer results in an error.
158855 ** non-RBU users. There is no harm in an application accessing the
158878 /************** Continuing where we left off in sqlite3rbu.c *****************/
158927 ** cases when another client appends frames to the *-wal file in the
158932 ** Valid if STAGE==1. The current change-counter cookie value in the
158936 ** Valid if STAGE==1. The size in bytes of the *-oal file.
158971 ** These values must match the values defined in wal.c for the equivalent
158980 ** A structure to store values read from the rbu_state table in memory.
159000 ** An iterator of this type is used to iterate through all objects in
159002 ** iterator visits, in order:
159010 ** it points to an array of flags nTblCol elements in size. The flag is
159028 int bCleanup; /* True in "cleanup" state */
159037 int nCol; /* Number of columns in current object */
159073 #define RBU_UPDATE 5 /* Update a row in a main table b-tree */
159108 u32 iMaxFrame; /* Largest iWalFrame value in aFrame[] */
159110 int nFrame; /* Entries in aFrame[] array */
159142 int nShm; /* Number of entries in apShm[] array */
159167 ** in *pz and is decremented once for each character in the integer.
159237 ** binary files) so we also have to pass in the length of the delta in
159240 ** This function returns the size of the output file in bytes (excluding
159399 ** Prepare the SQL statement in buffer zSql against database handle db.
159446 ** returned. In this case, *pzErrmsg may also be set to point to an error
159451 ** In this case SQLITE_NOMEM is returned and *ppStmt set to NULL.
159535 ** left in the RBU handle passed as the first argument. A copy of the
159599 ** accepts one argument - the name of a table in the RBU database. If the
159635 ** left in the RBU handle passed as the first argument. A copy of the
159644 "WHERE type IN ('table', 'view') AND target IS NOT NULL "
159663 ** an error code is stored in the RBU handle passed as the first argument.
159667 ** stored error code. In this case it still calls sqlite3_free() on any
159691 ** If an error occurs, an error code and error message is stored in the
159717 ** error code in the rbu handle passed as the first argument. Or, if an
159740 ** error code in the RBU handle passed as the first argument.
159759 ** returns a copy of the string in memory obtained from sqlite3_malloc().
159822 ** if( no entry exists in sqlite_master ){
159827 ** if( the index that is the pk exists in sqlite_master ){
159960 ** an error does occur, an error code and error message are also left in
160021 /* Check that all non-HIDDEN columns in the destination table are also
160022 ** present in the input table. Populate the abTblPk[], azTblType[] and
160068 ** column names currently stored in the pIter->azTblCol[] array.
160243 ** Return an expression that can be used in a WHERE clause to match the
160289 ** is something wrong with the rbu_control value in the rbu_control value
160290 ** stored in the (p->nCol+1)'th column. Set the error code and error message
160311 ** string, an error code is left in the rbu handle passed as the first
160365 ** string, an error code is left in the rbu handle passed as the first
160514 ** in the sqlite3rbu object indicated by the first argument and returns
160523 ** An imposter table is required in all cases except RBU_PK_VTAB. Only
160581 ** The number of bound variables is equal to the number of columns in
160689 /* Create the SELECT statement to read keys in sorted order */
160832 ** is not an error. Output variable *ppStmt is set to NULL in this case.
160844 /* In case an error occurs */
160849 ** to the number of statements currently in the cache and pUp to the
160850 ** last object in the list. */
160918 ** error occurs, leave an error code and message in the RBU handle.
160961 ** this call returns SQLITE_NOTFOUND, then the RBU vfs is not in use.
160978 ** filename in zBaseFilename is a URI with the "8_3_names=1" parameter and
160979 ** if filename in z[] has a suffix (a.k.a. "extension") that is longer than
161011 ** The checksum is store in the first page of xShmMap memory as an 8-byte
161037 ** being resumed. In this case, if the checksum of the wal-index-header
161038 ** following recovery is not the same as the checksum saved in the RbuState
161040 ** other client appends a transaction to the wal file in the middle of
161057 ** special behaviour in the rbu VFS:
161065 ** would be read/written are recorded in the sqlite3rbu.aFrame[]
161105 ** the rbu object is in capture mode. Record the frame number of the frame
161106 ** being read in the aFrame[] array.
161137 ** file while the rbu handle is in capture mode. Record the page number
161138 ** of the page being written in the aFrame[] array.
161202 ** The RBU handle is currently in RBU_STAGE_OAL state, with a SHARED lock
161204 ** then reopens the database file (this time in vanilla, non-oal, WAL mode).
161205 ** If an error occurs, leave an error code and error message in the rbu
161221 ** in WAL mode). So no other connection may be writing the db.
161223 ** In order to ensure that there are no database readers, an EXCLUSIVE
161291 ** message are left in the RBU handle and zero returned.
161354 ** and message is left in the RBU handle and a copy of the error code
161474 ** statement reads is page 1, which is guaranteed to be in the cache.
161489 ** value stored in the RBU_STATE_STAGE column is eStage. All other values
161649 ** If an error occurs, leave an error code and message in the rbu handle
161729 ** an rbu handle in OAL stage. If the rbu update has not started (i.e.
161734 ** If an error occurs, an error code and error message are left in the
161765 ** If there is a "*-oal" file in the file-system corresponding to the
161766 ** target database in the file-system, delete it. If an error occurs,
161767 ** leave an error code and error message in the rbu handle.
161782 ** specified a URI with a vfs=? option in place of a target database
161881 ** transaction is committed in rollback mode) currently stored on
161954 ** If the error code currently stored in the RBU handle is SQLITE_CONSTRAINT,
162049 ** of a standard VFS in the following ways:
162052 ** written, the value of the change-counter cookie is stored in
162054 ** database header field is stored in rbu_file.iWriteVer. This ensures
162068 ** associated with an RBU target database currently in RBU_STAGE_OAL
162073 ** target database may not be in wal mode already.
162079 ** the RBU target in RBU_STAGE_OAL stage, instead of opening the *-wal
162082 ** 3b. The *-shm pages returned by xShmMap() for a target db file in
162083 ** RBU_STAGE_OAL mode are actually stored in heap memory. This is to
162085 ** are no-ops on target database files in RBU_STAGE_OAL mode. This is
162086 ** because assert() statements in some VFS implementations fail if
162089 ** 3c. If an EXCLUSIVE lock is attempted on a target database file in any
162095 ** 3d. In RBU_STAGE_CAPTURE mode, all xRead() calls on the wal file, and
162314 /* First try to find another RBU vfs lower down in the vfs stack. If
162315 ** one is found, this vfs will operate in pass-through mode. The lower
162320 /* Now search for a zipvfs instance lower down in the VFS stack. If
162350 ** Return the sector-size in bytes for an rbuVfs-file.
162418 /* If not in RBU_STAGE_OAL, allow this call to pass through. Or, if this
162419 ** rbu is in the RBU_STAGE_OAL state, use heap memory for *-shm space
162570 ** pair of '\0' bytes in case the VFS attempts to extract a URI
162636 ** database connection exists, and the RBU update is in RBU_STAGE_OAL,
162641 ** in wal mode, even if the first page of the database file has
162665 ** to the pathname in zPath. zOut is guaranteed to point to a buffer
162698 ** Return a pointer to the symbol zSymbol in the dynamic library pHandle.
162737 ** Return the current time as a Julian Day number in *pTimeOut.
162854 ** The author disclaims copyright to this source code. In place of
162866 ** information from an SQLite database in order to implement the
162895 ** from. For example, the three overflow pages in a chain linked from
162899 ** '/1c2/000+000000' // First page in overflow chain
162900 ** '/1c2/000+000001' // Second page in overflow chain
162901 ** '/1c2/000+000002' // Third page in overflow chain
162904 ** the overflow pages associated with a cell will appear earlier in the
162919 " pgoffset INTEGER, /* Offset of page in file */" \
162933 int nOvfl; /* Entries in aOvfl[] */
162962 int iPage; /* Current entry in aPage[] */
163064 /* Records are always returned in ascending order of (name, path).
163282 ** Move a statvfs cursor to the next entry in the file.
163557 ** The author disclaims copyright to this source code. In place of
163572 ** a JSONB type in the future which stores a binary encoding of JSON in
163573 ** a BLOB, but there is no support for JSONB in the current implementation.
163612 ** the library isspace() function, resulting in a 7% overall performance
163636 /* Unsigned integer types. These are already defined in the sqliteInt.h,
163655 u64 nAlloc; /* Bytes of storage available in zBuf[] */
163703 u32 iKey; /* Key for ARRAY objects in json_tree() */
163822 /* Append the N-byte string in zIn to the end of the JsonString string
163823 ** under construction. Enclose the string in "..." and escape
163885 /* Make the JSON in p the result of the SQL function.
163907 ** by which the JsonNode counter should increment in order to go to the
164209 ** instance to the JsonParse. Return the index in pParse->aNode[] of the
164410 ** to fill in all the descendants of node i.
164437 ** Compute the parentage of all nodes in a completed parse.
164482 const char **pzErr /* Make *pzErr point to any syntax error in zPath */
164742 ** array that contains all values given in arguments. Or if any argument
164769 ** Return the number of elements in the top-level JSON array.
164851 ** object that contains all name/value given in arguments. Or if any name
164894 ** JSON or PATH arguments result in an error.
165079 u32 i; /* Index in sParse.aNode[] of current row */
165534 ** The following routines are the only publically visible identifiers in this
165535 ** file. Call the following routines in order to register the various SQL
165624 ** The author disclaims copyright to this source code. In place of
165633 ** Interfaces to extend FTS5. Using the interfaces defined in this file,
165665 int nVal, /* Number of values in apVal[] array */
165683 ** to the total number of tokens in the FTS5 table. Or, if iCol is
165684 ** non-negative but less than the number of columns in the table, return
165685 ** the total number of tokens in column iCol, considering all rows in
165689 ** in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g.
165694 ** Return the number of columns in the table.
165698 ** to the total number of tokens in the current row. Or, if iCol is
165699 ** non-negative but less than the number of columns in the table, set
165700 ** *pnToken to the number of tokens in column iCol of the current row.
165703 ** in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g.
165710 ** containing the text in utf-8 encoding, (*pn) is set to the size in bytes
165716 ** Returns the number of phrases in the current query expression.
165719 ** Returns the number of tokens in phrase iPhrase of the query. Phrases
165797 ** (set to NULL) before this function returns. In this case the xDelete,
165803 ** This function is used to retrieve the total number of rows in the table.
165804 ** In other words, the same value that would be returned by:
165895 ** be returned. In this case, fts5 assumes that the final value of *ppOut
165933 ** For each token in the input string, the supplied callback xToken() must
165937 ** size of the token in bytes. The 4th and 5th arguments are the byte offsets
165943 ** synonyms. In this case see the discussion below for details.
165945 ** FTS5 assumes the xToken() callback is invoked for each token in the
165958 ** Custom tokenizers may also support synonyms. Consider a case in which a
165960 ** built-in tokenizers, the FTS5 query 'first + place' will match instances
165962 ** such as "1st place". In some applications, it would be better to match
165964 ** the user specified in the MATCH query text.
165966 ** There are several ways to approach this in FTS5:
165968 ** <ol><li> By mapping all synonyms to a single token. In this case, the
165969 ** In the above example, this means that the tokenizer returns the
165970 ** same token for inputs "first" and "1st". Say that token is in
165978 ** In this case, when tokenizing query text, the tokenizer may
165987 ** first token in the MATCH query and FTS5 effectively runs a query
166007 ** 'first + place' or '1st + place', as there are entires in the
166027 ** by making multiple calls to xToken(FTS5_TOKEN_COLOCATED) in sequence.
166031 ** In many cases, method (1) above is the best approach. It does not add
166033 ** so it is efficient in terms of disk space and query speed. However, it
166043 ** For full prefix support, method (3) may be preferred. In this case,
166076 int nToken, /* Size of token in bytes */
166145 ** The author disclaims copyright to this source code. In place of
166193 ** directive in fts5_index.c will cause the build to fail.
166228 ** Each entry in the aiCol[] array is a column that may be matched.
166240 ** Interface to code in fts5_config.c. fts5_config.c contains contains code
166267 ** This exists in order to allow the fts5_index.c module to return a
166289 int *aPrefix; /* Sizes in bytes of nPrefix prefix indexes */
166300 int pgsz; /* Approximate page size used in %_data */
166350 ** End of interface to code in fts5_config.c.
166354 ** Interface to code in fts5_buffer.c.
166398 int n; /* Size of buffer at a[] in bytes */
166399 int i; /* Current offset in a[] */
166421 int *pi, /* IN/OUT: Offset within a[] */
166422 i64 *piOff /* IN/OUT: Current offset */
166433 ** End of interface to code in fts5_buffer.c.
166437 ** Interface to code in fts5_index.c. fts5_index.c contains contains code
166438 ** to access the data stored in the %_data table.
166448 #define FTS5INDEX_QUERY_DESC 0x0002 /* Docs in descending rowid order */
166508 ** For an insert, it must be called once for each token in the new document.
166510 ** unique token in the document with an iCol value less than zero. The iPos
166515 int iCol, /* Column token appears in (-ve -> delete) */
166531 ** Flush any data stored in the in-memory hash tables to the database.
166537 ** Discard any data stored in the in-memory hash tables. Do not write it
166539 ** table may have changed on disk. So any in-memory caches of %_data
166577 ** End of interface to code in fts5_index.c.
166581 ** Interface to code in fts5_varint.c.
166601 ** End of interface to code in fts5_varint.c.
166606 ** Interface to code in fts5.c.
166621 ** End of interface to code in fts5.c.
166625 ** Interface to code in fts5_hash.c.
166638 int iCol, /* Column token appears in (-ve -> delete) */
166653 int *pnDoclist /* OUT: Size of doclist in bytes */
166665 int *pnDoclist /* OUT: size of doclist in bytes */
166670 ** End of interface to code in fts5_hash.c.
166674 ** Interface to code in fts5_storage.c. fts5_storage.c contains contains
166675 ** code to access the data stored in the %_content and %_docsize tables.
166719 ** End of interface to code in fts5_storage.c.
166724 ** Interface to code in fts5_expr.c.
166735 int n; /* Size of buffer p in bytes */
166773 ** C code in this module. The interfaces below this point are called by
166774 ** the parser code in fts5parse.y. */
166815 ** End of interface to code in fts5_expr.c.
166821 ** Interface to code in fts5_aux.c.
166826 ** End of interface to code in fts5_aux.c.
166830 ** Interface to code in fts5_tokenizer.c.
166835 ** End of interface to code in fts5_tokenizer.c.
166839 ** Interface to code in fts5_vocab.c.
166845 ** End of interface to code in fts5_vocab.c.
166850 ** Interface to automatically generated code in fts5_unicode2.c.
166856 ** End of interface to code in fts5_unicode2.c.
166880 ** macros to disable tests that are needed in the case of a general
166881 ** LALR(1) grammar but which are always false in the
166885 ** in the input grammar file. */
166890 ** Disable all error recovery processing in the parser push-down
166900 /* Next is all token values, in a form suitable for use by makeheaders.
166908 ** Each symbol here is a terminal symbol in the grammar.
166923 ** number is used to fill in empty slots of the hash
166936 ** which is sqlite3Fts5ParserFTS5TOKENTYPE. The entry in the union
166947 ** fts5YYNRULE the number of rules in the grammar
166994 ** Applications can choose to define fts5yytestcase() in the %include section
166995 ** to a macro that can assist in verifying code coverage. For production
167026 ** slots in the fts5yy_action[] table.
167035 ** is equal to fts5YY_SHIFT_USE_DFLT, it means that the action is not in the table
167040 ** a reduce action) then the fts5yy_reduce_ofst[] array is used in place of
167041 ** the fts5yy_shift_ofst[] array and fts5YY_REDUCE_USE_DFLT is used in place of
167044 ** The following are the tables generated in this section:
167047 ** fts5yy_lookahead[] A table containing the lookahead for each entry in
167104 ** appears in the grammar, then ID becomes a fallback token for X, Y,
167120 ** (In other words, the "major" token.)
167123 ** the information used by the action routines in the grammar.
167131 fts5YYACTIONTYPE stateno; /* The state-number, or reduce action in SHIFTREDUCE */
167139 /* The state of the parser is completely contained in an instance of
167142 int fts5yyidx; /* Index of top element in stack */
167267 ** A pointer to a parser. This pointer is used in subsequent calls
167355 ** empty in SQLite. */
167387 /* In SQLite, we never try to destroy a parser that was not successfully
167388 ** created in the first place. */
167553 int fts5yyNewState, /* The new state to shift in */
167554 int fts5yyMajor, /* The major token to shift in */
167555 fts5YYMINORTYPE *fts5yypMinor /* Pointer to the minor token to shift in */
167590 unsigned char nrhs; /* Number of right-hand side symbols in the rule */
167645 ** in some cases when it is copied into the stack after the following
167655 ** from wireshark this week. Clearly they are stressing Lemon in ways
167783 ** onto the stack here, and skip the stack overflow test in fts5yy_shift().
167861 ** user wants (and specified in the grammar) and is available for
168038 ** The author disclaims copyright to this source code. In place of
168052 ** Object used to iterate through all "coalesced phrase instances" in
168053 ** a single column of the current row. If the phrase instances in the
168055 ** through them. Or, if they do overlap (share one or more tokens in
168079 int iStart; /* First token in coalesced phrase instance */
168080 int iEnd; /* Last token in coalesced phrase instance */
168116 ** iterate through coalesced phrase instances in column iCol.
168147 int iPos; /* Current token offset in zIn[] */
168153 int nIn; /* Size of input text in bytes */
168186 int nToken, /* Size of token in bytes */
168238 int nVal, /* Number of values in apVal[] array */
168287 int nVal, /* Number of values in apVal[] array */
168297 int nPhrase; /* Number of phrases in query */
168303 int nColSize = 0; /* Total size of iBestCol in tokens */
168411 int nPhrase; /* Number of phrases in query */
168412 double avgdl; /* Average number of tokens in each row */
168418 ** Callback used by fts5Bm25GetData() to count the number of rows in the
168446 int nPhrase; /* Number of phrases in query */
168447 sqlite3_int64 nRow = 0; /* Number of rows in table */
168448 sqlite3_int64 nToken = 0; /* Number of tokens in table */
168470 /* Calculate an IDF for each phrase in the query */
168480 ** where "N" is the total number of documents in the set and nHit
168486 ** (1e-6) - roughly the same as a term that appears in just over
168512 int nVal, /* Number of values in apVal[] array */
168522 double D = 0.0; /* Total number of tokens in row */
168525 /* Calculate the phrase frequency (symbol "f(qi,D)" in the documentation)
168526 ** for each phrase in the query for the current row. */
168542 /* Figure out the total size of the current row in tokens. */
168597 ** The author disclaims copyright to this source code. In place of
168636 ** pBuf. If an OOM error occurs, set the error code in p.
168662 ** the error code in p. If an error has already occurred when this function
168680 ** though this byte is not included in the pBuf->n count.
168698 ** included in the pBuf->n count.
168754 ** the error code in p. If an error has already occurred when this function
168769 int *pi, /* IN/OUT: Offset within a[] */
168770 i64 *piOff /* IN/OUT: Current offset */
168849 ** is non-negative, then it is the length of the string in bytes. Otherwise,
168903 ** The author disclaims copyright to this source code. In place of
168935 ** string. Return a pointer to the first character following *pIn in
168948 ** string. Return a pointer to the first character following *pIn in
169032 ** the string and, if found, dequotes the string in place and adds a new
169037 ** found, -1 is returned. If -1 is returned, the buffer is left in an
169074 ** the quote characters. The conversion is done in-place. If the
169101 ** may be left in *pzErr. It is the responsibility of the caller to
169175 *pzErr = sqlite3_mprintf("parse error in tokenize directive");
169244 ** Return a pointer to the character immediately following the last in
169258 int *pRc, /* IN/OUT: Error code */
169352 ** returned, *ppOut is set to NULL and an error message may be left in
169413 *pzErr = sqlite3_mprintf("parse error in \"%s\"", zOrig);
169436 /* If no zContent option was specified, fill in the default values. */
169527 ** The callback is invoked once for each token in the input text. The
169528 ** arguments passed to it are, in order:
169532 ** int nToken // Size of token in bytes
169535 ** int iPos // Position of token in input (first token is 0)
169561 ** Argument pIn points to the first character in what is expected to be
169590 ** + Zero or more SQL literals in a comma separated list
169771 ** The author disclaims copyright to this source code. In place of
169786 ** All token types in the generated fts5parse.h file are greater than 0.
169809 int bDesc; /* Iterate in descending rowid order */
169810 int nPhrase; /* Number of phrases in expression */
169848 Fts5ExprTerm *pSynonym; /* Pointer to first in list of synonyms */
169852 ** A phrase. One or more terms that must appear in a contiguous sequence
169858 int nTerm; /* Number of entries in aTerm[] */
169869 int nPhrase; /* Number of entries in aPhrase[] array */
169905 const char **pz, /* IN/OUT: Pointer into buffer */
170149 ** All individual term iterators in pPhrase are guaranteed to be valid and
170182 /* Initialize a term iterator for each term in the phrase */
170241 int n; /* Size of buffer a[] in bytes */
170242 int i; /* Current offset in position list */
170289 ** If in/out parameter (*pRc) is set to other than SQLITE_OK when this
170292 ** The return value is undefined in both these cases.
170389 ** Advance the first term iterator in the first phrase of pNear. Set output
170465 i64 *piLast, /* IN/OUT: Lastest rowid seen so far */
170491 i64 *piLast, /* IN/OUT: Lastest rowid seen so far */
170527 /* Check that each phrase in the nearset matches the current row.
170578 ** All individual term iterators in pNear are guaranteed to be valid when
170607 ** iterator skips through rowids in the default ascending order, this means
170650 ** Initialize all term iterators in the pNear object. If any term is found
170815 ** Nodes that point to values that come later in the iteration order are
171006 /* Initialize all term iterators in the NEAR object. */
171024 ** Begin iterating through the set of documents in index pIdx matched by
171026 ** parameter is passed a non-zero value, iteration is in descending rowid
171027 ** order. Or, if it is zero, in ascending order.
171029 ** If iterating in ascending rowid order (bDesc==0), the first document
171031 ** to parameter iFirst. Or, if iterating in ascending order (bDesc==1),
171046 /* If not at EOF but the current rowid occurs earlier than iFirst in
171180 int nToken, /* Size of token in bytes */
171264 ** string may or may not be quoted. In any case it is tokenized and a
171393 ** Token pTok has appeared in a MATCH expression where the NEAR operator
171395 ** in the pParse object.
171435 ** If an OOM error occurs, store an error code in pParse and return NULL.
171436 ** The old colset object (if any) is not freed in this case.
171443 int nCol = p ? p->nCol : 0; /* Num. columns already in colset object */
171466 /* Check that the array is in order and contains no duplicate entries. */
171531 ** OOM error), leave an error code in pParse and return NULL.
171971 ** Return the number of phrases in expression pExpr.
171978 ** Return the number of terms in the iPhrase'th phrase in pExpr.
172006 ** The author disclaims copyright to this source code. In place of
172023 ** This file contains the implementation of an in-memory hash table used
172031 int nEntry; /* Number of entries currently in hash */
172038 ** Each entry in the hash table is represented by an object of the
172040 ** are stored in a single memory allocation. The position list data
172041 ** immediately follows the key data in memory.
172043 ** The data that follows the key is in a similar, but not identical format
172044 ** to the doclist data stored in the database. It is:
172060 Fts5HashEntry *pScanNext; /* Next entry in sorted order */
172189 int nSz = (p->nData - p->iSzPoslist - 1); /* Size in bytes */
172209 int iCol, /* Column token appears in (-ve -> delete) */
172321 ** each sorted in key order. This function merges the two lists into a
172364 ** in sorted order. The hash table is cleared before returning. It is
172417 int *pnDoclist /* OUT: Size of doclist in bytes */
172458 int *pnDoclist /* OUT: size of doclist in bytes */
172478 ** The author disclaims copyright to this source code. In place of
172487 ** Low level access to the FTS index stored in the database file. The
172488 ** routines in this file file implement all read and write access to the
172490 ** the interface defined in fts5Int.h.
172502 ** * all segment b-tree leaf data is stored in fixed size page records
172504 ** taken to ensure it is possible to iterate in either direction through
172505 ** the entries in a doclist, or to seek to a specific entry within a
172513 ** * extra fields in the "structure record" record the state of ongoing
172520 #define FTS5_WORK_UNIT 64 /* Number of leaf pages in unit of work */
172544 ** recorded in a single record within the %_data table. The record consists
172559 ** + number of input segments in ongoing merge.
172560 ** + total number of segments in level.
172569 ** The first value is the number of rows in the index. Then, for each column
172570 ** from left to right, the total number of tokens in the column for all
172585 ** varint: number of bytes in common with previous term
172602 ** varint: size of poslist in bytes multiplied by 2, not including
172621 ** unsigned integer fields in big-endian format. They are:
172627 ** footer is 0 bytes in size, then this field is the same as the
172628 ** size of the leaf page in bytes.
172632 ** within the page, delta-compressed against the previous value. In
172633 ** other words, the first varint in the footer is the byte offset of
172639 ** the format is changed in two ways:
172646 ** + the first term on each page is stored in the same way as the
172684 ** Rowids for the averages and structure records in the %_data table.
172691 ** and nodes. All nodes and leaves are stored in the %_data table with large
172716 ** Maximum segments permitted in a single index
172747 int nn; /* Size of record in bytes */
172757 int nWorkUnit; /* Leaf pages in a "unit" of work */
172761 ** in-memory hash tables before they are flushed to disk.
172763 Fts5Hash *pHash; /* Hash table for in-memory data */
172794 ** using an Fts5Structure record in memory. Which uses instances of the
172799 int pgnoFirst; /* First leaf page number in segment */
172800 int pgnoLast; /* Last leaf page number in segment */
172803 int nMerge; /* Number of segments in incr-merge */
172810 int nSegment; /* Total segments in this structure */
172811 int nLevel; /* Number of levels in this index */
172835 u8 bFirstRowidInDoclist; /* True if next rowid is first in doclist */
172836 u8 bFirstRowidInPage; /* True if next rowid is first in page */
172838 u8 bFirstTermInPage; /* True if next term will be first in leaf */
172852 ** visiting each term/rowid pair in the merged data.
172861 ** even number, is stored in aFirst[(nSeg+N)/2]. The "result" of the
172862 ** comparison in this context is the index of the iterator that currently
172866 ** aFirst[1] contains the index in aSeg[] of the iterator that points to
172878 ** pair in the segment.
172905 ** has been exhausted. Do not proceed to the next term in the segment.
172909 ** it is set, iterate through rowid in descending order instead of the
172931 ** is the offset of the first rowid in the current doclist. */
172935 int iPgidxOff; /* Next offset in pgidx */
172939 int iRowidOffset; /* Current entry in aRowidOffset[] */
172948 int nPos; /* Number of bytes in current position list */
172986 int bRev; /* True to iterate in reverse order */
173036 ** Allocate and return a buffer at least nByte bytes in size.
173038 ** If an OOM error is encountered, return NULL and set the error code in
173049 ** +ve if pRight is smaller than pLeft. In other words:
173069 ** +ve if pRight is smaller than pLeft. In other words:
173111 ** If an error occurs, NULL is returned and an error left in the
173121 ** rollback since it was last used. In this case a new blob handle
173146 ** table, missing row, non-blob/text in block column - indicate
173301 ** Deserialize and return the structure record currently stored in serialized
173314 int nData, /* Size of buffer pData in bytes */
173439 ** If an error occurs, NULL is returned and an error code left in the
173467 ** Return the total number of segments in index structure pStruct. This
173487 ** If an error occurs, leave an error code in the Fts5Index object. If an
173580 ** as a result. Segments are promoted in two scenarios:
173704 ** points to the first rowid in the doclist-index.
173764 ** before the buffer in memory. */
173929 ** read. Before returning, set *pnSz to the number of bytes in the position
173945 ** in the following variables:
173987 ** the first term in the segment).
174023 ** Initialize the iterator object pIter to iterate through the entries in
174038 ** fts5TrimSegments() for details. In this case leave the iterator empty.
174066 ** The iterator is in an unusual state when this function is called: the
174074 ** is in its regular state - Fts5SegIter.iLeafOffset points to the first
174135 ** In this case leaf pIter->pLeaf==0, this iterator is at EOF. */
174314 ** Iterator pIter currently points to the first rowid in a doclist. This
174315 ** function sets the iterator up so that iterates in reverse order through
174342 /* The last rowid in the doclist may not be on the current page. Search
174363 ** lies on the page currently indicated by the iterator. In this case
174368 ** rowid. In this case configure the iterator so that it points to the
174432 ** iterator is left pointing to the smallest term in the segment that
174451 int iPgidx; /* Current offset in pgidx */
174462 /* Figure out how many new bytes are in this term */
174549 ** pSeg. If there is no such term in the index, the iterator is set to EOF.
174575 ** contain term (pTerm/nTerm), if it is present in the segment. */
174638 ** in-memory hash table. If there is no such term in the hash-table, the
174706 ** in *pRes is the correct result of comparing the current positions of the
174747 ** is compiled. In that case, this function is essentially an assert()
174792 ** in the pIter->aSeg[] array that is (a) not at EOF, and (b) pointing
174794 ** segment-iterator in the pSeg->aSeg[] array.
174958 int iMinset /* Minimum entry in aFirst[] to set */
175031 ** If an error occurs, an error code is left in Fts5Index.rc. It is not
175091 ** The new object will be used to iterate through data in structure pStruct.
175092 ** If iLevel is -ve, then all data in all segments is merged. Or, if iLevel
175096 ** The iterator initially points to the first term/rowid entry in the
175109 int nSeg = 0; /* Number of segment-iters in use */
175163 ** to the first entry in its segment. In this case initialize the
175323 ** returned in this case.
175351 ** Discard all data currently cached in the hash-tables.
175362 ** Return the size of the prefix, in bytes, that buffer (nNew/pNew) shares
175400 ** Grow the pWriter->aDlidx[] array to at least nLvl elements in size.
175425 ** If the current doclist-index accumulating in pWriter->aDlidx[] is large
175446 ** The doclist-index for that term is currently stored in-memory within the
175462 /* The following was already done in fts5WriteInit(): */
175478 ** If an error occurs, an error code is left in Fts5Index.rc. If an error
175665 /* This is the first term on a leaf that is not the leftmost leaf in
175666 ** the segment b-tree. In this case it is necessary to add a term to
175669 ** this term. In other words, a prefix of (pTerm/nTerm) that is one
175673 ** Usually, the previous term is available in pPage->term. The exception
175674 ** is if this is the first term written in an incremental-merge step.
175675 ** In this case the previous term is not available, so just write a
175723 ** rowid-pointer in the page-header. Also append a value to the dlidx
175724 ** buffer, in case a doclist-index is required. */
175782 int *pnLeaf /* OUT: Number of leaf pages in b-tree */
175820 /* Grow the two buffers to pgsz + padding bytes in size. */
175920 Fts5Structure **ppStruct, /* IN/OUT: Stucture of index */
175972 /* Read input from all segments in the input level */
176050 Fts5Structure **ppStruct, /* IN/OUT: Current structure of index */
176107 Fts5Structure **ppStruct, /* IN/OUT: Current structure of index */
176128 Fts5Structure **ppStruct /* IN/OUT: Current structure of index */
176158 ** in a 32-bit integer. Return the size of the largest prefix of this
176159 ** list nMax bytes or less in size.
176187 ** Flush the contents of in-memory hash table iHash to a new level-0
176197 int pgnoLast = 0; /* Last leaf page number in segment */
176208 Fts5Buffer *pBuf; /* Buffer in which to assemble leaf page */
176209 Fts5Buffer *pPgidx; /* Buffer in which to assemble pgidx */
176230 int nDoclist; /* Size of doclist in bytes */
176269 ** it in one go. */
176329 ** Flush any data stored in the in-memory hash tables to the database.
176518 ** IN/OUT parameter (*pa) points to a position list n bytes in size. If
176521 ** bytes in it returned. Or, if the argument position list does not
176525 const u8 **pa, /* IN/OUT: Pointer to poslist */
176526 int n, /* IN: Size of poslist in bytes */
176564 ** no entries in the position list, nothing is appended to the buffer (not
176567 ** If an error occurs, an error code is left in p->rc.
176673 i64 *piLastRowid, /* IN/OUT: Previous rowid written (if any) */
176693 ** If an error occurs, an error code is left in p->rc. If an error has
176783 int nToken, /* Size of buffer pToken in bytes */
176891 ** Discard any data stored in the in-memory hash tables. Do not write it
176893 ** table may have changed on disk. So any in-memory caches of %_data
176982 ** Argument p points to a buffer containing utf-8 text that is n bytes in
176983 ** size. Return the number of bytes in the nChar character prefix of the
176984 ** buffer, or 0 if there are less than nChar characters in total.
176999 ** pIn is a UTF-8 encoded string, nIn bytes in size. Return the number of
177000 ** unicode characters in the string.
177019 ** For an insert, it must be called once for each token in the new document.
177021 ** unique token in the document with an iCol value less than zero. The iPos
177026 int iCol, /* Column token appears in (-ve -> delete) */
177082 ** for internal sanity checking by the integrity-check in debug
177161 ** in ascending or descending rowid order.
177209 ** in bytes before returning.
177218 int *pn, /* OUT: Size of position-list in bytes */
177276 ** the number of user defined columns in the FTS table.
177380 ** FTS functionality, or even the integrity-check, in any way.
177384 ** iSegid/iLeaf is iterated in forwards or reverse order.
177426 int n, /* Size of index key in bytes */
177428 u64 *pCksum /* IN/OUT: Checksum value */
177462 ** FTS functionality, or even the integrity-check, in any way.
177478 int nTerm = pPrev->n-1; /* Size of zTerm in bytes */
177494 ** the index is disabled are the same. In both ASC and DESC order.
177637 /* If the leaf in question has already been trimmed from the segment,
177645 ** to or larger than the split-key in zIdxTerm. Also check that if there
177653 int nTerm; /* Size of term on leaf in bytes */
177736 /* Page iter.iLeaf must now be the rightmost leaf-page in the segment */
177780 ** based on all expected entries in the FTS index (including prefix index
177786 ** while doing a linear scan of each individual index in turn.
177796 int n; /* Size of term in bytes */
177832 i64 iRowid, /* Document term appears in */
177833 int iCol, /* Column term appears in */
177834 int iPos, /* Position term appears in */
177900 int *pRc, /* IN/OUT: error code */
177929 int *pRc, /* IN/OUT: error code */
177954 int *pRc, /* IN/OUT: error code */
178038 /* Make a copy of the second argument (a blob) in aBlob[]. The aBlob[]
178076 int szLeaf; /* Offset of pgidx in a[] */
178213 ** The author disclaims copyright to this source code. In place of
178230 ** assert() conditions in the fts5 code are activated - conditions that are
178255 ** in the xSync() callback. Current versions of SQLite ignore any errors
178265 ** in-memory. It is flushed to disk whenever xSync(), xRelease() or
178271 ** is not required, but it is used in the assert() statements executed by
178288 Fts5Auxiliary *pAux; /* First in list of all aux. functions */
178289 Fts5TokenizerModule *pTok; /* First in list of all tokenizer modules */
178291 Fts5Cursor *pCsr; /* First in list of all open cursors */
178338 int nTerm; /* Size of phrase in terms */
178346 ** There is one entry in the aIdx[] array for each phrase in the query,
178354 int nIdx; /* Number of entries in aIdx[] */
178368 ** cursor iterates in ascending order of rowids, iFirstRowid is the lower
178369 ** limit of rowids to return, and iLastRowid the upper. In other words, the
178370 ** WHERE clause in the user's query might have been:
178374 ** If the cursor iterates in descending order of rowid, iFirstRowid
178380 Fts5Cursor *pNext; /* Next cursor in Fts5Cursor.pCsr list */
178401 sqlite3_stmt *pRankArgStmt; /* Origin of objects in apRankArg[] */
178405 Fts5Auxdata *pAuxdata; /* First in linked list of saved aux-data */
178453 Fts5Auxdata *pNext; /* Next object in linked list */
178567 int argc, /* Number of elements in argv array */
178618 ** work is done in function fts5InitVtab().
178623 int argc, /* Number of elements in argv array */
178633 int argc, /* Number of elements in argv array */
178652 ** Set the SQLITE_INDEX_SCAN_UNIQUE flag in pIdxInfo->flags. Unless this
178654 ** support index-info flags. In that case this function is a no-op.
178681 ** a) If an unusable MATCH operator is present in the WHERE clause, the
178714 int iConsIndex; /* Index in pInfo->aConstraint[] */
178768 /* Calculate the estimated cost based on the flags set in idxFlags. */
178781 /* Assign argvIndex values to each constraint in use. */
178955 ** (in the cursors sort order - ASC or DESC) than the current rowid.
178987 in the table that matches the
179060 ** If SQLite a built-in statement cache, this wouldn't be a problem. */
179114 int n; /* Number of bytes in text at z */
179226 "parse error in rank function: %s", z
179266 int nVal, /* Number of elements in apVal */
179300 ** Note: The following set of if(...) statements must be in the same
179301 ** order as the corresponding entries in the struct at the top of
179452 ** be left in sqlite3_vtab.zErrMsg.
179495 ** This function is called to handle an FTS INSERT command. In other words,
179505 ** The commands implemented by this function are documented in the "Special
179958 int nToken, /* Size of token in bytes */
180229 ** If successful, set *pnCol to the number of indexed columns in the
180250 ** the current position-list for each phrase in the query associated with
180254 ** the number of phrases in the query. Following the varints are the
180255 ** concatenated position lists for each phrase, in order.
180314 ** useful in that it may be passed as the first argument to an FTS5
180426 int nName; /* Size of zName in bytes, including \0 */
180551 *pzErr = sqlite3_mprintf("error in tokenizer constructor");
180713 ** The author disclaims copyright to this source code. In place of
180731 i64 nTotalRow; /* Total number of rows in FTS table */
180914 int *pRc, /* IN/OUT: Error code */
181062 int szCol; /* Size of column value in tokens */
181072 int nToken, /* Size of token in bytes */
181085 ** If a row with rowid iDel is present in the %_content table, add the
181174 ** variables in the "averages" record on disk.
181297 ** Delete all entries in the FTS5 index.
181394 ** If the %_docsize table does not exist, SQLITE_MISMATCH is returned. In
181554 int nToken, /* Size of token in bytes */
181590 ** %_content table. This block stores the checksum in ctx.cksum. */
181723 int nCol = p->pConfig->nCol; /* Number of user columns in table */
181775 ** Flush any data currently held in-memory to disk.
181824 ** The author disclaims copyright to this source code. In place of
182052 int nFold; /* Size of aFold[] in bytes */
182311 /* Any tokens larger than this (in bytes) are passed through without
183027 ** Register all built-in tokenizers with FTS5.
183059 ** The author disclaims copyright to this source code. In place of
183084 /* Each unsigned integer in the following array corresponds to a contiguous
183088 ** The most significant 22 bits in each 32-bit value contain the first
183089 ** codepoint in the range. The least significant 10 bits are used to store
183090 ** the size of the range (always at least 1). In other words, the value
183209 ** in the ASCII range with a diacritic added, return the codepoint
183285 /* Each entry in the following array defines a rule for folding a range
183289 ** If the least significant bit in flags is clear, then the rule applies
183292 ** every second codepoint in the range, starting with codepoint C.
183294 ** The 7 most significant bits in flags are an index into the aiOff[]
183420 ** The author disclaims copyright to this source code. In place of
183480 /* A 32-bit varint is used to store size information in btrees.
183504 ** inline in order to work around bugs in the RVT compiler.
183515 ** Return the number of bytes read. The value is stored in *v.
183590 /* we can skip these cause they were (effectively) done above in calc'ing s */
183611 /* we can skip this cause it was (effectively) done above in calc'ing s */
183762 ** The author disclaims copyright to this source code. In place of
183781 ** instances of term $term in column $col (in any row of the fts5 table).
183786 ** One row for each term in the database. The value of $doc is set to
183789 ** $term in the database.
183895 ** or, for tables in the TEMP schema only.
183904 int argc, /* Number of elements in argv array */
183960 ** work is done in function fts5VocabInitVtab().
183965 int argc, /* Number of elements in argv array */
183975 int argc, /* Number of elements in argv array */
184073 ** Advance the cursor to the next row in the table.
184154 int nVal, /* Number of elements in apVal */
