1 <html> 2 <head> 3 <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> 4 <title>1.8.3 Manual</title> 5 </head> 6 <body> 7 <h1>1.8.3 Manual</h1> 8 <hr> 9 <a name="Contents"></a><h2>Contents</h2> 10 <ol> 11 <li><a href="#Chapter1">Introduction</a></li> 12 <li><a href="#Chapter2">Version</a></li> 13 <li><a href="#Chapter3">Tuning parameter</a></li> 14 <li><a href="#Chapter4">Simple Functions</a></li> 15 <li><a href="#Chapter5">Advanced Functions</a></li> 16 <li><a href="#Chapter6">Streaming Compression Functions</a></li> 17 <li><a href="#Chapter7">Streaming Decompression Functions</a></li> 18 <li><a href="#Chapter8">Unstable declarations</a></li> 19 <li><a href="#Chapter9">Private definitions</a></li> 20 <li><a href="#Chapter10">Obsolete Functions</a></li> 21 </ol> 22 <hr> 23 <a name="Chapter1"></a><h2>Introduction</h2><pre> 24 LZ4 is lossless compression algorithm, providing compression speed at 400 MB/s per core, 25 scalable with multi-cores CPU. It features an extremely fast decoder, with speed in 26 multiple GB/s per core, typically reaching RAM speed limits on multi-core systems. 27 28 The LZ4 compression library provides in-memory compression and decompression functions. 29 Compression can be done in: 30 - a single step (described as Simple Functions) 31 - a single step, reusing a context (described in Advanced Functions) 32 - unbounded multiple steps (described as Streaming compression) 33 34 lz4.h provides block compression functions. It gives full buffer control to user. 35 Decompressing an lz4-compressed block also requires metadata (such as compressed size). 36 Each application is free to encode such metadata in whichever way it wants. 37 38 An additional format, called LZ4 frame specification (doc/lz4_Frame_format.md), 39 take care of encoding standard metadata alongside LZ4-compressed blocks. 40 If your application requires interoperability, it's recommended to use it. 41 A library is provided to take care of it, see lz4frame.h. 42 <BR></pre> 43 44 <a name="Chapter2"></a><h2>Version</h2><pre></pre> 45 46 <pre><b>int LZ4_versionNumber (void); </b>/**< library version number; useful to check dll version */<b> 47 </b></pre><BR> 48 <pre><b>const char* LZ4_versionString (void); </b>/**< library version string; unseful to check dll version */<b> 49 </b></pre><BR> 50 <a name="Chapter3"></a><h2>Tuning parameter</h2><pre></pre> 51 52 <pre><b>#ifndef LZ4_MEMORY_USAGE 53 # define LZ4_MEMORY_USAGE 14 54 #endif 55 </b><p> Memory usage formula : N->2^N Bytes (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.) 56 Increasing memory usage improves compression ratio 57 Reduced memory usage may improve speed, thanks to cache effect 58 Default value is 14, for 16KB, which nicely fits into Intel x86 L1 cache 59 60 </p></pre><BR> 61 62 <a name="Chapter4"></a><h2>Simple Functions</h2><pre></pre> 63 64 <pre><b>int LZ4_compress_default(const char* src, char* dst, int srcSize, int dstCapacity); 65 </b><p> Compresses 'srcSize' bytes from buffer 'src' 66 into already allocated 'dst' buffer of size 'dstCapacity'. 67 Compression is guaranteed to succeed if 'dstCapacity' >= LZ4_compressBound(srcSize). 68 It also runs faster, so it's a recommended setting. 69 If the function cannot compress 'src' into a more limited 'dst' budget, 70 compression stops *immediately*, and the function result is zero. 71 Note : as a consequence, 'dst' content is not valid. 72 Note 2 : This function is protected against buffer overflow scenarios (never writes outside 'dst' buffer, nor read outside 'source' buffer). 73 srcSize : max supported value is LZ4_MAX_INPUT_SIZE. 74 dstCapacity : size of buffer 'dst' (which must be already allocated) 75 return : the number of bytes written into buffer 'dst' (necessarily <= dstCapacity) 76 or 0 if compression fails 77 </p></pre><BR> 78 79 <pre><b>int LZ4_decompress_safe (const char* src, char* dst, int compressedSize, int dstCapacity); 80 </b><p> compressedSize : is the exact complete size of the compressed block. 81 dstCapacity : is the size of destination buffer, which must be already allocated. 82 return : the number of bytes decompressed into destination buffer (necessarily <= dstCapacity) 83 If destination buffer is not large enough, decoding will stop and output an error code (negative value). 84 If the source stream is detected malformed, the function will stop decoding and return a negative result. 85 This function is protected against malicious data packets. 86 </p></pre><BR> 87 88 <a name="Chapter5"></a><h2>Advanced Functions</h2><pre></pre> 89 90 <pre><b>int LZ4_compressBound(int inputSize); 91 </b><p> Provides the maximum size that LZ4 compression may output in a "worst case" scenario (input data not compressible) 92 This function is primarily useful for memory allocation purposes (destination buffer size). 93 Macro LZ4_COMPRESSBOUND() is also provided for compilation-time evaluation (stack memory allocation for example). 94 Note that LZ4_compress_default() compresses faster when dstCapacity is >= LZ4_compressBound(srcSize) 95 inputSize : max supported value is LZ4_MAX_INPUT_SIZE 96 return : maximum output size in a "worst case" scenario 97 or 0, if input size is incorrect (too large or negative) 98 </p></pre><BR> 99 100 <pre><b>int LZ4_compress_fast (const char* src, char* dst, int srcSize, int dstCapacity, int acceleration); 101 </b><p> Same as LZ4_compress_default(), but allows selection of "acceleration" factor. 102 The larger the acceleration value, the faster the algorithm, but also the lesser the compression. 103 It's a trade-off. It can be fine tuned, with each successive value providing roughly +~3% to speed. 104 An acceleration value of "1" is the same as regular LZ4_compress_default() 105 Values <= 0 will be replaced by ACCELERATION_DEFAULT (currently == 1, see lz4.c). 106 </p></pre><BR> 107 108 <pre><b>int LZ4_sizeofState(void); 109 int LZ4_compress_fast_extState (void* state, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration); 110 </b><p> Same compression function, just using an externally allocated memory space to store compression state. 111 Use LZ4_sizeofState() to know how much memory must be allocated, 112 and allocate it on 8-bytes boundaries (using malloc() typically). 113 Then, provide this buffer as 'void* state' to compression function. 114 </p></pre><BR> 115 116 <pre><b>int LZ4_compress_destSize (const char* src, char* dst, int* srcSizePtr, int targetDstSize); 117 </b><p> Reverse the logic : compresses as much data as possible from 'src' buffer 118 into already allocated buffer 'dst', of size >= 'targetDestSize'. 119 This function either compresses the entire 'src' content into 'dst' if it's large enough, 120 or fill 'dst' buffer completely with as much data as possible from 'src'. 121 note: acceleration parameter is fixed to "default". 122 123 *srcSizePtr : will be modified to indicate how many bytes where read from 'src' to fill 'dst'. 124 New value is necessarily <= input value. 125 @return : Nb bytes written into 'dst' (necessarily <= targetDestSize) 126 or 0 if compression fails. 127 </p></pre><BR> 128 129 <pre><b>int LZ4_decompress_fast (const char* src, char* dst, int originalSize); 130 </b><p> This function used to be a bit faster than LZ4_decompress_safe(), 131 though situation has changed in recent versions, 132 and now `LZ4_decompress_safe()` can be as fast and sometimes faster than `LZ4_decompress_fast()`. 133 Moreover, LZ4_decompress_fast() is not protected vs malformed input, as it doesn't perform full validation of compressed data. 134 As a consequence, this function is no longer recommended, and may be deprecated in future versions. 135 It's only remaining specificity is that it can decompress data without knowing its compressed size. 136 137 originalSize : is the uncompressed size to regenerate. 138 `dst` must be already allocated, its size must be >= 'originalSize' bytes. 139 @return : number of bytes read from source buffer (== compressed size). 140 If the source stream is detected malformed, the function stops decoding and returns a negative result. 141 note : This function requires uncompressed originalSize to be known in advance. 142 The function never writes past the output buffer. 143 However, since it doesn't know its 'src' size, it may read past the intended input. 144 Also, because match offsets are not validated during decoding, 145 reads from 'src' may underflow. 146 Use this function in trusted environment **only**. 147 148 </p></pre><BR> 149 150 <pre><b>int LZ4_decompress_safe_partial (const char* src, char* dst, int srcSize, int targetOutputSize, int dstCapacity); 151 </b><p> Decompress an LZ4 compressed block, of size 'srcSize' at position 'src', 152 into destination buffer 'dst' of size 'dstCapacity'. 153 Up to 'targetOutputSize' bytes will be decoded. 154 The function stops decoding on reaching this objective, 155 which can boost performance when only the beginning of a block is required. 156 157 @return : the number of bytes decoded in `dst` (necessarily <= dstCapacity) 158 If source stream is detected malformed, function returns a negative result. 159 160 Note : @return can be < targetOutputSize, if compressed block contains less data. 161 162 Note 2 : this function features 2 parameters, targetOutputSize and dstCapacity, 163 and expects targetOutputSize <= dstCapacity. 164 It effectively stops decoding on reaching targetOutputSize, 165 so dstCapacity is kind of redundant. 166 This is because in a previous version of this function, 167 decoding operation would not "break" a sequence in the middle. 168 As a consequence, there was no guarantee that decoding would stop at exactly targetOutputSize, 169 it could write more bytes, though only up to dstCapacity. 170 Some "margin" used to be required for this operation to work properly. 171 This is no longer necessary. 172 The function nonetheless keeps its signature, in an effort to not break API. 173 174 </p></pre><BR> 175 176 <a name="Chapter6"></a><h2>Streaming Compression Functions</h2><pre></pre> 177 178 <pre><b>LZ4_stream_t* LZ4_createStream(void); 179 int LZ4_freeStream (LZ4_stream_t* streamPtr); 180 </b><p> LZ4_createStream() will allocate and initialize an `LZ4_stream_t` structure. 181 LZ4_freeStream() releases its memory. 182 183 </p></pre><BR> 184 185 <pre><b>void LZ4_resetStream (LZ4_stream_t* streamPtr); 186 </b><p> An LZ4_stream_t structure can be allocated once and re-used multiple times. 187 Use this function to start compressing a new stream. 188 189 </p></pre><BR> 190 191 <pre><b>int LZ4_loadDict (LZ4_stream_t* streamPtr, const char* dictionary, int dictSize); 192 </b><p> Use this function to load a static dictionary into LZ4_stream_t. 193 Any previous data will be forgotten, only 'dictionary' will remain in memory. 194 Loading a size of 0 is allowed, and is the same as reset. 195 @return : dictionary size, in bytes (necessarily <= 64 KB) 196 197 </p></pre><BR> 198 199 <pre><b>int LZ4_compress_fast_continue (LZ4_stream_t* streamPtr, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration); 200 </b><p> Compress 'src' content using data from previously compressed blocks, for better compression ratio. 201 'dst' buffer must be already allocated. 202 If dstCapacity >= LZ4_compressBound(srcSize), compression is guaranteed to succeed, and runs faster. 203 204 @return : size of compressed block 205 or 0 if there is an error (typically, cannot fit into 'dst'). 206 207 Note 1 : Each invocation to LZ4_compress_fast_continue() generates a new block. 208 Each block has precise boundaries. 209 It's not possible to append blocks together and expect a single invocation of LZ4_decompress_*() to decompress them together. 210 Each block must be decompressed separately, calling LZ4_decompress_*() with associated metadata. 211 212 Note 2 : The previous 64KB of source data is __assumed__ to remain present, unmodified, at same address in memory! 213 214 Note 3 : When input is structured as a double-buffer, each buffer can have any size, including < 64 KB. 215 Make sure that buffers are separated, by at least one byte. 216 This construction ensures that each block only depends on previous block. 217 218 Note 4 : If input buffer is a ring-buffer, it can have any size, including < 64 KB. 219 220 Note 5 : After an error, the stream status is invalid, it can only be reset or freed. 221 222 </p></pre><BR> 223 224 <pre><b>int LZ4_saveDict (LZ4_stream_t* streamPtr, char* safeBuffer, int maxDictSize); 225 </b><p> If last 64KB data cannot be guaranteed to remain available at its current memory location, 226 save it into a safer place (char* safeBuffer). 227 This is schematically equivalent to a memcpy() followed by LZ4_loadDict(), 228 but is much faster, because LZ4_saveDict() doesn't need to rebuild tables. 229 @return : saved dictionary size in bytes (necessarily <= maxDictSize), or 0 if error. 230 231 </p></pre><BR> 232 233 <a name="Chapter7"></a><h2>Streaming Decompression Functions</h2><pre> Bufferless synchronous API 234 <BR></pre> 235 236 <pre><b>LZ4_streamDecode_t* LZ4_createStreamDecode(void); 237 int LZ4_freeStreamDecode (LZ4_streamDecode_t* LZ4_stream); 238 </b><p> creation / destruction of streaming decompression tracking context. 239 A tracking context can be re-used multiple times. 240 241 </p></pre><BR> 242 243 <pre><b>int LZ4_setStreamDecode (LZ4_streamDecode_t* LZ4_streamDecode, const char* dictionary, int dictSize); 244 </b><p> An LZ4_streamDecode_t context can be allocated once and re-used multiple times. 245 Use this function to start decompression of a new stream of blocks. 246 A dictionary can optionally be set. Use NULL or size 0 for a reset order. 247 Dictionary is presumed stable : it must remain accessible and unmodified during next decompression. 248 @return : 1 if OK, 0 if error 249 250 </p></pre><BR> 251 252 <pre><b>int LZ4_decoderRingBufferSize(int maxBlockSize); 253 #define LZ4_DECODER_RING_BUFFER_SIZE(mbs) (65536 + 14 + (mbs)) </b>/* for static allocation; mbs presumed valid */<b> 254 </b><p> Note : in a ring buffer scenario (optional), 255 blocks are presumed decompressed next to each other 256 up to the moment there is not enough remaining space for next block (remainingSize < maxBlockSize), 257 at which stage it resumes from beginning of ring buffer. 258 When setting such a ring buffer for streaming decompression, 259 provides the minimum size of this ring buffer 260 to be compatible with any source respecting maxBlockSize condition. 261 @return : minimum ring buffer size, 262 or 0 if there is an error (invalid maxBlockSize). 263 264 </p></pre><BR> 265 266 <pre><b>int LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* src, char* dst, int srcSize, int dstCapacity); 267 int LZ4_decompress_fast_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* src, char* dst, int originalSize); 268 </b><p> These decoding functions allow decompression of consecutive blocks in "streaming" mode. 269 A block is an unsplittable entity, it must be presented entirely to a decompression function. 270 Decompression functions only accepts one block at a time. 271 The last 64KB of previously decoded data *must* remain available and unmodified at the memory position where they were decoded. 272 If less than 64KB of data has been decoded, all the data must be present. 273 274 Special : if decompression side sets a ring buffer, it must respect one of the following conditions : 275 - Decompression buffer size is _at least_ LZ4_decoderRingBufferSize(maxBlockSize). 276 maxBlockSize is the maximum size of any single block. It can have any value > 16 bytes. 277 In which case, encoding and decoding buffers do not need to be synchronized. 278 Actually, data can be produced by any source compliant with LZ4 format specification, and respecting maxBlockSize. 279 - Synchronized mode : 280 Decompression buffer size is _exactly_ the same as compression buffer size, 281 and follows exactly same update rule (block boundaries at same positions), 282 and decoding function is provided with exact decompressed size of each block (exception for last block of the stream), 283 _then_ decoding & encoding ring buffer can have any size, including small ones ( < 64 KB). 284 - Decompression buffer is larger than encoding buffer, by a minimum of maxBlockSize more bytes. 285 In which case, encoding and decoding buffers do not need to be synchronized, 286 and encoding ring buffer can have any size, including small ones ( < 64 KB). 287 288 Whenever these conditions are not possible, 289 save the last 64KB of decoded data into a safe buffer where it can't be modified during decompression, 290 then indicate where this data is saved using LZ4_setStreamDecode(), before decompressing next block. 291 </p></pre><BR> 292 293 <pre><b>int LZ4_decompress_safe_usingDict (const char* src, char* dst, int srcSize, int dstCapcity, const char* dictStart, int dictSize); 294 int LZ4_decompress_fast_usingDict (const char* src, char* dst, int originalSize, const char* dictStart, int dictSize); 295 </b><p> These decoding functions work the same as 296 a combination of LZ4_setStreamDecode() followed by LZ4_decompress_*_continue() 297 They are stand-alone, and don't need an LZ4_streamDecode_t structure. 298 Dictionary is presumed stable : it must remain accessible and unmodified during next decompression. 299 300 </p></pre><BR> 301 302 <a name="Chapter8"></a><h2>Unstable declarations</h2><pre> 303 Declarations in this section should be considered unstable. 304 Use at your own peril, etc., etc. 305 They may be removed in the future. 306 Their signatures may change. 307 <BR></pre> 308 309 <pre><b>void LZ4_resetStream_fast (LZ4_stream_t* streamPtr); 310 </b><p> Use this, like LZ4_resetStream(), to prepare a context for a new chain of 311 calls to a streaming API (e.g., LZ4_compress_fast_continue()). 312 313 Note: 314 Using this in advance of a non- streaming-compression function is redundant, 315 and potentially bad for performance, since they all perform their own custom 316 reset internally. 317 318 Differences from LZ4_resetStream(): 319 When an LZ4_stream_t is known to be in a internally coherent state, 320 it can often be prepared for a new compression with almost no work, only 321 sometimes falling back to the full, expensive reset that is always required 322 when the stream is in an indeterminate state (i.e., the reset performed by 323 LZ4_resetStream()). 324 325 LZ4_streams are guaranteed to be in a valid state when: 326 - returned from LZ4_createStream() 327 - reset by LZ4_resetStream() 328 - memset(stream, 0, sizeof(LZ4_stream_t)), though this is discouraged 329 - the stream was in a valid state and was reset by LZ4_resetStream_fast() 330 - the stream was in a valid state and was then used in any compression call 331 that returned success 332 - the stream was in an indeterminate state and was used in a compression 333 call that fully reset the state (e.g., LZ4_compress_fast_extState()) and 334 that returned success 335 336 When a stream isn't known to be in a valid state, it is not safe to pass to 337 any fastReset or streaming function. It must first be cleansed by the full 338 LZ4_resetStream(). 339 340 </p></pre><BR> 341 342 <pre><b>int LZ4_compress_fast_extState_fastReset (void* state, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration); 343 </b><p> A variant of LZ4_compress_fast_extState(). 344 345 Using this variant avoids an expensive initialization step. It is only safe 346 to call if the state buffer is known to be correctly initialized already 347 (see above comment on LZ4_resetStream_fast() for a definition of "correctly 348 initialized"). From a high level, the difference is that this function 349 initializes the provided state with a call to something like 350 LZ4_resetStream_fast() while LZ4_compress_fast_extState() starts with a 351 call to LZ4_resetStream(). 352 353 </p></pre><BR> 354 355 <pre><b>void LZ4_attach_dictionary(LZ4_stream_t *working_stream, const LZ4_stream_t *dictionary_stream); 356 </b><p> This is an experimental API that allows for the efficient use of a 357 static dictionary many times. 358 359 Rather than re-loading the dictionary buffer into a working context before 360 each compression, or copying a pre-loaded dictionary's LZ4_stream_t into a 361 working LZ4_stream_t, this function introduces a no-copy setup mechanism, 362 in which the working stream references the dictionary stream in-place. 363 364 Several assumptions are made about the state of the dictionary stream. 365 Currently, only streams which have been prepared by LZ4_loadDict() should 366 be expected to work. 367 368 Alternatively, the provided dictionary stream pointer may be NULL, in which 369 case any existing dictionary stream is unset. 370 371 If a dictionary is provided, it replaces any pre-existing stream history. 372 The dictionary contents are the only history that can be referenced and 373 logically immediately precede the data compressed in the first subsequent 374 compression call. 375 376 The dictionary will only remain attached to the working stream through the 377 first compression call, at the end of which it is cleared. The dictionary 378 stream (and source buffer) must remain in-place / accessible / unchanged 379 through the completion of the first compression call on the stream. 380 381 </p></pre><BR> 382 383 <a name="Chapter9"></a><h2>Private definitions</h2><pre> 384 Do not use these definitions. 385 They are exposed to allow static allocation of `LZ4_stream_t` and `LZ4_streamDecode_t`. 386 Using these definitions will expose code to API and/or ABI break in future versions of the library. 387 <BR></pre> 388 389 <pre><b>typedef struct { 390 const uint8_t* externalDict; 391 size_t extDictSize; 392 const uint8_t* prefixEnd; 393 size_t prefixSize; 394 } LZ4_streamDecode_t_internal; 395 </b></pre><BR> 396 <pre><b>typedef struct { 397 const unsigned char* externalDict; 398 size_t extDictSize; 399 const unsigned char* prefixEnd; 400 size_t prefixSize; 401 } LZ4_streamDecode_t_internal; 402 </b></pre><BR> 403 <pre><b>#define LZ4_STREAMSIZE_U64 ((1 << (LZ4_MEMORY_USAGE-3)) + 4) 404 #define LZ4_STREAMSIZE (LZ4_STREAMSIZE_U64 * sizeof(unsigned long long)) 405 union LZ4_stream_u { 406 unsigned long long table[LZ4_STREAMSIZE_U64]; 407 LZ4_stream_t_internal internal_donotuse; 408 } ; </b>/* previously typedef'd to LZ4_stream_t */<b> 409 </b><p> information structure to track an LZ4 stream. 410 init this structure before first use. 411 note : only use in association with static linking ! 412 this definition is not API/ABI safe, 413 it may change in a future version ! 414 415 </p></pre><BR> 416 417 <pre><b>#define LZ4_STREAMDECODESIZE_U64 4 418 #define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U64 * sizeof(unsigned long long)) 419 union LZ4_streamDecode_u { 420 unsigned long long table[LZ4_STREAMDECODESIZE_U64]; 421 LZ4_streamDecode_t_internal internal_donotuse; 422 } ; </b>/* previously typedef'd to LZ4_streamDecode_t */<b> 423 </b><p> information structure to track an LZ4 stream during decompression. 424 init this structure using LZ4_setStreamDecode (or memset()) before first use 425 note : only use in association with static linking ! 426 this definition is not API/ABI safe, 427 and may change in a future version ! 428 429 </p></pre><BR> 430 431 <a name="Chapter10"></a><h2>Obsolete Functions</h2><pre></pre> 432 433 <pre><b>#ifdef LZ4_DISABLE_DEPRECATE_WARNINGS 434 # define LZ4_DEPRECATED(message) </b>/* disable deprecation warnings */<b> 435 #else 436 # define LZ4_GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__) 437 # if defined (__cplusplus) && (__cplusplus >= 201402) </b>/* C++14 or greater */<b> 438 # define LZ4_DEPRECATED(message) [[deprecated(message)]] 439 # elif (LZ4_GCC_VERSION >= 405) || defined(__clang__) 440 # define LZ4_DEPRECATED(message) __attribute__((deprecated(message))) 441 # elif (LZ4_GCC_VERSION >= 301) 442 # define LZ4_DEPRECATED(message) __attribute__((deprecated)) 443 # elif defined(_MSC_VER) 444 # define LZ4_DEPRECATED(message) __declspec(deprecated(message)) 445 # else 446 # pragma message("WARNING: You need to implement LZ4_DEPRECATED for this compiler") 447 # define LZ4_DEPRECATED(message) 448 # endif 449 #endif </b>/* LZ4_DISABLE_DEPRECATE_WARNINGS */<b> 450 </b><p> Should deprecation warnings be a problem, 451 it is generally possible to disable them, 452 typically with -Wno-deprecated-declarations for gcc 453 or _CRT_SECURE_NO_WARNINGS in Visual. 454 Otherwise, it's also possible to define LZ4_DISABLE_DEPRECATE_WARNINGS 455 </p></pre><BR> 456 457 </html> 458 </body> 459
