| Up to higher level directory | |||
| Name | Date | Size | |
|---|---|---|---|
| ACKNOWLEDGEMENTS | 06-Dec-2016 | 6K | |
| Android.mk | 06-Dec-2016 | 133 | |
| CHANGES | 06-Dec-2016 | 26.7K | |
| COPYING | 06-Dec-2016 | 17.7K | |
| INSTALL | 06-Dec-2016 | 1.1K | |
| kernel/ | 06-Dec-2016 | ||
| kernel-2.4/ | 06-Dec-2016 | ||
| MODULE_LICENSE_GPL | 06-Dec-2016 | 0 | |
| NOTICE | 06-Dec-2016 | 17.7K | |
| README | 06-Dec-2016 | 864 | |
| RELEASE-README | 06-Dec-2016 | 40K | |
| RELEASE-READMEs/ | 06-Dec-2016 | ||
| squashfs-tools/ | 06-Dec-2016 | ||
README
1 Git status: 2 3 This is the squashfs.sourceforge.net CVS repository imported into this 4 new git repository. All new development will be under git. 5 6 Squashfs-tools: The new Squashfs 4.3 release (2014/05/05). 7 8 kernel: obsolete, the kernel code is now in mainline at www.kernel.org. 9 10 All Squashfs kernel development trees are stored on kernel.org, 11 under git.kernel.org:/linux/kernel/git/pkl/... 12 13 kernel-2.4: prehistoric, not updated since the 3.1 release. If you still need 14 Squashfs support in the 2.4 kernel then use the squashfs 3.1 release. This 15 is the last release that supported 2.4 kernels. 16 17 The kernel and kernel-2.4 directories (imported from CVS) are not really 18 relevant anymore, but are here temporarily while I decide where to put 19 them (I don't want to delete them and have all the pre-mainlining 20 kernel commit history disappear from public repositories). 21 22
RELEASE-README
1 SQUASHFS 4.3 - A squashed read-only filesystem for Linux 2 3 Copyright 2002-2014 Phillip Lougher <phillip (a] lougher.demon.co.uk> 4 5 Released under the GPL licence (version 2 or later). 6 7 Welcome to Squashfs version 4.3. Please read the README-4.3 and CHANGES files 8 for details of changes. 9 10 Squashfs is a highly compressed read-only filesystem for Linux. 11 It uses either gzip/xz/lzo/lz4 compression to compress both files, inodes 12 and directories. Inodes in the system are very small and all blocks are 13 packed to minimise data overhead. Block sizes greater than 4K are supported 14 up to a maximum of 1Mbytes (default block size 128K). 15 16 Squashfs is intended for general read-only filesystem use, for archival 17 use (i.e. in cases where a .tar.gz file may be used), and in constrained 18 block device/memory systems (e.g. embedded systems) where low overhead is 19 needed. 20 21 1. SQUASHFS OVERVIEW 22 -------------------- 23 24 1. Data, inodes and directories are compressed. 25 26 2. Squashfs stores full uid/gids (32 bits), and file creation time. 27 28 3. In theory files up to 2^64 bytes are supported. In theory filesystems can 29 be up to 2^64 bytes. 30 31 4. Inode and directory data are highly compacted, and packed on byte 32 boundaries. Each compressed inode is on average 8 bytes in length 33 (the exact length varies on file type, i.e. regular file, directory, 34 symbolic link, and block/char device inodes have different sizes). 35 36 5. Squashfs can use block sizes up to 1Mbyte (the default size is 128K). 37 Using 128K blocks achieves greater compression ratios than the normal 38 4K block size. 39 40 6. File duplicates are detected and removed. 41 42 7. Filesystems can be compressed with gzip, xz (lzma2), lzo or lz4 43 compression algorithms. 44 45 1.1 Extended attributes (xattrs) 46 -------------------------------- 47 48 Squashfs filesystems now have extended attribute support. The 49 extended attribute implementation has the following features: 50 51 1. Layout can store up to 2^48 bytes of compressed xattr data. 52 2. Number of xattrs per inode unlimited. 53 3. Total size of xattr data per inode 2^48 bytes of compressed data. 54 4. Up to 4 Gbytes of data per xattr value. 55 5. Inline and out-of-line xattr values supported for higher performance 56 in xattr scanning (listxattr & getxattr), and to allow xattr value 57 de-duplication. 58 6. Both whole inode xattr duplicate detection and individual xattr value 59 duplicate detection supported. These can obviously nest, file C's 60 xattrs can be a complete duplicate of file B, and file B's xattrs 61 can be a partial duplicate of file A. 62 7. Xattr name prefix types stored, allowing the redundant "user.", "trusted." 63 etc. characters to be eliminated and more concisely stored. 64 8. Support for files, directories, symbolic links, device nodes, fifos 65 and sockets. 66 67 Extended attribute support is in 2.6.35 and later kernels. Filesystems 68 with extended attributes can be mounted on 2.6.29 and later kernels, the 69 extended attributes will be ignored with a warning. 70 71 2. USING SQUASHFS 72 ----------------- 73 74 Squashfs filesystems should be mounted with 'mount' with the filesystem type 75 'squashfs'. If the filesystem is on a block device, the filesystem can be 76 mounted directly, e.g. 77 78 %mount -t squashfs /dev/sda1 /mnt 79 80 Will mount the squashfs filesystem on "/dev/sda1" under the directory "/mnt". 81 82 If the squashfs filesystem has been written to a file, the loopback device 83 can be used to mount it (loopback support must be in the kernel), e.g. 84 85 %mount -t squashfs image /mnt -o loop 86 87 Will mount the squashfs filesystem in the file "image" under 88 the directory "/mnt". 89 90 3. MKSQUASHFS 91 ------------- 92 93 3.1 Mksquashfs options and overview 94 ----------------------------------- 95 96 As squashfs is a read-only filesystem, the mksquashfs program must be used to 97 create populated squashfs filesystems. 98 99 SYNTAX:./mksquashfs source1 source2 ... dest [options] [-e list of exclude 100 dirs/files] 101 102 Filesystem build options: 103 -comp <comp> select <comp> compression 104 Compressors available: 105 gzip (default) 106 lzo 107 lz4 108 xz 109 -b <block_size> set data block to <block_size>. Default 128 Kbytes 110 Optionally a suffix of K or M can be given to specify 111 Kbytes or Mbytes respectively 112 -no-exports don't make the filesystem exportable via NFS 113 -no-sparse don't detect sparse files 114 -no-xattrs don't store extended attributes 115 -xattrs store extended attributes (default) 116 -noI do not compress inode table 117 -noD do not compress data blocks 118 -noF do not compress fragment blocks 119 -noX do not compress extended attributes 120 -no-fragments do not use fragments 121 -always-use-fragments use fragment blocks for files larger than block size 122 -no-duplicates do not perform duplicate checking 123 -all-root make all files owned by root 124 -force-uid uid set all file uids to uid 125 -force-gid gid set all file gids to gid 126 -nopad do not pad filesystem to a multiple of 4K 127 -keep-as-directory if one source directory is specified, create a root 128 directory containing that directory, rather than the 129 contents of the directory 130 131 Filesystem filter options: 132 -p <pseudo-definition> Add pseudo file definition 133 -pf <pseudo-file> Add list of pseudo file definitions 134 -sort <sort_file> sort files according to priorities in <sort_file>. One 135 file or dir with priority per line. Priority -32768 to 136 32767, default priority 0 137 -ef <exclude_file> list of exclude dirs/files. One per line 138 -wildcards Allow extended shell wildcards (globbing) to be used in 139 exclude dirs/files 140 -regex Allow POSIX regular expressions to be used in exclude 141 dirs/files 142 143 Filesystem append options: 144 -noappend do not append to existing filesystem 145 -root-becomes <name> when appending source files/directories, make the 146 original root become a subdirectory in the new root 147 called <name>, rather than adding the new source items 148 to the original root 149 150 Mksquashfs runtime options: 151 -version print version, licence and copyright message 152 -exit-on-error treat normally ignored errors as fatal 153 -recover <name> recover filesystem data using recovery file <name> 154 -no-recovery don't generate a recovery file 155 -info print files written to filesystem 156 -no-progress don't display the progress bar 157 -progress display progress bar when using the -info option 158 -processors <number> Use <number> processors. By default will use number of 159 processors available 160 -mem <size> Use <size> physical memory. Currently set to 1922M 161 Optionally a suffix of K, M or G can be given to specify 162 Kbytes, Mbytes or Gbytes respectively 163 164 Miscellaneous options: 165 -root-owned alternative name for -all-root 166 -noInodeCompression alternative name for -noI 167 -noDataCompression alternative name for -noD 168 -noFragmentCompression alternative name for -noF 169 -noXattrCompression alternative name for -noX 170 171 -Xhelp print compressor options for selected compressor 172 173 Compressors available and compressor specific options: 174 gzip (default) 175 -Xcompression-level <compression-level> 176 <compression-level> should be 1 .. 9 (default 9) 177 -Xwindow-size <window-size> 178 <window-size> should be 8 .. 15 (default 15) 179 -Xstrategy strategy1,strategy2,...,strategyN 180 Compress using strategy1,strategy2,...,strategyN in turn 181 and choose the best compression. 182 Available strategies: default, filtered, huffman_only, 183 run_length_encoded and fixed 184 lzo 185 -Xalgorithm <algorithm> 186 Where <algorithm> is one of: 187 lzo1x_1 188 lzo1x_1_11 189 lzo1x_1_12 190 lzo1x_1_15 191 lzo1x_999 (default) 192 -Xcompression-level <compression-level> 193 <compression-level> should be 1 .. 9 (default 8) 194 Only applies to lzo1x_999 algorithm 195 lz4 196 -Xhc 197 Compress using LZ4 High Compression 198 xz 199 -Xbcj filter1,filter2,...,filterN 200 Compress using filter1,filter2,...,filterN in turn 201 (in addition to no filter), and choose the best compression. 202 Available filters: x86, arm, armthumb, powerpc, sparc, ia64 203 -Xdict-size <dict-size> 204 Use <dict-size> as the XZ dictionary size. The dictionary size 205 can be specified as a percentage of the block size, or as an 206 absolute value. The dictionary size must be less than or equal 207 to the block size and 8192 bytes or larger. It must also be 208 storable in the xz header as either 2^n or as 2^n+2^(n+1). 209 Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K 210 etc. 211 212 Source1 source2 ... are the source directories/files containing the 213 files/directories that will form the squashfs filesystem. If a single 214 directory is specified (i.e. mksquashfs source output_fs) the squashfs 215 filesystem will consist of that directory, with the top-level root 216 directory corresponding to the source directory. 217 218 If multiple source directories or files are specified, mksquashfs will merge 219 the specified sources into a single filesystem, with the root directory 220 containing each of the source files/directories. The name of each directory 221 entry will be the basename of the source path. If more than one source 222 entry maps to the same name, the conflicts are named xxx_1, xxx_2, etc. where 223 xxx is the original name. 224 225 To make this clear, take two example directories. Source directory 226 "/home/phillip/test" contains "file1", "file2" and "dir1". 227 Source directory "goodies" contains "goodies1", "goodies2" and "goodies3". 228 229 usage example 1: 230 231 %mksquashfs /home/phillip/test output_fs 232 233 This will generate a squashfs filesystem with root entries 234 "file1", "file2" and "dir1". 235 236 example 2: 237 238 %mksquashfs /home/phillip/test goodies output_fs 239 240 This will create a squashfs filesystem with the root containing 241 entries "test" and "goodies" corresponding to the source 242 directories "/home/phillip/test" and "goodies". 243 244 example 3: 245 246 %mksquashfs /home/phillip/test goodies test output_fs 247 248 This is the same as the previous example, except a third 249 source directory "test" has been specified. This conflicts 250 with the first directory named "test" and will be renamed "test_1". 251 252 Multiple sources allow filesystems to be generated without needing to 253 copy all source files into a common directory. This simplifies creating 254 filesystems. 255 256 The -keep-as-directory option can be used when only one source directory 257 is specified, and you wish the root to contain that directory, rather than 258 the contents of the directory. For example: 259 260 example 4: 261 262 %mksquashfs /home/phillip/test output_fs -keep-as-directory 263 264 This is the same as example 1, except for -keep-as-directory. 265 This will generate a root directory containing directory "test", 266 rather than the "test" directory contents "file1", "file2" and "dir1". 267 268 The Dest argument is the destination where the squashfs filesystem will be 269 written. This can either be a conventional file or a block device. If the file 270 doesn't exist it will be created, if it does exist and a squashfs 271 filesystem exists on it, mksquashfs will append. The -noappend option will 272 write a new filesystem irrespective of whether an existing filesystem is 273 present. 274 275 3.2 Changing compression algorithm and compression specific options 276 ------------------------------------------------------------------- 277 278 By default Mksquashfs will compress using the gzip compression 279 algorithm. This algorithm offers a good trade-off between compression 280 ratio, and memory and time taken to decompress. 281 282 Squashfs also supports LZ4, LZO and XZ (LZMA2) compression. LZO offers worse 283 compression ratio than gzip, but is faster to decompress. XZ offers better 284 compression ratio than gzip, but at the expense of greater memory and time 285 to decompress (and significantly more time to compress). LZ4 is similar 286 to LZO, but, support for it is not yet in the mainline kernel, and so 287 its usefulness is currently limited to using Squashfs with Mksquashfs/Unsquashfs 288 as an archival system like tar. 289 290 If you're not building the squashfs-tools and kernel from source, then 291 the tools and kernel may or may not have been built with support for LZ4, LZO or 292 XZ compression. The compression algorithms supported by the build of 293 Mksquashfs can be found by typing mksquashfs without any arguments. The 294 compressors available are displayed at the end of the help message, e.g. 295 296 Compressors available and compressor specific options: 297 gzip (default) 298 -Xcompression-level <compression-level> 299 <compression-level> should be 1 .. 9 (default 9) 300 -Xwindow-size <window-size> 301 <window-size> should be 8 .. 15 (default 15) 302 -Xstrategy strategy1,strategy2,...,strategyN 303 Compress using strategy1,strategy2,...,strategyN in turn 304 and choose the best compression. 305 Available strategies: default, filtered, huffman_only, 306 run_length_encoded and fixed 307 lzo 308 -Xalgorithm <algorithm> 309 Where <algorithm> is one of: 310 lzo1x_1 311 lzo1x_1_11 312 lzo1x_1_12 313 lzo1x_1_15 314 lzo1x_999 (default) 315 -Xcompression-level <compression-level> 316 <compression-level> should be 1 .. 9 (default 8) 317 Only applies to lzo1x_999 algorithm 318 lz4 319 -Xhc 320 Compress using LZ4 High Compression 321 xz 322 -Xbcj filter1,filter2,...,filterN 323 Compress using filter1,filter2,...,filterN in turn 324 (in addition to no filter), and choose the best compression. 325 Available filters: x86, arm, armthumb, powerpc, sparc, ia64 326 -Xdict-size <dict-size> 327 Use <dict-size> as the XZ dictionary size. The dictionary size 328 can be specified as a percentage of the block size, or as an 329 absolute value. The dictionary size must be less than or equal 330 to the block size and 8192 bytes or larger. It must also be 331 storable in the xz header as either 2^n or as 2^n+2^(n+1). 332 Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K 333 etc. 334 335 If the compressor offers compression specific options (all the compressors now 336 have compression specific options except the deprecated lzma1 compressor) 337 then these options are also displayed (.i.e. in the above XZ is shown with two 338 compression specific options). The compression specific options are, obviously, 339 specific to the compressor in question, and the compressor documentation and 340 web sites should be consulted to understand their behaviour. In general 341 the Mksquashfs compression defaults for each compressor are optimised to 342 give the best performance for each compressor, where what constitutes 343 best depends on the compressor. For gzip/xz best means highest compression, 344 for LZO/LZ4 best means a tradeoff between compression and (de)-compression 345 overhead (LZO/LZ4 by definition are intended for weaker processors). 346 347 3.3 Changing global compression defaults used in mksquashfs 348 ----------------------------------------------------------- 349 350 There are a large number of options that can be used to control the 351 compression in mksquashfs. By and large the defaults are the most 352 optimum settings and should only be changed in exceptional circumstances! 353 Note, this does not apply to the block size, increasing the block size 354 from the default of 128Kbytes will increase compression (especially 355 for the xz compressor) and should increase I/O performance too. However, 356 a block size of greater than 128Kbytes may increase latency in certain 357 cases (where the filesystem contains lots of fragments, and no locality 358 of reference is observed). For this reason the block size default is 359 configured to the less optimal 128Kbytes. Users should experiment 360 with 256Kbyte sizes or above. 361 362 The -noI, -noD and -noF options (also -noInodeCompression, -noDataCompression 363 and -noFragmentCompression) can be used to force mksquashfs to not compress 364 inodes/directories, data and fragments respectively. Giving all options 365 generates an uncompressed filesystem. 366 367 The -no-fragments tells mksquashfs to not generate fragment blocks, and rather 368 generate a filesystem similar to a Squashfs 1.x filesystem. It will of course 369 still be a Squashfs 4.0 filesystem but without fragments, and so it won't be 370 mountable on a Squashfs 1.x system. 371 372 The -always-use-fragments option tells mksquashfs to always generate 373 fragments for files irrespective of the file length. By default only small 374 files less than the block size are packed into fragment blocks. The ends of 375 files which do not fit fully into a block, are NOT by default packed into 376 fragments. To illustrate this, a 100K file has an initial 64K block and a 36K 377 remainder. This 36K remainder is not packed into a fragment by default. This 378 is because to do so leads to a 10 - 20% drop in sequential I/O performance, as a 379 disk head seek is needed to seek to the initial file data and another disk seek 380 is need to seek to the fragment block. Specify this option if you want file 381 remainders to be packed into fragment blocks. Doing so may increase the 382 compression obtained BUT at the expense of I/O speed. 383 384 The -no-duplicates option tells mksquashfs to not check the files being 385 added to the filesystem for duplicates. This can result in quicker filesystem 386 generation and appending although obviously compression will suffer badly if 387 there is a lot of duplicate files. 388 389 The -b option allows the block size to be selected, both "K" and "M" postfixes 390 are supported, this can be either 4K, 8K, 16K, 32K, 64K, 128K, 256K, 512K or 391 1M bytes. 392 393 3.4 Specifying the UIDs/GIDs used in the filesystem 394 --------------------------------------------------- 395 396 By default files in the generated filesystem inherit the UID and GID ownership 397 of the original file. However, mksquashfs provides a number of options which 398 can be used to override the ownership. 399 400 The options -all-root and -root-owned (both do exactly the same thing) force all 401 file uids/gids in the generated Squashfs filesystem to be root. This allows 402 root owned filesystems to be built without root access on the host machine. 403 404 The "-force-uid uid" option forces all files in the generated Squashfs 405 filesystem to be owned by the specified uid. The uid can be specified either by 406 name (i.e. "root") or by number. 407 408 The "-force-gid gid" option forces all files in the generated Squashfs 409 filesystem to be group owned by the specified gid. The gid can be specified 410 either by name (i.e. "root") or by number. 411 412 3.5 Excluding files from the filesystem 413 --------------------------------------- 414 415 The -e and -ef options allow files/directories to be specified which are 416 excluded from the output filesystem. The -e option takes the exclude 417 files/directories from the command line, the -ef option takes the 418 exlude files/directories from the specified exclude file, one file/directory 419 per line. 420 421 Two styles of exclude file matching are supported: basic exclude matching, and 422 extended wildcard matching. Basic exclude matching is a legacy feature 423 retained for backwards compatibility with earlier versions of Mksquashfs. 424 Extended wildcard matching should be used in preference. 425 426 3.5.1 Basic exclude matching 427 ---------------------------- 428 429 Each exclude file is treated as an exact match of a file/directory in 430 the source directories. If an exclude file/directory is absolute (i.e. 431 prefixed with /, ../, or ./) the entry is treated as absolute, however, if an 432 exclude file/directory is relative, it is treated as being relative to each of 433 the sources in turn, i.e. 434 435 %mksquashfs /tmp/source1 source2 output_fs -e ex1 /tmp/source1/ex2 out/ex3 436 437 Will generate exclude files /tmp/source1/ex2, /tmp/source1/ex1, source2/ex1, 438 /tmp/source1/out/ex3 and source2/out/ex3. 439 440 3.5.2 Extended exclude file handling 441 ------------------------------------ 442 443 Extended exclude file matching treats each exclude file as a wildcard or 444 regex expression. To enable wildcard matching specify the -wildcards 445 option, and to enable regex matching specify the -regex option. In most 446 cases the -wildcards option should be used rather than -regex because wildcard 447 matching behaviour is significantly easier to understand! 448 449 In addition to wildcards/regex expressions, exclude files can be "anchored" or 450 "non-anchored". An anchored exclude is one which matches from the root of the 451 directory and nowhere else, a non-anchored exclude matches anywhere. For 452 example given the directory hierarchy "a/b/c/a/b", the anchored exclude 453 "a/b" will match "a/b" at the root of the directory hierarchy, but 454 it will not match the "/a/b" sub-directory within directory "c", whereas a 455 non-anchored exclude would. 456 457 A couple of examples should make this clearer. 458 459 Anchored excludes 460 461 1. mksquashfs example image.sqsh -wildcards -e 'test/*.gz' 462 463 Exclude all files matching "*.gz" in the top level directory "test". 464 465 2. mksquashfs example image.sqsh -wildcards -e '*/[Tt]est/example*' 466 467 Exclude all files beginning with "example" inside directories called 468 "Test" or "test", that occur inside any top level directory. 469 470 Using extended wildcards, negative matching is also possible. 471 472 3. mksquashfs example image.sqsh -wildcards -e 'test/!(*data*).gz' 473 474 Exclude all files matching "*.gz" in top level directory "test", 475 except those with "data" in the name. 476 477 Non-anchored excludes 478 479 By default excludes match from the top level directory, but it is 480 often useful to exclude a file matching anywhere in the source directories. 481 For this non-anchored excludes can be used, specified by pre-fixing the 482 exclude with "...". 483 484 Examples: 485 486 1. mksquashfs example image.sqsh -wildcards -e '... *.gz' 487 488 Exclude files matching "*.gz" anywhere in the source directories. 489 For example this will match "example.gz", "test/example.gz", and 490 "test/test/example.gz". 491 492 2. mksquashfs example image.sqsh -wildcards -e '... [Tt]est/*.gz' 493 494 Exclude files matching "*.gz" inside directories called "Test" or 495 "test" that occur anywhere in the source directories. 496 497 Again, using extended wildcards, negative matching is also possible. 498 499 3. mksquashfs example image.sqsh -wildcards -e '... !(*data*).gz' 500 501 Exclude all files matching "*.gz" anywhere in the source directories, 502 except those with "data" in the name. 503 504 3.5.3 Exclude files summary 505 --------------------------- 506 507 The -e and -ef exclude options are usefully used in archiving the entire 508 filesystem, where it is wished to avoid archiving /proc, and the filesystem 509 being generated, i.e. 510 511 %mksquashfs / /tmp/root.sqsh -e proc /tmp/root.sqsh 512 513 Multiple -ef options can be specified on the command line, and the -ef 514 option can be used in conjuction with the -e option. 515 516 3.6 Appending to squashfs filesystems 517 ------------------------------------- 518 519 Running squashfs with the destination directory containing an existing 520 filesystem will add the source items to the existing filesystem. By default, 521 the source items are added to the existing root directory. 522 523 To make this clear... An existing filesystem "image" contains root entries 524 "old1", and "old2". Source directory "/home/phillip/test" contains "file1", 525 "file2" and "dir1". 526 527 example 1: 528 529 %mksquashfs /home/phillip/test image 530 531 Will create a new "image" with root entries "old1", "old2", "file1", "file2" and 532 "dir1" 533 534 example 2: 535 536 %mksquashfs /home/phillip/test image -keep-as-directory 537 538 Will create a new "image" with root entries "old1", "old2", and "test". 539 As shown in the previous section, for single source directories 540 '-keep-as-directory' adds the source directory rather than the 541 contents of the directory. 542 543 example 3: 544 545 %mksquashfs /home/phillip/test image -keep-as-directory -root-becomes 546 original-root 547 548 Will create a new "image" with root entries "original-root", and "test". The 549 '-root-becomes' option specifies that the original root becomes a subdirectory 550 in the new root, with the specified name. 551 552 The append option with file duplicate detection, means squashfs can be 553 used as a simple versioning archiving filesystem. A squashfs filesystem can 554 be created with for example the linux-2.4.19 source. Appending the linux-2.4.20 555 source will create a filesystem with the two source trees, but only the 556 changed files will take extra room, the unchanged files will be detected as 557 duplicates. 558 559 3.7 Appending recovery file feature 560 ----------------------------------- 561 562 Recovery files are created when appending to existing Squashfs 563 filesystems. This allows the original filesystem to be recovered 564 if Mksquashfs aborts unexpectedly (i.e. power failure). 565 566 The recovery files are called squashfs_recovery_xxx_yyy, where 567 "xxx" is the name of the filesystem being appended to, and "yyy" is a 568 number to guarantee filename uniqueness (the PID of the parent Mksquashfs 569 process). 570 571 Normally if Mksquashfs exits correctly the recovery file is deleted to 572 avoid cluttering the filesystem. If Mksquashfs aborts, the "-recover" 573 option can be used to recover the filesystem, giving the previously 574 created recovery file as a parameter, i.e. 575 576 mksquashfs dummy image.sqsh -recover squashfs_recovery_image.sqsh_1234 577 578 The writing of the recovery file can be disabled by specifying the 579 "-no-recovery" option. 580 581 3.8 Pseudo file support 582 ----------------------- 583 584 Mksquashfs supports pseudo files, these allow fake files, directories, character 585 and block devices to be specified and added to the Squashfs filesystem being 586 built, rather than requiring them to be present in the source directories. 587 This, for example, allows device nodes to be added to the filesystem without 588 requiring root access. 589 590 Mksquashfs 4.1 added support for "dynamic pseudo files" and a modify operation. 591 Dynamic pseudo files allow files to be dynamically created when Mksquashfs 592 is run, their contents being the result of running a command or piece of 593 shell script. The modifiy operation allows the mode/uid/gid of an existing 594 file in the source filesystem to be modified. 595 596 Two Mksquashfs options are supported, -p allows one pseudo file to be specified 597 on the command line, and -pf allows a pseudo file to be specified containing a 598 list of pseduo definitions, one per line. 599 600 3.8.1. Creating a dynamic file 601 ------------------------------ 602 603 Pseudo definition 604 605 Filename f mode uid gid command 606 607 mode is the octal mode specifier, similar to that expected by chmod. 608 609 uid and gid can be either specified as a decimal number, or by name. 610 611 command can be an executable or a piece of shell script, and it is executed 612 by running "/bin/sh -c command". The stdout becomes the contents of 613 "Filename". 614 615 Examples: 616 617 Running a basic command 618 ----------------------- 619 620 /somedir/dmesg f 444 root root dmesg 621 622 creates a file "/somedir/dmesg" containing the output from dmesg. 623 624 Executing shell script 625 ---------------------- 626 627 RELEASE f 444 root root \ 628 if [ ! -e /tmp/ver ]; then \ 629 echo 0 > /tmp/ver; \ 630 fi; \ 631 ver=`cat /tmp/ver`; \ 632 ver=$((ver +1)); \ 633 echo $ver > /tmp/ver; \ 634 echo -n `cat /tmp/release`; \ 635 echo "-dev #"$ver `date` "Build host" `hostname` 636 637 Creates a file RELEASE containing the release name, date, build host, and 638 an incrementing version number. The incrementing version is a side-effect 639 of executing the shell script, and ensures every time Mksquashfs is run a 640 new version number is used without requiring any other shell scripting. 641 642 The above example also shows that commands can be split across multiple lines 643 using "\". Obviously as the script will be presented to the shell as a single 644 line, a semicolon is need to separate individual shell commands within the 645 shell script. 646 647 Reading from a device (or fifo/named socket) 648 -------------------------------------------- 649 650 input f 444 root root dd if=/dev/sda1 bs=1024 count=10 651 652 Copies 10K from the device /dev/sda1 into the file input. Ordinarily Mksquashfs 653 given a device, fifo, or named socket will place that special file within the 654 Squashfs filesystem, the above allows input from these special files to be 655 captured and placed in the Squashfs filesystem. 656 657 3.8.2. Creating a block or character device 658 ------------------------------------------- 659 660 Pseudo definition 661 662 Filename type mode uid gid major minor 663 664 Where type is either 665 b - for block devices, and 666 c - for character devices 667 668 mode is the octal mode specifier, similar to that expected by chmod. 669 670 uid and gid can be either specified as a decimal number, or by name. 671 672 For example: 673 674 /dev/chr_dev c 666 root root 100 1 675 /dev/blk_dev b 666 0 0 200 200 676 677 creates a character device "/dev/chr_dev" with major:minor 100:1 and 678 a block device "/dev/blk_dev" with major:minor 200:200, both with root 679 uid/gid and a mode of rw-rw-rw. 680 681 3.8.3. Creating a directory 682 --------------------------- 683 684 Pseudo definition 685 686 Filename d mode uid gid 687 688 mode is the octal mode specifier, similar to that expected by chmod. 689 690 uid and gid can be either specified as a decimal number, or by name. 691 692 For example: 693 694 /pseudo_dir d 666 root root 695 696 creates a directory "/pseudo_dir" with root uid/gid and mode of rw-rw-rw. 697 698 3.8.4. Modifying attributes of an existing file 699 ----------------------------------------------- 700 701 Pseudo definition 702 703 Filename m mode uid gid 704 705 mode is the octal mode specifier, similar to that expected by chmod. 706 707 uid and gid can be either specified as a decimal number, or by name. 708 709 For example: 710 711 dmesg m 666 root root 712 713 Changes the attributes of the file "dmesg" in the filesystem to have 714 root uid/gid and a mode of rw-rw-rw, overriding the attributes obtained 715 from the source filesystem. 716 717 3.9 Miscellaneous options 718 ------------------------- 719 720 The -info option displays the files/directories as they are compressed and 721 added to the filesystem. The original uncompressed size of each file 722 is printed, along with DUPLICATE if the file is a duplicate of a 723 file in the filesystem. 724 725 The -nopad option informs mksquashfs to not pad the filesystem to a 4K multiple. 726 This is performed by default to enable the output filesystem file to be mounted 727 by loopback, which requires files to be a 4K multiple. If the filesystem is 728 being written to a block device, or is to be stored in a bootimage, the extra 729 pad bytes are not needed. 730 731 4. UNSQUASHFS 732 ------------- 733 734 Unsquashfs allows you to decompress and extract a Squashfs filesystem without 735 mounting it. It can extract the entire filesystem, or a specific 736 file or directory. 737 738 The Unsquashfs usage info is: 739 740 SYNTAX: ./unsquashfs [options] filesystem [directories or files to extract] 741 -v[ersion] print version, licence and copyright information 742 -d[est] <pathname> unsquash to <pathname>, default "squashfs-root" 743 -n[o-progress] don't display the progress bar 744 -no[-xattrs] don't extract xattrs in file system 745 -x[attrs] extract xattrs in file system (default) 746 -u[ser-xattrs] only extract user xattrs in file system. 747 Enables extracting xattrs 748 -p[rocessors] <number> use <number> processors. By default will use 749 number of processors available 750 -i[nfo] print files as they are unsquashed 751 -li[nfo] print files as they are unsquashed with file 752 attributes (like ls -l output) 753 -l[s] list filesystem, but don't unsquash 754 -ll[s] list filesystem with file attributes (like 755 ls -l output), but don't unsquash 756 -f[orce] if file already exists then overwrite 757 -s[tat] display filesystem superblock information 758 -e[f] <extract file> list of directories or files to extract. 759 One per line 760 -da[ta-queue] <size> Set data queue to <size> Mbytes. Default 256 761 Mbytes 762 -fr[ag-queue] <size> Set fragment queue to <size> Mbytes. Default 763 256 Mbytes 764 -r[egex] treat extract names as POSIX regular expressions 765 rather than use the default shell wildcard 766 expansion (globbing) 767 768 Decompressors available: 769 gzip 770 lzo 771 lz4 772 xz 773 774 To extract a subset of the filesystem, the filenames or directory 775 trees that are to be extracted can be specified on the command line. The 776 files/directories should be specified using the full path to the 777 files/directories as they appear within the Squashfs filesystem. The 778 files/directories will also be extracted to those positions within the specified 779 destination directory. 780 781 The extract files can also be given in a file using the "-e[f]" option. 782 783 Similarly to Mksquashfs, wildcard matching is performed on the extract 784 files. Wildcard matching is enabled by default. 785 786 Examples: 787 788 1. unsquashfs image.sqsh 'test/*.gz' 789 790 Extract all files matching "*.gz" in the top level directory "test". 791 792 2. unsquashfs image.sqsh '[Tt]est/example*' 793 794 Extract all files beginning with "example" inside top level directories 795 called "Test" or "test". 796 797 Using extended wildcards, negative matching is also possible. 798 799 3. unsquashfs image.sqsh 'test/!(*data*).gz' 800 801 Extract all files matching "*.gz" in top level directory "test", 802 except those with "data" in the name. 803 804 805 4.1 Unsquashfs options 806 ---------------------- 807 808 The "-ls" option can be used to list the contents of a filesystem without 809 decompressing the filesystem data itself. The "-lls" option is similar 810 but it also displays file attributes (ls -l style output). 811 812 The "-info" option forces Unsquashfs to print each file as it is decompressed. 813 The -"linfo" is similar but it also displays file attributes. 814 815 The "-dest" option specifies the directory that is used to decompress 816 the filesystem data. If this option is not given then the filesystem is 817 decompressed to the directory "squashfs-root" in the current working directory. 818 819 The "-force" option forces Unsquashfs to output to the destination 820 directory even if files or directories already exist. This allows you 821 to update an existing directory tree, or to Unsquashfs to a partially 822 filled directory. Without the "-force" option, Unsquashfs will 823 refuse to overwrite any existing files, or to create any directories if they 824 already exist. This is done to protect data in case of mistakes, and 825 so the "-force" option should be used with caution. 826 827 The "-stat" option displays filesystem superblock information. This is 828 useful to discover the filesystem version, byte ordering, whether it has a NFS 829 export table, and what options were used to compress the filesystem, etc. 830 831 Unsquashfs can decompress all Squashfs filesystem versions, 1.x, 2.x, 3.x and 832 4.0 filesystems. 833 834 5. FILESYSTEM LAYOUT 835 -------------------- 836 837 A squashfs filesystem consists of a maximum of nine parts, packed together on a 838 byte alignment: 839 840 --------------- 841 | superblock | 842 |---------------| 843 | compression | 844 | options | 845 |---------------| 846 | datablocks | 847 | & fragments | 848 |---------------| 849 | inode table | 850 |---------------| 851 | directory | 852 | table | 853 |---------------| 854 | fragment | 855 | table | 856 |---------------| 857 | export | 858 | table | 859 |---------------| 860 | uid/gid | 861 | lookup table | 862 |---------------| 863 | xattr | 864 | table | 865 --------------- 866 867 Compressed data blocks are written to the filesystem as files are read from 868 the source directory, and checked for duplicates. Once all file data has been 869 written the completed super-block, compression options, inode, directory, 870 fragment, export, uid/gid lookup and xattr tables are written. 871 872 5.1 Compression options 873 ----------------------- 874 875 Compressors can optionally support compression specific options (e.g. 876 dictionary size). If non-default compression options have been used, then 877 these are stored here. 878 879 5.2 Inodes 880 ---------- 881 882 Metadata (inodes and directories) are compressed in 8Kbyte blocks. Each 883 compressed block is prefixed by a two byte length, the top bit is set if the 884 block is uncompressed. A block will be uncompressed if the -noI option is set, 885 or if the compressed block was larger than the uncompressed block. 886 887 Inodes are packed into the metadata blocks, and are not aligned to block 888 boundaries, therefore inodes overlap compressed blocks. Inodes are identified 889 by a 48-bit number which encodes the location of the compressed metadata block 890 containing the inode, and the byte offset into that block where the inode is 891 placed (<block, offset>). 892 893 To maximise compression there are different inodes for each file type 894 (regular file, directory, device, etc.), the inode contents and length 895 varying with the type. 896 897 To further maximise compression, two types of regular file inode and 898 directory inode are defined: inodes optimised for frequently occurring 899 regular files and directories, and extended types where extra 900 information has to be stored. 901 902 5.3 Directories 903 --------------- 904 905 Like inodes, directories are packed into compressed metadata blocks, stored 906 in a directory table. Directories are accessed using the start address of 907 the metablock containing the directory and the offset into the 908 decompressed block (<block, offset>). 909 910 Directories are organised in a slightly complex way, and are not simply 911 a list of file names. The organisation takes advantage of the 912 fact that (in most cases) the inodes of the files will be in the same 913 compressed metadata block, and therefore, can share the start block. 914 Directories are therefore organised in a two level list, a directory 915 header containing the shared start block value, and a sequence of directory 916 entries, each of which share the shared start block. A new directory header 917 is written once/if the inode start block changes. The directory 918 header/directory entry list is repeated as many times as necessary. 919 920 Directories are sorted, and can contain a directory index to speed up 921 file lookup. Directory indexes store one entry per metablock, each entry 922 storing the index/filename mapping to the first directory header 923 in each metadata block. Directories are sorted in alphabetical order, 924 and at lookup the index is scanned linearly looking for the first filename 925 alphabetically larger than the filename being looked up. At this point the 926 location of the metadata block the filename is in has been found. 927 The general idea of the index is ensure only one metadata block needs to be 928 decompressed to do a lookup irrespective of the length of the directory. 929 This scheme has the advantage that it doesn't require extra memory overhead 930 and doesn't require much extra storage on disk. 931 932 5.4 File data 933 ------------- 934 935 Regular files consist of a sequence of contiguous compressed blocks, and/or a 936 compressed fragment block (tail-end packed block). The compressed size 937 of each datablock is stored in a block list contained within the 938 file inode. 939 940 To speed up access to datablocks when reading 'large' files (256 Mbytes or 941 larger), the code implements an index cache that caches the mapping from 942 block index to datablock location on disk. 943 944 The index cache allows Squashfs to handle large files (up to 1.75 TiB) while 945 retaining a simple and space-efficient block list on disk. The cache 946 is split into slots, caching up to eight 224 GiB files (128 KiB blocks). 947 Larger files use multiple slots, with 1.75 TiB files using all 8 slots. 948 The index cache is designed to be memory efficient, and by default uses 949 16 KiB. 950 951 5.5 Fragment lookup table 952 ------------------------- 953 954 Regular files can contain a fragment index which is mapped to a fragment 955 location on disk and compressed size using a fragment lookup table. This 956 fragment lookup table is itself stored compressed into metadata blocks. 957 A second index table is used to locate these. This second index table for 958 speed of access (and because it is small) is read at mount time and cached 959 in memory. 960 961 5.6 Uid/gid lookup table 962 ------------------------ 963 964 For space efficiency regular files store uid and gid indexes, which are 965 converted to 32-bit uids/gids using an id look up table. This table is 966 stored compressed into metadata blocks. A second index table is used to 967 locate these. This second index table for speed of access (and because it 968 is small) is read at mount time and cached in memory. 969 970 5.7 Export table 971 ---------------- 972 973 To enable Squashfs filesystems to be exportable (via NFS etc.) filesystems 974 can optionally (disabled with the -no-exports Mksquashfs option) contain 975 an inode number to inode disk location lookup table. This is required to 976 enable Squashfs to map inode numbers passed in filehandles to the inode 977 location on disk, which is necessary when the export code reinstantiates 978 expired/flushed inodes. 979 980 This table is stored compressed into metadata blocks. A second index table is 981 used to locate these. This second index table for speed of access (and because 982 it is small) is read at mount time and cached in memory. 983 984 5.8 Xattr table 985 --------------- 986 987 The xattr table contains extended attributes for each inode. The xattrs 988 for each inode are stored in a list, each list entry containing a type, 989 name and value field. The type field encodes the xattr prefix 990 ("user.", "trusted." etc) and it also encodes how the name/value fields 991 should be interpreted. Currently the type indicates whether the value 992 is stored inline (in which case the value field contains the xattr value), 993 or if it is stored out of line (in which case the value field stores a 994 reference to where the actual value is stored). This allows large values 995 to be stored out of line improving scanning and lookup performance and it 996 also allows values to be de-duplicated, the value being stored once, and 997 all other occurences holding an out of line reference to that value. 998 999 The xattr lists are packed into compressed 8K metadata blocks. 1000 To reduce overhead in inodes, rather than storing the on-disk 1001 location of the xattr list inside each inode, a 32-bit xattr id 1002 is stored. This xattr id is mapped into the location of the xattr 1003 list using a second xattr id lookup table. 1004 1005 6. AUTHOR INFO 1006 -------------- 1007 1008 Squashfs was written by Phillip Lougher, email phillip (a] lougher.demon.co.uk, 1009 in Chepstow, Wales, UK. If you like the program, or have any problems, 1010 then please email me, as it's nice to get feedback! 1011
