Home | History | Annotate | only in /external/squashfs-tools/RELEASE-READMEs
Up to higher level directory
NameDateSize
DONATIONS08-Oct-2015663
PERFORMANCE.README08-Oct-20156.4K
pseudo-file.example08-Oct-20152.7K
README-2.008-Oct-20156.3K
README-2.0-AMD6408-Oct-2015958
README-2.108-Oct-20153.5K
README-3.008-Oct-20152K
README-3.108-Oct-20155.1K
README-3.208-Oct-20151,021
README-3.308-Oct-20155.6K
README-4.008-Oct-20151.5K
README-4.108-Oct-20158.2K
README-4.208-Oct-20151.9K
README-4.308-Oct-20157.2K

PERFORMANCE.README

      1 GENERAL INFORMATION ON PERFORMANCE TESTS
      2 ----------------------------------------
      3 
      4 The following performance tests were based on two file sets: the
      5 liveCD filesystem from the Ubuntu liveCD (Warty release), and the
      6 liveCD filesystem from the Damn Small Linux liveCD (release 0.8.4).
      7 The Ubuntu liveCD filesystem was used to test filesystem performance
      8 from CDROM and hard disk for Zisofs, Cloop, Squashfs 2.0 and Squashfs2.1.
      9 CRAMFS filesystem performance could not be tested for this filesystem
     10 bacause it exceeds the maximum supported size of CRAMFS.  To test
     11 CRAMFS performance against Squashfs, the liveCD filesystem from
     12 Damn Small Linux was used.
     13 
     14 NOTE: the usual warnings apply to these results, they are provided for
     15 illustrative purposes only, and due to different hardware and/or file data, you
     16 may obtain different results.  As such the results are provided "as is" without
     17 any warranty (either express or implied) and you assume all risks as to their
     18 quality and accuracy.
     19 
     20 1. Ubuntu liveCD performance tests
     21 
     22    ext3 uncompressed size      1.4 GB
     23    Zisofs compressed size      589.81 MB
     24    Cloop compressed size       471.89 MB
     25    Squashfs2.0 compressed size 448.58 MB
     26    Squashfs2.1 compressed size 448.58 MB
     27 
     28 1.1 Performance tests from CDROM
     29 
     30 1.1.1 Directory Lookup performance
     31 
     32   Time taken to perform "ls -lR --color=alawys | cat > /dev/null" on filesystem
     33   mounted from CDROM
     34 
     35   Zisofs       49.88 seconds (User 2.60 secs, Sys 11.19 secs)
     36   Cloop        20.80 seconds (User 2.71 secs, Sys 13.50 secs)
     37   Squashfs2.0  16.56 seconds (User 2.42 secs, Sys 10.37 secs)
     38   Squashfs2.1  10.14 seconds (User 2.48 secs, Sys 4.44 secs)
     39 
     40 1.1.2 Sequential I/O performance
     41 
     42   Time taken to perform "tar cf - | cat > /dev/null" on filesystem mounted
     43   from CDROM
     44 
     45   Zisofs       27 minutes 28.54 seconds (User 3.00 secs, Sys 1 min 4.80 secs)
     46   Cloop        5 minutes 55.72 seconds (User 2.90 secs, Sys 3 min 37.90 secs)
     47   Squashfs2.0  5 minutes 20.87 seconds (User 2.33 secs, Sys 56.98 secs)
     48   Squashfs2.1  5 minutes 15.46 seconds (user 2.28 secs, Sys 51.12 secs)
     49 
     50 1.1.3 Random I/O performance
     51 
     52   Random access pattern generated by "find /mnt -type f -printf "%s %p\n" | sort
     53   -g | awk '{ printf $2 }' > /tmp/sort
     54 
     55   Time taken to perform "cpio -o --quiet -H newc < /tmp/sort > /dev/null"
     56   on filesystem mounted from CDROM
     57 
     58   Zisofs       101 minutes 29.65 seconds (User 5.33 secs, Sys  1 min 17.20 secs)
     59   Cloop        35 minutes 27.51 seconds (user 5.93 secs, Sys 4 mins 30.23 secs)
     60   Squashfs2.0  21 minutes 53.05 seconds (user 5.71 secs, Sys 2 mins 36.59 secs)
     61   Squashfs2.1  21 minutes 46.99 seconds (User 5.80 secs, Sys 2 mins 31.88 secs)
     62 
     63 
     64 1.2 Performance tests from Hard disk
     65 
     66 1.2.1 Directory Lookup performance
     67 
     68   Time taken to perform "ls -lR --color=alawys | cat > /dev/null" on filesystem
     69   mounted from Hard disk 
     70 
     71   Zisofs       17.29 seconds (User 2.62 secs, Sys 11.08 secs)
     72   Cloop        16.46 seconds (User 2.63 secs, Sys 13.41 secs)
     73   Squashfs2.0  13.75 seconds (User 2.44 secs, Sys 11.00 secs)
     74   Squashfs2.1  6.94 seconds (User 2.44 secs, Sys 4.48 secs)
     75 
     76 1.2.2 Sequential I/O performance
     77 
     78   Time taken to perform "tar cf - | cat > /dev/null" on filesystem mounted
     79   from Hard disk
     80 
     81   Zisofs       1 minute 21.47 seconds (User 2.73 secs, Sys 54.44 secs)
     82   Cloop        1 minute 34.06 seconds (user 2.85 secs, Sys 1 min 12.13 secs)
     83   Squashfs2.0  1 minute 21.22 seconds (User 2.42 secs, Sys 56.21 secs)
     84   Squashfs2.1  1 minute 15.46 seconds (User 2.36 secs, Sys 49.78 secs)
     85 
     86 1.2.3 Random I/O performance
     87 
     88   Random access pattern generated by "find /mnt -type f -printf "%s %p\n" | sort
     89   -g | awk '{ printf $2 }' > /tmp/sort
     90 
     91   Time taken to perform "cpio -o --quiet -H newc < /tmp/sort > /dev/null"
     92   on filesystem mounted from Hard disk
     93 
     94   Zisofs       11 minutes 13.64 seconds (User 5.08 secs, Sys 52.62 secs)
     95   Cloop        5 minutes 37.93 seconds (user 6 secs, Sys 2 mins 22.38 secs)
     96   Squashfs2.0  5 minutes 7.11 seconds (user 5.63 secs, Sys 2 mins 35.23 secs)
     97   Squashfs2.1  5 minutes 1.87 seconds (User 5.71 secs, Sys 2 mins 29.98 secs)
     98 
     99 
    100 2. Damn Small Linux liveCD performance tests
    101 
    102    ext3 uncompressed size      126 MB
    103    CRAMFS compressed size      52.19 MB
    104    Squashfs2.0 compressed size 46.52 MB
    105    Squashfs2.1 compressed size 46.52 MB
    106 
    107 2.1 Performance tests from CDROM
    108 
    109 2.1.1 Directory Lookup performance
    110 
    111   Time taken to perform "ls -lR --color=alawys | cat > /dev/null" on filesystem
    112   mounted from CDROM
    113 
    114   CRAMFS       10.85 seconds (User 0.39 secs, Sys 0.98 secs)
    115   Squashfs2.0  2.97 seconds (User 0.36 secs, Sys 2.15 secs)
    116   Squashfs2.1  2.43 seconds (User 0.40 secs, Sys 1.42 secs)
    117 
    118 2.1.2 Sequential I/O performance
    119 
    120   Time taken to perform "tar cf - | cat > /dev/null" on filesystem mounted
    121   from CDROM
    122 
    123   CRAMFS       55.38 seconds (User 0.34 secs, Sys 6.98 secs)
    124   Squashfs2.0  35.99 seconds (User 0.30 secs, Sys 6.35 secs)
    125   Squashfs2.1  33.83 seconds (User 0.26 secs, Sys 5.56 secs)
    126 
    127 2.1.3 Random I/O performance
    128 
    129   Random access pattern generated by "find /mnt -type f -printf "%s %p\n" | sort
    130   -g | awk '{ printf $2 }' > /tmp/sort
    131 
    132   Time taken to perform "cpio -o --quiet -H newc < /tmp/sort > /dev/null"
    133   on filesystem mounted from CDROM
    134 
    135 
    136   CRAMFS        3 minutes 1.68 seconds (User 0.54 secs, Sys 9.51 secs)
    137   Squashfs2.0   1 minute 39.45 seconds (User 0.57 secs, Sys 13.14 secs)
    138   Squashfs2.1   1 minute 38.41 seconds (User 0.58 secs, Sys 13.08 secs)
    139 
    140 2.2 Performance tests from Hard disk
    141 
    142 2.2.1 Directory Lookup performance
    143 
    144   Time taken to perform "ls -lR --color=alawys | cat > /dev/null" on filesystem
    145   mounted from Hard disk
    146 
    147   CRAMFS       1.77 seconds (User 0.53 secs, Sys 1.21 secs)
    148   Squashfs2.0  2.67 seconds (User 0.41 secs, Sys 2.25 secs)
    149   Squashfs2.1  1.87 seconds (User 0.41 secs, Sys 1.46 secs)
    150 
    151 2.2.2 Sequential I/O performance
    152 
    153   Time taken to perform "tar cf - | cat > /dev/null" on filesystem mounted
    154   from Hard disk 
    155 
    156   CRAMFS       6.80 seconds (User 0.36 secs, Sys 6.02 secs)
    157   Squashfs2.0  7.23 seconds (User 0.29 secs, Sys 6.62 secs)
    158   Squashfs2.1  6.53 seconds (User 0.31 secs, Sys 5.82 secs)
    159 
    160 2.2.3 Random I/O performance
    161 
    162   Random access pattern generated by "find /mnt -type f -printf "%s %p\n" | sort
    163   -g | awk '{ printf $2 }' > /tmp/sort
    164 
    165   Time taken to perform "cpio -o --quiet -H newc < /tmp/sort > /dev/null"
    166   on filesystem mounted from Hard disk 
    167 
    168 
    169   CRAMFS       28.55 seconds (User 0.49 secs, Sys 6.49 secs)
    170   Squashfs2.0  25.44 seconds (User 0.58 secs, Sys 13.17 secs)
    171   Squashfs2.1  24.72 seconds (User 0.56 secs, Sys 13.15 secs)
    172

README-2.0

      1 NOTE:  This the original README for version 2.0.  It is retained as it
      2 contains information about the fragment design.  A description of the new 2.0
      3 mksquashfs options has been added to the main README file, and that
      4 file should now be consulted for these.
      5 
      6 	SQUASHFS 2.0 - A squashed read-only filesystem for Linux
      7 
      8 	Copyright 2004 Phillip Lougher (plougher (a] users.sourceforge.net)
      9 
     10 	Released under the GPL licence (version 2 or later).
     11 
     12 Welcome to the final release of Squashfs version 2.0!  A lot of changes to the
     13 filesystem have been made under the bonnet (hood).  Squashfs 2.0 uses fragment
     14 blocks and larger blocks (64K) to improve compression ratio by about 5 - 20%
     15 over Squashfs 1.0 depending on the files being compressed.  Using fragment
     16 blocks allows Squashfs 2.0 to achieve better compression than cloop and similar
     17 compression to tgz files while retaining the I/O efficiency of a compressed
     18 filesystem.
     19 
     20 Detailed changes:
     21 
     22 1. Squashfs 2.0 has added the concept of fragment blocks (see later discussion).
     23    Files smaller than the file block size (64K in Squashfs 2.0) and optionally
     24    the remainder of files that do not fit fully into a block (i.e. the last 32K
     25    in a 96K file) are packed into shared fragments and compressed together.
     26    This achieves on average 5 - 20% better compression than Squashfs 1.x.
     27 
     28 2. The maximum block size has been increased to 64K.
     29 
     30 3. The maximum number of UIDs has been increased to 256 (from 48 in 1.x).
     31 
     32 4. The maximum number of GIDs has been increased to 256 (from 15 in 1.x).
     33 
     34 5. New mksquashfs -all-root, -root-owned, -force-uid, and -force-gid
     35    options.  These allow the uids/gids of files in the generated
     36    filesystem to be specified, overriding the uids/gids in the
     37    source filesystem.
     38 
     39 6. Initrds are now supported for kernels 2.6.x.
     40 
     41 7. Removal of sleep_on() function call in 2.6.x patch, to allow Squashfs
     42    to work on the Fedora rc2 kernel.
     43 
     44 8. AMD64, check-data and gid bug fixes.
     45 
     46 9. Numerous small bug fixes have been made.
     47 
     48 10. New patch for Linux 2.6.7.
     49 
     50 
     51 New Squashfs 2.0 options
     52 ------------------------
     53 
     54 -noF or -noFragmentCompression
     55 
     56 	Do not compress the fragments.  Added for compatibility with noI and
     57 	noD, probably not that useful.
     58 
     59 -no-fragments
     60 
     61 	Do not use fragment blocks, and rather generate a filesystem
     62 	similar to a Squashfs 1.x filesystem.  It will of course still
     63 	be a Squashfs 2.0 filesystem but without fragments, and so
     64 	it won't be mountable on a Squashfs 1.x system.
     65 
     66 -always-use-fragments
     67 
     68 	By default only small files less than the block size are packed into
     69 	fragment blocks.  The ends of files which do not fit fully into a block,
     70 	are NOT by default packed into fragments.  To illustrate this, a
     71 	100K file has an initial 64K block and a 36K remainder.  This
     72 	36K remainder is not packed into a fragment by default.  This is
     73 	because to do so leads to a 10 - 20% drop in sequential I/O
     74 	performance, as a disk head seek is needed to seek to the initial
     75 	file data and another disk seek is need to seek to the fragment
     76 	block.
     77 
     78 	Specify this option if you want file remainders to be packed into
     79 	fragment blocks.  Doing so may increase the compression obtained
     80 	BUT at the expense of I/O speed.
     81 
     82 -no-duplicates
     83 
     84 	Do not detect duplicate files.
     85 
     86 -all-root
     87 -root-owned
     88 
     89 	These options (both do exactly the same thing), force all file
     90 	uids/gids in the generated Squashfs filesystem to be root.
     91 	This allows root owned filesystems to be built without root access
     92 	on the host machine.
     93 
     94 -force-uid uid
     95 
     96 	This option forces all files in the generated Squashfs filesystem to
     97 	be owned by the specified uid.  The uid can be specified either by
     98 	name (i.e. "root") or by number.
     99 
    100 -force-gid gid
    101 
    102 	This option forces all files in the generated Squashfs filesystem to
    103 	be group owned by the specified gid.  The gid can be specified either by
    104 	name (i.e. "root") or by number.
    105 
    106 
    107 Compression improvements example
    108 --------------------------------
    109 
    110 The following is the compression results obtained compressing the 2.6.6
    111 linux kernel source using CRAMFS, Cloop (with iso filesystem), Squashfs 1.3 and
    112 Squashfs 2.0 (results generated using big-endian filesystems).
    113 
    114 In decreasing order of size:
    115 
    116 	CRAMFS		62791680 bytes (59.9M)
    117 	Squashfs 1.x	51351552 bytes (48.9M)
    118 	Cloop		46118681 bytes (44.0M)
    119 	Squashfs 2.0	45604854 bytes (43.5M)
    120 
    121 
    122 The Squashfs 1.x filesystem is 12.6% larger than the new 2.0 filesystem.
    123 The cloop filesystem is 1.1% larger than the Squashfs 2.0 filesystem.
    124 
    125 
    126 Fragment blocks in Squashfs 2.0
    127 -------------------------------
    128 
    129 Squashfs like all other compressed filesystems compresses files individually
    130 on a block by block basis.  This is performed to allow mounting and
    131 de-compression of files on a block by block basis without requiring the entire
    132 filesystem to be decompressed.  This is in contrast to data-based compression
    133 schemes which compress without understanding the underlying filesystem (i.e.
    134 cloop and tgz files) and which, therefore, do not compress files individually.
    135 Each approach has advantages and disadvantages, data-based systems have better
    136 compression because compression is always performed at the maximum block size
    137 (64K in cloop) irrespective of the size of each file (which could be less than
    138 the block size).  Compressed filesystems tend to be faster at I/O because
    139 they understand the filesystem and therefore employ better caching stategies
    140 and read less un-needed data from the filesystem.
    141 
    142 Fragment blocks in Squashfs 2.0 solves this problem by packing files (and
    143 optionally the ends of files) which are smaller than the block size into
    144 shared blocks, which are compressed together.  For example five files each of
    145 10K will be packed into one shared fragment of 50K and compressed together,
    146 rather than being compressed in five 10K blocks.
    147 
    148 This scheme produces a hybrid filesystem, retaining the I/O efficiency
    149 of a compressed filesystem, while obtaining the compression efficiency
    150 of data-based schemes by compressing small files together.
    151 
    152 
    153 Squashfs 1.x and Squashfs 2.0 compatibility
    154 -------------------------------------------
    155 
    156 Appending to Squashfs 1.x filesystems is not supported.  If you wish to append
    157 to 1.x filesystems, then either use the original mksquashfs, or convert them
    158 to Squashfs 2.0 by mounting the filesystem and running the 2.0 mksquashfs
    159 on the mounted filesystem.
    160 
    161 Mounting Squashfs 1.x filesystems IS supported by the 2.0 kernel patch.
    162

README-2.0-AMD64

      1 Information for amd64 users
      2 ---------------------------
      3 
      4 All previous releases of Squashfs (2.0-alpha and older) generate incorrect
      5 filesystems on amd64 machines.  These filesystems work correctly on amd64
      6 machines, but cannot be mounted on non-amd64 machines.  Likewise, filesystems
      7 generated on non amd64 machines could not be mounted on amd64 machines.
      8 This bug was caused by the different size of the "time_t" definition used in
      9 SquashFS filesystem structures.
     10 
     11 This bug is now fixed in this release.  However, all amd64 filesystems
     12 generated by previous releases will not be mountable on amd64 machines
     13 with this release.  If you have pre-existing amd64 generated filesystems,
     14 it is important that you recreate the filesystem.  This can be performed
     15 by mounting the filesystem using a kernel with the original patch
     16 (i.e. a 2.0-alpha or older patch) and running the SquashFS 2.0
     17 (i.e. this release) mksquashfs tool to create a new SquashFS filesystem.
     18 
     19

README-2.1

      1 	SQUASHFS 2.1 - A squashed read-only filesystem for Linux
      2 
      3 	Copyright 2004 Phillip Lougher (plougher (a] users.sourceforge.net)
      4 
      5 	Released under the GPL licence (version 2 or later).
      6 
      7 Welcome to Squashfs version 2.1-r2.  Squashfs 2.1 introduces indexed
      8 directories which considerably speed up directory lookup (ls, find etc.) for
      9 directories which are greater than 8K in size.  All directories are now also
     10 sorted alphabetically which further speeds up directory lookup.  Many smaller
     11 improvements have also been made to this release, please see the CHANGES file
     12 entry for detailed changes.
     13 
     14 1. DIRECTORY SPEED IMPROVEMENT EXAMPLES
     15 ---------------------------------------
     16 
     17 To give an indication of the directory speed improvements a number of test
     18 results are shown here.  There is in addition a new PERFORMANCE.README file
     19 which gives details of I/O and lookup performance for Squashfs 2.1 against
     20 the Zisofs, Cloop and CRAMFS filesystems.
     21 
     22 example 1:
     23 
     24 Filesystems generated from a single directory of 72,784 files (2.6 MB
     25 directory size).  Each file is 10 bytes in size (the test is directory
     26 lookup and so the file size isn't an issue).  The ext3 uncompressed
     27 directory size is 288 MB (presumably because of one file per block).
     28 
     29 Zisofs compressed size        153.50 MB
     30 Cloop (isofs) compressed size 1.74 MB
     31 Squashfs2.1 compressed size   612 KB (0.60 MB)
     32 
     33 Time taken to perform "ls -lR --color=always | cat > /dev/null" on
     34 filesystems mounted on hard disk.
     35 
     36 Zisofs       35 minutes 7.895 seconds (User 7.868 secs, Sys 34 mins 5.621 secs)
     37 Cloop        35 minutes 12.765 seconds (User 7.771 secs, Sys 34 mins 3.869 secs)
     38 Squashfs2.1  19 seconds (User 5.119 secs, Sys 14.547 secs)
     39 
     40 example 2:
     41 
     42 Filesystems were generated from the Ubuntu Warty livecd (original uncompressed
     43 size on ext3 is 1.4 GB).
     44 
     45 Zisofs compressed size        589.81 MB
     46 Cloop (isofs) compressed size 471.19 MB
     47 Squashfs2.0 compressed size   448.58 MB
     48 Squashfs2.1 compressed size   448.58 MB
     49 
     50 Time taken to perform "ls -lR --color=always | cat > /dev/null" on
     51 filesystems mounted on hard disk.
     52 
     53 Zisofs        49.875 seconds (User time 2.589 secs, Sys 11.194 secs)
     54 Cloop         20.797 seconds (User time 2.706 secs, Sys 13.496 secs)
     55 Squashfs2.0   16.556 seconds (User time 2.424 secs, Sys 10.371 secs)
     56 Squashfs2.1   10.143 seconds (User time 2.475 secs, Sys 4.440 secs)
     57 
     58 
     59 NOTE: the usual warnings apply to these results, they are provided for
     60 illustrative purposes only, and due to different hardware and/or file data, you
     61 may obtain different results.  As such the results are provided "as is" without
     62 any warranty (either express or implied) and you assume all risks as to their
     63 quality and accuracy.
     64 
     65 2. NEW MKSQUASHFS OPTIONS
     66 -------------------------
     67 
     68 There is only one extra option "-2.0".  This tells mksquashfs to generate
     69 a filesystem which is mountable with Squashfs version 2.0.
     70 
     71 3. APPENDING AND MOUNTING SQUASHFS 2.0 FILESYSTEMS
     72 --------------------------------------------------
     73 
     74 Mounting 2.0 filesystems is supported by Squashfs 2.1.  In addition
     75 mksquashfs v2.1 can append to 2.0 filesystems, although the generated
     76 filesystem will still be a 2.0 filesystem.
     77 
     78 4. DONATIONS
     79 ------------
     80 
     81 If you find Squashfs useful then please consider making a donation,
     82 particularly if you use Squashfs in a commercial product.  Please consider
     83 giving something back especially if you're making money from it.
     84 
     85 Off the Squashfs subject somewhat I'm currently looking for another
     86 job doing Linux kernel or filesystems work.  If you know of any such
     87 work that can be performed from the UK then please get in touch.  Thanks.
     88

README-3.0

      1 	SQUASHFS 3.0 - A squashed read-only filesystem for Linux
      2 
      3 	Copyright 2002-2006 Phillip Lougher <phillip (a] lougher.org.uk>
      4 
      5 	Released under the GPL licence (version 2 or later).
      6 
      7 Welcome to the first release of Squashfs version 3.0.  Squashfs 3.0 has the
      8 the following improvements to 2.x.
      9 
     10 	1. Filesystems are no longer limited to 4 GB.  In
     11 	   theory 2^64 or 4 exabytes is now supported.
     12 
     13 	2. Files are no longer limited to 4 GB.  In theory the maximum
     14 	   file size is 4 exabytes.
     15 
     16 	3. Metadata (inode table and directory tables) are no longer
     17 	   restricted to 16 Mbytes.
     18 
     19 	4. Hardlinks are now suppported.
     20 
     21 	5. Nlink counts are now supported.
     22 
     23 	6. Readdir now returns '.' and '..' entries.
     24 
     25 	7. Special support for files larger than 256 MB has been added to
     26 	   the Squashfs kernel code for faster read access.
     27 
     28 	8. Inode numbers are now stored within the inode rather than being
     29 	   computed from inode location on disk (this is not so much an
     30 	   improvement, but a change forced by the previously listed
     31 	   improvements).
     32 
     33 There is a new Unsquashfs utility (in squashfs-tools) than can be used to
     34 decompress a filesystem without mounting it.
     35 
     36 Squashfs 3.0 supports 2.x filesystems.  Support for 1.x filesystems
     37 will be added in the future.
     38 
     39 1. UNSQUASHFS
     40 -------------
     41 
     42 Unsquashfs has the following options:
     43 
     44 SYNTAX: unsquashfs [-ls | -dest] filesystem
     45 	-version		print version, licence and copyright information
     46 	-info			print files as they are unsquashed
     47 	-ls			list filesystem only
     48 	-dest <pathname>	unsquash to <pathname>, default "squashfs-root"
     49 
     50 The "-ls" option can be used to list the contents of a filesystem without
     51 decompressing the filesystem data itself.
     52 
     53 The "-info" option forces Unsquashfs to print each file as it is decompressed.
     54 
     55 The "-dest" option specifies the directory that is used to decompress
     56 the filesystem data.  If this option is not given then the filesystem is
     57 decompressed to the directory "squashfs-root" in the current working directory.
     58 
     59 Unsquashfs can decompress 3.0 filesystems.  Support for 2.x and 1.x
     60 filesystems will be added in the future.
     61

README-3.1

      1 	SQUASHFS 3.1 - A squashed read-only filesystem for Linux
      2 
      3 	Copyright 2002-2006 Phillip Lougher <phillip (a] lougher.org.uk>
      4 
      5 	Released under the GPL licence (version 2 or later).
      6 
      7 Welcome to Squashfs version 3.1-r2.  Squashfs 3.1 has major improvements to
      8 the Squashfs tools (Mksquashfs and Unsquashfs), some major bug fixes, new
      9 kernel patches, and various other smaller improvements and bug fixes.
     10 Please see the CHANGES file for a detailed list.
     11 
     12 1. MKSQUASHFS
     13 -------------
     14 
     15 Mksquashfs has been rewritten and it is now multi-threaded.  It offers
     16 the following improvements:
     17 
     18 1. Parallel compression.  By default as many compression and fragment
     19 compression threads are created as there are available processors.
     20 This significantly speeds up performance on SMP systems.
     21 
     22 2. File input and filesystem output is peformed in parallel on separate
     23 threads to maximise I/O performance.  Even on single processor systems
     24 this speeds up performance by at least 10%.
     25 
     26 3. Appending has been significantly improved, and files within the
     27 filesystem being appended to are no longer scanned and checksummed.  This
     28 significantly improves append time for large filesystems.
     29 
     30 4. File duplicate checking has been optimised, and split into two separate
     31 phases.  Only files which are considered possible duplicates after the
     32 first phase are checksummed and cached in memory.
     33 
     34 5. The use of swap memory was found to significantly impact performance. The
     35 amount of memory used to cache the file is now a command line option, by default
     36 this is 512 Mbytes.
     37 
     38 1.1 NEW COMMAND LINE OPTIONS
     39 ----------------------------
     40 
     41 The new Mksquashfs program has a couple of extra command line options
     42 which can be used to control the new features:
     43 
     44 -processors <processors>
     45 
     46 This specifies the number of processors used by Mksquashfs.
     47 By default this is the number of available processors.
     48 
     49 -read_queue <size in Mbytes>
     50 
     51 This specifies the size of the file input queue used by the reader thread.
     52 This defaults to 64 Mbytes.
     53 
     54 -write_queue <size in Mbytes>
     55 
     56 This specifies the size of the filesystem output queue used by the
     57 writer thread.  It also specifies the maximum cache used in file
     58 duplicate detection (the output queue is shared between these tasks).
     59 This defaults to 512 Mbytes.
     60 
     61 1.2 PERFORMANCE RESULTS
     62 -----------------------
     63 
     64 The following results give an indication of the speed improvements.  Two
     65 example filesystems were tested, a liveCD filesystem (about 1.8 Gbytes
     66 uncompressed), and my home directory consisting largely of text files
     67 (about 1.3 Gbytes uncompressed).  Tests were run on a single core
     68 and a dual core system.
     69 
     70 Dual Core (AMDx2 3800+) system:
     71 Source directories on ext3.
     72 
     73 LiveCD, old mksquashfs:
     74 
     75 real    11m48.401s
     76 user    9m27.056s
     77 sys     0m15.281s
     78 
     79 LiveCD, new par_mksquashfs:
     80 
     81 real    4m8.736s
     82 user    7m11.771s
     83 sys     0m27.749s
     84 
     85 "Home", old mksquashfs:
     86 
     87 real    4m34.360s
     88 user    3m54.007s
     89 sys     0m32.155s
     90 
     91 "Home", new par_mksquashfs:
     92 
     93 real    1m27.381s
     94 user    2m7.304s
     95 sys     0m17.234s
     96 
     97 Single Core PowerBook (PowerPC G4 1.5 GHz Ubuntu Linux)
     98 Source directories on ext3.
     99 
    100 LiveCD, old mksquashs:
    101 
    102 real    11m38.472s
    103 user    9m6.137s
    104 sys     0m23.799s
    105 
    106 LiveCD,  par_mksquashfs:
    107 
    108 real    10m5.572s
    109 user    8m59.921s
    110 sys     0m16.145s
    111 
    112 "Home", old mksquashfs:
    113 
    114 real    3m42.298s
    115 user    2m49.478s
    116 sys     0m13.675s
    117 
    118 "Home", new par_mksquashfs:
    119 
    120 real    3m9.178s
    121 user    2m50.699s
    122 sys     0m9.069s
    123 
    124 I'll be interested in any performance results obtained, especially from SMP
    125 machines larger than my dual-core AMD box, as this will give an indication of
    126 the scalability of the code.  Obviously, I'm also interested in any problems,
    127 deadlocks, low performance etc.
    128 
    129 2. UNSQUASHFS
    130 -------------
    131 
    132 Unsquashfs now allows you to specify the filename or directory that is to be
    133 extracted from the Squashfs filesystem, rather than always extracting the
    134 entire filesystem.  It also has a new "-force" option, and all options can be
    135 specified in a short form (-i rather than -info).
    136 
    137 The Unsquashfs usage info is now:
    138 
    139 SYNTAX: ./unsquashfs [options] filesystem [directory or file to extract]
    140 	-v[ersion]		print version, licence and copyright information
    141 	-i[nfo]			print files as they are unsquashed
    142 	-l[s]			list filesystem only
    143 	-d[est] <pathname>	unsquash to <pathname>, default "squashfs-root"
    144 	-f[orce]		if file already exists then overwrite
    145 
    146 To extract a subset of the filesystem, the filename or directory
    147 tree that is to be extracted can now be specified on the command line.  The
    148 file/directory should be specified using the full path to the file/directory
    149 as it appears within the Squashfs filesystem.  The file/directory will also be
    150 extracted to that position within the specified destination directory.
    151 
    152 The new "-force" option forces Unsquashfs to output to the destination
    153 directory even if files or directories already exist.  This allows you
    154 to update an existing directory tree, or to Unsquashfs to a partially
    155 filled directory.  Without the "-force" option, Unsquashfs will
    156 refuse to overwrite any existing files, or to create any directories if they
    157 already exist.  This is done to protect data in case of mistakes, and
    158 so the "-force" option should be used with caution.
    159

README-3.2

      1 	SQUASHFS 3.2 - A squashed read-only filesystem for Linux
      2 
      3 	Copyright 2002-2007 Phillip Lougher <phillip (a] lougher.org.uk>
      4 
      5 	Released under the GPL licence (version 2 or later).
      6 
      7 Welcome to Squashfs version 3.2.  Squashfs 3.2 has support for NFS exporting,
      8 some improvements to the Squashfs tools (Mksquashfs and Unsquashfs), some
      9 major bug fixes, new kernel patches, and various other smaller improvements
     10 and bug fixes.  Please see the CHANGES file for a detailed list.
     11 
     12 1. MKSQUASHFS
     13 -------------
     14 
     15 New command line options:
     16 
     17 -no-exports
     18 
     19 	Squashfs now supports NFS exports.  By default the additional
     20 	information necessary is added to the filesystem by Mksquashfs.  If you
     21 	do not wish this extra information, then this option can be specified.
     22 	This will save a couple of bytes per file, and the filesystem
     23 	will be identical to Squashfs 3.1.
     24 
     25 -no-progress
     26 
     27 	Mksquashfs by default now displays a progress bar. This option disables
     28 	it.
     29 
     30 2. UNSQUASHFS
     31 -------------
     32 
     33 	Unsquashfs now supports Squashfs 2.x filesystems.
     34

README-3.3

      1 	SQUASHFS 3.3 - A squashed read-only filesystem for Linux
      2 
      3 	Copyright 2002-2007 Phillip Lougher <phillip (a] lougher.demon.co.uk>
      4 
      5 	Released under the GPL licence (version 2 or later).
      6 
      7 Welcome to another release of Squashfs.  This is the 22nd release in just
      8 over five years of work.  Squashfs 3.3 has lots of nice improvements,
      9 both to the filesystem itself (bigger blocks, and sparse files), but
     10 also to the Squashfs-tools Mksquashfs and Unsquashfs.  As usual the
     11 CHANGES file has a detailed list of all the improvements.
     12 
     13 Following is a description of the changes to  the Squashfs tools, usage
     14 guides to the new options, and a summary of the new options.
     15 
     16 1. MKSQUASHFS - EXTENDED EXCLUDE FILE HANDLING
     17 ----------------------------------------------
     18 
     19 1. Extended wildcard pattern matching now supported in exclude files
     20 
     21   Enabled by specifying -wildcards option
     22 
     23   Supports both anchored and non-anchored exclude files.
     24 
     25 1.1 Anchored excludes
     26 
     27   Similar to existing exclude files except with wildcards.  Exclude
     28   file matches from root of source directories.
     29 
     30   Examples:
     31 
     32   1. mksquashfs example image.sqsh -wildcards -e 'test/*.gz'
     33 
     34      Exclude all files matching "*.gz" in the top level directory "test".
     35 
     36   2. mksquashfs example image.sqsh -wildcards -e '*/[Tt]est/example*'
     37 
     38      Exclude all files beginning with "example" inside directories called
     39      "Test" or "test", that occur inside any top level directory.
     40 
     41   Using extended wildcards, negative matching is also possible.
     42 
     43   3. mksquashfs example image.sqsh -wildcards -e 'test/!(*data*).gz'
     44 
     45      Exclude all files matching "*.gz" in top level directory "test",
     46      except those with "data" in the name.
     47 
     48 1.2 Non-anchored excludes
     49 
     50   By default excludes match from the top level directory, but it is
     51   often useful to exclude a file matching anywhere in the source directories.
     52   For this non-anchored excludes can be used, specified by pre-fixing the
     53   exclude with "...".
     54 
     55   Examples:
     56 
     57   1. mksquashfs example image.sqsh -wildcards -e '... *.gz'
     58 
     59      Exclude files matching "*.gz" anywhere in the source directories.
     60      For example this will match "example.gz", "test/example.gz", and
     61      "test/test/example.gz".
     62 
     63   2. mksquashfs example image.sqsh -wildcards -e '... [Tt]est/*.gz'
     64 
     65      Exclude files matching "*.gz" inside directories called "Test" or
     66      "test" that occur anywhere in the source directories.
     67 
     68   Again, using extended wildcards, negative matching is also possible.
     69 
     70   3. mksquashfs example image.sqsh -wildcards -e '... !(*data*).gz'
     71 
     72      Exclude all files matching "*.gz" anywhere in the source directories,
     73      except those with "data" in the name.
     74 
     75 2. Regular expression pattern matching now supported in exclude files
     76 
     77   Enabled by specifying -regex option.  Identical behaviour to wild
     78 card pattern matching, except patterns are considered to be regular
     79 expressions.
     80 
     81   Supports both anchored and non-anchored exclude files.
     82 
     83 
     84 2. MKSQUASHFS - NEW RECOVERY FILE FEATURE
     85 -----------------------------------------
     86 
     87 Recovery files are now created when appending to existing Squashfs
     88 filesystems.  This allows the original filesystem to be recovered
     89 if Mksquashfs aborts unexpectedly (i.e. power failure).
     90 
     91 The recovery files are called squashfs_recovery_xxx_yyy, where
     92 "xxx" is the name of the filesystem being appended to, and "yyy" is a
     93 number to guarantee filename uniqueness (the PID of the parent Mksquashfs
     94 process).
     95 
     96 Normally if Mksquashfs exits correctly the recovery file is deleted to
     97 avoid cluttering the filesystem.  If Mksquashfs aborts, the "-recover"
     98 option can be used to recover the filesystem, giving the previously
     99 created recovery file as a parameter, i.e.
    100 
    101 mksquashfs dummy image.sqsh -recover squashfs_recovery_image.sqsh_1234
    102 
    103 The writing of the recovery file can be disabled by specifying the
    104 "-no-recovery" option.
    105 
    106 
    107 3. UNSQUASHFS - EXTENDED EXTRACT FILE HANDLING
    108 ----------------------------------------------
    109 
    110 1. Multiple extract files can now be specified on the command line, and the
    111 files/directories to be extracted can now also be given in a file.
    112 
    113 To specify a file containing the extract files use the "-e[f]" option.
    114 
    115 2. Extended wildcard pattern matching now supported in extract files
    116 
    117   Enabled by default.  Similar to existing extract files except with
    118 wildcards.
    119 
    120   Examples:
    121 
    122   1. unsquashfs image.sqsh 'test/*.gz'
    123 
    124      Extract all files matching "*.gz" in the top level directory "test".
    125 
    126   2. unsquashfs image.sqsh '[Tt]est/example*'
    127 
    128      Extract all files beginning with "example" inside top level directories
    129      called "Test" or "test".
    130 
    131   Using extended wildcards, negative matching is also possible.
    132 
    133   3. unsquashfs image.sqsh 'test/!(*data*).gz'
    134 
    135      Extract all files matching "*.gz" in top level directory "test",
    136      except those with "data" in the name.
    137 
    138 3. Regular expression pattern matching now supported in extract files
    139 
    140   Enabled by specifying -r[egex] option.  Identical behaviour to wild
    141 card pattern matching, except patterns are considered to be regular
    142 expressions.
    143 
    144 4. UNSQUASHFS - EXTENDED FILENAME PRINTING
    145 ------------------------------------------
    146 
    147 Filename printing has been enhanced and Unquashfs can now display filenames
    148 with file attributes ('ls -l' style output).
    149 
    150 New options:
    151 
    152   -ll[s]
    153 
    154    list filesystem with file attributes, but don't unsquash
    155 
    156   -li[nfo]
    157 
    158    print files as they are unsquashed with file attributes
    159 
    160 
    161 5. UNSQUASHFS - MISCELLANEOUS OPTIONS
    162 -------------------------------------
    163 
    164    -s[tat]
    165 
    166    Display the filesystem superblock information.  This is useful to
    167    discover the filesystem version, byte ordering, whether it has an
    168    NFS export table, and what options were used to compress
    169    the filesystem.
    170

README-4.0

      1 	SQUASHFS 4.0 - A squashed read-only filesystem for Linux
      2 
      3 	Copyright 2002-2009 Phillip Lougher <phillip (a] lougher.demon.co.uk>
      4 
      5 	Released under the GPL licence (version 2 or later).
      6 
      7 Welcome to Squashfs 4.0.  This is an initial tools only release to
      8 support users of the 2.6.29 kernel, following the mainlining of Squashfs
      9 earlier this year.
     10 
     11 Later releases will probably contain kernel patches supporting 4.0
     12 layouts for earlier kernels.
     13 
     14 New Mksquashfs options
     15 ----------------------
     16 
     17 Mksquashfs now supports pseudo files, these allow fake directories, character
     18 and block devices to be specified and added to the Squashfs filesystem being
     19 built, rather than requiring them to be present in the source directories.
     20 This, for example, allows device nodes to be added to the filesystem without
     21 requiring root access.
     22 
     23 Two options are supported, -p allows one pseudo file to be specified on the
     24 command line, and -pf allows a pseudo file to be specified containing a
     25 list of pseduo definitions, one per line.
     26 
     27 Pseudo device nodes are specified using 7 arguments
     28 
     29 Filename type mode uid gid major minor
     30 
     31 Where type is either
     32 	b - for block devices, and
     33 	c - for character devices
     34 
     35 mode is the octal mode specifier, similar to that expected by chmod.
     36 
     37 Uid and gid can be either specified as a decimal number, or by name.
     38 
     39 For example:
     40 
     41 /dev/chr_dev c 666 root root 100 1
     42 /dev/blk_dev b 444 0 0 200 200
     43 
     44 Directories are specified using 5 arguments
     45 
     46 Filename type mode uid gid
     47 
     48 Where type is d.
     49

README-4.1

      1 	SQUASHFS 4.1 - A squashed read-only filesystem for Linux
      2 
      3 	Copyright 2002-2010 Phillip Lougher <phillip (a] lougher.demon.co.uk>
      4 
      5 	Released under the GPL licence (version 2 or later).
      6 
      7 Welcome to Squashfs 4.1.  This is a tools only release, support for Squashfs
      8 file systems is in mainline (2.6.29 and later).
      9 
     10 New features in Squashfs-tools 4.1
     11 ----------------------------------
     12 
     13   1. Support for extended attributes
     14   2. Support for LZMA and LZO compression
     15   3. New pseudo file features
     16 
     17 Compatiblity
     18 ------------
     19 
     20 Mksquashfs 4.1 generates 4.0 filesystems.  These filesystems are fully
     21 compatible/interchangable with filesystems generated by Mksquashfs 4.0 and are
     22 mountable on 2.6.29 and later kernels.
     23 
     24 Extended attributes (xattrs)
     25 ----------------------------
     26 
     27 Squashfs file systems now have extended attribute support.  The
     28 extended attribute implementation has the following features:
     29 
     30 1. Layout can store up to 2^48 bytes of compressed xattr data.
     31 2. Number of xattrs per inode unlimited.
     32 3. Total size of xattr data per inode 2^48 bytes of compressed data.
     33 4. Up to 4 Gbytes of data per xattr value.
     34 5. Inline and out-of-line xattr values supported for higher performance
     35    in xattr scanning (listxattr & getxattr), and to allow xattr value
     36    de-duplication.
     37 6. Both whole inode xattr duplicate detection and individual xattr value
     38    duplicate detection supported.  These can obviously nest, file C's
     39    xattrs can be a complete duplicate of file B, and file B's xattrs
     40    can be a partial duplicate of file A.
     41 7. Xattr name prefix types stored, allowing the redundant "user.", "trusted."
     42    etc. characters to be eliminated and more concisely stored.
     43 8. Support for files, directories, symbolic links, device nodes, fifos
     44    and sockets.
     45 
     46 Extended attribute support is in 2.6.35 and later kernels.  File systems
     47 with extended attributes can be mounted on 2.6.29 and later kernels, the
     48 extended attributes will be ignored with a warning.
     49 
     50 LZMA and LZO compression
     51 ------------------------
     52 
     53 Squashfs now supports LZMA and LZO compression.
     54 
     55 LZO support is in 2.6.36 and newer kernels.  LZMA is not yet in mainline.
     56 
     57 New Mksquashfs options
     58 ----------------------
     59 
     60 -comp <comp>
     61 
     62     Select <comp> compression.  
     63 
     64     The compression algorithms supported by the build of Mksquashfs can be
     65     found by typing mksquashfs without any arguments.  The compressors available
     66     are displayed at the end of the help message, e.g.
     67 
     68     Compressors available:
     69 	gzip (default)
     70 	lzma
     71 	lzo
     72 
     73     The default compression used when -comp isn't specified on the command line
     74     is indicated by "(default)".
     75 
     76 -no-xattrs
     77     Don't store extended attributes
     78 
     79 -xattrs
     80     Store extended attributes
     81 
     82     The default behaviour of Mksquashfs with respect to extended attribute
     83     storage is build time selectable.  The Mksquashfs help message indicates
     84     whether extended attributes are stored or not, e.g.
     85 
     86 	-no-xattrs		don't store extended attributes
     87 	-xattrs			store extended attributes (default)
     88 
     89     shows that extended attributes are stored by default, and can be disabled
     90     by the -no-xattrs option.
     91 
     92 	-no-xattrs		don't store extended attributes (default)
     93 	-xattrs			store extended attributes 
     94 
     95     shows that extended attributes are not stored by default, storage can be
     96     enabled by the -xattrs option.
     97 
     98 
     99 -noX
    100 -noXattrCompression
    101     Don't compress extended attributes
    102 
    103 
    104 New Unsquashfs options
    105 ----------------------
    106 
    107 -n[o-xattrs]
    108     Don't extract xattrs in filesystem
    109 
    110 -x[attrs]
    111     Extract xattrs in filesystem
    112 
    113     The default behaviour of Unsquashfs with respect to extended attributes
    114     is build time selectable.  The Unsquashfs help message indicates whether
    115     extended attributes are stored or not, e.g.
    116 
    117 	-no[-xattrs]		don't extract xattrs in file system
    118 	-x[attrs]		extract xattrs in file system (default)
    119 
    120     shows that xattrs are extracted by default.
    121 
    122 	-no[-xattrs]		don't extract xattrs in file system (default)
    123 	-x[attrs]		extract xattrs in file system
    124 
    125     shows that xattrs are not extracted by default.
    126 
    127 
    128 New pseudo file support
    129 -----------------------
    130 
    131 Mksquashfs supports pseudo files, these allow fake files, directories, character
    132 and block devices to be specified and added to the Squashfs filesystem being
    133 built, rather than requiring them to be present in the source directories.
    134 This, for example, allows device nodes to be added to the filesystem without
    135 requiring root access.
    136 
    137 Mksquashfs 4.1 adds support for "dynamic pseudo files" and a modify operation.
    138 Dynamic pseudo files allow files to be dynamically created when Mksquashfs
    139 is run, their contents being the result of running a command or piece of
    140 shell script.  The modifiy operation allows the mode/uid/gid of an existing
    141 file in the source filesystem to be modified.
    142 
    143 Two Mksquashfs options are supported, -p allows one pseudo file to be specified
    144 on the command line, and -pf allows a pseudo file to be specified containing a
    145 list of pseduo definitions, one per line.
    146 
    147 Pseudo operations
    148 -----------------
    149 
    150 1. Creating a dynamic file
    151 --------------------------
    152 
    153 Pseudo definition
    154 
    155 Filename f mode uid gid command
    156 
    157 mode is the octal mode specifier, similar to that expected by chmod.
    158 
    159 uid and gid can be either specified as a decimal number, or by name.
    160 
    161 command can be an executable or a piece of shell script, and it is executed
    162 by running "/bin/sh -c command".   The stdout becomes the contents of
    163 "Filename".
    164 
    165 Examples:
    166 
    167 Running a basic command
    168 -----------------------
    169 
    170 /somedir/dmesg f 444 root root dmesg
    171 
    172 creates a file "/somedir/dmesg" containing the output from dmesg.
    173 
    174 Executing shell script
    175 ----------------------
    176 
    177 RELEASE f 444 root root \
    178 		if [ ! -e /tmp/ver ]; then \
    179 			echo 0 > /tmp/ver; \
    180 		fi; \
    181                 ver=`cat /tmp/ver`; \
    182                 ver=$((ver +1)); \
    183                 echo $ver > /tmp/ver; \
    184                 echo -n `cat /tmp/release`; \
    185                 echo "-dev #"$ver `date` "Build host" `hostname`
    186 
    187 Creates a file RELEASE containing the release name, date, build host, and
    188 an incrementing version number.  The incrementing version is a side-effect
    189 of executing the shell script, and ensures every time Mksquashfs is run a
    190 new version number is used without requiring any other shell scripting.
    191 
    192 The above example also shows that commands can be split across multiple lines
    193 using "\".  Obviously as the script will be presented to the shell as a single
    194 line, a semicolon is need to separate individual shell commands within the
    195 shell script.
    196 
    197 Reading from a device (or fifo/named socket)
    198 --------------------------------------------
    199 
    200 input f 444 root root dd if=/dev/sda1 bs=1024 count=10
    201 
    202 Copies 10K from the device /dev/sda1 into the file input.  Ordinarily Mksquashfs
    203 given a device, fifo, or named socket will place that special file within the
    204 Squashfs filesystem, the above allows input from these special files to be
    205 captured and placed in the Squashfs filesystem.
    206 
    207 2. Creating a block or character device
    208 ---------------------------------------
    209 
    210 Pseudo definition
    211 
    212 Filename type mode uid gid major minor
    213 
    214 Where type is either
    215 	b - for block devices, and
    216 	c - for character devices
    217 
    218 mode is the octal mode specifier, similar to that expected by chmod.
    219 
    220 uid and gid can be either specified as a decimal number, or by name.
    221 
    222 For example:
    223 
    224 /dev/chr_dev c 666 root root 100 1
    225 /dev/blk_dev b 666 0 0 200 200
    226 
    227 creates a character device "/dev/chr_dev" with major:minor 100:1 and
    228 a block device "/dev/blk_dev" with major:minor 200:200, both with root
    229 uid/gid and a mode of rw-rw-rw.
    230 
    231 3. Creating a directory
    232 -----------------------
    233 
    234 Pseudo definition
    235 
    236 Filename d mode uid gid
    237 
    238 mode is the octal mode specifier, similar to that expected by chmod.
    239 
    240 uid and gid can be either specified as a decimal number, or by name.
    241 
    242 For example:
    243 
    244 /pseudo_dir d 666 root root
    245 
    246 creates a directory "/pseudo_dir" with root uid/gid and mode of rw-rw-rw.
    247 
    248 4. Modifying attributes of an existing file
    249 -------------------------------------------
    250 
    251 Pseudo definition
    252 
    253 Filename m mode uid gid
    254 
    255 mode is the octal mode specifier, similar to that expected by chmod.
    256 
    257 uid and gid can be either specified as a decimal number, or by name.
    258 
    259 For example:
    260 
    261 dmesg m 666 root root
    262 
    263 Changes the attributes of the file "dmesg" in the filesystem to have
    264 root uid/gid and a mode of rw-rw-rw, overriding the attributes obtained
    265 from the source filesystem.
    266

README-4.2

      1 	SQUASHFS 4.2 - A squashed read-only filesystem for Linux
      2 
      3 	Copyright 2002-2011 Phillip Lougher <phillip (a] lougher.demon.co.uk>
      4 
      5 	Released under the GPL licence (version 2 or later).
      6 
      7 Welcome to Squashfs 4.2.  This is a tools only release, support for Squashfs
      8 filesystems is in mainline (2.6.29 and later).
      9 
     10 New features in Squashfs-tools 4.2
     11 ----------------------------------
     12 
     13   1. Support for XZ compression
     14   2. Support for compressor specific options
     15 
     16 Compatiblity
     17 ------------
     18 
     19 Mksquashfs 4.2 generates 4.0 filesystems.  These filesystems are fully
     20 compatible/interchangable with filesystems generated by Mksquashfs 4.0 and are
     21 mountable on 2.6.29 and later kernels.
     22 
     23 XZ compression
     24 --------------
     25 
     26 Squashfs now supports XZ compression.  
     27 
     28 XZ support is in 2.6.38 and newer kernels.
     29 
     30 New Mksquashfs options
     31 ----------------------
     32 
     33 -X<compressor-option>
     34 
     35   Compression algorithms can now support compression specific options.  These
     36 options are prefixed by -X, and are passed to the compressor for handling.
     37 
     38   The compression specific options supported by each compressor can be
     39 found by typing mksquashfs without any arguments.  They are displayed at the
     40 end of the help message, e.g. 
     41 
     42 Compressors available and compressor specific options:
     43 	gzip (no options) (default)
     44 	lzo (no options)
     45 	xz
     46 	  -Xbcj filter1,filter2,...,filterN
     47 		Compress using filter1,filter2,...,filterN in turn
     48 		(in addition to no filter), and choose the best compression.
     49 		Available filters: x86, arm, armthumb, powerpc, sparc, ia64
     50 	  -Xdict-size <dict-size>
     51 		Use <dict-size> as the XZ dictionary size.  The dictionary size
     52 		can be specified as a percentage of the block size, or as an
     53 		absolute value.  The dictionary size must be less than or equal
     54 		to the block size and 8192 bytes or larger.  It must also be
     55 		storable in the xz header as either 2^n or as 2^n+2^(n+1).
     56 		Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K
     57 		etc.
     58

README-4.3

      1 	SQUASHFS 4.3 - A squashed read-only filesystem for Linux
      2 
      3 	Copyright 2002-2014 Phillip Lougher <phillip (a] lougher.demon.co.uk>
      4 
      5 	Released under the GPL licence (version 2 or later).
      6 
      7 Welcome to Squashfs 4.3.  This is the first release in over 3 years, and
      8 there are substantial improvements to stability, new compression options
      9 and compressors, speed optimisations, and new options for Mksquashfs/Unsquashfs.
     10 
     11 This is a tools only release, support for Squashfs filesystems is
     12 in mainline (2.6.29 and later).
     13 
     14 Changes in Squashfs-tools 4.3
     15 -----------------------------
     16 
     17 1. Stability improvements.  Better checking of user input for out of
     18    range/invalid values.  Better handling of corrupted Squashfs filesystems
     19    (Mksquashfs append mode, and Unsquashfs).  Better handling of buffer
     20    overflow/underflow.
     21 
     22 2. GZIP compressor now supports compression options, allowing different
     23    compression levels to be used.
     24 
     25 3. Rewritten LZO compressor with compression options, allowing different
     26    LZO algorithms and different compression levels to be used.
     27 
     28 4. New LZ4 compressor (note not yet in mainline kernel)
     29 
     30 5. Better default memory usage for Mksquashfs.  Mksquashfs by default now
     31    uses 25% of physical memory.
     32 
     33 6. Duplicate checking in Mksquashfs further optimised.  With certain
     34    "problem filesystems" greater than 2x performance improvement.
     35    Filesystems with a lot of duplicates should see at least 10-20% speed
     36    improvement.
     37 
     38 7. The -stat option in Unsquashfs now displays the compression options
     39    used to generate the original filesystem.  Previously -stat only displayed
     40    the compression algorithm used.
     41 
     42 8. The file being compressed/uncompressed in Mksquashfs/Unsquashfs is now
     43    displayed if CTRL-\ (SIGQUIT from keyboard) typed.
     44 
     45 9. The status of the internal queues/caches in Mksquashfs/Unsquashfs is
     46    now displayed if CTRL-\ (SIGQUIT from keyboard) is typed twice within
     47    one second.  Normally only useful for "power users", but it can be
     48    used to discover if there's any bottlenecks affecting performance
     49    (the bottleneck will normally be the compressors/fragment compressors).
     50 
     51 10. Miscellaneous new options for Mksquashfs/Unsquashfs to fine tune behaviour.
     52 
     53 11. Fixes for CVE-2012-4024 and CVE-2012-4025.
     54 
     55 Compatiblity
     56 ------------
     57 
     58 Mksquashfs 4.3 generates 4.0 filesystems.  These filesystems are fully
     59 compatible/interchangable with filesystems generated by Mksquashfs 4.0 and are
     60 mountable on 2.6.29 and later kernels.
     61 
     62 Compressors
     63 -----------
     64 
     65 New compression options and compressors are now supported.
     66 
     67 The new options and compressors are:
     68 
     69 1. gzip
     70 	  -Xcompression-level <compression-level>
     71 		<compression-level> should be 1 .. 9 (default 9)
     72 	  -Xwindow-size <window-size>
     73 		<window-size> should be 8 .. 15 (default 15)
     74 	  -Xstrategy strategy1,strategy2,...,strategyN
     75 		Compress using strategy1,strategy2,...,strategyN in turn
     76 		and choose the best compression.
     77 		Available strategies: default, filtered, huffman_only,
     78 		run_length_encoded and fixed
     79 
     80 2. lzo
     81 	  -Xalgorithm <algorithm>
     82 		Where <algorithm> is one of:
     83 			lzo1x_1
     84 			lzo1x_1_11
     85 			lzo1x_1_12
     86 			lzo1x_1_15
     87 			lzo1x_999 (default)
     88 	  -Xcompression-level <compression-level>
     89 		<compression-level> should be 1 .. 9 (default 8)
     90 		Only applies to lzo1x_999 algorithm
     91 
     92 3. lz4
     93 	  -Xhc
     94 		Compress using LZ4 High Compression
     95 
     96 The compression specific options are, obviously, specific to the compressor
     97 in question, and you should read the compressor documentation and check
     98 their web sites to understand their behaviour.
     99 
    100 In general the defaults used by Mksquashfs for each compressor are optimised
    101 to give the best performance for each compressor, where what constitutes
    102 best depends on the compressor.  For gzip/xz best means highest compression
    103 (trying multiple filters/strategies can improve compression, but this is
    104 extremely expensive computationally, and hence, not suitable for the defaults),
    105 for LZO/LZ4 best means a tradeoff between compression and (de)-compression
    106 overhead (LZO/LZ4 by definition are intended for weaker processors).
    107 
    108 New Mksquashfs options
    109 ----------------------
    110 
    111 1. -mem <size>
    112 
    113   Set the amount of memory used by Mksquashfs to <size> bytes.  G/M and K
    114   post-fixes are supported.
    115 
    116   By default Mksquashfs uses 25% of the physical memory.  Increasing
    117   this with the -mem option can increase performance (note it does not have
    118   any effect on compression).  Reducing it can prevent thrashing if the
    119   system is busy and there is not 25% of physical memory free (again, note
    120   it does not have any effect on compression).
    121 
    122 2. -exit-on-error
    123 
    124   By default Mksquashfs treats certain errors as benign, if these
    125   errors occur Mksquashfs prints the error on the console but continues.
    126   These errors are typically failure to read a file from the source filesystem.
    127   This is deliberate, in many cases users prefer Mksquashfs to flag
    128   the error but continue rather than abort what may be hours of compression.
    129 
    130   But there are times where failure to read any file is considered critical,
    131   and users (especially in the case of automated scripts where the
    132   errors output to the console may be missed) prefer Mksquashfs to exit.
    133 
    134   The new -exit-on-error option can be used in this scenario.  This option
    135   makes Mksquashfs treat all benign errors as fatal.
    136 
    137 3. -progress
    138 
    139   By default if -info is specified, the progress bar is disabled as it gets
    140   in the way.  Occasionally you might want the progress bar enabled whilst
    141   -info is enabled.  This option forces Mksquashfs to output the progress
    142   bar when -info is specified.
    143 
    144 4. -Xhelp
    145 
    146   Display the usage text for the currently selected compressor.
    147 
    148 New Unsquashfs options
    149 ----------------------
    150 
    151 1. -u[ser-xattrs]
    152 
    153   Only write user xattrs.  This forces Unsquashfs to ignore system xattrs.
    154   This is useful when Unsquashing a filesystem as a non-root user, and the
    155   filesystem contains system xattrs which are only writable by root.
    156 
    157 Major bugs fixed
    158 ----------------
    159 
    160 1. If Mksquashfs ran out of space in the destination filesystem, this
    161    would not cause Mksquashfs to immediately abort, and Mksquashfs would
    162    continue to process the source filesystem.  Mksquashfs now immediately
    163    aborts on out of space in the destination filesystem.
    164 
    165 2. Unsquashfs ignored the maximum number of open files limit, and if that
    166    was lower than the default limit for Linux, it would run out of file
    167    descriptors.  Unsquashfs now limits the number of open files to the
    168    limit currently in force (e.g. specified by setrlimit).
    169 
    170 3. If huge numbers of dynamic pseudo files were specified, Mksquashfs
    171    could exceed the maximum number of open files limit.  This was because
    172    Mksquashfs created all the dynamic file processes up front before
    173    commencing source filesystem reading and compression.  Mksquashfs
    174    now creates the dynamic file processes on demand whilst reading
    175    and compressing the source filesystem, thus limiting the number of
    176    dynamic pseudo file processes in existence at any one time.
    177 
    178 4. When outputting Unsquashfs used to set the permissions of directories
    179    as it recursively descended.  This in hindsight had an obvious oversight,
    180    if a directory had only read permission (or was otherwise restricted), then
    181    Unsquashfs would fail to write its contents when descending into it.  Fixed
    182    by setting directory permissions as Unsquashfs recursively unwinds.
    183