Skip to main content

pigz: Parallel Compression Tool Guide

1. Introduction​

pigz is a parallel implementation of gzip that leverages multi-core processors to accelerate the compression process. Written by Mark Adler, pigz aims to provide the same functionality as gzip but with significantly improved compression speed through multi-threading.

2. Features​

  • Parallel compression: pigz uses multiple processor cores to compress data in parallel, significantly improving compression speed.
  • Compatibility: Compressed files produced by pigz are fully compatible with gzip and can be decompressed with gzip or gunzip.
  • Flexibility: Supports a variety of options and parameters, allowing you to adjust compression levels and thread counts as needed.

3. Installation​

pigz can be installed via the package manager on most Linux distributions:

On Debian/Ubuntu:​

sudo apt-get install pigz

On CentOS/RHEL:​

sudo yum install pigz

On macOS:​

brew install pigz

4. Usage​

4.1 Basic Usage​

The usage of pigz is essentially the same as gzip, except that it uses all available processor cores by default.

Compress a file​

pigz file.txt

This creates a file.txt.gz compressed file.

Decompress a file​

pigz -d file.txt.gz

This decompresses file.txt.gz back to file.txt.

4.2 Using Multiple Threads​

You can use the -p option to specify the number of threads. For example, compress with 4 threads:

pigz -p 4 file.txt

4.3 Using with tar​

pigz is often used together with the tar tool for archiving and compression.

Compress a directory​

tar -cvf - directory/ | pigz -p 4 > archive.tar.gz

Here, -p 4 means using 4 threads.

Decompress a directory​

pigz -d -c archive.tar.gz | tar -xvf -

5. Options and Parameters​

pigz supports a variety of options and parameters. Here are some commonly used ones:

  • -p, --processes <n>: Specify the number of threads to use.
  • -d, --decompress: Decompress a file.
  • -k, --keep: Keep the original file.
  • -f, --force: Force overwrite of existing files.
  • -1 to -9: Set the compression level. -1 is the fastest with the lowest compression ratio, -9 is the slowest with the highest compression ratio (default is -6).
  • --fast: Equivalent to -1.
  • --best: Equivalent to -9.

Examples​

Fast compression​

pigz --fast file.txt

High compression ratio​

pigz --best file.txt

Keep the original file​

pigz -k file.txt

6. Performance Comparison​

pigz significantly outperforms single-threaded gzip, especially on multi-core processors. Here is a simple performance comparison:

Single-threaded gzip​

time gzip largefile.txt

Multi-threaded pigz​

time pigz -p 8 largefile.txt

On multi-core processors, pigz compression speed is typically several times faster than gzip. The exact improvement depends on file size, compression level, and the number of available processor cores.

7. Summary​

pigz is a powerful parallel compression tool suitable for scenarios that require fast compression of large files. Its multi-threading capability can significantly improve compression speed while maintaining compatibility with gzip. Whether on personal computers or in server environments, pigz is a compression tool worth recommending.