Pcompress is an archiver that can do compression/decompression and deduplication in parallel by splitting input data into chunks. It has a modular structure and includes support for multiple algorithms like LZMA, Bzip2, PPMD, LZ4, etc., with KECCAK/BLAKE2/SHA-256/512 chunk checksums. SSE optimizations for the bundled LZMA are included. It also implements chunk-level Content-Aware Deduplication and Delta Compression features based on a Polynomial Fingerprinting scheme. It has low metadata overhead and overlaps I/O and compression to achieve maximum parallelism. It has AES encryption capability and uses Scrypt from Tarsnap to generate per-session unique keys from passwords. It can work in pipe mode, reading from stdin and writing to stdout. It also provides some adaptive compression modes in which a suitable algorithm is chosen per chunk based on heuristics.
Release Notes: This release adds many improvements and fixes, including ones for performance and stability. It adds the KECCAK Sha3 message digest. A new fast Delta variant detects embedded tables of binary numeric data and RLE encodes them. A matrix transform allows better compression of a Dedupe index. LZ4 and XXHash have been updated. The test suite has been expanded. Pcompress now builds without warnings with strict compiler flags. Alternate locations for external libraries are handled properly, and older OpenSSL versions up to 0.9.8e work. The debug statistics mode now prints additional throughput statistics.