Release Notes: This release fixes several important bugs in the 3.0 version, making it more stable. In addition, automatic scaling of compression buffers based on compression level has been added. This improves compression ratio for the simple usage modes.
Release Notes: This is a major release featuring complete archiving capability using Libarchive and advanced data type-based compression. Appropriate algorithms for the file type are used along with various filters like PackJPG, Dispack, Delta2, etc. Files are sorted to cluster similar content and improve compression. Data split boundaries are determined from file type and rolling hash changes. Other improvements include performance tweaks, reduced memory consumption, and simplified usage.
Release Notes: This release fixes several issues, including some corner case crashes and a couple of buffer overflows. Data Deduplication can now be done using blocks as small as 2KB, providing a much higher dedupe ratio than virtually any other deduplication software. Similarity based deduplication performance has been improved. Free memory detection accuracy has also been improved.
Release Notes: This release fixes a few bugs and provides several improvements in efficiency and performance. The Similarity detection effectiveness for similarity based near-exact deduplication has been improved. At the same time memory requirements for the index has been reduced. Accuracy of data partitioning between threads has been improved. Chunking and indexing performance have been improved and the KMV Sketch computation is now more accurate. This release moves all the core functionality into a shared library in preparation for an API interface that will be introduced in future releases.
Release Notes: This is primarily a Bugfix release. It fixes some crashes with invalid input and build problems on Debian6 and older non-SSE4 processors. The Min-heap based Similarity matching for Delta Encoding has been improved and made faster and more accurate. Accuracy of scalable Segmented Global Deduplication has been further improved to be greater than 95%. More testcases have been added.
Release Notes: This release adds many bugfixes and performance improvements. Accuracy in finding duplicates in Global Dedupe has been improved. SHA256 is now the default block hash algorithm for dedupe, with the ability to change it separately from the chunk verification hash. Overall, many performance improvements have been made, with better parallelism, more SSE vectorization, and faster sorting and improving the segment hash list file handling, resulting in smaller I/O and fewer random accesses. Bugs in calculating in-memory index size has been fixed to avoid overflowing free RAM and swapping to disk.
Release Notes: This release introduces the capability to do Global Deduplication. This performs deduplication across the entire dataset using an in-memory index as opposed to deduplication only within segments. Two kinds of indexes are used based on the dataset size. A full chunk hash index is used for small datasets. A special segmented similarity based index is used when the dataset is very large. The latter index size is just 0.002% of the dataset size with >90% efficiency of a full chunk index based exact dedupe using 4KB chunks. Streaming support allows optimization of network transfer of large data.
Release Notes: This update release adds several performance and security enhancements. AES code now includes AES-NI and VPAES optimizations. The fast XSalsa20 encryption algorithm has been added. Encryption key length can now be set at runtime (128/256 bits). Nonce, salt, etc. are now HMACed, and nonce generation randomized. Merkle Tree hashing via OpenMP is now used for all hashes when compressing an entire file in a single chunk (solid archive mode). Deduplication performance is improved by 95%. There is improved XML detection in adaptive modes. The file format has been updated, but backward compatibility retained.
Release Notes: This is a performance-focused release with improvements across the board. Extensive x86 SSE2/3/4/AVX vectorization has been done with runtime CPU feature detection. Deduplication performance is increased 3X. The Delta Compression algorithm has been tweaked for better performance and effectiveness with reduced memory usage. xxHash has been vectorized. Support for BLAKE2 checksum has been included. AES CTR mode has been vectorized. Intel's optimized SHA512, 512/256 is included for leading edge SHA2 performance in addition to parallel modes. LZMA performance is slightly boosted.
Release Notes: This release adds many improvements and fixes, including ones for performance and stability. It adds the KECCAK Sha3 message digest. A new fast Delta variant detects embedded tables of binary numeric data and RLE encodes them. A matrix transform allows better compression of a Dedupe index. LZ4 and XXHash have been updated. The test suite has been expanded. Pcompress now builds without warnings with strict compiler flags. Alternate locations for external libraries are handled properly, and older OpenSSL versions up to 0.9.8e work. The debug statistics mode now prints additional throughput statistics.