Google gets Snappy over compression

Google has released its Snappy Compression and Decompression library as Apache-licensed open source. Previously referred to as "Zippy", the C++ based Snappy aims for "very high speeds with reasonable compression". This means that it's compression level is only around half that achieved with programs such as zlib .

But on an single 2.26 GHz core of an i7 microprocessor, benchmarks have shown that it can compress at around 250 MB/sec and decompress at 500 MB/sec. Snappy is used widely within Google in applications such as BigTable or MapReduce and in the company's internal RPC systems. Google engineer, Steinar H. Gunderson, writes that the library is an order of magnitude faster than zlib, noting that Snappy gets its performance from sacrificing the secondary entropy reduction phase typical of "most LZ-style compressors". It also uses a fixed, hand-tuned (over many years) packing format, resulting in less space saving but much less CPU usage.

Snappy is optimised for 64-bit x86 compatible processors and is considered best suited for situations where developers want to reduce disk or network I/O without placing a load on CPU resources. If the requirement is to save space, Gunderson recommends sticking with zlib or similar more CPU intensive compression libraries. Snappy is hosted on Google Code and more information is available in the README file.

(djwm)