Re: nnml compression: state of the art?

[Stainless Steel Rat <ratinox@peorth.gweep.net>]

* Alan Shutko <ats@acm.org>  on Thu, 29 Mar 2001
| I disagree.  I tried it on a mail folder I have around:

Context: nnml files are generally ~4k, not 20MB.  A more relevant example:

-rw-------    1 ratinox  ratinox      4974 Mar 29 17:04 test
-rw-------    1 ratinox  ratinox      2640 Mar 29 17:04 test.1.gz
-rw-------    1 ratinox  ratinox      2557 Mar 29 17:04 test.9.gz

On a system like mine with 4k blocks, test requires two blocks (8k), while
test.1.gz and test.9.gz require one block (4k) each.  The net gain in free
space is one block (4k) regardess of -1 or -9.

On a system with 2k blocks, test requires three blocks (6k), while
test.1.gz and test.9.gz require two blocks (4k) each.  The net gain in free
space is one block (2k) regardless of -1 or -9.

On a system with 1k blocks, test requires five blocks (5k), while test.1.gz
and test.9.gz require three blocks (3k) each.  The net gain in free space
is two blocks (2k) regardless of -1 or -9.

On a system with 512 byte blocks, test requires 10 blocks (5k), while
test.1.gz requires 6 blocks (3k) and test.2.gz requires 5 blocks (2.5k).  A
net savings of 512 bytes.  This is reaching the point of diminishing
returns, because if test were 7 bytes longer then the gzip -9 file would be
slightly over 512 * 5 and require 6 blocks (3k), again for no net gain.

If you have a few very large files then gzip -9 is usually better if space
is the primary concern.  If you have lots and lots of small files then gzip
-1 is almost always better, except in a few odd cases.

By the way:

-rw-------    1 ratinox  ratinox      4974 Mar 29 17:04 test
-rw-------    1 ratinox  ratinox      2640 Mar 29 17:04 test.1.gz
-rw-------    1 ratinox  ratinox      2623 Mar 29 17:04 test.2.gz
-rw-------    1 ratinox  ratinox      2607 Mar 29 17:04 test.3.gz
-rw-------    1 ratinox  ratinox      2561 Mar 29 17:04 test.4.gz
-rw-------    1 ratinox  ratinox      2557 Mar 29 17:04 test.5.gz
-rw-------    1 ratinox  ratinox      2557 Mar 29 17:04 test.6.gz
-rw-------    1 ratinox  ratinox      2557 Mar 29 17:04 test.7.gz
-rw-------    1 ratinox  ratinox      2557 Mar 29 17:04 test.8.gz
-rw-------    1 ratinox  ratinox      2557 Mar 29 17:04 test.9.gz

Anything beyond -5 nets you no extra space savings for small files.
-- 
Rat <ratinox@peorth.gweep.net>    \ Do not use Happy Fun Ball on concrete.
Minion of Nathan - Nathan says Hi! \ 
PGP Key: at a key server near you!  \