From mboxrd@z Thu Jan 1 00:00:00 1970 From: list at eworm.de (Christian Hesse) Date: Wed, 26 Feb 2020 11:18:47 +0100 Subject: [PATCH 1/2] ui-snapshot: add support for zstd compression In-Reply-To: References: <20200226082148.42470-1-list@eworm.de> Message-ID: <20200226111847.2fab512c@leda> "Jason A. Donenfeld" on Wed, 2020/02/26 17:16: > We had been discussing which compression level to pass to zstd or to > leave it as default. I think Unit193 had some thoughts on that, so > CC'ing him in case he wants to jump in. Thought about that myself after I sent the mail... This is a trade-off... Do we prefer faster speed or better compression? We can not have both, just either of both or a mix. And do we want to go with multi-threaded compression? The default compression level is 3. For our Arch packages we decided to go with level 20... Well, the packages are compressed once and decompressed on a magnitude of machines - so it makes sense there. I would propose to take something in between... I think compression level 10 gives reasonable results. Any thoughts? Some numbers: % for I in $(seq 3 19); do zstd -$I < git-2.25.1.tar > git-2.25.1-$I.tar.zst; done level 3: 0,30s user 0,03s system 105% cpu 0,306 total level 4: 0,30s user 0,03s system 107% cpu 0,307 total level 5: 0,57s user 0,03s system 103% cpu 0,574 total level 6: 0,72s user 0,04s system 103% cpu 0,739 total level 7: 0,90s user 0,04s system 102% cpu 0,912 total level 8: 1,13s user 0,03s system 102% cpu 1,134 total level 9: 1,59s user 0,02s system 100% cpu 1,598 total level 10: 1,98s user 0,04s system 100% cpu 2,002 total level 11: 2,28s user 0,05s system 100% cpu 2,318 total level 12: 3,50s user 0,05s system 100% cpu 3,536 total level 13: 4,70s user 0,05s system 100% cpu 4,749 total level 14: 6,52s user 0,06s system 99% cpu 6,603 total level 15: 7,06s user 0,09s system 100% cpu 7,146 total level 16: 9,35s user 0,10s system 99% cpu 9,456 total level 17: 12,00s user 0,08s system 99% cpu 12,118 total level 18: 14,26s user 0,08s system 99% cpu 14,390 total level 19: 19,66s user 0,10s system 99% cpu 19,835 total % for I in $(seq 3 19); do zstd -T0 -$I < git-2.25.1.tar > git-2.25.1-$I-T0.tar.zst; done level 3: 0,41s user 0,05s system 182% cpu 0,251 total level 4: 0,52s user 0,05s system 200% cpu 0,285 total level 5: 0,81s user 0,04s system 195% cpu 0,434 total level 6: 1,59s user 0,06s system 188% cpu 0,877 total level 7: 1,25s user 0,04s system 189% cpu 0,681 total level 8: 1,63s user 0,04s system 187% cpu 0,888 total level 9: 2,12s user 0,05s system 187% cpu 1,156 total level 10: 2,58s user 0,07s system 178% cpu 1,482 total level 11: 2,85s user 0,07s system 176% cpu 1,650 total level 12: 4,33s user 0,12s system 173% cpu 2,560 total level 13: 5,50s user 0,07s system 188% cpu 2,948 total level 14: 6,42s user 0,09s system 192% cpu 3,389 total level 15: 8,21s user 0,13s system 185% cpu 4,492 total level 16: 10,28s user 0,07s system 179% cpu 5,778 total level 17: 12,08s user 0,10s system 117% cpu 10,411 total level 18: 16,24s user 0,11s system 114% cpu 14,271 total level 19: 24,78s user 0,21s system 125% cpu 19,843 total % xz < git-2.25.1.tar > git-2.25.1.tar.xz 17,07s user 0,26s system 139% cpu 12,453 total % time gzip < git-2.25.1.tar > git-2.25.1.tar.gz 2,16s user 0,04s system 361% cpu 0,607 total 37M git-2.25.1.tar 8,7M git-2.25.1.tar.gz 5,9M git-2.25.1.tar.xz 8,2M git-2.25.1-3.tar.zst 8,1M git-2.25.1-4.tar.zst 7,9M git-2.25.1-5.tar.zst 7,6M git-2.25.1-6.tar.zst 7,3M git-2.25.1-7.tar.zst 7,1M git-2.25.1-8.tar.zst 7,0M git-2.25.1-9.tar.zst 6,8M git-2.25.1-10.tar.zst 6,8M git-2.25.1-11.tar.zst 6,7M git-2.25.1-12.tar.zst 6,5M git-2.25.1-13.tar.zst 6,4M git-2.25.1-14.tar.zst 6,4M git-2.25.1-15.tar.zst 6,1M git-2.25.1-16.tar.zst 6,0M git-2.25.1-17.tar.zst 5,9M git-2.25.1-18.tar.zst 5,9M git-2.25.1-19.tar.zst -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);} -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: