From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/5057 Path: main.gmane.org!not-for-mail From: Albrecht Kadlec Newsgroups: gmane.emacs.gnus.general Subject: Archive: Zip-/Directories? (was: Re: Old messages from this list in Digest Format via FTP ?) Date: Tue, 6 Feb 96 14:06 MET Message-ID: References: <9602050537.AA25951@healdb.matat.health.gov.il> <199602050720.BAA29446@sina.hpc.uh.edu> Reply-To: albrecht@auto.tuwien.ac.at (Albrecht Kadlec) NNTP-Posting-Host: coloc-standby.netfonds.no X-Trace: main.gmane.org 1035145714 31648 80.91.224.250 (20 Oct 2002 20:28:34 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Sun, 20 Oct 2002 20:28:34 +0000 (UTC) Cc: ding@ifi.uio.no, larsi@ifi.uio.no Return-Path: ding-request@ifi.uio.no Original-Received: from ifi.uio.no (ifi.uio.no [129.240.64.2]) by miranova.com (8.7.3/8.6.9) with SMTP id GAA16340 for ; Tue, 6 Feb 1996 06:18:30 -0800 Original-Received: from kodiak.vlsivie.tuwien.ac.at (kodiak.vlsivie.tuwien.ac.at [128.130.40.140]) by ifi.uio.no with ESMTP (8.6.11/ifi2.4) id ; Tue, 6 Feb 1996 14:06:23 +0100 Original-Received: from anaphi.vlsivie.tuwien.ac.at by kodiak.vlsivie.tuwien.ac.at with esmtp (Smail3.1.29.1 #9) id m0tjn67-000TYIC; Tue, 6 Feb 96 14:06 MET Original-Received: by anaphi.vlsivie.tuwien.ac.at (Smail3.1.29.1 #9) id m0tjn66-0000z9C; Tue, 6 Feb 96 14:06 MET Original-To: Jason L Tibbitts III X-Attribution: A Gcc: nnfolder:misc-mail Xref: main.gmane.org gmane.emacs.gnus.general:5057 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:5057 >>>>> Jason L Tibbitts writes: >>>>> "AS" == Aharon Schkolnik writes: AS> Can I FTP the old messages from this list in digest format from AS> somewhere ? J> Well, I keep the archives and they're not digested. Perhaps you already J> know that you can type G a in the group buffer to get a newsgroup J> containing the last 500 articles from this list, or C-u G a to get the J> _entire_ archive. WARNING: that's currently over 7000 articles and the J> overview is 1.4MB. It's probably pretty useless at this point, but then J> info that old is probably useless, too. Whatever happened to the proposal J> to allow nnml folders to contain several subdirectories? I saw someone mentioning it again a month ago, but I couldn't find the article again. It was sth. like packing some articles, so that there were chunks of a specific length - say 10K. The reason was to keep ftp overhead to a minimum (1K requests were said to be very inefficient). I wanted to add a "please support zipped .overviews and chunks": "closed" .overviews could then be easily zipped as could the 10K chunks. only the active chunk and .overview don't necessarily have to be zipped. Sysadmins would jump at this too, since it saves MEGS of disk space and CPU time (automatic compression, when a .gz is requested) (BTW: Maybe sth along these ideas could be established as THE archive standard ?? Don't tweak me now - I might wake up ;^) ) But which format should the articles be chunked into? .tar.gz is unix standard. But you'll never get the DOS-OS/2-NT world to use it. - ok, maybe the OS/2 world, but not the DOS / WIN / NT people. I also think it's more elegant and easier to administrate, if we used a packed format with an internal directory, and an "ADD-TO / EXTRACT FROM" functionality like .zip files. The responsiveness of .zip archives should be much better than 'gzip -d | tar -x'. Also with .zip archives the sysadmin could set up a script to ADD each newly arrived article to the active chunk, switching chunks every 10 to 20 articles, and add it's headers to the .overview (switched every 200 articles, the average article nedds about 200 bytes in the .overview -> 40K, compressed 20K). The Gnus archive (a rather crowded example of a mailing list archive :-) ) would then consist of some 700 article chunks of 10 articles each (=18K) in some 35 directories ('a 20 files) and the corresponding .overview (cover 200 articles, also 20K each) files. 200 articles are about 2 weeks' articles on the gnus list. First Approach: (dislike) Gnus could request the entire .zip file and let jka-compress or some other package do the rest of the dirty stuff, simply requesting an article from the .zip file. #pragma dream on Second Approach: (mucho like) One of the "guru's" mentioned a new dired-like package about a half year ago. Is there anything like a find-file extension where I can give a pathname like "~/oldstuff.zip/.emacs.el". Anyone saw a package, that could do this stuff ? Would be way cool to simply specify: /ftp@ftp.hpc.uh.edu:/pub/emacs/ding-list-recent/articles7000-7009.zip/7001 and have ange-ftp fetch the zip file to a buffer, jka-compress extract the article, read it and then, if article 7002 is read, find-file already has the zip file locally and only calls jka-compress to extract the article. If such a beast doesn't exist, how about creating it - most of the code is there somewhere. As a first estimation, one would have to make jka-compress, ange-ftp and dired-mode work together (integrate?) from find-file-not-found-hook (or extend load-file to support jka-compress, as it supports ange-ftp). #pragma dream off I like the second one better, since all of emacs-country (all of Gnus!!!) would benefit from such an extension, and it's less complicated to use for Gnus: simply call find-file for the article fetching and saving. The argument needs some fiddling with numbers - though. (one could dream of a compressed cache with one .zip file per newsgroup: News/cache/alt.solar.thermal.zip/article* News/cache/alt.solar.thermal.overview.gz) How many man-years am I talking about? - is it impossible? J> You can also get the stuff as a single tar file; just get the J> (non-existant) file ding-list-recent.tar.gz or ding-list.tar.gz from J> ftp.hpc.uh.edu:/pub/emacs. The second will still be huge (~13MB). 7000 Articles, so the average article is 1.9K long (zipped !!!) Seems pretty much to me, but I based my calculations above on this info. J> If it would be of any use to anyone, I may be persuaded to batch up monthly J> chunks but I probably won't make them into digests unless someone provides J> a quick script to do so. One month seems to be roughly 500 articles (on the ding-list). Should be an easy one to create an .overview from 200 articles and put 10 articles into a zip file of it's own in a subdir ? albrecht