* Deleting duplicates from nnml:mail.misc
@ 2013-10-07 23:00 Alex Schroeder
2013-10-13 18:44 ` Alex Schroeder
0 siblings, 1 reply; 3+ messages in thread
From: Alex Schroeder @ 2013-10-07 23:00 UTC (permalink / raw)
To: info-gnus-english
I've started using Gnus again after many years. I had a ton of
mbox files I had created using fetchmail from my Gmail account,
but didn't really trust it, ran it a lot, thought that maybe it
didn't get my sent messages and therefore moved a lot of messages
from the Gmail IMAP to my nnml:mail.misc using B m ... and now I'm
suddenly having second thoughts. Is my nnml:mail.misc of about
60000 messages full of duplicates?
I've been wondering how to find them, if any. I've tried looking at the
.overview file...
#!/bin/env perl
my %count = ();
my %file = ();
my $overview = "/Volumes/Extern/Archives/Mail/mail/misc/.overview";
open(F, $overview) or die "Cannot open $overview: $!";
while(my $line = <F>) {
my @field = split(/\t/, $line);
$count{$field[4]}++;
push(@{$file{$field[4]}}, $field[0]);
}
close(F);
my @keys = sort { $count{$b} cmp $count{$a} } keys %count;
print join("\n", map {
$_ . "\t" . $count{$_} . "\t" . join(", ", @{$file{$_}})
} @keys[0 .. 3]) . "\n";
How would I best use this script to delete the duplicate messages? Can I
"regenerate" the overview file without loosing anything, perhaps by
regenrating something in the Server buffer? Or is there an elisp version
of the above that does the right thing?
Cheers
Alex
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Deleting duplicates from nnml:mail.misc
2013-10-07 23:00 Deleting duplicates from nnml:mail.misc Alex Schroeder
@ 2013-10-13 18:44 ` Alex Schroeder
2013-10-14 7:16 ` Thien-Thi Nguyen
0 siblings, 1 reply; 3+ messages in thread
From: Alex Schroeder @ 2013-10-13 18:44 UTC (permalink / raw)
To: info-gnus-english
Alex Schroeder <alex@gnu.org> writes:
> Is my nnml:mail.misc of about 60000 messages full of duplicates?
I think I found a way of doing this within Emacs. I'm sure this could be
made into a nice package, but hope to use it just once. And it takes
long seconds to build up the data structures.
Anyway, here's what I do:
;; get all the headers from the overview file
(setq asc:headers (nnheader-parse-overview-file
"/Volumes/Extern/Archives/Mail/mail/misc/.overview"))
;; how many mails in total?
(length asc:headers)
;; create an alist with key Message-ID and value being a list of
;; article numbers sharing this Message-Id (in other words, if there
;; are more than one article number, these are potential duplicates)
;; Example list item: ("<m28uxxr3jk.fsf@gnu.org>" 62335 62329)
(setq asc:duplicates nil)
(dolist (hdr asc:headers)
(let ((cell (assoc (aref hdr 4) asc:duplicates)))
(if cell
(setcdr cell (cons (aref hdr 0) (cdr cell)))
(setq asc:duplicates (cons (list (aref hdr 4)
(aref hdr 0))
asc:duplicates)))))
;; how many unique Message-IDs?
(length asc:duplicates)
;; look at an example entry
(car asc:duplicates)
;; check how many entries refer to more than one article number
(let ((count 0))
(dolist (item asc:duplicates)
(when (> (length item) 2)
(setq count (1+ count))))
count)
;; our todo list are the Message-IDs with more than one article number
(setq asc:todo (remove-if (lambda (item) (= (length item) 2)) asc:duplicates))
;; look at the first todo item
(car asc:todo)
;; if you want to check upon those numbers...
;; (gnus-summary-goto-article 62335)
;; (gnus-summary-goto-article 62329)
;; how many todo items?
(length asc:todo)
;; let's get a flat list of article numbers -- using nconc means
;; trashing asc:duplicates!
(setq asc:articles (apply 'nconc
(mapcar 'cdr
(remove-if (lambda (item) (= (length item) 2))
asc:duplicates))))
;; how many articles to look at?
(length asc:articles)
;; read them all
(gnus-group-read-group t t "nnml+mail:mail.misc" asc:articles)
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Deleting duplicates from nnml:mail.misc
2013-10-13 18:44 ` Alex Schroeder
@ 2013-10-14 7:16 ` Thien-Thi Nguyen
0 siblings, 0 replies; 3+ messages in thread
From: Thien-Thi Nguyen @ 2013-10-14 7:16 UTC (permalink / raw)
To: Alex Schroeder; +Cc: info-gnus-english
[-- Attachment #1.1: Type: text/plain, Size: 438 bytes --]
() Alex Schroeder <alex@gnu.org>
() Sun, 13 Oct 2013 20:44:49 +0200
And it takes long seconds to build up the data structures.
If order does not matter, you can use a hash table instead of an alist.
--
Thien-Thi Nguyen
GPG key: 4C807502
(if you're human and you know it)
read my lisp: (responsep (questions 'technical)
(not (via 'mailing-list)))
=> nil
[-- Attachment #1.2: Type: application/pgp-signature, Size: 197 bytes --]
[-- Attachment #2: Type: text/plain, Size: 162 bytes --]
_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2013-10-14 7:16 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-10-07 23:00 Deleting duplicates from nnml:mail.misc Alex Schroeder
2013-10-13 18:44 ` Alex Schroeder
2013-10-14 7:16 ` Thien-Thi Nguyen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).