Gnus development mailing list
 help / color / mirror / Atom feed
* X-Gnus-Article-Number cleanup
@ 2021-01-26 22:04 Brent Busby
  2021-01-28  5:03 ` Lars Ingebrigtsen
  0 siblings, 1 reply; 5+ messages in thread
From: Brent Busby @ 2021-01-26 22:04 UTC (permalink / raw)
  To: ding

[-- Attachment #1: Type: text/plain, Size: 2370 bytes --]

I'm writing a small Perl script (which would probably be more apropos in
Lisp, but I'm not good at Lisp) to cleanup Berkeley mbox format
mailboxes for Gnus after they've been edited by other mailreaders like
Alpine and Mutt.  Sometimes I find this is necessary even when other
mailreaders haven't done anything to the mailbox, as when copying a
message into an nnfolder within Gnus that came from elsewhere in Gnus,
like copying a message from a Usenet newsgroup or a different nnfolder,
using the gnus-summary-move-article or gnus-summary-copy-article
commands (B m or B c).

The problem seems to be that if a message doesn't have the
X-Gnus-Article-Number header in a mailbox where all the preceeding
messages before it all _did_ have that header, the message that's
missing it and all messages after that will be invisible in Gnus.  This
also seems to happen if a message somehow gets a X-Gnus-Article-Number
that's out of sequence, lower in number than its preceeding messages in
the same mailbox.  The Xref header seems to be correlated here too.
Using 'B m' or 'B c' seems to have a very high likelyhood of messages
retaining their old metadata in the destination, and becoming invisible
in the destination folder because the old header contains data that's
now inapplicable or wrong in the new one.

I've been trying to figure out what to do to repair mailboxes like this
so Gnus will see all of them again.  Using sed to completely get rid of
all X-Gnus-Article-Number and/or Xref lines and then doing meta-g on the
group seems to work, but also seems a little brutal.  All of this makes
it very hazardous to move messages around between mailboxes, especially
if copying into an existing nnfolder, even within Gnus, with no
involvement from any other mailreaders.  It becomes necessary to do
header cleanup afterward to make all messages visible again.

What do you think would be the recommended way to hammer a mailbox's
Gnus metadata back into shape in situations like these?

-- 
+ Brent A. Busby	 + "We've all heard that a million monkeys
+ Sr. UNIX Systems Admin +  banging on a million typewriters will
+ University of Chicago	 +  eventually reproduce the entire works of
+ James Franck Institute +  Shakespeare.  Now, thanks to the Internet,
+ Materials Research Ctr +  we know this is not true." -Robert Wilensky

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: X-Gnus-Article-Number cleanup
  2021-01-26 22:04 X-Gnus-Article-Number cleanup Brent Busby
@ 2021-01-28  5:03 ` Lars Ingebrigtsen
  2021-01-28 16:49   ` Brent Busby
  0 siblings, 1 reply; 5+ messages in thread
From: Lars Ingebrigtsen @ 2021-01-28  5:03 UTC (permalink / raw)
  To: Brent Busby; +Cc: ding

Brent Busby <brent@jfi.uchicago.edu> writes:

> I've been trying to figure out what to do to repair mailboxes like this
> so Gnus will see all of them again.  Using sed to completely get rid of
> all X-Gnus-Article-Number and/or Xref lines and then doing meta-g on the
> group seems to work, but also seems a little brutal.  All of this makes
> it very hazardous to move messages around between mailboxes, especially
> if copying into an existing nnfolder, even within Gnus, with no
> involvement from any other mailreaders.  It becomes necessary to do
> header cleanup afterward to make all messages visible again.
>
> What do you think would be the recommended way to hammer a mailbox's
> Gnus metadata back into shape in situations like these?

That sounds like a good way to do it, but it'd be nice if we could
figure out what's going wrong instead.  I'm assuming you're using
nnfolder?  And you say that `B c'-ing an article to one of these groups
doesn't result in the message getting the X-Gnus-Article-Number header?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: X-Gnus-Article-Number cleanup
  2021-01-28  5:03 ` Lars Ingebrigtsen
@ 2021-01-28 16:49   ` Brent Busby
  2021-02-01  8:58     ` Lars Ingebrigtsen
  0 siblings, 1 reply; 5+ messages in thread
From: Brent Busby @ 2021-01-28 16:49 UTC (permalink / raw)
  To: ding

[-- Attachment #1: Type: text/plain, Size: 2497 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Brent Busby <brent@jfi.uchicago.edu> writes:
>
>> I've been trying to figure out what to do to repair mailboxes like this
>> so Gnus will see all of them again.  Using sed to completely get rid of
>> all X-Gnus-Article-Number and/or Xref lines and then doing meta-g on the
>> group seems to work, but also seems a little brutal.  All of this makes
>> it very hazardous to move messages around between mailboxes, especially
>> if copying into an existing nnfolder, even within Gnus, with no
>> involvement from any other mailreaders.  It becomes necessary to do
>> header cleanup afterward to make all messages visible again.
>>
>> What do you think would be the recommended way to hammer a mailbox's
>> Gnus metadata back into shape in situations like these?
>
> That sounds like a good way to do it, but it'd be nice if we could
> figure out what's going wrong instead.  I'm assuming you're using
> nnfolder?  And you say that `B c'-ing an article to one of these groups
> doesn't result in the message getting the X-Gnus-Article-Number header?

Yes, I am using nnfolder for mailboxes that get long term storage.  What
seems to sometimes happen if I copy or move an email or Usenet message
into an nnfolder is either:

- The new copy in the nnfolder will get a X-Gnus-Article-Number number
  that is not in sequence compared to the other messages in the
  nnfolder.  In cases where the new messages number is actually lower
  than the one before it, Gnus will not display that message in the
  summary, or any messages that come after it.  You will have to use
  another mailreader to see these messages, or look at them raw with a
  pager like less.

- And/or, the Xref: line will still contain the name of the mailbox it
  came from, instead of the one it's in now.  This may be appropriate
  (don't know), but it's suspicious.

Getting rid of all Xref: and X-Gnus-Article-Number: lines in the whole
nnfolder is one way of dealing with this, but I'd like to make a Perl
script that's a little smarter, so I'm wondering what a true cleanup
would need to do.

-- 
+ Brent A. Busby	 + "We've all heard that a million monkeys
+ Sr. UNIX Systems Admin +  banging on a million typewriters will
+ University of Chicago	 +  eventually reproduce the entire works of
+ James Franck Institute +  Shakespeare.  Now, thanks to the Internet,
+ Materials Research Ctr +  we know this is not true." -Robert Wilensky

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: X-Gnus-Article-Number cleanup
  2021-01-28 16:49   ` Brent Busby
@ 2021-02-01  8:58     ` Lars Ingebrigtsen
  2021-02-01 15:23       ` Brent Busby
  0 siblings, 1 reply; 5+ messages in thread
From: Lars Ingebrigtsen @ 2021-02-01  8:58 UTC (permalink / raw)
  To: Brent Busby; +Cc: ding

Brent Busby <brent@jfi.uchicago.edu> writes:

> Yes, I am using nnfolder for mailboxes that get long term storage.  What
> seems to sometimes happen if I copy or move an email or Usenet message
> into an nnfolder is either:
>
> - The new copy in the nnfolder will get a X-Gnus-Article-Number number
>   that is not in sequence compared to the other messages in the
>   nnfolder.  In cases where the new messages number is actually lower
>   than the one before it, Gnus will not display that message in the
>   summary, or any messages that come after it.  You will have to use
>   another mailreader to see these messages, or look at them raw with a
>   pager like less.

Is this repeatable?  I tried this myself now, and the nnfolder message
got an X-Gnus-Article-Number header.

In any case, please report this with `M-x report-emacs-bug' so that we
can track the problem.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: X-Gnus-Article-Number cleanup
  2021-02-01  8:58     ` Lars Ingebrigtsen
@ 2021-02-01 15:23       ` Brent Busby
  0 siblings, 0 replies; 5+ messages in thread
From: Brent Busby @ 2021-02-01 15:23 UTC (permalink / raw)
  To: ding

[-- Attachment #1: Type: text/plain, Size: 1624 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Brent Busby <brent@jfi.uchicago.edu> writes:
>
>> Yes, I am using nnfolder for mailboxes that get long term storage.  What
>> seems to sometimes happen if I copy or move an email or Usenet message
>> into an nnfolder is either:
>>
>> - The new copy in the nnfolder will get a X-Gnus-Article-Number number
>>   that is not in sequence compared to the other messages in the
>>   nnfolder.  In cases where the new messages number is actually lower
>>   than the one before it, Gnus will not display that message in the
>>   summary, or any messages that come after it.  You will have to use
>>   another mailreader to see these messages, or look at them raw with a
>>   pager like less.
>
> Is this repeatable?  I tried this myself now, and the nnfolder message
> got an X-Gnus-Article-Number header.
>
> In any case, please report this with `M-x report-emacs-bug' so that we
> can track the problem.

Like I said, it _does_ get the header, but much of the time, it's out of
sequence.  (The number may sometimes be lower than the last message that
was already in the mailbox.)  It seems to depend on what the last
message number was in the mailbox it came from, rather than the one it's
being copied to.

-- 
+ Brent A. Busby	 + "We've all heard that a million monkeys
+ Sr. UNIX Systems Admin +  banging on a million typewriters will
+ University of Chicago	 +  eventually reproduce the entire works of
+ James Franck Institute +  Shakespeare.  Now, thanks to the Internet,
+ Materials Research Ctr +  we know this is not true." -Robert Wilensky

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2021-02-01 15:23 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-26 22:04 X-Gnus-Article-Number cleanup Brent Busby
2021-01-28  5:03 ` Lars Ingebrigtsen
2021-01-28 16:49   ` Brent Busby
2021-02-01  8:58     ` Lars Ingebrigtsen
2021-02-01 15:23       ` Brent Busby

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).