Correct ordering of items in mail threads

Correct ordering of items in mail threads

[Pankaj Jangid]

Following is the snapshot from the go-nuts mailing list. Note that the
last item in the list was received before the parent item. This should
not be possible. What changes should I make in the in the Gnus settings
so that it appears in correct order. By correct order, I mean the last
item should have appeard at the root level, instead of appearing as a
child of the one above it.

--8<---------------cut here---------------start------------->8---
R.  02:44 AM [ 123: Victor Giordano ] => Re: [go-nuts] Re: About allowing conversion from bool to int
R.  03:17 AM [ 142: Volker Dobler   ]   +-> 
R.  06:01 AM [  21: Ian Lance Taylor]   +-> 
R.  08:44 AM [  57: Victor Giordano ]     +-> 
R.  03:44 AM { 179: Victor Giordano }       +-> Fwd: [go-nuts] Re: About allowing conversion from bool to int
--8<---------------cut here---------------end--------------->8---

Following is the relevant section of my configuration:

--8<---------------cut here---------------start------------->8---
 ;; Summary buffer formating
 '(gnus-sum-opening-bracket             "[")
 '(gnus-sum-closing-bracket             "]")
 '(gnus-sum-opening-bracket-adopted     "{")
 '(gnus-sum-closing-bracket-adopted     "}")
 '(gnus-sum-thread-tree-root            "=> ")
 '(gnus-sum-thread-tree-false-root      ">> ")
 '(gnus-sum-thread-tree-single-indent   "=> ")
 '(gnus-sum-thread-tree-indent          "  ")
 '(gnus-sum-thread-tree-vertical        "| ")
 '(gnus-sum-thread-tree-leaf-with-other "+-> ")
 '(gnus-sum-thread-tree-single-leaf     "+-> ")

 '(gnus-user-date-format-alist
   '(((gnus-seconds-today) . "%H:%M %p")
     (t . "%d-%m-%y")))
 ;; '(gnus-summary-line-format
 ;;   "%U%R%z %-5&user-date; %(%[%*%-13,13f%]%) %B%s\n")
 '(gnus-summary-line-format
   "%U%R%z %-5&user-date; %(%[%4L: %-16,16f%]%) %B%s\n")
 
 ;; Threading
 '(gnus-thread-sort-functions
   '(gnus-thread-sort-by-number))
 '(gnus-summary-thread-gathering-function
   'gnus-gather-threads-by-references)
--8<---------------cut here---------------end--------------->8---

Re: Correct ordering of items in mail threads

[Eric S Fraga]

On Saturday, 21 Oct 2023 at 10:41, Pankaj Jangid wrote:
>  ;; Threading
>  '(gnus-thread-sort-functions
>    '(gnus-thread-sort-by-number))
>  '(gnus-summary-thread-gathering-function
>    'gnus-gather-threads-by-references)
>

I have

(setq gnus-sort-gathered-threads-function 'gnus-thread-sort-by-date)

along with similar (but slightly different) settings for the variables
you've set.  This variable says how to sort *within* a thread which may
be what you are looking for.

-- 
Eric S Fraga via gnus (Emacs 30.0.50 2023-09-14) on Debian 12.1

Re: Correct ordering of items in mail threads

[Pankaj Jangid]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Saturday, 21 Oct 2023 at 10:41, Pankaj Jangid wrote:
>>  ;; Threading
>>  '(gnus-thread-sort-functions
>>    '(gnus-thread-sort-by-number))
>>  '(gnus-summary-thread-gathering-function
>>    'gnus-gather-threads-by-references)
>>
>
> I have
>
> (setq gnus-sort-gathered-threads-function 'gnus-thread-sort-by-date)
>
> along with similar (but slightly different) settings for the variables
> you've set.  This variable says how to sort *within* a thread which may
> be what you are looking for.

In fact, I wanted to use 'gnus-thread-sort-by-date for
gnus-thread-sort-functions. But the docstring discouraged me.

--8<---------------cut here---------------start------------->8---
Also note that sorting by date is often much slower than sorting by
number, and the sorting order is very similar.  (Sorting by date means
sorting by the time the message was sent, sorting by number means
sorting by arrival time.)
--8<---------------cut here---------------end--------------->8---

Re: Correct ordering of items in mail threads

[Adam Sjøgren]

Pankaj writes:

> In fact, I wanted to use 'gnus-thread-sort-by-date for
> gnus-thread-sort-functions. But the docstring discouraged me.
>
> Also note that sorting by date is often much slower than sorting by
> number, and the sorting order is very similar.

These warnings are less and less relevant with the speed of personal
computers these days, I think.

If you give it a go, let us know if you can feel any difference!


  Best regards,

    Aam

-- 
 "It's either spectacular, unbelievable success, or         Adam Sjøgren
  crushing, hopeless defeat! There is no middle        asjo@koldfront.dk
  ground!"

Re: Correct ordering of items in mail threads

[Eric Abrahamsen]

Pankaj Jangid <pankaj@codeisgreat.org> writes:

> Eric S Fraga <e.fraga@ucl.ac.uk> writes:
>
>> On Saturday, 21 Oct 2023 at 10:41, Pankaj Jangid wrote:
>>>  ;; Threading
>>>  '(gnus-thread-sort-functions
>>>    '(gnus-thread-sort-by-number))
>>>  '(gnus-summary-thread-gathering-function
>>>    'gnus-gather-threads-by-references)
>>>
>>
>> I have
>>
>> (setq gnus-sort-gathered-threads-function 'gnus-thread-sort-by-date)
>>
>> along with similar (but slightly different) settings for the variables
>> you've set.  This variable says how to sort *within* a thread which may
>> be what you are looking for.
>
> In fact, I wanted to use 'gnus-thread-sort-by-date for
> gnus-thread-sort-functions. But the docstring discouraged me.
>
> Also note that sorting by date is often much slower than sorting by
> number, and the sorting order is very similar.  (Sorting by date means
> sorting by the time the message was sent, sorting by number means
> sorting by arrival time.)

I think this is the trade-off, though -- sorting by number _usually_
means sorting by arrival time, but there's no contract there. It's
entirely possible for messages to arrive out-of-order, and when they do,
you get a weird display. I think a lot of what was slow in Gnus is
gradually becoming less slow (with things like native compilation and
whatnot) and you might try sorting by date and just seeing how bad it is.

Re: Correct ordering of items in mail threads

[Adam Sjøgren]

Eric writes:

> sorting by number _usually_ means sorting by arrival time, but
> there's no contract there.

When do the numbers given to articles in Gnus not reflect _arrival_
time?


  Best regards,

    Adam

-- 
 "Some kind of night into your darkness                     Adam Sjøgren
  Colors your eyes with what's not there"              asjo@koldfront.dk

Re: Correct ordering of items in mail threads

[Eric Abrahamsen]

Adam Sjøgren <asjo@koldfront.dk> writes:

> Eric writes:
>
>> sorting by number _usually_ means sorting by arrival time, but
>> there's no contract there.
>
> When do the numbers given to articles in Gnus not reflect _arrival_
> time?

Sorry, that was backwards -- I meant the articles don't always arrive at
Gnus in the order they were originally authored.

Re: Correct ordering of items in mail threads

[Adam Sjøgren]

Eric writes:

> I meant the articles don't always arrive at Gnus in the order they
> were originally authored.

Got it :-)

I have configured Gnus to sort a bunch of groups by date instead of by
number, and I can't feel the difference (in speed) -- this on a 4 year
old laptop.

Maybe I'm just growing slower ;-)


  Best regards,

    Adam

-- 
 "What do you do for a living, man?"                        Adam Sjøgren
 "Oh, I confound expectations."                        asjo@koldfront.dk

Re: Correct ordering of items in mail threads

[Eric Abrahamsen]

Adam Sjøgren <asjo@koldfront.dk> writes:

> Eric writes:
>
>> I meant the articles don't always arrive at Gnus in the order they
>> were originally authored.
>
> Got it :-)
>
> I have configured Gnus to sort a bunch of groups by date instead of by
> number, and I can't feel the difference (in speed) -- this on a 4 year
> old laptop.
>
> Maybe I'm just growing slower ;-)

Or all this just seems to matter less!

Re: Correct ordering of items in mail threads

[Pankaj Jangid]

>> I have configured Gnus to sort a bunch of groups by date instead of by
>> number, and I can't feel the difference (in speed) -- this on a 4 year
>> old laptop.
>>
>> Maybe I'm just growing slower ;-)
>
> Or all this just seems to matter less!

I am using the following configuration since last 24h.

#+begin_src elisp
 ;; Threading
 '(gnus-thread-sort-functions
   '(gnus-thread-sort-by-date))
 '(gnus-summary-thread-gathering-function
   'gnus-gather-threads-by-references)
#+end_src

I confirm that there is no noticeable lag in generating the summary
buffer. However I have yet to verify the order of items.

--8<---------------cut here---------------start------------->8---
E   20-10-23 23:18 PM [ 107: Victor Giordano ] => Re: [go-nuts] Re: About allowing conversion from bool to int
O   21-10-23 02:44 AM [ 123: Victor Giordano ]   +-> 
O   21-10-23 03:17 AM [ 142: Volker Dobler   ]     +-> 
O   21-10-23 06:01 AM [  21: Ian Lance Taylor]     +-> 
O   21-10-23 08:44 AM [  57: Victor Giordano ]       +-> 
E   21-10-23 18:27 PM [  69: Victor Giordano ]         +-> 
O   21-10-23 03:44 AM { 179: Victor Giordano }           +-> Fwd: [go-nuts] Re: About allowing conversion from bool to int
--8<---------------cut here---------------end--------------->8---

In the above snapshot, items are still ordered incorrectly. This could
be because of some other configuration. But I am not sure what. The last
item in the above snapshot cannot be child of the one before it, even
when I am using gnus-gather-threads-by-references. Is the following
configuration correct? I am using this to decorate summary buffer.

 ;; Summary buffer formating
 '(gnus-sum-opening-bracket             "[")
 '(gnus-sum-closing-bracket             "]")
 '(gnus-sum-opening-bracket-adopted     "{")
 '(gnus-sum-closing-bracket-adopted     "}")
 '(gnus-sum-thread-tree-root            "=> ")
 '(gnus-sum-thread-tree-false-root      ">> ")
 '(gnus-sum-thread-tree-single-indent   "=> ")
 '(gnus-sum-thread-tree-indent          "  ")
 '(gnus-sum-thread-tree-vertical        "| ")
 '(gnus-sum-thread-tree-leaf-with-other "+-> ")
 '(gnus-sum-thread-tree-single-leaf     "+-> ")

 '(gnus-user-date-format-alist
   '((t . "%d-%m-%y %H:%M %p")))
 '(gnus-summary-line-format
   "%U%R%z %-5&user-date; %(%[%4L: %-16,16f%]%) %B%s\n")

Re: Correct ordering of items in mail threads

[Adam Sjøgren]

Pankaj writes:

> E   20-10-23 23:18 PM [ 107: Victor Giordano ] => Re: [go-nuts] Re: About allowing conversion from bool to int
> O   21-10-23 02:44 AM [ 123: Victor Giordano ]   +-> 
> O   21-10-23 03:17 AM [ 142: Volker Dobler   ]     +-> 
> O   21-10-23 06:01 AM [  21: Ian Lance Taylor]     +-> 
> O   21-10-23 08:44 AM [  57: Victor Giordano ]       +-> 
> E   21-10-23 18:27 PM [  69: Victor Giordano ]         +-> 
> O   21-10-23 03:44 AM { 179: Victor Giordano }           +-> Fwd: [go-nuts] Re: About allowing conversion from bool to int
>
> In the above snapshot, items are still ordered incorrectly.

Just to make sure: you are expecting the article in the last row to be
shown in the 4th row?

I would expect the reason that the last line is put last is because
the References: header includes the Message-ID of the second to last
article.

Is that not the case?

But wait, the curly braces means "adopted". What does that mean?

 · https://www.gnu.org/software/emacs/manual/html_node/gnus/Loose-Threads.html

What happens if you change the value of gnus-summary-make-false-root?
Perhaps to nil?


  Best regards,

   Adam

-- 
 "Our voodoo-dolls are full of hopes"                       Adam Sjøgren
                                                       asjo@koldfront.dk

Re: Correct ordering of items in mail threads

[Pankaj Jangid]

Adam Sjøgren <asjo@koldfront.dk> writes:

> Pankaj writes:
>
>> E   20-10-23 23:18 PM [ 107: Victor Giordano ] => Re: [go-nuts] Re: About allowing conversion from bool to int
>> O   21-10-23 02:44 AM [ 123: Victor Giordano ]   +-> 
>> O   21-10-23 03:17 AM [ 142: Volker Dobler   ]     +-> 
>> O   21-10-23 06:01 AM [  21: Ian Lance Taylor]     +-> 
>> O   21-10-23 08:44 AM [  57: Victor Giordano ]       +-> 
>> E   21-10-23 18:27 PM [  69: Victor Giordano ]         +-> 
>> O   21-10-23 03:44 AM { 179: Victor Giordano }           +-> Fwd: [go-nuts] Re: About allowing conversion from bool to int
>>
>> In the above snapshot, items are still ordered incorrectly.
>
> Just to make sure: you are expecting the article in the last row to be
> shown in the 4th row?

No. I want the indentation to be proper. It cannot be child of a later message.

> I would expect the reason that the last line is put last is because
> the References: header includes the Message-ID of the second to last
> article.
>
> Is that not the case?

As we can see this is not possible. Becase the second last message came
after the last one.

> But wait, the curly braces means "adopted". What does that mean?
>
>  · https://www.gnu.org/software/emacs/manual/html_node/gnus/Loose-Threads.html
>
> What happens if you change the value of gnus-summary-make-false-root?
> Perhaps to nil?

Wow. Good pointer. The default value of the variable is 'adopt'. I
experimented with the other three possible values. And here are the
results, along with the quoted docstring

> If this variable is ‘adopt’, Gnus will make one of the "children"
> the parent and mark all the step-children as such.

> If this variable is ‘none’, Gnus will not make a false root, but just
> present the sub-threads after another.

(setq gnus-summary-make-false-root 'none)

--8<---------------cut here---------------start------------->8---
E   20-10-23 23:18 PM [ 107: Victor Giordano ] => Re: [go-nuts] Re: About allowing conversion from bool to int
O   21-10-23 02:44 AM [ 123: Victor Giordano ] | +-> 
O   21-10-23 03:17 AM [ 142: Volker Dobler   ] |   +-> 
O   21-10-23 06:01 AM [  21: Ian Lance Taylor] |   +-> 
O   21-10-23 08:44 AM [  57: Victor Giordano ] |     +-> 
E   21-10-23 18:27 PM [  69: Victor Giordano ] |       +-> 
O   21-10-23 03:44 AM [ 179: Victor Giordano ] => Fwd: [go-nuts] Re: About allowing conversion from bool to int
--8<---------------cut here---------------end--------------->8---

> If this variable is ‘dummy’, Gnus will create a dummy root that will
> have all the sub-threads as children.

(setq gnus-summary-make-false-root 'dummy)

--8<---------------cut here---------------start------------->8---
   :                             : Re: [go-nuts] Re: About allowing conversion from bool to int
E   20-10-23 23:18 PM [ 107: Victor Giordano ] +-> 
O   21-10-23 02:44 AM [ 123: Victor Giordano ] | +-> 
O   21-10-23 03:17 AM [ 142: Volker Dobler   ] |   +-> 
O   21-10-23 06:01 AM [  21: Ian Lance Taylor] |   +-> 
O   21-10-23 08:44 AM [  57: Victor Giordano ] |     +-> 
E   21-10-23 18:27 PM [  69: Victor Giordano ] |       +-> 
O   21-10-23 03:44 AM [ 179: Victor Giordano ] +-> Fwd: [go-nuts] Re: About allowing conversion from bool to int
--8<---------------cut here---------------end--------------->8---

> If this variable is ‘empty’, the "children" are printed with empty
> subject fields.  (Or rather, they will be printed with a string given
> by the ‘gnus-summary-same-subject’ variable.)

(setq gnus-summary-make-false-root 'empty)

--8<---------------cut here---------------start------------->8---
E   20-10-23 23:18 PM [ 107: Victor Giordano ] => Re: [go-nuts] Re: About allowing conversion from bool to int
O   21-10-23 02:44 AM [ 123: Victor Giordano ] | +-> 
O   21-10-23 03:17 AM [ 142: Volker Dobler   ] |   +-> 
O   21-10-23 06:01 AM [  21: Ian Lance Taylor] |   +-> 
O   21-10-23 08:44 AM [  57: Victor Giordano ] |     +-> 
E   21-10-23 18:27 PM [  69: Victor Giordano ] |       +-> 
O   21-10-23 03:44 AM [ 179: Victor Giordano ] => Fwd: [go-nuts] Re: About allowing conversion from bool to int
--8<---------------cut here---------------end--------------->8---

I think 'none' and 'empty' works best for the situation. May be I should
try each of these for a couple of days and decide.

Re: Correct ordering of items in mail threads

[Adam Sjøgren]

Pankaj writes:

> Adam Sjøgren <asjo@koldfront.dk> writes:

>> Pankaj writes:

>>> In the above snapshot, items are still ordered incorrectly.

>> Just to make sure: you are expecting the article in the last row to be
>> shown in the 4th row?

> No. I want the indentation to be proper. It cannot be child of a later message.

This illustrates nicely that only stating that something is incorrect,
without saying what you expect, is insufficient :-)

> As we can see this is not possible.

I cannot see this without examining the headers, maybe you can :-)


  Best regards,

    Adam


(I wanted to point you at Eric S Fraga's suggestion when I stumbled
over the "adopted"-thing, but I could find it at the time - luckily he
has suggested looking at gnus-sort-gathered-threads-function again -
setting that does what you want, right?!)

-- 
 "Oh, great altar of passive entertainment... bestow        Adam Sjøgren
  upon me thy discordant images at such speed as to    asjo@koldfront.dk
  render linear thought impossible!"

Re: Correct ordering of items in mail threads

[Pankaj Jangid]

Adam Sjøgren <asjo@koldfront.dk> writes:

>> As we can see this is not possible.
>
> I cannot see this without examining the headers, maybe you can :-)

My apologies for any confusion. What I meant to say is that, based on
the date-time, an earlier message wouldn't typically have an
'in-reply-to' header referencing a later message.

Re: Correct ordering of items in mail threads

[Björn Bidar]

Pankaj Jangid <pankaj@codeisgreat.org> writes:

>> If this variable is ‘dummy’, Gnus will create a dummy root that will
>> have all the sub-threads as children.
>
> (setq gnus-summary-make-false-root 'dummy)

Personally I found dummy works the best for me until I press
A T to fetch the rest of thread.

No solution is perfect.

Re: Correct ordering of items in mail threads

[Pankaj Jangid]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

> Pankaj Jangid <pankaj@codeisgreat.org> writes:
>
>>> If this variable is ‘dummy’, Gnus will create a dummy root that will
>>> have all the sub-threads as children.
>>
>> (setq gnus-summary-make-false-root 'dummy)
>
> Personally I found dummy works the best for me until I press
> A T to fetch the rest of thread.
>
> No solution is perfect.

That's true.

And 'dummy' doesn't follow the summary line format so it looks ugly to
me. For now I am settling for 'none'.

Re: Correct ordering of items in mail threads

[Eric S Fraga]

Have you tried what I suggested initially:

(setq gnus-sort-gathered-threads-function 'gnus-thread-sort-by-date)

which tells gnus how to sort the individual posts in a thread, as
opposed to how to sort the threads in the summary?

-- 
Eric S Fraga via gnus (Emacs 30.0.50 2023-06-19) on Debian 12.0

Re: Correct ordering of items in mail threads

[Pankaj Jangid]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Have you tried what I suggested initially:
>
> (setq gnus-sort-gathered-threads-function 'gnus-thread-sort-by-date)
>

Yes. I tried this suggestion. But this doesn't have any effect on the
output. I tried it with all the combinations of
"gnus-summary-make-false-root". Default being 'adopt'.

Re: Correct ordering of items in mail threads

[Eric S Fraga]

On Tuesday, 24 Oct 2023 at 09:58, Pankaj Jangid wrote:
> Yes. I tried this suggestion. But this doesn't have any effect on the
> output. I tried it with all the combinations of
> "gnus-summary-make-false-root". Default being 'adopt'.

Interesting.  Adding this to my config (years ago) fixed the problem I
was having that, IIRC, was essentially the same as what you are seeing.
Oh well, complex systems...

-- 
Eric S Fraga via gnus (Emacs 30.0.50 2023-06-19) on Debian 12.0