Problem with thread sorting

Problem with thread sorting

[Peter Münster]

Hello,

When the root of a thread is not in the summary buffer, the thread
sorting is wrong.

Thread settings:
--8<---------------cut here---------------start------------->8---
(setq
 gnus-thread-sort-functions      '((not gnus-thread-sort-by-most-recent-date))
 gnus-sum-thread-tree-false-root nil
 gnus-sum-thread-tree-root       nil
 gnus-summary-dummy-line-format  "              %S\n"
 gnus-summary-make-false-root    'dummy
 gnus-summary-thread-gathering-function 'gnus-gather-threads-by-references)
--8<---------------cut here---------------end--------------->8---

For example in this group (gmane.emacs.gnus.general), when showing about
the last 25 articles, I get this:

--8<---------------cut here---------------start------------->8---
              Re: Asynchroneous image retrieval in HTML rendering
O   2 Thursday   David Engster    (3.8k) ├►
O   3 Thursday   Lars Magne Ingebr(4.9k) └►
O   4 Thursday   David Engster    (3.9k)  └►
O   5 Thursday   Lars Magne Ingebr(5.5k)   └►
O   6 Thursday   michael.zombok@go(3.2k) nnml backend unavailable when
                                         constructing some nnml-variables
--8<---------------cut here---------------end--------------->8---

But article 6 is older than 5.

When reopening the group with more articles (about 100), so that the
root of "Asynchroneous image retrieval in HTML rendering" shows up in
the summary buffer, the sorting is right.

How could I avoid the wrong sorting?

TIA for any hints,
-- 
           Peter

Re: Problem with thread sorting

[Dan Christensen]

pmlists@free.fr (Peter Münster) writes:

> When the root of a thread is not in the summary buffer, the thread
> sorting is wrong.

I've also noticed bad sorting behaviour, and it might be the same issue
you are seeing.  Like you, I sort threads by the most recent date, and
in my Inbox I often have every second article in a conversation (the
messages I sent are stored in my Sent group).  This causes Gnus to
gather the articles together rather than treat them as one thread.  I
think what is happening is that this gathering is done *after* the
threads are sorted.  If I am right, then the solution is to form
threads, gather them together to make "virtual" threads, and then sort
those.  However, I think the sorting code assumes that all threads are
trees, so false roots would be needed.  (Maybe false roots are always
created behind the scenes no matter how gnus-summary-make-false-root is
set?)

I'm swamped right now, so I won't be able to come up with a patch, but
maybe the above is enough to help someone else fix this.

Dan

Re: Problem with thread sorting

[Lars Magne Ingebrigtsen]

pmlists@free.fr (Peter Münster) writes:

> When reopening the group with more articles (about 100), so that the
> root of "Asynchroneous image retrieval in HTML rendering" shows up in
> the summary buffer, the sorting is right.

When sorting inside gathered threads,
`gnus-sort-gathered-threads-function' is used.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Problem with thread sorting

[Peter Münster]

On Mon, May 30 2011, Lars Magne Ingebrigtsen wrote:

> pmlists@free.fr (Peter Münster) writes:
>
>> When reopening the group with more articles (about 100), so that the
>> root of "Asynchroneous image retrieval in HTML rendering" shows up in
>> the summary buffer, the sorting is right.
>
> When sorting inside gathered threads,
> `gnus-sort-gathered-threads-function' is used.

I've already tried that, but it doesn't work. Here my setup:

(defun my-sort-gathered (h1 h2)
  (gnus-thread-sort-by-most-recent-date h2 h1))

(setq
 gnus-sort-gathered-threads-function 'my-sort-gathered
 gnus-thread-sort-functions   '((not gnus-thread-sort-by-most-recent-date)))

-- 
           Peter

Re: Problem with thread sorting

[Lars Magne Ingebrigtsen]

pmlists@free.fr (Peter Münster) writes:

> I've already tried that, but it doesn't work. Here my setup:
>
> (defun my-sort-gathered (h1 h2)
>   (gnus-thread-sort-by-most-recent-date h2 h1))
>
> (setq
>  gnus-sort-gathered-threads-function 'my-sort-gathered

I tried playing around with this a bit.  If I reverse the order of h1
and h2 in that function, the order of the gathered threads are
reversed.  So it's doing something.  But is it doing things in reverse
order? 

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Problem with thread sorting

[Dan Christensen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> pmlists@free.fr (Peter Münster) writes:
>
>> I've already tried that, but it doesn't work. Here my setup:
>>
>> (defun my-sort-gathered (h1 h2)
>>   (gnus-thread-sort-by-most-recent-date h2 h1))
>>
>> (setq
>>  gnus-sort-gathered-threads-function 'my-sort-gathered
>
> I tried playing around with this a bit.  If I reverse the order of h1
> and h2 in that function, the order of the gathered threads are
> reversed.  So it's doing something.  But is it doing things in reverse
> order? 

What do you think about my theory from another message in this thread?
I have included it below:

Dan Christensen wrote:

> I've also noticed bad sorting behaviour, and it might be the same issue
> you are seeing.  Like you, I sort threads by the most recent date, and
> in my Inbox I often have every second article in a conversation (the
> messages I sent are stored in my Sent group).  This causes Gnus to
> gather the articles together rather than treat them as one thread.  I
> think what is happening is that this gathering is done *after* the
> threads are sorted.  If I am right, then the solution is to form
> threads, gather them together to make "virtual" threads, and then sort
> those.  However, I think the sorting code assumes that all threads are
> trees, so false roots would be needed.  (Maybe false roots are always
> created behind the scenes no matter how gnus-summary-make-false-root is
> set?)
> 
> I'm swamped right now, so I won't be able to come up with a patch, but
> maybe the above is enough to help someone else fix this.
> 
> Dan

Re: Problem with thread sorting

[Lars Magne Ingebrigtsen]

Dan Christensen <jdc@uwo.ca> writes:

[...]

>> I tried playing around with this a bit.  If I reverse the order of h1
>> and h2 in that function, the order of the gathered threads are
>> reversed.  So it's doing something.  But is it doing things in reverse
>> order? 
>
> What do you think about my theory from another message in this thread?
> I have included it below:

But if I reversed the h1/h2 order, then the gathered thread was sorted
in the proper manner, I think?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Problem with thread sorting

[Peter Münster]

On Sat, Jun 04 2011, Lars Magne Ingebrigtsen wrote:

> But if I reversed the h1/h2 order, then the gathered thread was sorted
> in the proper manner, I think?

No. I see 2 issues: the sorting of the sub-threads in a gathered thread,
and the sorting of the gathered threads themselves. Both seem wrong to
me.

It would be nice, if the gathered threads behaved the same as normal
threads. The sorting I'm looking for:
(setq gnus-thread-sort-functions
      '((not gnus-thread-sort-by-most-recent-date)))

-- 
           Peter

Re: Problem with thread sorting

[Lars Magne Ingebrigtsen]

pmlists@free.fr (Peter Münster) writes:

> No. I see 2 issues: the sorting of the sub-threads in a gathered thread,
> and the sorting of the gathered threads themselves. Both seem wrong to
> me.

Right...

This is how it's done:

	   (gnus-sort-gathered-threads
	    (funcall gnus-summary-thread-gathering-function
		     (gnus-sort-threads
		      (gnus-cut-threads (gnus-make-threads)))))

First each thread is sorted, then they are gathered, and then the
threads inside the gathered threads are sorted individually.

You would rather that the threads be gathered first, then the threads
are sorted (somehow), and then the threads inside the gathered threads
would be sorted.

However, I don't think that can be done.  Take the default sorting
parameter now -- sort by root article number.  What's the root article
number of a gathered thread?  The article number of the first thread?
The article number of the first thread after sorting the sub-threads?
Or what?

So I think what it does now is reasonable.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Problem with thread sorting

[Dan Christensen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> This is how it's done:
>
> 	   (gnus-sort-gathered-threads
> 	    (funcall gnus-summary-thread-gathering-function
> 		     (gnus-sort-threads
> 		      (gnus-cut-threads (gnus-make-threads)))))
>
> First each thread is sorted, then they are gathered, and then the
> threads inside the gathered threads are sorted individually.
>
> You would rather that the threads be gathered first, then the threads
> are sorted (somehow), and then the threads inside the gathered threads
> would be sorted.
>
> However, I don't think that can be done.  Take the default sorting
> parameter now -- sort by root article number.  What's the root article
> number of a gathered thread?  The article number of the first thread?
> The article number of the first thread after sorting the sub-threads?
> Or what?

You are right that a somewhat arbitrary decision needs to be made here.

> So I think what it does now is reasonable.

I think the current way is also arbitrary, but is arbitrary in a way
that can't be controlled.  For example, suppose that before gathering,
the threads are sorted as

Thread 1
Thread 2 *
Thread 3
Thread 4 *
Thread 5

and that threads 2 and 4 are going to be gathered together in the next
step.  I believe that what happens now is that you end up with both in
either the position of thread 2 or thread 4, and that there are no user
settings to control this.  For sorting by root article number, it would
be equivalent to gather the threads first, and then sort them using one
of the arbitrary suggestions you make above, so there is no loss in
switching to this method.  But for sorting functions that traverse the
thread, like most-recent-date, sorting after gathering would produce
correct results.

In principle the current method doesn't really make sense, since why
would you sort things into a list, and then just grab bunches of them
and arbitrarily move them around.  You should instead do the grouping
first, so that the sort function has full information about the final
grouping.

As to how to treat the root, I guess the root of one of the gathered
threads should be simply chosen to be the root.  This could be done
using the existing settings for how gathered threads are handled,
or could be chosen using the sorting function for gathered threads.
Or a dummy root could be greated, and given a virtual article number,
date, score, etc, derived from the gathered threads.

[If I recall correctly, the order of articles within a thread is
controlled by the same sort function as the threads themselves.  If so,
it seems that this could extend to the gathered threads as well,
eliminating the need for a separate sort function to be set by the user.
In which case, the thread sorting function could be used to choose the
root of the thread.]

So there are some issues to sort out (no pun intended), but I think it
could make sorting much better and at least predictable.

Dan

PS: And just to emphasize, the current method seems to *always* get the
sorting wrong when gathering threads sorted by (not most-recent-date).

Re: Problem with thread sorting

[Lars Magne Ingebrigtsen]

Dan Christensen <jdc@uwo.ca> writes:

> So there are some issues to sort out (no pun intended), but I think it
> could make sorting much better and at least predictable.

I agree that the current sorting going on in relation to gathered
threads isn't particularly nice, but I'm not quite sure that it can be
made to be consistent, either, so...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/