What is a "virtual" group?

What is a "virtual" group?

[Dave Abrahams]

I'm not referring here to nnvirtual, but to the virtual property as it
appears in `gnus-valid-select-methods' or in 

  (gnus-declare-backend "nnir" 'mail 'virtual)

which is done in nnir.el

The context for my question is
http://article.gmane.org/gmane.emacs.gnus.general/82136, where I'm
wondering whether it's OK to label "nndoc" groups as virtual

Answers to the other questions at the end of that article would also be
most appreciated:

* Should gnus-warp-to-article really be restricting itself to virtual
  groups?
* Should I be doing something different in `gnus-goto-article' to build
  a group with my message in it?

-- 
Dave Abrahams
BoostPro Computing                  Software Development        Training
http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost

Re: What is a "virtual" group?

[Steinar Bang]

>>>>> Dave Abrahams <dave@boostpro.com>:

> I'm not referring here to nnvirtual, but to the virtual property as it
> appears in `gnus-valid-select-methods' or in 

>   (gnus-declare-backend "nnir" 'mail 'virtual)

> which is done in nnir.el

"Virtual group" in this context refers to a temporary group holding the
results of the nnir search.

> The context for my question is
> http://article.gmane.org/gmane.emacs.gnus.general/82136, where I'm
> wondering whether it's OK to label "nndoc" groups as virtual

Well,... I can't answer that.

> Answers to the other questions at the end of that article would also be
> most appreciated:

> * Should gnus-warp-to-article really be restricting itself to virtual
>   groups?

For nnir what it does is to move out of the group and into to the group
the article was found in, and show the article there.  In the group it
belongs to, it can be replied to with the appropriate sender address,
and have its tick marks changed.

> * Should I be doing something different in `gnus-goto-article' to build
>   a group with my message in it?

I can't answer that either.

Re: What is a "virtual" group?

[Dave Abrahams]

on Sun Sep 16 2012, Steinar Bang <sb-AT-dod.no> wrote:

>>>>>> Dave Abrahams <dave@boostpro.com>:
>
>> I'm not referring here to nnvirtual, but to the virtual property as it
>> appears in `gnus-valid-select-methods' or in 
>
>>   (gnus-declare-backend "nnir" 'mail 'virtual)
>
>> which is done in nnir.el
>
> "Virtual group" in this context refers to a temporary group holding
> the results of the nnir search.

Just to be clear, I'm trying to understand the purpose of labelling a
group "virtual."  In other words, what is that supposed to /mean/ in
Gnus-land?

The "holding the results of the nnir search" part of your response is
not particularly relevant to the question, since that just happens to be
the case for this particular "virtual" group.  So I'm left with

   virtual == temporary

Is that really all it means?

>> The context for my question is
>> http://article.gmane.org/gmane.emacs.gnus.general/82136, where I'm
>> wondering whether it's OK to label "nndoc" groups as virtual
>
> Well,... I can't answer that.

I think only someone who understands the intended meaning of "virtual"
could answer it.  Lars?

>> Answers to the other questions at the end of that article would also be
>> most appreciated:
>
>> * Should gnus-warp-to-article really be restricting itself to virtual
>>   groups?
>
> For nnir what it does is to move out of the group and into to the group
> the article was found in, and show the article there.  In the group it
> belongs to, it can be replied to with the appropriate sender address,
> and have its tick marks changed.

I know what gnus-warp-to-article does, intimately.  The question is,is
there a reason that gnus-warp-to-article only does something in groups
marked "virtual," and if so, what is that reason?

>> * Should I be doing something different in `gnus-goto-article' to build
>>   a group with my message in it?
>
> I can't answer that either.

Thanks for trying.

-- 
Dave Abrahams
BoostPro Computing                  Software Development        Training
http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost

Re: What is a "virtual" group?

[Adam Sjøgren]

On Sun, 16 Sep 2012 11:08:06 -0400, Dave wrote:

> The question is,is there a reason that gnus-warp-to-article only does
> something in groups marked "virtual," and if so, what is that reason?

Maybe 'virtual' means "groups that shows articles that don't live in
this group for real, thus warping to their original/real home makes
sense"?

If you see an article where it "physically" belongs, warping to it
doesn't quite make sense, because that wouldn'r change anything, right?


  Just a stupid guess,

    Adam

-- 
 "Hur långt man än har kommit                                 Adam Sjøgren
  Är det alltid längre kvar"                             asjo@koldfront.dk

Re: What is a "virtual" group?

[Steinar Bang]

>>>>> Dave Abrahams <dave@boostpro.com>:

> So I'm left with

>    virtual == temporary

> Is that really all it means?

I vaguely remember when first reading about virtual groups that it
looked like you could combine several groups into one.  But I never had
a use case for that, so I never investigated further.

I also may have been mistaken in my assumptions.

> I think only someone who understands the intended meaning of "virtual"
> could answer it.

Probably.

> Lars?

Very probably.

Re: What is a "virtual" group?

[Lars Ingebrigtsen]

Dave Abrahams <dave@boostpro.com> writes:

> I think only someone who understands the intended meaning of "virtual"
> could answer it.  Lars?

A virtual group is a group that has a different group or groups backing
it, so it has no real backing store of its own.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Lars Magne Ingebrigtsen

Re: What is a "virtual" group?

[Dave Abrahams]

on Tue Dec 25 2012, Lars Ingebrigtsen <larsi-AT-gnus.org> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
>> I think only someone who understands the intended meaning of "virtual"
>> could answer it.  Lars?
>
> A virtual group is a group that has a different group or groups backing
> it, so it has no real backing store of its own.

Thanks for answering, Lars!  I hope you don't mind addressing a few
follow-on questions....

* Is the "has a different group or groups backing it" part essential to
  nnvirtual, or is it really more about "has no real backing store of
  its own?"

* Suppose I write a function that takes a group as a parameter.  How
  does virtual-ness affect what my function should expect to be able to
  do with that group?  Are there certain operations that should be
  valid or invalid because of the group's virtual-ness?

* Should an nndoc group be considered virtual?

* Does the check for virtual-ness in gnus-warp-to-article actually
  belong there?  After all, real groups shouldn't implement the
  warp-to-article backend function, should they?

-- 
Dave Abrahams
BoostPro Computing                  Software Development        Training
http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost

Re: What is a "virtual" group?

[Lars Ingebrigtsen]

Dave Abrahams <dave@boostpro.com> writes:

> * Is the "has a different group or groups backing it" part essential to
>   nnvirtual, or is it really more about "has no real backing store of
>   its own?"

It's more the former.  It's a group that's used to access data from a
different backend.

> * Suppose I write a function that takes a group as a parameter.  How
>   does virtual-ness affect what my function should expect to be able to
>   do with that group?  Are there certain operations that should be
>   valid or invalid because of the group's virtual-ness?

It's possible.  Grep for "virtual".

> * Should an nndoc group be considered virtual?

No.

> * Does the check for virtual-ness in gnus-warp-to-article actually
>   belong there?  After all, real groups shouldn't implement the
>   warp-to-article backend function, should they?

I don't know.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Lars Magne Ingebrigtsen

Re: What is a "virtual" group?

[Dave Abrahams]

on Tue Dec 25 2012, Lars Ingebrigtsen <larsi-AT-gnus.org> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
>> * Is the "has a different group or groups backing it" part essential to
>>   nnvirtual, or is it really more about "has no real backing store of
>>   its own?"
>
> It's more the former.  It's a group that's used to access data from a
> different backend.

OK, thanks.  That's very clear.  In trying to suggest corresponding
documentation edits, though, I arrive at 6.7.1 Virtual Groups in the
manual and find:

  An "nnvirtual group" is really nothing more than a collection of
  other groups.

where I expect to find a definition of "virtual group."  Maybe we need
to insert something like:

   A "virtual group" is a group whose backing store is actually provided
   by a different group or groups.  At this writing there are two kinds
   of virtual groups: nnvirtual groups, which aggregate all articles
   from their backing groups, and nnir search groups, which aggregate
   specific articles found in their backing groups.

I'm not sure about there only being two types of virtual groups.  Is
nnagent a virtual group?  It does use nnvirtual-convert-headers.

Also, should "virtual" in the user interface and docstrings be changed
to "nnvirtual?"  That seems to be what it really means, most of the time.

>> * Suppose I write a function that takes a group as a parameter.  How
>>   does virtual-ness affect what my function should expect to be able to
>>   do with that group?  Are there certain operations that should be
>>   valid or invalid because of the group's virtual-ness?
>
> It's possible.  Grep for "virtual".

Well, I have.  I did try hard to answer this question for myself before
posting, but failed.

>> * Should an nndoc group be considered virtual?
>
> No.

Interesting.  So how do we ever get here:

  http://git.gnus.org/cgit/gnus.git/tree/lisp/gnus-sum.el#n9253

?

>
>> * Does the check for virtual-ness in gnus-warp-to-article actually
>>   belong there?  After all, real groups shouldn't implement the
>>   warp-to-article backend function, should they?
>
> I don't know.

Would you be open to a patch that removes the check?

-- 
Dave Abrahams
BoostPro Computing                  Software Development        Training
http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost

Re: What is a "virtual" group?

[Lars Ingebrigtsen]

Dave Abrahams <dave@boostpro.com> writes:

> OK, thanks.  That's very clear.  In trying to suggest corresponding
> documentation edits, though, I arrive at 6.7.1 Virtual Groups in the
> manual and find:
>
>   An "nnvirtual group" is really nothing more than a collection of
>   other groups.

It's a virtual group as far as the user is concerned.  The word
"virtual" is reused as a backend property to mean something else.

>>> * Should an nndoc group be considered virtual?
>>
>> No.
>
> Interesting.  So how do we ever get here:
>
>   http://git.gnus.org/cgit/gnus.git/tree/lisp/gnus-sum.el#n9253

I'm not sure what the question is?  It tries to open an nnvirtual group
and then says something about not being able to do so if it can't?

>>> * Does the check for virtual-ness in gnus-warp-to-article actually
>>>   belong there?  After all, real groups shouldn't implement the
>>>   warp-to-article backend function, should they?
>>
>> I don't know.
>
> Would you be open to a patch that removes the check?

I'm not familiar with that code.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Lars Magne Ingebrigtsen

Re: What is a "virtual" group?

[Dave Abrahams]

on Tue Dec 25 2012, Lars Ingebrigtsen <larsi-AT-gnus.org> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
>> OK, thanks.  That's very clear.  In trying to suggest corresponding
>> documentation edits, though, I arrive at 6.7.1 Virtual Groups in the
>> manual and find:
>>
>>   An "nnvirtual group" is really nothing more than a collection of
>>   other groups.
>
> It's a virtual group as far as the user is concerned.  The word
> "virtual" is reused as a backend property to mean something else.

Yes, I understand that.  However as with all things Emacs, the "user"
role flows into the "extender" and "developer" roles without any clear
boundary.  If that document is supposed to be just for "users," then we
need a separate one for people like me who are trying to contribute to
the codebase.

>>>> * Should an nndoc group be considered virtual?
>>>
>>> No.
>>
>> Interesting.  So how do we ever get here:
>>
>>   http://git.gnus.org/cgit/gnus.git/tree/lisp/gnus-sum.el#n9253
>
> I'm not sure what the question is?  It tries to open an nnvirtual group
> and then says something about not being able to do so if it can't?

OK, the message says "Couldn't select virtual nndoc group".  But nndoc
groups are not today (and according to you, should never be) virtual.
So is this essentially unreachable code unless something gets corrupted?
Or am I mis-reading the message somehow.

>>>> * Does the check for virtual-ness in gnus-warp-to-article actually
>>>>   belong there?  After all, real groups shouldn't implement the
>>>>   warp-to-article backend function, should they?
>>>
>>> I don't know.
>>
>> Would you be open to a patch that removes the check?
>
> I'm not familiar with that code.

Sorry, I don't know how to interpret your answer.

I am trying to figure out how to contribute a really cool (and for me,
essential) piece of functionality to Gnus: the ability to, given a
message-id, find and display the message in the context of the entire
thread in which that message participates, even across groups.  Of
course I'm presuming you think that's worth having in Gnus, but I could
be wrong.  If so, just tell me and I'll stop pestering you.

I have implemented something that works, but currently requires making
nndoc groups virtual or disabling that check.  You've already said that
making nndoc groups virtual is inappropriate.  If disabling the check is
appropriate, that would solve the problem easily.  

The other possible approach would be to discover a way to implement the
following function using some kind of virtual group (instead of nndoc).
I think that's probably the best approach but I have tried myself many
times and failed, and I have asked here many times for guidance without
success... so I eventually gave up.

#+begin_src lisp
(defun gnus-goto-article (message-id)
  (with-temp-buffer
    (erase-buffer)
    ;; Insert dummy article
    (insert (format "From nobody Tue Sep 13 22:05:34 2011\n\n"))
    (gnus-group-read-ephemeral-group
     message-id
     `(nndoc ,message-id
             (nndoc-address ,(current-buffer))
             (nndoc-article-type mbox))
     :activate
     (cons (current-buffer) gnus-current-window-configuration)
     (not :request-only)
     '(-1) ; :select-articles
     (not :parameters)
     0     ; :number
     )
    (gnus-summary-refer-article message-id)
    ))
#+end_src

-- 
Dave Abrahams
BoostPro Computing                  Software Development        Training
http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost

Re: What is a "virtual" group?

[Lars Ingebrigtsen]

Dave Abrahams <dave@boostpro.com> writes:

> OK, the message says "Couldn't select virtual nndoc group". But nndoc
> groups are not today (and according to you, should never be) virtual.

It creates an nnvirtual group over several nndoc groups.  Just look a
few lines up from the error message.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Lars Magne Ingebrigtsen

Re: What is a "virtual" group?

[Dave Abrahams]

on Tue Dec 25 2012, Lars Ingebrigtsen <larsi-AT-gnus.org> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
>> OK, the message says "Couldn't select virtual nndoc group". But nndoc
>> groups are not today (and according to you, should never be) virtual.
>
> It creates an nnvirtual group over several nndoc groups.  Just look a
> few lines up from the error message.

OK, getting back to this old thread...

The phrasing of the message implies the nndoc group itself is virtual,
the same way that "couldn't select that purple paper bag" implies
the paper bag is also purple, not that the paper bag is in another
bag that's purple.

If you changed it to "Couldn't select nndoc group <name> as source for
virtual group <name2>" or some such, it would go from
misleading/confusing to potentially useful.  Would that be possible?

-- 
Dave Abrahams
BoostPro Computing                  Software Development        Training
http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost

Re: What is a "virtual" group?

[Dave Abrahams]

on Tue Dec 25 2012, Dave Abrahams <dave-AT-boostpro.com> wrote:

>>>>> * Does the check for virtual-ness in gnus-warp-to-article actually
>>>>>   belong there?  After all, real groups shouldn't implement the
>>>>>   warp-to-article backend function, should they?
>>>>
>>>> I don't know.
>>>
>>> Would you be open to a patch that removes the check?
>>
>> I'm not familiar with that code.
>
> Sorry, I don't know how to interpret your answer.

Bump on this.  I get a sense the length of the message may have put you
off.  I'll try to make this super-simple for you:

* I would like to remove the check for virtual-ness in
  gnus-warp-to-article.

* I believe that removal to be OK.  I'll test it here, of course, and
  I'm a daily user of warping.

* If I don't find any problems with it, would that be enough to convince
  you that the change is OK and to accept the patch?

* If not, what would it take to convince you to accept the patch?

All I really need from you is answers to the last two bullets above

The rest of what's below is background, designed to convince you it
might be worth taking a risk on such a change to get a really cool
feature, but you can ignore it if you're too busy.

> I am trying to figure out how to contribute a really cool (and for me,
> essential) piece of functionality to Gnus: the ability to, given a
> message-id, find and display the message in the context of the entire
> thread in which that message participates, even across groups.  Of
> course I'm presuming you think that's worth having in Gnus, but I could
> be wrong.  If so, just tell me and I'll stop pestering you.
>
> I have implemented something that works, but currently requires making
> nndoc groups virtual or disabling that check.  You've already said that
> making nndoc groups virtual is inappropriate.  If disabling the check is
> appropriate, that would solve the problem easily.  
>
> The other possible approach would be to discover a way to implement the
> following function using some kind of virtual group (instead of nndoc).
> I think that's probably the best approach but I have tried myself many
> times and failed, and I have asked here many times for guidance without
> success... so I eventually gave up.
>
> #+begin_src lisp
> (defun gnus-goto-article (message-id)
>   (with-temp-buffer
>     (erase-buffer)
>     ;; Insert dummy article
>     (insert (format "From nobody Tue Sep 13 22:05:34 2011\n\n"))
>     (gnus-group-read-ephemeral-group
>      message-id
>      `(nndoc ,message-id
>              (nndoc-address ,(current-buffer))
>              (nndoc-article-type mbox))
>      :activate
>      (cons (current-buffer) gnus-current-window-configuration)
>      (not :request-only)
>      '(-1) ; :select-articles
>      (not :parameters)
>      0     ; :number
>      )
>     (gnus-summary-refer-article message-id)
>     ))
> #+end_src

-- 
Dave Abrahams
BoostPro Computing                  Software Development        Training
http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost

Re: What is a "virtual" group?

[Eric S Fraga]

Dave Abrahams <dave@boostpro.com> writes:

>> I am trying to figure out how to contribute a really cool (and for me,
>> essential) piece of functionality to Gnus: the ability to, given a
>> message-id, find and display the message in the context of the entire
>> thread in which that message participates, even across groups.  Of
>> course I'm presuming you think that's worth having in Gnus, but I could
>> be wrong.  If so, just tell me and I'll stop pestering you.

Dave,

I cannot help with this but did want to add my vote: I would love to
have this feature in gnus.

-- 
: Eric S Fraga, GnuPG: 0xC89193D8FFFCF67D
: in Emacs 24.3.50.1 + Ma Gnus v0.6
: BBDB version 3.02 ($Date: 2013/01/13 22:41:36 $)

Re: What is a "virtual" group?

[Dave Abrahams]

on Thu Jan 31 2013, Eric S Fraga <e.fraga-AT-ucl.ac.uk> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
>>> I am trying to figure out how to contribute a really cool (and for me,
>>> essential) piece of functionality to Gnus: the ability to, given a
>>> message-id, find and display the message in the context of the entire
>>> thread in which that message participates, even across groups.  Of
>>> course I'm presuming you think that's worth having in Gnus, but I could
>>> be wrong.  If so, just tell me and I'll stop pestering you.
>
> Dave,
>
> I cannot help with this but did want to add my vote: I would love to
> have this feature in gnus.

You could help by lobbying Lars to respond to my questions of Dec 25th
in this thread :-)

-- 
Dave Abrahams

Re: What is a "virtual" group?

[Leonidas Tsampros]

Dave Abrahams <dave@boostpro.com> writes:
> on Thu Jan 31 2013, Eric S Fraga <e.fraga-AT-ucl.ac.uk> wrote:
>
>> Dave Abrahams <dave@boostpro.com> writes:
>>
>>>> I am trying to figure out how to contribute a really cool (and for me,
>>>> essential) piece of functionality to Gnus: the ability to, given a
>>>> message-id, find and display the message in the context of the entire
>>>> thread in which that message participates, even across groups.  Of
>>>> course I'm presuming you think that's worth having in Gnus, but I could
>>>> be wrong.  If so, just tell me and I'll stop pestering you.
>>
>> Dave,
>>
>> I cannot help with this but did want to add my vote: I would love to
>> have this feature in gnus.
>
> You could help by lobbying Lars to respond to my questions of Dec 25th
> in this thread :-)

Are you referring only to a specific backend? I guess this will have to work
differently if we have to cross backends.

Re: What is a "virtual" group?

[Dave Abrahams]

on Mon Apr 22 2013, Leonidas Tsampros <ltsampros-AT-upnet.gr> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>> on Thu Jan 31 2013, Eric S Fraga <e.fraga-AT-ucl.ac.uk> wrote:
>>
>>> Dave Abrahams <dave@boostpro.com> writes:
>>>
>>>>> I am trying to figure out how to contribute a really cool (and for me,
>
>>>>> essential) piece of functionality to Gnus: the ability to, given a
>>>>> message-id, find and display the message in the context of the entire
>>>>> thread in which that message participates, even across groups.  Of
>>>>> course I'm presuming you think that's worth having in Gnus, but I could
>>>>> be wrong.  If so, just tell me and I'll stop pestering you.
>>>
>>> Dave,
>>>
>>> I cannot help with this but did want to add my vote: I would love to
>>> have this feature in gnus.
>>
>> You could help by lobbying Lars to respond to my questions of Dec 25th
>> in this thread :-)
>
> Are you referring only to a specific backend? I guess this will have to work
> differently if we have to cross backends.

No, it works across backends.  It doesn't even take all that much code.

-- 
Dave Abrahams

Re: What is a "virtual" group?

[Leonidas Tsampros]

Dave Abrahams <dave@boostpro.com> writes:
> on Mon Apr 22 2013, Leonidas Tsampros <ltsampros-AT-upnet.gr> wrote:
>
>> Dave Abrahams <dave@boostpro.com> writes:
>>> on Thu Jan 31 2013, Eric S Fraga <e.fraga-AT-ucl.ac.uk> wrote:
>>>
>>>> Dave Abrahams <dave@boostpro.com> writes:
>>>>
>>>>>> I am trying to figure out how to contribute a really cool (and for me,
>>
>>>>>> essential) piece of functionality to Gnus: the ability to, given a
>>>>>> message-id, find and display the message in the context of the entire
>>>>>> thread in which that message participates, even across groups.  Of
>>>>>> course I'm presuming you think that's worth having in Gnus, but I could
>>>>>> be wrong.  If so, just tell me and I'll stop pestering you.
>>>>
>>>> Dave,
>>>>
>>>> I cannot help with this but did want to add my vote: I would love to
>>>> have this feature in gnus.
>>>
>>> You could help by lobbying Lars to respond to my questions of Dec 25th
>>> in this thread :-)
>>
>> Are you referring only to a specific backend? I guess this will have to work
>> differently if we have to cross backends.
>
> No, it works across backends.  It doesn't even take all that much code.

Ok, I'm willing to test anything you send.

I did ask however because I had a similar idea the other day while
looking at Gmail's IMAP X-GM-EXT-1 extensions. They provide an
X-GM-THRID attribute that works like this:

a008 FETCH 1:4 (X-GM-THRID)
* 1 FETCH (X-GM-THRID 1278455344230334865)
* 2 FETCH (X-GM-THRID 1266894439832287888)
* 3 FETCH (X-GM-THRID 1266894439832287888)
* 4 FETCH (X-GM-THRID 1266894439832287888)
a008 OK FETCH (Success)

a009 UID SEARCH X-GM-THRID 1266894439832287888
* SEARCH 2 3 4
a009 OK Search (Success)

The above examples were copied from
https://developers.google.com/google-apps/gmail/imap_extensions

So theoretically it seems quite simple really to find 'a gmail-like
thread/conversation/whatever-marketing-decides-to-call-it-tommorow' on
an imap-gmail server.