Offline mail and group cooperation

Offline mail and group cooperation

[Kai Großjohann]

We are a bunch of folks all using Gnus.  Soon we will all have
laptops and work on them at home, at work, and in the train.  So now
is the time to think about disconnected usage.  We also cooperate, ie
we have common groups we post to.

I guess that nnimap and Gnus Agent is still one of the major
alternatives.  I remember that I tried it a long time ago and I was
confused about which messages it wanted to download automatically.  I
guess that we want to download all messages (regardless of marks) in
selected groups.  Is this possible with the Agent?  The common groups
could be handled via shared folders.  But I think we would want
separated marks.  Does Cyrus allow us to say which marks should be
per-user and which should be per-folder?  Or is there another IMAP
server we should use?  And then there is the thorny problem that
nnimap/agent do not allow moving of messages from one group to
another while offline, right?  (And if we do IMAP, then I think I
want to do my splitting using ifilter.  But that would have to be
done on the server side, and how does ifilter on the server know that
I've moved a message?)

Another alternative might be to install a news server for our group,
and everybody installs leafnode on their laptop.  That would be kind
of cool, really :-) Moving messages could be done with supersede, I
think.  But then we have to maintain a news server installation -- I
think leafnode won't do for the central server, for it allows no local
groups, right?  How difficult is it to install and administer a `real'
news server for this purpose?  Hm.  And then there is the
authentication issue -- we are not paranoid but maybe we want to hide
some groups from our students.  What do we do about personal mail in
this case?  And what about normal news groups like gnu.emacs.gnus?

I wonder if nnmaildir could somehow do the job?  Nnmaildir seems to
be nifty generally, and I'd believe it can make coffee for us :-)
Kudos to Paul.  Alas, I've never really looked at it so I don't have
an idea what it can _actually_ do.

And then, still, we could go the low-tech approach: instead of common
groups or shared folders, we just have mail aliases and everybody has
their own copy of the message.

A colleague also said that he ideally wants to be able to access the
mail from any computer, not only from the laptop.  This is for the
case where you are traveling and you find a computer with internet
access and putty, say, but you can't plug in your laptop for whatever
reason.

Are we asking for too many features?

I'm interested in the general approach to doing things at this time,
I think I can work out the nitty-gritty details later on.  All
comments appreciated.

kai
-- 
~/.signature is: umop 3p!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Alex Schroeder]

grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:

> I'm interested in the general approach to doing things at this time,
> I think I can work out the nitty-gritty details later on.  All
> comments appreciated.

At work, most use Outlook and IMAP, and confusion abounds: they don't
split mails into groups, if they do, everybody has his own system.  I
am the only one using Gnus and POP3, and I have a local copy of all
the mails on my computer.  This works perfectly well.  I can use Gnus,
and if all fails, I can grep.  It turns out that I have around 700MB
of mails on my HD, but who cares.  When other people are looking for
that missing email, they still have to ask me, because I am sure to
have it.

So my advice is: Everybody having his own setup works, so why fix it?

> We are a bunch of folks all using Gnus.  Soon we will all have
> laptops and work on them at home, at work, and in the train.  So now
> is the time to think about disconnected usage.  We also cooperate,
> ie we have common groups we post to.
> ...
> And then, still, we could go the low-tech approach: instead of
> common groups or shared folders, we just have mail aliases and
> everybody has their own copy of the message.

Your description did not contain any problem to solve.  What is the
problem you have with disconnected usage?  All I can think of is "If I
move a mail from folder X to folder Y, everybody else should move that
mail from folder X to folder Y."  Is this an accurate description of
the problem?  Perhaps another problem is that "I want all coworkers to
have the mails in the same folders as I do." -- but really, why do you
want that?  So basically, before you start looking for a solution, you
need to specify the problem.  ;)

Alex.

Re: Offline mail and group cooperation

[Kai Großjohann]

Alex Schroeder <alex@emacswiki.org> writes:

> Your description did not contain any problem to solve.  What is the
> problem you have with disconnected usage?  All I can think of is "If I
> move a mail from folder X to folder Y, everybody else should move that
> mail from folder X to folder Y."  Is this an accurate description of
> the problem?  Perhaps another problem is that "I want all coworkers to
> have the mails in the same folders as I do." -- but really, why do you
> want that?  So basically, before you start looking for a solution, you
> need to specify the problem.  ;)

Back when we were using mail aliases, it often happened that some
messages were sent to a specific person, but not to everybody who
would be interested.  Since we started using shared imap folders for
this, the problem disappeared because everybody automatically moves
mail regarding the foo project into the foo folder and so everybody
involved with the project knows about all messages.

So, what is the problem we want to solve?

* First of all, there is the disconnected thing.  People access mail
  from their laptop while at home, while at work, and they might want
  to read it while in the train.  So I want to slurp mails from home,
  work on them in the train on the way to work, then send my
  responses from the office.

* Secondly, there is the shared email thing.  People work together on
  a project and all email regarding the project should be seen by all
  people.

* Some of these mails are todo items.  It would be nice to have a way
  to see which items are done and which still need to be done.

Does that make things clearer?

kai
-- 
~/.signature is: umop 3p!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Clemens Fischer]

Kai Großjohann wrote:

>  * First of all, there is the disconnected thing.  People access
>    mail from their laptop while at home, while at work, and they
>    might want to read it while in the train.  So I want to slurp
>    mails from home, work on them in the train on the way to work,
>    then send my responses from the office.
>  
>  * Secondly, there is the shared email thing.  People work together
>    on a project and all email regarding the project should be seen
>    by all people.
>  
>  * Some of these mails are todo items.  It would be nice to have a
>    way to see which items are done and which still need to be done.

is cooperation the central aspect?  everybody working on and
contributing to the same set of documents?  that would be a wiki, but
this wiki would have to be a little like shared IMAP folders with
everybody having a separate, propably extensible set of marks and
access rights.  there would be public marks (ie. with agreed upon
semantics), manipulated by a central mechanism, public marks
manipulated by users and private marks with any meaning one can dream
up.

for disconnected operation, a local copy of something is needed, but
as you can't be sure somebody else changes the same thing at the same
time, you'd need CVS features.  collisions were the thing to worry
about:  you'd either want to fork off a separate version or merge
after discussion.

every document in such a database would need its own list of people
subscribed to notifications in case it is changed.

clemens

Re: Offline mail and group cooperation

[Kai Großjohann]

Clemens Fischer <ino@despammed.com> writes:

> is cooperation the central aspect?  everybody working on and
> contributing to the same set of documents?  that would be a wiki, but
> this wiki would have to be a little like shared IMAP folders with
> everybody having a separate, propably extensible set of marks and
> access rights.

No, we're talking about something like mails or news postings, not
documents.

(We have a project MIND, and there is a mail alias
mind@ls6.cs.uni-dortmund.de.  Messages sent to it arrive in the
shared IMAP folder so that everybody can read them and reply to
them.  If somebody inadvertently gets a personal message then it is
moved into the same shared IMAP folder.  I think you can see that
Wikis are not the answer :-)

kai
-- 
~/.signature is: umop 3p!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Alex Schroeder]

grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:

> Back when we were using mail aliases, it often happened that some
> messages were sent to a specific person, but not to everybody who
> would be interested.

We use per-project mailing lists for this.  It works fine most of the
time; sometimes people start sending private mail instead of using the
mailing list, but in these cases, I just reply to the mailing list,
with a complete copy of the mail cited, and then -- without offending
anybody :) -- everybody is informed.

> * First of all, there is the disconnected thing.  People access mail
>   from their laptop while at home, while at work, and they might want
>   to read it while in the train.  So I want to slurp mails from home,
>   work on them in the train on the way to work, then send my
>   responses from the office.

POP3 would be enough for that.

> * Secondly, there is the shared email thing.  People work together on
>   a project and all email regarding the project should be seen by all
>   people.

Mailing lists would be enough for that.

> * Some of these mails are todo items.  It would be nice to have a way
>   to see which items are done and which still need to be done.

No answer from me for this one.  But is IMAP the answer?

Alex.

Re: Offline mail and group cooperation

[news]

grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:

> * First of all, there is the disconnected thing.  People access mail
>   from their laptop while at home, while at work, and they might want
>   to read it while in the train.  So I want to slurp mails from home,
>   work on them in the train on the way to work, then send my
>   responses from the office.

I know you will do everything in gnus, but I'll throw a few
non-gnuscentric ideas out anyway!

If IMAP or nntp requires connecting too frequently, consider uucp to
handle the disconnected thing.  You can configure it to use ssh
as well:

  http://perso.linuxfr.org/penso/article/uucp+ssh.html#toc6
  http://www.internatif.org/bortzmeyer/uucp-over-ssh/

uucp is attractive for composing email while disconnected.  

mew also has a nice queue feature, which holds mail until a
connection to an MTA can be made, and maybe gnus could
incorporate that.

> * Secondly, there is the shared email thing.  People work together on
>   a project and all email regarding the project should be seen by all
>   people.

You are already thinking that nntp or IMAP is best for sharing
efficiently, otherwise, mailing lists and aliases.

> * Some of these mails are todo items.  It would be nice to have a way
>   to see which items are done and which still need to be done.

Good luck!  I think your group is pioneering these sharing
and coordination techniques.  Thanks for sharing your progress
and problems.

Chris

Re: Offline mail and group cooperation

[Kai Großjohann]

(Chris Beggy ) news@kippona.com writes:

> grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:
>
>> * First of all, there is the disconnected thing.  People access mail
>>   from their laptop while at home, while at work, and they might want
>>   to read it while in the train.  So I want to slurp mails from home,
>>   work on them in the train on the way to work, then send my
>>   responses from the office.
>
> I know you will do everything in gnus, but I'll throw a few
> non-gnuscentric ideas out anyway!

:-)  That's always a good idea.  Look over the edge of your plate, as
a German idiom says.

> If IMAP or nntp requires connecting too frequently, consider uucp to
> handle the disconnected thing.  You can configure it to use ssh
> as well:
>
>   http://perso.linuxfr.org/penso/article/uucp+ssh.html#toc6
>   http://www.internatif.org/bortzmeyer/uucp-over-ssh/
>
> uucp is attractive for composing email while disconnected.  

Hm.  As I understand, UUCP is just a mechanism to transport mail and
news to/from my machine.  So, functionality-wise, there is no
difference between invoking UUCP to fetch news and mail, and invoking
fetchnews (from leafnode) and fetchmail (from ESR) to fetch news and
mail.

But this does not do the shared email thing.

> mew also has a nice queue feature, which holds mail until a
> connection to an MTA can be made, and maybe gnus could
> incorporate that.

Gnus already has that: when you tell the Agent to go unplugged, then
outgoing mails and news are stored in nndraft:queue when you hit C-c
C-c.  You can then send the queued drafts with a command (`J S' from
the group buffer).

>> * Secondly, there is the shared email thing.  People work together on
>>   a project and all email regarding the project should be seen by all
>>   people.
>
> You are already thinking that nntp or IMAP is best for sharing
> efficiently, otherwise, mailing lists and aliases.

The question is, am I missing something?

For example, if there was a bidirectional file synchronization tool
for maildirs, then nnmaildir might be a good choice.

>> * Some of these mails are todo items.  It would be nice to have a way
>>   to see which items are done and which still need to be done.
>
> Good luck!  I think your group is pioneering these sharing
> and coordination techniques.  Thanks for sharing your progress
> and problems.

I have a hard time believing we are the first to do that.  There must
be thousands of people who have already solved this problem.

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Christoph Garbers]

* Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann):

>> If IMAP or nntp requires connecting too frequently, consider uucp to
>> handle the disconnected thing.  You can configure it to use ssh
>> as well:
>>
>>   http://perso.linuxfr.org/penso/article/uucp+ssh.html#toc6
>>   http://www.internatif.org/bortzmeyer/uucp-over-ssh/
>>
>> uucp is attractive for composing email while disconnected.  
> 
> Hm.  As I understand, UUCP is just a mechanism to transport mail and
> news to/from my machine.  So, functionality-wise, there is no
> difference between invoking UUCP to fetch news and mail, and invoking
> fetchnews (from leafnode) and fetchmail (from ESR) to fetch news and
> mail.

Correct. It is a protocol to copy files between unix systems.

Christoph

Re: Offline mail and group cooperation

[François Pinard]

[Christoph Garbers]
> * Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann):

>>> uucp is attractive for composing email while disconnected.  
>> 
>> So, functionality-wise, there is no difference between invoking UUCP to
>> fetch news and mail, and invoking fetchnews (from leafnode) and fetchmail
>> (from ESR) to fetch news and mail.

> Correct.  It is a protocol to copy files between unix systems.

UUCP is still ideal for fetching or delivering mail over sporadically
connected lines, in situations where the connection time is expensive,
either in money, or loss of commodity of the only phone line. :-) Note that
UUCP is not aware of the NNTP protocol.  A newsfeed through UUCP is rather a
massive, batch operation, while NNTP requires the interaction capabilities
of the Internet.  On the other hand, UUCP over PPP does not give much more,
however, because PPP is already not so economical of the communication line.

UUCP stands for Unix-to-Unix CoPy, but could do much more than mere copying.
The copy facilities of UUCP were a base over which all the rest was built.
UUCP predates the current Internet connectivity, and because of this, was
once popular for various services besides news and mail, which have been the
most well known applications of UUCP.  You could remotely execute any
program explicitly allowed to do so, and this has been used in many ways.

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard

Re: Offline mail and group cooperation

[Scott A Crosby]

On Fri, 27 Sep 2002 12:24:29 +0200, Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Gro~johann) writes:

> For example, if there was a bidirectional file synchronization tool
> for maildirs, then nnmaildir might be a good choice.

I use a program called 'unison' to syncronize my laptop homedir and my
desktop homedir. Making a program to syncronize maildirs is about 20
lines of perl.  (have each maildir contain a file called 'directory'
that records all files in that maildir as of the last syncronization,
then compare each maildir to its directory. If a file has been
removed, send a 'remove' command to the peer. If a file has been
added, send a 'add' command to the peer. )


Scott

Re: Offline mail and group cooperation

[Simon Josefsson]

grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:

> I guess that nnimap and Gnus Agent is still one of the major
> alternatives.  I remember that I tried it a long time ago and I was
> confused about which messages it wanted to download automatically.  I
> guess that we want to download all messages (regardless of marks) in
> selected groups.  Is this possible with the Agent?  

Right now the Agent is hard coded to never download articles marked as
read in Gnus.  I posted a (very lightly tested) patch to fix this long
time ago, but never committed it as it breaks backwards compatibility
(people that want the old behaviour had to add a `read' Agent
predicate to only download read articles, which was the old
behaviour).

What you describe is how most other disconnected clients work, so I
think it would be nice to support it, but I'm not sure how.  Is it
worth breaking backwards compatibility for this?  Perhaps adding a
`unread' Agent predicate which you'd need to add to get this behaviour
could work.

> The common groups could be handled via shared folders.  But I think
> we would want separated marks.  Does Cyrus allow us to say which
> marks should be per-user and which should be per-folder?  

Not fully fine grained, I think, but it may be enough.  Look at
setting ACLs on mailboxes, I remember being able to say whether a user
was permitted to set global flags.  ACLs could probably be used to
exclude students from mailboxes too.

> And then there is the thorny problem that nnimap/agent do not allow
> moving of messages from one group to another while offline, right?

Right.

> (And if we do IMAP, then I think I want to do my splitting using
> ifilter.  But that would have to be done on the server side, and how
> does ifilter on the server know that I've moved a message?)

Can't you split when you plug in?

Generally I think there are dragons in the IMAP and Agent interaction,
so other mechanisms are probably easier to get to work, but it would
be useful if you had time to try to get it to work.

Re: Offline mail and group cooperation

[Kai Großjohann]

Simon Josefsson <jas@extundo.com> writes:

> Right now the Agent is hard coded to never download articles marked as
> read in Gnus.  I posted a (very lightly tested) patch to fix this long
> time ago, but never committed it as it breaks backwards compatibility
> (people that want the old behaviour had to add a `read' Agent
> predicate to only download read articles, which was the old
> behaviour).
>
> What you describe is how most other disconnected clients work, so I
> think it would be nice to support it, but I'm not sure how.  Is it
> worth breaking backwards compatibility for this?  Perhaps adding a
> `unread' Agent predicate which you'd need to add to get this behaviour
> could work.

One idea: add a variable gnus-agent-support-read-predicate,
defaulting to nil.  If people set it to t, then they have to add a
`read' predicate to get the old behavior.

Of course, if adding the `unread' predicate is feasible, then this is
also good.

For our situation, I think it makes no sense to even try the current
behavior, as we surely want to refer to old articles.  (Hm.  But if
we start with always using the Agent, then it will have downloaded
the old articles when they were unread.  Hm.  But then we might
easily miss to download a message.  Not good.)

kai
-- 
~/.signature is: umop 3p!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Kai Großjohann]

Simon Josefsson <jas@extundo.com> writes:

> Generally I think there are dragons in the IMAP and Agent interaction,
> so other mechanisms are probably easier to get to work, but it would
> be useful if you had time to try to get it to work.

I might not have time to fix the problems.  Is anybody here
interested to work on this in the next weeks?

Hm.  Actually, probably we won't share our mail that soon.  So it's
not that urgent.  But once we try to make it work, it would be bad to
have too many problems for too long.

kai
-- 
~/.signature is: umop 3p!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Björn Torkelsson]

Simon Josefsson <jas@extundo.com> writes:

> grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:
>
>> I guess that nnimap and Gnus Agent is still one of the major
>> alternatives.  I remember that I tried it a long time ago and I was
>> confused about which messages it wanted to download automatically.  I
>> guess that we want to download all messages (regardless of marks) in
>> selected groups.  Is this possible with the Agent?  
>
> Right now the Agent is hard coded to never download articles marked as
> read in Gnus.  I posted a (very lightly tested) patch to fix this long
> time ago, but never committed it as it breaks backwards compatibility
> (people that want the old behaviour had to add a `read' Agent
> predicate to only download read articles, which was the old
> behaviour).
>
> What you describe is how most other disconnected clients work, so I
> think it would be nice to support it, but I'm not sure how.  Is it
> worth breaking backwards compatibility for this?  Perhaps adding a
> `unread' Agent predicate which you'd need to add to get this behaviour
> could work.

Isn't this what the cache is supposed to solve? 

When I read a mail, it's entered into the cache. When I delete the
mail (either when I mark it as expirable, or when it's expired) it is
also deleted from the cache. 

The only time I should have to manually download messages are when I
want to download, new, unread mail, or when I for some reason have
turned of the cache (in which case I should be able to specify which
mail to donwload)

That's at least my idea of how the agent/cache *should* work.

Pehaps it should also be an option to download new messages into the
cache when checking for new mail.

All my rant probably on apply for nnimap.

/torkel

Re: Offline mail and group cooperation

[Kai Großjohann]

torkel@acc.umu.se (Björn Torkelsson) writes:

> When I read a mail, it's entered into the cache. When I delete the
> mail (either when I mark it as expirable, or when it's expired) it is
> also deleted from the cache. 

I think this does NOT happen by default when you are plugged.  You
need to explicitly invoke some command to download messages.

But your suggestion is good: there should be a mode where Gnus works
this way.

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[dme]

* Kai.Grossjohann@CS.Uni-Dortmund.DE [2002-09-26 09:14:20]
> torkel@acc.umu.se (Björn Torkelsson) writes:
>
>> When I read a mail, it's entered into the cache. When I delete the
>> mail (either when I mark it as expirable, or when it's expired) it is
>> also deleted from the cache. 
>
> I think this does NOT happen by default when you are plugged.  You
> need to explicitly invoke some command to download messages.
>
> But your suggestion is good: there should be a mode where Gnus works
> this way.

I've used the following settings for a while to "aggressively" cache
things.  When I'm offline articles I've already seen can be loaded
from the cache.  It works well, though I do end up with lots of data
in the cache.  So far this hasn't been a problem (disks are large !).

I _think_ that you need to use a version of gnus which has the
combined cache/agent stuff, but I'm not really sure (it's in Oort I
believe).

(setq
 ; needs to be "t" for the cache to have any effect
 gnus-use-cache t
 ; only cache these groups
 gnus-cacheable-groups "^nnimap\\|^nntp"
 ; this means that all articles which are read will be stored in the
 ; cache, as well as the default set - good for ad-hoc offline
 ; re-reading
 gnus-cache-enter-articles '(ticked dormant unread read)
 gnus-cache-remove-articles nil
 )

Re: Offline mail and group cooperation

[Kai Großjohann]

dme@dme.org writes:

> I've used the following settings for a while to "aggressively" cache
> things.  When I'm offline articles I've already seen can be loaded
> from the cache.  It works well, though I do end up with lots of data
> in the cache.  So far this hasn't been a problem (disks are large !).

Wow, this is cool.  It doesn't quite do what I'd like but I guess
it's good enough: deleting messages is not _really_ necessary...

I was there when the agent/cache unification happened, but somehow I
didn't grok the implications of this.  So this really means that
articles I read online are entered into the agent/cache, and articles
downloaded by the agent are also entered into the same agent/cache?
Then I guess I'm happy.

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Wes Hardaker]

>>>>> On Thu, 26 Sep 2002 09:44:31 +0100, dme@dme.org said:

dme> gnus-cache-enter-articles '(ticked dormant unread read)

This will definitely be a help.  Thanks for suggesting it.

dme> gnus-cache-remove-articles nil

I think that we should add a 'expired' note to the list so that on
being marked as expiable we can remove it from the cache.  This would
help clean things up (I actually am frequently tight on disk space, as
I never delete things everywhere else on my disk).  I glanced at
gnus-cache.el to see how easy it would be, but am too sleepy to
attempt it at the moment.
-- 
"The trouble with having an open mind, of course, is that people will
 insist on coming along and trying to put things in it."   -- Terry Pratchett

Re: Offline mail and group cooperation

[Simon Josefsson]

dme@dme.org writes:

> I've used the following settings for a while to "aggressively" cache
> things.  When I'm offline articles I've already seen can be loaded
> from the cache.  It works well, though I do end up with lots of data
> in the cache.  So far this hasn't been a problem (disks are large !).

Interesting.  But it seems the cache stores things in ~/News/cache/
but the agent in ~/News/agent/, I thought they should re-use each
other's data.  Hm.  Do you see this too?

Re: Offline mail and group cooperation

[Kai Großjohann]

Simon Josefsson <jas@extundo.com> writes:

> Interesting.  But it seems the cache stores things in ~/News/cache/
> but the agent in ~/News/agent/, I thought they should re-use each
> other's data.  Hm.  Do you see this too?

Is there a way (function) to download an article into the Agent,
right away?  Then gnus-cache-enter-article could just call that
function, instead of rolling its own.  This would properly update the
.agentview files and stuff.

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[dme]

* jas@extundo.com [2002-09-26 17:59:48]
> Interesting.  But it seems the cache stores things in ~/News/cache/
> but the agent in ~/News/agent/, I thought they should re-use each
> other's data.  Hm.  Do you see this too?

I never use the agent commands to download articles, just the cache.

Re: Offline mail and group cooperation

[Kai Großjohann]

Simon Josefsson <jas@extundo.com> writes:

> grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:
>
>> I guess that nnimap and Gnus Agent is still one of the major
>> alternatives.  I remember that I tried it a long time ago and I was
>> confused about which messages it wanted to download automatically.  I
>> guess that we want to download all messages (regardless of marks) in
>> selected groups.  Is this possible with the Agent?  
>
> Right now the Agent is hard coded to never download articles marked as
> read in Gnus.  I posted a (very lightly tested) patch to fix this long
> time ago, but never committed it as it breaks backwards compatibility
> (people that want the old behaviour had to add a `read' Agent
> predicate to only download read articles, which was the old
> behaviour).

I would really appreciate it if we could get the new behavior.  It
seems that the trick of your patch is to change the return value of
gnus-agent-fetch-headers.  My suggestion is to make this depend on a
user option, which has the right value for backward compatibility.

Hm.

It is not pretty.  It would be much better if the predicate syntax
could just be augmented so that old predicates continue to retain
their meaning.  But I can't see how to do that.  Hm.  Hmmm...  So
maybe it is really better to just have a user option, even if it's
butt-ugly.  What do you think?

It's getting urgent for me.  I really want to be able to download all
articles, including the read ones.

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Kai Großjohann]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> It's getting urgent for me.  I really want to be able to download all
> articles, including the read ones.

I've been looking at the patch from
http://www.gnus.org/cgi-bin/wilma_hiliter/ding/200007/msg00006.html?line=30#hilite
-- this is the one you meant?

The first hunk cannot be applied easily because the code looks very
different around that part.  (If the comment still identifies the
spot.)

I've also been thinking about changing the predicate syntax so that
the old predicates are still valid (with the same meaning), and only
new-style predicates include the unread articles.  One possibility
would be to look for each predicate whether it contains the symbols
`read' and `unread'.  If the symbols are not present, it must be an
old-style predicate, and it is rewritten: let P be the predicate,
then the rewritten predicate is (and (not read) P).

If one of the symbols `read' or `unread' are present, then the
predicate is not rewritten but just taken as-is.

But then people have to say something atrocious like (or read true)
to just fetch all articles.  This is really ugly.  Or is there a
better way to do it so that the new predicates aren't so ugly?

Simon, would you be willing to update your patch to the current
gnus-agent?  I'd test it right away.

[time passes]

I even found a good name for that variable:
gnus-agent-consider-all-articles.  It would default to nil.  And
gnus-agent-fetch-headers can just have the current behavior or include
all articles, depending on the variable.

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Kai Großjohann]

Simon Josefsson <jas@extundo.com> writes:

> grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:
>
>> I guess that nnimap and Gnus Agent is still one of the major
>> alternatives.  I remember that I tried it a long time ago and I was
>> confused about which messages it wanted to download automatically.  I
>> guess that we want to download all messages (regardless of marks) in
>> selected groups.  Is this possible with the Agent?  
>
> Right now the Agent is hard coded to never download articles marked as
> read in Gnus.  I posted a (very lightly tested) patch to fix this long
> time ago, but never committed it as it breaks backwards compatibility
> (people that want the old behaviour had to add a `read' Agent
> predicate to only download read articles, which was the old
> behaviour).

So what do you think about the current implementation?  Is it really
horrible?

>> The common groups could be handled via shared folders.  But I think
>> we would want separated marks.  Does Cyrus allow us to say which
>> marks should be per-user and which should be per-folder?  
>
> Not fully fine grained, I think, but it may be enough.  Look at
> setting ACLs on mailboxes, I remember being able to say whether a user
> was permitted to set global flags.  ACLs could probably be used to
> exclude students from mailboxes too.

So when they are not allowed to set global flags, they automatically
set per-user ones, or what?  That's cool.

If the flag changes just don't happen at all, that would be bad.

>> And then there is the thorny problem that nnimap/agent do not allow
>> moving of messages from one group to another while offline, right?
>
> Right.

Yet another challenge.  I wish somebody would do something :-)
But who am I to complain.  I'm already enjoying immensely the new
agent header caching!

>> (And if we do IMAP, then I think I want to do my splitting using
>> ifilter.  But that would have to be done on the server side, and how
>> does ifilter on the server know that I've moved a message?)
>
> Can't you split when you plug in?

You mean, invoke ifile from nnimap-split-rule?  Hm.  I guess that
would work.  But it would be somewhat difficult to get ifile up to
speed with the current messages.

Does anyone have something which will download messages from the IMAP
server and hand them over to ifile to say "lookee-here, this message
was filed in the foo.bar group"?

Lessee now.  I think splitting is done on the headers, right?  So
even when it happens it would be nontrivial to pass the whole message
to ifile for examination.

> Generally I think there are dragons in the IMAP and Agent interaction,
> so other mechanisms are probably easier to get to work, but it would
> be useful if you had time to try to get it to work.

Well, at the moment I'm surprised I found so few problems.  No
dragons to be seen.

Do you have any idea where to look for dragons?

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Reiner Steib]

On Mon, Sep 23 2002, Kai Großjohann wrote:

> But then we have to maintain a news server installation -- I
> think leafnode won't do for the central server, for it allows no local
> groups, right?

The released versions 1.9x don't support local groups, but leafnode
2.0 (development version) does:

<URL: http://www-dt.e-technik.uni-dortmund.de/~ma/leafnode/beta/>
<URL: http://news.gmane.org/thread.php?group=gmane.network.leafnode>

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: Offline mail and group cooperation

[Kai Großjohann]

Reiner Steib <4uce.02.r.steib@gmx.net> writes:

> On Mon, Sep 23 2002, Kai Großjohann wrote:
>
>> But then we have to maintain a news server installation -- I
>> think leafnode won't do for the central server, for it allows no local
>> groups, right?
>
> The released versions 1.9x don't support local groups, but leafnode
> 2.0 (development version) does:

Ah, that's cool.  But will it help for our endeavor?  My idea is to
have everybody set up a leafnode on their laptop and to sync it with
a main server.  This way, the offline functionality is automatically
there.  Is this better or worse than relying on Gnus' Agent thingy in
conjunction with an IMAP server?

kai
-- 
~/.signature is: umop 3p!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Christoph Garbers]

* grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann):

> Another alternative might be to install a news server for our group,
> and everybody installs leafnode on their laptop.  That would be kind
> of cool, really :-) Moving messages could be done with supersede, I
> think.  But then we have to maintain a news server installation -- I
> think leafnode won't do for the central server, for it allows no local
> groups, right? 

The latest beta version (leafnode 2.x) does allow local groups, the
stable version (leafnode 1.9x) does not.

> How difficult is it to install and administer a `real' news server for
> this purpose?

Hm. I do not think that everyone has the time to understand and
configure INN for this purpose, although it would be possible.

Christoph

Re: Offline mail and group cooperation

[Frank Schmitt]

grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:

> We are a bunch of folks all using Gnus.  Soon we will all have
> laptops and work on them at home, at work, and in the train.  So now
> is the time to think about disconnected usage.  We also cooperate, ie
> we have common groups we post to.

Well, as a Windows user I don't know to well about this stuff, but
couldn't you do the following?

Everybody uses nnml as a backend. The folder where the folders for the
groups is stored is a CVS repository, if everybody should have his own
marks just keep the marks file out of the CVS repository.

You want to read mail while disconnected:
CVS update to get the stuff the others already committed, normal mail
retrieval, Mail splitting, CVS commit.

You want to read mail while connected:
Just do it.

You want to read mail from someone else's computer: Telnet to a server
which has the repository on disk, fire up a console emacs, read the
mail.

I guess that it will not work because of some reason I forgot (remember
that I'm only a dumb Windowser :-)) but perhaps...

-- 
One Ring to rule them all, One Ring to find them,
One Ring to bring them all and in the darkness bind them
In the Land of Mordor where the Shadows lie.

Re: Offline mail and group cooperation

[Kai Großjohann]

Frank Schmitt <usereplyto@Frank-Schmitt.net> writes:

> I guess that it will not work because of some reason I forgot (remember
> that I'm only a dumb Windowser :-)) but perhaps...

Alice does "cvs update", which creates group nnml:foo with messages
1, 2, 3.  Bob does the same.  Both go offline.  Alice moves a new
message into the group nnml:foo, which then gets number 4.  Bob does
the same.

Now both have a file named 4, and obviously the repository can have
only one such file.

Maybe nnmaildir helps because it creates unique filenames.  I'm not
sure.

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Frank Schmitt]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Frank Schmitt <usereplyto@Frank-Schmitt.net> writes:
>
>> I guess that it will not work because of some reason I forgot (remember
>> that I'm only a dumb Windowser :-)) but perhaps...
>
> Alice does "cvs update", which creates group nnml:foo with messages
> 1, 2, 3.  Bob does the same.  Both go offline.  Alice moves a new
> message into the group nnml:foo, which then gets number 4.  Bob does
> the same.
>
> Now both have a file named 4, and obviously the repository can have
> only one such file.

Looking at the code it seems to be easy to patch nnml.el to prefix the
files with e.g. the username of the person which created it + a
delimiter (for example fschmitt%78 instead of 78). Question is now:

-How about .marks, with or without prefix
-Can one easily delete the prefix while searching for the file to read?

-- 
One Ring to rule them all, One Ring to find them,
One Ring to bring them all and in the darkness bind them
In the Land of Mordor where the Shadows lie.

Re: Offline mail and group cooperation

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
> Alice does "cvs update", which creates group nnml:foo with messages
> 1, 2, 3.  Bob does the same.  Both go offline.  Alice moves a new
> message into the group nnml:foo, which then gets number 4.  Bob does
> the same.
>
> Now both have a file named 4, and obviously the repository can have
> only one such file.
>
> Maybe nnmaildir helps because it creates unique filenames.  I'm not
> sure.

nnmaildir would include (system-name) in the message filenames.  But I
don't know how CVS handles hard links; inode usage might skyrocket
with nnmaildir.


paul

Re: Offline mail and group cooperation

[Kai Großjohann]

grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:

> Another alternative might be to install a news server for our group,
> and everybody installs leafnode on their laptop.

I wonder how come that nobody commented on this alternative?

Is anyone out there using this?  How does it work?  Does anyone have
experience with this versus doing the shared folder thing via IMAP?

It would also be interesting to hear from people using an NNTP for
group-local communication, without that extra offline stuff.  Maybe I
can learn from it whether it is time-intensive to maintain your own
NNTP server installation, what do people do about emails (we have
mail aliases which send to bboard+shared.folder@imap-server so that
mails from outside partners end up in IMAP; I gather that a
mail-to-news gateway, while possible, is not quite as trivial).

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Offline mail and group cooperation

[Daniel Pittman]

On Sun, 06 Oct 2002, Kai Großjohann wrote:
> grossjoh@lothlorien.cs.uni-dortmund.de (Kai Großjohann) writes:
> 
>> Another alternative might be to install a news server for our group,
>> and everybody installs leafnode on their laptop.
> 
> I wonder how come that nobody commented on this alternative?

...because I didn't feel that I had very much to contribute. :)

> Is anyone out there using this?  

Not in exactly the situation you talk about, but for similar things,
yes.

> How does it work? Does anyone have experience with this versus doing
> the shared folder thing via IMAP?

It works well, though leafnode tends to slow down as the local cache
size grows, because it simply dumps all articles for a single group in
the one directory. Thus, performance is directly related to the (usually
poor) filesystem directory search implementation.

> It would also be interesting to hear from people using an NNTP for
> group-local communication, without that extra offline stuff. 

It's certainly very usable for this, what with being a many-to-many
communications medium. It tends to have better tools available for using
it than Shared IMAP, simply because it's been doing it longer.

> Maybe I can learn from it whether it is time-intensive to maintain
> your own NNTP server installation, 

AFAIK, no, for a local server only. If you actually want to do news
peering and exchange with other news servers (not including
leafnode-style caching servers) then it's more involved.

OTOH I run a cache-only server myself, so I may not have the full
picture in my head.

        Daniel

-- 
Reality is not as strong as perception. Perception all too often swallows
reality and spits it out in a new, unrecognizable form.
        -- Maytee Aspuro

Re: Offline mail and group cooperation

[Clemens Fischer]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann):

>> Another alternative might be to install a news server for our group,
>> and everybody installs leafnode on their laptop.
>
> I wonder how come that nobody commented on this alternative?
>
> Is anyone out there using this?  How does it work?  Does anyone have
> experience with this versus doing the shared folder thing via IMAP?

leafnode has local groups, but it is hard to find a good mail2news
gate.  one of the leafnode maintainers (joerg dietrich) has promised
to give his implementation a touch up, though.  then there is an often
referenced python script, but it writes to the spool directly.  gnus
users would propably prefer to use the nntp interface.

also, leafnode is easy to configure by design, and it is actively
developed by a very considerate developer (matthias andree).

clemens