Gnus Access mail spool by ssh

Gnus Access mail spool by ssh

[Harry Putnam]

[ also posted to freebsd-mobil list]

I have ssh-1.2.27 installed and use it extensively.  In fact I've set
my machines (personal at home machines) to only allow connections via
ssh.

Now I want to have a laptop running FreeBSD 4.0 networked to desktop
running Redhat linux, be able to  view/manipulate mail on the parent machine
but don't want to  setup pop, imap or  special sendmail stuff.  I
don't want more daemons running that can be exploited from the internet.
And haven't really looked into firewall technique.

I realize there are other ways via scp, rsync, mirror using ssh  but it seems
the most convient to have the laptop users inbox be a ssh connection
to the parent desktop at /var/spool/mail/$NAME

I'll  use procmail to send a copy of every mail to
/var/spool/mail/NAME, on the parent machine.  I know how to do that part.
Then both machines will have full access to the mail without screwing
up the others spool.

I would like to just have $MAIL on the laptop set to
`ssh PARENT:/var/spool/mail/$NAME' to access the procmail created 
spool on the parent machine.

I'm pretty lost how to script this so that when a mail client is
called, a ssh connection is initiated and /var/spool/mail/$NAME
becomes the inbox.

What would be nice is to just say `M-x gnus' and a ssh connection is
started and mail in PARENT:/var/spool/mail/$NAME is viewed, expired
saved etc.

I have `TRAMP' installed and can think of several ways that
application could be called in to do this job, but would rather find a
more general solution so that *any* mail client could use it.

I have ssh-agent setup and do understand how to avoid password
problems during connections. (Thanks Kai)

Re: Gnus Access mail spool by ssh

[Kai Großjohann]

One problem is file locking.  The program that reads
/var/spool/mail/jrl on the desktop machine needs to do file locking to
prevent sendmail (or whatever you use) from writing to the same file
at the same time.

Idea: given that you have an Emacs installation on the desktop
machine, you could do the following from the laptop:

# $desktop is the host name of the desktop machine.
# $movemail is the full path name of movemail on $desktop
# $mailfile is the full path name of /var/mail/foo on $desktop
# $desk_home is the home dir of the user on $desktop
# $lap_home is the home dir of the user on the laptop

ssh $desktop $movemail $mailfile $desk_home/TMPFILE
scp $desktop:TMPFILE $lap_home/TMPFILE

The above two commands could be put in a script which is used as
`:prescript' from mail-sources.  Then, tell Gnus to fetch mail from
TMPFILE.

How's that sound?
kai
-- 
I like BOTH kinds of music.

Re: Gnus Access mail spool by ssh

[Alan Shutko]

Harry Putnam <reader@newsguy.com> writes:

> Now I want to have a laptop running FreeBSD 4.0 networked to desktop
> running Redhat linux, be able to  view/manipulate mail on the parent machine
> but don't want to  setup pop, imap or  special sendmail stuff.  I
> don't want more daemons running that can be exploited from the internet.
> And haven't really looked into firewall technique.

Well, first, you should _really_ look into the firewall technique.
Otherwise, you'll be jumping through hoops that nobody else has
considered, because they've all set up a firewall and forgotten about
it.

If you don't want to do that... well, you could use the ssh to set up
a ppp tunnel between your computers, and run NFS, imap, or whatever
over that.  In some cases, that makes sense (ie, laptop halfway across
the continent, over very untrusted ethernet).  But not in a home
network.  Unless you have a very untrusted home network (ie, both
machines are just connected via a hub to a cable modem) it's much more
convenient to set up some firewall rules on the host connected to the
internet, and trust the internal network.  ssh was _not_ designed to
be a mail protocol, and as Kai pointed out, locking issues will be a
bitch.

I'm not sure precisely how you are expecting ssh to access
/var/spool/mail on the other host, as it doesn't set up any file
access from this side... you could send over ssh parent "cat
/var/spool/mail/ats", but editing the file will not be easy.  You
could do some weird use of files to lock on the other side, and scp
the file over... but it would be a mess.

-- 
Alan Shutko <ats@acm.org> - In a variety of flavors!
125 days, 13 hours, 37 minutes, 6 seconds till we run away.
A PC is for manipulation of data, not the manipulation of thought.

Re: Gnus Access mail spool by ssh

[Harry Putnam]

Alan Shutko <ats@acm.org> writes:

> Harry Putnam <reader@newsguy.com> writes:
> 
> > Now I want to have a laptop running FreeBSD 4.0 networked to desktop
> > running Redhat linux, be able to  view/manipulate mail on the parent machine
> > but don't want to  setup pop, imap or  special sendmail stuff.  I
> > don't want more daemons running that can be exploited from the internet.
> > And haven't really looked into firewall technique.

[...]
> 
> If you don't want to do that... well, you could use the ssh to set up
> a ppp tunnel between your computers, and run NFS, imap, or whatever
> over that.  In some cases, that makes sense (ie, laptop halfway across
> the continent, over very untrusted ethernet).  But not in a home
> network.  Unless you have a very untrusted home network (ie, both
> machines are just connected via a hub to a cable modem) it's much more
> convenient to set up some firewall rules on the host connected to the
> internet, and trust the internal network.  ssh was _not_ designed to
> be a mail protocol, and as Kai pointed out, locking issues will be a
> bitch.

I think Kai hit it right with a few modifications.  I guess I was
over complicating a bit.

Here is what I've come up with:

procmail copies all mail to $desktop_spool and a second  spool file on
$DESKTOP called  $HOME/spool/backup/lap.sp

Now using Kias formula:

ssh $desktop $movemail $mailfile $desk_home/TMPFILE
But with the modification that $mailfile is $lap.sp (a copy of
incoming).
 
So movemail moves $lap.sp to lap.tmp and when gnus slurps it up
through the ssh tunnel

The only monkey wrench here was that I couldn't find `movemail'  Its
nowhere in my path, which is very extensive.  A grep of
/usr/local/pgnus/lisp/*.el  finds many mentions of `movemail' but
none seem to show its address. 
Checking emacs info , a few hits but no address.

Finally the trusty gnus manual pages tipped it off and in the mail
sources dialog there is a nice line giving the address.

So a short script will do it all I think, except I suspect there could
be trouble about deleting the spool if procmail happens to be writing
to it right then.  

The cool thing is someone already wrote almost the exact script in 
gnus info at Mail Source Specifiers.

        (setq mail-sources
                '((file :prescript "ssh reader@reader ~/scripts/getmail >%t ")))

What does the specifier "t" do here?  Does just mean to the default?

Lifted almost bodily from gnus info.... I love it when there are good
examples like this!  I only had to add an `if/then' clause for a
little protection.  Since movemail doesn't delete the spool but
leaves an empty file.

  getmail.sh script 

  #!/bin/sh 
  MOVEMAIL=/usr/lib/emacs/20.5/i386-redhat-linux-gnu/movemail
  TMP=/home/reader/spool/backup/lap.tmp
  MAIL=/home/reader/spool/backup/lap.sp
  if [ -s $MAIL ];then 
  rm -f $TMP;$MOVEMAIL $MAIL $TMP >/dev/null && cat $TMP
  fi

Re: Gnus Access mail spool by ssh

[Kai Großjohann]

Harry Putnam <reader@newsguy.com> writes:

> The only monkey wrench here was that I couldn't find `movemail'  Its
> nowhere in my path, which is very extensive.

When you do `ssh $host $command', no environment is set on the remote
host.  In particular, /etc/profile and ~/.profile are not read, since
it's not a login shell.

There is a file ~/.ssh/environment or somesuch which gets executed,
see the ssh documentation for details.

> The cool thing is someone already wrote almost the exact script in 
> gnus info at Mail Source Specifiers.
> 
>         (setq mail-sources
>                 '((file :prescript "ssh reader@reader ~/scripts/getmail >%t ")))
> 
> What does the specifier "t" do here?  Does just mean to the default?

The mail-sources stuff writes the mail to a tmp file, maybe
~/.mail-source-crashbox or ~/Mail/Incoming*.  Since your script does
not know the tmp file name, mail-sources provides it.  If you put %t
into the command name, that's replaced by the tmp file name.

kai
-- 
I like BOTH kinds of music.

Re: Gnus Access mail spool by ssh

[Nuutti Kotivuori]

"Harry" == Harry Putnam <reader@newsguy.com> writes:
> [ also posted to freebsd-mobil list]
> 
> I have ssh-1.2.27 installed and use it extensively.  In fact I've set
> my machines (personal at home machines) to only allow connections via
> ssh.
> 
> Now I want to have a laptop running FreeBSD 4.0 networked to desktop
> running Redhat linux, be able to  view/manipulate mail on the parent machine
> but don't want to  setup pop, imap or  special sendmail stuff.  I
> don't want more daemons running that can be exploited from the internet.
> And haven't really looked into firewall technique.

<snip>

> I have ssh-agent setup and do understand how to avoid password
> problems during connections. (Thanks Kai)

Um. I have a similar problem. But my problem was that I wanted to
store all my mail folders on my server and on my laptop, including
sent mail - and still be able to read and send mail offline.

Firstly, there is no need to run an imap daemon lessening security,
since it can be run directly from a shell as well. The imap server
starts up as preauthenticated then, and ofcourse if it's run thru ssh,
the whole connection is encrypted. So, if you want to just fetch mail
from the spool (INBOX?), you can do something similar to my fetchmail
scripts. (The current script uses an identity without a passphrase
which only allows access to the imap daemon, but it can be changed to
whatever ofcourse)

bin/fetchmail-imap-wrapper:
exec ssh -i $HOME/.ssh/identity-imap -l xxxxx $1 /opt/net/etc/imapd

.fetchmailrc:
poll xxxxx with proto IMAP and options no dns
       preauth ssh plugin /usr/home/xxxxx/bin/fetchmail-imap-wrapper

I think this can be easily adapted into mail-sources fetching, correct
if I'm wrong though, but I'm not so sure about nnimap. If it can be
adapted to nnimap, you can effectively share mailboxes with your
server and use gnus-agent to download the articles to local storage,
possibly using a cache as well.

I'm going to experiment more on the remote accessing thru IMAP soon
enough, when I move my mail again.

-- Naked

Re: Gnus Access mail spool by ssh

[Harry Putnam]

Helmut Waitzmann <Helmut.Waitzmann@studserv.uni-stuttgart.de> writes:

> Harry Putnam <reader@newsguy.com> writes:
> 
> >  getmail.sh script 
> >
> >  #!/bin/sh 
> >  MOVEMAIL=/usr/lib/emacs/20.5/i386-redhat-linux-gnu/movemail
> >  TMP=/home/reader/spool/backup/lap.tmp
> >  MAIL=/home/reader/spool/backup/lap.sp
> >  if [ -s $MAIL ];then 
> >  rm -f $TMP;$MOVEMAIL $MAIL $TMP >/dev/null && cat $TMP
> >  fi
> 
> And now, imagine:  Movemail has just moved your mail to $TMP, when
> the connection breaks.  So cat won't be able to send you your
> mail.  The next time you fetch mail, your old $TMP file will be
> deleted and your mail will be gone...

Good point, maybe this is a little safer..:

#!/bin/sh 
MOVEMAIL=/usr/lib/emacs/20.5/i386-redhat-linux-gnu/movemail
TMP=/home/reader/spool/backup/lap.tmp
MAIL=/home/reader/spool/backup/lap.sp
if [ -s $TMP ];then
exit 1
else 
$MOVEMAIL $MAIL $TMP >/dev/null && cat $TMP && rm -f $TMP
fi

Then at least I can check it out before anything is deleted.

How can I change this (or the mail-source prescript) so that gnus tells
me if something untoward happens?

Re: Gnus Access mail spool by ssh

[Alexandre Oliva]

On Jul 11, 2000, Harry Putnam <reader@newsguy.com> wrote:

> $MOVEMAIL $MAIL $TMP >/dev/null && cat $TMP && rm -f $TMP

I've done that before, and I was bitten by it.  It happened that cat
finished and the file was removed but a significant part of my mailbox
was still in the network buffer, and the connection was broken before
it got to the other end.  Ever since, I've been using fetchmail, which
is far slower, but is quite safe.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Gnus Access mail spool by ssh

[Harry Putnam]

Alexandre Oliva <oliva@lsd.ic.unicamp.br> writes:

> On Jul 11, 2000, Harry Putnam <reader@newsguy.com> wrote:
> 
> > $MOVEMAIL $MAIL $TMP >/dev/null && cat $TMP && rm -f $TMP
> 
> I've done that before, and I was bitten by it.  It happened that cat
> finished and the file was removed but a significant part of my mailbox
> was still in the network buffer, and the connection was broken before
> it got to the other end.  Ever since, I've been using fetchmail, which
> is far slower, but is quite safe.

This is a home network where both machines set on the same table so
may be a little safer.  But can fetchmail be used to fetch a spool
file from a remote machine not running a special server?  The man page
seems to indicate not:

      The fetchmail program can gather mail  from  servers  sup­
       porting  any of the common mail-retrieval protocols: POP2,
       POP3, IMAP2bis, and IMAP4.  It can also use the ESMTP ETRN
       extension.   (The  RFCs describing all these protocols are
       listed at the end of this manual page.)

Re: Gnus Access mail spool by ssh

[Alexandre Oliva]

On Jul 11, 2000, Harry Putnam <reader@newsguy.com> wrote:

> But can fetchmail be used to fetch a spool
> file from a remote machine not running a special server?

Yup, as long as you can run the imapd program on the other end.  Then,
you write a script that `ssh' to the other end and runs imapd and
tells fetchmail to use it as a plugin with preauth ssh.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: Gnus Access mail spool by ssh

[Kai Großjohann]

On 11 Jul 2000, Alexandre Oliva wrote:

> On Jul 11, 2000, Harry Putnam <reader@newsguy.com> wrote:
> 
>> But can fetchmail be used to fetch a spool
>> file from a remote machine not running a special server?
> 
> Yup, as long as you can run the imapd program on the other end.
> Then, you write a script that `ssh' to the other end and runs imapd
> and tells fetchmail to use it as a plugin with preauth ssh.

I think Harry does not want to run a food on the other end, not even
an imapd :-)

But of course it would be the best situation.  Setting up a POP server
ought to be simple enough, for example.

kai
-- 
I like BOTH kinds of music.

Re: Gnus Access mail spool by ssh

[Nuutti Kotivuori]

"Kai" == Kai Großjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> writes:
> On 11 Jul 2000, Alexandre Oliva wrote:
>> On Jul 11, 2000, Harry Putnam <reader@newsguy.com> wrote:
>> 
>>> But can fetchmail be used to fetch a spool
>>> file from a remote machine not running a special server?
>> 
>> Yup, as long as you can run the imapd program on the other end.
>> Then, you write a script that `ssh' to the other end and runs imapd
>> and tells fetchmail to use it as a plugin with preauth ssh.
> 
> I think Harry does not want to run a food on the other end, not even
> an imapd :-)
> 
> But of course it would be the best situation.  Setting up a POP server
> ought to be simple enough, for example.

Err. He won't be forced to run imapd as a daemon listening to an imap
port. Just as a program to access mail. To clear things up:

_Not_:
$ telnet mail.host.foo imap2
Trying 127.0.0.1...
Connected to oro.
Escape character is '^]'.
* OK mail.host.foo IMAP4rev1 v12.264 server ready

_But_:
$ ssh mail.host.foo
mail$ /usr/sbin/imapd
* PREAUTH mail.host.foo IMAP4rev1 v12.264 server ready

Most imap servers are able to run from the command line and notice the
situation and start up preauthenticated. And this is what can be used
by fetchmail and the plugin option - and if nnimap does not support
this, it would be nice.

-- Naked

Re: Gnus Access mail spool by ssh

[Harry Putnam]

Nuutti Kotivuori <nuutti.kotivuori@sonera.com> writes:


[...]

> 
> Err. He won't be forced to run imapd as a daemon listening to an imap
> port. Just as a program to access mail. To clear things up:
> 
> _Not_:
> $ telnet mail.host.foo imap2
> Trying 127.0.0.1...
> Connected to oro.
> Escape character is '^]'.
> * OK mail.host.foo IMAP4rev1 v12.264 server ready
> 
> _But_:
> $ ssh mail.host.foo
> mail$ /usr/sbin/imapd
> * PREAUTH mail.host.foo IMAP4rev1 v12.264 server ready
> 
> Most imap servers are able to run from the command line and notice the
> situation and start up preauthenticated. And this is what can be used
> by fetchmail and the plugin option - and if nnimap does not support
> this, it would be nice.

This looks like an interesting approach.  
I'll have to get an imap program installed, but before I do:

It isn't clear to me how fetchmail fits in here.  Reading man
fetchail, particularly the `--plugin <command>' section:

              " . . . . . . . Fetchmail will write to  the
              plugin's stdin and read from the plugin's stdout.

It appears that the fetchmail addressed here will be running on the
parent desktop (in this case, $remote) or `mail.host.foo' so does it
just put its standard out into the ssh tunnel in the same way as the
mail-sources prescript `cat' command does, or is fetchmail asked to
deliver to sendmail on the local machine?

It sounds somewhat confusing.  Can you describe the process in a bit
more detail?

Re: Gnus Access mail spool by ssh

[Nuutti Kotivuori]

"Harry" == Harry Putnam <reader@newsguy.com> writes:
> Nuutti Kotivuori <nuutti.kotivuori@sonera.com> writes:
> [...]
> 
>> 
>> Err. He won't be forced to run imapd as a daemon listening to an imap
>> port. Just as a program to access mail. To clear things up:
>> 
>> _Not_:
>> $ telnet mail.host.foo imap2
>> Trying 127.0.0.1...
>> Connected to oro.
>> Escape character is '^]'.
>> * OK mail.host.foo IMAP4rev1 v12.264 server ready
>> 
>> _But_:
>> $ ssh mail.host.foo
>> mail$ /usr/sbin/imapd
>> * PREAUTH mail.host.foo IMAP4rev1 v12.264 server ready
>> 
>> Most imap servers are able to run from the command line and notice the
>> situation and start up preauthenticated. And this is what can be used
>> by fetchmail and the plugin option - and if nnimap does not support
>> this, it would be nice.
> 
> This looks like an interesting approach.  
> I'll have to get an imap program installed, but before I do:
> 
> It isn't clear to me how fetchmail fits in here.  Reading man
> fetchail, particularly the `--plugin <command>' section:
> 
>               " . . . . . . . Fetchmail will write to  the
>               plugin's stdin and read from the plugin's stdout.
> 
> It appears that the fetchmail addressed here will be running on the
> parent desktop (in this case, $remote) or `mail.host.foo' so does it
> just put its standard out into the ssh tunnel in the same way as the
> mail-sources prescript `cat' command does, or is fetchmail asked to
> deliver to sendmail on the local machine?
> 
> It sounds somewhat confusing.  Can you describe the process in a bit
> more detail?

Local machine runs fetchmail, which runs ssh to connect to the remote
machine and run imapd, which the fetchmail will access. I will put a
few configuration scripts to demonstrate.

Local .fetchmailrc:

poll mail.host.foo with proto IMAP and options no dns
       preauth ssh plugin /usr/home/xxx/bin/fetchmail-imap-wrapper

Local bin/fetchmail-imap-wrapper:

#!/bin/sh
exec ssh -i $HOME/.ssh/identity-imap -l xxx $1 /opt/net/etc/imapd

So, what nnimap should be able to do is to do exactly what --plugin
option in fetchmail does.

-- Naked

Re: Gnus Access mail spool by ssh

[Harry Putnam]

Nuutti Kotivuori <nuutti.kotivuori@sonera.com> writes:

> Local machine runs fetchmail, which runs ssh to connect to the remote
> machine and run imapd, which the fetchmail will access. I will put a
> few configuration scripts to demonstrate.

OK, starting to catch on a little.  Is Imapd left running by this
invocation?  Or does it shut down when ssh shuts down.

Also, any recommendations as to Imap server?

Re: Gnus Access mail spool by ssh

[Kai Großjohann]

On 12 Jul 2000, Harry Putnam wrote:

> Nuutti Kotivuori <nuutti.kotivuori@sonera.com> writes:
> 
>> Local machine runs fetchmail, which runs ssh to connect to the
>> remote machine and run imapd, which the fetchmail will access. I
>> will put a few configuration scripts to demonstrate.
> 
> OK, starting to catch on a little.  Is Imapd left running by this
> invocation?  Or does it shut down when ssh shuts down.

I think it will shut down.

> Also, any recommendations as to Imap server?

I think you want the UWash one -- Cyrus assumes a different setup.

(The UWash server plays ball with files in /var/spool/mail and user
home dirs and so on, whereas Cyrus wants to own all mail files.  On
the other hand, Cyrus is well suited for large installation where IMAP
users should not be given Unix accounts.)

kai
-- 
I like BOTH kinds of music.

Re: Gnus Access mail spool by ssh

[Harry Putnam]

Sorry, I'm still confused here.... Your description stops right at the
punch line.

Nuutti Kotivuori <nuutti.kotivuori@sonera.com> writes:

> Local machine runs fetchmail, which runs ssh to connect to the remote
> machine and run imapd, which the fetchmail will access. . . . 

What does fetchmail do with it.  Deliver to /var/mail/$USER or put it
on standard out of ssh command or what?

> Local .fetchmailrc:
> 
> poll mail.host.foo with proto IMAP and options no dns
>        preauth ssh plugin /usr/home/xxx/bin/fetchmail-imap-wrapper

OK .. I follow the .fetchmailrc recipe.
Is this wrapper necessary if ssh-agent is engaged?


> Local bin/fetchmail-imap-wrapper:
> 
> #!/bin/sh
> exec ssh -i $HOME/.ssh/identity-imap -l xxx $1 /opt/net/etc/imapd

You aim ssh at a specific id in ~/.ssh -- login as xxx.... what does
$1 do?  First argument to what... ssh?   --- imapd gets invoked and
passes new mail to fetchmail?  Which then does what?

I've tried this with the following scripts:

.fetchmailrc:
poll reader.local.lan with proto IMAP and options no dns
 preauth ssh plugin /home/reader/scripts/ssh-fetch.sh

ssh-fetch.sh
#!/bin/sh
exec ssh reader@reader /usr/sbin/imapd

I can't tell if its going to work because some other mess with my
FreeBSD install crops up to wreck it.

$ fetchmail 
/usr/libexec/ld-elf.so.1: Shared object "libssl.so.1" not found

Fetchmail installed from FreeBSD install with no complaints.

This is starting to look like too much pain in the ass for such a
simple chore.

Running just the shell script (ssh-fetch.sh) gives:

bsd > scripts/ssh-fetch.sh
* PREAUTH reader.ptw.com IMAP4rev1 v12.264 server ready

whoopee .... now what?

`man imapd' you say...  Well the one I have is just one tiny cut above
useless, lacking descriptive information.

I think I like the straight forward approach laid out by Kai and Lars.
What are the chances on a local network that the `cat' command in the
mail-sources prescript will die an ugly death?  I'm beginning to think
what ever risk, it is  much less problem than sorting out a bunch of
complicated soft ware for this one task.

Plus, it seems one could insert some kind of `trap' that would make it
safer. 

Especially since I have an overall backup of all incoming mail that
keeps the latest 1000 messages. 

Re: Gnus Access mail spool by ssh

[Alan Shutko]

Harry Putnam <reader@newsguy.com> writes:

> What does fetchmail do with it.  Deliver to /var/mail/$USER or put it
> on standard out of ssh command or what?

Fetchmail will send it to your local MTA (ie, sendmail).  It used to
drop it in the mail spool, but that feature was taken out for some
reason.

-- 
Alan Shutko <ats@acm.org> - In a variety of flavors!
121 days, 19 hours, 53 minutes, 23 seconds till we run away.
What we need is either less corruption, or more chance to participate in it.

Re: Gnus Access mail spool by ssh

[Harry Putnam]

Alan Shutko <ats@acm.org> writes:

> Harry Putnam <reader@newsguy.com> writes:
> 
> > What does fetchmail do with it.  Deliver to /var/mail/$USER or put it
> > on standard out of ssh command or what?
> 
> Fetchmail will send it to your local MTA (ie, sendmail).  It used to
> drop it in the mail spool, but that feature was taken out for some
> reason.

Thanks... I've cleaned up the crippled fetchmail and now can run the
scripts I posted .  Sure enough the mail is fetched to local incoming
spool.

*BUT* it is snatched from the parent incoming spool, so that the
 parent (remote) looses access.  This is not what I was after.  I want
 both machines to be able to process the same mail in whatever way
 needed.

Back to duplicating with procmail on parent (remote) .  But how to aim
fetchmail at the duplicated spool?  Maybe a dummy account for the
laptop to access... I'll try that.

Re: Gnus Access mail spool by ssh

[Nuutti Kotivuori]

"Harry" == Harry Putnam <reader@newsguy.com> writes:
> Sorry, I'm still confused here.... Your description stops right at the
> punch line.
> 
> Nuutti Kotivuori <nuutti.kotivuori@sonera.com> writes:
> 
>> Local machine runs fetchmail, which runs ssh to connect to the remote
>> machine and run imapd, which the fetchmail will access. . . . 
> 
> What does fetchmail do with it.  Deliver to /var/mail/$USER or put it
> on standard out of ssh command or what?

Well as pointed out, fetchmail does as it's told. For the very best
solutions, it likes to use a local SMTP server, if not, then MDA, if
not then maybe BSMTP delivery or whatever. That has nothing to do with
this setup, it's just fetchmail configuration.

>> Local .fetchmailrc:
>> 
>> poll mail.host.foo with proto IMAP and options no dns
>> preauth ssh plugin /usr/home/xxx/bin/fetchmail-imap-wrapper
> 
> OK .. I follow the .fetchmailrc recipe.
> Is this wrapper necessary if ssh-agent is engaged?

Err. Depending, see below.

>> Local bin/fetchmail-imap-wrapper:
>> 
>> #!/bin/sh
>> exec ssh -i $HOME/.ssh/identity-imap -l xxx $1 /opt/net/etc/imapd
> 
> You aim ssh at a specific id in ~/.ssh -- login as xxx.... what does
> $1 do?  First argument to what... ssh?   --- imapd gets invoked and
> passes new mail to fetchmail?  Which then does what?

$1 is the first argument passed to the script. And as the manual page
of fetchmail says, it will pass the name of the server and the port of
the server as arguments to the script. So $1 in that case expands to
the name of the server.

This is so if I have my .ssh/config including usernames for my hosts,
I can use the same wrapper script for all connections.

As for the imapd, it get's eof after fetchmail is finished and dies
away - as the ssh connection shuts down.

> I've tried this with the following scripts:
> 
> .fetchmailrc:
> poll reader.local.lan with proto IMAP and options no dns
>  preauth ssh plugin /home/reader/scripts/ssh-fetch.sh
> 
> ssh-fetch.sh
> #!/bin/sh
> exec ssh reader@reader /usr/sbin/imapd
> 
> I can't tell if its going to work because some other mess with my
> FreeBSD install crops up to wreck it.
> 
> $ fetchmail 
> /usr/libexec/ld-elf.so.1: Shared object "libssl.so.1" not found
> 
> Fetchmail installed from FreeBSD install with no complaints.

Well it looks ok. But it seems you are missing ssl support. It is not
needed and if you compile fetchmail yourself, you shouldn't include
ssl support unless you really need it.

This can probably be fixed by installing libssl. This can be obtained
from www.openssl.org, I do not know about FreeBSD installs.

> This is starting to look like too much pain in the ass for such a
> simple chore.

Umm. I found this to be very easy to set up. Then again, I already had
fetchmail installed and was using it.

> Running just the shell script (ssh-fetch.sh) gives:
> 
> bsd > scripts/ssh-fetch.sh
> * PREAUTH reader.ptw.com IMAP4rev1 v12.264 server ready
> 
> whoopee .... now what?

Well this seems to be doing fine, your script works. If you type
ctrl-d there, you'll see the connection close down cleanly. Or you can
type 'a LIST "" %'. Or 'a SELECT INBOX'. Anyway that's just extra if
you wanna see how the server works.

> `man imapd' you say...  Well the one I have is just one tiny cut above
> useless, lacking descriptive information.

www.imap.org has pointers to the IMAP4rev1 RFC that you should look
for info on how IMAP works, not that you need it.

> I think I like the straight forward approach laid out by Kai and Lars.
> What are the chances on a local network that the `cat' command in the
> mail-sources prescript will die an ugly death?  I'm beginning to think
> what ever risk, it is  much less problem than sorting out a bunch of
> complicated soft ware for this one task.
> 
> Plus, it seems one could insert some kind of `trap' that would make it
> safer. 
> 
> Especially since I have an overall backup of all incoming mail that
> keeps the latest 1000 messages. 

Well, it would be awfully nice if mail-sources and nnimap supported
using a command instead of a direct connection.

-- Naked

Re: Gnus Access mail spool by ssh

[Nuutti Kotivuori]

"Harry" == Harry Putnam <reader@newsguy.com> writes:
> Alan Shutko <ats@acm.org> writes:
>> Harry Putnam <reader@newsguy.com> writes:
>> 
>> > What does fetchmail do with it.  Deliver to /var/mail/$USER or put it
>> > on standard out of ssh command or what?
>> 
>> Fetchmail will send it to your local MTA (ie, sendmail).  It used to
>> drop it in the mail spool, but that feature was taken out for some
>> reason.
> 
> Thanks... I've cleaned up the crippled fetchmail and now can run the
> scripts I posted .  Sure enough the mail is fetched to local incoming
> spool.
> 
> *BUT* it is snatched from the parent incoming spool, so that the
>  parent (remote) looses access.  This is not what I was after.  I want
>  both machines to be able to process the same mail in whatever way
>  needed.

You can use the keep option to prevent this. But mail is still marked
as read if it's fetched by the fetchmail.

> Back to duplicating with procmail on parent (remote) .  But how to aim
> fetchmail at the duplicated spool?  Maybe a dummy account for the
> laptop to access... I'll try that.

Um. IMAP has multiple folder support. Find out how your IMAP server
handles multiple folder and store the messages to a folder named
remote for example. Then just pass folder option in your fetchmailrc.

-- Naked

Re: Gnus Access mail spool by ssh

[Harry Putnam]

Nuutti Kotivuori <nuutti.kotivuori@sonera.com> writes:

[...]

> 
> Well as pointed out, fetchmail does as it's told. For the very best
> solutions, it likes to use a local SMTP server, if not, then MDA, if
> not then maybe BSMTP delivery or whatever. That has nothing to do with
> this setup, it's just fetchmail configuration.

In this case it is using local SMTP.

[...]

> Um. IMAP has multiple folder support. Find out how your IMAP server
> handles multiple folder and store the messages to a folder named
> remote for example. Then just pass folder option in your fetchmailrc.

After cleaning up my fetchmail install but before studying up on imap
workings regarding `multiple folders' I just created a user account
for `laptop' on the parent machine.  Forward copies of all mail there,
and access that spool with the method you've explained.  This works
fine.

One problem has cropped up and may be fetchmail specific but the laptop
fetchmail (version 5.4.1) seems to be eating the `Return-Path: '
headers.

Viewing the messages on parent machine in /var/mail/spool/laptop  I
see those headers in tact.  But once moved to the laptop those headers
are gone.   The parent machine is fetching with fetchmail-5.3.1-1.

Could something in the methodology be causing this or do you think it
would be a fetchmail version issue?

Do you see something similar?

Re: Gnus Access mail spool by ssh

[Simon Josefsson]

Nuutti Kotivuori <nuutti.kotivuori@sonera.com> writes:

> $ ssh mail.host.foo
> mail$ /usr/sbin/imapd
> * PREAUTH mail.host.foo IMAP4rev1 v12.264 server ready
> 
> Most imap servers are able to run from the command line and notice the
> situation and start up preauthenticated. And this is what can be used
> by fetchmail and the plugin option - and if nnimap does not support
> this, it would be nice.

I've added support for this.  It should be possible to use a method
such as

(nnimap "yourmailhost"
        (nnimap-stream shell))

>From the info pages:

`nnimap-stream'
     The type of stream used to connect to your server.  By default,
     nnimap will detect and automatically use all of the below, with
     the exception of SSL. (SSL is being replaced by STARTTLS, which
     can be automatically detected, but it's not widely deployed yet).
...
        * "shell:" Use a shell command to start IMAP connection.
...
     For IMAP connections using the `shell' stream, the variable
     `imap-shell-program' specify what program to call.

The variable documentation:

`imap-shell-program' is a variable declared in Lisp.
  -- loaded from "imap"

Value: ("ssh %s imapd" "rsh %s imapd" "ssh %g ssh %s imapd" "rsh %g rsh %s imapd")

Documentation:
A list of strings, containing commands for IMAP connection.
Within a string, %s is replaced with the server address, %p with port
number on server, %g with `imap-shell-host', and %l with
`imap-default-user'.  The program should read IMAP commands from stdin
and write IMAP response to stdout. Each entry in the list is tried
until a successful connection is made.

Re: Gnus Access mail spool by ssh

[Nuutti Kotivuori]

"Harry" == Harry Putnam <reader@newsguy.com> writes:
> Nuutti Kotivuori <nuutti.kotivuori@sonera.com> writes:
> [...]
> 
>> 
>> Well as pointed out, fetchmail does as it's told. For the very best
>> solutions, it likes to use a local SMTP server, if not, then MDA,
>> if not then maybe BSMTP delivery or whatever. That has nothing to
>> do with this setup, it's just fetchmail configuration.
> 
> In this case it is using local SMTP.
> 
> [...]
> 
>> Um. IMAP has multiple folder support. Find out how your IMAP server
>> handles multiple folder and store the messages to a folder named
>> remote for example. Then just pass folder option in your
>> fetchmailrc.
> 
> After cleaning up my fetchmail install but before studying up on
> imap workings regarding `multiple folders' I just created a user
> account for `laptop' on the parent machine.  Forward copies of all
> mail there, and access that spool with the method you've explained.
> This works fine.
> 
> One problem has cropped up and may be fetchmail specific but the
> laptop fetchmail (version 5.4.1) seems to be eating the
> `Return-Path: ' headers.
> 
> Viewing the messages on parent machine in /var/mail/spool/laptop I
> see those headers in tact.  But once moved to the laptop those
> headers are gone.  The parent machine is fetching with
> fetchmail-5.3.1-1.
> 
> Could something in the methodology be causing this or do you think
> it would be a fetchmail version issue?
> 
> Do you see something similar?

Um. Not a version issue I'd say. Think it might be defined behaviour
even. There's a fetchmail option called 'invisible' which will try to
make it invisible to the mda (normally it inserts received headers and
the like). Also, no-bounce is worth looking into, since it's kinda
tied in with return-path.

-- Naked