[9fans] gmail's new POP features + upas/fs

[9fans] gmail's new POP features + upas/fs

[David Leimbach]

I've not had any luck making a connection based on the wiki
information for setting up upas/fs for mail with poptls or apoptls.

upas/fs -f/poptls/pop.gmail.com just hangs it seems.

I've gotten it to work with other servers, I wonder what the difference is.

Dave

Re: [9fans] gmail's new POP features + upas/fs

[Russ Cox]

I haven't actually tried this, but looking at the
documentation it appears that you should
use /pop/ and not /poptls/.

Russ


On Sun, 21 Nov 2004 22:07:16 -0800, David Leimbach <leimy2k@gmail.com> wrote:
> I've not had any luck making a connection based on the wiki
> information for setting up upas/fs for mail with poptls or apoptls.
> 
> upas/fs -f/poptls/pop.gmail.com just hangs it seems.
> 
> I've gotten it to work with other servers, I wonder what the difference is.
> 
> Dave
>

Re: [9fans] gmail's new POP features + upas/fs

[David Leimbach]

On Mon, 22 Nov 2004 08:05:20 -0500, Russ Cox <russcox@gmail.com> wrote:
> I haven't actually tried this, but looking at the
> documentation it appears that you should
> use /pop/ and not /poptls/.
> 

According to this it's POP3 with ssl:
http://gmail.google.com/support/bin/answer.py?answer=13287

is that done with /pop/?  I'm confused.

Dave

Re: [9fans] gmail's new POP features + upas/fs

[Russ Cox]

> According to this it's POP3 with ssl:
> http://gmail.google.com/support/bin/answer.py?answer=13287
> 
> is that done with /pop/?  I'm confused.

Actually it wasn't done at all.  But now it can
be done with /pops/.  The difference between
/pops/ and /poptls/ is that /pops/ means dial
port 995 and start SSL/TLS immediately, while
/poptls/ means dial port 110 as usual and 
then issue a STLS command during handshaking
to start SSL/TLS.  Why have one way to do it
when you could have two?

I just wrote the code and pushed it to sources --
pull to get a new source and binary.

I can now get at (the oldest 300 messages of) my
gmail with upas/fs -f/pops/pop.gmail.com/russcox.

Russ

Re: [9fans] gmail's new POP features + upas/fs

[Russ Cox]

> is this normal:
> 
>         plan9% upas/fs -f/pops/pop.gmail.com/mirtchovski
>         upas/fs pop3: server certificate 2C471E10D5C1E3F4417680B8EF5507B20FF62F33 not recognized
>         upas/fs: opening mailbox: bad server certificate
>         plan9%
> 
> if you have a solution to this problem i'd be happy to put it in the
> wiki :)

yes, you need to add a line like

x509 sha1=2C471E10D5C1E3F4417680B8EF5507B20FF62F33 pop.gmail.com

to /sys/lib/tls/mail.  the "pop.gmail.com" is just a comment:
everything after the hash is ignored.

russ

Re: [9fans] gmail's new POP features + upas/fs

[David Leimbach]

On Mon, 22 Nov 2004 16:01:46 -0500, Russ Cox <russcox@gmail.com> wrote:
> > is this normal:
> >
> >         plan9% upas/fs -f/pops/pop.gmail.com/mirtchovski
> >         upas/fs pop3: server certificate 2C471E10D5C1E3F4417680B8EF5507B20FF62F33 not recognized
> >         upas/fs: opening mailbox: bad server certificate
> >         plan9%
> >
> > if you have a solution to this problem i'd be happy to put it in the
> > wiki :)
> 
> yes, you need to add a line like
> 
> x509 sha1=2C471E10D5C1E3F4417680B8EF5507B20FF62F33 pop.gmail.com
> 
> to /sys/lib/tls/mail.  the "pop.gmail.com" is just a comment:
> everything after the hash is ignored.
> 
> russ
> 


I think it's already been in the wiki. :)

Dave

Re: [9fans] gmail's new POP features + upas/fs

[David Leimbach]

> I can now get at (the oldest 300 messages of) my
> gmail with upas/fs -f/pops/pop.gmail.com/russcox.
> 
> Russ
> 

Hmmm... not sure what I'm doing wrong.  I just did

upas/fs -f/pops/pop.gmail.com/leimy2k

and I see there is a mount of '#|/data1' in /mail/fs in my namespace. 
But when I run nedmail or Acme mail it says 0 messages.

Yet when I look in my gmail I've got 1 in my inbox.  

Also "ls -l /mail/fs" shows:

--rw-rw-rw- M 69 dave dave 0 Nov 22 11:56 /mail/fs/ctl
d-r-xr-xr-x M 57 dave dave 0 Nov 17 19:30 /mail/fs/mbox

and inside /mail/fs/mbox is another ctl file.

---w--w--w- M 64 dave dave 0 Nov 22 17:23 ctl

Not sure if this is all normal.

I fixed my time/date and timezone and rebooted and I still see no mail
when I clearly have some in the web client.

I'm probably just misusing upas/fs but I'm not sure how.

Dave

Re: [9fans] gmail's new POP features + upas/fs

[andrey mirtchovski]

> Yet when I look in my gmail I've got 1 in my inbox.  
> 

i have the same problem -- whenever i open a new upas/fs from gmail
the old messages disappear!  this happens even is i open the same file
descriptor served by the same upas/fs as /srv/upasfs$user.

the new mount shows no messages, the old one shows the messages as
'deleted'.  upas/fs needs a 'leave messages on server' option.

andrey

Re: [9fans] gmail's new POP features + upas/fs

[David Leimbach]

On Mon, 22 Nov 2004 18:36:25 -0700, andrey mirtchovski
<mirtchov@cpsc.ucalgary.ca> wrote:
> 
> > Yet when I look in my gmail I've got 1 in my inbox.
> >
> 
> i have the same problem -- whenever i open a new upas/fs from gmail
> the old messages disappear!  this happens even is i open the same file
> descriptor served by the same upas/fs as /srv/upasfs$user.
> 
> the new mount shows no messages, the old one shows the messages as
> 'deleted'.  upas/fs needs a 'leave messages on server' option.
> 
> andrey
> 
Actually I don't ever see any messages at all :).  I went back and
double checked my gmail settings and made sure I enabled POP support. 
There's two settings one for only new mail from now on and another for
all mail.

Strange.

Dave

Re: [9fans] gmail's new POP features + upas/fs

[andrey mirtchovski]

> Actually I don't ever see any messages at all :).  I went back and
> double checked my gmail settings and made sure I enabled POP support. 
> There's two settings one for only new mail from now on and another for
> all mail.
> 
> Strange.
> 
> Dave

even if you send a message to yourself it still doesn't appear?  try
with faces running too -- i just saw my mug appear in faces after a
test message was sent, only to disappear immediately when i did a
'mount /srv/upasfs.andrey'..

Re: [9fans] gmail's new POP features + upas/fs

[David Leimbach]

On Mon, 22 Nov 2004 19:51:00 -0700, andrey mirtchovski
<mirtchov@cpsc.ucalgary.ca> wrote:
> 
> > Actually I don't ever see any messages at all :).  I went back and
> > double checked my gmail settings and made sure I enabled POP support.
> > There's two settings one for only new mail from now on and another for
> > all mail.
> >
> > Strange.
> >
> > Dave
> 
> even if you send a message to yourself it still doesn't appear?  try
> with faces running too -- i just saw my mug appear in faces after a
> test message was sent, only to disappear immediately when i did a
> 'mount /srv/upasfs.andrey'..
> 

Actually when I just do the following:

upas/fs -f/imaps/mail.opendarwin.org/leimy

I can then go and see my inbox mail in /mail/fs/mbox and cd into the
mail and look at headers and all that good stuff.

With:

"upas/fs -f/pops/pop.gmail.com/leimy2k"

I get nothing at all like that in /mail/fs/mbox.

Just a lonely ctl file.

Dave

>

Re: [9fans] gmail's new POP features + upas/fs

[David Leimbach]

On Mon, 22 Nov 2004 19:06:21 -0800, David Leimbach <leimy2k@gmail.com> wrote:
> On Mon, 22 Nov 2004 19:51:00 -0700, andrey mirtchovski
> 
> 
> <mirtchov@cpsc.ucalgary.ca> wrote:
> >
> > > Actually I don't ever see any messages at all :).  I went back and
> > > double checked my gmail settings and made sure I enabled POP support.
> > > There's two settings one for only new mail from now on and another for
> > > all mail.
> > >
> > > Strange.
> > >
> > > Dave
> >
> > even if you send a message to yourself it still doesn't appear?  try
> > with faces running too -- i just saw my mug appear in faces after a
> > test message was sent, only to disappear immediately when i did a
> > 'mount /srv/upasfs.andrey'..
> >
> 
> Actually when I just do the following:
> 
> upas/fs -f/imaps/mail.opendarwin.org/leimy
> 
> I can then go and see my inbox mail in /mail/fs/mbox and cd into the
> mail and look at headers and all that good stuff.
> 
> With:
> 
> "upas/fs -f/pops/pop.gmail.com/leimy2k"
> 
> I get nothing at all like that in /mail/fs/mbox.
> 
> Just a lonely ctl file.
> 
> Dave
> 
Even more mysterious is this dir that just showed up:

/mail/fs/leimy1101173179 that has a date of Dec 31 1969.

When I try to cd to it I am told it doesn't exist.

Dave


> >
>

Re: [9fans] gmail's new POP features + upas/fs

[David Leimbach]

On Mon, 22 Nov 2004 19:08:43 -0800, David Leimbach <leimy2k@gmail.com> wrote:
> On Mon, 22 Nov 2004 19:06:21 -0800, David Leimbach <leimy2k@gmail.com> wrote:
> 
> 
> > On Mon, 22 Nov 2004 19:51:00 -0700, andrey mirtchovski
> >
> >
> > <mirtchov@cpsc.ucalgary.ca> wrote:
> > >
> > > > Actually I don't ever see any messages at all :).  I went back and
> > > > double checked my gmail settings and made sure I enabled POP support.
> > > > There's two settings one for only new mail from now on and another for
> > > > all mail.
> > > >
> > > > Strange.
> > > >
> > > > Dave
> > >
> > > even if you send a message to yourself it still doesn't appear?  try
> > > with faces running too -- i just saw my mug appear in faces after a
> > > test message was sent, only to disappear immediately when i did a
> > > 'mount /srv/upasfs.andrey'..
> > >
> >
> > Actually when I just do the following:
> >
> > upas/fs -f/imaps/mail.opendarwin.org/leimy
> >
> > I can then go and see my inbox mail in /mail/fs/mbox and cd into the
> > mail and look at headers and all that good stuff.
> >
> > With:
> >
> > "upas/fs -f/pops/pop.gmail.com/leimy2k"
> >
> > I get nothing at all like that in /mail/fs/mbox.
> >
> > Just a lonely ctl file.
> >
> > Dave
> > 
> Even more mysterious is this dir that just showed up:
> 
> /mail/fs/leimy1101173179 that has a date of Dec 31 1969.
> 
> When I try to cd to it I am told it doesn't exist.
> 
> Dave

Ok now I see two emails in 'faces' with google's gmail logo on em.

Not sure what to do with it :)

Dave

> 
> > >
> >
>

Re: [9fans] gmail's new POP features + upas/fs

[Russ Cox]

> the new mount shows no messages, the old one shows the messages as
> 'deleted'.  upas/fs needs a 'leave messages on server' option.

'leave messages on server'
is the only way it behaves.  it only
deletes messages if you delete them 
during nedmail or Mail or whatever you
read mail with.

perhaps you've told gmail to delete
messages once pop downloads them.
(it's a gmail settings option in the pop section.)
the next time upas/fs logs in and looks
at your mail, they're not there, so it assumes
they've been deleted and forgets about them.

russ

Re: [9fans] gmail's new POP features + upas/fs

[andrey mirtchovski]

> perhaps you've told gmail to delete
> messages once pop downloads them.
> (it's a gmail settings option in the pop section.)
> the next time upas/fs logs in and looks
> at your mail, they're not there, so it assumes
> they've been deleted and forgets about them.
> 
> russ

i had them on archive first, then changed to 'leave in inbox'.  gmail
indicates that they're sending a 'copy to pop3', so don't think that's
to blame.

here's what i do:

	- kill all 'fs' in the current session, even better -- start a
	new one with drawterm and without starting upas/fs.

	- upas/fs -s -f/..../

	- mount /srv/upasfs.andrey and start acme Mail on that dir

	- send a message to myself on my gmail account

	- when the message appears in acme either do a new mount of
	/srv/upasfs...  or wait about a minute and the message will be
	marked as 'deleted' in the old Mail.  it never appears in the
	new one.

exiting the original acme mail and restarting it doesn't show the
message either, even though Mail wouldn't really delete the message
unless its window has been 'Put'.

i can verify that this problem doesn't exist with imaps and imap on
other accounts i have.

andrey

Re: [9fans] gmail's new POP features + upas/fs

[Russ Cox]

I think gmail is doing funny things.

Specifically, they make some noises in the
pop documentation (such as it is) that the
first time you check your mail you get the
first batch (say 300) of messages, then
the next time you get the next 300 or so,
and so on.  If that's really going on, that
they remember which messages they've
handed to pop and don't hand in a message
more than once, then upas/fs will not deal
well with that.

Upas/fs really expects that it's manipulating
a permanent mailbox.  It's not a typical usage
of pop (the typical usage is download, perhaps delete
from server, and keep locally even if the message
disappears from the server) but it usually works.
Gmail might be breaking the rules.

You could use a trivial little pop3 client to download
everything once and put it in an mbox, but
that's silly and you might as well just turn on 
forwarding instead.

What I really want is to have gmail pop present
exactly my inbox -- nothing less nothing more --
and when I "delete" something I really want it 
just moved out of my inbox and archived.
That doesn't seem to be an option.

Russ

Re: [9fans] gmail's new POP features + upas/fs

[andrey mirtchovski]

> I think gmail is doing funny things.
> 

i understand much better now, thanks!

Re: [9fans] gmail's new POP features + upas/fs

[Richard Miller]

russcox@gmail.com says:

> Upas/fs really expects that it's manipulating
> a permanent mailbox.  It's not a typical usage
> of pop (the typical usage is download, perhaps delete
> from server, and keep locally even if the message
> disappears from the server) but it usually works.
> Gmail might be breaking the rules.

The upas/fs pop3 code uses message numbers, not UIDs,
to identify messages on the server, and expects that
messages will be in the same order each time the client
reconnects.  I found this expectation wasn't true with
the demon.co.uk server, and indeed it doesn't seem to
be stipulated anywhere in the RFCs.

I made a modification to upas/fs/pop3.c to use UIDs,
which seemed to work reliably with demon - can't
remember whether I submitted it to sources, but I
switched to a different provider soon afterwards so
I stopped using it.

The same modification might help with gmail - if I
had a gmail account I could try it and see ;)

-- Richard

Re: [9fans] gmail's new POP features + upas/fs

[andrey mirtchovski]

> The same modification might help with gmail - if I
> had a gmail account I could try it and see ;)

i sent you an invite to "9fans at hamnavoe...", mail me privately if that's
not the right address.

i have a few more invites, if anyone wants one email me.

andrey

Re: [9fans] gmail's new POP features + upas/fs

[Christopher Nielsen]

On Tue, Nov 23, 2004 at 02:45:06PM -0700, andrey mirtchovski wrote:
> 
> i have a few more invites, if anyone wants one email me.

same here.

-- 
Christopher Nielsen
"They who can give up essential liberty for temporary
safety, deserve neither liberty nor safety." --Benjamin Franklin

Re: [9fans] gmail's new POP features + upas/fs

[andrey mirtchovski]

well, someone beat me to it. 

here's a free-for-all, just so we don't let it expire:

http://gmail.google.com/gmail/a-51fa9542ef-67719eb957-a37fa73555

Re: [9fans] gmail's new POP features + upas/fs

[Russ Cox]

> The upas/fs pop3 code uses message numbers, not UIDs,
> to identify messages on the server, and expects that
> messages will be in the same order each time the client
> reconnects.  I found this expectation wasn't true with
> the demon.co.uk server, and indeed it doesn't seem to
> be stipulated anywhere in the RFCs.

Well, yes and no.  It does use the UIDs but also depends
on ordering.  It expects new messages to appear at the
end of the list, which isn't stipulated in the RFCs but has
been true of every POP3 server I've ever seen (except the
original Plan 9 one, but I fixed that).  At the time I wrote
the code, it seemed that there were enough other mail
readers expecting the oldest->newest sort order that
it seemed a safe assumption.  To be fair, nowhere in the
spec does it say that the server has to support UIDs either,
so there's not really a safe way to do the upas/fs 
"stay in sync with the server" trick.  

I don't think this is Gmail's problem.  If that were happening,
the messages would just look reordered.  I'm pretty sure
gmail remembers which messages the POP client has 
downloaded and never shows them to the client again.
This is kind of necessary to work around the fact that POP
doesn't scale so well.  Each time you connect you get the
next 300 messages you haven't downloaded yet.

Russ

Re: [9fans] gmail's new POP features + upas/fs

[boyd, rounin]

> I don't think this is Gmail's problem.

neither do i, on many fronts.  the worst being that upas thinks that things are static and can haul 'em in ram.

if this is of any any interest, myself and brucee wrote this pop3 thing in limbo.

is may be simple, but it scales:

    http://www.insultant.net/repo/pop.bundle

Re: [9fans] gmail's new POP features + upas/fs

[boyd, rounin]

> The upas/fs pop3 code uses message numbers, not UIDs,
> to identify messages on the server, and expects that
> messages will be in the same order each time the client
> reconnects. 

this is not a problem as long as client side has it together.

my approach has been to pull each message across.  smashing it into a upas f/s is your choice.

Re: [9fans] gmail's new POP features + upas/fs

[David Leimbach]

On Mon, 22 Nov 2004 20:14:18 -0700, andrey mirtchovski
<mirtchov@cpsc.ucalgary.ca> wrote:
> 
> > perhaps you've told gmail to delete
> > messages once pop downloads them.
> > (it's a gmail settings option in the pop section.)
> > the next time upas/fs logs in and looks
> > at your mail, they're not there, so it assumes
> > they've been deleted and forgets about them.
> >
> > russ
> 
> i had them on archive first, then changed to 'leave in inbox'.  gmail
> indicates that they're sending a 'copy to pop3', so don't think that's
> to blame.
> 
> here's what i do:
> 
>         - kill all 'fs' in the current session, even better -- start a
>         new one with drawterm and without starting upas/fs.
> 
>         - upas/fs -s -f/..../
> 
>         - mount /srv/upasfs.andrey and start acme Mail on that dir
> 
>         - send a message to myself on my gmail account
> 
>         - when the message appears in acme either do a new mount of
>         /srv/upasfs...  or wait about a minute and the message will be
>         marked as 'deleted' in the old Mail.  it never appears in the
>         new one.
> 
> exiting the original acme mail and restarting it doesn't show the
> message either, even though Mail wouldn't really delete the message
> unless its window has been 'Put'.
> 
> i can verify that this problem doesn't exist with imaps and imap on
> other accounts i have.
> 
> andrey
> 
> 
Andrey,

I am sitting here watching faces and I got your race... I just got one
of russco on my machine too... and now it vanished...

It must take things that are in a regular inbox and move them to some
other directory?

quite weird :).  Perhaps Rob Pike can save us :)

Dave

Re: [9fans] gmail's new POP features + upas/fs

[Richard Miller]

russcox@gmail.com says:

> Well, yes and no.  It does use the UIDs but also depends
> on ordering.  ...
> 
> I don't think this is Gmail's problem.  If that were happening,
> the messages would just look reordered.

Having tried it myself now, I see what Russ means.  I did find with
demon.co.uk that the server's reordering of messages confused upas/fs
sufficiently that some would seem to disappear.  But gmail's trouble
is more fundamental - after you've fetched the messages once, it
replies +OK 0 0 to subsequent STAT commands.  Because upas/fs
keeps reattaching to the server to refresh its list, it sees an
apparently empty list and therefore deletes its own copy of messages
to keep in sync.

I wonder if this causes trouble with any other pop clients?

I suppose we could add a "gmail compatibility" flag to upas/fs/pop3
to stop it from deleting its copies of messages except by
explicit local request.  This would allow it to be used for
fetching stuff (once) from gmail to be copied to a local mailbox.

-- Richard

Re: [9fans] gmail's new POP features + upas/fs

[Russ Cox]

> I suppose we could add a "gmail compatibility" flag to upas/fs/pop3
> to stop it from deleting its copies of messages except by
> explicit local request.  This would allow it to be used for
> fetching stuff (once) from gmail to be copied to a local mailbox.

I don't think this makes much sense, since it will
only last until you reboot your terminal, and then
there go your messages.  You really want to just
download them with a trivial pop fetch program,
or forward them to your Plan 9 account directly,
so that they'll be on disk in your mbox.  Either
way you're maintaining two separate mailboxes
now.

Russ

Re: [9fans] gmail's new POP features + upas/fs

[Richard Miller]

Richard:
>> I suppose we could add a "gmail compatibility" flag to upas/fs/pop3
>> to stop it from deleting its copies of messages except by
>> explicit local request.  This would allow it to be used for
>> fetching stuff (once) from gmail to be copied to a local mailbox.

Russ:
> I don't think this makes much sense, since it will
> only last until you reboot your terminal, and then
> there go your messages.

Richard:
Yes, that's why I said "to be copied to a local mailbox".
I should have inserted "immediately".

Russ:
> You really want to just
> download them with a trivial pop fetch program,

Richard:
Yes, I'm still using the one you posted to 9fans many years ago.
Simple & reliable.

Re: [9fans] gmail's new POP features + upas/fs

[David Leimbach]

> 
> Russ:
> > You really want to just
> > download them with a trivial pop fetch program,
> 
> Richard:
> Yes, I'm still using the one you posted to 9fans many years ago.
> Simple & reliable.
> 
> 

is this basically "fetchmail-but simpler" for plan 9?  I might like to
try it out.

On that note has anyone used Gmail's SMTP+TLS service to send outgoing mail?

It'd be cool to handle all my gmail "locally" through plan 9.  

This would be probably the only daily use I've found for my plan 9 box
so far... except for learning about plan 9 which is also kind of fun
but not very productive in getting things done day-to-day.

Dave

Re: [9fans] gmail's new POP features + upas/fs

[Steve Simon]

I put a copy of getpop3.c in /n/sources/steve
Its Russ's code, but I bodged in support for factotum.

I use it to to pull from CCmail which is so slow it makes
direct upas/fs access painfull.

-Steve