Gnus development mailing list
 help / color / mirror / Atom feed
* gnutls status
@ 2010-11-25 17:29 Julien Danjou
  2010-11-26  0:28 ` Lars Magne Ingebrigtsen
  0 siblings, 1 reply; 28+ messages in thread
From: Julien Danjou @ 2010-11-25 17:29 UTC (permalink / raw)
  To: ding

[-- Attachment #1: Type: text/plain, Size: 233 bytes --]

Hi,

gnutls support has been added to Emacs 24. What's the status of it?
Could we use it in Gnus? Or is this still an Emacs side problem to
resolve?

-- 
Julien Danjou
// ᐰ <julien@danjou.info>   http://julien.danjou.info

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-25 17:29 gnutls status Julien Danjou
@ 2010-11-26  0:28 ` Lars Magne Ingebrigtsen
  2010-11-26 12:13   ` Ted Zlatanov
  0 siblings, 1 reply; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-26  0:28 UTC (permalink / raw)
  To: ding

Julien Danjou <julien@danjou.info> writes:

> gnutls support has been added to Emacs 24. What's the status of it?

It seems to work.

> Could we use it in Gnus? Or is this still an Emacs side problem to
> resolve?

nnimap has support for using it, but it's probably the wrong place to
put the support.  It should be put into tls.el, probably, so that all
the users don't have to know about the stuff...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-26  0:28 ` Lars Magne Ingebrigtsen
@ 2010-11-26 12:13   ` Ted Zlatanov
  2010-11-26 12:51     ` Julien Danjou
  2010-11-26 14:10     ` Lars Magne Ingebrigtsen
  0 siblings, 2 replies; 28+ messages in thread
From: Ted Zlatanov @ 2010-11-26 12:13 UTC (permalink / raw)
  To: ding; +Cc: emacs-devel

On Fri, 26 Nov 2010 01:28:46 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Julien Danjou <julien@danjou.info> writes:
>> gnutls support has been added to Emacs 24. What's the status of it?

LMI> It seems to work.

It's missing some features.  See
http://thread.gmane.org/gmane.emacs.devel/131441/focus=131551 and the
rest of that thread for the details.  The biggest one is callbacks.

We need callbacks to implement host name verification and certificate
chain checking, which are both IMO essential to making the Emacs GnuTLS
support "official."  In GnuTLS 2.8.x you can't set a callback function,
so the C code would need (from doc/examples/ex-rfc2818.c):

  if (gnutls_x509_crt_get_expiration_time (cert) < time (0))
    {
      printf ("The certificate has expired\n");
      return;
    }

  if (gnutls_x509_crt_get_activation_time (cert) > time (0))
    {
      printf ("The certificate is not yet activated\n");
      return;
    }

  if (!gnutls_x509_crt_check_hostname (cert, hostname))
    {
      printf ("The certificate's owner does not match hostname '%s'\n",
              hostname);
      return;
    }

which is very inflexible compared to a callback function.  We'd need to
add custom API options for each of the three checks above plus another
for the certificate chain verification; in addition it would be harder
to interact with the user and store trusted certificates from C.

2.10.x and above let us set a callback function, which would make all of
the above easier and more convenient from ELisp-land.  The problem is
that 2.10.x hasn't been widely adopted in Debian and thus won't work by
default.

So we'd need to either 1) require 2.10.x, or 2) complicate the C code
and API using just 2.8.x features and maybe figure out how to set up our
own callback mechanism, or 3) use the 2.10.x features only when it's
available, using autoconf detection, which is twice as complicated as
(2).

(2) and (3) will require a lot of new code that is completely
unnecessary under 2.10.x.

This is essentially why I haven't worked on the GnuTLS support in a bit:
I don't know the best way forward.  If anyone can suggest a good way to
do it, I'm all ears.  I also don't know about 2.10.x's status in the
major distros and whether Emacs can require that version or higher
specifically.

>> Could we use it in Gnus? Or is this still an Emacs side problem to
>> resolve?

LMI> nnimap has support for using it, but it's probably the wrong place to
LMI> put the support.  It should be put into tls.el, probably, so that all
LMI> the users don't have to know about the stuff...

I think every package should explicitly choose to support gnutls.el, it
shouldn't be an Emacs-wide choice.  There's too many configuration
options that depend on the purpose.  For instance IMAP and HTTPS have
really different security needs.  Also, GnuTLS is much more configurable
than the older command-line interfaces so normalizing the API is not so
easy.  This is why I removed the old tls.el compatibility code from
gnutls.el and it only has gnutls-* functions now (`open-gnutls-stream'
and `gnutls-negotiate' being the main entry points from the ELisp side,
while `gnutls-boot' is the main entry point from the C side).

IMO tls.el and friends should be made deprecated as soon as gnutls.el is
capable to verify a certificate chain, the expiration date, and the
hostname.  They are insecure and the cause of many bug reports over the
years, especially on the W32 platform.

As with the C issues above, if you have opinions or suggestions on how
to do all this better, including Gnus but also for Emacs in general,
tell me.  I'm CC-ing to emacs-devel as well.

Ted




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-26 12:13   ` Ted Zlatanov
@ 2010-11-26 12:51     ` Julien Danjou
  2010-11-26 15:02       ` Stefan Monnier
  2010-11-26 14:10     ` Lars Magne Ingebrigtsen
  1 sibling, 1 reply; 28+ messages in thread
From: Julien Danjou @ 2010-11-26 12:51 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: ding, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 1977 bytes --]

On Fri, Nov 26 2010, Ted Zlatanov wrote:

> 2.10.x and above let us set a callback function, which would make all of
> the above easier and more convenient from ELisp-land.  The problem is
> that 2.10.x hasn't been widely adopted in Debian and thus won't work by
> default.

It's in experimental. It won't go in sid until squeeze is released,
which should be very soon now. So there's nothing to wait IMHO.
At the time Emacs 24 will be released, gnutls 2.10 would have been
since a very long in Debian. :)

> This is essentially why I haven't worked on the GnuTLS support in a bit:
> I don't know the best way forward.  If anyone can suggest a good way to
> do it, I'm all ears.  I also don't know about 2.10.x's status in the
> major distros and whether Emacs can require that version or higher
> specifically.

My opinion is go with 2.10.

Some distro do not update often packages if they're is no urgent need
to, anyhow. So you will just be pushing them a bit, which is not a bad
thing. :)

Gentoo, OpenBSD, FreeBSD already have it FWIW. And Debian in
experimental, so it will end up in Debian and Ubuntu in a couple of
month top[1].

This is the development model I used for the last years in various
project like the awesome window manager, often requiring recent version
of libraries it uses. I consider I don't have enough time to write
compatibility code for older library release, while smart people write
better interface for us to use.
The only downside is for people using old OS for whatever reason. But
they *will have* to upgrade other time anyhow, so why not doing sooner
than later? Since we're talking a user program (not a production Web
server), there's no big counter-argument to this, IMHO.

Of course, YMMV, as your amount of spare time to be used writing
compatibility code. ;)

[1]  Depends if squeeze is released soon, of course.

-- 
Julien Danjou
// ᐰ <julien@danjou.info>   http://julien.danjou.info

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-26 12:13   ` Ted Zlatanov
  2010-11-26 12:51     ` Julien Danjou
@ 2010-11-26 14:10     ` Lars Magne Ingebrigtsen
  2010-11-27 14:18       ` Lars Magne Ingebrigtsen
  2010-12-14 22:59       ` gnutls status Ted Zlatanov
  1 sibling, 2 replies; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-26 14:10 UTC (permalink / raw)
  To: ding; +Cc: emacs-devel

Ted Zlatanov <tzz@lifelogs.com> writes:

> So we'd need to either 1) require 2.10.x, or 2) complicate the C code
> and API using just 2.8.x features and maybe figure out how to set up our
> own callback mechanism, or 3) use the 2.10.x features only when it's
> available, using autoconf detection, which is twice as complicated as
> (2).

Is 2.10.x at least backwards-compatible, so that if we do implement the
complicated 2.8.x features, it'll continue to work in the future, too?

> I think every package should explicitly choose to support gnutls.el, it
> shouldn't be an Emacs-wide choice.  There's too many configuration
> options that depend on the purpose.  For instance IMAP and HTTPS have
> really different security needs.

True.  On the other hand, look at `nnimap-open-connection' and weep.  It
started off as a simple function and grew into a monster.  That's why I
haven't adapted nntp/pop3/etc to use gnutls yet -- I don't want to have
to repeat the same code all over the place.

And the STARTTLS situation, in particular, is a nightmare, since the
sane gnutls way of doing it and the really awkward starttls.el way of
doing it require different code paths.

And we're going to have to support gnutls/tls.el/starttls.el in all the
connections for the unforeseeable future.  I kinda want to write a new
connection framework that'll just separate out all this cruft into a
separate package, but I can't really see a sensible unified interface.
Especially with the STARTTLS stuff.

I mean, ideally, in the future whenever you do a
pop3/imap/nntp/smtp/etc connection, Emacs should opportunistically
switch on STARTTLS if the server supports it.  Encryption is good.
Switching on STARTTLS is virtually free if you have gnutls.  It's really
expensive if you don't.  I mean, time-wise for the user.

But I don't really see an easy way to do that.  You'd need
protocol-specific callbacks and stuff.  And I hate frameworks.  And
it'll be brittle, anyway, since starttls.el is kinda brittle.

But I do think we need some kinda of compat library to avoid going
totally insane.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-26 12:51     ` Julien Danjou
@ 2010-11-26 15:02       ` Stefan Monnier
  2010-11-26 15:56         ` Julien Danjou
  0 siblings, 1 reply; 28+ messages in thread
From: Stefan Monnier @ 2010-11-26 15:02 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: ding, emacs-devel

>> 2.10.x and above let us set a callback function, which would make all of
>> the above easier and more convenient from ELisp-land.  The problem is
>> that 2.10.x hasn't been widely adopted in Debian and thus won't work by
>> default.

> It's in experimental. It won't go in sid until squeeze is released,
> which should be very soon now. So there's nothing to wait IMHO.
> At the time Emacs 24 will be released, gnutls 2.10 would have been
> since a very long in Debian. :)

Many people use Debian stable, so until it's not in Debian stable, it's
not really in Debian.


        Stefan



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-26 15:02       ` Stefan Monnier
@ 2010-11-26 15:56         ` Julien Danjou
  2010-11-26 18:42           ` Stefan Monnier
  0 siblings, 1 reply; 28+ messages in thread
From: Julien Danjou @ 2010-11-26 15:56 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Ted Zlatanov, ding, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 299 bytes --]

On Fri, Nov 26 2010, Stefan Monnier wrote:

> Many people use Debian stable, so until it's not in Debian stable, it's
> not really in Debian.

But does many people use Debian stable with emacs development version?

-- 
Julien Danjou
// ᐰ <julien@danjou.info>   http://julien.danjou.info

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-26 15:56         ` Julien Danjou
@ 2010-11-26 18:42           ` Stefan Monnier
  2010-11-27  9:36             ` Julien Danjou
  0 siblings, 1 reply; 28+ messages in thread
From: Stefan Monnier @ 2010-11-26 18:42 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: ding, emacs-devel

>> Many people use Debian stable, so until it's not in Debian stable, it's
>> not really in Debian.
> But does many people use Debian stable with emacs development version?

That's not the relevant question: the relevant question is whether 2.10
will be in Debian stable by the time Emacs-24 is released.  It might,
but I'm not completely sure.


        Stefan



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-26 18:42           ` Stefan Monnier
@ 2010-11-27  9:36             ` Julien Danjou
  2010-11-27 14:28               ` Stefan Monnier
  2010-11-28  9:55               ` Lars Magne Ingebrigtsen
  0 siblings, 2 replies; 28+ messages in thread
From: Julien Danjou @ 2010-11-27  9:36 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Ted Zlatanov, ding, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 786 bytes --]

On Fri, Nov 26 2010, Stefan Monnier wrote:

> That's not the relevant question: the relevant question is whether 2.10
> will be in Debian stable by the time Emacs-24 is released.  It might,
> but I'm not completely sure.

Depends on how you think about it. Emacs 24 will be probably released
while Squeeze is stable, which does not have gnutls 2.10 (but may have
it through backports).

Next Debian stable would have Emacs 24, and gnutls 2.10.

So gnutls 2.10 will not be in Debian stable when Emacs 24 get released
(unless Emacs 24 is released after mid-2012). But most users won't try
to install Emacs 24 on it anyway.

OTOH Emacs 24 and gnutls 2.10 will be in the next Debian stable. :)

-- 
Julien Danjou
// ᐰ <julien@danjou.info>   http://julien.danjou.info

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-26 14:10     ` Lars Magne Ingebrigtsen
@ 2010-11-27 14:18       ` Lars Magne Ingebrigtsen
  2010-11-27 14:40         ` Lars Magne Ingebrigtsen
  2010-12-14 22:59       ` gnutls status Ted Zlatanov
  1 sibling, 1 reply; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-27 14:18 UTC (permalink / raw)
  To: ding

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> But I don't really see an easy way to do that.  You'd need
> protocol-specific callbacks and stuff.  And I hate frameworks.  And
> it'll be brittle, anyway, since starttls.el is kinda brittle.

I've gone ahead and done so, anyway.  nnimap got smaller, and I think it
should be easy enough to adapt it to NNTP, SMTP and POP3, but, so far,
my only use case is IMAP, so I would guess that I haven't covered all
bases yet.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27  9:36             ` Julien Danjou
@ 2010-11-27 14:28               ` Stefan Monnier
  2010-11-28  9:55               ` Lars Magne Ingebrigtsen
  1 sibling, 0 replies; 28+ messages in thread
From: Stefan Monnier @ 2010-11-27 14:28 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: ding, emacs-devel

> So gnutls 2.10 will not be in Debian stable when Emacs 24 get released
> (unless Emacs 24 is released after mid-2012). But most users won't try
> to install Emacs 24 on it anyway.

That's a good point.  Especially since gnutls is not indispensable for
Emacs, so as long as we properly handle the case where the gnutls library
to too old (either by not using it, or by using some fallback code),
then I'm OK with it.


        Stefan



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27 14:18       ` Lars Magne Ingebrigtsen
@ 2010-11-27 14:40         ` Lars Magne Ingebrigtsen
  2010-11-27 15:31           ` Lars Magne Ingebrigtsen
  0 siblings, 1 reply; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-27 14:40 UTC (permalink / raw)
  To: ding

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I've gone ahead and done so, anyway.  nnimap got smaller, and I think it
> should be easy enough to adapt it to NNTP, SMTP and POP3, but, so far,
> my only use case is IMAP, so I would guess that I haven't covered all
> bases yet.

I've now pushed this for nnimap, and it Works For Me, but it needs more
testing.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27 14:40         ` Lars Magne Ingebrigtsen
@ 2010-11-27 15:31           ` Lars Magne Ingebrigtsen
  2010-11-27 16:04             ` Lars Magne Ingebrigtsen
  2010-11-28  2:36             ` Automatic STARTTLS upgrades (was: gnutls status) Lars Magne Ingebrigtsen
  0 siblings, 2 replies; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-27 15:31 UTC (permalink / raw)
  To: ding

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I've now pushed this for nnimap, and it Works For Me, but it needs more
> testing.

What the hey.  I've now altered NNTP to use proto-stream, and it'll
switch on STARTTLS now if the server supports it.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27 15:31           ` Lars Magne Ingebrigtsen
@ 2010-11-27 16:04             ` Lars Magne Ingebrigtsen
  2010-11-27 16:37               ` Steinar Bang
  2010-11-28  2:36             ` Automatic STARTTLS upgrades (was: gnutls status) Lars Magne Ingebrigtsen
  1 sibling, 1 reply; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-27 16:04 UTC (permalink / raw)
  To: ding

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> What the hey.  I've now altered NNTP to use proto-stream, and it'll
> switch on STARTTLS now if the server supports it.

I don't have access to any STARTTLS NNTP servers, so please give it a
whirl.

I've now also tested this stuff with a non-gnutls-enabled Emacs, and
after some further hacking away, it seems to work for me with nnimap +
tls and starttls.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27 16:04             ` Lars Magne Ingebrigtsen
@ 2010-11-27 16:37               ` Steinar Bang
  2010-11-27 16:41                 ` Lars Magne Ingebrigtsen
  0 siblings, 1 reply; 28+ messages in thread
From: Steinar Bang @ 2010-11-27 16:37 UTC (permalink / raw)
  To: ding

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> I don't have access to any STARTTLS NNTP servers,...

You can't spin up that functionality on the gmane INN?




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27 16:37               ` Steinar Bang
@ 2010-11-27 16:41                 ` Lars Magne Ingebrigtsen
  2010-11-27 16:59                   ` Lars Magne Ingebrigtsen
  0 siblings, 1 reply; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-27 16:41 UTC (permalink / raw)
  To: ding

Steinar Bang <sb@dod.no> writes:

> You can't spin up that functionality on the gmane INN?

I don't know whether it supports it?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27 16:41                 ` Lars Magne Ingebrigtsen
@ 2010-11-27 16:59                   ` Lars Magne Ingebrigtsen
  2010-11-27 17:33                     ` Dan Christensen
  0 siblings, 1 reply; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-27 16:59 UTC (permalink / raw)
  To: ding

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I don't know whether it supports it?

It apparently does.  I've now recompiled inn with openssl support, but I
haven't put it into production yet.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27 16:59                   ` Lars Magne Ingebrigtsen
@ 2010-11-27 17:33                     ` Dan Christensen
  2010-11-27 17:36                       ` Lars Magne Ingebrigtsen
  0 siblings, 1 reply; 28+ messages in thread
From: Dan Christensen @ 2010-11-27 17:33 UTC (permalink / raw)
  To: ding

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> It apparently does.  I've now recompiled inn with openssl support, but I
> haven't put it into production yet.

Is that going to add overhead to gmane connections (and add load to the
server)?  I don't really need my gmane connections encrypted...

Dan




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27 17:33                     ` Dan Christensen
@ 2010-11-27 17:36                       ` Lars Magne Ingebrigtsen
  2010-11-27 17:42                         ` Lars Magne Ingebrigtsen
  0 siblings, 1 reply; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-27 17:36 UTC (permalink / raw)
  To: ding

Dan Christensen <jdc@uwo.ca> writes:

> Is that going to add overhead to gmane connections (and add load to the
> server)?  I don't really need my gmane connections encrypted...

Well, setting up the connection is going to be marginally slower, since
there are a few additional round trips added.  The added load on the
server is pretty marginal.  SSL doesn't take much CPU power.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27 17:36                       ` Lars Magne Ingebrigtsen
@ 2010-11-27 17:42                         ` Lars Magne Ingebrigtsen
  0 siblings, 0 replies; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-27 17:42 UTC (permalink / raw)
  To: ding

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Well, setting up the connection is going to be marginally slower, since
> there are a few additional round trips added.  The added load on the
> server is pretty marginal.  SSL doesn't take much CPU power.

But I've added a new variable for people who want to disable this:
`proto-stream-always-use-starttls'.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Automatic STARTTLS upgrades (was: gnutls status)
  2010-11-27 15:31           ` Lars Magne Ingebrigtsen
  2010-11-27 16:04             ` Lars Magne Ingebrigtsen
@ 2010-11-28  2:36             ` Lars Magne Ingebrigtsen
  2010-11-28 12:28               ` Automatic STARTTLS upgrades Lars Magne Ingebrigtsen
  1 sibling, 1 reply; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-28  2:36 UTC (permalink / raw)
  To: ding

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> What the hey.  I've now altered NNTP to use proto-stream, and it'll
> switch on STARTTLS now if the server supports it.

But I'm wondering about the defaults a bit.  This could be brittle and
slow if things don't actually work.

So let's take the worst-case scenarios:

* First we open a normal network connection, and we say "CAPABILITY" in
  one way or another.  This should be safe, but it's one extra round
  trip.

* If the server says that it supports STARTTLS, we enable either:

** If Emacs has gnutls support, we just enable STARTTLS.  One extra
   round trip.

** If Emacs doesn't, we sever the connection, and start it again with
   starttls.el, which defaults to gnutls-cli.  This may fail because of
   a faulty gnutls-cli installation, in which case we error out.  But it
   it's successful, then it's a close, an open, and an STARTTLS
   enablement.  So that's at least a ... big handful of extra round
   trips. 

* We then do the TLS negotiation.  This is at least one round trip, but
  it's probably more.  It may fail for any number of reasons.  For
  instance, inn may be compiled with TLS support, but not have the
  certificates.  Or they may be grossly invalid.  In that case, we error
  out.

* On success, we do CAPABILITY again, since they have a tendency to
  change after STARTTLS in some protocols.

* We're done!

So, as you can see, this introduces many new, interesting failure modes,
which have to be mitigated somehow.

First of all, I've now switched off opportunistic STARTTLS upgrades if
we don't have an Emacs with built-in gnutls support.

Secondly, I think if the STARTTLS stuff fails for any reason, the
connection should go back to being a plain network connection.  Is that
possible with the built-in gnutls stuff, Ted?  Otherwise, proto-stream
has to reopen the connection...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-27  9:36             ` Julien Danjou
  2010-11-27 14:28               ` Stefan Monnier
@ 2010-11-28  9:55               ` Lars Magne Ingebrigtsen
  1 sibling, 0 replies; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-28  9:55 UTC (permalink / raw)
  To: emacs-devel; +Cc: ding

Julien Danjou <julien@danjou.info> writes:

> Next Debian stable would have Emacs 24, and gnutls 2.10.
>
> So gnutls 2.10 will not be in Debian stable when Emacs 24 get released
> (unless Emacs 24 is released after mid-2012). But most users won't try
> to install Emacs 24 on it anyway.

True.  And people who use Emacs 24 on older machines can use the
external gnutls-cli/openssl solutions.  They're slower and slightly less
secure, but people have been living with that solution for decades, so
they can do so a year more.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Automatic STARTTLS upgrades
  2010-11-28  2:36             ` Automatic STARTTLS upgrades (was: gnutls status) Lars Magne Ingebrigtsen
@ 2010-11-28 12:28               ` Lars Magne Ingebrigtsen
  2010-11-28 13:34                 ` Lars Magne Ingebrigtsen
  0 siblings, 1 reply; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-28 12:28 UTC (permalink / raw)
  To: ding

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> * We then do the TLS negotiation.  This is at least one round trip, but
>   it's probably more.  It may fail for any number of reasons.  For
>   instance, inn may be compiled with TLS support, but not have the
>   certificates.  Or they may be grossly invalid.  In that case, we error
>   out.

I've now fixed this.  If the server announces STARTTLS, but gives an
error when we try to enable it, the connection will just stay
unencrypted.  This works both with the built-in gnutls support and the
starttls.el support.  (Although in the latter case, the stream will
still pass through gnutls-cli, but unencrypted.  So it'll be marginally
slower, but it shouldn't matter much.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: Automatic STARTTLS upgrades
  2010-11-28 12:28               ` Automatic STARTTLS upgrades Lars Magne Ingebrigtsen
@ 2010-11-28 13:34                 ` Lars Magne Ingebrigtsen
  0 siblings, 0 replies; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2010-11-28 13:34 UTC (permalink / raw)
  To: ding

I guess I could convert pop3.el to using proto-stream.el for
opportunistic STARTTLS upgrades, but I should probably wait a bit and
see how well this works with IMAP and NNTP first, I guess...  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-11-26 14:10     ` Lars Magne Ingebrigtsen
  2010-11-27 14:18       ` Lars Magne Ingebrigtsen
@ 2010-12-14 22:59       ` Ted Zlatanov
  2011-03-01 21:52         ` Ted Zlatanov
  1 sibling, 1 reply; 28+ messages in thread
From: Ted Zlatanov @ 2010-12-14 22:59 UTC (permalink / raw)
  To: ding; +Cc: emacs-devel

On Fri, 26 Nov 2010 15:10:39 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Is 2.10.x at least backwards-compatible, so that if we do implement the
LMI> complicated 2.8.x features, it'll continue to work in the future, too?

Yes.  They try really hard to keep backwards compatibility.  I'd guess
for all 2.x releases we'll be OK unless there's newer features we simply
must have :)

>> I think every package should explicitly choose to support gnutls.el, it
>> shouldn't be an Emacs-wide choice.  There's too many configuration
>> options that depend on the purpose.  For instance IMAP and HTTPS have
>> really different security needs.

LMI> True.  On the other hand, look at `nnimap-open-connection' and weep.  It
LMI> started off as a simple function and grew into a monster.  That's why I
LMI> haven't adapted nntp/pop3/etc to use gnutls yet -- I don't want to have
LMI> to repeat the same code all over the place.

...thanks for working on proto-stream.el, it's made all this much simpler.

On Fri, 26 Nov 2010 13:51:09 +0100 Julien Danjou <julien@danjou.info> wrote: 

JD> My opinion is go with 2.10.

On Sat, 27 Nov 2010 09:28:11 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> So gnutls 2.10 will not be in Debian stable when Emacs 24 get released
>> (unless Emacs 24 is released after mid-2012). But most users won't try
>> to install Emacs 24 on it anyway.

SM> That's a good point.  Especially since gnutls is not indispensable for
SM> Emacs, so as long as we properly handle the case where the gnutls library
SM> to too old (either by not using it, or by using some fallback code),
SM> then I'm OK with it.

OK, I will put upgrading Emacs' support of GnuTLS to 2.10 on my TODO
list.  Which means it won't happen soon, so if anyone else is feeling
frisky, go ahead and do it.  I listed what's necessary in my original
e-mail here; basically it's callbacks but there's other features in the
2.10 API we should use if possible.

I think that supporting 2.8 is counterproductive so I'll happily target
only 2.10.  I think that's reasonable and doesn't put a big burden on
the users and distros.

Ted




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2010-12-14 22:59       ` gnutls status Ted Zlatanov
@ 2011-03-01 21:52         ` Ted Zlatanov
  2011-03-05 11:01           ` Lars Magne Ingebrigtsen
  0 siblings, 1 reply; 28+ messages in thread
From: Ted Zlatanov @ 2011-03-01 21:52 UTC (permalink / raw)
  To: ding; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 1682 bytes --]

On Tue, 14 Dec 2010 16:59:34 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Fri, 26 Nov 2010 15:10:39 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
LMI> Is 2.10.x at least backwards-compatible, so that if we do implement the
LMI> complicated 2.8.x features, it'll continue to work in the future, too?

TZ> Yes.  They try really hard to keep backwards compatibility.  I'd guess
TZ> for all 2.x releases we'll be OK unless there's newer features we simply
TZ> must have :)

Argh, GnuTLS 2.8.x is still standard on Ubuntu 10.10, so practically we
should support it.  Below is my first (untested) patch to generate the
HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE and
HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY definitions in the configure.in
using AC_CHECK_FUNCS and then use them (currently just #ifdef
placeholders) in gnutls.c.  I plan to retrieve them from the :callbacks
alist parameter to `gnutls-boot'.

Regenerating "configure" failed for me.  I get this error at the end:

./configure: line 12620: gl_ASSERT_NO_GNULIB_POSIXCHECK: command not found
./configure: line 12621: gl_ASSERT_NO_GNULIB_TESTS: command not found
./configure: line 12622: gl_INIT: command not found
checking for lstat... yes
./configure: line 12648: syntax error near unexpected token `lstat'
./configure: line 12648: `gl_SYS_STAT_MODULE_INDICATOR(lstat)'

at the end.  But it gets far enough that I can tell the tests are being
run.  This is why the patch is untested; I'll see if I can figure out
why that's happening.  It may be an Ubuntu oddity.

Please let me know if the proposed approach is reasonable and if you
have any comments.  In theory this should be pretty trivial.

Thanks
Ted



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: callbacks.patch --]
[-- Type: text/x-diff, Size: 2684 bytes --]

=== modified file 'configure.in'
--- configure.in	2011-02-24 04:28:17 +0000
+++ configure.in	2011-03-01 21:39:23 +0000
@@ -1972,12 +1972,26 @@
 AC_SUBST(LIBSELINUX_LIBS)
 
 HAVE_GNUTLS=no
+HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY=no
+HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE=no
 if test "${with_gnutls}" = "yes" ; then
   PKG_CHECK_MODULES([LIBGNUTLS], [gnutls >= 2.2.4], HAVE_GNUTLS=yes, HAVE_GNUTLS=no)
   if test "${HAVE_GNUTLS}" = "yes"; then
     AC_DEFINE(HAVE_GNUTLS, 1, [Define if using GnuTLS.])
   fi
+
+  AC_CHECK_FUNCS(gnutls_certificate_set_verify_function, HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY=yes)
+  AC_CHECK_FUNCS(gnutls_certificate_client_set_retrieve_function, HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE=yes)
+
+  if test "${HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE}" = "yes"; then
+    AC_DEFINE(HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE, 1, [Define if using GnuTLS certificate retrieval callbacks.])
+  fi
+
+  if test "${HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY}" = "yes"; then
+    AC_DEFINE(HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY, 1, [Define if using GnuTLS certificate verification callbacks.])
+  fi
 fi
+
 AC_SUBST(LIBGNUTLS_LIBS)
 AC_SUBST(LIBGNUTLS_CFLAGS)
 
@@ -3667,6 +3681,8 @@
 echo "  Does Emacs use -lgconf?                                 ${HAVE_GCONF}"
 echo "  Does Emacs use -lselinux?                               ${HAVE_LIBSELINUX}"
 echo "  Does Emacs use -lgnutls?                                ${HAVE_GNUTLS}"
+echo "  Does Emacs use -lgnutls certificate verify callbacks?   ${HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY}"
+echo "  Does Emacs use -lgnutls certificate retrieve callbacks? ${HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE}"
 echo "  Does Emacs use -lxml2?                                  ${HAVE_LIBXML2}"
 
 echo "  Does Emacs use -lfreetype?                              ${HAVE_FREETYPE}"

=== modified file 'src/gnutls.c'
--- src/gnutls.c	2011-01-25 04:08:28 +0000
+++ src/gnutls.c	2011-03-01 21:41:36 +0000
@@ -484,6 +484,16 @@
 
   GNUTLS_INITSTAGE (proc) = GNUTLS_STAGE_FILES;
 
+  GNUTLS_LOG (1, max_log_level, "gnutls callbacks");
+
+  GNUTLS_INITSTAGE (proc) = GNUTLS_STAGE_CALLBACKS;
+
+#ifdef HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY
+#endif
+
+#ifdef HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE
+#endif
+
   GNUTLS_LOG (1, max_log_level, "gnutls_init");
 
   ret = gnutls_init (&state, GNUTLS_CLIENT);

=== modified file 'src/gnutls.h'
--- src/gnutls.h	2011-01-25 04:08:28 +0000
+++ src/gnutls.h	2011-03-01 21:32:17 +0000
@@ -28,6 +28,7 @@
   GNUTLS_STAGE_EMPTY = 0,
   GNUTLS_STAGE_CRED_ALLOC,
   GNUTLS_STAGE_FILES,
+  GNUTLS_STAGE_CALLBACKS,
   GNUTLS_STAGE_INIT,
   GNUTLS_STAGE_PRIORITY,
   GNUTLS_STAGE_CRED_SET,


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2011-03-01 21:52         ` Ted Zlatanov
@ 2011-03-05 11:01           ` Lars Magne Ingebrigtsen
  2011-03-05 14:46             ` Ted Zlatanov
  0 siblings, 1 reply; 28+ messages in thread
From: Lars Magne Ingebrigtsen @ 2011-03-05 11:01 UTC (permalink / raw)
  To: ding; +Cc: emacs-devel

Ted Zlatanov <tzz@lifelogs.com> writes:

> Regenerating "configure" failed for me.  I get this error at the end:
>
> ./configure: line 12620: gl_ASSERT_NO_GNULIB_POSIXCHECK: command not found
> ./configure: line 12621: gl_ASSERT_NO_GNULIB_TESTS: command not found
> ./configure: line 12622: gl_INIT: command not found
> checking for lstat... yes
> ./configure: line 12648: syntax error near unexpected token `lstat'
> ./configure: line 12648: `gl_SYS_STAT_MODULE_INDICATOR(lstat)'

I tried applying your patch to configure.in, and it worked for me.

Here's the output from around the lstat check:

checking for working GNU getopt function... yes
checking whether getenv is declared... yes
checking for lstat... yes
checking for alarm... yes


-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen




^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: gnutls status
  2011-03-05 11:01           ` Lars Magne Ingebrigtsen
@ 2011-03-05 14:46             ` Ted Zlatanov
  0 siblings, 0 replies; 28+ messages in thread
From: Ted Zlatanov @ 2011-03-05 14:46 UTC (permalink / raw)
  To: ding; +Cc: emacs-devel

On Sat, 05 Mar 2011 12:01:39 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Regenerating "configure" failed for me.  I get this error at the end:
>> 
>> ./configure: line 12620: gl_ASSERT_NO_GNULIB_POSIXCHECK: command not found
>> ./configure: line 12621: gl_ASSERT_NO_GNULIB_TESTS: command not found
>> ./configure: line 12622: gl_INIT: command not found
>> checking for lstat... yes
>> ./configure: line 12648: syntax error near unexpected token `lstat'
>> ./configure: line 12648: `gl_SYS_STAT_MODULE_INDICATOR(lstat)'

LMI> I tried applying your patch to configure.in, and it worked for me.

LMI> Here's the output from around the lstat check:

LMI> checking for working GNU getopt function... yes
LMI> checking whether getenv is declared... yes
LMI> checking for lstat... yes
LMI> checking for alarm... yes

Something is broken in my autoconf setup at home, because the exact same
configure.in worked on another machine (same Ubuntu 10.10 setup!)
yesterday.  Weird.

Anyhow, since GnuTLS 2.10 is not available even in Ubuntu 10.10, I'm
working on a way to control the verification with the GnuTLS
verification flags.  It's not as good as a verification callback but it
will be more useful for the vast majority of people right now.  I hope
to have something ready this weekend.

Ted




^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2011-03-05 14:46 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-11-25 17:29 gnutls status Julien Danjou
2010-11-26  0:28 ` Lars Magne Ingebrigtsen
2010-11-26 12:13   ` Ted Zlatanov
2010-11-26 12:51     ` Julien Danjou
2010-11-26 15:02       ` Stefan Monnier
2010-11-26 15:56         ` Julien Danjou
2010-11-26 18:42           ` Stefan Monnier
2010-11-27  9:36             ` Julien Danjou
2010-11-27 14:28               ` Stefan Monnier
2010-11-28  9:55               ` Lars Magne Ingebrigtsen
2010-11-26 14:10     ` Lars Magne Ingebrigtsen
2010-11-27 14:18       ` Lars Magne Ingebrigtsen
2010-11-27 14:40         ` Lars Magne Ingebrigtsen
2010-11-27 15:31           ` Lars Magne Ingebrigtsen
2010-11-27 16:04             ` Lars Magne Ingebrigtsen
2010-11-27 16:37               ` Steinar Bang
2010-11-27 16:41                 ` Lars Magne Ingebrigtsen
2010-11-27 16:59                   ` Lars Magne Ingebrigtsen
2010-11-27 17:33                     ` Dan Christensen
2010-11-27 17:36                       ` Lars Magne Ingebrigtsen
2010-11-27 17:42                         ` Lars Magne Ingebrigtsen
2010-11-28  2:36             ` Automatic STARTTLS upgrades (was: gnutls status) Lars Magne Ingebrigtsen
2010-11-28 12:28               ` Automatic STARTTLS upgrades Lars Magne Ingebrigtsen
2010-11-28 13:34                 ` Lars Magne Ingebrigtsen
2010-12-14 22:59       ` gnutls status Ted Zlatanov
2011-03-01 21:52         ` Ted Zlatanov
2011-03-05 11:01           ` Lars Magne Ingebrigtsen
2011-03-05 14:46             ` Ted Zlatanov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).