* gnutls status @ 2010-11-25 17:29 Julien Danjou 2010-11-26 0:28 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 28+ messages in thread From: Julien Danjou @ 2010-11-25 17:29 UTC (permalink / raw) To: ding [-- Attachment #1: Type: text/plain, Size: 233 bytes --] Hi, gnutls support has been added to Emacs 24. What's the status of it? Could we use it in Gnus? Or is this still an Emacs side problem to resolve? -- Julien Danjou // ᐰ <julien@danjou.info> http://julien.danjou.info [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-25 17:29 gnutls status Julien Danjou @ 2010-11-26 0:28 ` Lars Magne Ingebrigtsen 2010-11-26 12:13 ` Ted Zlatanov 0 siblings, 1 reply; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-26 0:28 UTC (permalink / raw) To: ding Julien Danjou <julien@danjou.info> writes: > gnutls support has been added to Emacs 24. What's the status of it? It seems to work. > Could we use it in Gnus? Or is this still an Emacs side problem to > resolve? nnimap has support for using it, but it's probably the wrong place to put the support. It should be put into tls.el, probably, so that all the users don't have to know about the stuff... -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-26 0:28 ` Lars Magne Ingebrigtsen @ 2010-11-26 12:13 ` Ted Zlatanov 2010-11-26 12:51 ` Julien Danjou 2010-11-26 14:10 ` Lars Magne Ingebrigtsen 0 siblings, 2 replies; 28+ messages in thread From: Ted Zlatanov @ 2010-11-26 12:13 UTC (permalink / raw) To: ding; +Cc: emacs-devel On Fri, 26 Nov 2010 01:28:46 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: LMI> Julien Danjou <julien@danjou.info> writes: >> gnutls support has been added to Emacs 24. What's the status of it? LMI> It seems to work. It's missing some features. See http://thread.gmane.org/gmane.emacs.devel/131441/focus=131551 and the rest of that thread for the details. The biggest one is callbacks. We need callbacks to implement host name verification and certificate chain checking, which are both IMO essential to making the Emacs GnuTLS support "official." In GnuTLS 2.8.x you can't set a callback function, so the C code would need (from doc/examples/ex-rfc2818.c): if (gnutls_x509_crt_get_expiration_time (cert) < time (0)) { printf ("The certificate has expired\n"); return; } if (gnutls_x509_crt_get_activation_time (cert) > time (0)) { printf ("The certificate is not yet activated\n"); return; } if (!gnutls_x509_crt_check_hostname (cert, hostname)) { printf ("The certificate's owner does not match hostname '%s'\n", hostname); return; } which is very inflexible compared to a callback function. We'd need to add custom API options for each of the three checks above plus another for the certificate chain verification; in addition it would be harder to interact with the user and store trusted certificates from C. 2.10.x and above let us set a callback function, which would make all of the above easier and more convenient from ELisp-land. The problem is that 2.10.x hasn't been widely adopted in Debian and thus won't work by default. So we'd need to either 1) require 2.10.x, or 2) complicate the C code and API using just 2.8.x features and maybe figure out how to set up our own callback mechanism, or 3) use the 2.10.x features only when it's available, using autoconf detection, which is twice as complicated as (2). (2) and (3) will require a lot of new code that is completely unnecessary under 2.10.x. This is essentially why I haven't worked on the GnuTLS support in a bit: I don't know the best way forward. If anyone can suggest a good way to do it, I'm all ears. I also don't know about 2.10.x's status in the major distros and whether Emacs can require that version or higher specifically. >> Could we use it in Gnus? Or is this still an Emacs side problem to >> resolve? LMI> nnimap has support for using it, but it's probably the wrong place to LMI> put the support. It should be put into tls.el, probably, so that all LMI> the users don't have to know about the stuff... I think every package should explicitly choose to support gnutls.el, it shouldn't be an Emacs-wide choice. There's too many configuration options that depend on the purpose. For instance IMAP and HTTPS have really different security needs. Also, GnuTLS is much more configurable than the older command-line interfaces so normalizing the API is not so easy. This is why I removed the old tls.el compatibility code from gnutls.el and it only has gnutls-* functions now (`open-gnutls-stream' and `gnutls-negotiate' being the main entry points from the ELisp side, while `gnutls-boot' is the main entry point from the C side). IMO tls.el and friends should be made deprecated as soon as gnutls.el is capable to verify a certificate chain, the expiration date, and the hostname. They are insecure and the cause of many bug reports over the years, especially on the W32 platform. As with the C issues above, if you have opinions or suggestions on how to do all this better, including Gnus but also for Emacs in general, tell me. I'm CC-ing to emacs-devel as well. Ted ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-26 12:13 ` Ted Zlatanov @ 2010-11-26 12:51 ` Julien Danjou 2010-11-26 15:02 ` Stefan Monnier 2010-11-26 14:10 ` Lars Magne Ingebrigtsen 1 sibling, 1 reply; 28+ messages in thread From: Julien Danjou @ 2010-11-26 12:51 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1977 bytes --] On Fri, Nov 26 2010, Ted Zlatanov wrote: > 2.10.x and above let us set a callback function, which would make all of > the above easier and more convenient from ELisp-land. The problem is > that 2.10.x hasn't been widely adopted in Debian and thus won't work by > default. It's in experimental. It won't go in sid until squeeze is released, which should be very soon now. So there's nothing to wait IMHO. At the time Emacs 24 will be released, gnutls 2.10 would have been since a very long in Debian. :) > This is essentially why I haven't worked on the GnuTLS support in a bit: > I don't know the best way forward. If anyone can suggest a good way to > do it, I'm all ears. I also don't know about 2.10.x's status in the > major distros and whether Emacs can require that version or higher > specifically. My opinion is go with 2.10. Some distro do not update often packages if they're is no urgent need to, anyhow. So you will just be pushing them a bit, which is not a bad thing. :) Gentoo, OpenBSD, FreeBSD already have it FWIW. And Debian in experimental, so it will end up in Debian and Ubuntu in a couple of month top[1]. This is the development model I used for the last years in various project like the awesome window manager, often requiring recent version of libraries it uses. I consider I don't have enough time to write compatibility code for older library release, while smart people write better interface for us to use. The only downside is for people using old OS for whatever reason. But they *will have* to upgrade other time anyhow, so why not doing sooner than later? Since we're talking a user program (not a production Web server), there's no big counter-argument to this, IMHO. Of course, YMMV, as your amount of spare time to be used writing compatibility code. ;) [1] Depends if squeeze is released soon, of course. -- Julien Danjou // ᐰ <julien@danjou.info> http://julien.danjou.info [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-26 12:51 ` Julien Danjou @ 2010-11-26 15:02 ` Stefan Monnier 2010-11-26 15:56 ` Julien Danjou 0 siblings, 1 reply; 28+ messages in thread From: Stefan Monnier @ 2010-11-26 15:02 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, emacs-devel >> 2.10.x and above let us set a callback function, which would make all of >> the above easier and more convenient from ELisp-land. The problem is >> that 2.10.x hasn't been widely adopted in Debian and thus won't work by >> default. > It's in experimental. It won't go in sid until squeeze is released, > which should be very soon now. So there's nothing to wait IMHO. > At the time Emacs 24 will be released, gnutls 2.10 would have been > since a very long in Debian. :) Many people use Debian stable, so until it's not in Debian stable, it's not really in Debian. Stefan ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-26 15:02 ` Stefan Monnier @ 2010-11-26 15:56 ` Julien Danjou 2010-11-26 18:42 ` Stefan Monnier 0 siblings, 1 reply; 28+ messages in thread From: Julien Danjou @ 2010-11-26 15:56 UTC (permalink / raw) To: Stefan Monnier; +Cc: Ted Zlatanov, ding, emacs-devel [-- Attachment #1: Type: text/plain, Size: 299 bytes --] On Fri, Nov 26 2010, Stefan Monnier wrote: > Many people use Debian stable, so until it's not in Debian stable, it's > not really in Debian. But does many people use Debian stable with emacs development version? -- Julien Danjou // ᐰ <julien@danjou.info> http://julien.danjou.info [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-26 15:56 ` Julien Danjou @ 2010-11-26 18:42 ` Stefan Monnier 2010-11-27 9:36 ` Julien Danjou 0 siblings, 1 reply; 28+ messages in thread From: Stefan Monnier @ 2010-11-26 18:42 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, emacs-devel >> Many people use Debian stable, so until it's not in Debian stable, it's >> not really in Debian. > But does many people use Debian stable with emacs development version? That's not the relevant question: the relevant question is whether 2.10 will be in Debian stable by the time Emacs-24 is released. It might, but I'm not completely sure. Stefan ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-26 18:42 ` Stefan Monnier @ 2010-11-27 9:36 ` Julien Danjou 2010-11-27 14:28 ` Stefan Monnier 2010-11-28 9:55 ` Lars Magne Ingebrigtsen 0 siblings, 2 replies; 28+ messages in thread From: Julien Danjou @ 2010-11-27 9:36 UTC (permalink / raw) To: Stefan Monnier; +Cc: Ted Zlatanov, ding, emacs-devel [-- Attachment #1: Type: text/plain, Size: 786 bytes --] On Fri, Nov 26 2010, Stefan Monnier wrote: > That's not the relevant question: the relevant question is whether 2.10 > will be in Debian stable by the time Emacs-24 is released. It might, > but I'm not completely sure. Depends on how you think about it. Emacs 24 will be probably released while Squeeze is stable, which does not have gnutls 2.10 (but may have it through backports). Next Debian stable would have Emacs 24, and gnutls 2.10. So gnutls 2.10 will not be in Debian stable when Emacs 24 get released (unless Emacs 24 is released after mid-2012). But most users won't try to install Emacs 24 on it anyway. OTOH Emacs 24 and gnutls 2.10 will be in the next Debian stable. :) -- Julien Danjou // ᐰ <julien@danjou.info> http://julien.danjou.info [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 9:36 ` Julien Danjou @ 2010-11-27 14:28 ` Stefan Monnier 2010-11-28 9:55 ` Lars Magne Ingebrigtsen 1 sibling, 0 replies; 28+ messages in thread From: Stefan Monnier @ 2010-11-27 14:28 UTC (permalink / raw) To: Ted Zlatanov; +Cc: ding, emacs-devel > So gnutls 2.10 will not be in Debian stable when Emacs 24 get released > (unless Emacs 24 is released after mid-2012). But most users won't try > to install Emacs 24 on it anyway. That's a good point. Especially since gnutls is not indispensable for Emacs, so as long as we properly handle the case where the gnutls library to too old (either by not using it, or by using some fallback code), then I'm OK with it. Stefan ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 9:36 ` Julien Danjou 2010-11-27 14:28 ` Stefan Monnier @ 2010-11-28 9:55 ` Lars Magne Ingebrigtsen 1 sibling, 0 replies; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-28 9:55 UTC (permalink / raw) To: emacs-devel; +Cc: ding Julien Danjou <julien@danjou.info> writes: > Next Debian stable would have Emacs 24, and gnutls 2.10. > > So gnutls 2.10 will not be in Debian stable when Emacs 24 get released > (unless Emacs 24 is released after mid-2012). But most users won't try > to install Emacs 24 on it anyway. True. And people who use Emacs 24 on older machines can use the external gnutls-cli/openssl solutions. They're slower and slightly less secure, but people have been living with that solution for decades, so they can do so a year more. :-) -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-26 12:13 ` Ted Zlatanov 2010-11-26 12:51 ` Julien Danjou @ 2010-11-26 14:10 ` Lars Magne Ingebrigtsen 2010-11-27 14:18 ` Lars Magne Ingebrigtsen 2010-12-14 22:59 ` gnutls status Ted Zlatanov 1 sibling, 2 replies; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-26 14:10 UTC (permalink / raw) To: ding; +Cc: emacs-devel Ted Zlatanov <tzz@lifelogs.com> writes: > So we'd need to either 1) require 2.10.x, or 2) complicate the C code > and API using just 2.8.x features and maybe figure out how to set up our > own callback mechanism, or 3) use the 2.10.x features only when it's > available, using autoconf detection, which is twice as complicated as > (2). Is 2.10.x at least backwards-compatible, so that if we do implement the complicated 2.8.x features, it'll continue to work in the future, too? > I think every package should explicitly choose to support gnutls.el, it > shouldn't be an Emacs-wide choice. There's too many configuration > options that depend on the purpose. For instance IMAP and HTTPS have > really different security needs. True. On the other hand, look at `nnimap-open-connection' and weep. It started off as a simple function and grew into a monster. That's why I haven't adapted nntp/pop3/etc to use gnutls yet -- I don't want to have to repeat the same code all over the place. And the STARTTLS situation, in particular, is a nightmare, since the sane gnutls way of doing it and the really awkward starttls.el way of doing it require different code paths. And we're going to have to support gnutls/tls.el/starttls.el in all the connections for the unforeseeable future. I kinda want to write a new connection framework that'll just separate out all this cruft into a separate package, but I can't really see a sensible unified interface. Especially with the STARTTLS stuff. I mean, ideally, in the future whenever you do a pop3/imap/nntp/smtp/etc connection, Emacs should opportunistically switch on STARTTLS if the server supports it. Encryption is good. Switching on STARTTLS is virtually free if you have gnutls. It's really expensive if you don't. I mean, time-wise for the user. But I don't really see an easy way to do that. You'd need protocol-specific callbacks and stuff. And I hate frameworks. And it'll be brittle, anyway, since starttls.el is kinda brittle. But I do think we need some kinda of compat library to avoid going totally insane. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-26 14:10 ` Lars Magne Ingebrigtsen @ 2010-11-27 14:18 ` Lars Magne Ingebrigtsen 2010-11-27 14:40 ` Lars Magne Ingebrigtsen 2010-12-14 22:59 ` gnutls status Ted Zlatanov 1 sibling, 1 reply; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-27 14:18 UTC (permalink / raw) To: ding Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > But I don't really see an easy way to do that. You'd need > protocol-specific callbacks and stuff. And I hate frameworks. And > it'll be brittle, anyway, since starttls.el is kinda brittle. I've gone ahead and done so, anyway. nnimap got smaller, and I think it should be easy enough to adapt it to NNTP, SMTP and POP3, but, so far, my only use case is IMAP, so I would guess that I haven't covered all bases yet. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 14:18 ` Lars Magne Ingebrigtsen @ 2010-11-27 14:40 ` Lars Magne Ingebrigtsen 2010-11-27 15:31 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-27 14:40 UTC (permalink / raw) To: ding Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > I've gone ahead and done so, anyway. nnimap got smaller, and I think it > should be easy enough to adapt it to NNTP, SMTP and POP3, but, so far, > my only use case is IMAP, so I would guess that I haven't covered all > bases yet. I've now pushed this for nnimap, and it Works For Me, but it needs more testing. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 14:40 ` Lars Magne Ingebrigtsen @ 2010-11-27 15:31 ` Lars Magne Ingebrigtsen 2010-11-27 16:04 ` Lars Magne Ingebrigtsen 2010-11-28 2:36 ` Automatic STARTTLS upgrades (was: gnutls status) Lars Magne Ingebrigtsen 0 siblings, 2 replies; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-27 15:31 UTC (permalink / raw) To: ding Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > I've now pushed this for nnimap, and it Works For Me, but it needs more > testing. What the hey. I've now altered NNTP to use proto-stream, and it'll switch on STARTTLS now if the server supports it. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 15:31 ` Lars Magne Ingebrigtsen @ 2010-11-27 16:04 ` Lars Magne Ingebrigtsen 2010-11-27 16:37 ` Steinar Bang 2010-11-28 2:36 ` Automatic STARTTLS upgrades (was: gnutls status) Lars Magne Ingebrigtsen 1 sibling, 1 reply; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-27 16:04 UTC (permalink / raw) To: ding Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > What the hey. I've now altered NNTP to use proto-stream, and it'll > switch on STARTTLS now if the server supports it. I don't have access to any STARTTLS NNTP servers, so please give it a whirl. I've now also tested this stuff with a non-gnutls-enabled Emacs, and after some further hacking away, it seems to work for me with nnimap + tls and starttls. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 16:04 ` Lars Magne Ingebrigtsen @ 2010-11-27 16:37 ` Steinar Bang 2010-11-27 16:41 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 28+ messages in thread From: Steinar Bang @ 2010-11-27 16:37 UTC (permalink / raw) To: ding >>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>: > I don't have access to any STARTTLS NNTP servers,... You can't spin up that functionality on the gmane INN? ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 16:37 ` Steinar Bang @ 2010-11-27 16:41 ` Lars Magne Ingebrigtsen 2010-11-27 16:59 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-27 16:41 UTC (permalink / raw) To: ding Steinar Bang <sb@dod.no> writes: > You can't spin up that functionality on the gmane INN? I don't know whether it supports it? -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 16:41 ` Lars Magne Ingebrigtsen @ 2010-11-27 16:59 ` Lars Magne Ingebrigtsen 2010-11-27 17:33 ` Dan Christensen 0 siblings, 1 reply; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-27 16:59 UTC (permalink / raw) To: ding Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > I don't know whether it supports it? It apparently does. I've now recompiled inn with openssl support, but I haven't put it into production yet. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 16:59 ` Lars Magne Ingebrigtsen @ 2010-11-27 17:33 ` Dan Christensen 2010-11-27 17:36 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 28+ messages in thread From: Dan Christensen @ 2010-11-27 17:33 UTC (permalink / raw) To: ding Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > It apparently does. I've now recompiled inn with openssl support, but I > haven't put it into production yet. Is that going to add overhead to gmane connections (and add load to the server)? I don't really need my gmane connections encrypted... Dan ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 17:33 ` Dan Christensen @ 2010-11-27 17:36 ` Lars Magne Ingebrigtsen 2010-11-27 17:42 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-27 17:36 UTC (permalink / raw) To: ding Dan Christensen <jdc@uwo.ca> writes: > Is that going to add overhead to gmane connections (and add load to the > server)? I don't really need my gmane connections encrypted... Well, setting up the connection is going to be marginally slower, since there are a few additional round trips added. The added load on the server is pretty marginal. SSL doesn't take much CPU power. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-27 17:36 ` Lars Magne Ingebrigtsen @ 2010-11-27 17:42 ` Lars Magne Ingebrigtsen 0 siblings, 0 replies; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-27 17:42 UTC (permalink / raw) To: ding Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > Well, setting up the connection is going to be marginally slower, since > there are a few additional round trips added. The added load on the > server is pretty marginal. SSL doesn't take much CPU power. But I've added a new variable for people who want to disable this: `proto-stream-always-use-starttls'. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Automatic STARTTLS upgrades (was: gnutls status) 2010-11-27 15:31 ` Lars Magne Ingebrigtsen 2010-11-27 16:04 ` Lars Magne Ingebrigtsen @ 2010-11-28 2:36 ` Lars Magne Ingebrigtsen 2010-11-28 12:28 ` Automatic STARTTLS upgrades Lars Magne Ingebrigtsen 1 sibling, 1 reply; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-28 2:36 UTC (permalink / raw) To: ding Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > What the hey. I've now altered NNTP to use proto-stream, and it'll > switch on STARTTLS now if the server supports it. But I'm wondering about the defaults a bit. This could be brittle and slow if things don't actually work. So let's take the worst-case scenarios: * First we open a normal network connection, and we say "CAPABILITY" in one way or another. This should be safe, but it's one extra round trip. * If the server says that it supports STARTTLS, we enable either: ** If Emacs has gnutls support, we just enable STARTTLS. One extra round trip. ** If Emacs doesn't, we sever the connection, and start it again with starttls.el, which defaults to gnutls-cli. This may fail because of a faulty gnutls-cli installation, in which case we error out. But it it's successful, then it's a close, an open, and an STARTTLS enablement. So that's at least a ... big handful of extra round trips. * We then do the TLS negotiation. This is at least one round trip, but it's probably more. It may fail for any number of reasons. For instance, inn may be compiled with TLS support, but not have the certificates. Or they may be grossly invalid. In that case, we error out. * On success, we do CAPABILITY again, since they have a tendency to change after STARTTLS in some protocols. * We're done! So, as you can see, this introduces many new, interesting failure modes, which have to be mitigated somehow. First of all, I've now switched off opportunistic STARTTLS upgrades if we don't have an Emacs with built-in gnutls support. Secondly, I think if the STARTTLS stuff fails for any reason, the connection should go back to being a plain network connection. Is that possible with the built-in gnutls stuff, Ted? Otherwise, proto-stream has to reopen the connection... -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: Automatic STARTTLS upgrades 2010-11-28 2:36 ` Automatic STARTTLS upgrades (was: gnutls status) Lars Magne Ingebrigtsen @ 2010-11-28 12:28 ` Lars Magne Ingebrigtsen 2010-11-28 13:34 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-28 12:28 UTC (permalink / raw) To: ding Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > * We then do the TLS negotiation. This is at least one round trip, but > it's probably more. It may fail for any number of reasons. For > instance, inn may be compiled with TLS support, but not have the > certificates. Or they may be grossly invalid. In that case, we error > out. I've now fixed this. If the server announces STARTTLS, but gives an error when we try to enable it, the connection will just stay unencrypted. This works both with the built-in gnutls support and the starttls.el support. (Although in the latter case, the stream will still pass through gnutls-cli, but unencrypted. So it'll be marginally slower, but it shouldn't matter much.) -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: Automatic STARTTLS upgrades 2010-11-28 12:28 ` Automatic STARTTLS upgrades Lars Magne Ingebrigtsen @ 2010-11-28 13:34 ` Lars Magne Ingebrigtsen 0 siblings, 0 replies; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2010-11-28 13:34 UTC (permalink / raw) To: ding I guess I could convert pop3.el to using proto-stream.el for opportunistic STARTTLS upgrades, but I should probably wait a bit and see how well this works with IMAP and NNTP first, I guess... :-) -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-11-26 14:10 ` Lars Magne Ingebrigtsen 2010-11-27 14:18 ` Lars Magne Ingebrigtsen @ 2010-12-14 22:59 ` Ted Zlatanov 2011-03-01 21:52 ` Ted Zlatanov 1 sibling, 1 reply; 28+ messages in thread From: Ted Zlatanov @ 2010-12-14 22:59 UTC (permalink / raw) To: ding; +Cc: emacs-devel On Fri, 26 Nov 2010 15:10:39 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: LMI> Is 2.10.x at least backwards-compatible, so that if we do implement the LMI> complicated 2.8.x features, it'll continue to work in the future, too? Yes. They try really hard to keep backwards compatibility. I'd guess for all 2.x releases we'll be OK unless there's newer features we simply must have :) >> I think every package should explicitly choose to support gnutls.el, it >> shouldn't be an Emacs-wide choice. There's too many configuration >> options that depend on the purpose. For instance IMAP and HTTPS have >> really different security needs. LMI> True. On the other hand, look at `nnimap-open-connection' and weep. It LMI> started off as a simple function and grew into a monster. That's why I LMI> haven't adapted nntp/pop3/etc to use gnutls yet -- I don't want to have LMI> to repeat the same code all over the place. ...thanks for working on proto-stream.el, it's made all this much simpler. On Fri, 26 Nov 2010 13:51:09 +0100 Julien Danjou <julien@danjou.info> wrote: JD> My opinion is go with 2.10. On Sat, 27 Nov 2010 09:28:11 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: >> So gnutls 2.10 will not be in Debian stable when Emacs 24 get released >> (unless Emacs 24 is released after mid-2012). But most users won't try >> to install Emacs 24 on it anyway. SM> That's a good point. Especially since gnutls is not indispensable for SM> Emacs, so as long as we properly handle the case where the gnutls library SM> to too old (either by not using it, or by using some fallback code), SM> then I'm OK with it. OK, I will put upgrading Emacs' support of GnuTLS to 2.10 on my TODO list. Which means it won't happen soon, so if anyone else is feeling frisky, go ahead and do it. I listed what's necessary in my original e-mail here; basically it's callbacks but there's other features in the 2.10 API we should use if possible. I think that supporting 2.8 is counterproductive so I'll happily target only 2.10. I think that's reasonable and doesn't put a big burden on the users and distros. Ted ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2010-12-14 22:59 ` gnutls status Ted Zlatanov @ 2011-03-01 21:52 ` Ted Zlatanov 2011-03-05 11:01 ` Lars Magne Ingebrigtsen 0 siblings, 1 reply; 28+ messages in thread From: Ted Zlatanov @ 2011-03-01 21:52 UTC (permalink / raw) To: ding; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1682 bytes --] On Tue, 14 Dec 2010 16:59:34 -0600 Ted Zlatanov <tzz@lifelogs.com> wrote: TZ> On Fri, 26 Nov 2010 15:10:39 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: LMI> Is 2.10.x at least backwards-compatible, so that if we do implement the LMI> complicated 2.8.x features, it'll continue to work in the future, too? TZ> Yes. They try really hard to keep backwards compatibility. I'd guess TZ> for all 2.x releases we'll be OK unless there's newer features we simply TZ> must have :) Argh, GnuTLS 2.8.x is still standard on Ubuntu 10.10, so practically we should support it. Below is my first (untested) patch to generate the HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE and HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY definitions in the configure.in using AC_CHECK_FUNCS and then use them (currently just #ifdef placeholders) in gnutls.c. I plan to retrieve them from the :callbacks alist parameter to `gnutls-boot'. Regenerating "configure" failed for me. I get this error at the end: ./configure: line 12620: gl_ASSERT_NO_GNULIB_POSIXCHECK: command not found ./configure: line 12621: gl_ASSERT_NO_GNULIB_TESTS: command not found ./configure: line 12622: gl_INIT: command not found checking for lstat... yes ./configure: line 12648: syntax error near unexpected token `lstat' ./configure: line 12648: `gl_SYS_STAT_MODULE_INDICATOR(lstat)' at the end. But it gets far enough that I can tell the tests are being run. This is why the patch is untested; I'll see if I can figure out why that's happening. It may be an Ubuntu oddity. Please let me know if the proposed approach is reasonable and if you have any comments. In theory this should be pretty trivial. Thanks Ted [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: callbacks.patch --] [-- Type: text/x-diff, Size: 2684 bytes --] === modified file 'configure.in' --- configure.in 2011-02-24 04:28:17 +0000 +++ configure.in 2011-03-01 21:39:23 +0000 @@ -1972,12 +1972,26 @@ AC_SUBST(LIBSELINUX_LIBS) HAVE_GNUTLS=no +HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY=no +HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE=no if test "${with_gnutls}" = "yes" ; then PKG_CHECK_MODULES([LIBGNUTLS], [gnutls >= 2.2.4], HAVE_GNUTLS=yes, HAVE_GNUTLS=no) if test "${HAVE_GNUTLS}" = "yes"; then AC_DEFINE(HAVE_GNUTLS, 1, [Define if using GnuTLS.]) fi + + AC_CHECK_FUNCS(gnutls_certificate_set_verify_function, HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY=yes) + AC_CHECK_FUNCS(gnutls_certificate_client_set_retrieve_function, HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE=yes) + + if test "${HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE}" = "yes"; then + AC_DEFINE(HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE, 1, [Define if using GnuTLS certificate retrieval callbacks.]) + fi + + if test "${HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY}" = "yes"; then + AC_DEFINE(HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY, 1, [Define if using GnuTLS certificate verification callbacks.]) + fi fi + AC_SUBST(LIBGNUTLS_LIBS) AC_SUBST(LIBGNUTLS_CFLAGS) @@ -3667,6 +3681,8 @@ echo " Does Emacs use -lgconf? ${HAVE_GCONF}" echo " Does Emacs use -lselinux? ${HAVE_LIBSELINUX}" echo " Does Emacs use -lgnutls? ${HAVE_GNUTLS}" +echo " Does Emacs use -lgnutls certificate verify callbacks? ${HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY}" +echo " Does Emacs use -lgnutls certificate retrieve callbacks? ${HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE}" echo " Does Emacs use -lxml2? ${HAVE_LIBXML2}" echo " Does Emacs use -lfreetype? ${HAVE_FREETYPE}" === modified file 'src/gnutls.c' --- src/gnutls.c 2011-01-25 04:08:28 +0000 +++ src/gnutls.c 2011-03-01 21:41:36 +0000 @@ -484,6 +484,16 @@ GNUTLS_INITSTAGE (proc) = GNUTLS_STAGE_FILES; + GNUTLS_LOG (1, max_log_level, "gnutls callbacks"); + + GNUTLS_INITSTAGE (proc) = GNUTLS_STAGE_CALLBACKS; + +#ifdef HAVE_GNUTLS_CALLBACK_CERTIFICATE_VERIFY +#endif + +#ifdef HAVE_GNUTLS_CALLBACK_CERTIFICATE_RETRIEVE +#endif + GNUTLS_LOG (1, max_log_level, "gnutls_init"); ret = gnutls_init (&state, GNUTLS_CLIENT); === modified file 'src/gnutls.h' --- src/gnutls.h 2011-01-25 04:08:28 +0000 +++ src/gnutls.h 2011-03-01 21:32:17 +0000 @@ -28,6 +28,7 @@ GNUTLS_STAGE_EMPTY = 0, GNUTLS_STAGE_CRED_ALLOC, GNUTLS_STAGE_FILES, + GNUTLS_STAGE_CALLBACKS, GNUTLS_STAGE_INIT, GNUTLS_STAGE_PRIORITY, GNUTLS_STAGE_CRED_SET, ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2011-03-01 21:52 ` Ted Zlatanov @ 2011-03-05 11:01 ` Lars Magne Ingebrigtsen 2011-03-05 14:46 ` Ted Zlatanov 0 siblings, 1 reply; 28+ messages in thread From: Lars Magne Ingebrigtsen @ 2011-03-05 11:01 UTC (permalink / raw) To: ding; +Cc: emacs-devel Ted Zlatanov <tzz@lifelogs.com> writes: > Regenerating "configure" failed for me. I get this error at the end: > > ./configure: line 12620: gl_ASSERT_NO_GNULIB_POSIXCHECK: command not found > ./configure: line 12621: gl_ASSERT_NO_GNULIB_TESTS: command not found > ./configure: line 12622: gl_INIT: command not found > checking for lstat... yes > ./configure: line 12648: syntax error near unexpected token `lstat' > ./configure: line 12648: `gl_SYS_STAT_MODULE_INDICATOR(lstat)' I tried applying your patch to configure.in, and it worked for me. Here's the output from around the lstat check: checking for working GNU getopt function... yes checking whether getenv is declared... yes checking for lstat... yes checking for alarm... yes -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: gnutls status 2011-03-05 11:01 ` Lars Magne Ingebrigtsen @ 2011-03-05 14:46 ` Ted Zlatanov 0 siblings, 0 replies; 28+ messages in thread From: Ted Zlatanov @ 2011-03-05 14:46 UTC (permalink / raw) To: ding; +Cc: emacs-devel On Sat, 05 Mar 2011 12:01:39 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: LMI> Ted Zlatanov <tzz@lifelogs.com> writes: >> Regenerating "configure" failed for me. I get this error at the end: >> >> ./configure: line 12620: gl_ASSERT_NO_GNULIB_POSIXCHECK: command not found >> ./configure: line 12621: gl_ASSERT_NO_GNULIB_TESTS: command not found >> ./configure: line 12622: gl_INIT: command not found >> checking for lstat... yes >> ./configure: line 12648: syntax error near unexpected token `lstat' >> ./configure: line 12648: `gl_SYS_STAT_MODULE_INDICATOR(lstat)' LMI> I tried applying your patch to configure.in, and it worked for me. LMI> Here's the output from around the lstat check: LMI> checking for working GNU getopt function... yes LMI> checking whether getenv is declared... yes LMI> checking for lstat... yes LMI> checking for alarm... yes Something is broken in my autoconf setup at home, because the exact same configure.in worked on another machine (same Ubuntu 10.10 setup!) yesterday. Weird. Anyhow, since GnuTLS 2.10 is not available even in Ubuntu 10.10, I'm working on a way to control the verification with the GnuTLS verification flags. It's not as good as a verification callback but it will be more useful for the vast majority of people right now. I hope to have something ready this weekend. Ted ^ permalink raw reply [flat|nested] 28+ messages in thread
end of thread, other threads:[~2011-03-05 14:46 UTC | newest] Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-11-25 17:29 gnutls status Julien Danjou 2010-11-26 0:28 ` Lars Magne Ingebrigtsen 2010-11-26 12:13 ` Ted Zlatanov 2010-11-26 12:51 ` Julien Danjou 2010-11-26 15:02 ` Stefan Monnier 2010-11-26 15:56 ` Julien Danjou 2010-11-26 18:42 ` Stefan Monnier 2010-11-27 9:36 ` Julien Danjou 2010-11-27 14:28 ` Stefan Monnier 2010-11-28 9:55 ` Lars Magne Ingebrigtsen 2010-11-26 14:10 ` Lars Magne Ingebrigtsen 2010-11-27 14:18 ` Lars Magne Ingebrigtsen 2010-11-27 14:40 ` Lars Magne Ingebrigtsen 2010-11-27 15:31 ` Lars Magne Ingebrigtsen 2010-11-27 16:04 ` Lars Magne Ingebrigtsen 2010-11-27 16:37 ` Steinar Bang 2010-11-27 16:41 ` Lars Magne Ingebrigtsen 2010-11-27 16:59 ` Lars Magne Ingebrigtsen 2010-11-27 17:33 ` Dan Christensen 2010-11-27 17:36 ` Lars Magne Ingebrigtsen 2010-11-27 17:42 ` Lars Magne Ingebrigtsen 2010-11-28 2:36 ` Automatic STARTTLS upgrades (was: gnutls status) Lars Magne Ingebrigtsen 2010-11-28 12:28 ` Automatic STARTTLS upgrades Lars Magne Ingebrigtsen 2010-11-28 13:34 ` Lars Magne Ingebrigtsen 2010-12-14 22:59 ` gnutls status Ted Zlatanov 2011-03-01 21:52 ` Ted Zlatanov 2011-03-05 11:01 ` Lars Magne Ingebrigtsen 2011-03-05 14:46 ` Ted Zlatanov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).