Aloha, I'm sure many of you received Google's email this morning announcing an eventual end to non-oauth access to gmail. There are various dates starting in February 2020 and extending into 2021 depending on the situation, but this will pose a problem for those of us who rely on directly sending/receiving with gmail via gnus. Note I'm not talking about offline/download solutions but direct access through gnus. There is gnus-gmail-oauth.el on github, almost four years old. I haven't tried it and it may not work for both sending and receiving. Perhaps there are other things. Does anyone know more about this and about what the best course of action might be? We have a little time, but not necessarily tons of it. Mahalo, -- Bob Newell Honolulu, Hawai`i - Via Gnus/BBDB/Org/Emacs/Linux
* Bob Newell:
> I'm sure many of you received Google's email this morning
> announcing an eventual end to non-oauth access to gmail. There
> are various dates starting in February 2020 and extending into
> 2021 depending on the situation, but this will pose a problem
> for those of us who rely on directly sending/receiving with
> gmail via gnus.
Wow. Does this mean that support for device passwords will be removed
completely?
Florian Weimer <fw@deneb.enyo.de> writes:
> Wow. Does this mean that support for device passwords will be removed
> completely?
The way I read it, yes, at least if I understand your
question. Email clients on devices will have to use oauth.
--
Bob Newell
Honolulu, Hawai`i
- Via Gnus/BBDB/Org/Emacs/Linux
By the way I accidently reversed dates: warnings start in June 2020 and everything ends Feb 2021. -- Bob Newell Honolulu, Hawai`i - Via Gnus/BBDB/Org/Emacs/Linux
That being the case, I'll have to learn how to have google forward all
my gmail to a different email account permanently or until google
discontinues that forwarding service too.
On Mon, 16 Dec 2019, Bob Newell wrote:
> Date: Mon, 16 Dec 2019 17:45:06
> From: Bob Newell <bobnewell@bobnewell.net>
> To: Florian Weimer <fw@deneb.enyo.de>, ding@gnus.org
> Subject: Re: oauth to be required for gmail
>
> Florian Weimer <fw@deneb.enyo.de> writes:
>
> > Wow. Does this mean that support for device passwords will be removed
> > completely?
>
> The way I read it, yes, at least if I understand your
> question. Email clients on devices will have to use oauth.
>
>
--
>> I'm sure many of you received Google's email this morning
>> announcing an eventual end to non-oauth access to gmail. There
>> are various dates starting in February 2020 and extending into
>> 2021 depending on the situation, but this will pose a problem
>> for those of us who rely on directly sending/receiving with
>> gmail via gnus.
>
> Wow. Does this mean that support for device passwords will be removed
> completely?
Yes. We must implement oauth now. I have done it in other apps. But I
have not much experience with Emacs Lisp. I am willing to do with some
guidance from leaders here.
>>>>> On Mon, 16 Dec 2019 09:48:58 -1000, Bob Newell <bobnewell@bobnewell.net> said: Bob> Aloha, Bob> I'm sure many of you received Google's email this morning Bob> announcing an eventual end to non-oauth access to gmail. There Bob> are various dates starting in February 2020 and extending into Bob> 2021 depending on the situation, but this will pose a problem Bob> for those of us who rely on directly sending/receiving with Bob> gmail via gnus. Bob> Note I'm not talking about offline/download solutions but Bob> direct access through gnus. Bob> There is gnus-gmail-oauth.el on github, almost four years Bob> old. I haven't tried it and it may not work for both sending Bob> and receiving. Perhaps there are other things. I think the canonical package for doing this is <https://github.com/ccrusius/auth-source-xoauth2>, although I haven't tried it yet. Robert
Robert Pluim <rpluim@gmail.com> writes: > I think the canonical package for doing this is > <https://github.com/ccrusius/auth-source-xoauth2>, although I haven't > tried it yet. It's comically convoluted to use oauth for open source software: https://github.com/ccrusius/auth-source-xoauth2/blob/8cef77b0d390f8a45e7690eaf8772173e1995a9b/auth-source-xoauth2.el#L79 Here's a recipe for Mutt, which one Hackernews described as "you just have to run a script": https://luxing.im/mutt-integration-with-gmail-using-oauth/ Now, what closed source software does is do all the first steps for you: Register an app on Google, get a secret key etc, so all the user has to do it go through a web-based login thing. I don't think this is much of an option for Gnus? Gnus Inc registering an app that's allowed to access Gmail accounts? I'd have to go through a security audit process (typically costing $50K, if I understand correctly) and, still, it's pretty much impossible as the secret key would have to be distributed some way, wouldn't it? Or... is that a solved problem? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no
>>>>>> On Mon, 16 Dec 2019 09:48:58 -1000, Bob Newell <bobnewell@bobnewell.net> said:
>
> Bob> Aloha,
> Bob> I'm sure many of you received Google's email this morning
> Bob> announcing an eventual end to non-oauth access to gmail. There
> Bob> are various dates starting in February 2020 and extending into
> Bob> 2021 depending on the situation, but this will pose a problem
> Bob> for those of us who rely on directly sending/receiving with
> Bob> gmail via gnus.
>
> Bob> Note I'm not talking about offline/download solutions but
> Bob> direct access through gnus.
>
> Bob> There is gnus-gmail-oauth.el on github, almost four years
> Bob> old. I haven't tried it and it may not work for both sending
> Bob> and receiving. Perhaps there are other things.
>
> I think the canonical package for doing this is
> <https://github.com/ccrusius/auth-source-xoauth2>, although I haven't
> tried it yet.
There is the 'oauth2' package in ELPA. I use it for org-caldav and it
works fine.
The problem with OAuth2 is not the technical side, which is pretty easy
to do. The problem is that OAuth2 allows the serving side to control
which application may access their services. They may forbid it
entirely, or they may limit the API access depending on the application.
They can do this since you need to register your application (in this
case with Google), and you get a "client ID" and "client secret" with
which your application identifies itself. AFAIK, for accessing GMail,
you even need to go through an additional verification process to get
full access.
Of course, the "client secret" is pointless if you openly put it into
your source, so there is no way to register an "official" client
ID/secret for Gnus which anyone could use. Last I checked, publishing
the client secret would be considered a violation of Google's services,
with all consequences this may imply (terminating the developer's
account, etc.). So usually, each user needs to register the application
separately; this works for org-caldav, since the CalDAV API does not
require additional verification, but this may very well be different for
the Mail API. It's been some time since I dealt with this, so maybe
things have gotten better in the meantime.
-David
> Robert Pluim <rpluim@gmail.com> writes: > >> I think the canonical package for doing this is >> <https://github.com/ccrusius/auth-source-xoauth2>, although I haven't >> tried it yet. > > It's comically convoluted to use oauth for open source software: > > https://github.com/ccrusius/auth-source-xoauth2/blob/8cef77b0d390f8a45e7690eaf8772173e1995a9b/auth-source-xoauth2.el#L79 > > Here's a recipe for Mutt, which one Hackernews described as "you just > have to run a script": > > https://luxing.im/mutt-integration-with-gmail-using-oauth/ > > Now, what closed source software does is do all the first steps for you: > Register an app on Google, get a secret key etc, so all the user has to > do it go through a web-based login thing. > > I don't think this is much of an option for Gnus? Gnus Inc registering > an app that's allowed to access Gmail accounts? I'd have to go through > a security audit process (typically costing $50K, if I understand > correctly) and, still, it's pretty much impossible as the secret key > would have to be distributed some way, wouldn't it? Or... is that a > solved problem? I don't think so, see my other mail I've just written. See also this answer here, which I think is still the current situation: https://stackoverflow.com/a/28109307 -David
Some further info: like it or not this appears to be the way big commercial email services are going. AOL is ending "less secure" logins in a couple of months, and they don't currently publish a method to get oauth2 tokens. (Okay, it's AOL, but it's an example of the trend.) Yahoo, who owns AOL, hasn't announced anything that I can find, but I imagine they'll go the same way too. I really think that gnus will have to support oauth natively in order to keep up. Not necessarily a pleasant thought, and yes, there are current workarounds as described in this thread, but ... gnus is not alone. Other mail clients and mail fetchers will have to follow suit. Doggone it, I thought it was a sort of cool irony to access AOL mail via gnus. This will leave a big hole in my life :)
> Yes. We must implement oauth now. I have done it in other apps. But I
> have not much experience with Emacs Lisp. I am willing to do with some
> guidance from leaders here.
Plus we need to documents for oauth2. That seems not easy to
understand...
Thanks for shareing^^^
Sincerely,
--
^고맙습니다 _地平天成_ 감사합니다_^))//
> I don't think this is much of an option for Gnus? Gnus Inc registering > an app that's allowed to access Gmail accounts? I'd have to go through > a security audit process (typically costing $50K, if I understand > correctly) and, still, it's pretty much impossible as the secret key > would have to be distributed some way, wouldn't it? Or... is that a > solved problem? So I wondered: How does Thunderbird does it? Oh, there are the ID's and secrets: https://dxr.mozilla.org/comm-central/source/comm/mailnews/base/util/OAuth2Providers.jsm But it seems if you put a comment above it which says "Don't copy these values for your own application--register it yourself", then it's fine. This whole OAuth2 stuff is ridiculous. -David
* 황병희: >> Yes. We must implement oauth now. I have done it in other apps. But I >> have not much experience with Emacs Lisp. I am willing to do with some >> guidance from leaders here. > > Plus we need to documents for oauth2. That seems not easy to > understand... There is this: <https://developers.google.com/gmail/imap/xoauth2-protocol> <https://github.com/google/gmail-oauth2-tools/wiki/OAuth2DotPyRunThrough> But I can't see this working for people who use a domain account where they are not administrator. At the very least, it requires agreement to separate service terms, which is typically not allowed by policy.
Hellow Florian^^^
Florian Weimer <fw@deneb.enyo.de> writes:
> * 황병희:
>
>>> Yes. We must implement oauth now. I have done it in other apps. But I
>>> have not much experience with Emacs Lisp. I am willing to do with some
>>> guidance from leaders here.
>>
>> Plus we need to documents for oauth2. That seems not easy to
>> understand...
>
> There is this:
>
> <https://developers.google.com/gmail/imap/xoauth2-protocol>
> <https://github.com/google/gmail-oauth2-tools/wiki/OAuth2DotPyRunThrough>
>
> But I can't see this working for people who use a domain account where
> they are not administrator. At the very least, it requires agreement
> to separate service terms, which is typically not allowed by policy.
Wow your kind comments are very useful to me. Also i added the urls to
firefox's bookmark, thanks^^^
Sincerely,
--
^고맙습니다 _地平天成_ 감사합니다_^))//
* David Engster:
> I don't think so, see my other mail I've just written. See also this
> answer here, which I think is still the current situation:
>
> https://stackoverflow.com/a/28109307
In this case, it's probably best to pick a widely used secret, so that
it is pretty much impossible to blacklist it for Google.
* David Engster:
> So I wondered: How does Thunderbird does it?
>
> Oh, there are the ID's and secrets:
>
> https://dxr.mozilla.org/comm-central/source/comm/mailnews/base/util/OAuth2Providers.jsm
>
> But it seems if you put a comment above it which says "Don't copy these
> values for your own application--register it yourself", then it's
> fine.
>
> This whole OAuth2 stuff is ridiculous.
Isn't there a different mode for Thunderbird, which performs a regular
web login and can also support third-party authentication for
enterprise accounts? Admittedly, it's been a year or two since I
tried this.
Basically, what seems to happen is that Thunderbird sees the OAuth2
request in the IMAP handshake, starts its internal web browser,
renders the Google login page. With an enterprise domain, Google then
automatically redirects to the external authentication source (based
on its preconfigured records), which can do any authentication it
wants (e.g., use Kerberos, so that the user doesn't even have to enter
a password), and then redirects back to Google, at which point Google
serves something back via the web browser which can be used to
complete the IMAP handshake.
My point is that it is pretty much impossible to complete that
sequence without a complete, Javascript-enabled web browser. But that
mode, while ridiculously complex, still isn't as pointless as the
approach with static password that is not actually secret and thus
does not serve any purpose at all.
> My point is that it is pretty much impossible to complete that
> sequence without a complete, Javascript-enabled web browser. But that
> mode, while ridiculously complex, still isn't as pointless as the
> approach with static password that is not actually secret and thus
> does not serve any purpose at all.
The real solution would be if Google supported RFC 7591 and RFC
7628. This is also what the comment above the hard-coded credentials in
Thunderbird says. I predict this will never happen because the providers
are actually quite happy with the current situation. It allows them to
control which applications may access their services, and it pushes more
users away from using third-party desktop mail client. Remember that
Google wants you to use the web while being logged in, since this makes
it much easier for them to track you.
I think the "solution" is to tell users they need to set those
credentials themselves. There are no widespred OAuth credentials which
we could simply use without being shady. The real solution of course is
that users simply stop using Google Mail and choose a proper mail
provider.
-David
> The real solution of course is
> that users simply stop using Google Mail and choose a proper mail
> provider.
>
> -David
In the end I'll likely just set up the oauth credentials and put in
the necessary software changes, but I have serious considered your
"real" solution. There are a couple of problems, though. I have a lot
of gmail-specific coding (perhaps a bad decision on my part) and a lot
of stuff in the gmail ecosystem. But my fear is that, should I change
to another provider --- I don't know, FastMail or someone--- they
might in time go the same route. Eliminating "less secure" login
methods certainly seems to be trending.
--
Bob Newell
Honolulu, Hawai`i
Via Linux/Emacs/Gnus/BBDB.
Bob Newell <bobnewell@bobnewell.net> writes: > But my fear is that, should I change to another provider --- I don't > know, FastMail or someone--- they might in time go the same > route. Eliminating "less secure" login methods certainly seems to be > trending. Yup. I think I read somewhere that Microsoft is disabling user-name/password auth on their IMAP offerings, too. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no
>>>>> Lars Ingebrigtsen <larsi@gnus.org>: > Bob Newell <bobnewell@bobnewell.net> writes: >> But my fear is that, should I change to another provider --- I don't >> know, FastMail or someone--- they might in time go the same >> route. Eliminating "less secure" login methods certainly seems to be >> trending. > Yup. I think I read somewhere that Microsoft is disabling > user-name/password auth on their IMAP offerings, too. FWIW a third option is to become your own provider: 1. Get a Virtual Private Server (VPS) 2. Get a DNS domain (any domain will do) 3. Set up a MTA on the VPS (postfix, exim, etc.) 4. Set up an IMAP server on the VPS (I recommend dovecot) (It's what I do)
> In the end I'll likely just set up the oauth credentials and put in
> the necessary software changes, but I have serious considered your
> "real" solution. There are a couple of problems, though. I have a lot
> of gmail-specific coding (perhaps a bad decision on my part) and a lot
> of stuff in the gmail ecosystem. But my fear is that, should I change
> to another provider --- I don't know, FastMail or someone--- they
> might in time go the same route. Eliminating "less secure" login
> methods certainly seems to be trending.
Anything is possible, but I highly doubt FastMail would go that route. I
don't think they even have their own OAuth2 domain. Instead, they
support different forms of two-factor authentication (U2F, OTP, SMS,
Yubikeyk. authenticator apps, etc.). For passwords, they have a very
nice system were you can set passwords separately for each app you're
using. You can view statistics like number of failed logins and remove
passwords if you think they might be compromised. They even still
support unencrypted protocols like FTP (but you need to set a specific
password for it). If I wouldn't self-host, I'd surely consider using
FastMail. From what I can see, they really try to support as many
use-cases as possible.
-David
Steinar Bang <sb@dod.no> writes:
>>>>>> Lars Ingebrigtsen <larsi@gnus.org>:
>> Bob Newell <bobnewell@bobnewell.net> writes:
>
>>> But my fear is that, should I change to another provider --- I
>>> don't know, FastMail or someone--- they might in time go the
>>> same route. Eliminating "less secure" login methods certainly
>>> seems to be trending.
>
>> Yup. I think I read somewhere that Microsoft is disabling
>> user-name/password auth on their IMAP offerings, too.
>
> FWIW a third option is to become your own provider:
> 1. Get a Virtual Private Server (VPS) 2. Get a DNS domain (any
> domain will do) 3. Set up a MTA on the VPS (postfix, exim,
> etc.) 4. Set up an IMAP server on the VPS (I recommend
> dovecot)
It's probably a good idea to get your own domain name in any
case. You can tell a provider like Fastmail to handle the email
for that domain, and if they ever start misbehaving, you can move
your domain elsewhere without updating your email address in
everyone's address book.
You can also do a similar thing communally: iki.fi is a non-profit
that gives you an email address and forwards email to wherever you
like (but only open to residents of Finland). Forwarding to Gmail
is a bit problematic because Google occasionally and seemingly
randomly blocks or delays mail from small-volume senders, but I've
been happily forwarding to Fastmail for a few years now.
David Engster writes:
> The real solution of course is that users simply stop using
> Google Mail and choose a proper mail provider.
Any suggestions?
--
Jorge.
"Jorge A. Alfaro-Murillo" <jorge.alfaro-murillo@yale.edu> writes:
> David Engster writes:
>
>> The real solution of course is that users simply stop using Google
>> Mail and choose a proper mail provider.
>
> Any suggestions?
I host my own, but I've heard good things about fastmail.
> David Engster writes: > >> The real solution of course is that users simply stop using Google >> Mail and choose a proper mail provider. > > Any suggestions? Stay flexble. Make sure to separate your email address from the mail hosting provider. This means: Register your own domain at some domain registrar. I would advise to only use this domain for mail, because this makes things easier. Then pay another(!) company to host your mail for that domain. I've heard good things about FastMail[1], but other mail companies provide this service as well. As long as they allow normal IMAP access, you will always be able to download all your mail and migrate to another provider whenever you're not satisfied. -David [1] https://www.fastmail.com/help/ourservice/hosteddomains.html
>>> The real solution of course is that users simply stop using Google >>> Mail and choose a proper mail provider. >> >> Any suggestions? > > Stay flexble. Make sure to separate your email address from the mail > hosting provider. > > This means: Register your own domain at some domain registrar. I would > advise to only use this domain for mail, because this makes things > easier. Then pay another(!) company to host your mail for that > domain. I've heard good things about FastMail[1], but other mail > companies provide this service as well. As long as they allow normal > IMAP access, you will always be able to download all your mail and > migrate to another provider whenever you're not satisfied. In addition to the above. I use mbsync[1] utility to keep IMAP folders in sync at multiple providers. It is very flexible in that sense. Sync between local and IMAP folders or sync between two IMAP providers. [1] http://isync.sourceforge.net/mbsync.html Regards -- Pankaj Jangid
> Stay flexble. Make sure to separate your email address from the mail > hosting provider. > > This means: Register your own domain at some domain registrar. I would > advise to only use this domain for mail, because this makes things > easier. Then pay another(!) company to host your mail for that > domain. I've heard good things about FastMail[1], but other mail > companies provide this service as well. As long as they allow normal > [1] https://www.fastmail.com/help/ourservice/hosteddomains.html Forgot to add. I am currently using zoho[1]. It is very low cost as compared to others; at least in India. [1] https://www.zoho.com/mail/zohomail-pricing.html Regards, -- Pankaj Jangid
Hellow, There is a good news for oauth of gmail? Sometimes my gmail login is not good. Sincerely, Gnus fan Byung-Hee -- ^고맙습니다 _布德天下_ 감사합니다_^))//
soyeomul@doraji.xyz (황병희) writes: > Hellow, > > There is a good news for oauth of gmail? > Sometimes my gmail login is not good. Just for the record, i attach related PR: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41386 Sincerely, Byung-Hee (daum-kakao account) -- ^고맙습니다 _地平天成_ 감사합니다_^))//