From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/87120 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.gnus.general Subject: Re: gnus-cloud work Date: Wed, 06 Jul 2016 10:50:17 -0400 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <87lh1eg6s6.fsf@lifelogs.com> References: <87twzdc86f.fsf@building.gnus.org> <8761bhzz0f.fsf@lifelogs.com> <878ugdxpuc.fsf@lifelogs.com> <86twv7p7nu.fsf_-_@dod.no> <86r3pnciof.fsf_-_@dod.no> <87io0y4bw4.fsf_-_@lifelogs.com> <87lh3439r6.fsf@dod.no> <87bn3vs1tb.fsf@lifelogs.com> <877fe0svto.fsf@lifelogs.com> <87r3by2sfd.fsf_-_@lifelogs.com> <87wplnzhvb.fsf@lifelogs.com> <8737nukab5.fsf@ericabrahamsen.net> <87a8i2n2i7.fsf@lifelogs.com> <878txm6rsy.fsf@ericabrahamsen.net> <87poqxjls5.fsf@lifelogs.com> <87shvsj1lg.fsf@lifelogs.com> Reply-To: ding@gnus.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1467816763 383 80.91.229.3 (6 Jul 2016 14:52:43 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 6 Jul 2016 14:52:43 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M35340@lists.math.uh.edu Wed Jul 06 16:52:28 2016 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from lists1.math.uh.edu ([129.7.128.208]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bKoBd-0004y2-Bd for ding-account@gmane.org; Wed, 06 Jul 2016 16:52:25 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by lists1.math.uh.edu with smtp (Exim 4.86_2) (envelope-from ) id 1bKo9q-0003po-Rt; Wed, 06 Jul 2016 09:50:34 -0500 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by lists1.math.uh.edu with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.86_2) (envelope-from ) id 1bKo9m-0003p9-8P for ding@lists.math.uh.edu; Wed, 06 Jul 2016 09:50:30 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtps (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.86_2) (envelope-from ) id 1bKo9k-0003Q2-E8 for ding@lists.math.uh.edu; Wed, 06 Jul 2016 09:50:30 -0500 Original-Received: from plane.gmane.org ([80.91.229.3]) by quimby.gnus.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.80) (envelope-from ) id 1bKo9i-0001RU-Lf for ding@gnus.org; Wed, 06 Jul 2016 16:50:26 +0200 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1bKo9g-0003is-Es for ding@gnus.org; Wed, 06 Jul 2016 16:50:24 +0200 Original-Received: from c-98-229-60-157.hsd1.ma.comcast.net ([98.229.60.157]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 06 Jul 2016 16:50:24 +0200 Original-Received: from tzz by c-98-229-60-157.hsd1.ma.comcast.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 06 Jul 2016 16:50:24 +0200 X-Injected-Via-Gmane: http://gmane.org/ Mail-Followup-To: ding@gnus.org Original-Lines: 50 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: c-98-229-60-157.hsd1.ma.comcast.net X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6;d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Copies-To: never User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) Cancel-Lock: sha1:d+ifGN3XU+5UAMlHx5FZyf1vEbI= X-Spam-Score: -4.2 (----) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:87120 Archived-At: On Fri, 01 Jul 2016 21:00:27 -0400 Ted Zlatanov wrote: TZ> Update: TZ> * newsrc data format changed, new entries have type :newsrc-data and the TZ> contents are inlined TZ> * `M-x gnus-cloud-upload-all-data' will upload all the data, files and TZ> newsrc both TZ> * (this must be run manually for now) `M-: (gnus-cloud-download-data t 1)' TZ> will get all the download data and install the newsrc and file data TZ> Only full uploads/downloads work. TZ> This is worth another test if people are adventurous, but *make sure you TZ> back up your newsrc.eld* before downloading the data. I made a few more tweaks: * only update if data is newer (can be overriden) * add `gnus-cloud-interactive' variable to confirm updates (good for testing and paranoia) The last piece missing is where to hook the upload and download code. Upload could happen every 30 minutes or every time more than 10 groups are updated? We have to be careful of race conditions where two machines are not aware that the other has uploaded "locally newer" data. But maybe that's OK, if we agree that the typical use case is one user on multiple machines, not multiple users on multiple machines. Download... not sure... maybe every time Gnus refreshes all groups with `g'? Or maybe a background process could continuously monitor IMAP for new articles in the cloud group? I don't know how that would work from the Gnus side, but it would be super nice for the user. Any IMAP/nnimap experts? Finally, currently the sequence number is a local number. Each machine starts from 1 and walks up to the global max sequence (from the IMAP articles). I think that's not ideal, because two machines can easily end up uploading with the same sequence number without careful synchronization. Global counters are hard to implement. Maybe it's better to simply use a timestamp string or the raw IMAP article number as the sequence number. The IMAP article number sequence in particular is guaranteed by the IMAP server, so I'm leaning towards that. What do you think? Once upload, download, and sequence numbers are settled, I'll squash all the commits and merge this into Emacs master branch. Ted