Announcements and discussions for Gnus, the GNU Emacs Usenet newsreader
 help / color / mirror / Atom feed
* RSS with Gnus (elfeed backend?)
@ 2021-04-01 18:59 Garjola Dindi
  2021-04-01 19:19 ` Jose A. Ortega Ruiz
  2021-04-02 15:04 ` Jeffrey DeLeo
  0 siblings, 2 replies; 11+ messages in thread
From: Garjola Dindi @ 2021-04-01 18:59 UTC (permalink / raw)
  To: info-gnus-english


Hi,

I read RSS/Atom feeds in Gnus everyday using a couple of ways.

The first one, is the standard nnrss backend with the following trick I
found somewhere to convert Atom to RSS:
,----
| (require 'mm-url)
| (defadvice mm-url-insert (after DE-convert-atom-to-rss () )
|   "Converts atom to RSS by calling xsltproc."
|   (when (re-search-forward "xmlns=\"http://www.w3.org/.*/Atom\"" 
| 			   nil t)
|     (goto-char (point-min))
|     (message "Converting Atom to RSS... ")
|     (call-process-region (point-min) (point-max) 
| 			 "xsltproc" 
| 			 t t nil 
| 			 (expand-file-name "~/.emacs.d/atom2rss.xsl") "-")
|     (goto-char (point-min))
|     (message "Converting Atom to RSS... done")))
`----

Since the nnrss fetching blocks Emacs, this is not useable for a large
number of feeds. So I reserve this approach to a few feeds which are
responsive and that I don't need to check while offline.

For most of my feeds, I use feed2imap
https://github.com/feed2imap/feed2imap which allows me to download to
disk and view offline using an e-mail backend (I use nnmaildir).
Feed2imap seems to be unmaintained and is not very robust wrt updated
items on the feeds.

Looking for alternatives I found this:
https://protesilaos.com/dotemacs/#h:7e4448ec-3878-4ea2-9cd6-75e9faaac373

which actually describes rather well my issues. I think I am more of a
Gnus fan person than Protesilaos, and I resist to split my inboxes
between Elfeed (https://github.com/skeeto/elfeed) for feeds and Gnus for
everything else.

What I was wondering (but it may be completely dumb) is that if elfeed
itself (which is written in Elisp) could be used as a backend for Gnus.

Since there seem to be Gnus backends for nearly everything, maybe that
interfacing with Elfeed is possible.

What do you think? 

I am not an elisp programmer, but I may try to have a look at it if it
doesn't seem a bad idea to people having written backends for Gnus.

Thanks for your feedback.
  
-- 


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RSS with Gnus (elfeed backend?)
  2021-04-01 18:59 RSS with Gnus (elfeed backend?) Garjola Dindi
@ 2021-04-01 19:19 ` Jose A. Ortega Ruiz
  2021-04-02  6:43   ` Garjola Dindi
  2021-04-02 15:04 ` Jeffrey DeLeo
  1 sibling, 1 reply; 11+ messages in thread
From: Jose A. Ortega Ruiz @ 2021-04-01 19:19 UTC (permalink / raw)
  To: info-gnus-english


Hi,

Not exactly what you're suggesting, but i've used rss2email for years
now, and it works pretty well (the package in debian keeps getting
updates, and it's working with the latest 3.9 python).  Instead of
putting the emails in a maildir directly, it sends an email to a (local)
address of your choice.

HTH,
jao
-- 
The surest way to corrupt a youth is to instruct him to hold in higher
esteem those who think alike than those who think
differently. -Friedrich Nietzsche, philosopher (1844-1900)


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RSS with Gnus (elfeed backend?)
  2021-04-01 19:19 ` Jose A. Ortega Ruiz
@ 2021-04-02  6:43   ` Garjola Dindi
  2021-04-02 20:44     ` Jose A. Ortega Ruiz
  0 siblings, 1 reply; 11+ messages in thread
From: Garjola Dindi @ 2021-04-02  6:43 UTC (permalink / raw)
  To: info-gnus-english; +Cc: Jose A. Ortega Ruiz

On Thu 01-Apr-2021 at 21:19:20 +02, "Jose A. Ortega Ruiz" <jao@gnu.org>
wrote: 
> Hi,
>
> Not exactly what you're suggesting, but i've used rss2email for years
> now, and it works pretty well (the package in debian keeps getting
> updates, and it's working with the latest 3.9 python).  Instead of
> putting the emails in a maildir directly, it sends an email to a (local)
> address of your choice.
>
> HTH,
> jao

Hi José,

Thanks for the hint! It seems an interesting replacement for feeds2imap.
I guess that all feeds go to the same e-mail address and that you use
splitting to create different groups? Would you be willing to share your
configuration?

I am still interested in the pure elisp route. I have had a look at the
elfeed source code and is seems rather modular and using its API seems
feasible.

On the other hand, I have had a look at the code of some Gnus backends
(hacker news, reddit) and I don't understand what is needed to build
one. 

Is there any example of Gnus backend which can be suggested as an
introductory tutorial? I am going to study the relevant section of the
Gnus info manual, but any advice would be welcome.

Thanks everybody.

G.

-- 


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RSS with Gnus (elfeed backend?)
  2021-04-01 18:59 RSS with Gnus (elfeed backend?) Garjola Dindi
  2021-04-01 19:19 ` Jose A. Ortega Ruiz
@ 2021-04-02 15:04 ` Jeffrey DeLeo
  2021-04-02 15:29   ` Garjola Dindi
  1 sibling, 1 reply; 11+ messages in thread
From: Jeffrey DeLeo @ 2021-04-02 15:04 UTC (permalink / raw)
  To: info-gnus-english

Have you tried gwene.org?
You register the rss feed with the site, and it shows up as a newsgroup
you can subscribe to via news.gwene.org. I follow many
sites in gnus via this method. You may find many of the site you wish to
follow are already there...


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RSS with Gnus (elfeed backend?)
  2021-04-02 15:04 ` Jeffrey DeLeo
@ 2021-04-02 15:29   ` Garjola Dindi
  2021-04-02 18:21     ` Tim Landscheidt
  0 siblings, 1 reply; 11+ messages in thread
From: Garjola Dindi @ 2021-04-02 15:29 UTC (permalink / raw)
  To: info-gnus-english

On Fri 02-Apr-2021 at 17:04:35 +02, Jeffrey DeLeo
<JeffreyDeLeo@gmail.com> wrote: 
> Have you tried gwene.org?
> You register the rss feed with the site, and it shows up as a newsgroup
> you can subscribe to via news.gwene.org. I follow many
> sites in gnus via this method. You may find many of the site you wish to
> follow are already there...

Hi JeffreyDeLeo,

Thanks for the suggestion. I am aware of gwene and I have used it in the
past. But I would prefer something which does not depend on a
centralized server. This defeats one of the purposes of RSS itself, in
my opinion. And there is also the issue of private feeds, of course.

I really think that nrss should be improved to solve its shortcomings
and it seems that elfeed is a very good feed reader. I wish I had the
skills to propose some patches or something.

I will study the code and the documentation on how to build a new
backend, but it will take months for me. Maybe if I start something this
will motivate real elisp programmers to chime in?

Cheers.

G.
-- 


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RSS with Gnus (elfeed backend?)
  2021-04-02 15:29   ` Garjola Dindi
@ 2021-04-02 18:21     ` Tim Landscheidt
  2021-04-03  7:30       ` Garjola Dindi
  0 siblings, 1 reply; 11+ messages in thread
From: Tim Landscheidt @ 2021-04-02 18:21 UTC (permalink / raw)
  To: info-gnus-english

Garjola Dindi <garjola@garjola.net> wrote:

>> Have you tried gwene.org?
>> You register the rss feed with the site, and it shows up as a newsgroup
>> you can subscribe to via news.gwene.org. I follow many
>> sites in gnus via this method. You may find many of the site you wish to
>> follow are already there...

> Thanks for the suggestion. I am aware of gwene and I have used it in the
> past. But I would prefer something which does not depend on a
> centralized server. This defeats one of the purposes of RSS itself, in
> my opinion. And there is also the issue of private feeds, of course.

> I really think that nrss should be improved to solve its shortcomings
> and it seems that elfeed is a very good feed reader. I wish I had the
> skills to propose some patches or something.

> I will study the code and the documentation on how to build a new
> backend, but it will take months for me. Maybe if I start something this
> will motivate real elisp programmers to chime in?

It's even worse because Emacs already has yet another (rath-
er fragile) RSS reader, newsticker, that AFAIR does not
share code with either, and nnrss has an asynchronous, yet
somewhat manual out-of-Gnus retriever
(nnrss-generate-download-script), the keys for moving to the
next or the next unread article in newsticker are "reversed"
compared to Gnus, elfeed seems to have a separate system for
scoring articles, etc., etc., etc.  newsticker as well as
nnrss feel like "technology demonstrators", i. e., yeah, you
can read RSS feeds in Emacs or Gnus, but nobody really does.

So if someone would implement a (preferably new) nice, clean
backend for Gnus that DWIM, I'd be eternally grateful :-).

Tim


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RSS with Gnus (elfeed backend?)
  2021-04-02  6:43   ` Garjola Dindi
@ 2021-04-02 20:44     ` Jose A. Ortega Ruiz
  2021-04-03  7:34       ` Garjola Dindi
  0 siblings, 1 reply; 11+ messages in thread
From: Jose A. Ortega Ruiz @ 2021-04-02 20:44 UTC (permalink / raw)
  To: info-gnus-english

On Fri, Apr 02 2021, Garjola Dindi wrote:

> Hi José,
>
> Thanks for the hint! It seems an interesting replacement for feeds2imap.
> I guess that all feeds go to the same e-mail address and that you use
> splitting to create different groups?

Yes, more or less.  When adding new feeds to the r2e list, one can
specify the email address per feed, and i set it to things like
jao+programming@localhost, jao+news@localhost, etc., and then filter on
the part after + in the address to group my emails into categories (in
my case, the filter is a sieve script, since i use dovecot for other
things and have it my LDA, but gnus can do similar filtering).  To ease
filtering, r2e also lets you define a message header of your choice that
will be added to all emails.

> Would you be willing to share your configuration?

This is an extract of my config (r2e updates it when you use commands to
add or remove feeds and the like).  Reading it i discovered that r2e is
also able to deliver to maildirs, BTW (setting email-protocol to
maildir):

        [DEFAULT]
        from = r2e@localhost
        user-agent = rss2email/__VERSION__ (__URL__)
        use-8bit = True
        force-from = False
        use-publisher-email = True
        name-format = {feed-title}: {author}
        to = jao@localhost
        unicode-snob = True
        links-after-each-paragraph = True
        body-width = 0
        email-protocol = smtp
        sendmail = /usr/sbin/sendmail
        smtp-auth = False
        smtp-username = username
        smtp-password = password
        smtp-server = localhost
        smtp-port = 25
        smtp-ssl = False

        [feed.morningpaper]
        url = https://blog.acolyer.org/feed/
        to = jao+programming@localhost

        [feed.notevenwrong]
        url = http://www.math.columbia.edu/~woit/wordpress/?feed=rss2
        to = jao+physics@localhost

        [feed.wondermark]
        url = http://feeds.feedburner.com/wondermark
        to = jao+fun@localhost

        [feed.smbc]
        url = https://www.smbc-comics.com/comic/rss
        to = jao+fun@localhost

        [feed.existential]
        url = https://existentialcomics.com/rss.xml
        to = jao+fun@localhost

        [feed.emacsplanet]
        url = https://planet.emacslife.com/atom.xml
        to = jao+emacs@localhost

        [feed.jao]
        url = https://jao.io/blog/rss.xml
        to = jao+programming@localhost

        [feed.in-our-time]
        url = https://podcasts.files.bbci.co.uk/b006qykl.rss
        to = jao+podcasts@localhost

As an aside, a very nice side-effect of using something like r2e is that
the contents of the feeds are indexed and searchable by Gnus (i like to
use the dovecot and notmuch backends).

Cheers,
jao
-- 
Since when do we have to agree with people to defend them from
injustice? -Lillian Hellman, playwright (20 Jun 1905-1984)


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RSS with Gnus (elfeed backend?)
  2021-04-02 18:21     ` Tim Landscheidt
@ 2021-04-03  7:30       ` Garjola Dindi
  2021-04-04  0:06         ` Eric Abrahamsen
  0 siblings, 1 reply; 11+ messages in thread
From: Garjola Dindi @ 2021-04-03  7:30 UTC (permalink / raw)
  To: info-gnus-english

On Fri 02-Apr-2021 at 20:21:07 +02, Tim Landscheidt
<tim@tim-landscheidt.de> wrote: 
> Garjola Dindi <garjola@garjola.net> wrote:
>>
> It's even worse because Emacs already has yet another (rath-
> er fragile) RSS reader, newsticker, that AFAIR does not
> share code with either, and nnrss has an asynchronous, yet
> somewhat manual out-of-Gnus retriever
> (nnrss-generate-download-script), the keys for moving to the
> next or the next unread article in newsticker are "reversed"
> compared to Gnus, elfeed seems to have a separate system for
> scoring articles, etc., etc., etc.  newsticker as well as
> nnrss feel like "technology demonstrators", i. e., yeah, you
> can read RSS feeds in Emacs or Gnus, but nobody really does.
>

And there is also org-feed.el which generates an org-mode outline from
RSS or Atom feeds.

> So if someone would implement a (preferably new) nice, clean
> backend for Gnus that DWIM, I'd be eternally grateful :-).

I have spent some time reading elfeed's source code and it provides a
nice API. What I don't like about elfeed is that it is «entry-oriented»,
that is, all articles are dumped in the same inbox and all the selection
operations are done through tags. I think that a Gnus back-end could use
elfeed's API to filter the entries and generate the content of Gnus's
groups (one group per feed, for instance), while all the feed updating,
downloading, etc. could be delegated to elfeed.

What I have in mind is writing a light Gnus backend (inspired for
instance from the one for Twitter that Lars Ingebrigtsen wrote [1]) that
just uses elfeed behind the scenes.

The difficulty is that I do not understand how a Gnus backend works, so
I will have to study this first. 

If anybody can point to an easy to understand Gnus backend to get me
started, I would be very grateful.

Thanks.


Footnotes:
[1]  https://lars.ingebrigtsen.no/2020/09/19/you-can-read-twitter-with-gnus/

-- 


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RSS with Gnus (elfeed backend?)
  2021-04-02 20:44     ` Jose A. Ortega Ruiz
@ 2021-04-03  7:34       ` Garjola Dindi
  0 siblings, 0 replies; 11+ messages in thread
From: Garjola Dindi @ 2021-04-03  7:34 UTC (permalink / raw)
  To: info-gnus-english

On Fri 02-Apr-2021 at 22:44:37 +02, "Jose A. Ortega Ruiz" <jao@gnu.org>
wrote: 
> On Fri, Apr 02 2021, Garjola Dindi wrote:
>
>> Hi José,
>>

[...]

>
>> Would you be willing to share your configuration?
>
> This is an extract of my config (r2e updates it when you use commands to
> add or remove feeds and the like).  Reading it i discovered that r2e is
> also able to deliver to maildirs, BTW (setting email-protocol to
> maildir):
>

[...]

> As an aside, a very nice side-effect of using something like r2e is that
> the contents of the feeds are indexed and searchable by Gnus (i like to
> use the dovecot and notmuch backends).
>

Thanks for the configuration and advices. This will be useful for me in
the likely case that I fail to do something to use elfeed from Gnus.

I really think that elfeed is very well designed and its only downside
is that it is not a Gnus backend ;)

-- 


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RSS with Gnus (elfeed backend?)
  2021-04-03  7:30       ` Garjola Dindi
@ 2021-04-04  0:06         ` Eric Abrahamsen
  2021-04-04  9:59           ` Garjola Dindi
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Abrahamsen @ 2021-04-04  0:06 UTC (permalink / raw)
  To: Garjola Dindi; +Cc: info-gnus-english

Garjola Dindi <garjola@garjola.net> writes:

> On Fri 02-Apr-2021 at 20:21:07 +02, Tim Landscheidt
> <tim@tim-landscheidt.de> wrote: 
>> Garjola Dindi <garjola@garjola.net> wrote:
>>>
>> It's even worse because Emacs already has yet another (rath-
>> er fragile) RSS reader, newsticker, that AFAIR does not
>> share code with either, and nnrss has an asynchronous, yet
>> somewhat manual out-of-Gnus retriever
>> (nnrss-generate-download-script), the keys for moving to the
>> next or the next unread article in newsticker are "reversed"
>> compared to Gnus, elfeed seems to have a separate system for
>> scoring articles, etc., etc., etc.  newsticker as well as
>> nnrss feel like "technology demonstrators", i. e., yeah, you
>> can read RSS feeds in Emacs or Gnus, but nobody really does.
>>
>
> And there is also org-feed.el which generates an org-mode outline from
> RSS or Atom feeds.
>
>> So if someone would implement a (preferably new) nice, clean
>> backend for Gnus that DWIM, I'd be eternally grateful :-).
>
> I have spent some time reading elfeed's source code and it provides a
> nice API. What I don't like about elfeed is that it is «entry-oriented»,
> that is, all articles are dumped in the same inbox and all the selection
> operations are done through tags. I think that a Gnus back-end could use
> elfeed's API to filter the entries and generate the content of Gnus's
> groups (one group per feed, for instance), while all the feed updating,
> downloading, etc. could be delegated to elfeed.
>
> What I have in mind is writing a light Gnus backend (inspired for
> instance from the one for Twitter that Lars Ingebrigtsen wrote [1]) that
> just uses elfeed behind the scenes.
>
> The difficulty is that I do not understand how a Gnus backend works, so
> I will have to study this first. 
>
> If anybody can point to an easy to understand Gnus backend to get me
> started, I would be very grateful.

Check the "Gnus Reference Guide" section of the Gnus manual, in
particular the second section, "Back End Interface". But you won't like
it :)

_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RSS with Gnus (elfeed backend?)
  2021-04-04  0:06         ` Eric Abrahamsen
@ 2021-04-04  9:59           ` Garjola Dindi
  0 siblings, 0 replies; 11+ messages in thread
From: Garjola Dindi @ 2021-04-04  9:59 UTC (permalink / raw)
  To: info-gnus-english

On Sun 04-Apr-2021 at 02:06:32 +02, Eric Abrahamsen
<eric@ericabrahamsen.net> wrote: 
> Garjola Dindi <garjola@garjola.net> writes:
>
>> On Fri 02-Apr-2021 at 20:21:07 +02, Tim Landscheidt
>> <tim@tim-landscheidt.de> wrote: 
>>> Garjola Dindi <garjola@garjola.net> wrote:
>> What I have in mind is writing a light Gnus backend (inspired for
>> instance from the one for Twitter that Lars Ingebrigtsen wrote [1]) that
>> just uses elfeed behind the scenes.
>>
>> The difficulty is that I do not understand how a Gnus backend works, so
>> I will have to study this first. 
>>
>> If anybody can point to an easy to understand Gnus backend to get me
>> started, I would be very grateful.
>
> Check the "Gnus Reference Guide" section of the Gnus manual, in
> particular the second section, "Back End Interface". But you won't like
> it :)

Hi Eric,

I have (tried to) read this part of the manual. Not easy, particularly
the fact that the 2 sections on required and optional functions do not
mention de OO interface which appears later.

Right now, I am studying these backends:

nndiscourse - https://github.com/dickmao/nndiscourse.git
nnhackernews - https://github.com/dickmao/nnhackernews.git
gnus-twit - https://github.com/larsmagne/gnus-twit
nntwitter - https://github.com/dickmao/nntwitter.git

And I go back and forth between the manual and the source code of those.

I have also succeeded in generating buffers with contents from elfeed
functions, so I guess its just now a matter of time and struggle.

It is not yet clear to me if there would be an interest in deriving from
nnrss and just reimplementing the appropriate functions.

I have started a repository
(https://gitlab.com/garjola-emacs/nnelfeed/-/tree/master). There is
nothing useful there yet. Just some ideas, but I hope I will be able to
have some progress in the next months.

If anybody can give some pointers to other ressources I could have a
look at, I would be very grateful.

Thanks.

-- 


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2021-04-04  9:59 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-01 18:59 RSS with Gnus (elfeed backend?) Garjola Dindi
2021-04-01 19:19 ` Jose A. Ortega Ruiz
2021-04-02  6:43   ` Garjola Dindi
2021-04-02 20:44     ` Jose A. Ortega Ruiz
2021-04-03  7:34       ` Garjola Dindi
2021-04-02 15:04 ` Jeffrey DeLeo
2021-04-02 15:29   ` Garjola Dindi
2021-04-02 18:21     ` Tim Landscheidt
2021-04-03  7:30       ` Garjola Dindi
2021-04-04  0:06         ` Eric Abrahamsen
2021-04-04  9:59           ` Garjola Dindi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).