function to access raw article

function to access raw article

[Alexander Baier]

Hello,

I want to access the raw (unwashed) version of an article from emacs
lisp to search for a regexp. I do this in order to extract information
from the body of the article to be used in the summary line. I used
`gnus-summary-show-raw-article' which did not seem to work. What is the
preferred way of doing this?

Regards,
-- 
Alexander Baier

Re: function to access raw article

[Igor Sosa Mayor]

Alexander Baier <alexander.baier@mailbox.org> writes:

> Hello,
>
> I want to access the raw (unwashed) version of an article from emacs
> lisp to search for a regexp. I do this in order to extract information
> from the body of the article to be used in the summary line. I used
> `gnus-summary-show-raw-article' which did not seem to work. What is the
> preferred way of doing this?

I think gnus-summary-show-raw-article should work...

But in any way, I don't think this is necessary to get the summary line
(do you mean 'subject')? What do you want to achieve exactly?

-- 
:: Igor Sosa Mayor     :: joseleopoldo1792@gmail.com ::
:: GnuPG: 0x1C1E2890   :: http://www.gnupg.org/      ::
:: jabberid: rogorido  ::                            ::

Re: function to access raw article

[Alexander Baier]

On 2014-10-31 09:53 Igor Sosa Mayor wrote:
> Alexander Baier <alexander.baier@mailbox.org> writes:
>
>> Hello,
>>
>> I want to access the raw (unwashed) version of an article from emacs
>> lisp to search for a regexp. I do this in order to extract information
>> from the body of the article to be used in the summary line. I used
>> `gnus-summary-show-raw-article' which did not seem to work. What is the
>> preferred way of doing this?
>
> I think gnus-summary-show-raw-article should work...
>
> But in any way, I don't think this is necessary to get the summary line
> (do you mean 'subject')? What do you want to achieve exactly?

I have information in the body of the the article that I want to display
in the summary line. To do this, I wrote a function
gnus-user-format-function-Y that extracts said information from the
body. I then include %uY in my gnus-summary-line-format to include the
information in the summary line. My current function looks like this:

#+BEGIN_SRC emacs-lisp
  (defun gnus-user-format-function-Y (header)
    "Returns the title of a youtube video."
    (with-current-buffer gnus-article-buffer
      (gnus-summary-show-raw-article)
      (gnus-summary-select-article-buffer)
      (goto-char (point-min))
      (search-forward-regexp "\n.*just uploaded a video\n\\(.*\\)\nhttp:.*")
      (match-string-no-properties 1)))
#+END_SRC

If I enter a group with this function specified in the
sumary-line-format I get the following messages in the *Messages* buffer:

Error updating the summary line
Can't find article 832 [2 times]

Furthermore gnus displays
"Hiding all threads... <some fast growing number>" in the echo area and
hangs. When I press C-g I see a garbled summary buffer.

Regards,
-- 
Alexander Baier

Re: function to access raw article

[Steinar Bang]

>>>>> Alexander Baier <alexander.baier@mailbox.org>:

> I have information in the body of the the article that I want to
> display in the summary line.

One problem with this, could be that some backends don't have the
article body available at the time of rendering the summary buffer...?
(I think this is the case for nntp, and possibly also nnimap...?)

I think the summary buffer uses what's available in what's returned by
NNTP as NOV data:
 http://tools.ietf.org/html/rfc3977#section-8

Re: function to access raw article

[Alexander Baier]

On 2014-10-31 12:04 Steinar Bang wrote:
>>>>>> Alexander Baier <alexander.baier@mailbox.org>:
>
>> I have information in the body of the the article that I want to
>> display in the summary line.
>
> One problem with this, could be that some backends don't have the
> article body available at the time of rendering the summary buffer...?
> (I think this is the case for nntp, and possibly also nnimap...?)

The backend in question is nnmaildir, which should have access to the
articles body, as it is already locally available in my file
system. The gnus-summary-line-format is set as a group parameter on only
one group, so that it does not interfere with my normal summary line.

Maybe gnus does not expect me to call some article display commands
during the rendering of the summary buffer? Does anyone know any
specifics on this?

> I think the summary buffer uses what's available in what's returned by
> NNTP as NOV data:
>  http://tools.ietf.org/html/rfc3977#section-8

Thank you for the pointer! Sadly this would not be enough for me. :(

Regards,
-- 
Alexander Baier

Re: function to access raw article

[Igor Sosa Mayor]

Steinar Bang <sb@dod.no> writes:

>>>>>> Alexander Baier <alexander.baier@mailbox.org>:
>
>> I have information in the body of the the article that I want to
>> display in the summary line.
>
> One problem with this, could be that some backends don't have the
> article body available at the time of rendering the summary buffer...?
> (I think this is the case for nntp, and possibly also nnimap...?)

I also think this could be the problem... Have you trie with other
backends? 

-- 
:: Igor Sosa Mayor     :: joseleopoldo1792@gmail.com ::
:: GnuPG: 0x1C1E2890   :: http://www.gnupg.org/      ::
:: jabberid: rogorido  ::                            ::

Re: function to access raw article

[Alexander Baier]

On 2014-10-31 12:45 Igor Sosa Mayor wrote:
> Steinar Bang <sb@dod.no> writes:
>
>>>>>>> Alexander Baier <alexander.baier@mailbox.org>:
>>
>>> I have information in the body of the the article that I want to
>>> display in the summary line.
>>
>> One problem with this, could be that some backends don't have the
>> article body available at the time of rendering the summary buffer...?
>> (I think this is the case for nntp, and possibly also nnimap...?)
>
> I also think this could be the problem... Have you trie with other
> backends? 

See my previous answer to Steinar's post. I only have nntp and nnmaildir
backends available here. The nnmaildir is the one I am trying it with.

Regards,
-- 
Alexander Baier

Re: function to access raw article

[Ted Zlatanov]

On Fri, 31 Oct 2014 09:29:41 +0100 Alexander Baier <alexander.baier@mailbox.org> wrote: 

AB> I want to access the raw (unwashed) version of an article from emacs
AB> lisp to search for a regexp. I do this in order to extract information
AB> from the body of the article to be used in the summary line. I used
AB> `gnus-summary-show-raw-article' which did not seem to work. What is the
AB> preferred way of doing this?

Try `spam-get-article-as-string' in spam.el, which is used by spam
backends that need the full article.  But as others said, it may not
work :)

Ted

Re: function to access raw article

[Alexander Baier]

On 2014-10-31 13:55 Ted Zlatanov wrote:
> On Fri, 31 Oct 2014 09:29:41 +0100 Alexander Baier
> <alexander.baier@mailbox.org> wrote:
>
> AB> I want to access the raw (unwashed) version of an article from emacs
> AB> lisp to search for a regexp. I do this in order to extract information
> AB> from the body of the article to be used in the summary line. I used
> AB> `gnus-summary-show-raw-article' which did not seem to work. What is the
> AB> preferred way of doing this?
>
> Try `spam-get-article-as-string' in spam.el, which is used by spam
> backends that need the full article.  But as others said, it may not
> work :)
>

This is it! The following works like a charm. It is designed to extract
the title of the video from a mail you get from youtube when someone you
are subscribed to uploads a new video.

#+BEGIN_SRC emacs-lisp
  (defun gnus-user-format-function-Y (header)
    "Returns the title of a youtube video."
    (let ((body (spam-get-article-as-string (elt header 0))))
      (string-match "\n.*just uploaded a video\n\\(.*\\)\nhttp:.*" body)
      (match-string-no-properties 1 body)))
#+END_SRC

Thank you Ted, Steinar and Igor! :)

Regards,
-- 
Alexander Baier

Re: function to access raw article

[Alexander Baier]

On 2014-10-31 14:38 Alexander Baier wrote:
> On 2014-10-31 13:55 Ted Zlatanov wrote:
>> On Fri, 31 Oct 2014 09:29:41 +0100 Alexander Baier
>> <alexander.baier@mailbox.org> wrote:
>>
>> AB> I want to access the raw (unwashed) version of an article from emacs
>> AB> lisp to search for a regexp. I do this in order to extract information
>> AB> from the body of the article to be used in the summary line. I used
>> AB> `gnus-summary-show-raw-article' which did not seem to work. What is the
>> AB> preferred way of doing this?
>>
>> Try `spam-get-article-as-string' in spam.el, which is used by spam
>> backends that need the full article.  But as others said, it may not
>> work :)
>>
>
> This is it! The following works like a charm. It is designed to extract
> the title of the video from a mail you get from youtube when someone you
> are subscribed to uploads a new video.
>
> #+BEGIN_SRC emacs-lisp
>   (defun gnus-user-format-function-Y (header)
>     "Returns the title of a youtube video."
>     (let ((body (spam-get-article-as-string (elt header 0))))
>       (string-match "\n.*just uploaded a video\n\\(.*\\)\nhttp:.*" body)
>       (match-string-no-properties 1 body)))
> #+END_SRC
>
> Thank you Ted, Steinar and Igor! :)
>
> Regards,

I have updated the regexp to catch multiline titles:

#+BEGIN_SRC emacs-lisp
  (defun gnus-user-format-function-Y (header)
    "Returns the title of a youtube video."
    (let ((body (spam-get-article-as-string (elt header 0)))
          (regex (concat "Content-Type: text/plain;.*\n"
                         "\n.*just uploaded a video\n"
                         "\\(.*\\(?:\n.*\\)*?\\)\nhttp:.*")))
      (string-match regex body)
      (s-replace "\n" "" (match-string-no-properties 1 body))))
#+END_SRC

Regards,
-- 
Alexander Baier

Re: function to access raw article

[Eric Abrahamsen]

Alexander Baier <alexander.baier@mailbox.org> writes:

> On 2014-10-31 09:53 Igor Sosa Mayor wrote:
>> Alexander Baier <alexander.baier@mailbox.org> writes:
>>
>>> Hello,
>>>
>>> I want to access the raw (unwashed) version of an article from emacs
>>> lisp to search for a regexp. I do this in order to extract information
>>> from the body of the article to be used in the summary line. I used
>>> `gnus-summary-show-raw-article' which did not seem to work. What is the
>>> preferred way of doing this?
>>
>> I think gnus-summary-show-raw-article should work...
>>
>> But in any way, I don't think this is necessary to get the summary line
>> (do you mean 'subject')? What do you want to achieve exactly?
>
> I have information in the body of the the article that I want to display
> in the summary line. To do this, I wrote a function
> gnus-user-format-function-Y that extracts said information from the
> body. I then include %uY in my gnus-summary-line-format to include the
> information in the summary line. My current function looks like this:
>
> #+BEGIN_SRC emacs-lisp
>   (defun gnus-user-format-function-Y (header)
>     "Returns the title of a youtube video."
>     (with-current-buffer gnus-article-buffer
>       (gnus-summary-show-raw-article)
>       (gnus-summary-select-article-buffer)
>       (goto-char (point-min))
>       (search-forward-regexp "\n.*just uploaded a video\n\\(.*\\)\nhttp:.*")
>       (match-string-no-properties 1)))
> #+END_SRC

Try something like this:

(when (buffer-live-p (get-buffer gnus-original-article-buffer))
  (with-current-buffer gnus-original-article-buffer
    (goto-char (point-min))
    ;;etc
    ))

At any rate, I'm pretty sure you want to be using
gnus-original-article-buffer.

HTH,
Eric

Re: function to access raw article

[Peter Münster]

On Fri, Oct 31 2014, Alexander Baier wrote:

> This is it! The following works like a charm. It is designed to extract
> the title of the video from a mail you get from youtube when someone you
> are subscribed to uploads a new video.
>
>   (defun gnus-user-format-function-Y (header)

Hi,

If you ever get performance problems (delays when generating the summary
buffer), then you could let procmail + formail do the job. These tools
can add new header lines, for example:
X-youtube-titles: title 1, title 2, etc.

Or you can rewrite the subject. Then, Gnus won't need to parse the body
no more.

-- 
           Peter

Re: function to access raw article

[Alexander Baier]

On 2014-10-31 15:19 Eric Abrahamsen wrote:
> Alexander Baier <alexander.baier@mailbox.org> writes:
>
>> On 2014-10-31 09:53 Igor Sosa Mayor wrote:
>>> Alexander Baier <alexander.baier@mailbox.org> writes:
>>>
>>>> Hello,
>>>>
>>>> I want to access the raw (unwashed) version of an article from emacs
>>>> lisp to search for a regexp. I do this in order to extract information
>>>> from the body of the article to be used in the summary line. I used
>>>> `gnus-summary-show-raw-article' which did not seem to work. What is the
>>>> preferred way of doing this?
>>>
>>> I think gnus-summary-show-raw-article should work...
>>>
>>> But in any way, I don't think this is necessary to get the summary line
>>> (do you mean 'subject')? What do you want to achieve exactly?
>>
>> I have information in the body of the the article that I want to display
>> in the summary line. To do this, I wrote a function
>> gnus-user-format-function-Y that extracts said information from the
>> body. I then include %uY in my gnus-summary-line-format to include the
>> information in the summary line. My current function looks like this:
>>
>> #+BEGIN_SRC emacs-lisp
>>   (defun gnus-user-format-function-Y (header)
>>     "Returns the title of a youtube video."
>>     (with-current-buffer gnus-article-buffer
>>       (gnus-summary-show-raw-article)
>>       (gnus-summary-select-article-buffer)
>>       (goto-char (point-min))
>>       (search-forward-regexp "\n.*just uploaded a video\n\\(.*\\)\nhttp:.*")
>>       (match-string-no-properties 1)))
>> #+END_SRC
>
> Try something like this:
>
> (when (buffer-live-p (get-buffer gnus-original-article-buffer))
>   (with-current-buffer gnus-original-article-buffer
>     (goto-char (point-min))
>     ;;etc
>     ))
>
> At any rate, I'm pretty sure you want to be using
> gnus-original-article-buffer.

I tried that and the buffer is never live.

Thanks,
-- 
Alexander Baier

Re: function to access raw article

[Alexander Baier]

On 2014-10-31 15:22 Peter Münster wrote:
> On Fri, Oct 31 2014, Alexander Baier wrote:
>
>> This is it! The following works like a charm. It is designed to extract
>> the title of the video from a mail you get from youtube when someone you
>> are subscribed to uploads a new video.
>>
>>   (defun gnus-user-format-function-Y (header)
>
> Hi,
>
> If you ever get performance problems (delays when generating the summary
> buffer), then you could let procmail + formail do the job. These tools
> can add new header lines, for example:
> X-youtube-titles: title 1, title 2, etc.
>
> Or you can rewrite the subject. Then, Gnus won't need to parse the body
> no more.

Thank you for the pointer! I don't think there will be performance
problems in my particular case, as I always display < 100 articles in
that group.

The advantage of doing it in gnus, is of course, that I do not have to
learn another tool. :)

Regards,
-- 
Alexander Baier

Re: function to access raw article

[Eric Abrahamsen]

Alexander Baier <alexander.baier@mailbox.org> writes:

> On 2014-10-31 15:19 Eric Abrahamsen wrote:
>> Alexander Baier <alexander.baier@mailbox.org> writes:
>>
>>> On 2014-10-31 09:53 Igor Sosa Mayor wrote:
>>>> Alexander Baier <alexander.baier@mailbox.org> writes:
>>>>
>>>>> Hello,
>>>>>
>>>>> I want to access the raw (unwashed) version of an article from emacs
>>>>> lisp to search for a regexp. I do this in order to extract information
>>>>> from the body of the article to be used in the summary line. I used
>>>>> `gnus-summary-show-raw-article' which did not seem to work. What is the
>>>>> preferred way of doing this?
>>>>
>>>> I think gnus-summary-show-raw-article should work...
>>>>
>>>> But in any way, I don't think this is necessary to get the summary line
>>>> (do you mean 'subject')? What do you want to achieve exactly?
>>>
>>> I have information in the body of the the article that I want to display
>>> in the summary line. To do this, I wrote a function
>>> gnus-user-format-function-Y that extracts said information from the
>>> body. I then include %uY in my gnus-summary-line-format to include the
>>> information in the summary line. My current function looks like this:
>>>
>>> #+BEGIN_SRC emacs-lisp
>>>   (defun gnus-user-format-function-Y (header)
>>>     "Returns the title of a youtube video."
>>>     (with-current-buffer gnus-article-buffer
>>>       (gnus-summary-show-raw-article)
>>>       (gnus-summary-select-article-buffer)
>>>       (goto-char (point-min))
>>>       (search-forward-regexp "\n.*just uploaded a video\n\\(.*\\)\nhttp:.*")
>>>       (match-string-no-properties 1)))
>>> #+END_SRC
>>
>> Try something like this:
>>
>> (when (buffer-live-p (get-buffer gnus-original-article-buffer))
>>   (with-current-buffer gnus-original-article-buffer
>>     (goto-char (point-min))
>>     ;;etc
>>     ))
>>
>> At any rate, I'm pretty sure you want to be using
>> gnus-original-article-buffer.
>
> I tried that and the buffer is never live.

Ah, I wasn't sure if that would be the case. Good to know, I guess...