Attaching notes to messages

Attaching notes to messages

[Hannu Koivisto]

Greetings,

Has anyone thought about or, even better, implemented some way of
attaching arbitrary notes to messages?  Many times I mark some
messages with important URLs or other information persistent and
ticked, but usually their subjects don't give any indication of the
reason why I saved them, so I can't later find them just by
interactively searching for something in the summary buffer.

Built-in full-text regexp search or using some external indexing
engine may help in some cases, but not always.  The reason why I
saved the article may not show from its contents or headers and my
brains associate information based on this kind of reasons.  So,
after I've read some article that I consider worth saving, I'd like
to press some key that would, for the speed's sake, do `*', `!'/`?'
and open a buffer for notes that I can then write about the
article.  These attached notes should be searchable so that if a
match is found, one could get to the article they are attached to.

-- 
Hannu

Re: Attaching notes to messages

[François Pinard]

Hannu Koivisto <azure@iki.fi> writes:

> Has anyone thought about or, even better, implemented some way of
> attaching arbitrary notes to messages?

I usually edit my notes within the messages themselves.  I do this really
all the time, and I suspect I may be the "biggest" message editor of the
whole `ding' list :-).  One very usual problem for me is that some backends
just do not allow message editing.

But your suggestion is quite enlightening, in that if Gnus was offering
ways to add not only annotations, but _also_ hide parts of any message,
or even have some per-message washing options, all of this without
touching the originals, maybe my need to edit all the time would decrease
dramatically, and I would be glad being able to ask Gnus to recover the
original unedited/unannotated message.

I guess that all these annotations should be taken under consideration while
preparing the quoted contents of a reply to a message, unless one first
visited the message without annotations, something that should be possible.

Expiry, moving, searching, and other things handling articles would have
to cope with annotations as well.  So, despite the capability might be
very attractive (for me at least), it also means non-negligible work in
Gnus internals -- it might be a major undertaking.

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard

Re: Attaching notes to messages

[François Pinard]

Jonas Steverud <d4jonas@dtek.chalmers.se> écrit:

> > Expiry, moving, searching, and other things handling articles would have
> > to cope with annotations as well.  So, despite the capability might
> > be very attractive (for me at least), it also means non-negligible
> > work in Gnus internals -- it might be a major undertaking.

> One idea: For backends that allow writing, e.g. nnfolder, nnmh and nnml,
> we can add an "X-Gnus-Annotaion: 5673762".  The problem is for backends
> like nntp that doesn't allow writing.

Annotations should be individual, maybe more related to `.newsrc' (or
`.newsrc.eld') than articles themselves.  The main idea is to leave
original articles alone, so I do not impose my own annotations to other
people reading the same spool.  This is less necessary for `nnml', which is
often personal by nature, but yet, the same principles could be used there.

Just to insist a bit, it looks essential to me that if we have annotations,
these should be able to subtract parts of the messages, rendering them
invisibly, and not just add text.

Maybe that all message rendering could be rethought around annotations.
All washing/hiding functions would merely generate ephemeral annotations :-)

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard

Re: Attaching notes to messages

[Eric Marsden]

>>>>> "fp" == François Pinard <pinard@iro.umontreal.ca> writes:

  fp> Annotations should be individual, maybe more related to
  fp> `.newsrc' (or `.newsrc.eld') than articles themselves. The main
  fp> idea is to leave original articles alone, so I do not impose my
  fp> own annotations to other people reading the same spool. This is
  fp> less necessary for `nnml', which is often personal by nature,
  fp> but yet, the same principles could be used there.

yes, using an nnpg backend which stores annotations and other metadata
in a PostgreSQL database. Sure, Postgres is not as nice as PLOB! but
it does have some OO features, it's free software, widely used,
there's even a (&/%¤#) elisp interface to it :-)

   <URL:http://www.chez.com/emarsden/downloads/pg.el>
  
-- 
   L'avantage du fromage sur les américains, 
   c'est qu'il y a une culture dedans. 
   -+- MZ in: Guide du Cabaliste Usenet - chapitre 9 - le gros 8 -+-

storing messages in DBMSes (Was: Attaching notes to messages)

[Steinar Bang]

>>>>> Eric Marsden <emarsden@mail.dotcom.fr>:

> yes, using an nnpg backend which stores annotations and other metadata
> in a PostgreSQL database. Sure, Postgres is not as nice as PLOB! but
> it does have some OO features, it's free software, widely used,
> there's even a (&/%¤#) elisp interface to it :-)

>    <URL:http://www.chez.com/emarsden/downloads/pg.el>

Ah!  This is only the elisp interface to Postgres.  Presumable the
nnpg backend doesn't exist yet.

But that makes me wonder: with the scaling problems all folders seem
to have, has anyone ever thought about using database systems to store 
the messages themselves?

Re: storing messages in DBMSes (Was: Attaching notes to messages)

[Eric Marsden]

>>>>> "sb" == Steinar Bang <sb@metis.no> writes:

  sb> Ah! This is only the elisp interface to Postgres. Presumable the
  sb> nnpg backend doesn't exist yet.

indeed. Just hoping to inspire someone to write it :-)
  

  sb> But that makes me wonder: with the scaling problems all folders
  sb> seem to have, has anyone ever thought about using database
  sb> systems to store the messages themselves?

I'm not very familiar with how Gnus backends work, but I doubt that it
would be worth trying to store messages directly in a DBMS. PostgreSQL
at least can only store 8k-long text fields; anything longer needs to
go in a BLOB.

Standard filesystems are probably good enough for this (for example,
qmail gets very high performance for its message queue from standard
filesystems), but maybe SQL would be useful for overview files.

-- 
Eric Marsden
It's elephants all the way down

Re: storing messages in DBMSes (Was: Attaching notes to messages)

[Fabrice POPINEAU]

It has been mentionned on the xemacs-nt list that gnus under xemacs (who has a
native support for gdbm) could make use of gdbm instead of .overview
files.

-- 
Fabrice Popineau

Re: Attaching notes to messages

[Kai Großjohann]

Jonas Steverud <d4jonas@dtek.chalmers.se> writes:

> One idea: For backends that allow writing, e.g. nnfolder, nnmh and
> nnml, we can add an "X-Gnus-Annotaion: 5673762". The problem is for
> backends like nntp that doesn't allow writing.

Well, we have the cache, and it ought to be possible to edit the
cached copy of a message, right?

OT1H, it seems to be a good idea to keep the annotation information in
the .newsrc.eld file, but I'm thinking of nnimap users where all state
information is supposed to be kept on the server.  Hm.  With article
editing, we could create a special nnimap group for annotations and
put them there.  The X-Gnus-Annotation header would then contain the
identifier of the corresponding article in the annotation group.

Sadly, both choices seem to be less than ideal :-(

kai
-- 
Life is hard and then you die.

Re: Attaching notes to messages

[Harry Putnam]

Hannu Koivisto <azure@iki.fi> writes:

> Greetings,
> 
> Has anyone thought about or, even better, implemented some way of
> attaching arbitrary notes to messages?  Many times I mark some
> messages with important URLs or other information persistent and
> ticked, but usually their subjects don't give any indication of the
> reason why I saved them, so I can't later find them just by
> interactively searching for something in the summary buffer.

This may be too amateurish for what you are after but it is a method
I've used for some time on groups where 'gnus-edit-article' is
allowed.

Editing is also possibe in News groups with  `C-u e'

Add a "Keywords:" header where you can put brief comments or single
words that will allow you to grep that topic later.
(G for gnus)

Keywords: $TOPIC  edit articles attach notes annotate messages

Where $TOPIC is a few characters.

You can then easily scan the "Keywords" lines with grep or rgrep,
or get a file number to use in gnus summary buffer with "j".

rgrep -r "^Keywords: $TOPIC" ~/Mail
To see all keyword lines for that topic.

A more developed method is possible esp. if you don't care to save
the original form.  But even if you do, below should be getting close
to what you describe.

I have code pieced together with help from Kai and Lars (the
botchiness is all mine) That allows the user to save an edited message
from  any gnus editing mode to any group with a few key strokes. And
leaves the original in place untouched.

It also allows a way to do this:

Press "F" or "R" on a message you wish to keep and annotate

Type in your annotation between [] or similar, then save that portion
to file with M-x save-to-file.

Remove the annotation with C-w then attach it with 'C-c a'

Save the Message plus attachment (to desired group) with the code I
mentioned above then kill the edit buffer.

The result of all this is an annotation attached to a copy of original
message, saved in a special group.  While leaving the original message
untouched in its group.

I'll post the code on a web site or here if you'd like to see if its
useful to you.

Re: Attaching notes to messages

[Hannu Koivisto]

Harry Putnam <reader@newsguy.com> writes:

| Editing is also possibe in News groups with  `C-u e'

Hmm, what exactly happens in this case?  Does the article get saved
to a local disk like in the case of marking it persistent?

| I have code pieced together with help from Kai and Lars (the
| botchiness is all mine) That allows the user to save an edited message
| from  any gnus editing mode to any group with a few key strokes. And
| leaves the original in place untouched.
| 
| It also allows a way to do this:
| 
| Press "F" or "R" on a message you wish to keep and annotate
| 
| Type in your annotation between [] or similar, then save that portion
| to file with M-x save-to-file.
| 
| Remove the annotation with C-w then attach it with 'C-c a'

Are these SAVE-TO-FILE and `C-c a' things part of your code?  I
can't see the connection and, on the other hand, they are not
default stuff.

| Save the Message plus attachment (to desired group) with the code I
| mentioned above then kill the edit buffer.
| 
| The result of all this is an annotation attached to a copy of original
| message, saved in a special group.  While leaving the original message
| untouched in its group.

As far as I can see, this doesn't provide any way to get to the
annotation from the message...

| I'll post the code on a web site or here if you'd like to see if its
| useful to you.

...so I don't think so.  What you suggested first and what Jonas
Steverund also suggested, i.e. editing the message and adding some
`X-Gnus-Annotation: foob' line, might be considerable, but first I
want to know what happens to by-force -edited news articles.
Although it still feels a bit hacky, I think this sounds the best
way to go at least if I manage to wrap up some functions to ease
using it.  Oh well, one can't get everything.

-- Warning! Dont continue further; what follows is just
   venting of my manic-depressive mind

What I really would like to do is to write a annotated-article
mixin class for article class and put slots for it for special
information and one for arbitrary text and be done with it (well,
of course I'd have add method to article display protocol too to
make Gnus display notes on a separate window along the actual
article and perhaps to printing protocol to make Gnus print these
annotations with the article).  I would not have to take care of
ugly saving details based on whether some backend by default stores
the actual article content on the local disk or somewhere else;
article objects would just end up to a PLOB!-like object database,
along with the article text or just a reference to nntp or imap
server.

But oooh no, I'm stuck to this freaking alternative reality that
somehow splitted up somewhere in the '80s and everything just
started going backwards.  I'm stuck to this pale shade of Lisp OS
called Emacs based on this &/%¤# Elisp and C running on top of this
everything-is-text Unix-lookalike, kept together with chewing-gum
and Perl (that looks like cat's vomit and leads to even more
disgusting chewing-gum-and-Perl software).  On this Emacs I run
Gnus that I need to turn upside down every time I want to even
consider adding some small extension, and in this case all I would
get is a text file, at the corner of some stupid filesystem with
close to zero accessibility on different machines I use, with no
semantic content and associations to my other personal information
anyway.  "Fortunately" I don't have time to do Gnus hacking right
now as after about 7h sleep I need to wake up to write AI code that
I eventually need to translate from CL to C++, which is a
requirement on our course.  Not to mention that for the evening
I've scheduled myself to write Java for a course whose lecturer
looks world through C++/Java-glasses and preaches about design
patterns that really are just a service pack to get around C++'s
limitations, just like there are oodles of services packs and small
tune-this-and-that utils to give a feeling that one has momentarily
got around the brokeness of MS' software.  At some point this whole
tower of crap will collapse and I want to be somewhere else then.

*phew*.  Did I mention that I need a holiday? :)

-- 
Hannu
(loop (luuk |Pink Floyd|:|Sorrow| :loud t))

Re: Attaching notes to messages

[Harry Putnam]

Hannu Koivisto <azure@iki.fi> writes:

> Harry Putnam <reader@newsguy.com> writes:
> 
> | Editing is also possibe in News groups with  `C-u e'
> 
> Hmm, what exactly happens in this case?  Does the article get saved
> to a local disk like in the case of marking it persistent?

It just allows the user to Edit a News article and then (manually) save
it to whatever, nnml group etc.  It saves the step of having to save it
first then move to group where saved to edit.

> |. . . . . . . . . . . . .  then save that portion
> | to file with M-x save-to-file.
> |                  ^^^^^^^^^^^^^
                     append to file

> | Remove the annotation with C-w then attach it with 'C-c a'
>                                                       ^^^^^
                                                        C-c C-a
> Are these SAVE-TO-FILE and `C-c a' things part of your code?  I
> can't see the connection and, on the other hand, they are not
> default stuff.
> 
Read M-x append-to-file .... and C-c C-a  The normal keycombo  to
attach a file when in edit mode.

> | Save the Message plus attachment (to desired group) with the code I
> | mentioned above then kill the edit buffer.
> | 
> | The result of all this is an annotation attached to a copy of original
> | message, saved in a special group.  While leaving the original message
> | untouched in its group.
> 
> As far as I can see, this doesn't provide any way to get to the
> annotation from the message... 

Press "i" over the button.

This was only a way to attach memos to a message without altering the
message. I never do this.  Better to put the edits in the message.
I've never needed a pristine original so far, so don't mind typing all
I want into the message itself.

However, with the method I use the original remains unchanged and in
place until an expiry rule gets to it.

> want to know what happens to by-force -edited news articles.

I think they dissappear when group is re-entered or message is closed
then opened after saving edit, if not saved in some way.  

The utility of it is that it allows you to make your edit *before*
saving to disk.  It illiminates a step in the save, edit, save
cycle. It becomes -- edit, save

> Although it still feels a bit hacky, I think this sounds the best
> way to go at least if I manage to wrap up some functions to ease
> using it.  Oh well, one can't get everything.
> 

[...]  snipped fierce rant

> 
> *phew*.  Did I mention that I need a holiday? :)

Use "nn" for six weeks    : )