note metadata for files

note metadata for files

[shawn wilson]

Maybe not a zsh specific question, but...

Does anyone have a system for attaching notes to a file?

The issue is I track down why a change was made to a file
(annotate/blame/whatever) and then I remember that I've tracked down
that change before. It would be useful if I could attach a note to
files outside of any repo (or the repo they belong to anyway) that
pops something up (maybe desktop-notify) when I look at the file (not
stat, but manually grep/cat/less/diff/whatever from a cli).

This could almost be a shell script, but I'm not sure how to make an
event that would trigger the script with a filename parameter - is
there a way to do this?

Re: note metadata for files

[Roman Neuhauser]

# ag4ve.us@gmail.com / 2014-07-02 13:59:24 -0400:
> Maybe not a zsh specific question, but...
> 
> Does anyone have a system for attaching notes to a file?

git, mercurial and others
 
> The issue is I track down why a change was made to a file
> (annotate/blame/whatever) and then I remember that I've tracked down
> that change before.

git notes might be useful

> It would be useful if I could attach a note to files outside of any
> repo

you can run git init anywhere, like in /etc.

> (or the repo they belong to anyway) that pops something up (maybe
> desktop-notify) when I look at the file (not stat, but manually
> grep/cat/less/diff/whatever from a cli).
> 
> This could almost be a shell script, but I'm not sure how to make an
> event that would trigger the script with a filename parameter - is
> there a way to do this?

see preexec in zshmisc(1), you can tokenize the first argument with
"${(@Z:C:)1}" (zshezpn(1)), and see if any word in the resulting array
names the file you're interested in.

-- 
roman

Re: note metadata for files

[Peter Stephenson]

On Wed, 2 Jul 2014 13:59:24 -0400
shawn wilson <ag4ve.us@gmail.com> wrote:
> Maybe not a zsh specific question, but...
> 
> Does anyone have a system for attaching notes to a file?

You might be interested in Oliver's article on using Linux file
attributes for metadata with zsh:

http://www.linux-mag.com/id/2126/

pws

Re: note metadata for files

[Mikael Magnusson]

On 2 July 2014 19:59, shawn wilson <ag4ve.us@gmail.com> wrote:
> Maybe not a zsh specific question, but...
>
> Does anyone have a system for attaching notes to a file?
>
> The issue is I track down why a change was made to a file
> (annotate/blame/whatever) and then I remember that I've tracked down
> that change before. It would be useful if I could attach a note to
> files outside of any repo (or the repo they belong to anyway) that
> pops something up (maybe desktop-notify) when I look at the file (not
> stat, but manually grep/cat/less/diff/whatever from a cli).
>
> This could almost be a shell script, but I'm not sure how to make an
> event that would trigger the script with a filename parameter - is
> there a way to do this?

Closest I have is
http://mika.l3ib.org/code/zsh-functions/note
http://cgit.mika.l3ib.org/cgit/zsh-cvs/commit/?h=mika&id=45550f3971d50f8aa0ba63b1a55ce6dd28662200
http://cgit.mika.l3ib.org/cgit/zsh-cvs/commit/?h=mika&id=2d0d2e6750383151551b62980f322ac90de07ec4

zstyle ':completion:dirprop:*' file-list dirprop=user.notes
zstyle ':completion:dirprop:*' dirinfo-format '%f '${(%):-%F{11}}%i${(%):-%f}
zle -C dirprop complete-word _generic
bindkey "^_^V"    dirprop

(or just set the style for your default context to always disable the
notes on tab completion, but it overrides filetype coloring which is
why i don't).

Also note that most programs will silently delete all xattrs when you
edit/copy/etc a file.

-- 
Mikael Magnusson

Re: note metadata for files

[shawn wilson]

Thanks all. I didn't know of hooks... I looked for something like this
in zsh a few years ago and didn't find anything (so didn't think to
look for that). I also like using xattr like that way - I'm going to
have to look into what can overwrite this though - if vim or cvs or
even touch can overwrite this, that's kinda a non-starter unless I can
find a way around it with cgroup or an LSM.

On Thu, Jul 3, 2014 at 5:44 PM, Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
> On Wed, 2 Jul 2014 13:59:24 -0400
> shawn wilson <ag4ve.us@gmail.com> wrote:
>> Maybe not a zsh specific question, but...
>>
>> Does anyone have a system for attaching notes to a file?
>
> You might be interested in Oliver's article on using Linux file
> attributes for metadata with zsh:
>
> http://www.linux-mag.com/id/2126/
>
> pws

Re: note metadata for files

[François Revol]

On 04/07/2014 05:55, shawn wilson wrote:
> Thanks all. I didn't know of hooks... I looked for something like this
> in zsh a few years ago and didn't find anything (so didn't think to
> look for that). I also like using xattr like that way - I'm going to
> have to look into what can overwrite this though - if vim or cvs or
> even touch can overwrite this, that's kinda a non-starter unless I can
> find a way around it with cgroup or an LSM.
> 
> On Thu, Jul 3, 2014 at 5:44 PM, Peter Stephenson
> <p.w.stephenson@ntlworld.com> wrote:
>> On Wed, 2 Jul 2014 13:59:24 -0400
>> shawn wilson <ag4ve.us@gmail.com> wrote:
>>> Maybe not a zsh specific question, but...
>>>
>>> Does anyone have a system for attaching notes to a file?
>>
>> You might be interested in Oliver's article on using Linux file
>> attributes for metadata with zsh:
>>
>> http://www.linux-mag.com/id/2126/

As for xattrs, you'll want to check the XDG guidelines instead of
redefining your own:

http://www.freedesktop.org/wiki/CommonExtendedAttributes/

Also, keep in mind xattrs are anything but standardized (OSX has a
different API and namespace layout (reverse DNS vs
{user.*,system.*...}), Windows (NTFS) has both xattrs and named streams...

In addition of the issues mentionned (applications forget about them
when moving things around), many filesystems either don't support them,
or use their own mangling scheme, which aren't idempotent.
And of course, if most kernels now finally support them out of the box,
in many cases the filesystems are still not mounted with them enabled by
default.

It's really a shame because xattrs can be very useful. BeOS and Haiku
use them extensively (for storing MIME type, icons, email headers...).

In case you're interested in the xattr compatibility issue, I've written
a paper about this:
http://dcevents.dublincore.org/IntConf/dc-2011/paper/view/53
Sadly it seems nobody cares.

François.

Re: note metadata for files

[shawn wilson]

So, I'm pushing this more into the realm of OT here (ie, this no
longer has anything to do with zsh and for this reason and possibly
others, this discussion might want to go offlist)

On Fri, Jul 4, 2014 at 6:50 AM, François Revol <revol@free.fr> wrote:
> On 04/07/2014 05:55, shawn wilson wrote:

>> On Thu, Jul 3, 2014 at 5:44 PM, Peter Stephenson
>> <p.w.stephenson@ntlworld.com> wrote:
>>> On Wed, 2 Jul 2014 13:59:24 -0400
>>> shawn wilson <ag4ve.us@gmail.com> wrote:
>>>> Maybe not a zsh specific question, but...
>>>>
>>>> Does anyone have a system for attaching notes to a file?
>>>
>>> You might be interested in Oliver's article on using Linux file
>>> attributes for metadata with zsh:
>>>
>>> http://www.linux-mag.com/id/2126/
>
> As for xattrs, you'll want to check the XDG guidelines instead of
> redefining your own:
>
> http://www.freedesktop.org/wiki/CommonExtendedAttributes/
>

Heh, I had no idea there was a standard (kinda your point I guess) :)

> In case you're interested in the xattr compatibility issue, I've written
> a paper about this:
> http://dcevents.dublincore.org/IntConf/dc-2011/paper/view/53
> Sadly it seems nobody cares.

Hmmmm, so how far has your research gone here? What software ignores
xattr on which fs (how well does something like ncase deal with it)?
What security mechanisms rely on EAs and have you tried to mangle them
in order to bypass these mechanisms (either OS or application
policies)? I'd be more interested in the networked ramifications of
mangled EAs, but a copy from a thumb drive might be interesting as
well (though I doubt that would be very fruitful since moving into a
higher security realm is a bit harder in kernel space).

Re: note metadata for files

[François Revol]

On 07/07/2014 00:32, shawn wilson wrote:
> So, I'm pushing this more into the realm of OT here (ie, this no
> longer has anything to do with zsh and for this reason and possibly
> others, this discussion might want to go offlist)
[...]
>> In case you're interested in the xattr compatibility issue, I've written
>> a paper about this:
>> http://dcevents.dublincore.org/IntConf/dc-2011/paper/view/53
>> Sadly it seems nobody cares.
> 
> Hmmmm, so how far has your research gone here? What software ignores
> xattr on which fs (how well does something like ncase deal with it)?

Well, even things like /bin/cp -p (--preserve) by default only preserves
mode,owner‐ship,timestamps but not xattr...

I think rsync supports the linux xattrs, probably not enabled by default
again, but I didn't try yet.

Don't know about ncase.

> What security mechanisms rely on EAs and have you tried to mangle them
> in order to bypass these mechanisms (either OS or application
> policies)?

I didn't really consider this question, given Samba alone already has a
hard time enough converting the ACLs.

Note, xattrs on Linux (and the now-withdrawn POSIX draft about xattrs)
have only been done on purpose of implementing ACLs atop.

> I'd be more interested in the networked ramifications of
> mangled EAs, but a copy from a thumb drive might be interesting as
> well (though I doubt that would be very fruitful since moving into a
> higher security realm is a bit harder in kernel space).

Besides, things like Samba and NTFS-3g differ (at least used to) on the
way the mangle named-stream inside xattrs...

François.