Re: Failing to write to history file if containing directory doesn't exist

Re: Failing to write to history file if containing directory doesn't exist

[Matthew Wozniski]

On Mon, Oct 08, 2007 at 10:14:03AM -0400, Clint Adams wrote:
> Personally, I'd be a little shocked and annoyed if zsh started creating
> directories without my explicit permission.

I don't know that I'd be shocked, but I would err on the side of
a smaller, faster, more maintainable shell, given how easy to handle
in shell code the problem is.  Shell code is more easily maintained
than C, and for an error condition that should only exist when the
HISTFILE parameter is set, I think adding code to the shell itself
would be overkill.  If you're already setting HISTFILE yourself, it's
not too much extra work to add an mkdir -p above it.

~Matt

Re: Failing to write to history file if containing directory doesn't exist

[Nikolai Weibull]

On 10/8/07, Matthew Wozniski <godlygeek@gmail.com> wrote:

> On Mon, Oct 08, 2007 at 10:14:03AM -0400, Clint Adams wrote:

> > Personally, I'd be a little shocked and annoyed if zsh started creating
> > directories without my explicit permission.

> I don't know that I'd be shocked, but I would err on the side of
> a smaller, faster, more maintainable shell, given how easy to handle
> in shell code the problem is.  Shell code is more easily maintained
> than C, and for an error condition that should only exist when the
> HISTFILE parameter is set, I think adding code to the shell itself
> would be overkill.  If you're already setting HISTFILE yourself, it's
> not too much extra work to add an mkdir -p above it.

And what if the directory gets deleted between the time HISTFILE is
set and the shell exits?

I'd rather err on the side of a more intelligent, "work hard for the
user"-type shell that doesn't let my data disappear out into oblivion
without doing its best to keep said data around.

Re: Failing to write to history file if containing directory doesn't exist

[Matthew Wozniski]

On Mon, Oct 08, 2007 at 05:30:08PM +0200, Nikolai Weibull wrote:
> On 10/8/07, Matthew Wozniski wrote:
> 
> > I don't know that I'd be shocked, but I would err on the side of
> > a smaller, faster, more maintainable shell, given how easy to handle
> > in shell code the problem is.  Shell code is more easily maintained
> > than C, and for an error condition that should only exist when the
> > HISTFILE parameter is set, I think adding code to the shell itself
> > would be overkill.  If you're already setting HISTFILE yourself, it's
> > not too much extra work to add an mkdir -p above it.
> 
> And what if the directory gets deleted between the time HISTFILE is
> set and the shell exits?
The only way that should ever happen is if the user removed (or moved)
the directory himself.  If that's the case, the shell should certainly
not recreate a directory that the user intentionally deleted, each and
every time the history is updated.  That would make it impossible for
a user with HISTFILE=~/.zsh/history to do something as simple as:

% mv .zsh .zsh-4.3.4
% ln -s .zsh-4.3.4 .zsh

without first unsetting and later setting HISTFILE, since the
directory would be recreated between the two commands.

> I'd rather err on the side of a more intelligent, "work hard for the
> user"-type shell that doesn't let my data disappear out into oblivion
> without doing its best to keep said data around.

But really, you're talking about recreating a directory that was just
deleted.  In this case we really ought to err on the principle of
least surprise.  After all, that directory didn't just get up and walk
away, someone had to delete it.  ;)

~Matt

Re: Failing to write to history file if containing directory doesn't exist

[Nikolai Weibull]

On 10/8/07, Matthew Wozniski <godlygeek@gmail.com> wrote:

> On Mon, Oct 08, 2007 at 05:30:08PM +0200, Nikolai Weibull wrote:

> > And what if the directory gets deleted between the time HISTFILE is
> > set and the shell exits?

> The only way that should ever happen is if the user removed (or moved)
> the directory himself.  If that's the case, the shell should certainly
> not recreate a directory that the user intentionally deleted, each and
> every time the history is updated.  That would make it impossible for
> a user with HISTFILE=~/.zsh/history to do something as simple as:
>
> % mv .zsh .zsh-4.3.4
> % ln -s .zsh-4.3.4 .zsh
>
> without first unsetting and later setting HISTFILE, since the
> directory would be recreated between the two commands.

Only under the condition that HISTFILE is written to between the two commands.

> > I'd rather err on the side of a more intelligent, "work hard for the
> > user"-type shell that doesn't let my data disappear out into oblivion
> > without doing its best to keep said data around.

> But really, you're talking about recreating a directory that was just
> deleted.  In this case we really ought to err on the principle of
> least surprise.  After all, that directory didn't just get up and walk
> away, someone had to delete it.  ;)

What if someone figured that it'd be easier to

% rm -r ~/.local/var/zsh

than to

% rm ~/.local/var/zsh/history

(if, for example, ~/.local/var/zsh contains other transient data to be removed).

I still want to play a "don't lose data without trying to save it"
card, but it seems I'm the only one playing cards of that suite.

Re: Failing to write to history file if containing directory doesn't exist

[Bart Schaefer]

On Oct 9, 12:32am, Nikolai Weibull wrote:
}
} > every time the history is updated.  That would make it impossible for
} > a user with HISTFILE=~/.zsh/history to do something as simple as:
} >
} > % mv .zsh .zsh-4.3.4
} > % ln -s .zsh-4.3.4 .zsh
} >
} > without first unsetting and later setting HISTFILE, since the
} > directory would be recreated between the two commands.
} 
} Only under the condition that HISTFILE is written to between the two
} commands.

Which will be always, if the SHARE_HISTORY and/or INC_APPEND_HISTORY
options are set.

} I still want to play a "don't lose data without trying to save it"
} card, but it seems I'm the only one playing cards of that suite.

History is only maintained for interactive shells, so you might get
more traction with an option to prompt the user to create the directory
and/or fix $HISTFILE when the file does not exist or is not writable.

Re: Failing to write to history file if containing directory doesn't exist

[Matthew Wozniski]

On Mon, Oct 08, 2007 at 06:44:45PM -0700, Bart Schaefer wrote:
> } Only under the condition that HISTFILE is written to between the two
> } commands.
> 
> Which will be always, if the SHARE_HISTORY and/or INC_APPEND_HISTORY
> options are set.

Exactly so.

> } I still want to play a "don't lose data without trying to save it"
> } card, but it seems I'm the only one playing cards of that suite.
> 
> History is only maintained for interactive shells, so you might get
> more traction with an option to prompt the user to create the directory
> and/or fix $HISTFILE when the file does not exist or is not writable.

Along that vein, and still leaning towards a solution in shell code,
might it be reasonable to put a precmd hook into the default
/etc/zshrc to check up on HISTFILE?  That seems like a compromise that
allows everyone to get it but for it still to be maintained in shell,
rather than c.

~Matt Wozniski

Re: Failing to write to history file if containing directory doesn't exist

[Nikolai Weibull]

On 10/9/07, Matthew Wozniski <godlygeek@gmail.com> wrote:

> On Mon, Oct 08, 2007 at 06:44:45PM -0700, Bart Schaefer wrote:

[I wrote:]

> > } Only under the condition that HISTFILE is written to between the two
> > } commands.

> > Which will be always, if the SHARE_HISTORY and/or INC_APPEND_HISTORY
> > options are set.

> Exactly so.

I was well aware of this.

> > } I still want to play a "don't lose data without trying to save it"
> > } card, but it seems I'm the only one playing cards of that suite.

> > History is only maintained for interactive shells, so you might get
> > more traction with an option to prompt the user to create the directory
> > and/or fix $HISTFILE when the file does not exist or is not writable.

That's a good suggestion.

> Along that vein, and still leaning towards a solution in shell code,
> might it be reasonable to put a precmd hook into the default
> /etc/zshrc to check up on HISTFILE?  That seems like a compromise that
> allows everyone to get it but for it still to be maintained in shell,
> rather than c.

Perhaps putting that in /etc/zlogout would work well enough.
Otherwise, zshexit might be the function to hook.  Of course, if you
use either sharehistory or incappendhistory, perhaps more frequent
checks are desirable.

Re: Failing to write to history file if containing directory doesn't exist

[Richard Hartmann]

On 09/10/2007, Matthew Wozniski <godlygeek@gmail.com> wrote:


> > History is only maintained for interactive shells, so you might get
> > more traction with an option to prompt the user to create the directory
> > and/or fix $HISTFILE when the file does not exist or is not writable.

If such an option were introduced, it could have potential security issues,
leaving a shell open that the user expects to close immediately (insert
CHECK_JOBS and HUP reference here).


> Along that vein, and still leaning towards a solution in shell code,
> might it be reasonable to put a precmd hook into the default
> /etc/zshrc to check up on HISTFILE?  That seems like a compromise that
> allows everyone to get it but for it still to be maintained in shell,
> rather than c.

Perhaps check when the shell starts and in the history write function.
precmd should be as light as possible.


Richard

Re: Failing to write to history file if containing directory doesn't exist

[Matthew Wozniski]

On Wed, Oct 17, 2007 at 01:16:23PM +0200, Richard Hartmann wrote:
> On 09/10/2007, Matthew Wozniski <godlygeek@gmail.com> wrote:
> > Along that vein, and still leaning towards a solution in shell code,
> > might it be reasonable to put a precmd hook into the default
> > /etc/zshrc to check up on HISTFILE?  That seems like a compromise that
> > allows everyone to get it but for it still to be maintained in shell,
> > rather than c.
> 
> Perhaps check when the shell starts and in the history write function.
> precmd should be as light as possible.

As Bart noted earlier in this thread, history writes occur once per
command anyway for users with SHARE_HISTORY or INC_APPEND_HISTORY set.
And a solution handled during a history write can't be handled
directly in shell code.  That's exactly what my suggestion was aimed
at avoiding.  The only advantage I can see to handling it in C would
be that it would add only one extra instruction for an unexceptional
write (if (errno != ...) ) instead of the full cost of a glob and
a parameter expansion, which are presumably much more expensive.  But,
either way, I'm still against this proposed feature; I still find it
to be unnecessary at best.

~Matt

Re: Failing to write to history file if containing directory doesn't exist

[Richard Hartmann]

On 17/10/2007, Matthew Wozniski <godlygeek@gmail.com> wrote:

> > Perhaps check when the shell starts and in the history write function.
> > precmd should be as light as possible.
>
> As Bart noted earlier in this thread, history writes occur once per
> command anyway for users with SHARE_HISTORY or INC_APPEND_HISTORY set.

Yah, that is why I worded it that way :)


> And a solution handled during a history write can't be handled
> directly in shell code.  That's exactly what my suggestion was aimed
> at avoiding.  The only advantage I can see to handling it in C would
> be that it would add only one extra instruction for an unexceptional
> write (if (errno != ...) ) instead of the full cost of a glob and
> a parameter expansion, which are presumably much more expensive.

Exactly. To me, the only clean solution would be a new option.


> But, either way, I'm still against this proposed feature; I still find it
> to be unnecessary at best.

I would not want to use it, either. My comments are more geared towards
'if one would do it, this could be a good way'.


An easy, light and scriptable solution that solves 99% of all cases has
already been hinted at:

export HISTDIR=foo
mkdir -p $HISTDIR || echo 'Woe is me!'
export HISTFILE=$HISTDIR/bar


Richard
>
> ~Matt
>

Re: Failing to write to history file if containing directory doesn't exist

[Nikolai Weibull]

On 10/17/07, Richard Hartmann <richih.mailinglist@gmail.com> wrote:

> On 17/10/2007, Matthew Wozniski <godlygeek@gmail.com> wrote:

> > But, either way, I'm still against this proposed feature; I still find it
> > to be unnecessary at best.

> I would not want to use it, either. My comments are more geared towards
> 'if one would do it, this could be a good way'.

I still haven't seen a good reason for Zsh to not deal with this error
condition appropriately, besides worries that Zsh start creating
directories.  This argument fails, because we actually asked it to
store a file in that directory, so we probably won't mind if Zsh makes
sure it exists when it's needed.

> An easy, light and scriptable solution that solves 99% of all cases has
> already been hinted at:
>
> export HISTDIR=foo
> mkdir -p $HISTDIR || echo 'Woe is me!'
> export HISTFILE=$HISTDIR/bar

I'm sorry, but how does it solve 99% of all cases?  There are two cases:

1.  $HISTFILE:h exists when writing $HISTFILE
2.  $HISTFILE:h does not exist when writing $HISTFILE

The above solution makes case 1 more likely, sure, but it doesn't solve case 2.

Re: Failing to write to history file if containing directory doesn't exist

[Matthew Wozniski]

On Wed, Oct 17, 2007 at 03:24:16PM +0200, Nikolai Weibull wrote:
> On 10/17/07, Richard Hartmann <richih.mailinglist@gmail.com> wrote:
> > An easy, light and scriptable solution that solves 99% of all cases has
> > already been hinted at:
> >
> > export HISTDIR=foo
> > mkdir -p $HISTDIR || echo 'Woe is me!'
> > export HISTFILE=$HISTDIR/bar
> 
> I'm sorry, but how does it solve 99% of all cases?  There are two cases:
> 
> 1.  $HISTFILE:h exists when writing $HISTFILE
> 2.  $HISTFILE:h does not exist when writing $HISTFILE
> 
> The above solution makes case 1 more likely, sure, but it doesn't solve case 2.

I don't believe case 2 to be a problem.  That directory didn't just
disappear, it was removed.  You told zsh to write to $HISTFILE, then
you removed $HISTFILE:h, and now zsh can't write to $HISTFILE anymore:
in what way is that behavior unexpected or undesirable?  I still don't
see it.

~Matt

Re: Failing to write to history file if containing directory doesn't exist

[Duncan Sinclair]

[-- Attachment #1: Type: text/plain, Size: 1195 bytes --]

>> An easy, light and scriptable solution that solves 99% of all  
>> cases has
>> already been hinted at:
>>
>> export HISTDIR=foo
>> mkdir -p $HISTDIR || echo 'Woe is me!'
>> export HISTFILE=$HISTDIR/bar
>
> I'm sorry, but how does it solve 99% of all cases?  There are two  
> cases:
>
> 1.  $HISTFILE:h exists when writing $HISTFILE
> 2.  $HISTFILE:h does not exist when writing $HISTFILE
>
> The above solution makes case 1 more likely, sure, but it doesn't  
> solve case 2.

I think that the proposed code to create directories violates the  
principle of least surprise, and should not be included in the  
standard zsh distribution.

The reality is that in 99.99% of cases $HISTFILE:h is going to be  
someone's home directory.  If that doesn't exist you're pretty much  
screwed anyway.

Then in the 0.01% of cases where people put their $HISTFILE  
elsewhere, the code above, placed in their .zshenv file will work  
99.99% of the time.

So I'd say that 99.9999% of cases are covered.

In other words, it's a one in a million thing.


-- 
Duncan Sinclair  |  {+44|0}141 548 3095  |  cis.strath.ac.uk
System Administrator
Computer and Information Sciences
University of Strathclyde




[-- Attachment #2: Type: text/html, Size: 3662 bytes --]