Dangerous feature - multios with globbing

Dangerous feature - multios with globbing

[Nadav Har'El]

[-- Attachment #1: Type: text/plain, Size: 1565 bytes --]

Hi, I've been a very happy zsh user for the last 30 years (yes, really :-)
before that, I had used ksh).
Today I stumbled across a dangerous feature that I have never noticed
before. I *wanted* to type:

     xv *

In a directory with 4000 photos, to view them.
But by mistake, I typed:

    xv > *

Note the extra ">".

This mistake should have been benign - ksh says in this case "ksh: *:
Ambiguous", and bash says "bash: *: ambiguous redirect", and creating a
file called "*" would also be fine. But I wasn't that lucky :-( What
actually happened was that all my files - 4000 of them - were truncated to
size 0!

It turns out that this dangerous feature is actually documented in the zsh
manual page:

       If  the MULTIOS option is set, the word after a redirection operator
is
       also subjected to filename generation (globbing).  Thus
              : > *
       will truncate all files in the current directory, assuming  there's
 at
       least  one.  (Without the MULTIOS option, it would create an empty
file
       called `*'.)

But the fact that this behavior is documented doesn't make it any less
dangerous, and to be honest - it makes absolutely no sense to me.

I quickly added "set +o multios" in my ~/.zshrc so this will never happen
to me again - at the cost of losing the multios feature completely.
But maybe just this multios-plus-globbing feature should be disabled (or
made a separate option, disabled by default) so that future zsh users don't
loose 4000 of their photos, or don't need to disable the multios feature?

Thanks,
Nadav.

[-- Attachment #2: Type: text/html, Size: 2026 bytes --]

Re: Dangerous feature - multios with globbing

[Mikael Magnusson]

On Sat, Sep 21, 2024 at 6:26 PM Nadav Har'El <nyharel@gmail.com> wrote:
>
> Hi, I've been a very happy zsh user for the last 30 years (yes, really :-) before that, I had used ksh).
> Today I stumbled across a dangerous feature that I have never noticed before. I *wanted* to type:
>
>      xv *
>
> In a directory with 4000 photos, to view them.
> But by mistake, I typed:
>
>     xv > *
>
> Note the extra ">".
>
> This mistake should have been benign - ksh says in this case "ksh: *: Ambiguous", and bash says "bash: *: ambiguous redirect", and creating a file called "*" would also be fine. But I wasn't that lucky :-( What actually happened was that all my files - 4000 of them - were truncated to size 0!
>
> It turns out that this dangerous feature is actually documented in the zsh manual page:
>
>        If  the MULTIOS option is set, the word after a redirection operator is
>        also subjected to filename generation (globbing).  Thus
>               : > *
>        will truncate all files in the current directory, assuming  there's  at
>        least  one.  (Without the MULTIOS option, it would create an empty file
>        called `*'.)
>
> But the fact that this behavior is documented doesn't make it any less dangerous, and to be honest - it makes absolutely no sense to me.
>
> I quickly added "set +o multios" in my ~/.zshrc so this will never happen to me again - at the cost of losing the multios feature completely.
> But maybe just this multios-plus-globbing feature should be disabled (or made a separate option, disabled by default) so that future zsh users don't loose 4000 of their photos, or don't need to disable the multios feature?

Another option is setopt NOCLOBBER, then you would have gotten this result:
% true > *
zsh: file exists: a
(unless you accidentally type >! * sometimes too)

-- 
Mikael Magnusson

Re: Dangerous feature - multios with globbing

[Eric Cook]

On 9/21/24 12:25 PM, Nadav Har'El wrote:
> Hi, I've been a very happy zsh user for the last 30 years (yes, really :-) before that, I had used ksh).
> Today I stumbled across a dangerous feature that I have never noticed before. I *wanted* to type:
>
>       xv *
>
> In a directory with 4000 photos, to view them.
> But by mistake, I typed:
>
>      xv > *
>
> Note the extra ">".
>
> This mistake should have been benign - ksh says in this case "ksh: *: Ambiguous", and bash says "bash: *: ambiguous redirect", and creating a file called "*" would also be fine. But I wasn't that lucky :-( What actually happened was that all my files -
> 4000 of them - were truncated to size 0!
>
> It turns out that this dangerous feature is actually documented in the zsh manual page:
>
>         If  the MULTIOS option is set, the word after a redirection operator is
>         also subjected to filename generation (globbing).  Thus
>                : > *
>         will truncate all files in the current directory, assuming  there's  at
>         least  one.  (Without the MULTIOS option, it would create an empty file
>         called `*'.)
>
> But the fact that this behavior is documented doesn't make it any less dangerous, and to be honest - it makes absolutely no sense to me.
>
> I quickly added "set +o multios" in my ~/.zshrc so this will never happen to me again - at the cost of losing the multios feature completely.
> But maybe just this multios-plus-globbing feature should be disabled (or made a separate option, disabled by default) so that future zsh users don't loose 4000 of their photos, or don't need to disable the multios feature?
>
> Thanks,
> Nadav.

While indifferent to the option default state and it leading to other more common encountered confusion, the fact that it took you
30 years to discover the behavior on accident speaks to how likely/dangerous one is to actually occur said problem.

Re: Dangerous feature - multios with globbing

[Bart Schaefer]

On Sat, Sep 21, 2024 at 9:25 AM Nadav Har'El <nyharel@gmail.com> wrote:
>
> But by mistake, I typed:
>
>     xv > *
>
> What actually happened was that all my files - 4000 of them - were truncated to size 0!

It might be possible to make this particular construct issue a
"really?" prompt when CLOBBER + MULTIOS are set, similar to what "rm"
does when RM_STAR_SILENT is not set.

Re: Dangerous feature - multios with globbing

[Nadav Har'El]

On Sun, Sep 22, 2024 at 2:01 AM Eric Cook <llua@gmx.com> wrote:
> While indifferent to the option default state and it leading to other more common encountered confusion, the fact that it took you
> 30 years to discover the behavior on accident speaks to how likely/dangerous one is to actually occur said problem.

That's an optimistic way to look at it. A more pessimistic way is to
say that in the last 30 years, I lost an average of 133 files
per year because of that feature ;-)
Although this calculation sounds like a joke, it actually emphasises
my main point - that the globbing+multios+clobber
combination - on by default - is dangerous exactly because it can
destroy many files at once.

I still wonder who really wants the ability to clobber many files at
once. The manual page gives examples of truncating many
files, and of replacing many shell scripts with a single script. But
does any user ever want to do such a thing? Or, in the rare
case that you do, wouldn't using a loop be safer than the multios
syntax? It will also be more general - the multios syntax
tries to open all the files at once, so will fail when "*" evaluates
to more than the number of allowed file descriptors (ulimit -n).
This is how I actually lost in this incident "only" the first few
thousands files and not all the files I had in that directory.

Nadav.