zsh-users
 help / color / mirror / code / Atom feed
* Feature request: set extension for =( ) created file
@ 2016-09-24 17:51 Sebastian Gniazdowski
  2016-09-24 17:55 ` Vadim Zeitlin
  2016-09-24 18:22 ` Bart Schaefer
  0 siblings, 2 replies; 21+ messages in thread
From: Sebastian Gniazdowski @ 2016-09-24 17:51 UTC (permalink / raw)
  To: Zsh Users

Hello,
when e.g. editing a file directly from git:

vim =( git -C /home/user/github/proj.git cat-file blob 1e08920 )

One will not get proper syntax highlighting because the file will be
e.g.: /tmp/zshoF8Hlc, without extension like .java.

Best regards,
Sebastian Gniazdowski


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 17:51 Feature request: set extension for =( ) created file Sebastian Gniazdowski
@ 2016-09-24 17:55 ` Vadim Zeitlin
  2016-09-24 18:43   ` Lawrence Velázquez
  2016-09-24 18:56   ` Sebastian Gniazdowski
  2016-09-24 18:22 ` Bart Schaefer
  1 sibling, 2 replies; 21+ messages in thread
From: Vadim Zeitlin @ 2016-09-24 17:55 UTC (permalink / raw)
  To: Zsh Users

[-- Attachment #1: Type: TEXT/PLAIN, Size: 662 bytes --]

On Sat, 24 Sep 2016 19:51:26 +0200 Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:

SG> Hello,
SG> when e.g. editing a file directly from git:
SG> 
SG> vim =( git -C /home/user/github/proj.git cat-file blob 1e08920 )
SG> 
SG> One will not get proper syntax highlighting because the file will be
SG> e.g.: /tmp/zshoF8Hlc, without extension like .java.

 How could the shell possibly do anything that would be better than the
usual workaround of

	vim -c 'setf java' =(...)

? You would still need to specify the extension somehow and specifying the
file type for Vim directly is exactly as simple and doesn't require any
changes to the shell.

 Regards,
VZ

[-- Attachment #2: Type: APPLICATION/PGP-SIGNATURE, Size: 196 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 17:51 Feature request: set extension for =( ) created file Sebastian Gniazdowski
  2016-09-24 17:55 ` Vadim Zeitlin
@ 2016-09-24 18:22 ` Bart Schaefer
  1 sibling, 0 replies; 21+ messages in thread
From: Bart Schaefer @ 2016-09-24 18:22 UTC (permalink / raw)
  To: Zsh Users

On Sep 24,  7:51pm, Sebastian Gniazdowski wrote:
}
} vim =( git -C /home/user/github/proj.git cat-file blob 1e08920 )
} 
} One will not get proper syntax highlighting because the file will be
} e.g.: /tmp/zshoF8Hlc, without extension like .java.

How about this?

    vimtmp () {
      local filetype="$(file $1)"
      case "$filetype" in
      (*[Jj]ava*) filetype=java;;
      (*) : etc.
      esac
      vim -c ":set filetype=$filetype" $1
    }

    vimtmp =( git -C /home/user/github/proj.git cat-file blob 1e08920 )

You can extend this with a cached mapping of "file" output to file types
and something to prompt/update the cache on unrecognized output.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 17:55 ` Vadim Zeitlin
@ 2016-09-24 18:43   ` Lawrence Velázquez
  2016-09-24 19:30     ` Sebastian Gniazdowski
  2016-09-24 18:56   ` Sebastian Gniazdowski
  1 sibling, 1 reply; 21+ messages in thread
From: Lawrence Velázquez @ 2016-09-24 18:43 UTC (permalink / raw)
  To: zsh-users

> On Sep 24, 2016, at 1:55 PM, Vadim Zeitlin <vz-zsh@zeitlins.org> wrote:
> 
> How could the shell possibly do anything that would be better than the
> usual workaround of
> 
> 	vim -c 'setf java' =(...)
> 
> ? You would still need to specify the extension somehow and specifying the
> file type for Vim directly is exactly as simple and doesn't require any
> changes to the shell.

I posit that it is not the shell's responsibility to guess file types
just so an editor will show the right colors.

vq


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 17:55 ` Vadim Zeitlin
  2016-09-24 18:43   ` Lawrence Velázquez
@ 2016-09-24 18:56   ` Sebastian Gniazdowski
  1 sibling, 0 replies; 21+ messages in thread
From: Sebastian Gniazdowski @ 2016-09-24 18:56 UTC (permalink / raw)
  To: Vadim Zeitlin; +Cc: Zsh Users

On 24 September 2016 at 19:55, Vadim Zeitlin <vz-zsh@zeitlins.org> wrote:
> On Sat, 24 Sep 2016 19:51:26 +0200 Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
>
> SG> Hello,
> SG> when e.g. editing a file directly from git:
> SG>
> SG> vim =( git -C /home/user/github/proj.git cat-file blob 1e08920 )
> SG>
> SG> One will not get proper syntax highlighting because the file will be
> SG> e.g.: /tmp/zshoF8Hlc, without extension like .java.
>
>  How could the shell possibly do anything that would be better than the
> usual workaround of
>
>         vim -c 'setf java' =(...)
>
> ? You would still need to specify the extension somehow and specifying the
> file type for Vim directly is exactly as simple and doesn't require any
> changes to the shell.
>

Yes coming up with how to pass extension to =( ) is a TODO. Maybe it
indeed isn't worth implementing, but IMO it's almost always the case
when using =( ) causes an inevitable drawback of feeding the program
with random file name which is somewhat a random data at the start.
Extensions are vital part of file names, what program doesn't use
them? In my script I use $EDITOR, not exactly vim so I will be
implementing a switch statement and supporting whatever editor there
is and is worth it – except NOT, will not do this, because it also
complicates the command that I'm pasting into BUFFER.

Best regards,
Sebastian Gniazdowski


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 18:43   ` Lawrence Velázquez
@ 2016-09-24 19:30     ` Sebastian Gniazdowski
  2016-09-24 19:55       ` Lawrence Velázquez
  0 siblings, 1 reply; 21+ messages in thread
From: Sebastian Gniazdowski @ 2016-09-24 19:30 UTC (permalink / raw)
  To: Lawrence Velázquez; +Cc: Zsh Users

On 24 September 2016 at 20:43, Lawrence Velázquez <larryv@macports.org> wrote:
>> On Sep 24, 2016, at 1:55 PM, Vadim Zeitlin <vz-zsh@zeitlins.org> wrote:
> I posit that it is not the shell's responsibility to guess file types
> just so an editor will show the right colors.

What vim does is a fancy ignorable thing for some, what other program
might do might not be. Extensions are vital parts of file names.

I'm doing git ls-tree and feed selected blob to $EDITOR =( ). Bam,
Github's "Browse files" implemented. Will be crying having to move to
mktemp or something different, =( ) "idiom" is great. But I rather
leave this as it is, user will rather not be offended when doing :w
~/extracted_file.java or :set ft=java. Extension-aware =( ) would be a
masterpiece feature for Zsh, though, IMO.

Best regards,
Sebastian Gniazdowski


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 19:30     ` Sebastian Gniazdowski
@ 2016-09-24 19:55       ` Lawrence Velázquez
  2016-09-24 20:14         ` Sebastian Gniazdowski
  2016-09-24 21:06         ` Bart Schaefer
  0 siblings, 2 replies; 21+ messages in thread
From: Lawrence Velázquez @ 2016-09-24 19:55 UTC (permalink / raw)
  To: Sebastian Gniazdowski; +Cc: zsh-users

> On Sep 24, 2016, at 3:30 PM, Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
> 
> What vim does is a fancy ignorable thing for some, what other program
> might do might not be. Extensions are vital parts of file names.

Not really. Any competently written program that cares about the
structure of its input has some other way of validating said structure,
like actually parsing it or checking for a file signature / header
/ magic number. Programs that process stdin already have to do this.

> Extension-aware =( ) would be a masterpiece feature for Zsh, though,
> IMO.

Unless it were implemented using libmagic or something, I would consider
it a massive waste of time and effort.

vq

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 19:55       ` Lawrence Velázquez
@ 2016-09-24 20:14         ` Sebastian Gniazdowski
  2016-09-24 20:36           ` Lawrence Velázquez
  2016-09-25  0:05           ` Nikolay Aleksandrovich Pavlov (ZyX)
  2016-09-24 21:06         ` Bart Schaefer
  1 sibling, 2 replies; 21+ messages in thread
From: Sebastian Gniazdowski @ 2016-09-24 20:14 UTC (permalink / raw)
  To: Lawrence Velázquez; +Cc: Zsh Users

On 24 September 2016 at 21:55, Lawrence Velázquez <larryv@macports.org> wrote:
>> On Sep 24, 2016, at 3:30 PM, Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
>>
>> What vim does is a fancy ignorable thing for some, what other program
>> might do might not be. Extensions are vital parts of file names.
>
> Not really. Any competently written program that cares about the
> structure of its input has some other way of validating said structure,
> like actually parsing it or checking for a file signature / header
> / magic number. Programs that process stdin already have to do this.

Yeah, one of such programs is Vim and it fails recognizing e.g.
ctags/awk.c as ft=zsh.

>> Extension-aware =( ) would be a masterpiece feature for Zsh, though,
>> IMO.
>
> Unless it were implemented using libmagic or something, I would consider
> it a massive waste of time and effort.

What time and effort? I guess it can be a beautiful few lines patch.
Modify name of file? Something like strcat. Parsing clever syntax that
would provide the extension? Also possibly basic.

Best regards,
Sebastian Gniazdowski


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 20:14         ` Sebastian Gniazdowski
@ 2016-09-24 20:36           ` Lawrence Velázquez
  2016-09-24 20:44             ` Sebastian Gniazdowski
  2016-09-25  0:05           ` Nikolay Aleksandrovich Pavlov (ZyX)
  1 sibling, 1 reply; 21+ messages in thread
From: Lawrence Velázquez @ 2016-09-24 20:36 UTC (permalink / raw)
  To: Sebastian Gniazdowski; +Cc: zsh-users

> On Sep 24, 2016, at 4:14 PM, Sebastian Gniazdowski
> <sgniazdowski@gmail.com> wrote:
> 
>> On 24 September 2016 at 21:55, Lawrence Velázquez
>> <larryv@macports.org> wrote:
>> 
>> Any competently written program that cares about the structure of its
>> input has some other way of validating said structure, like actually
>> parsing it or checking for a file signature / header / magic number.
>> Programs that process stdin already have to do this.
> 
> Yeah, one of such programs is Vim and it fails recognizing e.g.
> ctags/awk.c as ft=zsh.

Then vim's file detection logic needs work. (Or maybe your vim is not
configured correctly. I downloaded ctags literally 30 seconds ago and
opened awk.c, and my vim set filetype=c correctly.) Not zsh's problem.

>>> Extension-aware =( ) would be a masterpiece feature for Zsh,
>>> though, IMO.
>> 
>> Unless it were implemented using libmagic or something, I would
>> consider it a massive waste of time and effort.
> 
> What time and effort? I guess it can be a beautiful few lines patch.
> Modify name of file? Something like strcat. Parsing clever syntax that
> would provide the extension? Also possibly basic.

A few lines (of C!) to reliably detect dozens/hundreds/thousands of file
formats? Okay.

(FWIW, file(1) -- a whole program/library made expressly for this
purpose -- gives me hilariously bad results on a regular basis.)

vq

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 20:36           ` Lawrence Velázquez
@ 2016-09-24 20:44             ` Sebastian Gniazdowski
  2016-09-24 21:06               ` Lawrence Velázquez
  0 siblings, 1 reply; 21+ messages in thread
From: Sebastian Gniazdowski @ 2016-09-24 20:44 UTC (permalink / raw)
  To: Lawrence Velázquez; +Cc: Zsh Users

On 24 September 2016 at 22:36, Lawrence Velázquez <larryv@macports.org> wrote:
>> On Sep 24, 2016, at 4:14 PM, Sebastian Gniazdowski
>> <sgniazdowski@gmail.com> wrote:
>> Yeah, one of such programs is Vim and it fails recognizing e.g.
>> ctags/awk.c as ft=zsh.
>
> Then vim's file detection logic needs work. (Or maybe your vim is not
> configured correctly. I downloaded ctags literally 30 seconds ago and
> opened awk.c, and my vim set filetype=c correctly.) Not zsh's problem.

I looked at the temporary file and it has "zsh" in it, my fault. Vim
probably doesn't check for ".zshrc" / ".zshenv" but for *zsh*. But if
I did cp awk.c ~/O3R9qv; vim ~/O3R9qv then it didn't detect the file
type.

> A few lines (of C!) to reliably detect dozens/hundreds/thousands of file
> formats? Okay.
>
> (FWIW, file(1) -- a whole program/library made expressly for this
> purpose -- gives me hilariously bad results on a regular basis.)

If someone approaches this or if a green light will be given I might try

Best regards,
Sebastian Gniazdowski


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 19:55       ` Lawrence Velázquez
  2016-09-24 20:14         ` Sebastian Gniazdowski
@ 2016-09-24 21:06         ` Bart Schaefer
  1 sibling, 0 replies; 21+ messages in thread
From: Bart Schaefer @ 2016-09-24 21:06 UTC (permalink / raw)
  To: zsh-users

On Sep 24,  3:55pm, Lawrence Velazquez wrote:
}
} > Extension-aware =( ) would be a masterpiece feature for Zsh, though,
} > IMO.
} 
} Unless it were implemented using libmagic or something, I would consider
} it a massive waste of time and effort.

Hence my suggestion of a wrapper function that calls "file".

You can generalize my example to have a mapping from "file" output strings
to file extensions, and then do something like this:

appendfileext() {
  local filetype="$(file $1)"
  if [[ -n "${filextcache[$filetype]} ]]; then
    REPLY="$1.${filextcache[$filetype]}"
  else
    case "$filetype" in 
    (...) : you fill this in;;
    (*) unset REPLY;;
    esac
  fi
}

fixfiletype() {
  local REPLY
  integer i
  declare -a removes
  { 
    for ((i=1; i <= $#; ++i)); do
      if [[ "${argv[i]}" = "$TMPPREFIX"* ]]; then
        appendfileext "${argv[i]}"
        if [[ -n "$REPLY" ]];then
          removes+="$REPLY"
          ln "${argv[i]}" "$REPLY"
          argv[i]="$REPLY"
        fi
      fi
    done
    "${argv[@]}"
  } always {
    [[ -n "$removes" ]] && /bin/rm -f "${removes[@]}"
  }
}

alias vim='fixfiletype vim '


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 20:44             ` Sebastian Gniazdowski
@ 2016-09-24 21:06               ` Lawrence Velázquez
  2016-09-24 21:24                 ` Bart Schaefer
  0 siblings, 1 reply; 21+ messages in thread
From: Lawrence Velázquez @ 2016-09-24 21:06 UTC (permalink / raw)
  To: Sebastian Gniazdowski; +Cc: zsh-users

> On Sep 24, 2016, at 4:44 PM, Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
> 
> I looked at the temporary file and it has "zsh" in it, my fault. Vim
> probably doesn't check for ".zshrc" / ".zshenv" but for *zsh*.

Close.

https://github.com/vim/vim/blob/v8.0.0008/runtime/filetype.vim#L1964

vq

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 21:06               ` Lawrence Velázquez
@ 2016-09-24 21:24                 ` Bart Schaefer
  2016-09-24 21:58                   ` Lawrence Velázquez
  0 siblings, 1 reply; 21+ messages in thread
From: Bart Schaefer @ 2016-09-24 21:24 UTC (permalink / raw)
  To: zsh-users

On Sep 24,  4:36pm, Lawrence Velazquez wrote:
}
} > What time and effort? I guess it can be a beautiful few lines patch.
} > Modify name of file? Something like strcat. Parsing clever syntax that
} > would provide the extension? Also possibly basic.
} 
} A few lines (of C!) to reliably detect dozens/hundreds/thousands of file
} formats? Okay.

I think Sebastian is asking for a way to change the file name returned
by =(...) and still have it automatically removed, not a way to figure
out TO WHAT to change the name.

} (FWIW, file(1) -- a whole program/library made expressly for this
} purpose -- gives me hilariously bad results on a regular basis.)

Yes, "file" is generally a lot better at figuring out what a file is
NOT than what a file IS.  Still, it's the best thing we have lying
around.

On Sep 24,  5:06pm, Lawrence Velazquez wrote:
} Subject: Re: Feature request: set extension for =( ) created file
}
} > I looked at the temporary file and it has "zsh" in it, my fault. Vim
} > probably doesn't check for ".zshrc" / ".zshenv" but for *zsh*.
} 
} Close.

You can change the value of $TMPPREFIX to cause a different string to
be embedded in the file name, and a different location for the files
as well.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 21:24                 ` Bart Schaefer
@ 2016-09-24 21:58                   ` Lawrence Velázquez
  0 siblings, 0 replies; 21+ messages in thread
From: Lawrence Velázquez @ 2016-09-24 21:58 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: zsh-users

> On Sep 24, 2016, at 5:24 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> 
> On Sep 24,  4:36pm, Lawrence Velazquez wrote:
> } A few lines (of C!) to reliably detect dozens/hundreds/thousands of file
> } formats? Okay.
> 
> I think Sebastian is asking for a way to change the file name returned
> by =(...) and still have it automatically removed, not a way to figure
> out TO WHAT to change the name.

That sounds much more feasible, but how might the caller choose the new
temporary name?

> } (FWIW, file(1) -- a whole program/library made expressly for this
> } purpose -- gives me hilariously bad results on a regular basis.)
> 
> Yes, "file" is generally a lot better at figuring out what a file is
> NOT than what a file IS.  Still, it's the best thing we have lying
> around.

Yeah, I didn't mean to hate on file(1) so much as assert the difficulty
of the task.

vq

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-24 20:14         ` Sebastian Gniazdowski
  2016-09-24 20:36           ` Lawrence Velázquez
@ 2016-09-25  0:05           ` Nikolay Aleksandrovich Pavlov (ZyX)
  2016-09-25  5:17             ` Bart Schaefer
  1 sibling, 1 reply; 21+ messages in thread
From: Nikolay Aleksandrovich Pavlov (ZyX) @ 2016-09-25  0:05 UTC (permalink / raw)
  To: Sebastian Gniazdowski, Lawrence Velázquez; +Cc: Zsh Users

25.09.2016, 02:04, "Sebastian Gniazdowski" <sgniazdowski@gmail.com>:
> On 24 September 2016 at 21:55, Lawrence Velázquez <larryv@macports.org> wrote:
>>>  On Sep 24, 2016, at 3:30 PM, Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
>>>
>>>  What vim does is a fancy ignorable thing for some, what other program
>>>  might do might not be. Extensions are vital parts of file names.
>>
>>  Not really. Any competently written program that cares about the
>>  structure of its input has some other way of validating said structure,
>>  like actually parsing it or checking for a file signature / header
>>  / magic number. Programs that process stdin already have to do this.
>
> Yeah, one of such programs is Vim and it fails recognizing e.g.
> ctags/awk.c as ft=zsh.
>
>>>  Extension-aware =( ) would be a masterpiece feature for Zsh, though,
>>>  IMO.
>>
>>  Unless it were implemented using libmagic or something, I would consider
>>  it a massive waste of time and effort.
>
> What time and effort? I guess it can be a beautiful few lines patch.
> Modify name of file? Something like strcat. Parsing clever syntax that
> would provide the extension? Also possibly basic.
>
> Best regards,
> Sebastian Gniazdowski

I would consider another thing: while extensions are significant part of the file names, sometimes complete last path component or the whole path matters. I would suggest that if syntax is extended to allow controlling some properties of a temporary file name it should allow to control everything:

1. Provide just an extension: use names like `/tmp/zshUkxFmK.ext`.
2. Optionally provide full file name: use names like `/tmp/zshUkxFmK/name.ext`, in this case temporary directory is generated and then recursively deleted.
3. Provide full path: allow generating automatically removed temporary files at any selected location.
4. Optionally make this feature generate temporary directories and not files, if code for 2. is written this should be easy.

The key point is that I sometimes need temporary file in a specific location, but I almost never needed temporary file with a specific extension.

Currently `echo =(echo foo).bar` produces something like `/tmp/zshUkxFmK.bar` which is not very useful because this file does not exist. I would thus suggest extending syntax in one of the following ways:

1. `=(cmd).bar` produces the same thing it does now, but file does exist. `=(cmd):bar` results in `/tmp/zshUkxFmK/bar`. `=(cmd)=bar` uses `./bar` as a temporary file (`=(cmd)=/bar` uses `/bar`, of course).

   This is backwards incompatible: if some people use this “feature” (e.g. in Vim it is possible to implement BufReadCmd which reads `/tmp/zshUkxFmK` when receiving `/tmp/zshUkxFmK.bar`) their scripts will be broken.
2. Currently `=(cmd)(…)` acts like a usual glob expression: `=(cmd)` is replaced with `/tmp/zshUkxFmK` and then `(…)` applies (e.g. `=(cmd)(/)` will yield “zsh: no match” because produced file is not a directory). Since such syntax is an error in most cases I would suggest to reuse this: `=(cmd)(.bar)`: extension, `=(cmd)(bar)`: exact file name in /tmp, `=(cmd)(/bar)`: absolute file name, `=(cmd)(./bar)`: file name in the current directory. Specifically:

   - `({xxx})` is treated as a part of the file name under /tmp if first character in `{xxx}` is not a dot or slash
   - `(.{xxx})` is treated as an “extension” if first character in `{xxx}` is not a dot or slash
   - in all other cases it is treated like a regular glob expression like if it was after `>`; program will receive whatever glob was expanded to.

   As an implementation detail I would expect zsh to run parameter expansion, filename expansion and filename generation and match the result according to the above patterns.

In both options if path ends with `/` then it is considered a temporary directory and is an error unless invoked with empty command (this is parse error now).


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-25  0:05           ` Nikolay Aleksandrovich Pavlov (ZyX)
@ 2016-09-25  5:17             ` Bart Schaefer
  2016-09-25  6:17               ` Sebastian Gniazdowski
  2016-09-25 16:58               ` Nikolay Aleksandrovich Pavlov (ZyX)
  0 siblings, 2 replies; 21+ messages in thread
From: Bart Schaefer @ 2016-09-25  5:17 UTC (permalink / raw)
  To: Zsh Users

On Sep 25,  3:05am, Nikolay Aleksandrovich Pavlov (ZyX) wrote:
}
} I would consider another thing: while extensions are significant part
} of the file names, sometimes complete last path component or the whole
} path matters. I would suggest that if syntax is extended to allow
} controlling some properties of a temporary file name it should allow
} to control everything:

One thing that has to be considered is the security aspects of temporary
file creation.  Files created with =(...) have unpredictable names for
a reason:  A malicious actor should not be able to create such a file in
advance, because if that's possible then the shell can be caused to read
and/or write anywhere that the malicious actor desires.

We even swept through the completion shell-functions not long ago and
revised them all to create their temporary files using =(...) and then
hardlink them with the actual desired file names.  The linked files
are then removed before functions exit, via an "always" block.

} 1. Provide just an extension: use names like `/tmp/zshUkxFmK.ext`.
} 2. Optionally provide full file name: use names like
} `/tmp/zshUkxFmK/name.ext`, in this case temporary directory is generated and
} then recursively deleted.
} 3. Provide full path: allow generating automatically removed temporary files
} at any selected location.
} 4. Optionally make this feature generate temporary directories and not files,
} if code for 2. is written this should be easy.

The "mktemp" utility is designed to cover nearly all of these cases,
and I don't know of any likely situation in which use of an "always"
block (see next example) would leave something behind that the use
of =(...) wouild not also leave behind.

  () {
    {
      local tempdir=$(mktemp -d)
      # ... do stuff in $tempdir ...
    } always {
      [[ -n $tempdir ]] && /bin/rm -r $tempdir
    }
  }

} The key point is that I sometimes need temporary file in a specific
} location, but I almost never needed temporary file with a specific
} extension.
[...]
} As an implementation detail I would expect zsh to run parameter
} expansion, filename expansion and filename generation and match the
} result according to the above patterns.

You can combine mktemp and zsh temp files, too:

    local tempdir=$(mktemp -d -p /*/${specific}/location)
    local TMPPREFIX=$tempdir/zsh
    ln =(<<<'this is a silly example') $tempdir/sillyexample.txt
    ls -l $tempdir

In short it's pretty unlikely that we're going to re-implement mktemp
by inventing convoluted variations of =(...) syntax.

Futhermore, the definition of =(...) is that the contents of the
resulting file are the standard output of the command inside the
parens.  What would that mean in the case of a directory?

What might be more reasonable is to add $TMPSUFFIX, which unlike
$TMPPREFIX would not have a value by default, and when it is set,
append that to the temporary name.

However, neither that nor any of your suggestions would accomplish
altering the filename based on the file content.  The file has to be
created in the parent shell before the command is even run, so that
it can be opened as the command's output; and I would not advocate
having the shell peering at the content without being explicitly
scripted to do so.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-25  5:17             ` Bart Schaefer
@ 2016-09-25  6:17               ` Sebastian Gniazdowski
  2016-09-25 15:21                 ` Lawrence Velázquez
  2016-09-26 18:03                 ` Bart Schaefer
  2016-09-25 16:58               ` Nikolay Aleksandrovich Pavlov (ZyX)
  1 sibling, 2 replies; 21+ messages in thread
From: Sebastian Gniazdowski @ 2016-09-25  6:17 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh Users

On 25 September 2016 at 07:17, Bart Schaefer <schaefer@brasslantern.com> wrote:
> However, neither that nor any of your suggestions would accomplish
> altering the filename based on the file content.  The file has to be
> created in the parent shell before the command is even run, so that
> it can be opened as the command's output; and I would not advocate
> having the shell peering at the content without being explicitly
> scripted to do so.

Yes yes I was surprised my post could be understood as peering at
content. It's like in the git example: extension is known (is obtained
from git ls-tree, just like SHA of the BLOB), and can be provided to
=( ) which runs command that accesses the content. Other example:
downloading a HTTP address, having Content-Type available, knowing
what the extension should be. And a standard situation: creating
specific content for specific program expecting specific, known
extension (a case if only it's not typical and of course common less
program, it's part of my point in general: "what programs do not use
extensions?").

First thing that came to my mind is use of [], not () like Nikolay
proposed, so e.g. vim =( cat ... )[java], although it also suffers
from colliding with globbing, which is IMO a serious drawback.

Best regards,
Sebastian Gniazdowski


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-25  6:17               ` Sebastian Gniazdowski
@ 2016-09-25 15:21                 ` Lawrence Velázquez
  2016-09-26 18:03                 ` Bart Schaefer
  1 sibling, 0 replies; 21+ messages in thread
From: Lawrence Velázquez @ 2016-09-25 15:21 UTC (permalink / raw)
  To: Sebastian Gniazdowski; +Cc: zsh-users

> On Sep 25, 2016, at 2:17 AM, Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
> 
> On 25 September 2016 at 07:17, Bart Schaefer <schaefer@brasslantern.com> wrote:
>> However, neither that nor any of your suggestions would accomplish
>> altering the filename based on the file content.  The file has to be
>> created in the parent shell before the command is even run, so that
>> it can be opened as the command's output; and I would not advocate
>> having the shell peering at the content without being explicitly
>> scripted to do so.
> 
> Yes yes I was surprised my post could be understood as peering at
> content. It's like in the git example: extension is known (is obtained
> from git ls-tree, just like SHA of the BLOB), and can be provided to
> =( ) which runs command that accesses the content. Other example:
> downloading a HTTP address, having Content-Type available, knowing
> what the extension should be.

According to Bart, the parent shell has to create the temporary file
before executing the command inside the subshell. Unless I'm mistaken,
that implies that the parent would have to inspect said command before
executing it in order to decide how to obtain your suggested metadata.
If the subshell contains "git cat-file", the parent shell would have to
know to run "git ls-tree". If the subshell contains "curl", the parent
shell would have to know to run "curl -I" first.

Guessing an extension by peering at arbitrary, context-specific metadata
is no better than doing so by peering at the data itself. It's just
another form of content detection that would balloon into a convoluted
mess and shouldn't be implemented in the shell itself.

> And a standard situation: creating specific content for specific
> program expecting specific, known extension

Would Bart's TMPSUFFIX suggestion satisfy this use case?

> (a case if only it's not typical and of course common less program,
> it's part of my point in general: "what programs do not use
> extensions?").


I could spend all day (or at least a minute!) listing Unix programs that
don't care about extensions. This is really not a good argument.

vq

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-25  5:17             ` Bart Schaefer
  2016-09-25  6:17               ` Sebastian Gniazdowski
@ 2016-09-25 16:58               ` Nikolay Aleksandrovich Pavlov (ZyX)
  2016-09-25 22:37                 ` Bart Schaefer
  1 sibling, 1 reply; 21+ messages in thread
From: Nikolay Aleksandrovich Pavlov (ZyX) @ 2016-09-25 16:58 UTC (permalink / raw)
  To: Bart Schaefer, Zsh Users



25.09.2016, 08:19, "Bart Schaefer" <schaefer@brasslantern.com>:
> On Sep 25, 3:05am, Nikolay Aleksandrovich Pavlov (ZyX) wrote:
> }
> } I would consider another thing: while extensions are significant part
> } of the file names, sometimes complete last path component or the whole
> } path matters. I would suggest that if syntax is extended to allow
> } controlling some properties of a temporary file name it should allow
> } to control everything:
>
> One thing that has to be considered is the security aspects of temporary
> file creation. Files created with =(...) have unpredictable names for
> a reason: A malicious actor should not be able to create such a file in
> advance, because if that's possible then the shell can be caused to read
> and/or write anywhere that the malicious actor desires.
>
> We even swept through the completion shell-functions not long ago and
> revised them all to create their temporary files using =(...) and then
> hardlink them with the actual desired file names. The linked files
> are then removed before functions exit, via an "always" block.

If user explicitly requested temporary file at `/home/foo/bar` then he is not satisfied with usual `=()` location and thus simply will use other options in place of `=()`. Providing extension to a `=()` syntax which allows specifying whole name should not open any security holes because it is not the default and it is a replacement for less convenient alternatives (i.e. `always` block). Specifying part of the name (extension or trailing path component) as I suggested is not different from the current situation: in both cases it simply adds a known suffix to a random name.

>
> } 1. Provide just an extension: use names like `/tmp/zshUkxFmK.ext`.
> } 2. Optionally provide full file name: use names like
> } `/tmp/zshUkxFmK/name.ext`, in this case temporary directory is generated and
> } then recursively deleted.
> } 3. Provide full path: allow generating automatically removed temporary files
> } at any selected location.
> } 4. Optionally make this feature generate temporary directories and not files,
> } if code for 2. is written this should be easy.
>
> The "mktemp" utility is designed to cover nearly all of these cases,
> and I don't know of any likely situation in which use of an "always"
> block (see next example) would leave something behind that the use
> of =(...) wouild not also leave behind.
>
>   () {
>     {
>       local tempdir=$(mktemp -d)
>       # ... do stuff in $tempdir ...
>     } always {
>       [[ -n $tempdir ]] && /bin/rm -r $tempdir
>     }
>   }

This is an argument against `=()` as a whole, not against my proposal. I can use `mktemp`+`always` block everywhere I use `=()`, but yet `=()` exists.

>
> } The key point is that I sometimes need temporary file in a specific
> } location, but I almost never needed temporary file with a specific
> } extension.
> [...]
> } As an implementation detail I would expect zsh to run parameter
> } expansion, filename expansion and filename generation and match the
> } result according to the above patterns.
>
> You can combine mktemp and zsh temp files, too:
>
>     local tempdir=$(mktemp -d -p /*/${specific}/location)
>     local TMPPREFIX=$tempdir/zsh
>     ln =(<<<'this is a silly example') $tempdir/sillyexample.txt
>     ls -l $tempdir
>
> In short it's pretty unlikely that we're going to re-implement mktemp
> by inventing convoluted variations of =(...) syntax.
>
> Futhermore, the definition of =(...) is that the contents of the
> resulting file are the standard output of the command inside the
> parens. What would that mean in the case of a directory?

This is why I suggested to allow only empty command if one needs a directory.

>
> What might be more reasonable is to add $TMPSUFFIX, which unlike
> $TMPPREFIX would not have a value by default, and when it is set,
> append that to the temporary name.
>
> However, neither that nor any of your suggestions would accomplish
> altering the filename based on the file content. The file has to be
> created in the parent shell before the command is even run, so that
> it can be opened as the command's output; and I would not advocate
> having the shell peering at the content without being explicitly
> scripted to do so.

This feature is not what OP requested.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-25 16:58               ` Nikolay Aleksandrovich Pavlov (ZyX)
@ 2016-09-25 22:37                 ` Bart Schaefer
  0 siblings, 0 replies; 21+ messages in thread
From: Bart Schaefer @ 2016-09-25 22:37 UTC (permalink / raw)
  To: Zsh Users

On Sep 25,  7:58pm, Nikolay Aleksandrovich Pavlov (ZyX) wrote:
} 
} 25.09.2016, 08:19, "Bart Schaefer" <schaefer@brasslantern.com>:
} > The "mktemp" utility is designed to cover nearly all of these cases,
} 
} This is an argument against `=()` as a whole, not against my proposal.

Quite the contrary.  =() is intended to solve a very specific and very
common case:  capture the output of a command and immediately make use
of it, then discard it automatically.  It differs from a pipe in only
two very deliberate ways:  it provides a file name when a command is
not able to read stdin, and it is seek-able.

There is no corresponding *common* case for any of your suggestions; it
rarely makes sense to create and immediately discard an empty directory,
and it seldom makes sense to create and discard a file with a specific
name.[*]  The existence of a more complete solution for complicated
cases neither invalidates the simple solution to the simple case nor
justifies turning the simple solution into a complicated one.

[*] This is not to say that there isn't a use case that might be done
a hundred times in a day that needs a specific name, just that there
are not a large number of different commonly-occuring such use cases.

} > However, neither that nor any of your suggestions would accomplish
} > altering the filename based on the file content.
} 
} This feature is not what OP requested.

Nearly everyone seems to have interpreted it that way, even though
Sebastian has now said that's not what he meant ... but:

What's potentially wanted is a syntax that means "take file X, make
a link to it named Y, include Y in the set of files to be unlinked
when this command context is finished, and substitute Y into the
command line in place of X."  I simulated that with my "fixfiletype"
precommand-modifier-ish wrapper function.  How you generate Y should
not need to have anything to do with where X came from.

The TMPSUFFIX idea is part-way to this, but it still ties Y to X.
It might even be possible to have a TMPFILENAME value that entirely
repaces the mktemp(3) file name, though that wouldn't work if there
are multiple =() on the same command line.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Feature request: set extension for =( ) created file
  2016-09-25  6:17               ` Sebastian Gniazdowski
  2016-09-25 15:21                 ` Lawrence Velázquez
@ 2016-09-26 18:03                 ` Bart Schaefer
  1 sibling, 0 replies; 21+ messages in thread
From: Bart Schaefer @ 2016-09-26 18:03 UTC (permalink / raw)
  To: Zsh Users

On Sep 25,  8:17am, Sebastian Gniazdowski wrote:
}
} First thing that came to my mind is use of [], not () like Nikolay
} proposed, so e.g. vim =( cat ... )[java], although it also suffers
} from colliding with globbing, which is IMO a serious drawback.

Hmm ... maybe colliding with globbing is exactly what we need.

Consider (line breaks added for readability):

vim =(
    git -C /home/user/github/proj.git cat-file blob 1e08920
  )(e?'reply=($REPLY.java); ln $REPLY $reply'?)

This covers everything that's wanted except for the auto-removal of
the .java file; even "java" can be replaced by a $(git ls-tree ...)
to obtain the file extension.

So if we could replace "ln" with a builtin that implements "make
this link, but remove it when the current command context finishes"
we'd have a complete solution.

One step further to make it it's own glob qualifier (hardest part
may be coming up with what character to use) and we're set.  In the
meantime,

    alias vimtmp='() { { vim $1 } always { rm $1 } }'

And, good grief, I just realized this is another answer to FAQ #2.3.


^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2016-09-26 18:03 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-24 17:51 Feature request: set extension for =( ) created file Sebastian Gniazdowski
2016-09-24 17:55 ` Vadim Zeitlin
2016-09-24 18:43   ` Lawrence Velázquez
2016-09-24 19:30     ` Sebastian Gniazdowski
2016-09-24 19:55       ` Lawrence Velázquez
2016-09-24 20:14         ` Sebastian Gniazdowski
2016-09-24 20:36           ` Lawrence Velázquez
2016-09-24 20:44             ` Sebastian Gniazdowski
2016-09-24 21:06               ` Lawrence Velázquez
2016-09-24 21:24                 ` Bart Schaefer
2016-09-24 21:58                   ` Lawrence Velázquez
2016-09-25  0:05           ` Nikolay Aleksandrovich Pavlov (ZyX)
2016-09-25  5:17             ` Bart Schaefer
2016-09-25  6:17               ` Sebastian Gniazdowski
2016-09-25 15:21                 ` Lawrence Velázquez
2016-09-26 18:03                 ` Bart Schaefer
2016-09-25 16:58               ` Nikolay Aleksandrovich Pavlov (ZyX)
2016-09-25 22:37                 ` Bart Schaefer
2016-09-24 21:06         ` Bart Schaefer
2016-09-24 18:56   ` Sebastian Gniazdowski
2016-09-24 18:22 ` Bart Schaefer

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).