spaces in filenames should be a crime.

spaces in filenames should be a crime.

[Ray Andrews]

Gentlemen:

This function:

     mostrecent ()
     {
         ls -l ${(f)$( find . -type f -printf "%T@ %p\n" | sort -n | cut 
-d' ' -f 2- | tail -n 1 )}
     }

Testing it on this dummy directory tree:

├── [   0]  a
├── [4.0K]  absolute junk
│   └── [   0]  junk
├── [   0]  b
└── [   0]  c

If the most recent file is under 'absolute junk', the function works, 
but it requires the ' ${(f) ... ' treatment, otherwise it barfs at the 
space in the directory name.  However, if I change things to ' tail -n 3 
"  it shows me:

ls: cannot access './a ./absolute junk/junk ./c': No such file or directory

So whereas the ' ${(f) ' treatment fixes the one problem, it creates the 
other, which is obviously that everything is one string.  Can I have it 
both ways?  Protected spaces in the directory name, and still have 
multiple items for the listing? I've done stuff like this before, but 
using arrays.   Or can printf handle the situation by itself?

Re: spaces in filenames should be a crime.

[Vadim A. Misbakh-Soloviov]

[-- Attachment #1: Type: text/plain, Size: 24 bytes --]

how about doublequoting?

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: spaces in filenames should be a crime.

[Daniel Shahaf]

Ray Andrews wrote on Sun, Mar 26, 2017 at 13:22:24 -0700:
> Gentlemen:
> 
> This function:
> 
>     mostrecent ()
>     {
>         ls -l ${(f)$( find . -type f -printf "%T@ %p\n" | sort -n | cut -d'
> ' -f 2- | tail -n 1 )}
>     }

You could replace the whole function with:

    mostrecent() { ls -l -- **/*(.om[1]) }

That glob qualifier restricts the match to files ('.'), sorts them by
modified date ('om'), and selects the first result ('[1]').

You can then use [1,3] to select several files.

> Testing it on this dummy directory tree:
> 
> ├── [   0]  a
> ├── [4.0K]  absolute junk
> │   └── [   0]  junk
> ├── [   0]  b
> └── [   0]  c
> 
> If the most recent file is under 'absolute junk', the function works, but it
> requires the ' ${(f) ... ' treatment, otherwise it barfs at the space in the
> directory name.  However, if I change things to ' tail -n 3 "  it shows me:
> 
> ls: cannot access './a ./absolute junk/junk ./c': No such file or directory
> 
> So whereas the ' ${(f) ' treatment fixes the one problem, it creates the
> other, which is obviously that everything is one string.  Can I have it both
> ways?  Protected spaces in the directory name, and still have multiple items
> for the listing? I've done stuff like this before, but using arrays.
>
> Or can printf handle the situation by itself?

As Vadim said, putting the $() in double quotes makes this example work.

Another option here is to use find -print0 with ${(0)}.  That only
matter if you need to DTRT when filenames contain NL characters.

Cheers,

Daniel

Re: spaces in filenames should be a crime.

[Ray Andrews]

On 26/03/17 02:18 PM, Daniel Shahaf wrote:

Thanks all:

Yes, Vadim's  fix seems to be fine.  Nuts, I still don't have much sense 
of the logic of these things.
> You could replace the whole function with:
>      mostrecent() { ls -l -- **/*(.om[1]) }

Yeah, my original was just something I found on the net and didn't 
bother to improve upon until it crashed and burned on a space in the 
directory name.  I'm not surprised that our 'ls' can do it all by itself.
> You can then use [1,3] to select several files.

Very fine.  Not forgetting to add '-U' to 'ls'.  Mind, why is that 
needed, since the input is not  redirected from some other program it 
seems strange to have to tell it not to resort incorrectly what has been 
specifically requested, no?
>
> Another option here is to use find -print0 with ${(0)}.  That only
> matter if you need to DTRT when filenames contain NL characters.
>
DTRT?

Re: spaces in filenames should be a crime.

[Bart Schaefer]

[-- Attachment #1: Type: text/plain, Size: 100 bytes --]

On Mar 26, 2017 4:12 PM, "Ray Andrews" <rayandrews@eastlink.ca> wrote:


DTRT?


Do The Right Thing

Re: spaces in filenames should be a crime.

[Ray Andrews]

On 26/03/17 04:20 PM, Bart Schaefer wrote:
> On Mar 26, 2017 4:12 PM, "Ray Andrews" <rayandrews@eastlink.ca> wrote:
>
>
> DTRT?
>
>
> Do The Right Thing
>
Damn, that seems basic.  Never seen it before.

Re: spaces in filenames should be a crime.

[Chris ccb]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]

It seems to me like, no matter whether you're writing a 4 line script for
home/personal use, or working on some some serious business secure
real-time application to pull some crazy wonder out of thin air, it always
pays to "DTRT" just in the long term time/effort savings alone. Not to
mention the skill/habit forming benefits and potentially reputation that
good code brings along with it. Am I wrong?

On Sun, Mar 26, 2017 at 8:47 PM Ray Andrews <rayandrews@eastlink.ca> wrote:

> On 26/03/17 04:20 PM, Bart Schaefer wrote:
> > On Mar 26, 2017 4:12 PM, "Ray Andrews" <rayandrews@eastlink.ca> wrote:
> >
> >
> > DTRT?
> >
> >
> > Do The Right Thing
> >
> Damn, that seems basic.  Never seen it before.
>
>

Re: spaces in filenames should be a crime.

[Ray Andrews]

On 26/03/17 06:11 PM, Chris ccb wrote:
> It seems to me like, no matter whether you're writing a 4 line script for
> home/personal use, or working on some some serious business secure
> real-time application to pull some crazy wonder out of thin air, it always
> pays to "DTRT" just in the long term time/effort savings alone. Not to
> mention the skill/habit forming benefits and potentially reputation that
> good code brings along with it. Am I wrong?

Heck no.  I hope I didn't sound like I was saying otherwise.    The 
thing that bugs the heck out of me is exactly these issues of 
robustness, there's always another gotcha.  It would be fantastic to be 
able to make things bullet proof right out of the gate, but that seems 
to be considered an accretion to be learned later on.  Mind, anyone who 
puts newlines in filenames should be burned at the stake, it's sick and 
twisted.  Bart will know how such things were ever permitted in the 
first place.  Mere anarchy was loosed upon the world.

Re: spaces in filenames should be a crime.

[Martijn Dekker]

Op 27-03-17 om 00:41 schreef Ray Andrews:
> On 26/03/17 02:18 PM, Daniel Shahaf wrote:
>> You could replace the whole function with:
>>      mostrecent() { ls -l -- **/*(.om[1]) }
> 
> Yeah, my original was just something I found on the net and didn't
> bother to improve upon until it crashed and burned on a space in the
> directory name.  I'm not surprised that our 'ls' can do it all by itself.

Note that 'ls' is an external command that doesn't do anything but list
files given to it as arguments. It cannot expand any glob patterns (with
*, etc.); that's the shell's job. The shell resolves the glob pattern
and hands 'ls' the expanded list of arguments. This applies to every
other command as well, and even to shell functions.

(This is a fundamental difference with DOS/Windows batch files, where
every command is expected to resolve patterns by itself.)

Shell globbing can also be dangerous if any file names start with '-';
the command may interpret these as options. Most commands support a
standard method of neutralising that: the special option '--' to disable
option parsing for any subsequent command. A common alternative idiom is
to prepend './' to the glob pattern so that every argument expanded from
it also starts with './' (and hence never '-').

> DTRT?

FYI: http://www.acronymfinder.com/

HTH. HAND. ;-)

- M.

Re: spaces in filenames should be a crime.

[Ray Andrews]

On 26/03/17 06:44 PM, Martijn Dekker wrote:
>
> Note that 'ls' is an external command that doesn't do anything but list ..

Of course.  I worded that poorly.  I shouldn't have said: " our 'ls' ", 
but something like "we can leverage 'ls' to do the whole thing".
> (This is a fundamental difference with DOS/Windows batch files, where
> every command is expected to resolve patterns by itself.)

Yes, and how hard that was to get through my skull.
>
> A common alternative idiom is
> to prepend './' to the glob pattern so that every argument expanded from
> it also starts with './' (and hence never '-').
Cool.

Re: spaces in filenames should be a crime.

[Jesper Nygårds]

[-- Attachment #1: Type: text/plain, Size: 365 bytes --]

If you'd suggest a solution using 'ls' on the bash-help mailing list, you'd
get lynched. Although I have always found the tone on that list less than
welcoming, I've taken that recommendation to heart. What is the opinion
here? Should 'ls' as a basis for file name generation be avoided at all
cost? It's pretty clear that it is never a necessary tool for the job.

Re: spaces in filenames should be a crime.

[Mikael Magnusson]

On Mon, Mar 27, 2017 at 8:53 PM, Jesper Nygårds
<jesper.nygards@gmail.com> wrote:
> If you'd suggest a solution using 'ls' on the bash-help mailing list, you'd
> get lynched. Although I have always found the tone on that list less than
> welcoming, I've taken that recommendation to heart. What is the opinion
> here? Should 'ls' as a basis for file name generation be avoided at all
> cost? It's pretty clear that it is never a necessary tool for the job.

You accidentally removed all context from your mail, but assuming
you're referring to daniel's
mostrecent() { ls -l -- **/*(.om[1]) }
then it doesn't use ls for generating a file list, only for formatting
output printed to the console, which is totally fine.

-- 
Mikael Magnusson

Re: spaces in filenames should be a crime.

[Ray Andrews]

On 27/03/17 04:18 PM, Mikael Magnusson wrote:
>
> You accidentally removed all context from your mail, but assuming
> you're referring to daniel's
> mostrecent() { ls -l -- **/*(.om[1]) }
> then it doesn't use ls for generating a file list, only for formatting
> output printed to the console, which is totally fine.
>
It does make me think tho, since zsh is doing all the hard work, how 
much more would be needed to bypass ls entirely and handle the output?  
This is baloney but:

$ echo $(**/*(.om[1])

... idea/question being that instead of ls receiving the list of files, maybe it could be done more directly.  Not that there's any need for such a thing, I'm just curious.

Re: spaces in filenames should be a crime.

[Martin Richter]

Hi Ray,

On 28.03.2017 02:46, Ray Andrews wrote:
> echo $(**/*(.om[1])

I'm afraid now you are holding it wrong :-)

1. **/*(.om[1]) will be expanded and therefore lead to a list of
matching filenames

2. the $(...) part will try to execute it as a command
But this is not what you want (usually there is no command named like
the first file found not is this what you usually want to do)

I think you are after

$ echo **/*(.om[1])

or

$ print -- **/*(.om[1])

which will just do what you want.

Cheers,
M.


PS:
Maybe you can also throw an extra (N) into the game to also handle the
case you should ever execute it in empty directories:

$ mkdir /tmp/foo_bar
$ cd /tmp/foo_bar
$ print -- **/*(.Nom[1])

does print nothing instead of throwing an "zsh: no matches found:
**/*(.om[1])" error or re-printing the pattern (depending on your
settings of nomatch and nullglob)

... Plus it gives the globbing modifiers a proximity to the Cookie
Monster's "Nom nom nom" ;-)

Re: spaces in filenames should be a crime.

[Ray Andrews]

On 28/03/17 02:08 AM, Martin Richter wrote:

echo $(**/*(.om[1])


> I think you are after
>
> $ print -- **/*(.Nom[1])
>
Ha!  You know  I was actually speculating, that wasn't supposed to work 
even remotely, yet I see it was not far from reality -- zsh can and does 
make a list of the files, it does not only do it when 'helping'  ls or 
some other binary.  So ls is merely supplying extra information about 
the files as may be desired, it does not 'make' the list of files.  
That's residual DOSthink again -- ok the shell globs the filenames, but 
only 'with' ls, that is, ls 'asks' zsh to expand a list of files.  NO!  
zsh creates the list all by itself ls or no ls.   ls is a passive 
recipient, there is no 'request'.  Yes?

Re: spaces in filenames should be a crime.

[Daniel Shahaf]

Ray Andrews wrote on Tue, Mar 28, 2017 at 07:29:24 -0700:
> ls is a passive recipient, there is no 'request'.  Yes?

Yes:

     1	% strace -fe execve sh -c 'ls -d /*' 
     2	execve("/bin/sh", ["sh", "-c", "ls -d /*"], [/* 47 vars */]) = 0
     3	Process 8717 attached
     4	[pid  8717] execve("/bin/ls", ["ls", "-d", "/bin", "/boot", "/dev", "/etc", "/home", "/initrd.img", "/lib", "/lib64", "/lost+found", "/media", "/mnt", ...], [/* 47 vars */]) = 0

That execve() line shows the 'argv' array in ls's main() will be.  You
can see that it has no asterisks.

Re: spaces in filenames should be a crime.

[Ray Andrews]

On 28/03/17 09:10 AM, Daniel Shahaf wrote:
> Ray Andrews wrote on Tue, Mar 28, 2017 at 07:29:24 -0700:
>> ls is a passive recipient, there is no 'request'.  Yes?
> Yes:
>
>       1	% strace -fe execve sh -c 'ls -d /*'
>       2	execve("/bin/sh", ["sh", "-c", "ls -d /*"], [/* 47 vars */]) = 0
>       3	Process 8717 attached
>       4	[pid  8717] execve("/bin/ls", ["ls", "-d", "/bin", "/boot", "/dev", "/etc", "/home", "/initrd.img", "/lib", "/lib64", "/lost+found", "/media", "/mnt", ...], [/* 47 vars */]) = 0
>
> That execve() line shows the 'argv' array in ls's main() will be.  You
> can see that it has no asterisks.
>
It must be frustrating for the list that after several years I still 
make these DOSy mistakes.   The above code is most educational, thanks.

Re: spaces in filenames should be a crime.

[Martijn Dekker]

Op 28-03-17 om 16:29 schreef Ray Andrews:
> zsh creates the list all by itself ls or no ls.   ls is a passive
> recipient, there is no 'request'.  Yes?

Yes. And this applies to nearly[*] every command.

But 'ls' is not entirely passive. There are things that 'ls' *does* do
with the arguments it receives. It's true it doesn't resolve glob
patterns, but:

* If any of the arguments to 'ls' (after resolving glob patterns) are
directories, it will list the contents of those directories, and not
just the directories themselves, unless the '-d' option is given.

* It will provide extra information about the files if an option like
'-l' is given.

* It formats the output into columns if it finds itself writing directly
to a terminal. (To see the difference, try 'ls' and then
'ls | cat')

As always, read the manual page for more information.

HTH,

- M.

[*] One command it does *not* apply to is 'find'; it does glob pattern
matching by itself. But that does not disable the shell's glob pattern
resolver; the shell treats it like any other command. That's why you
have to tell the shell NOT to resolve those glob patterns by quoting
them, e.g.: find /some/dir -name '*.txt'

Re: spaces in filenames should be a crime.

[Stephane Chazelas]

2017-03-28 11:08:43 +0200, Martin Richter:
[...]
> $ print -- **/*(.om[1])
> 
> which will just do what you want.
[...]

You need -r for print:

print -r -- **/*(.om[1])

or use printf instead:

printf '%s\n' **/*(.om[1])

To be equivalent to the "find" approach, you'd also want the "D"
glob qualifier to not exclude hidden files, and to traverse
hidden directories:

print -r -- **/*(D.om[1])

-- 
Stephane

Re: spaces in filenames should be a crime.

[Ray Andrews]

On 29/03/17 04:27 AM, Stephane Chazelas wrote:
> print -r -- **/*(D.om[1])

Thanks.  Sorta like buying a boat that, out of the factory, has 15 holes 
by design.  Slowly you find out where they all are and plug them and 
then she's seaworthy. Dunno, ships have sea cocks, but they're closed by 
default; the culture with shells is that they are all open by default.  
I wonder what software would look like if things were robust by default 
and then you subtracted functionality that you didn't want, so " **/* " 
would mean every damn thing even on Tuesday and then you'd add options 
to filter stuff out rather than adding it in.  Maybe impossible.  When I 
go shopping my cart doesn't start out with everything in the store in it 
and I take out what I don't want.  Yet I'm glad that boats don't sink by 
default.  What would the masters of the Tao say?

Re: spaces in filenames should be a crime.

[Peter Stephenson]

On Wed, 29 Mar 2017 07:50:34 -0700
Ray Andrews <rayandrews@eastlink.ca> wrote:
> On 29/03/17 04:27 AM, Stephane Chazelas wrote:
> > print -r -- **/*(D.om[1])
> 
> Thanks.  Sorta like buying a boat that, out of the factory, has 15 holes 
> by design.  Slowly you find out where they all are and plug them and 
> then she's seaworthy. Dunno, ships have sea cocks, but they're closed by 
> default; the culture with shells is that they are all open by default.  
> I wonder what software would look like if things were robust by default 
> and then you subtracted functionality that you didn't want, so " **/* " 
> would mean every damn thing even on Tuesday and then you'd add options 
> to filter stuff out rather than adding it in.

(Obviously this is a pointless discussion, but while I'm waiting for
things to compile...)

The only thing missing by default is files beginning "." (that's the
D).  The "." and the "om[1]" are indeed subtracting things.

By demanding that globbing matches files beginning with "." by default,
you've just removed one of the earliest safety features built into
shells, so that people are condemned to have their initialisation files
show up with every glob.  It happens that most people don't consider
this "robust" for their purposes.  But if you're happy to sacrifice
usability. you can set the option GLOB_DOTS, and it works the way
you want.

pws

Re: spaces in filenames should be a crime.

[Ray Andrews]

On 29/03/17 08:21 AM, Peter Stephenson wrote:
> On Wed, 29 Mar 2017 07:50:34 -0700
> Ray Andrews <rayandrews@eastlink.ca> wrote:
> (Obviously this is a pointless discussion, but while I'm waiting for
> things to compile...)
Is the design philosophy of software pointless?
> The only thing missing by default is files beginning "." (that's the
> D).  The "." and the "om[1]" are indeed subtracting things.
>
> By demanding
Not demanding, rather musing on the paradigm.

> that globbing matches files beginning with "." by default,
> you've just removed one of the earliest safety features built into
> shells, so that people are condemned to have their initialisation files
> show up with every glob.  It happens that most people don't consider
> this "robust" for their purposes.  But if you're happy to sacrifice
> usability. you can set the option GLOB_DOTS, and it works the way
> you want.
>
> pws
>
I guess it's both -- D and N are includers and '.om ' are excluders.  
Anyway, sorry, just dreaming out loud.  Probably can't be a dogmatic 
answer one way or the other.  Not bitching.

Re: spaces in filenames should be a crime.

[Bart Schaefer]

On Wed, Mar 29, 2017 at 9:21 AM, Ray Andrews <rayandrews@eastlink.ca> wrote:
> On 29/03/17 08:21 AM, Peter Stephenson wrote:
>>
>> On Wed, 29 Mar 2017 07:50:34 -0700
>> Ray Andrews <rayandrews@eastlink.ca> wrote:
>> (Obviously this is a pointless discussion, but while I'm waiting for
>> things to compile...)
>
> Is the design philosophy of software pointless?

No, but unless you're planning to write your own shell, there's zero
possibility that this discussion is going to lead anywhere.

Re: spaces in filenames should be a crime.

[Matthew Martin]

On Mon, Mar 27, 2017 at 08:53:17PM +0200, Jesper Nygårds wrote:
> If you'd suggest a solution using 'ls' on the bash-help mailing list, you'd
> get lynched. Although I have always found the tone on that list less than
> welcoming, I've taken that recommendation to heart. What is the opinion
> here? Should 'ls' as a basis for file name generation be avoided at all
> cost? It's pretty clear that it is never a necessary tool for the job.

I wouldn't say never. There are some inventive uses in the tree.


Completion/Unix/Command/_cvs:
    if _cvs_loadstat; then
      id="$(LC_ALL=C builtin zstat -g +mtime -F '%Y/%m/%d-%T' "$cvspassfile")"
    else
      id="$(LC_ALL=C ls -l "$cvspassfile")"
    fi
    if [[ "$id" != "$_cvs_pass_id" ]]; then
      slash=/
      _cvs_roots=($_cvs_roots ${${${${(f)"$(<$cvspassfile)"}#/1 }%% *}/:2401${slash}/:/})
      _cvs_pass_id="$id"
    fi


Test/C02cond.ztst:
  else
    unmodified_ls="$(ls -lu $unmodified)"
    print -u $ZTST_fd 'This test takes up to 60 seconds...'
  fi
  sleep 2
  [...]
       { (( $+unmodified_ls )) && SECONDS=0 &&
         ! until (( SECONDS >= 58 )); do
             ZTST_hashmark; sleep 2; cat $unmodified
             [[ $unmodified_ls != "$(ls -lu $unmodified)" ]] && break
	   done }; then
    ZTST_skip="[[ -N file ]] not supported with noatime file system"


While not technically correct, I don't know if there's a better way
Completion/Zsh/Type/_file_descriptors:
	if link=$(ls -l $proc/$i); then
	  list+=( "${(r.$#fds[-1].)i} $sep ${(D)link#* -> }" )


_hg and _remote_files could probably use printf '%s\0' with some glob,
but there are systems that don't have printf (notably android which is
why _adb uses ls).

Re: spaces in filenames should be a crime.

[Peter Stephenson]

On Sun, 26 Mar 2017 13:22:24 -0700
Ray Andrews <rayandrews@eastlink.ca> wrote:
>      mostrecent ()
>      {
>          ls -l ${(f)$( find . -type f -printf "%T@ %p\n" | sort -n | cut 
> -d' ' -f 2- | tail -n 1 )}
>      }

As Vadim says:

      mostrecent ()
      {
         ls -l ${(f)"$( find . -type f -printf "%T@ %p\n" | sort -n | cut -d' ' -f 2- | tail -n 1 )"}
      }

That's how (f) was designed to work.

But "find -print0 <whatever> | xargs -0 <whatever>' might also be your
friend, and you might not actually need find.  In fact, I'm sure there
are ways of tidying this up I haven't got time for just now.

pws