zsh-users
 help / color / mirror / Atom feed
* question mark in filename.
@ 2021-01-10  4:49 Ray Andrews
  2021-01-10  5:39 ` Lawrence Velázquez
  2021-01-10  7:02 ` Roman Perepelitsa
  0 siblings, 2 replies; 11+ messages in thread
From: Ray Andrews @ 2021-01-10  4:49 UTC (permalink / raw)
  To: Zsh Users

Not that there should ever be such a thing, but I was curious:  I throw 
strings around mostly successfully but not with the stupid question mark:


    function test1 ()
    {
         input=( "$@" )         # 'input' is fine: "ls s,7*"
         eval $input            # this works fine.
         output=( $(eval ${()input}) )
         print "$output"        # ... but 'output' looses the question
    marks.
    }

    function test2 ()
    {
         string="ls s,7*"       # 'ls' finds: 's,7,big?improvements?in?code'
         test1 $string
    }

    $ . test; test2
    s,7,big?improvements?in?code
    s,7,big improvements in code

I've tried various invocations in the parenthesis, I thought '(q)' 
should work but no luck.  I'm know that protecting special characters is 
possible.  'output' prefers to be separate words, broken on the '?' but 
the outer parenthesis at least get it all back on one line.


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: question mark in filename.
  2021-01-10  4:49 question mark in filename Ray Andrews
@ 2021-01-10  5:39 ` Lawrence Velázquez
  2021-01-10  5:46   ` Ray Andrews
  2021-01-10  7:02 ` Roman Perepelitsa
  1 sibling, 1 reply; 11+ messages in thread
From: Lawrence Velázquez @ 2021-01-10  5:39 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

> On Jan 9, 2021, at 11:49 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:
> 
> Not that there should ever be such a thing

Nonsense. '?' is valid in Unix filenames and should not be stigmatized.

> but I was curious:  I throw strings around mostly successfully but not with the stupid question mark:
> 
> 
>   function test1 ()
>   {
>        input=( "$@" )         # 'input' is fine: "ls s,7*"
>        eval $input            # this works fine.
>        output=( $(eval ${()input}) )
>        print "$output"        # ... but 'output' looses the question
>   marks.
>   }
> 
>   function test2 ()
>   {
>        string="ls s,7*"       # 'ls' finds: 's,7,big?improvements?in?code'
>        test1 $string
>   }
> 
>   $ . test; test2
>   s,7,big?improvements?in?code
>   s,7,big improvements in code
> 
> I've tried various invocations in the parenthesis, I thought '(q)' should work but no luck.  I'm know that protecting special characters is possible.  'output' prefers to be separate words, broken on the '?' but the outer parenthesis at least get it all back on one line.

Your example works for me in a clean shell.

	% zsh --version
	zsh 5.8 (x86_64-apple-darwin18.7.0)
	% cat foo.zsh
	function test1 ()
	{
	    input=( "$@" )
	    eval $input
	    output=( $(eval ${()input}) )
	    print "$output"
	}
	
	function test2 ()
	{
	    string="ls s,7*"
	    test1 $string
	}
	
	touch 's,7,big?improvements?in?code'
	test2
	rm 's,7,big?improvements?in?code'
	% ls
	foo.zsh
	% zsh -f foo.zsh
	s,7,big?improvements?in?code
	s,7,big?improvements?in?code

What options do you have set? Aliases? Functions? What are the
contents of your current working directory?

vq

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: question mark in filename.
  2021-01-10  5:39 ` Lawrence Velázquez
@ 2021-01-10  5:46   ` Ray Andrews
  2021-01-10  6:06     ` Lawrence Velázquez
  0 siblings, 1 reply; 11+ messages in thread
From: Ray Andrews @ 2021-01-10  5:46 UTC (permalink / raw)
  To: zsh-users

On 2021-01-09 9:39 p.m., Lawrence Velázquez wrote:
> Your example works for me in a clean shell.
> What options do you have set? Aliases? Functions? What are the
> contents of your current working directory?
>
> vq
Isn't that interesting.

$ zsh -f
g5--9-Debian1# . test; test2
s,7,big?improvements?in?code
s,7,big improvements in code
g5--9-Debian1#

... so I should have an equally pure shell there no?  What could be the 
difference?



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: question mark in filename.
  2021-01-10  5:46   ` Ray Andrews
@ 2021-01-10  6:06     ` Lawrence Velázquez
  2021-01-10  6:21       ` Ray Andrews
  0 siblings, 1 reply; 11+ messages in thread
From: Lawrence Velázquez @ 2021-01-10  6:06 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

> On Jan 10, 2021, at 12:46 AM, Ray Andrews <rayandrews@eastlink.ca> wrote:
> 
> On 2021-01-09 9:39 p.m., Lawrence Velázquez wrote:
>> What are the contents of your current working directory?

You didn't clarify this for us.

> Isn't that interesting.
> 
> $ zsh -f
> g5--9-Debian1# . test; test2
> s,7,big?improvements?in?code
> s,7,big improvements in code
> g5--9-Debian1#
> 
> ... so I should have an equally pure shell there no?  What could be the difference?

Depends on what's in your /etc/zshenv, or your system's equivalent
global zshenv. I don't have one.

vq

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: question mark in filename.
  2021-01-10  6:06     ` Lawrence Velázquez
@ 2021-01-10  6:21       ` Ray Andrews
  0 siblings, 0 replies; 11+ messages in thread
From: Ray Andrews @ 2021-01-10  6:21 UTC (permalink / raw)
  To: zsh-users


> Depends on what's in your /etc/zshenv, or your system's equivalent
> global zshenv. I don't have one.
>
> vq
I'm not sure what's sourced by the '-f' shell.  No /etc/zshenv here.  I 
have one startup file but since nothing in it seems active, I presume 
it's not being sourced.  I hid it and nothing changed.



^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: question mark in filename.
  2021-01-10  4:49 question mark in filename Ray Andrews
  2021-01-10  5:39 ` Lawrence Velázquez
@ 2021-01-10  7:02 ` Roman Perepelitsa
  2021-01-10 15:03   ` Ray Andrews
  1 sibling, 1 reply; 11+ messages in thread
From: Roman Perepelitsa @ 2021-01-10  7:02 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Sun, Jan 10, 2021 at 5:50 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Not that there should ever be such a thing, but I was curious:  I throw
> strings around mostly successfully but not with the stupid question mark:
>
>
>     function test1 ()
>     {
>          input=( "$@" )         # 'input' is fine: "ls s,7*"
>          eval $input            # this works fine.
>          output=( $(eval ${()input}) )
>          print "$output"        # ... but 'output' looses the question
>     marks.
>     }
>
>     function test2 ()
>     {
>          string="ls s,7*"       # 'ls' finds: 's,7,big?improvements?in?code'
>          test1 $string
>     }
>
>     $ . test; test2
>     s,7,big?improvements?in?code
>     s,7,big improvements in code

My guess is that the name of this file contains something interesting
in place of question marks (ascii 63). Try this:

    ls s,7* | cat -v
    print -r -- s,7* | od -t x1 -An -v

Roman.


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: question mark in filename.
  2021-01-10  7:02 ` Roman Perepelitsa
@ 2021-01-10 15:03   ` Ray Andrews
  2021-01-10 15:28     ` Roman Perepelitsa
  0 siblings, 1 reply; 11+ messages in thread
From: Ray Andrews @ 2021-01-10 15:03 UTC (permalink / raw)
  To: zsh-users

On 2021-01-09 11:02 p.m., Roman Perepelitsa wrote:
>
> My guess is that the name of this file contains something interesting
> in place of question marks (ascii 63). Try this:
>
>      ls s,7* | cat -v
>      print -r -- s,7* | od -t x1 -An -v
>
> Roman.
>
Right! I create a similar file:

$ touch s,8,big?improvements?in?code

$ ls s,8*
s,8,big?improvements?in?code

$ print -r -- s,7* | od -t x1 -An -v
  73 2c 37 2c 62 69 67 0a 69 6d 70 72 6f 76 65 6d
  65 6e 74 73 0a 69 6e 0a 63 6f 64 65 0a

$ print -r -- s,8* | od -t x1 -An -v
  73 2c 38 2c 62 69 67 3f 69 6d 70 72 6f 76 65 6d
  65 6e 74 73 3f 69 6e 3f 63 6f 64 65 0a

pasting the two outputs and trimming to the first question mark:

  73 2c 37 2c 62 69 67 0a  << A bleeding linefeed!
73 2c 38 2c 62 69 67 3f  << A real question mark.

... which is exactly what tried to show when printing my '$output', it 
broke on the 'question marks' that are really newlines anyway.  So a 
better question is why ls does this:

$ ls -1 s,[7,8]*
s,7,big?improvements?in?code
s,8,big?improvements?in?code

Anyway the former was auto generated by a utility I have and it was 
clearly in a buggy state when it made that file.  And now we know that 
'ls' replaces linefeeds with question marks, thanks Richard Stallman.  
Maybe any non-visible character?  But then again 'eval' uses question 
marks the first time and spaces the second so it's not just an issue 
with 'ls'.

Nice call Roman.





^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: question mark in filename.
  2021-01-10 15:03   ` Ray Andrews
@ 2021-01-10 15:28     ` Roman Perepelitsa
  2021-01-10 15:55       ` Ray Andrews
  0 siblings, 1 reply; 11+ messages in thread
From: Roman Perepelitsa @ 2021-01-10 15:28 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Sun, Jan 10, 2021 at 4:04 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> And now we know that 'ls' replaces linefeeds with question marks, thanks Richard Stallman.

It does that only when stdout is a tty. The output in this mode has
been improved recently. Here's what I get:

  % ls
  's,7,big'$'\n''improvements'$'\n''in'$'\n''code'$'\n'

  % ls | cat
  s,7,big
  improvements
  in
  code

> But then again 'eval' uses question marks the first time

That's just the output of `ls` when stdout is a tty.

> ... and spaces the second

This time stdout of `ls` is not a tty, so it prints the file name
verbatim. After that your code splits the output on IFS and joins it
back together with spaces. It's WAI.

Roman.


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: question mark in filename.
  2021-01-10 15:28     ` Roman Perepelitsa
@ 2021-01-10 15:55       ` Ray Andrews
  2021-01-10 16:32         ` Roman Perepelitsa
  0 siblings, 1 reply; 11+ messages in thread
From: Ray Andrews @ 2021-01-10 15:55 UTC (permalink / raw)
  To: zsh-users

On 2021-01-10 7:28 a.m., Roman Perepelitsa wrote:
> It does that only when stdout is a tty. The output in this mode has
> been improved recently. Here's what I get:
>
>    % ls
>    's,7,big'$'\n''improvements'$'\n''in'$'\n''code'$'\n'
That's a whole lot more honest.  A linefeed is not a question mark. This 
can be put to bed now, but if I had one remaining question it is why (or 
how) I might have protected the rogue characters.  As I mentioned I 
tried ' ${(q)input} ' and variations because  that seems to swallow most 
things and I was suspicious.  Nevermind.
>    % ls | cat
>    s,7,big
>    improvements
>    in
>    code
Yeah, after I captured the output to a variable, that's exactly what it 
wanted to show and of course now I know that's exactly what it should 
show.  Linefeeds are linefeeds.

> That's just the output of `ls` when stdout is a tty.
Ok, so 'eval' is innocent of all this.  As usual the man page is less 
than helpful, they could mention all this and save us all the trouble.




^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: question mark in filename.
  2021-01-10 15:55       ` Ray Andrews
@ 2021-01-10 16:32         ` Roman Perepelitsa
  2021-01-10 20:57           ` Ray Andrews
  0 siblings, 1 reply; 11+ messages in thread
From: Roman Perepelitsa @ 2021-01-10 16:32 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Sun, Jan 10, 2021 at 4:56 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I tried ' ${(q)input} ' and variations because  that seems to swallow most
> things and I was suspicious.

If you want to extract the list of files from the output of `ls`,
you'll need to pass --quoting-style=shell and then parse the output
with ${(z)...}. If you just want to get a list of files that match a
pattern, you don't need `ls`.

    files=(s,7*)

I understand that your problem is likely more complex than the test
you've shown but even then it's likely that you can avoid parsing the
output of `ls`.

Roman.


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: question mark in filename.
  2021-01-10 16:32         ` Roman Perepelitsa
@ 2021-01-10 20:57           ` Ray Andrews
  0 siblings, 0 replies; 11+ messages in thread
From: Ray Andrews @ 2021-01-10 20:57 UTC (permalink / raw)
  To: zsh-users

On 2021-01-10 8:32 a.m., Roman Perepelitsa wrote:
> On Sun, Jan 10, 2021 at 4:56 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>>
>>      files=(s,7*)
>>
>> I understand that your problem is likely more complex than the test
>> you've shown but even then it's likely that you can avoid parsing the
>> output of `ls`.
>>
>> Roman.
>>
You know, I'm just realizing that now.  I need 'ls' for output but not 
for getting file matches.



^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2021-01-10 20:57 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-10  4:49 question mark in filename Ray Andrews
2021-01-10  5:39 ` Lawrence Velázquez
2021-01-10  5:46   ` Ray Andrews
2021-01-10  6:06     ` Lawrence Velázquez
2021-01-10  6:21       ` Ray Andrews
2021-01-10  7:02 ` Roman Perepelitsa
2021-01-10 15:03   ` Ray Andrews
2021-01-10 15:28     ` Roman Perepelitsa
2021-01-10 15:55       ` Ray Andrews
2021-01-10 16:32         ` Roman Perepelitsa
2021-01-10 20:57           ` Ray Andrews

zsh-users

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.vuxu.org/zsh-users

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 zsh-users zsh-users/ http://inbox.vuxu.org/zsh-users \
		zsh-users@zsh.org
	public-inbox-index zsh-users

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.zsh.users


code repositories for the project(s) associated with this inbox:

	https://git.vuxu.org/mirror/zsh/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git