* Re: cat as a builtin command
2014-08-29 1:40 cat as a builtin command Izumi Natsuka
@ 2014-08-29 11:58 ` Roman Neuhauser
2014-08-29 12:24 ` Dirk Heinrichs
` (2 subsequent siblings)
3 siblings, 0 replies; 10+ messages in thread
From: Roman Neuhauser @ 2014-08-29 11:58 UTC (permalink / raw)
To: Izumi Natsuka; +Cc: zsh-users
# izumi.natsuka@hotmail.com / 2014-08-29 09:40:29 +0800:
> Hello, I'm going to write a shell function that provides a basic
> functionality (print the content of a file or stdin) of cat[0], in
> order to avoid forking too many process when I call it in a loop[1].
> exec {file}<${1:-0}
> read -Erd '' -u ${file}
> It works perfectly except when I want to cat a binary file:
>
> $ zstat +size archlinux-2012.09.07-dual.iso
> 411041792
> $ cat archlinux-2012.09.07-dual.iso | wc -c
> 39
>
> Seems that the file was cut by some special raw bytes.
i thought i could get it through with empty $IFS, alas, it stops on
the first ^M anyway. which is what the manpage says:
Read one line and break it into fields using the characters in $IFS
as separators, except as noted below.
there *is* a way, it's just slow as hell:
setopt localoptions nomultibyte
declare -A st
zstat -H st +size ${1:-0}
read -Erd '' -u ${file} -k ${st[size]}
> As I don't known how to avoid that I tried to use another method (via
> sysread):
> sysread -i ${file} -o 1 -s $(zstat +size ${1})
see `zstat -H` above.
none of that solves the FIFO problem.
> Is there any way that can perform the basic functionality of cat
> without calling external command?
i'd go for a zsh module (see zshmodules(1)). the sources for
zsh/example and zsh/files should give you a kickstart (i found
them useful for my needs).
--
roman
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: cat as a builtin command
2014-08-29 1:40 cat as a builtin command Izumi Natsuka
2014-08-29 11:58 ` Roman Neuhauser
@ 2014-08-29 12:24 ` Dirk Heinrichs
2014-08-29 13:20 ` Roman Neuhauser
2014-08-29 13:03 ` Peter Stephenson
2014-09-01 7:53 ` Han Pingtian
3 siblings, 1 reply; 10+ messages in thread
From: Dirk Heinrichs @ 2014-08-29 12:24 UTC (permalink / raw)
To: zsh-users
[-- Attachment #1: Type: text/plain, Size: 858 bytes --]
Am 29.08.2014 um 03:40 schrieb Izumi Natsuka:
> Hello, I'm going to write a shell function that provides a basic
> functionality(print the content of a file or stdin) of cat[0], in
> order to avoid forking too many process when I call it in a loop[1].
I try to avoid cat alltogether, as many uses belong to the "Useless use
of cat" category, like this one:
>
> $ cat archlinux-2012.09.07-dual.iso | wc -c
> 39
Could also be "wc -c archlinux-2012.09.07-dual.iso".
Results in mostly cat-free shell scripts (I like dog(s) more, anyway ;-) )
HTH...
Dirk
--
*Dirk Heinrichs*, Senior Systems Engineer, Engineering Solutions
*Recommind GmbH*, Von-Liebig-Straße 1, 53359 Rheinbach
*Tel*: +49 2226 1596666 (Ansage) 1149
*Email*: dhs@recommind.com <mailto:dhs@recommind.com>
*Skype*: dirk.heinrichs.recommind
www.recommind.com <http://www.recommind.com>
[-- Attachment #2.1: Type: text/html, Size: 1717 bytes --]
[-- Attachment #2.2: Logo.gif --]
[-- Type: image/gif, Size: 1537 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: cat as a builtin command
2014-08-29 12:24 ` Dirk Heinrichs
@ 2014-08-29 13:20 ` Roman Neuhauser
2014-08-29 13:33 ` Dirk Heinrichs
0 siblings, 1 reply; 10+ messages in thread
From: Roman Neuhauser @ 2014-08-29 13:20 UTC (permalink / raw)
To: Dirk Heinrichs; +Cc: zsh-users
# dhs@recommind.com / 2014-08-29 14:24:38 +0200:
> Am 29.08.2014 um 03:40 schrieb Izumi Natsuka:
>
> > Hello, I'm going to write a shell function that provides a basic
> > functionality(print the content of a file or stdin) of cat[0], in
> > order to avoid forking too many process when I call it in a loop[1].
>
> I try to avoid cat alltogether, as many uses belong to the "Useless use
> of cat" category, like this one:
>
> >
> > $ cat archlinux-2012.09.07-dual.iso | wc -c
> > 39
>
> Could also be "wc -c archlinux-2012.09.07-dual.iso".
definitely, but that doesn't fit the no-fork(2) restriction;
i understand the OP used wc(1) simply to demonstrate the defect
in his implementation.
--
roman
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: cat as a builtin command
2014-08-29 13:20 ` Roman Neuhauser
@ 2014-08-29 13:33 ` Dirk Heinrichs
0 siblings, 0 replies; 10+ messages in thread
From: Dirk Heinrichs @ 2014-08-29 13:33 UTC (permalink / raw)
To: zsh-users
[-- Attachment #1: Type: text/plain, Size: 625 bytes --]
Am 29.08.2014 um 15:20 schrieb Roman Neuhauser:
> definitely, but that doesn't fit the no-fork(2) restriction;
> i understand the OP used wc(1) simply to demonstrate the defect
> in his implementation.
Sure. But OTOH he also didn't state how he uses cat in his loop, so I
wanted to mention it, just un case...
Bye...
Dirk
--
*Dirk Heinrichs*, Senior Systems Engineer, Engineering Solutions
*Recommind GmbH*, Von-Liebig-Straße 1, 53359 Rheinbach
*Tel*: +49 2226 1596666 (Ansage) 1149
*Email*: dhs@recommind.com <mailto:dhs@recommind.com>
*Skype*: dirk.heinrichs.recommind
www.recommind.com <http://www.recommind.com>
[-- Attachment #2.1: Type: text/html, Size: 1238 bytes --]
[-- Attachment #2.2: Logo.gif --]
[-- Type: image/gif, Size: 1537 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: cat as a builtin command
2014-08-29 1:40 cat as a builtin command Izumi Natsuka
2014-08-29 11:58 ` Roman Neuhauser
2014-08-29 12:24 ` Dirk Heinrichs
@ 2014-08-29 13:03 ` Peter Stephenson
2014-08-30 10:11 ` Mikael Magnusson
2014-09-01 7:53 ` Han Pingtian
3 siblings, 1 reply; 10+ messages in thread
From: Peter Stephenson @ 2014-08-29 13:03 UTC (permalink / raw)
To: zsh-users
On Fri, 29 Aug 2014 09:40:29 +0800
Izumi Natsuka <izumi.natsuka@hotmail.com> wrote:
> Hello, I'm going to write a shell function that provides a basic
> functionality(print the content of a file or stdin) of cat[0], in
> order to avoid forking too many process when I call it in a
> loop[1]. And I have put the following in my zshrc:
If it's just printing the contents of a file, investigate zsh/mapfile.
This turns off multibyte mode to make it easier to do binary files ---
there should be no disadvantage here as you just want a byte stream.
mycat() {
emulate -L zsh
unsetopt multibyte
zmodload zsh/mapfile
print -r $mapfile[$1]
}
This only works for files in the file system, not file descriptors,
pipes etc. For those you can often optimise out the cat entirely.
pws
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: cat as a builtin command
2014-08-29 13:03 ` Peter Stephenson
@ 2014-08-30 10:11 ` Mikael Magnusson
0 siblings, 0 replies; 10+ messages in thread
From: Mikael Magnusson @ 2014-08-30 10:11 UTC (permalink / raw)
To: Zsh Users
On 29 August 2014 15:03, Peter Stephenson <p.stephenson@samsung.com> wrote:
> On Fri, 29 Aug 2014 09:40:29 +0800
> Izumi Natsuka <izumi.natsuka@hotmail.com> wrote:
>> Hello, I'm going to write a shell function that provides a basic
>> functionality(print the content of a file or stdin) of cat[0], in
>> order to avoid forking too many process when I call it in a
>> loop[1]. And I have put the following in my zshrc:
>
> If it's just printing the contents of a file, investigate zsh/mapfile.
> This turns off multibyte mode to make it easier to do binary files ---
> there should be no disadvantage here as you just want a byte stream.
>
>
> mycat() {
> emulate -L zsh
> unsetopt multibyte
> zmodload zsh/mapfile
> print -r $mapfile[$1]
> }
>
>
> This only works for files in the file system, not file descriptors,
> pipes etc. For those you can often optimise out the cat entirely.
>
> pws
mycat() { for i { print -r - "$(< $i)" } }
(also only works for files)
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: cat as a builtin command
2014-08-29 1:40 cat as a builtin command Izumi Natsuka
` (2 preceding siblings ...)
2014-08-29 13:03 ` Peter Stephenson
@ 2014-09-01 7:53 ` Han Pingtian
2014-09-01 18:39 ` Bart Schaefer
3 siblings, 1 reply; 10+ messages in thread
From: Han Pingtian @ 2014-09-01 7:53 UTC (permalink / raw)
To: zsh-users
On Fri, Aug 29, 2014 at 09:40:29AM +0800, Izumi Natsuka wrote:
> Hello, I'm going to write a shell function that provides a basic functionality(print the content of a file or stdin) of cat[0], in order to avoid forking too many process when I call it in a loop[1]. And I have put the following in my zshrc:
>
> cat() {
> if [[ $# -le 1 ]] && [[ ${1:0:1} != '-' || ${1} == '-' ]];then
> local file
> exec {file}<${1:-0}
If there is no argument to cat, this line would be
exec {file}<0
It should be
exec {file}<&0
right? But I get error mesage for it:
localhost% (){
function> local fd
function> exec {fd}<&0
function> read -E -u $fd
function> exec {fd}<&-
function> }
(anon):2: 0: bad file descriptor
(anon):read:3: argument expected: -u
(anon):4: failed to close file descriptor 0: bad file descriptor
0 isn't a good descriptor here?
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: cat as a builtin command
2014-09-01 7:53 ` Han Pingtian
@ 2014-09-01 18:39 ` Bart Schaefer
2014-09-02 1:08 ` Han Pingtian
0 siblings, 1 reply; 10+ messages in thread
From: Bart Schaefer @ 2014-09-01 18:39 UTC (permalink / raw)
To: Han Pingtian, zsh-users
On Sep 1, 3:53pm, Han Pingtian wrote:
}
} It should be
}
} exec {file}<&0
}
} right? But I get error mesage for it:
}
} localhost% (){
} function> local fd
} function> exec {fd}<&0
} function> read -E -u $fd
} function> exec {fd}<&-
} function> }
} (anon):2: 0: bad file descriptor
} (anon):read:3: argument expected: -u
} (anon):4: failed to close file descriptor 0: bad file descriptor
I'm not able to reproduce this. Is this in a newly started shell?
torch% (){
local fd
function> exec {fd}<&0
function> read -E -u $fd
function> exec {fd}<&-
function> }
hello
hello
torch%
I *suspect* that what happened is that while you were experimenting, some
previous "exec <&-" has already closed descriptor 0. Closing stdin is
not fatal to an interactive zsh, it maintains its own descriptor for ZLE
to access /dev/tty.
E.g., if I explicitly do:
torch% exec 0<&-
Then up-history a couple of times and:
torch% (){
local fd
exec {fd}<&0
read -E -u $fd
exec {fd}<&-
}
(anon):2: 0: bad file descriptor
(anon):read:3: argument expected: -u
(anon):4: failed to close file descriptor 0: bad file descriptor
torch%
What's puzzling to me is why line 4 says "0: bad file descriptor" rather
than this:
torch% exec {fd}<&-
zsh: parameter fd does not contain a file descriptor
It appears that only an UNSET parameter name triggers the "does not
contain" error; a set-but-empty parameter is treated as 0 and closes
standard input, which is likely how you got into this situation.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: cat as a builtin command
2014-09-01 18:39 ` Bart Schaefer
@ 2014-09-02 1:08 ` Han Pingtian
0 siblings, 0 replies; 10+ messages in thread
From: Han Pingtian @ 2014-09-02 1:08 UTC (permalink / raw)
To: zsh-users
On Mon, Sep 01, 2014 at 11:39:40AM -0700, Bart Schaefer wrote:
> On Sep 1, 3:53pm, Han Pingtian wrote:
> }
> } It should be
> }
> } exec {file}<&0
> }
> } right? But I get error mesage for it:
> }
> } localhost% (){
> } function> local fd
> } function> exec {fd}<&0
> } function> read -E -u $fd
> } function> exec {fd}<&-
> } function> }
> } (anon):2: 0: bad file descriptor
> } (anon):read:3: argument expected: -u
> } (anon):4: failed to close file descriptor 0: bad file descriptor
>
> I'm not able to reproduce this. Is this in a newly started shell?
>
> torch% (){
> local fd
> function> exec {fd}<&0
> function> read -E -u $fd
> function> exec {fd}<&-
> function> }
> hello
> hello
> torch%
>
> I *suspect* that what happened is that while you were experimenting, some
> previous "exec <&-" has already closed descriptor 0. Closing stdin is
> not fatal to an interactive zsh, it maintains its own descriptor for ZLE
> to access /dev/tty.
>
> E.g., if I explicitly do:
>
> torch% exec 0<&-
>
> Then up-history a couple of times and:
>
> torch% (){
> local fd
> exec {fd}<&0
> read -E -u $fd
> exec {fd}<&-
> }
> (anon):2: 0: bad file descriptor
> (anon):read:3: argument expected: -u
> (anon):4: failed to close file descriptor 0: bad file descriptor
> torch%
Thanks, I cannot reproduce it after a reboot now. I think you are right,
I must closed 0 before the test. Thanks so much.
^ permalink raw reply [flat|nested] 10+ messages in thread