Re: zsh and line breaks

["Peter A. Castro" <doctor@fruitbat.org>]

On Thu, 1 Apr 2004, Oliver Kiddle wrote:

Warning: this is long!!

> "Peter A. Castro" wrote:
> > > It is easy for us to add `#ifdef __CYGWIN__' around changes or #define
> > > O_TEXT to zero on other systems so if you do correct the problem,
> > > please send the changes back to us.
> >
> > There are about 43 open() calls which I've updated with the O_TEXT
> > option.  Having all those ifdef's seemed rather ugly (makes the code hard
> > to look at, expectially when they are within a few lines of each other)
> > so I took a more "elegent" approach, though you may want to revise it if
> > it doesn't meet your style requirements :)
>
> I can believe that adding ifdef's to all is ugly. That's what I meant
> by "#define O_TEXT to zero on other systems" - just one thought on a
> possible more "elegant" approach.

I might as well describe what I've done so you can give it a thumbs up or
down.  In system.h I've added a #define in the #ifdef __CYGWIN__ section of:
#define ORO_TEXT | O_TEXT
And for the #else case:
#define ORO_TEXT

Then in code which needs it I have modified it to look like this:

if ((fd = open(name, O_RDONLY ORO_TEXT)) < 0) {

It's really just utilizing the macro ability of the compiler, and it's a
style judgement call.  I don't have a problem with it, obviously, but
others might look at it and wonder how it could possible compile if they
didn't look in system.h first.  If you think that might present
confusion, then I'll change it to be explicitly "| OR_TEXT" and have
#define O_TEXT to be 0 if not defined at all.

Any thoughts on this?

> > Yep, I'm experimenting with this right now.  As it stands, tests which
> > print out to a file and then cat it back in (currently A04redirect and
> > E01options) produce a diff, but don't otherwise seem to have any
> > problems.
>
> Are the diffs just the line endings? From what I understand, the
> reported problems were with text files used as input to the shell (i.e.
> scripts, sourced files, autoloaded functions and stdin). Quite whether
> it is also right for redirected output, I wouldn't know.

The primary problem is with running scripts with CR/LFs.  That gets fixed
with adding O_TEXT everywhere.  A secondary problem is with redirected
input and/or output which is processed by the shell.

I've been reviewing this problem and I think maybe I've been attacking it
incorrectly.  I had though that adding O_TEXT everywhere would solve this
problem.  However, the environment is complicating things :)

Here's the deal: Cygwin can "mount" a part of the native Windows
filesystem into the virtual Cygwin filesystem with an attribute of
textmode or binmode.  The meaning of this is on textmode mounts, files
opened as "text files" (ie: not as a binary file), will have CR/LF
translation performed on them (ie: reads will remove CR and writes will
write CR/LF).  On binmode mounts, files opened as "text files" will
retain their line terminators, so no translation is performed on either
reads or writes.  So where the file located in the filesystem determines
the default handling of translation when opened as a text file.
This is for normal unix style coding of opens without any O_TEXT or
O_BINARY cruft.

Now, adding an explicit O_TEXT or O_BINARY forces one mode or the other,
ignoring the filesystem mount attributes.  The problem is, I don't want
to force the mode, at least not on writes.  See, if the user has his
files on a textmode mounted filesystem, he'll be expecting files written
to it to have CR/LFs.  If his files are on a binmode mounted filesystem,
he'll be expecting files written to not have any CRs.  But, zsh,
internally, can't handle CRs (unless I hack the parser...no, don't want
to do that!), so I need to ensure that all opened files for read are
forced via O_TEXT (ie: strip off CRs), while allowing opened files for
write to default to the filesystem mode.  The trouble is, there are
several places in the code where files are opened as Read/Write!

This gets more complicated when, say, a user opens normal scripts/files
on a binmode mount, but /tmp (the place zsh makes temp files) is mounted
in textmode!  I need to analyse the code a little more to determine just
what the RW files that zsh opens are used for.  If it's purely internal
usage and never provide as input or output to external programs via pipes
or redirection, then I can make it O_BINARY and that should solve things.
But, if not... well... I'm not sure how to handle it (yet)...  First the
analyse.

I'd remember going through this same exercise a while back with 4.0 of
Zsh, and remember that I had the same issues and decided it would be
better to simply tell people to keep their mounts consistent with their
data files.  Now, however, I really want to fix this to work correctly.

Thanks for listening.  Any suggestions are welcome.

> Oliver

-- 
Peter A. Castro <doctor@fruitbat.org> or <Peter.Castro@oracle.com>
	"Cats are just autistic Dogs" -- Dr. Tony Attwood