(concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

(concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Kai Großjohann]

In the Gnus source code, we often have constructs like this:

    (concat "/dir/name/" "file-name")

I think this should be replaced with:

    (expand-file-name "file-name" "/dir/name/")

What do you think?

kai
-- 
I like BOTH kinds of music.

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> In the Gnus source code, we often have constructs like this:
> 
>     (concat "/dir/name/" "file-name")
> 
> I think this should be replaced with:
> 
>     (expand-file-name "file-name" "/dir/name/")
> 
> What do you think?

It depends on the contents of "file-name".  If it might contain
something that really needs to be normalized, then it should be handed
to expand-file-name.  If it should be interpreted literally, then
expand-file-name might do something unintended.  Consider, e.g.,
"foo/../bar", where foo is a symlink.

The default maildir in nnmaildir is expressed as
(concat (file-name-as-directory (expand-file-name "~")) "Maildir").
User-supplied maildir names are also normalized once with
expand-file-name, but then after that, dir/file is always accessed
with (concat (file-name-as-directory dir) file).  I think
file-name-as-directory and directory-file-name are idempotent on all
platforms, although they aren't documented as such, AFAICT.  nnmaildir
doesn't depend on their idempotency, as long as user-supplied maildir
names are what would be returned from directory-file-name.


paul

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Kai Großjohann]

On 13 Oct 2000, Paul Jarc wrote:
> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
>> In the Gnus source code, we often have constructs like this:
>> 
>>     (concat "/dir/name/" "file-name")
>> 
>> I think this should be replaced with:
>> 
>>     (expand-file-name "file-name" "/dir/name/")
>> 
>> What do you think?
> 
> It depends on the contents of "file-name".  If it might contain
> something that really needs to be normalized, then it should be
> handed to expand-file-name.  If it should be interpreted literally,
> then expand-file-name might do something unintended.  Consider,
> e.g., "foo/../bar", where foo is a symlink.

Ah.  Right.  Okay, I won't go into a frenzy to change all them concats
to expand-file-name, then :-)

Your comment about expanding a file name once and to reuse the
expanded file name also makes sense.  Hm.

kai
-- 
I like BOTH kinds of music.

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Dave Love]

>>>>> "PJ" == Paul Jarc <prj@po.cwru.edu> writes:

 PJ> It depends on the contents of "file-name".  If it might contain
 PJ> something that really needs to be normalized, then it should be
 PJ> handed to expand-file-name.  If it should be interpreted
 PJ> literally, then expand-file-name might do something unintended.
 PJ> Consider, e.g., "foo/../bar", where foo is a symlink.

I don't understand this.  One of the points about expand-file-name is
that it gets to do magic depending on the file name.  I have changed
such uses of `concat' where I've found it in the sources and I'd need
some convincing that that's wrong, as well as a contribution to the
Lisp manual.

 PJ> I think file-name-as-directory and directory-file-name are
 PJ> idempotent on all platforms, although they aren't documented as
 PJ> such, AFAICT.

I'm not sure why that's relevant, but even if it's generally true in
the absence of handlers -- I don't know -- I don't think you can say
what a random handler might do, e.g. for remote files on an arbitrary
system.

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Paul Jarc]

Dave Love <d.love@dl.ac.uk> writes:
> I don't understand this.  One of the points about expand-file-name is
> that it gets to do magic depending on the file name.

Right.  Now suppose you have a file name that would invoke magic if
expanded, but that you want to interpret literally - maybe it came
from directory-files, for example - so it might contain arbitrary
characters, but you know that it needs no massaging to become the name
of an actual file.  In that case, expand-file-name won't necessarily
give you the file name you want.

>  PJ> I think file-name-as-directory and directory-file-name are
>  PJ> idempotent on all platforms, although they aren't documented as
>  PJ> such, AFAICT.
> 
> I'm not sure why that's relevant,

I'm not sure *that* it's relevant.  I mentioned it only in passing.

> but even if it's generally true in the absence of handlers -- I
> don't know -- I don't think you can say what a random handler might
> do, e.g. for remote files on an arbitrary system.

expand-file-name, file-name-as-directory and directory-file-name are
just string operations.  Handlers aren't involved until I/O is
attempted, AFAIK.


paul

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Daniel Pittman]

On 16 Oct 2000, Paul Jarc <prj@po.cwru.edu> wrote:

> Dave Love <d.love@dl.ac.uk> writes:

[...]

>> but even if it's generally true in the absence of handlers -- I
>> don't know -- I don't think you can say what a random handler might
>> do, e.g. for remote files on an arbitrary system.
> 
> expand-file-name, file-name-as-directory and directory-file-name are
> just string operations.  Handlers aren't involved until I/O is
> attempted, AFAIK.

No, they are not. At least, they are not under XEmacs 20.4, 21.1 and
21.2 and, unless I am very much mistakes, Emacs in any of it's recent
incarnations.

Heck, I believe that TRAMP (a remote file access mechanism) runs under
Emacs 19 and I /know/ that it fails to do the right thing without
handlers being invoked for `file-name-as-directory'.

        Daniel

-- 
Of course I lie to people. But I lie altruistically - for our mutual
good. The lie is the basic building block of good manners. That may
seem mildly shocking to a moralist - but then what isn't?
        -- Quentin Crisp

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Dave Love]

>>>>> "PJ" == Paul Jarc <prj@po.cwru.edu> writes:

 PJ> In that case, expand-file-name won't necessarily give you the
 PJ> file name you want.

I don't understand this.  It sounds like cause for a bug report.

 PJ> Handlers aren't involved until I/O is attempted, AFAIK.

I believe the operations listed under Info (elisp)Magic File Names
accurately reflect the code.  If not, please make a bug report.

[I can speak for Emacs 20 and Emacs 21 development code.  I recall
that XEmacs appeared to have some problems connected with file name
handlers, but I don't recall what they are.]

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Paul Jarc]

Dave Love <d.love@dl.ac.uk> writes:
>  PJ> In that case, expand-file-name won't necessarily give you the
>  PJ> file name you want.
> 
> I don't understand this.  It sounds like cause for a bug report.

Not with regard to the implementation of expand-file-name, but perhaps
with various uses of it.  Consider "foo/../bar".  expand-file-name
will normalize it to "bar", but it's possible that foo is a symlink to
something other than a sibling directory.  In that case, "foo/../bar"
and "bar" might *both* name existing files, but not the same file.  Or
one might name an existing file, and the other might not.  Which
string names the file we actually want?  In each particular case of
deciding whether to use expand-file-name, you have to decide which
file you would want: the file named literally by the string you've
got, or the file named by the string's expansion.  There is no general
answer; you have to decide case by case.  Also, you want to be careful
not to expand an already-expanded name, if expand-file-name is indeed
not necessarily idempotent.  Otherwise you'll be dealing with two
different filenames, when you really want to work with just one file.

>  PJ> Handlers aren't involved until I/O is attempted, AFAIK.
> 
> I believe the operations listed under Info (elisp)Magic File Names
> accurately reflect the code.  If not, please make a bug report.

I was probably mistaken there, then.  I can't find that node in my
manual, but the Index node is inaccessible, so maybe my manual is
broken.
#    This Info file contains edition 2.5 of the GNU Emacs Lisp Reference
# Manual, corresponding to GNU Emacs version 20.3.
or is there a newer manual?


paul

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Dave Love]

>>>>> "PJ" == Paul Jarc <prj@po.cwru.edu> writes:

 PJ> In each particular case of deciding whether to use
 PJ> expand-file-name,

I don't think you should be making any such decision.

 PJ> you have to decide which file you would want: the file named
 PJ> literally by the string you've got, or the file named by the
 PJ> string's expansion.

If you think you should be able to suppress contraction of ../ for
some reason, make a bug report with enough justification.  I'm not
convinced.

 PJ> I was probably mistaken there, then.  

And, I think, elsewhere.  If in doubt, read the source.

 PJ> I can't find that node in my manual, but the Index node is
 PJ> inaccessible, so maybe my manual is broken.

Yes.  Please fix it and read the relevant parts.  If they're not
clear, please make a bug report.  I doubt we've changed much
significant in that area, but there is a tarball of a more recent
version of the Lisp manual somewhere on alpha.gnu.org.

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Paul Jarc]

Dave Love <d.love@dl.ac.uk> writes:
>  PJ> In each particular case of deciding whether to use
>  PJ> expand-file-name,
> 
> I don't think you should be making any such decision.

Let me rephrase.  Whether a filename is to be expanded should be a
documented part of any interface that uses filenames.  Expansion is
generally useful for users, so it would be done for most or all user
interfaces, but not necessarily for programmatic interfaces.  (This
has the cost that a user typically can't use a name like "foo/../bar"
literally when foo is a symlink, but we can live with that.)  Then
programs have to be careful to avoid, e.g., passing an expanded (or
not expanded) name to another interface that expects an unexpanded (or
expanded) name.  You want to avoid multiple expansions in general,
since expansion need not be idempotent, and you want to avoid
expanding a name which is known to be literal but which might invoke
magic - e.g., it came from directory-files - because its expansion
might be a different name, naming a different file.  So take a name
given by the user, expand it once, then keep that expanded name for
future use, and for building other filenames.  For example:
(setq expanded-name (expand-file-name directory-name-from-user)) ; once
(setq directory-name (file-name-as-directory expanded-name))     ; once
(concat directory-name literal-relative-name)                    ; whenever

Avoiding multiple expansions also helps with performance, regardless
of the correctness problems that might occur.

> If you think you should be able to suppress contraction of ../ for
> some reason, make a bug report with enough justification.  I'm not
> convinced.

../ was perhaps a bad example.  But I can't make a bug report, because
I haven't audited all the code that uses expand-file-name to see where
names might be expanded twice.  It would be easier and faster for the
various people who've written the code to audit it themselves, if they
hadn't thought of this issue when they wrote it.  Then they'll also be
more aware of the potential problem when they write new code.

> I doubt we've changed much significant in that area, but there is a
> tarball of a more recent version of the Lisp manual somewhere on
> alpha.gnu.org.

Got it, thanks.  Yes, I see that expand-file-name,
directory-file-name, and file-name-as-directory are subject to
handlers, so idempotency cannot be generally guaranteed.  It should
probably be a goal of any handler, though.


paul

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Kai Großjohann]

I have tried hard to come up with a reason why expand-file-name could
fail to be idempotent, but I can't come up with a reason.  The only
possibility I can come up with is that file name expansion might be a
multi-step operation, and expand-file-name might only make one of
those steps.  But I think that expand-file-name should be doing all of
the steps right at the first call.

But I agree that functions which expect file names might need to
document whether or not expand-file-name is called on those file
names.  OTOH, it's just an efficiency issue if expand-file-name is
idempotent.

But in any case, using concat to create a file name from a directory
and a relative path name is not a good thing, I think.

kai
-- 
I like BOTH kinds of music.

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> I have tried hard to come up with a reason why expand-file-name could
> fail to be idempotent, but I can't come up with a reason.

A mischevious, badly-written, or maybe just an unconventional handler
might give an expanded name that would, when re-expanded, give a
different name.  But this probably doesn't happen in most normal
situations.

> But in any case, using concat to create a file name from a directory
> and a relative path name is not a good thing, I think.

Why not?


paul

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Kai Großjohann]

On 22 Oct 2000, Paul Jarc wrote:
> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
>> I have tried hard to come up with a reason why expand-file-name
>> could fail to be idempotent, but I can't come up with a reason.
> 
> A mischevious, badly-written, or maybe just an unconventional
> handler might give an expanded name that would, when re-expanded,
> give a different name.  But this probably doesn't happen in most
> normal situations.

So it's a bug when expand-file-name fails to be idempotent?  (What's
the opposite of idempotent?  Idemimpotent?)

>> But in any case, using concat to create a file name from a
>> directory and a relative path name is not a good thing, I think.
> 
> Why not?

Hm.  Maybe it isn't too bad after all, but one has to be very careful
to say (concat (file-name-as-directory D) F), not just (concat D F).
But I think you were saying that file-name-as-directory might not be
idempotent, either, right?  So we have just shifted the problem.

kai
-- 
I like BOTH kinds of music.

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> On 22 Oct 2000, Paul Jarc wrote:
> > A mischevious, badly-written, or maybe just an unconventional
> > handler might give an expanded name that would, when re-expanded,
> > give a different name.  But this probably doesn't happen in most
> > normal situations.
> 
> So it's a bug when expand-file-name fails to be idempotent?

In such a case, it's a bug to expand an already-expanded name, yes.  I
don't know whether any code actually has this bug.

> Hm.  Maybe it isn't too bad after all, but one has to be very careful
> to say (concat (file-name-as-directory D) F), not just (concat D F).
> But I think you were saying that file-name-as-directory might not be
> idempotent, either, right?  So we have just shifted the problem.

True, if you don't know whether D is as-directory to begin with.  But
expand-file-name does other things as well; file-name-as-directory
does only one simple thing, so it has a better chance of being
idempotent, even if only by accident.


paul

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Daniel Pittman]

On Wed, 18 Oct 2000, Dave Love <d.love@dl.ac.uk> wrote:

>>>>>> "PJ" == Paul Jarc <prj@po.cwru.edu> writes:
> 
>  PJ> In that case, expand-file-name won't necessarily give you the
>  PJ> file name you want.
> 
> I don't understand this.  It sounds like cause for a bug report.
> 
>  PJ> Handlers aren't involved until I/O is attempted, AFAIK.
> 
> I believe the operations listed under Info (elisp)Magic File Names
> accurately reflect the code.  If not, please make a bug report.
> 
> [I can speak for Emacs 20 and Emacs 21 development code.  I recall
> that XEmacs appeared to have some problems connected with file name
> handlers, but I don't recall what they are.]

I don't know of any particular issues with XEmacs vs filename handlers,
and have worked in the area. If you do happen to find what they are,
please let me know.

        Daniel

-- 
You may be a geek, you may have geek written all over you; you should aim to
be one geek they'll never forget. Don't aim to be civilized.
        -- Bruce Sterling

Re: (concat "/dir/name/" "foo") --> (expand-file-name "foo" "/dir/name/")

[Dave Love]

>>>>> "DP" == Daniel Pittman <daniel@rimspace.net> writes:

 DP> I don't know of any particular issues with XEmacs vs filename
 DP> handlers, and have worked in the area. If you do happen to find
 DP> what they are, please let me know.

Hrvoje was aware of whatever it is/was.