use of (defvar <foo>)

use of (defvar <foo>)

[Dave Love]

Gnus has lots of instances of `(defvar <foo>)' that aren't protected
by `eval-when-compile' and should be.  I'll fix them when I get a
chance, but people should beware of that in future.

Re: use of (defvar <foo>)

[Reiner Steib]

On Thu, Apr 06 2006, Dave Love wrote:

> Gnus has lots of instances of `(defvar <foo>)' that aren't protected
> by `eval-when-compile' and should be.

I learned on emacs-devel that we don't need `eval-when-compile' in
this case (unless I'm misunderstanding what you are referring to):

,----[ http://article.gmane.org/gmane.emacs.devel/42998 ]
| From: Stefan Monnier
| Subject: Re: defvars at compile time
| 
| [...]
| 
| The form (defvar foo) was specifically designed as a byte-compiler
| directive, so please make use of it.
`----

,----[ http://article.gmane.org/gmane.emacs.devel/43027 ]
| From: Richard M. Stallman
| Subject: Re: defvars at compile time
|
| The only effect of a defvar with no initial value
| is to silence the compiler.  So it is superfluous
| to put it inside eval-when-compile.
`----

We had this discussion about it recently:
http://thread.gmane.org/gmane.emacs.devel/51304/focus=11990

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: use of (defvar <foo>)

[Dave Love]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> ,----[ http://article.gmane.org/gmane.emacs.devel/43027 ]
> | From: Richard M. Stallman
> | Subject: Re: defvars at compile time
> |
> | The only effect of a defvar with no initial value
> | is to silence the compiler.  So it is superfluous
> | to put it inside eval-when-compile.
> `----

That's certainly not true in Emacs 21:

(progn
  (with-temp-file "/tmp/x.el"
    (insert "(defvar foo)\n"))
  (byte-compile-file "/tmp/x.el")
  (with-temp-buffer 
    (insert-file-contents-literally "/tmp/x.elc")
    (goto-char (point-max))
    (delete-region 1 (line-beginning-position 0))
    (buffer-string)))
  => "(byte-code \"\\301\bB\x10\\301\\207\" [current-load-list foo] 2)
"

You can completely screw Emacs if you unload a feature that does that
on important variables.  It also causes confusion with find-function &
al.  I fixed a fair number of cases in the Emacs source that could
clobber things like `font-lock-keywords' and documented using
eval-when-compile in the Lisp manual.  I don't remember what the
justification was for the one-arg defvar modifying the load history at
the time; that sort of policy decision often gets reversed.

Re: use of (defvar <foo>)

[Reiner Steib]

On Fri, Apr 07 2006, Dave Love wrote on ding@gnus.org:

> Reiner Steib <reinersteib+gmane@imap.cc> writes:

[ Quote from initial mail added for emacs-devel ]
>> On Thu, Apr 06 2006, Dave Love wrote:
>>> Gnus has lots of instances of `(defvar <foo>)' that aren't protected
>>> by `eval-when-compile' and should be.

>> ,----[ http://article.gmane.org/gmane.emacs.devel/43027 ]
>> | From: Richard M. Stallman
>> | Subject: Re: defvars at compile time
>> |
>> | The only effect of a defvar with no initial value
>> | is to silence the compiler.  So it is superfluous
>> | to put it inside eval-when-compile.
>> `----
>
> That's certainly not true in Emacs 21:
>
> (progn
>   (with-temp-file "/tmp/x.el"
>     (insert "(defvar foo)\n"))
>   (byte-compile-file "/tmp/x.el")
>   (with-temp-buffer 
>     (insert-file-contents-literally "/tmp/x.elc")
>     (goto-char (point-max))
>     (delete-region 1 (line-beginning-position 0))
>     (buffer-string)))
>   => "(byte-code \"\\301\bB\x10\\301\\207\" [current-load-list foo] 2)
> "
>
> You can completely screw Emacs if you unload a feature that does that
> on important variables.  It also causes confusion with find-function &
> al.  I fixed a fair number of cases in the Emacs source that could
> clobber things like `font-lock-keywords' and documented using
> eval-when-compile in the Lisp manual.  I don't remember what the
> justification was for the one-arg defvar modifying the load history at
> the time; that sort of policy decision often gets reversed.

Could people on emacs-devel please comment on this?

Should we use (defvar <foo>) or (eval-when-compile (defvar <foo>)) in
Gnus for compatibility with Emacs 21?  (Probably the same holds for
MH-E.)

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: use of (defvar <foo>)

[Stefan Monnier]

> Could people on emacs-devel please comment on this?
> Should we use (defvar <foo>) or (eval-when-compile (defvar <foo>)) in Gnus
> for compatibility with Emacs 21?  (Probably the same holds for MH-E.)

>From the future's point of view (Emacs-22 and up), placing such defvars
inside eval-when-compile is a bad practice (in the future it may even fail
to work since there's nothing that guarantees that the content of
eval-when-compile is actually compiled rather than just eval'd).

In Emacs-21 and older, such a defvar will have some undesirable minor
side-effects (via the load-history) unless it is placed inside an
eval-when-compile *and* the file is byte-compiled.  Indeed placing the
defvar inside the eval-when-compile has no effect if the file is not
byte-compiled.

I'd recommend to not use eval-when-compile since that's what many packages
have been doing for many years without suffering much (if ever) of the
occasional side-effects (which only affect unload-feature and sometimes
C-h v).

But it's your call, really.


        Stefan

Re: use of (defvar <foo>)

[Bill Wohler]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Could people on emacs-devel please comment on this?
>> Should we use (defvar <foo>) or (eval-when-compile (defvar <foo>)) in Gnus
>> for compatibility with Emacs 21?  (Probably the same holds for MH-E.)
>
>>From the future's point of view (Emacs-22 and up), placing such defvars
> inside eval-when-compile is a bad practice (in the future it may even fail
> to work since there's nothing that guarantees that the content of
> eval-when-compile is actually compiled rather than just eval'd).
>
> In Emacs-21 and older, such a defvar will have some undesirable minor
> side-effects (via the load-history) unless it is placed inside an
> eval-when-compile *and* the file is byte-compiled.  Indeed placing the
> defvar inside the eval-when-compile has no effect if the file is not
> byte-compiled.
>
> I'd recommend to not use eval-when-compile since that's what many packages
> have been doing for many years without suffering much (if ever) of the
> occasional side-effects (which only affect unload-feature and sometimes
> C-h v).
>
> But it's your call, really.

I was planning on stripping the eval-when-compile defvar wrappers for
the MH-E 8.0/Emacs 22.1 release. Reiner, is the problem you describe
real or theoretical?

-- 
Bill Wohler <wohler@newt.com>  http://www.newt.com/wohler/  GnuPG ID:610BD9AD
Maintainer of comp.mail.mh FAQ and MH-E. Vote Libertarian!
If you're passed on the right, you're in the wrong lane.

Re: use of (defvar <foo>)

[Dave Love]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>From the future's point of view (Emacs-22 and up), placing such defvars
> inside eval-when-compile is a bad practice (in the future it may even fail
> to work since there's nothing that guarantees that the content of
> eval-when-compile is actually compiled rather than just eval'd).

Huh?  You don't want it compiled -- that's the point.  You're actually
considering changing what eval-when-compile does and breaking more
existing code??  Anyway, the point is that Gnus is supposed to be
supporting Emacs 21 (and XEmacs), the eval-when-compile DTRT
everywhere, and Emacs 22 looks rather a forlorn hope.  The
compatibility stuff is typically ugly, but this idiom isn't.

> In Emacs-21 and older, such a defvar will have some undesirable minor
> side-effects (via the load-history) unless it is placed inside an
> eval-when-compile *and* the file is byte-compiled.  Indeed placing the
> defvar inside the eval-when-compile has no effect if the file is not
> byte-compiled.

That's the point.  Without the wrapper it prevents you using the
load-history mechanisms reliably (as I was trying to do to contribute
to Gnus).

> I'd recommend to not use eval-when-compile since that's what many packages
> have been doing for many years without suffering much (if ever) of the
> occasional side-effects (which only affect unload-feature and sometimes
> C-h v).

Actually, such usage was fixed widely in Emacs 21 because it caused
trouble -- try to test a change by unloading a feature and render your
session unusable.

Re: use of (defvar <foo>)

[David Kastrup]

Dave Love <fx@gnu.org> writes:

> Actually, such usage was fixed widely in Emacs 21 because it caused
> trouble -- try to test a change by unloading a feature and render
> your session unusable.

Unloading features is so unreliable in Emacs 21 that there is little
point in accommodating it.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: use of (defvar <foo>)

[Dave Love]

David Kastrup <dak@gnu.org> writes:

> Unloading features is so unreliable in Emacs 21 that there is little
> point in accommodating it.

Actually doing the work (when it was possible to help) obviously gave
me a different perspective.  I realize it's not relevant, though.

Re: use of (defvar <foo>)

[Stefan Monnier]

>> From the future's point of view (Emacs-22 and up), placing such defvars
>> inside eval-when-compile is a bad practice (in the future it may even fail
>> to work since there's nothing that guarantees that the content of
>> eval-when-compile is actually compiled rather than just eval'd).

> Huh?  You don't want it compiled -- that's the point.

By "compiled" I only mean that it goes through the byte-compiler (before
being eval'd).  Not that the resulting byte-code is written in the
output file.

If the content of eval-when-compile were not compiled, then
(eval-when-compile (defvar foo)) would be 100% equivalent to a nop.
It's only because it's both compiled&eval'd that the byte-compiler notices
the defvar and thus silences warnings about subsequent uses of
this variable.

> You're actually considering changing what eval-when-compile does and
> breaking more existing code??

Compiling the content of eval-when-compile before eval'ing doesn't
remove/hide any bug AFAIK, quite the opposite.


        Stefan

Re: use of (defvar <foo>)

[Dave Love]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Huh?  You don't want it compiled -- that's the point.

[Actually, I meant the code rather than the eval result.]

> By "compiled" I only mean that it goes through the byte-compiler (before
> being eval'd).

It goes through the compiler _after_ being evalled, by definition.

> If the content of eval-when-compile were not compiled, then
> (eval-when-compile (defvar foo)) would be 100% equivalent to a nop.

But it isn't compiled as such.

> Compiling the content of eval-when-compile before eval'ing doesn't
> remove/hide any bug AFAIK, quite the opposite.

Doing that would be a bug.

If the effect of the defvar on load-history has been redefined as a
bug, the eval-when-compile wrapper does avoid a bug, and I don't see
how it could reasonably be harmful in a future Emacs.  CL's
`(declare (special ...))' often is cleaner, though.

Re: use of (defvar <foo>)

[David Kastrup]

Dave Love <fx@gnu.org> writes:

> If the effect of the defvar on load-history has been redefined as a
> bug, the eval-when-compile wrapper does avoid a bug, and I don't see
> how it could reasonably be harmful in a future Emacs.

defvar is the main way to inform the system a variable could be bound
dynamically instead of just lexically.  Having it omitted on runtime
could lead to hard to find bugs if we change to lexical binding at one
time.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: use of (defvar <foo>)

[Dave Love]

David Kastrup <dak@gnu.org> writes:

> defvar is the main way to inform the system a variable could be bound
> dynamically instead of just lexically.  Having it omitted on runtime
> could lead to hard to find bugs if we change to lexical binding at one
> time.

[I'll believe that when I see it (even more than Emacs 22).]

defvars are often not added with the intention of actually
distinguishing dynamic and putative lexical binding, but specifically
to suppress the compiler warning when people care.  They may well not
be in the right place to signal that bindings are lexical.

If that ever becomes an issue, you need diagnostics for the existing
code which correctly (in Emacs 21) uses the guard, and for `declare',
at least.

Re: use of (defvar <foo>)

[Stefan Monnier]

>>> Huh?  You don't want it compiled -- that's the point.
> [Actually, I meant the code rather than the eval result.]
>> By "compiled" I only mean that it goes through the byte-compiler (before
>> being eval'd).
> It goes through the compiler _after_ being evalled, by definition.

No, go check bytecomp.el: the code in eval-when-compile is first compiled;
then the resulting byte-code is evaluated and then the resuting value goes
through the normal processing.

It's this first compilation step before eval'ing which notices the `defvar'
and makes it that (eval-when-compile (defvar foo)) is not a complete noop.
If the contents were not compiled before evaluation, then (eval-when-compile
(defvar foo)) would have no effect whatsoever (i.e. it wouldn't silence the
byte compiler's warnings).

>> If the content of eval-when-compile were not compiled, then
>> (eval-when-compile (defvar foo)) would be 100% equivalent to a nop.

> But it isn't compiled as such.

Judge for yourself:

    (eval-when-compile . (lambda (&rest body)
			   (list 'quote
				 (byte-compile-eval (byte-compile-top-level
						     (cons 'progn body))))))

`byte-compile-eval' is just the evaluation function, but
byte-compile-top-level is the normal byte-compilation function, and I'd argue
that it should be removed (and if it is removed, then (eval-when-compile
(defvar foo)) won't have any silencing effect any more).


        Stefan

Re: use of (defvar <foo>)

[Dave Love]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> No, go check bytecomp.el: the code in eval-when-compile is first compiled;
> then the resulting byte-code is evaluated and then the resuting value goes
> through the normal processing.

I beg your pardon.  Either I never knew how it works or, more likely,
I forgot from when there were issues in that area.

Then the documentation is at best misleading in just saying that
`The result of the body appears to the compiler as a quoted constant.'

Re: use of (defvar <foo>)

[Richard Stallman]

    Should we use (defvar <foo>) or (eval-when-compile (defvar <foo>)) in
    Gnus for compatibility with Emacs 21?  (Probably the same holds for
    MH-E.)

Use plain defvar.  That has been the standard way to quiet
a compiler warning since the beginning.

Re: use of (defvar <foo>)

[Dave Love]

Richard Stallman <rms@gnu.org> writes:

> Use plain defvar.  That has been the standard way to quiet
> a compiler warning since the beginning.

In the beginning the load-history features didn't exist, so that
didn't cause any trouble.  In Emacs 21 this use of eval-when-compile
was documented after discussion about defvar affecting load-history,
with a policy decision from either you or gerd.  The policy may have
changed, but that's irrelevant for people working with the available
Emacs.

Re: use of (defvar <foo>)

[Lars Magne Ingebrigtsen]

Dave Love <fx@gnu.org> writes:

>> | The only effect of a defvar with no initial value
>> | is to silence the compiler.  So it is superfluous
>> | to put it inside eval-when-compile.
>> `----
>
> That's certainly not true in Emacs 21:

If Emacs 21 has screwed this up, then Emacs 21 should be fixed, in my
opinion.

If you have to wrap every single one-argument defvar with
`eval-when-compile', then single-argument defvars should do that
itself automatically.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen