gnus-agentize versus emacs 20 mapc

gnus-agentize versus emacs 20 mapc

[Kevin Ryde]

I tried to startup today's cvs under emacs 20 but it died in
gnus-agentize, apparently missing mapc

    Signaling: (void-function mapc)
      mapc(#[(server-or-method) "^H\211^Y\203^O^@   <\203^O^@       \206\227^@\306 \$
      gnus-agentize()
      gnus-1(nil nil nil)
      gnus(nil)
      call-interactively(gnus)
      execute-extended-command(nil)
      call-interactively(execute-extended-command)

Does one have to require 'cl to use mapc, not just eval-when-compile?
I'm not up with the way that stuff is supposed to work.

If gnus-agentize is only iterating over a few methods then perhaps
just mapcar would be enough.

Re: gnus-agentize versus emacs 20 mapc

[Katsumi Yamaoka]

>>>>> In <87oev1uyzw.fsf@zip.com.au>
>>>>>	Kevin Ryde <user42@zip.com.au> wrote:

> I tried to startup today's cvs under emacs 20 but it died in
> gnus-agentize, apparently missing mapc

>     Signaling: (void-function mapc)

[...]

> Does one have to require 'cl to use mapc, not just eval-when-compile?
> I'm not up with the way that stuff is supposed to work.

> If gnus-agentize is only iterating over a few methods then perhaps
> just mapcar would be enough.

I have the same opinion with you.  Developers, if you think Gnus
still supports Emacs 20, please do so.  Or should I add the
compiler macro to dgnushack.el for `mapc'?
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: gnus-agentize versus emacs 20 mapc

[Kevin Greiner]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

>>>>>> In <87oev1uyzw.fsf@zip.com.au>
>>>>>>	Kevin Ryde <user42@zip.com.au> wrote:
>
>> I tried to startup today's cvs under emacs 20 but it died in
>> gnus-agentize, apparently missing mapc
>
>>     Signaling: (void-function mapc)
>
> [...]
>
>> Does one have to require 'cl to use mapc, not just eval-when-compile?
>> I'm not up with the way that stuff is supposed to work.

Macros can be evaluated during compilation as they get expanded by
the compiler.  However, functions such as mapc must be evaluated at
runtime.

>> If gnus-agentize is only iterating over a few methods then perhaps
>> just mapcar would be enough.
>
> I have the same opinion with you.  Developers, if you think Gnus
> still supports Emacs 20, please do so.  Or should I add the
> compiler macro to dgnushack.el for `mapc'?

The mapc function currently appears in 
  gnus-agent.el
  gnus-registry.el
  gnus.el
  spam-stat.el
  spam.el
so it would appear that multiple authors have made the same mistake.
Rather than perpetuate the issue, I'd vote for the compiler macro in
dgnushack.el.  Can someone who uses Emacs 20 write and test it?

Kevin

Re: gnus-agentize versus emacs 20 mapc

[Katsumi Yamaoka]

>>>>> In <uk75prtzg.fsf@xpediantsolutions.com>
>>>>>	Kevin Greiner <kgreiner@xpediantsolutions.com> wrote:

> The mapc function currently appears in 
>   gnus-agent.el
>   gnus-registry.el
>   gnus.el
>   spam-stat.el
>   spam.el
> so it would appear that multiple authors have made the same mistake.
> Rather than perpetuate the issue, I'd vote for the compiler macro in
> dgnushack.el.  Can someone who uses Emacs 20 write and test it?

I've added it.
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: gnus-agentize versus emacs 20 mapc

[Kevin Ryde]

Kevin Greiner <kgreiner@xpediantsolutions.com> writes:
>
>   spam-stat.el

Looks like this one has its own trickery to drag in 'cl when required.
With dgnushack doing it perhaps that's no longer wanted.

Re: gnus-agentize versus emacs 20 mapc

[Katsumi Yamaoka]

>>>>> In <87k75oktn7.fsf@zip.com.au>
>>>>>	Kevin Ryde <user42@zip.com.au> wrote:

> Kevin Greiner <kgreiner@xpediantsolutions.com> writes:
>>
>>   spam-stat.el

> Looks like this one has its own trickery to drag in 'cl when required.
> With dgnushack doing it perhaps that's no longer wanted.

That is only for smap-stat and Emacs 20 users.  However, mapc
you wrote about is in gnus-agent.el and gnus.el which may be
used by most of Gnus and Emacs 20 users.
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Using CL in Gnus (was: gnus-agentize versus emacs 20 mapc)

[Reiner Steib]

On Wed, Nov 26 2003, Katsumi Yamaoka wrote:
>>>>>>	Kevin Ryde <user42@zip.com.au> wrote:

[ spam-stat.el ]
>> Looks like this one has its own trickery to drag in 'cl when required.
>> With dgnushack doing it perhaps that's no longer wanted.
>
> That is only for smap-stat and Emacs 20 users.  However, mapc
> you wrote about is in gnus-agent.el and gnus.el which may be
> used by most of Gnus and Emacs 20 users.

IMHO, the point isn't how many users are affected by loading cl, but
that all GNU Emacs packages (thus Gnus) should not load it at run time
at all:

,----[ (info "(cl)Overview") in Emacs CVS HEAD ]
|    *Please note:* the "CL" functions are not standard parts of the
| Emacs Lisp name space, so it is legitimate for users to define them
| with other, conflicting meanings.  To avoid conflicting with those user
| activities, we have a policy that packages installed in Emacs must not
| load "CL" at run time.  (It is ok for them to load "CL" at compile time
| only, with `eval-when-compile', and use the macros it provides.)  If
| you are writing packages that you plan to distribute and invite
| widespread use for, you might want to observe the same rule.
`----

OTOH, Gnus 5.10.x (labeled Gnus 5.11) will be included in Emacs 21.y
where the relevant functions for `spam-stat.el' (gethash
hash-table-count make-hash-table mapc puthash) are built-in functions
already.  So I'm not sure how relevant it is to avoid mapc and friends
in Gnus 5.10.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: Using CL in Gnus

[Katsumi Yamaoka]

>>>>> In <v9y8u3cqiq.fsf@marauder.physik.uni-ulm.de>
>>>>>	Reiner Steib <4.uce.03.r.s@nurfuerspam.de> wrote:

> IMHO, the point isn't how many users are affected by loading cl, but
> that all GNU Emacs packages (thus Gnus) should not load it at run time
> at all:

That's quite right.  However, Gnus 5.11 to be shipped is for
Emacs 21.4, not for Emacs 20.  And the latest compiled Gnus
doesn't require cl in Emacs 21 as far as I know.

> OTOH, Gnus 5.10.x (labeled Gnus 5.11) will be included in Emacs 21.y
> where the relevant functions for `spam-stat.el' (gethash
> hash-table-count make-hash-table mapc puthash) are built-in functions
> already.  So I'm not sure how relevant it is to avoid mapc and friends
> in Gnus 5.10.

Well, is it impossible that Gnus 5.11 will not support Emacs 20
as Emacs 19 was once forsaken by the September doctrine?  It is
a block to program freely, isn't it?  Although dgnushack.el
probably will not be contained in Emacs 21.4, there are odd
items and old fashioned codes for Emacs 20 here there and
elsewhere (which also means that it may be hard to remove
them).
-- 
Katsumi Yamaoka <yamaoka@jpl.org>

Re: Using CL in Gnus

[Reiner Steib]

On Wed, Nov 26 2003, Katsumi Yamaoka wrote:

>>>>>> In <v9y8u3cqiq.fsf@marauder.physik.uni-ulm.de>
>>>>>>	Reiner Steib <4.uce.03.r.s@nurfuerspam.de> wrote:
[...]
> That's quite right.  However, Gnus 5.11 to be shipped is for
> Emacs 21.4, not for Emacs 20.  

Yes (this is what I mentioned below).

> And the latest compiled Gnus doesn't require cl in Emacs 21 as far
> as I know.

Yes, it doesn't.

>> OTOH, Gnus 5.10.x (labeled Gnus 5.11) will be included in Emacs 21.y
>> where the relevant functions for `spam-stat.el' (gethash
>> hash-table-count make-hash-table mapc puthash) are built-in functions
>> already.  So I'm not sure how relevant it is to avoid mapc and friends
>> in Gnus 5.10.
>
> Well, is it impossible that Gnus 5.11 will not support Emacs 20 [...]?

[ Gnus 5.11 will be more or less identical to Gnus 5.10.x, as Gnus 5.9
(from Emacs 21.[1-3]) is more or less Gnus 5.8.8.]  IIRC it is still
our policy to support Emacs 20.7 (and XEmacs 21.1) and up, see (info
"(gnus)Emacsen").

> It is a block to program freely, isn't it?  Although dgnushack.el
> probably will not be contained in Emacs 21.4, there are odd items
> and old fashioned codes for Emacs 20 here there and elsewhere (which
> also means that it may be hard to remove them).

I agree.  Probably we should drop Emacs 20.7 (and XEmacs < 21.4?) in
the No Gnus series.  There are not enough developers using and/or
testing with this Emacs versions, so it's (sometimes) unsure that it
really works.

,----[ grep -n -w -e mapc *.el ]
| dgnushack.el:83:  (define-compiler-macro mapc (&whole form fn seq &rest rest)
| gnus-agent.el:602:    (mapc
| gnus-registry.el:294:    (mapc
| gnus.el:3197:    (mapc (lambda (name-method)
| gnus.el:3202:    (mapc
| gnus.el:3204:       (mapc (lambda (name-method)
| spam-stat.el:200:				  mapc puthash)))
| spam-stat.el:433:    (mapc (lambda (l)
| spam-stat.el:496:	    (mapc (lambda (entry)
`----

Do I understand the docs correctly, that mapc and mapcar _only_ differ
in the result they give, but not in the side effects?  Did anyone
check if we could use mapcar instead of mapc in gnus.el and
gnus-agent?  IMHO we should replace them, if there's no disadvantage.
*time passes* ...  Now I tried so (and renamed your compiler-macro in
dgnushack.el) and it worked for me with Emacs 20.7.

For `gnus-registry.el' and `spam-stat.el' it's probably not so easy
(more/other cl-functions).  But for people who want those features,
it's probably okay to load CL at run time when using Emacs 20, as
those are quite advanced features.

To make clear that the code in Gnus 5.10 _should_ not use mapc (beside
the two packages `gnus-registry.el' and `spam-stat.el'), I support
your point from your previous article: "Developers, if you think Gnus
still supports Emacs 20, please do so.".  :-)

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: Using CL in Gnus

[Karl Pflästerer]

On 26 Nov 2003, Reiner Steib <- 4.uce.03.r.s@nurfuerspam.de wrote:

> Do I understand the docs correctly, that mapc and mapcar _only_ differ
> in the result they give, but not in the side effects?  Did anyone

No they differ  more.  `mapc' can take an arbitrary number of sequences
as arguments `mapcar' only one.  Furthermore IMO if you read `mapc' you
know a function is used only for the side effects it has; on the other
hand `mapcar' shouldn't be used for side effects IMO.

> check if we could use mapcar instead of mapc in gnus.el and
> gnus-agent?  IMHO we should replace them, if there's no disadvantage.

Better check for the Emacs version and provide a macro.  People have to
read code and the meaning of mapc is different from the one of mapcar.


[...]
> To make clear that the code in Gnus 5.10 _should_ not use mapc (beside

I don't understand your point here. 


   KP

-- 
      And has thou slain the Jabberwock?
          Come to my arms, my beamish boy!
      O frabjous day!  Callooh!  Callay!'
          He chortled in his joy.   "Lewis Carroll" "Jabberwocky"

Re: Using CL in Gnus

[Reiner Steib]

On Wed, Nov 26 2003, Karl Pflästerer wrote:

> On 26 Nov 2003, Reiner Steib <- 4.uce.03.r.s@nurfuerspam.de wrote:
>
>> Do I understand the docs correctly, that mapc and mapcar _only_ differ
>> in the result they give, but not in the side effects?
>
> No they differ  more.  `mapc' can take an arbitrary number of sequences
> as arguments `mapcar' only one.

I don't doubt your explanations, but I don't understand it from the
doc-strings (GNU Emacs 21.3):

,----[ C-h f mapc RET ]
| mapc is a built-in function.
| (mapc FUNCTION SEQUENCE)
| 
| Apply FUNCTION to each element of SEQUENCE for side effects only.
| Unlike `mapcar', don't accumulate the results.  Return SEQUENCE.
| SEQUENCE may be a list, a vector, a bool-vector, or a string.
`----

,----[ C-h f mapcar RET ]
| mapcar is a built-in function.
| (mapcar FUNCTION SEQUENCE)
| 
| Apply FUNCTION to each element of SEQUENCE, and make a list of the results.
| The result is a list just as long as SEQUENCE.
| SEQUENCE may be a list, a vector, a bool-vector, or a string.
`----

> Furthermore IMO if you read `mapc' you know a function is used only
> for the side effects it has; on the other hand `mapcar' shouldn't be
> used for side effects IMO.

Thanks for your explanations.  Is this the case in the examples of
mapc from gnus.el and gnus-agent.el?

>> check if we could use mapcar instead of mapc in gnus.el and
>> gnus-agent?  IMHO we should replace them, if there's no disadvantage.
>
> Better check for the Emacs version and provide a macro.  People have to
> read code and the meaning of mapc is different from the one of mapcar.
>
> [...]
>> To make clear that the code in Gnus 5.10 _should_ not use mapc (beside
>
> I don't understand your point here. 

Well, if programmers see mapc at many places in Gnus code, they might
be tempted to use it without thinking about possible alternatives
(alternatives w/o using CL functions).  But -as we are in feature
freeze- we will not have a significant amount of _new_ code.  Thus
_if_ there is no disadvantage in using mapcar in gnus.el and
gnus-agent.el (only 4 occurrences in total, see above) we _may_
replace them instead of providing a "compatibility hack".

Maybe it's better if I cease this thread, because I'm sure you and
Katsumi know much more about this (not being ironic at all here!) and
he did the right thing.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: Using CL in Gnus

[Karl Pflästerer]

On 27 Nov 2003, Reiner Steib <- 4.uce.03.r.s@nurfuerspam.de wrote:

> On Wed, Nov 26 2003, Karl Pflästerer wrote:

>> No they differ  more.  `mapc' can take an arbitrary number of sequences
>> as arguments `mapcar' only one.

> I don't doubt your explanations, but I don't understand it from the
> doc-strings (GNU Emacs 21.3):

> ,----[ C-h f mapc RET ]
> | mapc is a built-in function.
> | (mapc FUNCTION SEQUENCE)
> | 
> | Apply FUNCTION to each element of SEQUENCE for side effects only.
> | Unlike `mapcar', don't accumulate the results.  Return SEQUENCE.
> | SEQUENCE may be a list, a vector, a bool-vector, or a string.
> `----

> ,----[ C-h f mapcar RET ]
> | mapcar is a built-in function.
> | (mapcar FUNCTION SEQUENCE)
> | 
> | Apply FUNCTION to each element of SEQUENCE, and make a list of the results.
> | The result is a list just as long as SEQUENCE.
> | SEQUENCE may be a list, a vector, a bool-vector, or a string.
> `----

I tried it now with an Emacs I have here (only for testing it's a
precompiled version and does not really fit in my cygwin environment as
I couldn't compile an Emacs under Cygwin).

And here is another extremly silly difference between Emacs and XEmacs.
The XEmacs `mapc' can take an arbitrary number of lists as arguments as
the original CL `mapc'; the Emacs `mapc' can take only one list as
argument.  I can't understand why the function was written that way.
Someone who knows `mapc' from CL will know that it can work on multiple
lists (and in XEmacs it can).  I can't iagine why soeone would want to
create such a silly incompatibility.

>> Furthermore IMO if you read `mapc' you know a function is used only
>> for the side effects it has; on the other hand `mapcar' shouldn't be
>> used for side effects IMO.

> Thanks for your explanations.  Is this the case in the examples of
> mapc from gnus.el and gnus-agent.el?

Speaking og `gnus-method-to-server' in gnus.el: IMO instead of mapc or
mapcar (which would give the completely wrong intention here for a
reader since the loops are only for the side effects) `dolist' could be
used here.  That's in cl-macs.el so could be used with eval-when-compile
and for a reader is easier to see what happens (IMO).  The same is true
for gnus-agent.el

That leads me to another point of mine.  If we start No Gnus we should
look for a lot of places in the code where mapping over a list is
written like that:
        (let ((lst inputlist))
          (while lst
            (do-something-with-list lst)
            .
            .
            .
            (setq lst (cdr lst))))
To cdr-down a list is not uncommon (above code would be nicer as
recursive version) but `dolist' is IMO a much better alternative at such
places.

>> Better check for the Emacs version and provide a macro.  People have to
>> read code and the meaning of mapc is different from the one of mapcar.

>> [...]
>>> To make clear that the code in Gnus 5.10 _should_ not use mapc (beside

>> I don't understand your point here. 

> Well, if programmers see mapc at many places in Gnus code, they might
> be tempted to use it without thinking about possible alternatives

That might be true; but a programmer who doesn't think why he uses mapc
instead of mapcar or dolist at a special place should not provide code
for Gnus.

> (alternatives w/o using CL functions).  But -as we are in feature

CL functions are not evil.

> freeze- we will not have a significant amount of _new_ code.  Thus
> _if_ there is no disadvantage in using mapcar in gnus.el and
> gnus-agent.el (only 4 occurrences in total, see above) we _may_
> replace them instead of providing a "compatibility hack".

Think about dolist.  Also if syntactilly you may use mapcar at these
places, semantically it would be completely wrong.
Here a famous quote I like to make my point clear:
,----
| "Programs must be written for people to read, and only incidentally
| for machines to execute."
|                 -- Abelson & Sussman, SICP (preface to the first edition)
`----



   KP

-- 
 `Beware the Jabberwock, my son!
     The jaws that bite, the claws that catch!
  Beware the Jubjub bird, and shun
    The frumious Bandersnatch!'   "Lewis Carroll" "Jabberwocky"

Re: Using CL in Gnus

[Jesper Harder]

sigurd@12move.de (Karl Pflästerer) writes:

> `dolist' could be used here.  That's in cl-macs.el so could be used
> with eval-when-compile and for a reader is easier to see what
> happens (IMO).

In Emacs 21.3 `dolist' has been promoted to subr.el, so it's not even
necessary to eval-when-compile.

> If we start No Gnus we should look for a lot of places in the code
> where mapping over a list is written like that:
>
>         (let ((lst inputlist))
>           (while lst
>             (do-something-with-list lst)
>             .
>             (setq lst (cdr lst))))
>
> To cdr-down a list is not uncommon (above code would be nicer as
> recursive version) but `dolist' is IMO a much better alternative at
> such places.

Yes.  But in a lot of places the list isn't copied first.  In that
case -- and if speed matters -- `dolist' would degrade performance and
consing slightly.

Re: gnus-agentize versus emacs 20 mapc

[Kevin Ryde]

[-- Attachment #1: Type: text/plain, Size: 416 bytes --]

Katsumi Yamaoka <yamaoka@jpl.org> writes:
>
> That is only for smap-stat and Emacs 20 users.

What I meant is that since you've added mapc as a macro there's no
need for spam-stat.el to test mapc when it thinks about loading 'cl.

        * spam-stat.el: Remove mapc from functions tested to load from 'cl,
        now that mapc is in dgnushack.el.

But I don't know how to use spam-stat, so I haven't tested this.


[-- Attachment #2: spam-stat.el.mapc.diff --]
[-- Type: text/plain, Size: 405 bytes --]

--- spam-stat.el.~6.14.~	2003-11-26 07:41:18.000000000 +1000
+++ spam-stat.el	2003-11-29 07:26:13.000000000 +1000
@@ -197,7 +197,7 @@ (defvar spam-stat-buffer-name " *spam st
 
 (when (memq nil (mapcar 'fboundp
 			'(gethash hash-table-count make-hash-table
-				  mapc puthash)))
+				  puthash)))
   (require 'cl)
   (unless (fboundp 'puthash)
     ;; alias puthash is missing from Emacs 20 cl-extra.el

Re: Using CL in Gnus

[Miles Bader]

Jesper Harder <harder@ifa.au.dk> writes:
> Yes.  But in a lot of places the list isn't copied first.  In that
> case -- and if speed matters -- `dolist' would degrade performance and
> consing slightly.

Huh?  dolist doesn't copy the list...

-Miles
-- 
`Suppose Korea goes to the World Cup final against Japan and wins,' Moon said.
`All the past could be forgiven.'   [NYT]

Re: Using CL in Gnus

[Jesper Harder]

Miles Bader <miles@gnu.org> writes:

> Jesper Harder <harder@ifa.au.dk> writes:
>
>> But in a lot of places the list isn't copied first.  In that case
>> -- and if speed matters -- `dolist' would degrade performance and
>> consing slightly.
>
> Huh?  dolist doesn't copy the list...

dolist establishes a let-binding for the list, which is often
unnecessary.  E.g.

(defun foo (list)
  (let (x)
    (while (setq x (pop list))
      ...)))

vs.

(defun bar (list)
  (let ((temp list) x)
    (while temp
      (setq x (car temp))
      (setq temp (cdr temp))
      ...))

Re: Using CL in Gnus

[Miles Bader]

Jesper Harder <harder@ifa.au.dk> writes:
> > Huh?  dolist doesn't copy the list...
> 
> dolist establishes a let-binding for the list, which is often
> unnecessary.  E.g.

Ah; so strong emphasis on `slightly'...

-Miles
-- 
Freedom's just another word, for nothing left to lose   --Janis Joplin

Re: Using CL in Gnus (was: gnus-agentize versus emacs 20 mapc)

[Per Abrahamsen]

The only Emacs where we need to worry about not including the cl
runtime is "the next version of GNU Emacs".

RMS do not want the cl namespace to clutter the pristine GNU Emacs,
which mean that Gnus that the bundled version of Gnus should not load
the cl runtime.  Since this list concern the development version of
Gnus, which will be available unbundled for older Emacsen and bundled
only in a future Emacs release, the "future Emacs release" is the only
version where we need to avoid cl.

In other words, using functions like mapc that requires the cl runtime
in older Emacsen (or XEmacs) are fine, as long as they don't require
cl in the CVS version of Emacs.

Re: Using CL in Gnus

[Ted Zlatanov]

mapc is used by gnus-registry.el thus:

(defun alist-to-hashtable (alist)
  "Build a hashtable from the values in ALIST."
  (let ((ht (make-hash-table                        
             :size 4096
             :test 'equal)))
    (mapc
     (lambda (kv-pair)
       (puthash (car kv-pair) (cdr kv-pair) ht))
     alist)
     ht))

This is trivial to switch to mapcar as far as I can see.  I only
wanted to emphasize that there is no return value.

Ted

Re: Using CL in Gnus

[Katsumi Yamaoka]

mapc is faster than mapcar, but dolist is a bit faster than mapc,
though they may change with cases.  What should be mentioned
especially is that dolist can be used without a problem also in
Emacs 20. :)

(defvar numbers-list (let ((n 0) rest)
		       (while (< n 65536)
			 (setq rest (cons n rest)
			       n (1+ n)))
		       (nreverse rest)))
(defvar numbers-sum 0)

(progn
  (setq numbers-sum 0)
  (benchmark 10
    (dolist (n numbers-list numbers-sum)
      (setq numbers-sum (+ numbers-sum n)))))
 => 0.1352449655532837


(progn
  (setq numbers-sum 0)
  (benchmark 10
    (mapc (lambda (n)
	    (setq numbers-sum (+ numbers-sum n)))
	  numbers-list)))
 => 0.1569150686264038


(progn
  (setq numbers-sum 0)
  (benchmark 10
    (mapcar (lambda (n)
	      (setq numbers-sum (+ numbers-sum n)))
	    numbers-list)))
 => 0.678508996963501

;; http://quimby.gnus.org/elisp/benchmark.el
-- 
Katsumi Yamaoka <yamaoka@jpl.org>