Humor value: user error

Humor value: user error

[Karl Kleinpaste]

a short route to an xemacs core dump under gnus:

# share scorefiles between news servers.
# e.g., usual server and test host on local machine.
cd ~/News
ln -s . nntp+localhost
xemacs
M-x gnus
SPC
"Finding all score files..."

observe that it's, um, taking quite a while, hm.
do `ps aux | sort -nr +4 | head' to see how big things are getting.

make shocked look on face at observing that xemacs is getting *REALLY*
*REALLY* *BIG*.  i mean, let's be serious, 94Mb, and it wasn't done yet...

watch xemacs complain, toss chunks, and drop core file.
make far-too-belated realization that xemacs is chasing infinitely
across nntp+localhost -> . in search of scorefiles, and (gee, wow,
howzabout that) managing to find more and more and More and MORE and
*MORE* of them.

bozo.  sheesh.  *sigh*

(if it matters to anyone, XEmacs 19.14 under RH4.0 Linux.)

Re: Humor value: user error

[Steven L Baur]

>>>>> "Karl" == Karl Kleinpaste <karl@jprc.com> writes:

Karl> a short route to an xemacs core dump under gnus:
Karl> # share scorefiles between news servers.
Karl> # e.g., usual server and test host on local machine.
Karl> cd ~/News
Karl> ln -s . nntp+localhost
Karl> xemacs
Karl> M-x gnus
Karl> SPC
Karl> "Finding all score files..."

Karl> observe that it's, um, taking quite a while, hm.
Karl> do `ps aux | sort -nr +4 | head' to see how big things are getting.

Karl> make shocked look on face at observing that xemacs is getting
Karl> *REALLY* *REALLY* *BIG*.  i mean, let's be serious, 94Mb, and it
Karl> wasn't done yet...

Karl> watch xemacs complain, toss chunks, and drop core file.
Karl> make far-too-belated realization that xemacs is chasing infinitely
Karl> across nntp+localhost -> . in search of scorefiles, and (gee, wow,
Karl> howzabout that) managing to find more and more and More and MORE and
Karl> *MORE* of them.

Karl> bozo.  sheesh.  *sigh*

Karl> (if it matters to anyone, XEmacs 19.14 under RH4.0 Linux.)

1.  A current workaround is to use long filenames for scorefiles.

2.  This is a Gnus bug, and not an XEmacs bug (or an Emacs bug) since
    the problem occurs in the function `gnus-score-score-files-1' and
    it chokes and dies just as readily in Emacs 19.34 when there's a
    circular directory structure.

A fix would be to keep a cache of expanded directory names, and
not search the same directory twice after expansion of symbolic links.

-- 
steve@miranova.com baur
Unsolicited commercial e-mail will be billed at $250/message.
"Bill Clinton is a bore.  He doesn't have a creative bone in his
body."  -- David Brinkley

Re: Humor value: user error

[Rich Pieri]

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "SLB" == Steven L Baur <steve@miranova.com> writes:

SLB> 2.  This is a Gnus bug, and not an XEmacs bug (or an Emacs bug)

It is not really a Gnus bug per se, since Gnus is doing exactly what it
is supposed to do: recurse through the directory.  The "bug" is in the
level that symbolic links are being dealt with.  Any function which
"chases" symlinks will fall into this kind of recursion.

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2
Charset: noconv

iQCVAwUBMpYJwJ6VRH7BJMxHAQG7GAQAopxr1Dxxj8WQFkn+XqENrSqpV7Wa4tgC
Ix5lZrvZg83xKqsJNS0UUcQmCT0t8zNFxtP61oKwuspE7Di5hsfU/m5g3LnFATzS
xtrp19pZ1BGB3YNILGinQCEcRHF0ut0iuMcVv82/VZ2xIFkLGwCL6n0K+cEaADmm
kIxCiJiuSQ0=
=/etJ
-----END PGP SIGNATURE-----
-- 
Rich Pieri <rich.pieri@prescienttech.com> | Climb your way to the top; that's
      Prescient Technologies, Inc.        | why the drapes are there.
       A Stone & Webster Company          | --A cat's guide to life
  I speak for myself, not PTI or SWEC     | 

Re: Humor value: user error

[Michael R Cook]

>>>>> "RP" == Rich Pieri <rich.pieri@PrescientTech.com> writes:

 RP> It is not really a Gnus bug per se, since Gnus is doing exactly what it
 RP> is supposed to do: recurse through the directory.  The "bug" is in the
 RP> level that symbolic links are being dealt with.

 RP> Any function which "chases" symlinks will fall into this kind of
 RP> recursion.

Unless it is smart enough not to.  Because gnus is _not_ smart enough, one
coule argue that the bug is in gnus.

Re: Humor value: user error

[Karl Kleinpaste]

Michael R Cook <mcook@cognex.com> writes:
> Unless it is smart enough not to.  Because gnus is _not_ smart enough, one
> coule argue that the bug is in gnus.

Well, first, it was really a pretty stupid thing to do.  I shouldn't
have done it, and if I'd given it slightly more thought, I wouldn't
have.  But it was impressive to see the VM reports from ps(1).

Anyhow, a workaround that doesn't involve hacking Gnus is
	mkdir ~/News/nntp+localhost
	cd ~/News/nntp+localhost
	ln -s ../* .
	rm nn*		# to get rid of nntp self-references &c.
which seems to give me exactly what I want, unless I manage to start
reading a hierarchy not supported in common among the others, in which
case I'll have to generalize the special case such a situation creates.
(By moving the new hierarchy up to ~/News, and adding another symlink.)

Re: Humor value: user error

[Rich Pieri]

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "MRC" == Michael R Cook <mcook@cognex.com> writes:

MRC> Because gnus is _not_ smart enough, one coule argue that the bug is
MRC> in gnus.

A program or operating system that gives you the power to shoot yourself
in the foot is not buggy in this regard.  If as root I do an 'rm -rf /'
do I blame the operating system or the rm command for my own stupidity?
Of course not.

Same thing, really.

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2
Charset: noconv

iQCVAwUBMpmorJ6VRH7BJMxHAQGHJAQAvwMfaoLrheFLctHRKcZ4BM2KNV5dGefq
xxwsLVWbSnsw9r5g/w7zbc1Ofq1OyT/RskJDIJ44XEupEVhWKFWyUkpyuyfMXgfC
uKUw8XxoaKRBe3/3ofdfAjuujYE+jI7oUj61aK6UQMgoTFhNQHwRJ0WknSG69ey/
BZM2yVkKYEk=
=4ll+
-----END PGP SIGNATURE-----
-- 
Rich Pieri <rich.pieri@prescienttech.com> | If you're not receiving enough
      Prescient Technologies, Inc.        | attention, try knocking over some
       A Stone & Webster Company          | very expensive antique lamps.
  I speak for myself, not PTI or SWEC     | --A cat's guide to life

Re: Humor value: user error

[David Moore]

Steven L Baur <steve@miranova.com> writes:
> 1.  A current workaround is to use long filenames for scorefiles.
> 
> 2.  This is a Gnus bug, and not an XEmacs bug (or an Emacs bug) since
>     the problem occurs in the function `gnus-score-score-files-1' and
>     it chokes and dies just as readily in Emacs 19.34 when there's a
>     circular directory structure.
> 
> A fix would be to keep a cache of expanded directory names, and
> not search the same directory twice after expansion of symbolic links.

	Patch below for a couple of these situations in rgnus-0.68.
Then again you might just want to avoid making cycles in your Gnus
directory trees with symlinks. ;-)


*** ChangeLog.orig	Thu Nov 21 18:10:48 1996
--- ChangeLog	Fri Nov 22 12:30:51 1996
***************
*** 1,3 ****
--- 1,25 ----
+ Fri Nov 22 12:17:14 1996  David Moore  <dmoore@ucsd.edu>
+ 
+ 	* nnml.el (nnml-generate-nov-databases-1): Don't infloop.
+ 
+ 	* gnus-score.el (gnus-score-score-files-1): Don't infloop, be
+ 	slightly faster.
+ 
  Thu Nov 21 05:33:24 1996  Lars Magne Ingebrigtsen  <larsi@ifi.uio.no>
  
  	* nnoo.el (nnoo-change-server): Protect against void vars.
*** gnus-score.el.orig	Fri Nov 22 11:57:49 1996
--- gnus-score.el	Fri Nov 22 12:16:44 1996
***************
*** 2361,2378 ****
  
  (defun gnus-score-score-files-1 (dir)
    "Return all possible score files under DIR."
!   (let ((files (directory-files (expand-file-name dir) t nil t))
  	(regexp (gnus-score-file-regexp))
  	(case-fold-search nil)
! 	out file)
      (while (setq file (pop files))
        (cond 
         ;; Ignore "." and "..".
         ((member (file-name-nondirectory file) '("." ".."))
  	nil)
!        ;; Recurse down directories.
!        ((file-directory-p file)
! 	(setq out (nconc (gnus-score-score-files-1 file) out)))
         ;; Add files to the list of score files.
         ((string-match regexp file)
  	(push file out))))
--- 2361,2380 ----
  
  (defun gnus-score-score-files-1 (dir)
    "Return all possible score files under DIR."
!   (let ((files (list (expand-file-name dir)))
  	(regexp (gnus-score-file-regexp))
  	(case-fold-search nil)
! 	seen out file)
      (while (setq file (pop files))
        (cond 
         ;; Ignore "." and "..".
         ((member (file-name-nondirectory file) '("." ".."))
  	nil)
!        ;; Add subtrees of directory to also be searched.
!        ((and (file-directory-p file)
! 	     (not (member file seen)))
! 	(push file seen)
! 	(setq files (nconc (directory-files file t nil t) files)))
         ;; Add files to the list of score files.
         ((string-match regexp file)
  	(push file out))))
*** nnml.el.orig	Fri Nov 22 12:24:55 1996
--- nnml.el	Fri Nov 22 12:29:12 1996
***************
*** 706,731 ****
    ;; Save the active file.
    (nnmail-save-active nnml-group-alist nnml-active-file))
  
! (defun nnml-generate-nov-databases-1 (dir)
    (setq dir (file-name-as-directory dir))
!   ;; We descend recursively 
!   (let ((dirs (directory-files dir t nil t))
! 	dir)
!     (while dirs 
!       (setq dir (pop dirs))
!       (when (and (not (member (file-name-nondirectory dir) '("." "..")))
! 		 (file-directory-p dir))
! 	(nnml-generate-nov-databases-1 dir))))
!   ;; Do this directory.
!   (let ((files (sort
! 		(mapcar
! 		 (lambda (name) (string-to-int name))
! 		 (directory-files dir nil "^[0-9]+$" t))
! 		'<)))
!     (when files
!       (funcall nnml-generate-active-function dir)
!       ;; Generate the nov file.
!       (nnml-generate-nov-file dir files))))
  
  (defvar files)
  (defun nnml-generate-active-info (dir)
--- 706,733 ----
    ;; Save the active file.
    (nnmail-save-active nnml-group-alist nnml-active-file))
  
! (defun nnml-generate-nov-databases-1 (dir &optional seen)
    (setq dir (file-name-as-directory dir))
!   ;; Only scan this sub-tree if we haven't been here yet.
!   (unless (member dir seen)
!     (push dir seen)
!     ;; We descend recursively
!     (let ((dirs (directory-files dir t nil t))
! 	  dir)
!       (while (setq dir (pop dirs))
! 	(when (and (not (member (file-name-nondirectory dir) '("." "..")))
! 		   (file-directory-p dir))
! 	  (nnml-generate-nov-databases-1 dir seen))))
!     ;; Do this directory.
!     (let ((files (sort
! 		  (mapcar
! 		   (lambda (name) (string-to-int name))
! 		   (directory-files dir nil "^[0-9]+$" t))
! 		  '<)))
!       (when files
! 	(funcall nnml-generate-active-function dir)
! 	;; Generate the nov file.
! 	(nnml-generate-nov-file dir files)))))
  
  (defvar files)
  (defun nnml-generate-active-info (dir)

Re: Humor value: user error

[Steven L Baur]

>>>>> "David" == David Moore <dmoore@UCSD.EDU> writes:

> Steven L Baur <steve@miranova.com> writes:

sb> A fix would be to keep a cache of expanded directory names, and
sb> not search the same directory twice after expansion of symbolic links.

David> 	Patch below for a couple of these situations in rgnus-0.68.
David> Then again you might just want to avoid making cycles in your Gnus
David> directory trees with symlinks. ;-)

Thanks for the patches David, but there are a couple of problems.  The
ChangeLog patch is corrupted (replace the 1,25 with 1,10).  And the
patches to nnml.el and gnus-score.el do not quite correct the problem.

Apply the following patch on top of David's patch to gnus-score.el to
get one fix.

[Tested with GNU Emacs 19.34 & XEmacs 19.15]

--- gnus-score.el~	Fri Nov 22 13:54:00 1996
+++ gnus-score.el	Fri Nov 22 14:21:57 1996
@@ -2372,8 +2372,8 @@
 	nil)
        ;; Add subtrees of directory to also be searched.
        ((and (file-directory-p file)
-	     (not (member file seen)))
-	(push file seen)
+	     (not (member (file-truename file) seen)))
+	(push (file-truename file) seen)
 	(setq files (nconc (directory-files file t nil t) files)))
        ;; Add files to the list of score files.
        ((string-match regexp file)

The same correction needs to be made to the nnml.el patch.  You need
to compare (file-truename)s to really defend against symbolic links.

Even with this fix, all files will get added twice due to the trailing
slash passed in, using Karl's example.  (Which is fine enough with me).

-- 
steve@miranova.com baur
Unsolicited commercial e-mail will be billed at $250/message.
"Bill Clinton is a bore.  He doesn't have a creative bone in his
body."  -- David Brinkley

Re: Humor value: user error

[Rich Pieri]

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "KK" == Karl Kleinpaste <karl@jprc.com> writes:

KK> ln -s . nntp+localhost

Well, gee, that is pretty stupid.  No wonder it went into an infinite
loop.  Anything that would recurse through the directory tree will loop
on that... because you told it to do that.

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2
Charset: noconv

iQCVAwUBMpYAaJ6VRH7BJMxHAQFN7AP/YCaMu1CU2fI7MjGdZWrQvVvCYjXfCSYh
yqhuxje2aSDZIXO7sUB85dcGqLqTnKBzUtKOawqfyHc2A+a2fi3pdOJljkL8scL3
nvSo8Hoe3YsnuYq79yK47GH66ncymjGf5FgelzpfBuJD1oS+XVEi7NOFK4XmhkYN
Bb5wordHFwE=
=dAcZ
-----END PGP SIGNATURE-----
-- 
Rich Pieri <rich.pieri@prescienttech.com> | When eating out, think nothing of
      Prescient Technologies, Inc.        | sending back your meal twenty or
       A Stone & Webster Company          | thirty times.
  I speak for myself, not PTI or SWEC     | --A cat's guide to life