'g' in the group buffer still very slow

'g' in the group buffer still very slow

[Sebastian D.B. Krause]

[-- Attachment #1: Type: text/plain, Size: 542 bytes --]

This problem was already on this list several month ago (posted by
another person): When I start Oort Gnus or press 'g' in the Groups
Buffer, it needs several seconds to read the active file. Someone
mailed a small patch to this list which solved this problem and put
it into my local CVS where the change version stayed over the
month. As I see now, this patch never made its way into the official
CVS, so this problem still exists.

What does this patch (attached to this email) change and why has it
never been applied to the current CVS?

[-- Attachment #2: nntp.el.diff --]
[-- Type: text/plain, Size: 705 bytes --]

--- nntp.el.old	Sun Jul 21 21:58:51 2002
+++ nntp.el	Fri Jul 12 22:50:12 2002
@@ -404,17 +404,7 @@
 	    (nntp-retrieve-data command
 				nntp-address nntp-port-number
 				nntp-server-buffer
-				wait-for nnheader-callback-function)
-	  ;; If nothing to wait for, still remove possibly echo'ed commands
-	  (unless wait-for
-	    (nntp-accept-response)
-	    (save-excursion
-	      (set-buffer buffer)
-	      (goto-char pos)
-	      (if (looking-at (regexp-quote command))
-		  (delete-region pos (progn (forward-line 1)
-					    (gnus-point-at-bol))))
-	      )))
+				wait-for nnheader-callback-function))
       (nnheader-report 'nntp "Couldn't open connection to %s."
 		       nntp-address))))
 

Re: 'g' in the group buffer still very slow

[Simon Josefsson]

krause@sdbk.de (Sebastian D.B. Krause) writes:

> This problem was already on this list several month ago (posted by
> another person): When I start Oort Gnus or press 'g' in the Groups
> Buffer, it needs several seconds to read the active file. Someone
> mailed a small patch to this list which solved this problem and put
> it into my local CVS where the change version stayed over the
> month. As I see now, this patch never made its way into the official
> CVS, so this problem still exists.
>
> What does this patch (attached to this email) change and why has it
> never been applied to the current CVS?

The code is needed if you use e.g. telnet with local echo to the
server.  Unfortunately I think the code contained a hidden delay or
buffer search somewhere, causing it to be slow.  I'm not sure how to
best solve it, if the code is removed, some people might be affected.

> --- nntp.el.old	Sun Jul 21 21:58:51 2002
> +++ nntp.el	Fri Jul 12 22:50:12 2002
> @@ -404,17 +404,7 @@
>  	    (nntp-retrieve-data command
>  				nntp-address nntp-port-number
>  				nntp-server-buffer
> -				wait-for nnheader-callback-function)
> -	  ;; If nothing to wait for, still remove possibly echo'ed commands
> -	  (unless wait-for
> -	    (nntp-accept-response)
> -	    (save-excursion
> -	      (set-buffer buffer)
> -	      (goto-char pos)
> -	      (if (looking-at (regexp-quote command))
> -		  (delete-region pos (progn (forward-line 1)
> -					    (gnus-point-at-bol))))
> -	      )))
> +				wait-for nnheader-callback-function))
>        (nnheader-report 'nntp "Couldn't open connection to %s."
>  		       nntp-address))))
>  

Re: 'g' in the group buffer still very slow

[Kai Großjohann]

Simon Josefsson <jas@extundo.com> writes:

> The code is needed if you use e.g. telnet with local echo to the
> server.  Unfortunately I think the code contained a hidden delay or
> buffer search somewhere, causing it to be slow.

The hidden delay is probably in nntp-accept-response, which calls
nntp-accept-process-output, which has a 1 sec timeout.

kai
-- 
A large number of young women don't trust men with beards.  (BFBS Radio)

Re: 'g' in the group buffer still very slow

[Sebastian D.B. Krause]

[-- Attachment #1: Type: text/plain, Size: 2423 bytes --]

On 3250 September 1993, Simon Josefsson wrote:
> krause@sdbk.de (Sebastian D.B. Krause) writes:
>> This problem was already on this list several month ago (posted by
>> another person): When I start Oort Gnus or press 'g' in the Groups
>> Buffer, it needs several seconds to read the active file. Someone
>> mailed a small patch to this list which solved this problem and put
>> it into my local CVS where the change version stayed over the
>> month. As I see now, this patch never made its way into the official
>> CVS, so this problem still exists.
>>
>> What does this patch (attached to this email) change and why has it
>> never been applied to the current CVS?
>
> The code is needed if you use e.g. telnet with local echo to the
> server.  Unfortunately I think the code contained a hidden delay or
> buffer search somewhere, causing it to be slow.  I'm not sure how to
> best solve it, if the code is removed, some people might be affected.

Okay, as we don't need this code when Gnus has a direct NNTP
connection to the server (when nntp-open-connection-function is set
to nntp-open-network-stream), I suggest -- as a workaround -- to
enable this code only when nntp-open-connection-function is *not*
nntp-open-network-stream. I've tested my small patch (see below) and
it makes fetching the active file much faster without the problems I
had with telnet and local echo when I removed the code with the old
patch quoted below.

I know that this is no generic solution, but because most users are
using nntp-open-network-stream and some had this problem with the
delay, too, IMHO it is okay to add this patch to the CVS.

>> --- nntp.el.old	Sun Jul 21 21:58:51 2002
>> +++ nntp.el	Fri Jul 12 22:50:12 2002
>> @@ -404,17 +404,7 @@
>>  	    (nntp-retrieve-data command
>>  				nntp-address nntp-port-number
>>  				nntp-server-buffer
>> -				wait-for nnheader-callback-function)
>> -	  ;; If nothing to wait for, still remove possibly echo'ed commands
>> -	  (unless wait-for
>> -	    (nntp-accept-response)
>> -	    (save-excursion
>> -	      (set-buffer buffer)
>> -	      (goto-char pos)
>> -	      (if (looking-at (regexp-quote command))
>> -		  (delete-region pos (progn (forward-line 1)
>> -					    (gnus-point-at-bol))))
>> -	      )))
>> +				wait-for nnheader-callback-function))
>>        (nnheader-report 'nntp "Couldn't open connection to %s."
>>  		       nntp-address))))
>>  

Here is the new patch:


[-- Attachment #2: nntp.el.diff --]
[-- Type: text/plain, Size: 937 bytes --]

--- nntp.el.old	Sun Dec  8 17:45:38 2002
+++ nntp.el	Sun Dec  8 17:42:44 2002
@@ -406,15 +406,16 @@
 				nntp-server-buffer
 				wait-for nnheader-callback-function)
 	  ;; If nothing to wait for, still remove possibly echo'ed commands
-	  (unless wait-for
-	    (nntp-accept-response)
-	    (save-excursion
-	      (set-buffer buffer)
-	      (goto-char pos)
-	      (if (looking-at (regexp-quote command))
-		  (delete-region pos (progn (forward-line 1)
-					    (gnus-point-at-bol))))
-	      )))
+	  (unless nntp-open-connection-function 'nntp-open-network-stream
+		  (unless wait-for
+		    (nntp-accept-response)
+		    (save-excursion
+		      (set-buffer buffer)
+		      (goto-char pos)
+		      (if (looking-at (regexp-quote command))
+			  (delete-region pos (progn (forward-line 1)
+						    (gnus-point-at-bol))))
+		      ))))
       (nnheader-report 'nntp "Couldn't open connection to %s."
 		       nntp-address))))
 

[-- Attachment #3: Type: text/plain, Size: 11 bytes --]


Sebastian

Re: 'g' in the group buffer still very slow

[Kai Großjohann]

krause@my.gnus.org (Sebastian D.B. Krause) writes:

> Okay, as we don't need this code when Gnus has a direct NNTP
> connection to the server (when nntp-open-connection-function is set
> to nntp-open-network-stream), I suggest -- as a workaround -- to
> enable this code only when nntp-open-connection-function is *not*
> nntp-open-network-stream. I've tested my small patch (see below) and
> it makes fetching the active file much faster without the problems I
> had with telnet and local echo when I removed the code with the old
> patch quoted below.

There is a bug.  You write (unless nntp-open-connection-function
'nntp-open-network-stream statements...).  The condition here is the
variable nntp-open-connection-function, it is tested whether the
variable is nil.  The quoted symbol is an expression which is
evaluated by has no effect.

I've committed something similar to your patch.

I think what should be done is to look, after the connection has been
established, whether the commands are echoed.  If so, set a
variable.  Then change the condition to look at that variable.

-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: 'g' in the group buffer still very slow

[Sebastian D.B. Krause]

On 3385 September 1993, Kai Großjohann wrote:
> There is a bug.  You write (unless nntp-open-connection-function
> 'nntp-open-network-stream statements...).  The condition here is the
> variable nntp-open-connection-function, it is tested whether the
> variable is nil.  The quoted symbol is an expression which is
> evaluated by has no effect.

Oh, sorry.

> I've committed something similar to your patch.

This doesn't seem to fix the problem. The and-expression seems to be
the problem in the patch you commited. If I write

(unless (equal nntp-open-connection-function
         'nntp-open-network-stream)
  (unless wait-for
  [...]

instead, the bug you described above seems to be fixed (I hope) and
the delay is gone, too. Uhh ... Lisp is a bit too compilicated for
me, sorry if the code above has bugs again.

Sebastian

Re: 'g' in the group buffer still very slow

[Martin Rohde]

On Day 50 of The Aftermath 3168, Kai Großjohann wrote:

> krause@my.gnus.org (Sebastian D.B. Krause) writes:
>
>> Okay, as we don't need this code when Gnus has a direct NNTP
>> connection to the server (when nntp-open-connection-function is set
>> to nntp-open-network-stream), I suggest -- as a workaround -- to
>> enable this code only when nntp-open-connection-function is *not*
>> nntp-open-network-stream. I've tested my small patch (see below) and
>> it makes fetching the active file much faster without the problems I
>> had with telnet and local echo when I removed the code with the old
>> patch quoted below.
>
> There is a bug.  You write (unless nntp-open-connection-function
> 'nntp-open-network-stream statements...).  The condition here is the
> variable nntp-open-connection-function, it is tested whether the
> variable is nil.  The quoted symbol is an expression which is
> evaluated by has no effect.
>
> I've committed something similar to your patch.

Like Sebastian mentioned before,

,----
| 
|  (unless (and wait-for
| 	     (equal nntp-open-connection-function
| 		    'nntp-open-network-stream))
`----

doesn't work. It has to be "or" instead of "and"...

> I think what should be done is to look, after the connection has been
> established, whether the commands are echoed.  If so, set a
> variable.  Then change the condition to look at that variable.

If I get it right, you can almost take the code for that from
nntp-send-command* and put it into nntp-open-connection. I will send
a patch, if it works.


Martin

Re: 'g' in the group buffer still very slow

[Kai Großjohann]

Martin Rohde <martin.rohde@gmx.de> writes:

> If I get it right, you can almost take the code for that from
> nntp-send-command* and put it into nntp-open-connection. I will send
> a patch, if it works.

That would be great!  (I didn't see code for finding out if there is
an echoed command.  Maybe I was just blind.)

-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: 'g' in the group buffer still very slow

[Martin Rohde]

[-- Attachment #1: Type: text/plain, Size: 959 bytes --]

On Day 51 of The Aftermath 3168, Kai Großjohann wrote:

> Martin Rohde <martin.rohde@gmx.de> writes:
>
>> If I get it right, you can almost take the code for that from
>> nntp-send-command* and put it into nntp-open-connection. I will send
>> a patch, if it works.
>
> That would be great!  (I didn't see code for finding out if there is
> an echoed command.  Maybe I was just blind.)

Well, nntp-send-command looks, if there's an echo, and then deletes
it. I don't understand most of the code in nntp-open-connection, so
I don't know if anything of this patch makes sense. And I don't know
how to test it, because i don't know how to force such a local echo.

What this code does, is to send a "MODE READER" as a No-Op command,
when the connection is established (I think ;), and then it looks,
just like in nntp-send-command, if the command is echoed in the
server-buffer. If so, it deletes it and sets nntp-delete-echo to t
for future reference (I hope).


[-- Attachment #2: nntp.el.diff --]
[-- Type: text/plain, Size: 2112 bytes --]

--- nntp.el.old	2002-12-09 13:22:57.000000000 +0100
+++ nntp.el	2002-12-09 14:35:30.000000000 +0100
@@ -251,6 +251,9 @@
   (autoload 'mail-source-read-passwd "mail-source")
   (autoload 'open-ssl-stream "ssl"))
 
+(defvar nntp-delete-echo nil)
+
+
 \f
 
 ;;; Internal functions.
@@ -408,17 +411,17 @@
 	  ;; If nothing to wait for, still remove possibly echo'ed commands.
 	  ;; We don't have echos if nntp-open-connection-function
 	  ;; is `nntp-open-network-stream', so we skip this in that case.
-	  (unless (or wait-for
-		      (equal nntp-open-connection-function
-			     'nntp-open-network-stream))
-	    (nntp-accept-response)
-	    (save-excursion
-	      (set-buffer buffer)
-	      (goto-char pos)
-	      (if (looking-at (regexp-quote command))
-		  (delete-region pos (progn (forward-line 1)
-					    (gnus-point-at-bol))))
-	      )))
+	  
+	    (when (and (not wait-for)
+		       nntp-delete-echo)
+	      (nntp-accept-response)
+	      (save-excursion
+		(set-buffer buffer)
+		(goto-char pos)
+		(if (looking-at (regexp-quote command))
+		    (delete-region pos (progn (forward-line 1)
+					      (gnus-point-at-bol))))
+		)))
       (nnheader-report 'nntp "Couldn't open connection to %s."
 		       nntp-address))))
 
@@ -1031,7 +1034,25 @@
 	      (set-buffer pbuffer)
 	      (nntp-read-server-type)
 	      (erase-buffer)
+
 	      (set-buffer nntp-server-buffer)
+
+	      ;; Send a No-Op command to recognize if we need to delete local echo
+	      ;; in future. This only makes sense, if we are using a non-direct connection...
+
+	      (unless (equal nntp-open-connection-function
+			     'nntp-open-network-stream)
+
+		(let ((command "MODE READER")
+		      (pos (point)))
+		  (nntp-send-string process command)
+		  (nntp-wait-for process "\n" (current-buffer))
+		  (goto-char pos)
+		  (when (looking-at (regexp-quote command))
+		    (setq nntp-delete-echo t)
+		    (delete-region pos (progn (forward-line 1)
+					      (gnus-point-at-bol))))))
+	      
 	      (let ((nnheader-callback-function nil))
 		(run-hooks 'nntp-server-opened-hook)
 		(nntp-send-authinfo t))))

Re: 'g' in the group buffer still very slow

[Kai Großjohann]

Martin Rohde <martin.rohde@gmx.de> writes:

> Well, nntp-send-command looks, if there's an echo, and then deletes
> it. I don't understand most of the code in nntp-open-connection, so
> I don't know if anything of this patch makes sense. And I don't know
> how to test it, because i don't know how to force such a local echo.

If nobody steps forward to test it (I don't have the time, sadly),
then you'll need to find a way to do it yourself.  I'm not sure, but
you might be successful with using telnet to open a connection.

You could create a new server definition that uses telnet instead of
connecting directly.  See the variable nntp-open-connection-function
and the function nntp-open-telnet-stream, referenced from the
variable.  Maybe you can twiddle the telnet args such that echoing is
used?  You might wish to manually send stuff to the server in the
server buffer (" *server foo *nntpd*" or somesuch, mind the leading
space) using process-send-string.  Then you can see whether the
server echoes commands.  (I hope that the echo removal isn't done in
a process filter.)

> What this code does, is to send a "MODE READER" as a No-Op command,
> when the connection is established (I think ;), and then it looks,
> just like in nntp-send-command, if the command is echoed in the
> server-buffer. If so, it deletes it and sets nntp-delete-echo to t
> for future reference (I hope).

I'm afraid that spot is too early.  Some servers might require
authentication to log in.  Thinking...  Hm...  Oh!  It doesn't matter:
if the server requires authentication, then it just sends an error
message which is just as good for your purposes as a success message.

But "mode reader" is not a no-op, could that be a problem?
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: 'g' in the group buffer still very slow

[Martin Rohde]

On Day 51 of The Aftermath 3168, Kai Großjohann wrote:

> You could create a new server definition that uses telnet instead of
> connecting directly.  See the variable nntp-open-connection-function
> and the function nntp-open-telnet-stream, referenced from the
> variable.  Maybe you can twiddle the telnet args such that echoing is
> used?  You might wish to manually send stuff to the server in the
> server buffer (" *server foo *nntpd*" or somesuch, mind the leading
> space) using process-send-string.  Then you can see whether the
> server echoes commands.  (I hope that the echo removal isn't done in
> a process filter.)

Ok, now i used nntp-open-telnet-stream and looked at the server
buffer (thanks for the hints). There were several lines like "LIST
ACTIVE <newsgroup>", so commands are echoed and the lines are not
deleted. With nntp-open-network-stream or with an older version of
nntp.el the buffer is "clean".

Obviously my patch didn't worked, but now I figured out, that it
acted in the wrong buffer. And I found another problem: 

After the test-command is send, I somehow have to find out, when the
response has arrived, so that I can look at the echo then. I thought
a (nntp-wait-for "^2.*\n" process buffer [...]) would work, but then
the echo is not found for unknown reasons (I have to confess, that I
don't really know, what I am doing here, maybe nntp-wait-for is
erasing the buffer for some reason...). 

I just tried it with a (sleep-for 2) right after the
nntp-send-string-command to wait until the echo is in the server
buffer, just to see if the rest of my code is ok, and it "works"
now. (That means, nntp-delete-echo is set properly) 

Of course, this is too ugly, so I need a reasonable possibility to
wait for the response. I thought of a kind of loop, but this
wouldn't be that much prettier IMHO. Is there a function i can use 
for this?

> But "mode reader" is not a no-op, could that be a problem?

I'm not quite sure. I wanted to send a command, that doesn't have
affects on gnus and I read, that "mode reader" often is used as a
no-op. Maybe "slave" would be a better idea? (I don't have a clue
about nntp :)

Thanks for your advise,
Martin.

Re: 'g' in the group buffer still very slow

[Martin Rohde]

[-- Attachment #1: Type: text/plain, Size: 593 bytes --]

Ok, now it works, I think:

Using telnet with local echo, leads to setting nntp-delete-echo to
t, and the lines in the server-buffer are deleted.
When I disable local echo, nntp-delete-echo is nil and the lines are
not touched. 

Because "mode reader" and "slave" still _could_ affect something
(whatever that would be), I now think, that the best way is to send 
something stupid like "MyNOOP", for NNTP doesn't close the
connection on wrong commands. 

Furthermore I changed the functions nntp-send-command-nodelete and
nntp-send-command-and-decode, that had the code to delete echos,
too.


[-- Attachment #2: nntp.el.diff --]
[-- Type: text/plain, Size: 2582 bytes --]

--- nntp.el.old	2002-12-09 13:22:57.000000000 +0100
+++ nntp.el	2002-12-10 18:48:10.000000000 +0100
@@ -251,6 +251,9 @@
   (autoload 'mail-source-read-passwd "mail-source")
   (autoload 'open-ssl-stream "ssl"))
 
+(defvar nntp-delete-echo nil)
+
+
 \f
 
 ;;; Internal functions.
@@ -408,9 +411,9 @@
 	  ;; If nothing to wait for, still remove possibly echo'ed commands.
 	  ;; We don't have echos if nntp-open-connection-function
 	  ;; is `nntp-open-network-stream', so we skip this in that case.
-	  (unless (or wait-for
-		      (equal nntp-open-connection-function
-			     'nntp-open-network-stream))
+	  
+	  (when (and (not wait-for)
+		     nntp-delete-echo)
 	    (nntp-accept-response)
 	    (save-excursion
 	      (set-buffer buffer)
@@ -435,7 +438,8 @@
 				nntp-server-buffer
 				wait-for nnheader-callback-function)
 	  ;; If nothing to wait for, still remove possibly echo'ed commands
-	  (unless wait-for
+	  (when (and (not wait-for)
+		     nntp-delete-echo)
 	    (nntp-accept-response)
 	    (save-excursion
 	      (set-buffer buffer)
@@ -465,14 +469,15 @@
 				nntp-server-buffer
 				wait-for nnheader-callback-function t)
 	  ;; If nothing to wait for, still remove possibly echo'ed commands
-	  (unless wait-for
+	  (unless (and (not wait-for)
+		       nntp-delete-echo)
 	    (nntp-accept-response)
 	    (save-excursion
-	  (set-buffer buffer)
-	  (goto-char pos)
-	  (if (looking-at (regexp-quote command))
-	      (delete-region pos (progn (forward-line 1) (gnus-point-at-bol))))
-	  )))
+	      (set-buffer buffer)
+	      (goto-char pos)
+	      (if (looking-at (regexp-quote command))
+		  (delete-region pos (progn (forward-line 1) (gnus-point-at-bol))))
+	      )))
       (nnheader-report 'nntp "Couldn't open connection to %s."
 		       nntp-address))))
 
@@ -1031,7 +1036,22 @@
 	      (set-buffer pbuffer)
 	      (nntp-read-server-type)
 	      (erase-buffer)
+
+	      ;; Send a No-Op command to recognize if we need to delete local echo
+	      ;; in future. This only makes sense, if we are using a non-direct connection...
+	      (unless (equal nntp-open-connection-function
+			     'nntp-open-network-stream)
+		(let ((command "MyNOOP")
+		      (pos (point)))
+		  (nntp-send-string process command)
+		  (accept-process-output process)
+		  (goto-char pos)
+		  (when (looking-at (regexp-quote command))
+		    (setq nntp-delete-echo t))))
+
+	      (erase-buffer)
 	      (set-buffer nntp-server-buffer)
+	      
 	      (let ((nnheader-callback-function nil))
 		(run-hooks 'nntp-server-opened-hook)
 		(nntp-send-authinfo t))))

Re: 'g' in the group buffer still very slow

[kgreiner]

Martin Rohde <martin.rohde@gmx.de> writes:

> Ok, now it works, I think:
>
> Using telnet with local echo, leads to setting nntp-delete-echo to
> t, and the lines in the server-buffer are deleted.
> When I disable local echo, nntp-delete-echo is nil and the lines are
> not touched. 
>
> Because "mode reader" and "slave" still _could_ affect something
> (whatever that would be), I now think, that the best way is to send 
> something stupid like "MyNOOP", for NNTP doesn't close the
> connection on wrong commands. 
>
> Furthermore I changed the functions nntp-send-command-nodelete and
> nntp-send-command-and-decode, that had the code to delete echos,
> too.

I believe that nntp-delete-echo should be made buffer local as setting
this globally may break configurations that use secondary nntp
servers.

Kevin

Re: 'g' in the group buffer still very slow

[Martin Rohde]

On Day 53 of The Aftermath 3168, kgreiner@xpediantsolutions.com wrote:
> I believe that nntp-delete-echo should be made buffer local as setting
> this globally may break configurations that use secondary nntp
> servers.

I think, that the echo depends on your local configuration, because
it is /telnet/, that is sending these echos and not the server.
Ok, then it is not necessary to test it every time you connect to a
server, but when should this be done instead?

But implementing this would be no problem, so if I am wrong and
there are situations, where the servers could echo something, too,
tell me.

(It isn't possible to set nntp-open-connection-function for each
server differently, is it?)

> Kevin

Martin

Re: 'g' in the group buffer still very slow

[kgreiner]

Martin Rohde <martin.rohde@gmx.de> writes:

> On Day 53 of The Aftermath 3168, kgreiner@xpediantsolutions.com wrote:
>> I believe that nntp-delete-echo should be made buffer local as setting
>> this globally may break configurations that use secondary nntp
>> servers.
>
> I think, that the echo depends on your local configuration, because
> it is /telnet/, that is sending these echos and not the server.
> Ok, then it is not necessary to test it every time you connect to a
> server, but when should this be done instead?

You're correct.  I was thinking that the echo came from the server
rather than the transport.

But since you asked, why not test the value in nntp-send-command?
What if nntp-send-command looked like this?

(defvar nntp-delete-echo t)

(defsubst nntp-send-command (wait-for &rest strings)
  "Send STRINGS to server and wait until WAIT-FOR returns."
  (when (and (not nnheader-callback-function)
             (not nntp-inhibit-output))
    (save-excursion
      (set-buffer nntp-server-buffer)
      (erase-buffer)))
  (let* ((command (mapconcat 'identity strings " "))
         (process (nntp-find-connection nntp-server-buffer))
         (buffer (and process (process-buffer process)))
         (pos (and buffer (with-current-buffer buffer (point)))))
    (if process
        (prog1
            (nntp-retrieve-data command
                                nntp-address nntp-port-number
                                nntp-server-buffer
                                wait-for nnheader-callback-function)
          ;; If nothing to wait for, still remove possibly echo'ed commands.
          ;; We don't have echos if nntp-open-connection-function
          ;; is `nntp-open-network-stream', so we skip this in that case.
          (when (and (not wait-for)
                     nntp-delete-echo)
            (nntp-accept-response)
            (save-excursion
              (set-buffer buffer)
              (goto-char pos)
              (if (setq nntp-delete-echo (looking-at (regexp-quote command)))
                  (delete-region pos (progn (forward-line 1)
                                            (gnus-point-at-bol))))
              )))
      (nnheader-report 'nntp "Couldn't open connection to %s."
                       nntp-address))))

Of course, you'd have to do the same in nntp-send-command-nodelete and
nntp-send-command-and-decode.  So the first time any of these
functions are called, the delete echo code is called.  That code calls
looking-at to confirm that the echo actually occurred.  The difference
is that we now assume that, if the current command did not echo, then
no future command will echo (i.e. the value returned by looking-at is
stored in nntp-delete-echo so, once looking-at returns nil, you'll
never call looking-at again to change that value.)

BTW, this is entirely untested so you'll have to decide if you are
interested in trying it.

>
> But implementing this would be no problem, so if I am wrong and
> there are situations, where the servers could echo something, too,
> tell me.

None that I'm aware of.


>
> (It isn't possible to set nntp-open-connection-function for each
> server differently, is it?)

That's correct, nntp-open-connection-function is set globally.

>
>> Kevin
>
> Martin

Kevin

Re: 'g' in the group buffer still very slow

[Kai Großjohann]

Martin Rohde <martin.rohde@gmx.de> writes:

> On Day 53 of The Aftermath 3168, kgreiner@xpediantsolutions.com wrote:
>> I believe that nntp-delete-echo should be made buffer local as setting
>> this globally may break configurations that use secondary nntp
>> servers.
>
> I think, that the echo depends on your local configuration, because
> it is /telnet/, that is sending these echos and not the server.
> Ok, then it is not necessary to test it every time you connect to a
> server, but when should this be done instead?

It is necessary to test it every time you connect to a server, because...

> But implementing this would be no problem, so if I am wrong and
> there are situations, where the servers could echo something, too,
> tell me.
>
> (It isn't possible to set nntp-open-connection-function for each
> server differently, is it?)

it *is* possible to set nntp-open-connection-function differently for
each server.  That's the whole point, actually.

You can put the definition in the server parameters, like this:

(add-to-list 'gnus-secondary-select-methods
             '(nntp "servername"
               (nntp-address "host.name.of.server")
               (nntp-open-connection-function nntp-open-telnet-stream)
               ...other.nntp-open-telnet.settings...))

-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: 'g' in the group buffer still very slow

[Martin Rohde]

[-- Attachment #1: Type: text/plain, Size: 1960 bytes --]

On Day 54 of The Aftermath 3168, Kai Großjohann wrote:

> Martin Rohde <martin.rohde@gmx.de> writes:
>
>> On Day 53 of The Aftermath 3168, kgreiner@xpediantsolutions.com wrote:
>>> I believe that nntp-delete-echo should be made buffer local as setting
>>> this globally may break configurations that use secondary nntp
>>> servers.
>>
>> I think, that the echo depends on your local configuration, because
>> it is /telnet/, that is sending these echos and not the server.
>> Ok, then it is not necessary to test it every time you connect to a
>> server, but when should this be done instead?
>
> It is necessary to test it every time you connect to a server, because...
>
>> But implementing this would be no problem, so if I am wrong and
>> there are situations, where the servers could echo something, too,
>> tell me.
>>
>> (It isn't possible to set nntp-open-connection-function for each
>> server differently, is it?)
>
> it *is* possible to set nntp-open-connection-function differently for
> each server.  That's the whole point, actually.
>
> You can put the definition in the server parameters, like this:
>
> (add-to-list 'gnus-secondary-select-methods
>              '(nntp "servername"
>                (nntp-address "host.name.of.server")
>                (nntp-open-connection-function nntp-open-telnet-stream)
>                ...other.nntp-open-telnet.settings...))

Oops. Sorry, then Kevin was right, nntp-delete-echo has to be
buffer-local.

I fixed the cleaning of the server-buffer after sending "MyNOOP"
like you suggested and it works, but I like the version of Kevin
very much. In my opinion it works just as well as testing for an
echo in nntp-open-connection, but saves code and time.

Hm, but it is right to refer to nntp-open-connection-function
globally, isn't it? It is connection-dependend, too, but I just
found it as a global variable...(That's why I thought, that it
would be independent...)

If so, here is yet another try:


[-- Attachment #2: nntp.el.diff --]
[-- Type: text/plain, Size: 3191 bytes --]

--- nntp.el.old	2002-12-13 16:10:35.000000000 +0100
+++ nntp.el	2002-12-13 16:30:41.000000000 +0100
@@ -218,6 +218,7 @@
 (defvar nntp-process-to-buffer nil)
 (defvar nntp-process-callback nil)
 (defvar nntp-process-decode nil)
+(defvar nntp-process-delete-echo nil)
 (defvar nntp-process-start-point nil)
 (defvar nntp-inside-change-function nil)
 (defvoo nntp-last-command-time nil)
@@ -411,14 +412,15 @@
 	  (unless (or wait-for
 		      (equal nntp-open-connection-function
 			     'nntp-open-network-stream))
-	    (nntp-accept-response)
 	    (save-excursion
 	      (set-buffer buffer)
-	      (goto-char pos)
-	      (if (looking-at (regexp-quote command))
-		  (delete-region pos (progn (forward-line 1)
-					    (gnus-point-at-bol))))
-	      )))
+	      (when nntp-process-delete-echo
+		(nntp-accept-response)
+		(goto-char pos)
+		(if (setq nntp-process-delete-echo (looking-at (regexp-quote command)))
+		    (delete-region pos (progn (forward-line 1)
+					      (gnus-point-at-bol))))
+		))))
       (nnheader-report 'nntp "Couldn't open connection to %s."
 		       nntp-address))))
 
@@ -435,15 +437,18 @@
 				nntp-server-buffer
 				wait-for nnheader-callback-function)
 	  ;; If nothing to wait for, still remove possibly echo'ed commands
-	  (unless wait-for
-	    (nntp-accept-response)
+	  (unless (or wait-for
+		      (equal nntp-open-connection-function
+			     'nntp-open-network-stream))
 	    (save-excursion
 	      (set-buffer buffer)
-	      (goto-char pos)
-	      (if (looking-at (regexp-quote command))
-		  (delete-region pos (progn (forward-line 1)
-					    (gnus-point-at-bol))))
-	      )))
+	      (when nntp-process-delete-echo
+		(nntp-accept-response)
+		(goto-char pos)
+		(if (setq nntp-process-delete-echo (looking-at (regexp-quote command)))
+		    (delete-region pos (progn (forward-line 1)
+					      (gnus-point-at-bol))))
+		))))
       (nnheader-report 'nntp "Couldn't open connection to %s."
 		       nntp-address))))
 
@@ -465,14 +470,18 @@
 				nntp-server-buffer
 				wait-for nnheader-callback-function t)
 	  ;; If nothing to wait for, still remove possibly echo'ed commands
-	  (unless wait-for
-	    (nntp-accept-response)
+	  (unless (or wait-for
+		      (equal nntp-open-connection-function
+			     'nntp-open-network-stream))
 	    (save-excursion
-	  (set-buffer buffer)
-	  (goto-char pos)
-	  (if (looking-at (regexp-quote command))
-	      (delete-region pos (progn (forward-line 1) (gnus-point-at-bol))))
-	  )))
+	      (set-buffer buffer)
+	      (when nntp-process-delete-echo
+		(nntp-accept-response)
+		(goto-char pos)
+		(if (setq nntp-process-delete-echo (looking-at (regexp-quote command)))
+		    (delete-region pos (progn (forward-line 1)
+					      (gnus-point-at-bol))))
+		))))
       (nnheader-report 'nntp "Couldn't open connection to %s."
 		       nntp-address))))
 
@@ -1048,6 +1057,7 @@
     (set (make-local-variable 'nntp-process-to-buffer) nil)
     (set (make-local-variable 'nntp-process-start-point) nil)
     (set (make-local-variable 'nntp-process-decode) nil)
+    (set (make-local-variable 'nntp-process-delete-echo) t)
     (current-buffer)))
 
 (defun nntp-open-connection (buffer)

Re: 'g' in the group buffer still very slow

[Kai Großjohann]

Martin Rohde <martin.rohde@gmx.de> writes:

> On Day 54 of The Aftermath 3168, Kai Großjohann wrote:
>
>> You can put the definition in the server parameters, like this:
>>
>> (add-to-list 'gnus-secondary-select-methods
>>              '(nntp "servername"
>>                (nntp-address "host.name.of.server")
>>                (nntp-open-connection-function nntp-open-telnet-stream)
>>                ...other.nntp-open-telnet.settings...))
>
> Oops. Sorry, then Kevin was right, nntp-delete-echo has to be
> buffer-local.

Yes.

> I fixed the cleaning of the server-buffer after sending "MyNOOP"
> like you suggested and it works, but I like the version of Kevin
> very much. In my opinion it works just as well as testing for an
> echo in nntp-open-connection, but saves code and time.

Hm?  If the code is simpler, I'm all for it, but I don't see what time
it saves.  Is it quicker to implement or easier to maintain, or does
it run quicker?

> Hm, but it is right to refer to nntp-open-connection-function
> globally, isn't it? It is connection-dependend, too, but I just
> found it as a global variable...(That's why I thought, that it
> would be independent...)

Code referring to a buffer-local variable looks the same as code
referring to a global variable.  See this:

(defvar x nil)          ; declare
(setq x 1)              ; global value
; switch to some buffer foo
(make-local-variable 'x); now it will be local to the foo buffer
x => 1                  ; local value same as global initially
(setq x 2)              ; set the local value
x => 2                  ; local value
; switch back to previous buffer
x => 1                  ; access global value again

As you can see, the setq looks the same and the access also looks the
same.  The only difference is the make-local-variable.

> If so, here is yet another try:

Looks good to me, even though I'm not so sure about the logic.

First of all, you still test the connection function, is that still
necessary?  If the look-for-echo test works well, it should be okay to
apply it always, making the code (a tiny bit) simpler.

Secondly, and this is the logic part, the check-for-echo variable
starts out as nil.  I didn't look very closely, but I thought that the
echo check is skipped when the variable is nil...

-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: 'g' in the group buffer still very slow

[Martin Rohde]

On Day 55 of The Aftermath 3168, Kai Großjohann wrote:

> Martin Rohde <martin.rohde@gmx.de> writes:
>
>> On Day 54 of The Aftermath 3168, Kai Großjohann wrote:
>>
>>> You can put the definition in the server parameters, like this:
>>>
>>> (add-to-list 'gnus-secondary-select-methods
>>>              '(nntp "servername"
>>>                (nntp-address "host.name.of.server")
>>>                (nntp-open-connection-function nntp-open-telnet-stream)
>>>                ...other.nntp-open-telnet.settings...))
>>
>> Oops. Sorry, then Kevin was right, nntp-delete-echo has to be
>> buffer-local.
>
> Yes.
>
>> I fixed the cleaning of the server-buffer after sending "MyNOOP"
>> like you suggested and it works, but I like the version of Kevin
>> very much. In my opinion it works just as well as testing for an
>> echo in nntp-open-connection, but saves code and time.
>
> Hm?  If the code is simpler, I'm all for it, but I don't see what time
> it saves.  Is it quicker to implement or easier to maintain, or does
> it run quicker?

Sorry, I mixed some things up.
All it saves is the sending of "MyNOOP". It is wrong to speak of
"saving time". 

It just saves the additional code in nntp-open-connection, that's
why I prefer it. But the other version works as well, so if you're
prefering your way, I can send you the working patch, too.

>> Hm, but it is right to refer to nntp-open-connection-function
>> globally, isn't it? It is connection-dependend, too, but I just
>> found it as a global variable...(That's why I thought, that it
>> would be independent...)
>
> Code referring to a buffer-local variable looks the same as code
> referring to a global variable.  See this:
>
> (defvar x nil)          ; declare
> (setq x 1)              ; global value
> ; switch to some buffer foo
> (make-local-variable 'x); now it will be local to the foo buffer
> x => 1                  ; local value same as global initially
> (setq x 2)              ; set the local value
> x => 2                  ; local value
> ; switch back to previous buffer
> x => 1                  ; access global value again
>
> As you can see, the setq looks the same and the access also looks the
> same.  The only difference is the make-local-variable.

Yes, but i couldn't see a 

(make-local-variable nntp-open-connection-function) 

anywhere. Furthermore, I am not in a special buffer, when I check it
(or am I?), so that's why I asked.

>> If so, here is yet another try:
>
> Looks good to me, even though I'm not so sure about the logic.
>
> First of all, you still test the connection function, is that still
> necessary?  If the look-for-echo test works well, it should be okay to
> apply it always, making the code (a tiny bit) simpler.

I thought, it would make the start of gnus a bit faster when you are
using nntp-open-network-stream, because otherwise you would still
have the delay from nntp-accept-response when nntp-send-command is
called for the first time.

> Secondly, and this is the logic part, the check-for-echo variable
> starts out as nil.  I didn't look very closely, but I thought that the
> echo check is skipped when the variable is nil...

Its global value is nil, but in nntp-make-process-buffer it is
initialized as

(set (make-local-variable 'nntp-process-delete-echo) t)

The (defvar nntp-process-delete-echo nil) ist just a dummy
declaration (because of "free variable"-warnings). For it has to be
checked locally in the server-buffers, it doesn't matter, what its
global value is, as you explained before. 

Martin

Re: 'g' in the group buffer still very slow

[Kai Großjohann]

Martin Rohde <martin.rohde@gmx.de> writes:

> Sorry, I mixed some things up.
> All it saves is the sending of "MyNOOP". It is wrong to speak of
> "saving time". 

Ah, it saves sending that command!  Okay, I completely missed that.
That's a minor point, but it *is* an advantage.

> It just saves the additional code in nntp-open-connection, that's
> why I prefer it. But the other version works as well, so if you're
> prefering your way, I can send you the working patch, too.

No, now I'm convinced that the no-MyNOOP way is better.  So I agree
with you.

> Yes, but i couldn't see a 
>
> (make-local-variable nntp-open-connection-function) 
>
> anywhere. Furthermore, I am not in a special buffer, when I check it
> (or am I?), so that's why I asked.

Whee.  Well, err.  The Gnus backend code is pretty convoluted, and I
believe any number of wonders that can be done with nnoo.

Ah, and nntp-open-connection-function is defvoo'd, not defvar'd.  That
explains it all.  I started to look in defvoo and then in nnoo-define,
but I get lost in a twisty maze of little backquotes...  Anyhow, I'm
almost sure that nnoo has some magic to make the variable local.

Anyone?

> I thought, it would make the start of gnus a bit faster when you are
> using nntp-open-network-stream, because otherwise you would still
> have the delay from nntp-accept-response when nntp-send-command is
> called for the first time.

Hm.  This happens if the first command doesn't produce a response,
right?  Anyhow, it's not so important.  Just leave it like that but a
comment (connections created with nntp-open-network-stream will never
echo; don't bother checking for echoes there) would be nice.

>> Secondly, and this is the logic part, the check-for-echo variable
>> starts out as nil.  I didn't look very closely, but I thought that the
>> echo check is skipped when the variable is nil...
>
> Its global value is nil, but in nntp-make-process-buffer it is
> initialized as
>
> (set (make-local-variable 'nntp-process-delete-echo) t)
>
> The (defvar nntp-process-delete-echo nil) ist just a dummy
> declaration (because of "free variable"-warnings). For it has to be
> checked locally in the server-buffers, it doesn't matter, what its
> global value is, as you explained before. 

What do you think about making it (defvar nntp-process-delete-echo t),
then then, in nntp-make-process-buffer, to just do
(make-local-variable 'nntp-process-delete-echo)?

It might make it a little bit clearer what is the default value of the
variable, so outsiders have a better chance of understanding the code.

Just my two cents.

In general, this is good stuff!
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: 'g' in the group buffer still very slow

[Martin Rohde]

[-- Attachment #1: Type: text/plain, Size: 1196 bytes --]

On Day 56 of The Aftermath 3168, Kai Großjohann wrote:

>> I thought, it would make the start of gnus a bit faster when you are
>> using nntp-open-network-stream, because otherwise you would still
>> have the delay from nntp-accept-response when nntp-send-command is
>> called for the first time.
>
> Hm.  This happens if the first command doesn't produce a response,
> right?  Anyhow, it's not so important.  Just leave it like that but a
> comment (connections created with nntp-open-network-stream will never
> echo; don't bother checking for echoes there) would be nice.

In nntp-send-command, there still is the comment of your last change
of this part, which is just right for this purpose. (Oh, but it's
absent in the other two functions, so I added it there.)

> What do you think about making it (defvar nntp-process-delete-echo t),
> then then, in nntp-make-process-buffer, to just do
> (make-local-variable 'nntp-process-delete-echo)?
>
> It might make it a little bit clearer what is the default value of the
> variable, so outsiders have a better chance of understanding the code.
>
> Just my two cents.

I agree with you, I missed that make-local-variable adopts the
global value.


[-- Attachment #2: nntp.el.diff --]
[-- Type: text/plain, Size: 3437 bytes --]

--- nntp.el.old	2002-12-14 14:22:49.000000000 +0100
+++ nntp.el	2002-12-14 14:27:55.000000000 +0100
@@ -218,6 +218,7 @@
 (defvar nntp-process-to-buffer nil)
 (defvar nntp-process-callback nil)
 (defvar nntp-process-decode nil)
+(defvar nntp-process-delete-echo t)
 (defvar nntp-process-start-point nil)
 (defvar nntp-inside-change-function nil)
 (defvoo nntp-last-command-time nil)
@@ -411,14 +412,15 @@
 	  (unless (or wait-for
 		      (equal nntp-open-connection-function
 			     'nntp-open-network-stream))
-	    (nntp-accept-response)
 	    (save-excursion
 	      (set-buffer buffer)
-	      (goto-char pos)
-	      (if (looking-at (regexp-quote command))
-		  (delete-region pos (progn (forward-line 1)
-					    (gnus-point-at-bol))))
-	      )))
+	      (when nntp-process-delete-echo
+		(nntp-accept-response)
+		(goto-char pos)
+		(if (setq nntp-process-delete-echo (looking-at (regexp-quote command)))
+		    (delete-region pos (progn (forward-line 1)
+					      (gnus-point-at-bol))))
+		))))
       (nnheader-report 'nntp "Couldn't open connection to %s."
 		       nntp-address))))
 
@@ -435,15 +437,20 @@
 				nntp-server-buffer
 				wait-for nnheader-callback-function)
 	  ;; If nothing to wait for, still remove possibly echo'ed commands
-	  (unless wait-for
-	    (nntp-accept-response)
+	  ;; We don't have echos if nntp-open-connection-function
+	  ;; is `nntp-open-network-stream', so we skip this in that case.
+	  (unless (or wait-for
+		      (equal nntp-open-connection-function
+			     'nntp-open-network-stream))
 	    (save-excursion
 	      (set-buffer buffer)
-	      (goto-char pos)
-	      (if (looking-at (regexp-quote command))
-		  (delete-region pos (progn (forward-line 1)
-					    (gnus-point-at-bol))))
-	      )))
+	      (when nntp-process-delete-echo
+		(nntp-accept-response)
+		(goto-char pos)
+		(if (setq nntp-process-delete-echo (looking-at (regexp-quote command)))
+		    (delete-region pos (progn (forward-line 1)
+					      (gnus-point-at-bol))))
+		))))
       (nnheader-report 'nntp "Couldn't open connection to %s."
 		       nntp-address))))
 
@@ -465,14 +472,20 @@
 				nntp-server-buffer
 				wait-for nnheader-callback-function t)
 	  ;; If nothing to wait for, still remove possibly echo'ed commands
-	  (unless wait-for
-	    (nntp-accept-response)
+	  ;; We don't have echos if nntp-open-connection-function
+	  ;; is `nntp-open-network-stream', so we skip this in that case.
+	  (unless (or wait-for
+		      (equal nntp-open-connection-function
+			     'nntp-open-network-stream))
 	    (save-excursion
-	  (set-buffer buffer)
-	  (goto-char pos)
-	  (if (looking-at (regexp-quote command))
-	      (delete-region pos (progn (forward-line 1) (gnus-point-at-bol))))
-	  )))
+	      (set-buffer buffer)
+	      (when nntp-process-delete-echo
+		(nntp-accept-response)
+		(goto-char pos)
+		(if (setq nntp-process-delete-echo (looking-at (regexp-quote command)))
+		    (delete-region pos (progn (forward-line 1)
+					      (gnus-point-at-bol))))
+		))))
       (nnheader-report 'nntp "Couldn't open connection to %s."
 		       nntp-address))))
 
@@ -1048,6 +1061,7 @@
     (set (make-local-variable 'nntp-process-to-buffer) nil)
     (set (make-local-variable 'nntp-process-start-point) nil)
     (set (make-local-variable 'nntp-process-decode) nil)
+    (make-local-variable 'nntp-process-delete-echo)
     (current-buffer)))
 
 (defun nntp-open-connection (buffer)

Re: 'g' in the group buffer still very slow

[Kai Großjohann]

Martin Rohde <martin.rohde@gmx.de> writes:

> +	      ;; Send a No-Op command to recognize if we need to delete local echo
> +	      ;; in future. This only makes sense, if we are using a non-direct connection...
> +	      (unless (equal nntp-open-connection-function
> +			     'nntp-open-network-stream)
> +		(let ((command "MyNOOP")
> +		      (pos (point)))
> +		  (nntp-send-string process command)
> +		  (accept-process-output process)
> +		  (goto-char pos)
> +		  (when (looking-at (regexp-quote command))
> +		    (setq nntp-delete-echo t))))
> +
> +	      (erase-buffer)

I think it is dangerous to just erase the buffer here.  Consider the
following:

Gnus sends MyNOOP command.  The response arrives in two chunks: first
the echo arrives, then the response from the remote server ("500
what?" or so) arrives.

Then accept-process-output would return after the first chunk, and
erase-buffer would erase too little.

>  	      (set-buffer nntp-server-buffer)
> +	      
>  	      (let ((nnheader-callback-function nil))
>  		(run-hooks 'nntp-server-opened-hook)
>  		(nntp-send-authinfo t))))

I think what you want to do is to wait until all of the response from
the server has appeared.  Ie, you wait until you see the line with
the status code.

You can't use nntp-wait-for to do this because it looks at the
beginning of the buffer (you'll see (goto-char (point-min)) if you
look at the code).  (That's also why your previous approach didn't
work I think.)

Idea: after accept-process-output, you look for the echo.  If there
is one, remove it (and only it) from the buffer.  Then do
nntp-wait-for.  Then erase-buffer.  Do you think that might work?

-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: 'g' in the group buffer still very slow

[Kai Großjohann]

Martin Rohde <martin.rohde@gmx.de> writes:

> doesn't work. It has to be "or" instead of "and"...

Eek.  Committed.  Thanks!  Sorry, Sebastian.

-- 
~/.signature is: umop ap!sdn    (Frank Nobis)