8bit mails via POP mail-source

8bit mails via POP mail-source

[Pavel Janík ml.]

Hi again,

Wed, 9 Aug 2000 15:27:19 +0200 I asked almost similar question, but Gnus
still behaves user non-friendly.

I have something like this in my .gnus:

(setq mail-sources '((pop :server "POPSERVER" :user "USER")))

So, I'm reading my mail from POPSERVER as user USER. It works, but
sometimes (~40 from 200) I'm asked to choose coding-system:

These default coding systems were tried:
  iso-8859-2
However, none of them safely encodes the target text.

Select one of the following safe coding systems:
  raw-text emacs-mule no-conversion

I use czech language environment. Everything is OK when I do RET for each
mail which cause this.

Do we have any solution to this? Some variable *-coding-system should be
set better?

GNU Emacs 21, Gnus 5.9.0 or CVS version (does not matter).
-- 
Pavel Janík ml.
Pavel@Janik.cz
http://www.janik.cz

Re: 8bit mails via POP mail-source

[Stainless Steel Rat]

* Pavel@Janik.cz (Pavel Janík ml.)  on Tue, 12 Dec 2000
| These default coding systems were tried:
|   iso-8859-2
| However, none of them safely encodes the target text.

Of course it doesn't.  POP3 is a binary data stream, not text.  It will
break if you try to treat it as text.  pop3.el *should* do the right thing,
but I haven't maintained that code since the FSF *and* XEmacs developers
yanked it out from under me.
-- 
Rat <ratinox@peorth.gweep.net>    \ When not in use, Happy Fun Ball should be
Minion of Nathan - Nathan says Hi! \ returned to its special container and
PGP Key: at a key server near you!  \ kept under refrigeration.

Re: 8bit mails via POP mail-source

[Pavel Janík ml.]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

   From: Stainless Steel Rat <ratinox@peorth.gweep.net>
   Date: 12 Dec 2000 16:36:17 -0500

Hi,

   > * Pavel@Janik.cz (Pavel Janík ml.)  on Tue, 12 Dec 2000
   > | These default coding systems were tried:
   > |   iso-8859-2
   > | However, none of them safely encodes the target text.
   > 
   > Of course it doesn't.  POP3 is a binary data stream, not text.  It will
   > break if you try to treat it as text.  pop3.el *should* do the right thing,
   > but I haven't maintained that code since the FSF *and* XEmacs developers
   > yanked it out from under me.

[-- Attachment #2: Type: text/plain, Size: 93 bytes --]


OK, so is there any solution or not?
-- 
Pavel Janík ml.
Pavel@Janik.cz
http://www.janik.cz

Re: 8bit mails via POP mail-source

[ShengHuo ZHU]

Pavel@Janik.cz (Pavel Janík ml.) writes:

>    From: Stainless Steel Rat <ratinox@peorth.gweep.net>
>    Date: 12 Dec 2000 16:36:17 -0500
> 
> Hi,
> 
>    > * Pavel@Janik.cz (Pavel Janík ml.)  on Tue, 12 Dec 2000
>    > | These default coding systems were tried:
>    > |   iso-8859-2
>    > | However, none of them safely encodes the target text.
>    > 
>    > Of course it doesn't.  POP3 is a binary data stream, not text.  It will
>    > break if you try to treat it as text.  pop3.el *should* do the right thing,
>    > but I haven't maintained that code since the FSF *and* XEmacs developers
>    > yanked it out from under me.
> 
> OK, so is there any solution or not?

Try (setq pop3-movemail-file-coding-system 'binary).

ShengHuo

Re: 8bit mails via POP mail-source

[Stainless Steel Rat]

* ShengHuo ZHU <zsh@cs.rochester.edu>  on Wed, 13 Dec 2000
| Try (setq pop3-movemail-file-coding-system 'binary).

/me checks stuff...  Well, that is dumb.  There shouldn't be a coding
system set in the movemail function.  That gets handled by the read/write
coding system settings in pop3-open-server, which hard-sets it to 'binary
which is the only appropriate setting for a raw data stream.  By no means
should it be settable by the user because anything other than 'binary will
break.  Please file a bug report.

FWIW, the version of pop3.el that ships with XEmacs 21 is not broken in
this manner.  Nor is the real 1.3s version (the FSF has failed to increment
version numbers with the changes they have made).
-- 
Rat <ratinox@peorth.gweep.net>    \ When not in use, Happy Fun Ball should be
Minion of Nathan - Nathan says Hi! \ returned to its special container and
PGP Key: at a key server near you!  \ kept under refrigeration.

Re: 8bit mails via POP mail-source

[Vladimir Volovich]

"Rat" == Stainless Steel Rat writes:

[ btw, i "suffer" from this bug in pop3.el for several years now...
  i asked to fix it, but you answered as as you did in your previous
  mail ]

 Rat> * ShengHuo ZHU <zsh@cs.rochester.edu> on Wed, 13 Dec 2000 | Try
 Rat> (setq pop3-movemail-file-coding-system 'binary).

 Rat> /me checks stuff...  Well, that is dumb.  There shouldn't be a
 Rat> coding system set in the movemail function.  That gets handled
 Rat> by the read/write coding system settings in pop3-open-server,
 Rat> which hard-sets it to 'binary which is the only appropriate
 Rat> setting for a raw data stream.  By no means should it be
 Rat> settable by the user because anything other than 'binary will
 Rat> break.  Please file a bug report.

well... why ShengHuo or you should file a bug report to anyone? isn't
pop3.el part of gnus?! just fix the bug as you want, and the problem
is over!

 Rat> FWIW, the version of pop3.el that ships with XEmacs 21 is not
 Rat> broken in this manner.  Nor is the real 1.3s version (the FSF
 Rat> has failed to increment version numbers with the changes they
 Rat> have made).

could you please merge your latest version into gnus?

	Best regards, -- Vladimir.

Re: 8bit mails via POP mail-source

[Stainless Steel Rat]

* Vladimir Volovich <vvv@vvv.vsu.ru>  on Thu, 14 Dec 2000
| well... why ShengHuo or you should file a bug report to anyone? isn't
| pop3.el part of gnus?! just fix the bug as you want, and the problem
| is over!

pop3.el is not part of Gnus, it is part of GNU Emacs.  The FSF owns it and
they maintain it.  I don't, and since they yanked maintainence out from
under me without any notice whatsoever I have no interest in resuming any
of it.  If you want it fixed you have to tell them to fix it.

[...]
| could you please merge your latest version into gnus?

My latest version is bundled with whatever version of Gnus was current
circa 1999-06-18 (according to my changelog).  That is the real 1.3s.
-- 
Rat <ratinox@peorth.gweep.net>    \ Happy Fun Ball may stick to certain types
Minion of Nathan - Nathan says Hi! \ of skin.
PGP Key: at a key server near you!  \ 

Re: 8bit mails via POP mail-source

[Kai Großjohann]

On Thu, 14 Dec 2000, Stainless Steel Rat wrote:

> pop3.el is not part of Gnus, it is part of GNU Emacs.

Why is it included in the Gnus tarball, then?  Hm.  It appears that if
some change is made to the Gnus distribution, then the modified
pop3.el will sometime end up in Emacs.  Maybe someone else can do the
work if you don't want to.

kai
-- 
A large number of young women don't trust men with beards.  (BFBS Radio)

Re: 8bit mails via POP mail-source

[Stainless Steel Rat]

* Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann)  on Thu, 14 Dec 2000
| Why is it included in the Gnus tarball, then?

Because I pulled a trick on RMS.  He didn't want to include it in the core
FSF Emacs distribution on its own (at the time the Win32 port was occouring
on its own and the patches were not being folded into the main code tree).
I got around that by getting Lars to include it in Gnus -- which *is* a
part of GNU Emacs.

| Hm.  It appears that if some change is made to the Gnus distribution,
| then the modified pop3.el will sometime end up in Emacs.  Maybe someone
| else can do the work if you don't want to.

That doesn't work.  The FSF had been ignoring the code I had been giving to
Lars from around 1.3m.  1.3s was my last version, which counts for some 6
major bugs fixed that they did not fold into their code tree.  I had a big
argument with someone at the FSF last year over this.  They finally did
rewrite based on v1.3s, but there is stuff in there now that I just don't
understand.

If someone else wants to take over, then be my guest.  But you have to go
through the FSF for that.
-- 
Rat <ratinox@peorth.gweep.net>    \ If Happy Fun Ball begins to smoke, get
Minion of Nathan - Nathan says Hi! \ away immediately. Seek shelter and cover
PGP Key: at a key server near you!  \ head.

Re: 8bit mails via POP mail-source

[vvv]

"Rat" == Stainless Steel Rat writes:

 Rat> My latest version is bundled with whatever version of Gnus was
 Rat> current circa 1999-06-18 (according to my changelog).  That is
 Rat> the real 1.3s.

is this real 1.3s available somewhere? could you send it to me?

i've reported the bug to emacs-pretesters-bug@gnu.org mailing list,
and got a reply that they agree to fix the problem with incorrect
pop3-movemail-file-coding-system setting. [they said that either the
default for that variable should be changed to 'binary or it should be
removed]

i'd like to suggest them to use your real version.

Best,
v.

Re: 8bit mails via POP mail-source

[Kenichi Handa]

vvv@vsu.ru writes:
> i'd like to ask Emacs maintainers to fix the problem in pop3.el which
> is there for several years now. The problem is that Emacs tries to use
> some non-binary coding system for buffers where it receives raw pop3
> stream. The text of messages received via pop3 could be in any
> charset, even mixed charsets are possible. So, the only possible
> choice is to use binary coding system for those buffers.

I agree with you.

I've just read pop3.el included in Emacs 21 pretest version.

The default value of pop3-movemail-file-coding-system is
nil, and pop3-movemail is the only function that uses this
variable.  It seems that what pop3-movemail does is to write
out data received from pop server into a file.  And the data
is received by `binary' coding system.  Then, we should use
`binary' also for writing to a file.

---
Ken'ichi HANDA
handa@etl.go.jp

Re: 8bit mails via POP mail-source

[Stainless Steel Rat]

* vvv@vsu.ru  on Fri, 15 Dec 2000
| is this real 1.3s available somewhere? could you send it to me?

I don't have it readilly available.  As I said, it is in the Gnus tarball
ca 1999-06-18, which is on Lars's server, so if you grab that tarball you
can get the hopefully working version.

| i've reported the bug to emacs-pretesters-bug@gnu.org mailing list,
| and got a reply that they agree to fix the problem with incorrect
| pop3-movemail-file-coding-system setting. [they said that either the
| default for that variable should be changed to 'binary or it should be
| removed]

It *must* be removed.  pop3-open-server sets the coding system to 'binary,
which is the only coding system that works correctly.  Anything else *will*
corrupt the mailbox.  Not may, will.
-- 
Rat <ratinox@peorth.gweep.net>    \ If Happy Fun Ball begins to smoke, get
Minion of Nathan - Nathan says Hi! \ away immediately. Seek shelter and cover
PGP Key: at a key server near you!  \ head.

Re: 8bit mails via POP mail-source

[Stainless Steel Rat]

Okay, the copy sent to emacs-pretesters-bug bounced :P.
Someone want to forward this to them?  Thanks.

* Stainless Steel Rat <ratinox@peorth.gweep.net>  on Fri, 15 Dec 2000
| * vvv@vsu.ru  on Fri, 15 Dec 2000
| | is this real 1.3s available somewhere? could you send it to me?

| I don't have it readilly available.  As I said, it is in the Gnus tarball
| ca 1999-06-18, which is on Lars's server, so if you grab that tarball you
| can get the hopefully working version.

| | i've reported the bug to emacs-pretesters-bug@gnu.org mailing list,
| | and got a reply that they agree to fix the problem with incorrect
| | pop3-movemail-file-coding-system setting. [they said that either the
| | default for that variable should be changed to 'binary or it should be
| | removed]

| It *must* be removed.  pop3-open-server sets the coding system to 'binary,
| which is the only coding system that works correctly.  Anything else *will*
| corrupt the mailbox.  Not may, will.
-- 
Rat <ratinox@peorth.gweep.net>    \ When not in use, Happy Fun Ball should be
Minion of Nathan - Nathan says Hi! \ returned to its special container and
PGP Key: at a key server near you!  \ kept under refrigeration.

Re: 8bit mails via POP mail-source

[ShengHuo ZHU]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> Okay, the copy sent to emacs-pretesters-bug bounced :P.
> Someone want to forward this to them?  Thanks.

It is emacs-pretest-bug@gnu.org.  I CC'd this message.

> 
> * Stainless Steel Rat <ratinox@peorth.gweep.net>  on Fri, 15 Dec 2000
> | * vvv@vsu.ru  on Fri, 15 Dec 2000
> | | is this real 1.3s available somewhere? could you send it to me?
> 
> | I don't have it readilly available.  As I said, it is in the Gnus tarball
> | ca 1999-06-18, which is on Lars's server, so if you grab that tarball you
> | can get the hopefully working version.
> 
> | | i've reported the bug to emacs-pretesters-bug@gnu.org mailing list,
> | | and got a reply that they agree to fix the problem with incorrect
> | | pop3-movemail-file-coding-system setting. [they said that either the
> | | default for that variable should be changed to 'binary or it should be
> | | removed]
> 
> | It *must* be removed.  pop3-open-server sets the coding system to 'binary,
> | which is the only coding system that works correctly.  Anything else *will*
> | corrupt the mailbox.  Not may, will.

I've removed pop3-movemail-file-coding-system in both Gnus CVS and
Emacs 21 CVS.

ShengHuo

Re: 8bit mails via POP mail-source

[vvv]

"Rat" == Stainless Steel Rat writes:

 Rat> is this real 1.3s available somewhere?
 Rat> I don't have it readilly available.  As I said, it is in the
 Rat> Gnus tarball ca 1999-06-18, which is on Lars's server, so if you
 Rat> grab that tarball you can get the hopefully working version.

i see there (on ftp://quimby.gnus.org/pub/gnus) two versions:

-rw-r--r--   1 larsi    larsi      901759 Jun 15  1999 pgnus-0.88.tar.gz
-rw-r--r--   1 larsi    larsi      902941 Jun 28  1999 pgnus-0.89.tar.gz

one of them was released earlier than 1999-06-18, and one was released
later. i'm grabbing both versions to see whether they contain
different pop3.el; how can i find whether the version of pop3.el is
"real"?

Best,
v.

Re: 8bit mails via POP mail-source

[vvv]

> i'm grabbing both versions to see whether they contain different pop3.el;

they appeared to contain the same pop3.el (no differences), but the
version number was set to 1.3r.

Re: 8bit mails via POP mail-source

[Stainless Steel Rat]

* vvv@vsu.ru  on Sat, 16 Dec 2000
| they appeared to contain the same pop3.el (no differences), but the
| version number was set to 1.3r.

Nope.  1.3s is the last version I released.

The last non-cosmetic change I made was the generation of Content-Length
headers in pop3-munge-message-separator.

Other than that, the easiest way to tell my code from the FSF branch is
that mine makes the statement that both FSF Emacs and XEmacs are stupid
about coding systems (which is true :).
-- 
Rat <ratinox@peorth.gweep.net>    \ When not in use, Happy Fun Ball should be
Minion of Nathan - Nathan says Hi! \ returned to its special container and
PGP Key: at a key server near you!  \ kept under refrigeration.

Re: 8bit mails via POP mail-source

[Dave Love]

>>>>> "ZSH" == ShengHuo ZHU <zsh@cs.rochester.edu> writes:

 >> | It *must* be removed.  pop3-open-server sets the coding system to 'binary,
 >> | which is the only coding system that works correctly.  Anything else *will*
 >> | corrupt the mailbox.  Not may, will.

 ZSH> I've removed pop3-movemail-file-coding-system in both Gnus CVS and
 ZSH> Emacs 21 CVS.

For people reading this on ding who might be confused, note that the
coding system should be bound non-nil when writing the crashbox.  (It
now is, even if the variable added by the Mule maintainer has been
removed).  That was not done in the version merged into the Emacs
repository from Gnus when we found we weren't being sent changes.

Note that Emacs asks the question that was the subject of the bug
report to avoid corrupting the data by using an unsafe coding system.
This seems not to be the case in XEmacs.

Re: 8bit mails via POP mail-source

[Stainless Steel Rat]

* Dave Love <d.love@dl.ac.uk>  on Thu, 21 Dec 2000
| For people reading this on ding who might be confused, note that the
| coding system should be bound non-nil when writing the crashbox.

Again, it must be 'binary.  Anything else will cause corruption of the
mailbox if the coding systems don't match (ie, Russian text and a Japanese
coding system).  If you are going to do coding systems then they should be
done on a per-message basis, not a per-mailbox basis.
-- 
Rat <ratinox@peorth.gweep.net>    \ Caution: Happy Fun Ball may suddenly
Minion of Nathan - Nathan says Hi! \ accelerate to dangerous speeds.
PGP Key: at a key server near you!  \ 

Re: 8bit mails via POP mail-source

[Stainless Steel Rat]

* Miles Bader <miles@gnu.org>  on Sat, 23 Dec 2000
| This was fixed a while ago.

Yay.
Now please test it to make sure it really is fixed.

| > As for my rants about maintenance, the fact that I have to deal with Dave
| > Love's insults just to get a bugfix in speaks for itself.
| Pot, kettle...

Nobody ever had to brow-beat me into fixing a bug in code I maintained.
-- 
Rat <ratinox@peorth.gweep.net>    \ Ingredients of Happy Fun Ball include an
Minion of Nathan - Nathan says Hi! \ unknown glowing substance which fell to
PGP Key: at a key server near you!  \ Earth, presumably from outer space.

Re: 8bit mails via POP mail-source

[Miles Bader]

On Fri, Dec 22, 2000 at 05:47:23PM -0500, Stainless Steel Rat wrote:
> The problem is known.  The fix is known.  So please shut the fuck up and
> fix the code.

This was fixed a while ago.

> As for my rants about maintenance, the fact that I have to deal with Dave
> Love's insults just to get a bugfix in speaks for itself.

Pot, kettle...

-Miles
-- 
P.S.  All information contained in the above letter is false,
      for reasons of military security.

Re: 8bit mails via POP mail-source

[Stainless Steel Rat]

"DL" == Dave Love writes:
DL> Note that Pieri is wrong about the fix.  Please also disregard
DL> his rants about maintenance.

My code which forces the coding-system 'binary works.
The FSF code which sets it 'nil through a variable breaks 8-bit mailboxes.
The FSF code with the same variable set to 'binary works.

I went through this in October and November of 1998 after MULE became a
"requirement".  If the coding system is not 'binary then 8-bit mailboxes
will be corrupted.  This is a proven fact.

The problem is known.  The fix is known.  So please shut the fuck up and
fix the code.

As for my rants about maintenance, the fact that I have to deal with Dave
Love's insults just to get a bugfix in speaks for itself.
-- 
Rat <ratinox@peorth.gweep.net>    \ Warning: pregnant women, the elderly, and
Minion of Nathan - Nathan says Hi! \ children under 10 should avoid prolonged
PGP Key: at a key server near you!  \ exposure to Happy Fun Ball.

Re: 8bit mails via POP mail-source

[vvv]

"DL" == Dave Love writes:

 vvv> I've attached a mail digest from the gnus mailing list which
 vvv> describes the problem. In particular, see the message from
 vvv> ratinox@peorth.gweep.net with Message-ID
 vvv> <m3g0jrsu3d.fsf@peorth.gweep.net>.

 DL> Note that Pieri is wrong about the fix.  Please also disregard
 DL> his rants about maintenance.

well, could you please explain in more details why Pieri is wrong
about the fix?

the current pop3.el behaves buggy. it asks annoying and nonsence
questions when receiving mail via POP3. the original pop3.el by Pieri
does not have such broken behavior.

if you find Pieri's arguments wrong, then please fix pop3.el as you
think it is correct. but current pop3.el is broken.

Best,
v.