PDF's as base64 vs. Q/P encoded attachments

PDF's as base64 vs. Q/P encoded attachments

[Graham Todd]

This is just for the information of anyone who has a similar problem
since I'm not too knowledgeable about MIME and encoding schemes.

There was a thread a while ago about how to decide between encoding
schemes for attachments and Lars wrote a function that switched
between them based on efficiency. I think it was decided that if
roughly 1/6 of the characters were non-ASCII, then base64 should be
used otherwise Q/P (I may be wrong here).

So Gnus now tries to encode PDF's I create as Q/P (I'm pretty sure it
didn't do this before).  The problem is that receiving PDF's as Q/P
seems to break the MIME handling of the Windows version of Netscape
Mail somehow so that Windows users that I correspond with couldn't
read the attached PDF files in Acrobat. Acrobat gave an error message
saying the files were broken. As an aside I can detach and read PDF's
sent as Q/P content to myself using the Unix (linux & freebsd)
versions of Acrobat and Netscape just fine.

If the cut-off point for encoding is 1/6 or more non-ASCII characters
most PDF's would/should get encoded as base64 wouldn't they?  Anyway
they appear ASCIIish to Gnus.

I did something like this to fix it:

(push '("application/pdf" base64) mm-content-transfer-encoding-alist)

so that PDF's are always encoded base64.

regards,

-- 
Graham Todd        

Re: PDF's as base64 vs. Q/P encoded attachments

[Toby Speight]

Graham> Graham Todd <URL:mailto:gtodd@yorku.ca>

0> In <URL:news:x67lo12n1p.fsf@ninga.ellerbeck.org>, Graham wrote:

Graham> So Gnus now tries to encode PDF's I create as Q/P (I'm pretty sure
Graham> it didn't do this before).  The problem is that receiving PDF's as
Graham> Q/P seems to break the MIME handling of the Windows version of
Graham> Netscape Mail somehow so that Windows users that I correspond with
Graham> couldn't read the attached PDF files in Acrobat. Acrobat gave an
Graham> error message saying the files were broken. As an aside I can
Graham> detach and read PDF's sent as Q/P content to myself using the Unix
Graham> (linux & freebsd) versions of Acrobat and Netscape just fine.

I wonder if PDF is sensitive to newline-mangling?  AIUI, PDF is based
on Postscript, which should be okay, but it's possible that some
application/* types need newlines preserving as "=0A=\n".  A pain!

Could you check this hypothesis?  Compose a mail on a Unix platform,
with two attachments - one q-p encoded normally, and one with newlines
explicitly kept, as above (use query-replace or something, after
MIME-encoding).  Then try reading both on a Windows platform.


Graham> I did something like this to fix it:
Graham>   (push '("application/pdf" base64)
Graham>         mm-content-transfer-encoding-alist)
Graham> so that PDF's are always encoded base64.

Seems a sensible workaround, but if there's a true bug, it would be
nice to nail it.

BTW - Lars, did you remember to include '=' with the non-ASCII
characters for the purposes of estimating q-p size?

Re: PDF's as base64 vs. Q/P encoded attachments

[Robert Bihlmeyer]

Hi,

>>>>> On Fri, 16 Jul 1999 05:13:18 GMT
>>>>> Graham Todd <gtodd@yorku.ca> said:

 Graham> There was a thread a while ago about how to decide between
 Graham> encoding schemes for attachments and Lars wrote a function
 Graham> that switched between them based on efficiency.

Is this already in the distribution?

 Graham> If the cut-off point for encoding is 1/6 or more non-ASCII
 Graham> characters most PDF's would/should get encoded as base64
 Graham> wouldn't they?

At least the compressed ones are sure to end up having about half of
their characters outside ASCII.

        Robbe

-- 
Robert Bihlmeyer	reads: Deutsch, English, MIME, Latin-1, NO SPAM!
<robbe@orcus.priv.at>	<http://stud2.tuwien.ac.at/~e9426626/sig.html>

Re: PDF's as base64 vs. Q/P encoded attachments

[Graham Todd]

>>>>> "Toby" == Toby Speight <Toby.Speight@streapadair.freeserve.co.uk> writes:

 Graham> Graham Todd <URL:mailto:gtodd@yorku.ca>

 0> In <URL:news:x67lo12n1p.fsf@ninga.ellerbeck.org>, Graham wrote:

 Graham> ... Gnus now tries to encode PDF's I create as Q/P ...

<snip>

 Toby> I wonder if PDF is sensitive to newline-mangling?  AIUI, PDF
 Toby> is based on Postscript, which should be okay, but it's
 Toby> possible that some application/* types need newlines
 Toby> preserving as "=0A=\n".  A pain!

 Toby> Could you check this hypothesis?  Compose a mail on a Unix
 Toby> platform, with two attachments - one q-p encoded normally,
 Toby> and one with newlines explicitly kept, as above (use
 Toby> query-replace or something, after MIME-encoding).  Then try
 Toby> reading both on a Windows platform.

I MIME encoded and then did the search replace and sent both as QP -
neither worked in Windows clients. Of course I might have screw

 Graham> I did something like this to fix it: (push
 Graham> '("application/pdf" base64)
 Graham> mm-content-transfer-encoding-alist) so that PDF's are
 Graham> always encoded base64.

Actually I think the correct line should be:

(push '("application/pdf" base64) mm-content-transfer-encoding-defaults)

It works for me (TM).

-- 
Graham Todd        

Re: PDF's as base64 vs. Q/P encoded attachments

[Graham Todd]

>>>>> "Robbe" == Robert Bihlmeyer <e9426626@stud4.tuwien.ac.at> writes:
Message-ID: <x6so6h7ifj.fsf@ninga.ellerbeck.org>
Lines: 13
User-Agent: Gnus/5.070095 (Pterodactyl Gnus v0.95) XEmacs/20.4 (Emerald)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Date: Wed, 21 Jul 1999 22:19:41 GMT
NNTP-Posting-Host: 206.172.12.231
X-Complaints-To: news@sunsite.auc.dk
X-Trace: sunsite.auc.dk 932595581 206.172.12.231 (Thu, 22 Jul 1999 00:19:41 MET DST)
NNTP-Posting-Date: Thu, 22 Jul 1999 00:19:41 MET DST
Organization: SunSITE Denmark (sunsite.auc.dk)

>>>>> On Fri, 16 Jul 1999 05:13:18 GMT
>>>>> Graham Todd <gtodd@yorku.ca> said:

    Graham> There was a thread a while ago about how to decide between
    Graham> encoding schemes for attachments and Lars wrote a function
    Graham> that switched between them based on efficiency.

    Robbe> Is this already in the distribution?

I assume so due to the changed behaviour.

-- 
Graham Todd        

Re: PDF's as base64 vs. Q/P encoded attachments

[Lars Magne Ingebrigtsen]

Toby Speight <Toby.Speight@streapadair.freeserve.co.uk> writes:

> BTW - Lars, did you remember to include '=' with the non-ASCII
> characters for the purposes of estimating q-p size?

Nope.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: PDF's as base64 vs. Q/P encoded attachments

[Lars Magne Ingebrigtsen]

Graham Todd <gtodd@yorku.ca> writes:

> So Gnus now tries to encode PDF's I create as Q/P (I'm pretty sure it
> didn't do this before).  The problem is that receiving PDF's as Q/P
> seems to break the MIME handling of the Windows version of Netscape
> Mail somehow so that Windows users that I correspond with couldn't
> read the attached PDF files in Acrobat.

Ack.  I was afraid of that.

What I was afraid of is this: When a user agent receives something
that is qp'd, it "knows" that it has gotten text.  So it decodes the
qp, and then does lots of "helpful" things, like (say) doing LF-CRLF
conversion, or highlighting URLs, or folding lines, or whatever.

Could we try to find out whether this is what's happening, or whether
it's Gnus' qp-ing that's buggy?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

[PATCH] [BUG] Binary attachments use wrong encoding (was Re: PDF's as base64 vs. Q/P encoded attachments)

[Kim-Minh Kaplan]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]

Graham Todd writes:

> [...] I think it was decided that if roughly 1/6 of the characters
> were non-ASCII, then base64 should be used otherwise Q/P [...] most
> PDF's would/should get encoded as base64 wouldn't they?  Anyway they
> appear ASCIIish to Gnus.

There is a bug in the function that chooses between quoted printable
and base64.  The following patch should fix it.

Kim-Minh.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Fix for binary MIME attachments --]
[-- Type: text/x-patch, Size: 865 bytes --]

--- mm-encode.el.orig	Wed Sep 22 11:31:49 1999
+++ mm-encode.el	Wed Sep 22 11:32:41 1999
@@ -126,10 +126,12 @@
 
 (defun mm-qp-or-base64 ()
   (save-excursion
-    (save-restriction
-      (narrow-to-region (point-min) (min (+ (point-min) 1000) (point-max)))
-      (goto-char (point-min))
-      (let ((8bit 0))
+    (let ((8bit 0)
+	  (start (point-min))
+	  (end (min (+ (point-min) 1000) (point-max))))
+      (narrow-to-region start end)
+      (goto-char start)
+      (save-restriction
 	(cond
 	 ((not (featurep 'mule))
 	  (while (re-search-forward "[^\x20-\x7f\r\n\t]" nil t)
@@ -141,9 +143,9 @@
 	    (unless (eobp)
 	      (forward-char 1)
 	      (incf 8bit)))))
-	(if (> (/ (* 8bit 1.0) (buffer-size)) 0.166)
+	(if (> (/ (* 8bit 1.0) (- end start)) 0.166)
 	    'base64
-	  'quoted-printable)))))
+	  'quoted-printable))))) 
 
 (provide 'mm-encode)
 

Re: [PATCH] [BUG] Binary attachments use wrong encoding (was Re: PDF's as base64 vs. Q/P encoded attachments)

[Kim-Minh Kaplan]

Hum, in the following, the 'SAVE-RESTRICTION' has to be a couple of
lines above.  Sorry.

Kim-Minh Kaplan <kmkaplan@vocatex.fr> writes:

> --- mm-encode.el.orig	Wed Sep 22 11:31:49 1999
> +++ mm-encode.el	Wed Sep 22 11:32:41 1999
> @@ -126,10 +126,12 @@
>  
>  (defun mm-qp-or-base64 ()
>    (save-excursion
> -    (save-restriction
> -      (narrow-to-region (point-min) (min (+ (point-min) 1000) (point-max)))
> -      (goto-char (point-min))
> -      (let ((8bit 0))
> +    (let ((8bit 0)
> +	  (start (point-min))
> +	  (end (min (+ (point-min) 1000) (point-max))))
> +      (narrow-to-region start end)
> +      (goto-char start)
> +      (save-restriction
>  	(cond
>  	 ((not (featurep 'mule))
>  	  (while (re-search-forward "[^\x20-\x7f\r\n\t]" nil t)
> @@ -141,9 +143,9 @@
>  	    (unless (eobp)
>  	      (forward-char 1)
>  	      (incf 8bit)))))
> -	(if (> (/ (* 8bit 1.0) (buffer-size)) 0.166)
> +	(if (> (/ (* 8bit 1.0) (- end start)) 0.166)
>  	    'base64
> -	  'quoted-printable)))))
> +	  'quoted-printable))))) 
>  
>  (provide 'mm-encode)
>