* [ISSUE] Confusing mshow behaviour with multipart/form-data
@ 2021-08-29 16:09 Earnestly
2021-08-29 18:07 ` leahneukirchen
` (14 more replies)
0 siblings, 15 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 16:09 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 2364 bytes --]
New issue by Earnestly on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212
Description:
Version: https://github.com/leahneukirchen/mblaze/commit/4ccf2f08c1aa8b15f31ac469edebe6c4710d74f1
For full disclosure I'm attempting to reuse `mshow` for POST data instead of email so perhaps this expected but I was hoping you may have some insight into this behaviour I'm witnessing.
Here is an example file I am working with, it included CRLFs (I'm not sure if github strips them):
```
Content-Type: multipart/form-data; boundary=------------------------55a586f81559face
--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="foo"
Content-Type: application/octet-stream
foo
--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream
bar
--------------------------55a586f81559face--
```
When in this form it appears to work as I would expect:
```
% mshow -t - < example
/dev/stdin
1: multipart/form-data size=346
2: application/octet-stream size=4 name="foo"
3: application/octet-stream size=4 name="bar"
```
However if any of the content contains a newline it appears to throw `mshow` off.
```
Content-Type: multipart/form-data; boundary=------------------------55a586f81559face
--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="foo"
Content-Type: application/octet-stream
foo
baz ^ newline
--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream
bar
--------------------------55a586f81559face--
```
```
% mshow -t - < example
/dev/stdin
1: multipart/form-data size=212 name="foo"
2: application/octet-stream size=4 name="bar"
% mshow -x - < example
foo
bar
%head -n-0 foo bar
==> foo <==
baz ^ newline
--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream
bar
--------------------------55a586f81559face--
==> bar <==
bar
```
If I include a newline in `bar` then `mshow` would produce an empty file.
Am I doing anything wrong or is `mshow` not an appropriate for POST multipart/form-data?
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
@ 2021-08-29 18:07 ` leahneukirchen
2021-08-29 18:08 ` leahneukirchen
` (13 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-29 18:07 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 253 bytes --]
New comment by leahneukirchen on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907840837
Comment:
Please upload the payload somewhere binary-safe (e.g. an attachment here), I can't reproduce from your description.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
2021-08-29 18:07 ` leahneukirchen
@ 2021-08-29 18:08 ` leahneukirchen
2021-08-29 19:18 ` Earnestly
` (12 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-29 18:08 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 210 bytes --]
New comment by leahneukirchen on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907841011
Comment:
Or are you getting fooled by your tty where `\r` causes carriage return?
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
2021-08-29 18:07 ` leahneukirchen
2021-08-29 18:08 ` leahneukirchen
@ 2021-08-29 19:18 ` Earnestly
2021-08-29 19:22 ` Earnestly
` (11 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 19:18 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 349 bytes --]
New comment by Earnestly on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907854245
Comment:
I believe 0x0 should keep the content intact: https://0x0.st/-t59.txt
I don't think I'm being fooled although it is possible; I do try to check with either `cat -A` or `sed -n l` if in doubt (or throw `od` at it).
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (2 preceding siblings ...)
2021-08-29 19:18 ` Earnestly
@ 2021-08-29 19:22 ` Earnestly
2021-08-29 19:36 ` Earnestly
` (10 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 19:22 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 492 bytes --]
New comment by Earnestly on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907854245
Comment:
I believe 0x0 should keep the content intact: https://0x0.st/-t59.txt
The paste should be of the working example, to replicate the issue you'll only have to add a newline above or below the existing content.
I don't think I'm being fooled although it is possible; I do try to check with either `cat -A` or `sed -n l` if in doubt (or throw `od` at it).
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (3 preceding siblings ...)
2021-08-29 19:22 ` Earnestly
@ 2021-08-29 19:36 ` Earnestly
2021-08-29 19:36 ` Earnestly
` (9 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 19:36 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 673 bytes --]
New comment by Earnestly on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907854245
Comment:
I believe 0x0 should keep the content intact: https://0x0.st/-t59.txt
The paste should be of the working example, to replicate the issue you'll only have to add a newline above or below the existing content.
I don't think I'm being fooled although it is possible; I do try to check with either `cat -A` or `sed -n l` if in doubt (or throw `od` at it).
**Oh, I forget that github can do attachments but for now when I try it tells me: `Something went really wrong, and we can’t process that file.`. This may be my fault though.**
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (4 preceding siblings ...)
2021-08-29 19:36 ` Earnestly
@ 2021-08-29 19:36 ` Earnestly
2021-08-29 19:37 ` Earnestly
` (8 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 19:36 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 671 bytes --]
New comment by Earnestly on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907854245
Comment:
I believe 0x0 should keep the content intact: https://0x0.st/-t59.txt
The paste should be of the working example, to replicate the issue you'll only have to add a newline above or below the existing content.
I don't think I'm being fooled although it is possible; I do try to check with either `cat -A` or `sed -n l` if in doubt (or throw `od` at it).
*Oh, I forget that github can do attachments but for now when I try it tells me: `Something went really wrong, and we can’t process that file.`. This may be my fault though.*
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (5 preceding siblings ...)
2021-08-29 19:36 ` Earnestly
@ 2021-08-29 19:37 ` Earnestly
2021-08-30 11:44 ` leahneukirchen
` (7 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 19:37 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 668 bytes --]
New comment by Earnestly on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907854245
Comment:
I believe 0x0 should keep the content intact: https://0x0.st/-t59.txt
The paste should be of the working example, to replicate the issue you'll only have to add a newline above or below the existing content.
I don't think I'm being fooled although it is possible; I do try to check with either `cat -A` or `sed -n l` if in doubt (or throw `od` at it).
Oh, I forget that github can do attachments. Perhaps redundant but it may last longer in the end: [foo.txt](https://github.com/leahneukirchen/mblaze/files/7073115/foo.txt)
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (6 preceding siblings ...)
2021-08-29 19:37 ` Earnestly
@ 2021-08-30 11:44 ` leahneukirchen
2021-08-30 11:46 ` leahneukirchen
` (6 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-30 11:44 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 178 bytes --]
New comment by leahneukirchen on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908273270
Comment:
Those are the cases that work fine, tho?
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (7 preceding siblings ...)
2021-08-30 11:44 ` leahneukirchen
@ 2021-08-30 11:46 ` leahneukirchen
2021-08-30 11:59 ` leahneukirchen
` (5 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-30 11:46 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 182 bytes --]
New comment by leahneukirchen on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908274078
Comment:
Ok, I managed to edit it to trigger the bug.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (8 preceding siblings ...)
2021-08-30 11:46 ` leahneukirchen
@ 2021-08-30 11:59 ` leahneukirchen
2021-08-30 16:13 ` leahneukirchen
` (4 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-30 11:59 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 232 bytes --]
New comment by leahneukirchen on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908281112
Comment:
Ok, a `\n\n` triggers the header end in `blaze822_mem` mistakenly. Let me think of a solution.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (9 preceding siblings ...)
2021-08-30 11:59 ` leahneukirchen
@ 2021-08-30 16:13 ` leahneukirchen
2021-08-30 18:12 ` Earnestly
` (3 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-30 16:13 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 150 bytes --]
New comment by leahneukirchen on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908475904
Comment:
Please test.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (10 preceding siblings ...)
2021-08-30 16:13 ` leahneukirchen
@ 2021-08-30 18:12 ` Earnestly
2021-08-30 18:19 ` leahneukirchen
` (2 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-30 18:12 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 1419 bytes --]
New comment by Earnestly on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908571531
Comment:
With that PR it appears to work with `mshow -x`. I get all files made with the correct content, however `mshow -O` appears to always print out the headers which happens to include the content, before ultimately printing the content, e.g.
```
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary0CUPAVuuy9FgOBzU
Content-Length: 356
------WebKitFormBoundary0CUPAVuuy9FgOBzU
Content-Disposition: form-data; name="post"; filename="bar"
Content-Type: application/octet-stream
bar
------WebKitFormBoundary0CUPAVuuy9FgOBzU
Content-Disposition: form-data; name="post"; filename="foo"
Content-Type: application/octet-stream
foo
newline
------WebKitFormBoundary0CUPAVuuy9FgOBzU--
```
Produces:
```
< out ~/mblaze/mshow -O -
------WebKitFormBoundary0CUPAVuuy9FgOBzU
Content-Disposition: form-data; name="post"; filename="bar"
Content-Type: application/octet-stream
bar
------WebKitFormBoundary0CUPAVuuy9FgOBzU
Content-Disposition: form-data; name="post"; filename="foo"
Content-Type: application/octet-stream
foo
newline
------WebKitFormBoundary0CUPAVuuy9FgOBzU--
bar
foo
newline
```
Is this intentional? (I tried various flags listed in the manpage but didn't come across anything to change this behaviour)
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (11 preceding siblings ...)
2021-08-30 18:12 ` Earnestly
@ 2021-08-30 18:19 ` leahneukirchen
2021-08-30 18:24 ` [ISSUE] [CLOSED] " Earnestly
2021-08-30 18:24 ` Earnestly
14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-30 18:19 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 304 bytes --]
New comment by leahneukirchen on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908576323
Comment:
mshow will parse three parts: the multipart/form-data, and the two form parts. `-O` will print all of them, use `mshow -O ./file 2 3` etc to only show the form parts.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [ISSUE] [CLOSED] Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (12 preceding siblings ...)
2021-08-30 18:19 ` leahneukirchen
@ 2021-08-30 18:24 ` Earnestly
2021-08-30 18:24 ` Earnestly
14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-30 18:24 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 2914 bytes --]
Closed issue by Earnestly on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212
Description:
Version: https://github.com/leahneukirchen/mblaze/commit/4ccf2f08c1aa8b15f31ac469edebe6c4710d74f1
For full disclosure I'm attempting to reuse `mshow` for POST data instead of email so perhaps this expected but I was hoping you may have some insight into this behaviour I'm witnessing.
Here is an example file I am working with, it included CRLFs (I'm not sure if github strips them):
```
Content-Type: multipart/form-data; boundary=------------------------55a586f81559face
--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="foo"
Content-Type: application/octet-stream
foo
--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream
bar
--------------------------55a586f81559face--
```
When in this form it appears to work as I would expect:
```
% mshow -t - < example
/dev/stdin
1: multipart/form-data size=346
2: application/octet-stream size=4 name="foo"
3: application/octet-stream size=4 name="bar"
```
However if any of the content contains a newline it appears to throw `mshow` off.
```
Content-Type: multipart/form-data; boundary=------------------------55a586f81559face
--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="foo"
Content-Type: application/octet-stream
foo
baz ^ newline
--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream
bar
--------------------------55a586f81559face--
```
```
% mshow -t - < example
/dev/stdin
1: multipart/form-data size=212 name="foo"
2: application/octet-stream size=4 name="bar"
% mshow -x - < example
foo
bar
% head -n-0 foo bar
==> foo <==
baz ^ newline
--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream
bar
--------------------------55a586f81559face--
==> bar <==
bar
```
If I include a newline in `bar` then `mshow` would produce an empty file.
Am I doing anything wrong or is `mshow` not an appropriate for POST multipart/form-data?
Edit:
Quick attempt with python's `email` module gets me this:
```
>>> import email
>>> f = open('example')
>>> m = email.message_from_file(f)
>>> for p in m.get_payload(): print(p)
...
Content-Disposition: form-data; name="a"; filename="foo"
Content-Type: application/octet-stream
foo
baz ^ newline
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream
bar
>>> for p in m.get_payload(): print(p.get_payload(decode=True))
...
b'foo\n\nbaz ^ newline\n'
b'bar\n\n'
```
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Confusing mshow behaviour with multipart/form-data
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
` (13 preceding siblings ...)
2021-08-30 18:24 ` [ISSUE] [CLOSED] " Earnestly
@ 2021-08-30 18:24 ` Earnestly
14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-30 18:24 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 285 bytes --]
New comment by Earnestly on mblaze repository
https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908580199
Comment:
Ah, so it does. This now all appears to work wonderfully. I'll close the issue as PR #213 does fix everything from the view of my tests.
Thank you
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2021-08-30 18:24 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
2021-08-29 18:07 ` leahneukirchen
2021-08-29 18:08 ` leahneukirchen
2021-08-29 19:18 ` Earnestly
2021-08-29 19:22 ` Earnestly
2021-08-29 19:36 ` Earnestly
2021-08-29 19:36 ` Earnestly
2021-08-29 19:37 ` Earnestly
2021-08-30 11:44 ` leahneukirchen
2021-08-30 11:46 ` leahneukirchen
2021-08-30 11:59 ` leahneukirchen
2021-08-30 16:13 ` leahneukirchen
2021-08-30 18:12 ` Earnestly
2021-08-30 18:19 ` leahneukirchen
2021-08-30 18:24 ` [ISSUE] [CLOSED] " Earnestly
2021-08-30 18:24 ` Earnestly
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).