Github messages for mblaze
 help / color / mirror / Atom feed
* [ISSUE] Confusing mshow behaviour with multipart/form-data
@ 2021-08-29 16:09 Earnestly
  2021-08-29 18:07 ` leahneukirchen
                   ` (14 more replies)
  0 siblings, 15 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 16:09 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 2364 bytes --]

New issue by Earnestly on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212

Description:
Version: https://github.com/leahneukirchen/mblaze/commit/4ccf2f08c1aa8b15f31ac469edebe6c4710d74f1

For full disclosure I'm attempting to reuse `mshow` for POST data instead of email so perhaps this expected but I was hoping you may have some insight into this behaviour I'm witnessing.

Here is an example file I am working with, it included CRLFs (I'm not sure if github strips them):

```
Content-Type: multipart/form-data; boundary=------------------------55a586f81559face

--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="foo"
Content-Type: application/octet-stream

foo

--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream

bar

--------------------------55a586f81559face--
```

When in this form it appears to work as I would expect:

```
% mshow -t - < example
/dev/stdin
  1: multipart/form-data size=346
    2: application/octet-stream size=4 name="foo"
    3: application/octet-stream size=4 name="bar"
```

However if any of the content contains a newline it appears to throw `mshow` off.

```
Content-Type: multipart/form-data; boundary=------------------------55a586f81559face

--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="foo"
Content-Type: application/octet-stream

foo

baz ^ newline

--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream

bar

--------------------------55a586f81559face--
```

```
% mshow -t - < example
/dev/stdin
  1: multipart/form-data size=212 name="foo"
    2: application/octet-stream size=4 name="bar"

% mshow -x - < example
foo
bar

%head -n-0 foo bar
==> foo <==
baz ^ newline

--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream

bar

--------------------------55a586f81559face--

==> bar <==
bar
```

If I include a newline in `bar` then `mshow` would produce an empty file.

Am I doing anything wrong or is `mshow` not an appropriate for POST multipart/form-data?


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
@ 2021-08-29 18:07 ` leahneukirchen
  2021-08-29 18:08 ` leahneukirchen
                   ` (13 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-29 18:07 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 253 bytes --]

New comment by leahneukirchen on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907840837

Comment:
Please upload the payload somewhere binary-safe (e.g. an attachment here), I can't reproduce from your description.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
  2021-08-29 18:07 ` leahneukirchen
@ 2021-08-29 18:08 ` leahneukirchen
  2021-08-29 19:18 ` Earnestly
                   ` (12 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-29 18:08 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 210 bytes --]

New comment by leahneukirchen on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907841011

Comment:
Or are you getting fooled by your tty where `\r` causes carriage return?

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
  2021-08-29 18:07 ` leahneukirchen
  2021-08-29 18:08 ` leahneukirchen
@ 2021-08-29 19:18 ` Earnestly
  2021-08-29 19:22 ` Earnestly
                   ` (11 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 19:18 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 349 bytes --]

New comment by Earnestly on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907854245

Comment:
I believe 0x0 should keep the content intact: https://0x0.st/-t59.txt

I don't think I'm being fooled although it is possible; I do try to check with either `cat -A` or `sed -n l` if in doubt (or throw `od` at it).

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (2 preceding siblings ...)
  2021-08-29 19:18 ` Earnestly
@ 2021-08-29 19:22 ` Earnestly
  2021-08-29 19:36 ` Earnestly
                   ` (10 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 19:22 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 492 bytes --]

New comment by Earnestly on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907854245

Comment:
I believe 0x0 should keep the content intact: https://0x0.st/-t59.txt

The paste should be of the working example, to replicate the issue you'll only have to add a newline above or below the existing content.

I don't think I'm being fooled although it is possible; I do try to check with either `cat -A` or `sed -n l` if in doubt (or throw `od` at it).


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (3 preceding siblings ...)
  2021-08-29 19:22 ` Earnestly
@ 2021-08-29 19:36 ` Earnestly
  2021-08-29 19:36 ` Earnestly
                   ` (9 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 19:36 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 673 bytes --]

New comment by Earnestly on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907854245

Comment:
I believe 0x0 should keep the content intact: https://0x0.st/-t59.txt

The paste should be of the working example, to replicate the issue you'll only have to add a newline above or below the existing content.

I don't think I'm being fooled although it is possible; I do try to check with either `cat -A` or `sed -n l` if in doubt (or throw `od` at it).

**Oh, I forget that github can do attachments but for now when I try it tells me: `Something went really wrong, and we can’t process that file.`.  This may be my fault though.**

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (4 preceding siblings ...)
  2021-08-29 19:36 ` Earnestly
@ 2021-08-29 19:36 ` Earnestly
  2021-08-29 19:37 ` Earnestly
                   ` (8 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 19:36 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 671 bytes --]

New comment by Earnestly on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907854245

Comment:
I believe 0x0 should keep the content intact: https://0x0.st/-t59.txt

The paste should be of the working example, to replicate the issue you'll only have to add a newline above or below the existing content.

I don't think I'm being fooled although it is possible; I do try to check with either `cat -A` or `sed -n l` if in doubt (or throw `od` at it).

*Oh, I forget that github can do attachments but for now when I try it tells me: `Something went really wrong, and we can’t process that file.`.  This may be my fault though.*

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (5 preceding siblings ...)
  2021-08-29 19:36 ` Earnestly
@ 2021-08-29 19:37 ` Earnestly
  2021-08-30 11:44 ` leahneukirchen
                   ` (7 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-29 19:37 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 668 bytes --]

New comment by Earnestly on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-907854245

Comment:
I believe 0x0 should keep the content intact: https://0x0.st/-t59.txt

The paste should be of the working example, to replicate the issue you'll only have to add a newline above or below the existing content.

I don't think I'm being fooled although it is possible; I do try to check with either `cat -A` or `sed -n l` if in doubt (or throw `od` at it).

Oh, I forget that github can do attachments.  Perhaps redundant but it may last longer in the end: [foo.txt](https://github.com/leahneukirchen/mblaze/files/7073115/foo.txt)


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (6 preceding siblings ...)
  2021-08-29 19:37 ` Earnestly
@ 2021-08-30 11:44 ` leahneukirchen
  2021-08-30 11:46 ` leahneukirchen
                   ` (6 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-30 11:44 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 178 bytes --]

New comment by leahneukirchen on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908273270

Comment:
Those are the cases that work fine, tho?

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (7 preceding siblings ...)
  2021-08-30 11:44 ` leahneukirchen
@ 2021-08-30 11:46 ` leahneukirchen
  2021-08-30 11:59 ` leahneukirchen
                   ` (5 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-30 11:46 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 182 bytes --]

New comment by leahneukirchen on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908274078

Comment:
Ok, I managed to edit it to trigger the bug.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (8 preceding siblings ...)
  2021-08-30 11:46 ` leahneukirchen
@ 2021-08-30 11:59 ` leahneukirchen
  2021-08-30 16:13 ` leahneukirchen
                   ` (4 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-30 11:59 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 232 bytes --]

New comment by leahneukirchen on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908281112

Comment:
Ok, a `\n\n` triggers the header end in `blaze822_mem` mistakenly. Let me think of a solution.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (9 preceding siblings ...)
  2021-08-30 11:59 ` leahneukirchen
@ 2021-08-30 16:13 ` leahneukirchen
  2021-08-30 18:12 ` Earnestly
                   ` (3 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-30 16:13 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 150 bytes --]

New comment by leahneukirchen on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908475904

Comment:
Please test.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (10 preceding siblings ...)
  2021-08-30 16:13 ` leahneukirchen
@ 2021-08-30 18:12 ` Earnestly
  2021-08-30 18:19 ` leahneukirchen
                   ` (2 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-30 18:12 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 1419 bytes --]

New comment by Earnestly on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908571531

Comment:
With that PR it appears to work with `mshow -x`.  I get all files made with the correct content, however `mshow -O` appears to always print out the headers which happens to include the content, before ultimately printing the content, e.g.


```
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary0CUPAVuuy9FgOBzU
Content-Length: 356

------WebKitFormBoundary0CUPAVuuy9FgOBzU
Content-Disposition: form-data; name="post"; filename="bar"
Content-Type: application/octet-stream


bar

------WebKitFormBoundary0CUPAVuuy9FgOBzU
Content-Disposition: form-data; name="post"; filename="foo"
Content-Type: application/octet-stream

foo

newline

------WebKitFormBoundary0CUPAVuuy9FgOBzU--
```

Produces:

```
< out ~/mblaze/mshow -O -
------WebKitFormBoundary0CUPAVuuy9FgOBzU
Content-Disposition: form-data; name="post"; filename="bar"
Content-Type: application/octet-stream


bar

------WebKitFormBoundary0CUPAVuuy9FgOBzU
Content-Disposition: form-data; name="post"; filename="foo"
Content-Type: application/octet-stream

foo

newline

------WebKitFormBoundary0CUPAVuuy9FgOBzU--

bar
foo

newline
```

Is this intentional?  (I tried various flags listed in the manpage but didn't come across anything to change this behaviour)


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (11 preceding siblings ...)
  2021-08-30 18:12 ` Earnestly
@ 2021-08-30 18:19 ` leahneukirchen
  2021-08-30 18:24 ` Earnestly
  2021-08-30 18:24 ` [ISSUE] [CLOSED] " Earnestly
  14 siblings, 0 replies; 16+ messages in thread
From: leahneukirchen @ 2021-08-30 18:19 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 304 bytes --]

New comment by leahneukirchen on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908576323

Comment:
mshow will parse three parts: the multipart/form-data, and the two form parts. `-O` will print all of them, use `mshow -O ./file 2 3` etc to only show the form parts.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [ISSUE] [CLOSED] Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (13 preceding siblings ...)
  2021-08-30 18:24 ` Earnestly
@ 2021-08-30 18:24 ` Earnestly
  14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-30 18:24 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 2914 bytes --]

Closed issue by Earnestly on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212

Description:
Version: https://github.com/leahneukirchen/mblaze/commit/4ccf2f08c1aa8b15f31ac469edebe6c4710d74f1

For full disclosure I'm attempting to reuse `mshow` for POST data instead of email so perhaps this expected but I was hoping you may have some insight into this behaviour I'm witnessing.

Here is an example file I am working with, it included CRLFs (I'm not sure if github strips them):

```
Content-Type: multipart/form-data; boundary=------------------------55a586f81559face

--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="foo"
Content-Type: application/octet-stream

foo

--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream

bar

--------------------------55a586f81559face--
```

When in this form it appears to work as I would expect:

```
% mshow -t - < example
/dev/stdin
  1: multipart/form-data size=346
    2: application/octet-stream size=4 name="foo"
    3: application/octet-stream size=4 name="bar"
```

However if any of the content contains a newline it appears to throw `mshow` off.

```
Content-Type: multipart/form-data; boundary=------------------------55a586f81559face

--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="foo"
Content-Type: application/octet-stream

foo

baz ^ newline

--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream

bar

--------------------------55a586f81559face--
```

```
% mshow -t - < example
/dev/stdin
  1: multipart/form-data size=212 name="foo"
    2: application/octet-stream size=4 name="bar"

% mshow -x - < example
foo
bar

% head -n-0 foo bar
==> foo <==
baz ^ newline

--------------------------55a586f81559face
Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream

bar

--------------------------55a586f81559face--

==> bar <==
bar
```

If I include a newline in `bar` then `mshow` would produce an empty file.

Am I doing anything wrong or is `mshow` not an appropriate for POST multipart/form-data?

Edit:
Quick attempt with python's `email` module gets me this:

```
>>> import email
>>> f = open('example')
>>> m = email.message_from_file(f)
>>> for p in m.get_payload(): print(p)
...
Content-Disposition: form-data; name="a"; filename="foo"
Content-Type: application/octet-stream

foo

baz ^ newline

Content-Disposition: form-data; name="a"; filename="bar"
Content-Type: application/octet-stream

bar


>>> for p in m.get_payload(): print(p.get_payload(decode=True))
... 
b'foo\n\nbaz ^ newline\n'
b'bar\n\n'
```


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Confusing mshow behaviour with multipart/form-data
  2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
                   ` (12 preceding siblings ...)
  2021-08-30 18:19 ` leahneukirchen
@ 2021-08-30 18:24 ` Earnestly
  2021-08-30 18:24 ` [ISSUE] [CLOSED] " Earnestly
  14 siblings, 0 replies; 16+ messages in thread
From: Earnestly @ 2021-08-30 18:24 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 285 bytes --]

New comment by Earnestly on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/212#issuecomment-908580199

Comment:
Ah, so it does.  This now all appears to work  wonderfully.  I'll close the issue as PR #213 does fix everything from the view of my tests.

Thank you

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2021-08-30 18:24 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-29 16:09 [ISSUE] Confusing mshow behaviour with multipart/form-data Earnestly
2021-08-29 18:07 ` leahneukirchen
2021-08-29 18:08 ` leahneukirchen
2021-08-29 19:18 ` Earnestly
2021-08-29 19:22 ` Earnestly
2021-08-29 19:36 ` Earnestly
2021-08-29 19:36 ` Earnestly
2021-08-29 19:37 ` Earnestly
2021-08-30 11:44 ` leahneukirchen
2021-08-30 11:46 ` leahneukirchen
2021-08-30 11:59 ` leahneukirchen
2021-08-30 16:13 ` leahneukirchen
2021-08-30 18:12 ` Earnestly
2021-08-30 18:19 ` leahneukirchen
2021-08-30 18:24 ` Earnestly
2021-08-30 18:24 ` [ISSUE] [CLOSED] " Earnestly

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).