9fans - fans of the OS Plan 9 from Bell Labs
 help / color / mirror / Atom feed
* [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
@ 2022-07-31 17:58 adr
  2022-07-31 18:08 ` hiro
                   ` (2 more replies)
  0 siblings, 3 replies; 23+ messages in thread
From: adr @ 2022-07-31 17:58 UTC (permalink / raw)
  To: 9fans

[-- Attachment #1: Type: text/plain, Size: 1668 bytes --]

https://es.aliexpress.com/item/1005003167747779.html

First time using usb/ether, so let me share my experience for other users.

This is on the last 9pi image at 9legacy.

First, I added a line with did=0x8153 to /sys/src/cmd/usb/usbd/usbdb.
Browsing the list's archives I thought that ether1=type=usb should be
added to the configuration file (cmdline.txt on the pi), but etherusb
wasn't in the link section of the pi4 kernel configuration file, so #l1
wasn't created. I added, compiled the kernel and then a dummy #l1 was
created, impossible to configure. I could use manually usb/ether and
configure the device mounted in /net, but usbd didn't detect the usb
card. Then I removed the ether parameters in cmdline.txt and etherusb
from pi4 to be sure, now usbd recognize the card and creates the
device /dev/etherU0. Other programs expect the devices mounted in
/net. I'm binding the device inside a directory in /tmp and then
binding this directory to /net. I would appreciate if someone shares
the correct way of doing this.

In summary, the sequence was:
ether1=type=usb in plan9.ini (cmdline.txt)
etherusb in kernel config
#l1 is created but useless
usbd doesn't work because etherusb
usb/ether works and the device appears on /net/
remove ether1=type=usb and etherusb from the kernel
now usbd works but the device doesn't appear on /net but on /dev

I finally made it work, but this is a mess, really.
------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-Mf7759a301a20f1a6e655ffc0
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 2815 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-07-31 17:58 [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4) adr
@ 2022-07-31 18:08 ` hiro
  2022-07-31 18:09   ` adr
  2022-07-31 21:35 ` Richard Miller
  2022-08-01  9:16 ` Richard Miller
  2 siblings, 1 reply; 23+ messages in thread
From: hiro @ 2022-07-31 18:08 UTC (permalink / raw)
  To: 9fans

"Esta página no existe :("

On 7/31/22, adr@sdf.org <adr@sdf.org> wrote:
> https://es.aliexpress.com/item/1005003167747779.html
> 
> First time using usb/ether, so let me share my experience for other users.
> 
> This is on the last 9pi image at 9legacy.
> 
> First, I added a line with did=0x8153 to /sys/src/cmd/usb/usbd/usbdb.
> Browsing the list's archives I thought that ether1=type=usb should be
> added to the configuration file (cmdline.txt on the pi), but etherusb
> wasn't in the link section of the pi4 kernel configuration file, so #l1
> wasn't created. I added, compiled the kernel and then a dummy #l1 was
> created, impossible to configure. I could use manually usb/ether and
> configure the device mounted in /net, but usbd didn't detect the usb
> card. Then I removed the ether parameters in cmdline.txt and etherusb
> from pi4 to be sure, now usbd recognize the card and creates the
> device /dev/etherU0. Other programs expect the devices mounted in
> /net. I'm binding the device inside a directory in /tmp and then
> binding this directory to /net. I would appreciate if someone shares
> the correct way of doing this.
> 
> In summary, the sequence was:
> ether1=type=usb in plan9.ini (cmdline.txt)
> etherusb in kernel config
> #l1 is created but useless
> usbd doesn't work because etherusb
> usb/ether works and the device appears on /net/
> remove ether1=type=usb and etherusb from the kernel
> now usbd works but the device doesn't appear on /net but on /dev
> 
> I finally made it work, but this is a mess, really.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M41f3c473944e9028dfd35bfa
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-07-31 18:08 ` hiro
@ 2022-07-31 18:09   ` adr
  0 siblings, 0 replies; 23+ messages in thread
From: adr @ 2022-07-31 18:09 UTC (permalink / raw)
  To: 9fans

[-- Attachment #1: Type: text/plain, Size: 354 bytes --]

Sorry hiro, try this
https://aliexpress.com/item/1005003167747779.html <https://es.aliexpress.com/item/1005003167747779.html>
------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M77a66abad7a13dbfad7f19b7
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 875 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-07-31 17:58 [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4) adr
  2022-07-31 18:08 ` hiro
@ 2022-07-31 21:35 ` Richard Miller
  2022-08-01  9:16 ` Richard Miller
  2 siblings, 0 replies; 23+ messages in thread
From: Richard Miller @ 2022-07-31 21:35 UTC (permalink / raw)
  To: 9fans

What you're trying to do should work. I've just checked an ASIX usb2 ether
dongle on a pi4 as '#l1', usbd recognises it and I can mount '#l1' on /net
and configure it. I did have to add etherusb to the kernel config and rebuild
first, as you did.

The tricky part is that the real usb ether driver is /sys/src/cmd/usb/ether,
which makes the device appear in /dev because that's the default directory
where usbd mounts itself. (Use the -m flag for a different dir if you really
want to.)

The etherusb 'device' in the kernel is just a stub, which passes packets
directly between the usb endpoint to the network interface, without having
to go back and forth to the usb driver process. Just a performance shortcut;
you can use a usb ether driver without etherusb but it's slower.

When you start a usb ether driver, it tries a "bind" command with each
of the '#l' ctl files in the kernel to see if one of them works as etherusb.
If etherusb is configured in the kernel and enabled by a kernel parameter
eg ether1=type=usb, the bind will succeed and link the '#l1' device with
the usb ether driver which did the bind. You can then mount '#l1' on /net
in the usual way and configure /net/ether1.

You should see a kernel message when the etherusb and usb ether drivers
link up, with the MAC address of the dongle. For instance I see

etherusb asix: 000606e00ae7

This should happen whether usbd starts the driver automatically or you
do it by hand with usb/ether.

Do you get any interesting error messages if you try 'usb/ether -d' ?


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-Mfe74a807004d1901910c0e32
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-07-31 17:58 [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4) adr
  2022-07-31 18:08 ` hiro
  2022-07-31 21:35 ` Richard Miller
@ 2022-08-01  9:16 ` Richard Miller
  2022-08-01 16:29   ` adr
  2022-08-01 16:43   ` adr
  2 siblings, 2 replies; 23+ messages in thread
From: Richard Miller @ 2022-08-01  9:16 UTC (permalink / raw)
  To: 9fans

> Other programs expect the devices mounted in
> /net. I'm binding the device inside a directory in /tmp and then
> binding this directory to /net. I would appreciate if someone shares
> the correct way of doing this.

Two ideas come to mind:

% mount -a /srv/usb /net
  (then all usb devices appear in /net as well as /dev)
% aux/stub -d /net/etherU0 && bind /dev/etherU0 /net/etherU0
  (for a single alias without extra clutter)

But I think anything expecting a device in /net can be given an explicit
alternative, eg

% ip/ipconfig ether /dev/etherU0 [...params]

Are there exceptions I'm missing?

Note that we can even boot with a fileserver connection via usb ether
by passing extra ipconfig args in cmdline.txt eg
  bootargs='tcp ether /dev/etherU0'
(same thing with plan9.ini on other platforms, without the quotes)


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M64be1cd32d8d81cd3dfcf1f0
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-01  9:16 ` Richard Miller
@ 2022-08-01 16:29   ` adr
  2022-08-02  8:50     ` Richard Miller
  2022-08-01 16:43   ` adr
  1 sibling, 1 reply; 23+ messages in thread
From: adr @ 2022-08-01 16:29 UTC (permalink / raw)
  To: 9fans

[-- Attachment #1: Type: text/plain, Size: 1415 bytes --]

I've been experimenting again. With etherusb in the kernel, usbd is
triggering usb/ether and the device is mounted at /dev. I don't know
what happened before.

But there is no etherusb device.

; echo noauto ether > /dev/usbdctl
(unplug and plug again the device)
; sed /ether/q /dev/usb/ctl
...
ep9.0 enabled control rw speed super maxpkt 512 pollival 0 samplesz 0 hz 0 hub 0 port 1 rootport 2 addr 7 idle
255 csp 0x00ffff csp 0x000602 csp 0x00000a vid 0x0bda did 0x8153 Realtek 'USB 10/100/1000 LAN' xhci
; usb/ether -ddd /dev/usb/ep9.0
[...]
ether: ep in /dev/usb/ep9.1 maxpkt 1024; ep out /dev/usb/ep9.2 maxpkt 1024
usb/ether: etherusb bind #l0: bad process or channel control request
usb/ether: etherusb bind #l1: bad maxpkt "bind cdc #u/usb/ep9.1/data #u/usb/ep9.2/data 00e04c680cfb 2000 1024"
usb/ether: etherusb bind #l2: no free devices
usb/ether: 1 buffers
usb/ether: dev ref 3
usb/ether: etherbread
usb/ether: fsadd etherU9

So bad maxpkt... etherusb check for maxpkt < 512 which is the max for
bulk transfers in usb2 high speed. Using 1024 instead fixes the problem.

Should it chek for the speed of the usb device and set the limit accordingly?


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M829ca30175f489f095e58ae9
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 2416 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-01  9:16 ` Richard Miller
  2022-08-01 16:29   ` adr
@ 2022-08-01 16:43   ` adr
  2022-08-01 17:06     ` adr
  2022-08-02  9:07     ` Richard Miller
  1 sibling, 2 replies; 23+ messages in thread
From: adr @ 2022-08-01 16:43 UTC (permalink / raw)
  To: 9fans

[-- Attachment #1: Type: text/plain, Size: 488 bytes --]

> % aux/stub -d /net/etherU0 && bind /dev/etherU0 /net/etherU0
>   (for a single alias without extra clutter)

Thanks Richard, I didn't know about stub.

Shouln't the usb type be documented in the ethernet section of plan9ini(8), and noauto in usbd(4)?

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M74e6d499395c257d841cfecf
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 1125 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-01 16:43   ` adr
@ 2022-08-01 17:06     ` adr
  2022-08-02  3:45       ` Lucio De Re
  2022-08-02  9:07     ` Richard Miller
  1 sibling, 1 reply; 23+ messages in thread
From: adr @ 2022-08-01 17:06 UTC (permalink / raw)
  To: 9fans

>> % aux/stub -d /net/etherU0 && bind /dev/etherU0 /net/etherU0
>>   (for a single alias without extra clutter)
>
> Thanks Richard, I didn't know about stub.

Oh, and thanks for trying with your pi, I know it takes time and
I appreaciate it.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-Mb495cd40eae04d7f02c3e15a
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-01 17:06     ` adr
@ 2022-08-02  3:45       ` Lucio De Re
  0 siblings, 0 replies; 23+ messages in thread
From: Lucio De Re @ 2022-08-02  3:45 UTC (permalink / raw)
  To: 9fans

On 8/1/22, adr <adr@sdf.org> wrote:
>
> Oh, and thanks for trying with your pi, I know it takes time and
> I appreaciate it.
>
Well, let me say thanks to you for instigating some interesting and
seemingly fruitful discussion - a rare gem and a precious one.

Lucio.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M93e94100d9b9a8283c7222c2
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-01 16:29   ` adr
@ 2022-08-02  8:50     ` Richard Miller
  2022-08-03  0:34       ` adr
  0 siblings, 1 reply; 23+ messages in thread
From: Richard Miller @ 2022-08-02  8:50 UTC (permalink / raw)
  To: 9fans

> So bad maxpkt... etherusb check for maxpkt < 512 which is the max for
> bulk transfers in usb2 high speed. Using 1024 instead fixes the problem.

Thanks, the code was correct when written but the world has moved on!
Is your adapter transmitting packets successfully now?

> Should it chek for the speed of the usb device and set the limit accordingly?

Actually the limit on maxpkt in that context wasn't strictly necessary anyway.
Just remove the check.

diff /n/dump/2022/0802/sys/src/9/bcm/etherusb.c /sys/src/9/bcm/etherusb.c
328c328
<       if(maxpkt < 8 || maxpkt > 512)
---
>       if(maxpkt < 8)

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M7a798f1d29caac2cbe00034f
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-01 16:43   ` adr
  2022-08-01 17:06     ` adr
@ 2022-08-02  9:07     ` Richard Miller
  2022-08-03  0:52       ` adr
  1 sibling, 1 reply; 23+ messages in thread
From: Richard Miller @ 2022-08-02  9:07 UTC (permalink / raw)
  To: 9fans

> Shouln't the usb type be documented in the ethernet section of plan9ini(8),

It should be documented somewhere, but at present it's only implemented
for the bcm kernel, and raspberry pi doesn't have a plan9.ini. (It sould be
possible to move etherusb.c to /sys/src/9/port and use it on other platforms
too. But it's a bit of an ugly hack.)

Until rpi4, the built-in ethernet adapter on the raspberry pi was via usb,
so 'ether0=type=usb' was the default and didn't need stating explicitly.
The rpi4 has proper (non-usb) ethernet, so etherusb shouldn't be needed there
except for niche situations (like a network appliance with multiple ethers?).

> and noauto in usbd(4)?

Yes it should.


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M73cd87940d3916d92640771c
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-02  8:50     ` Richard Miller
@ 2022-08-03  0:34       ` adr
  2022-08-03  3:00         ` adr
  0 siblings, 1 reply; 23+ messages in thread
From: adr @ 2022-08-03  0:34 UTC (permalink / raw)
  To: 9fans

On Tue, 2 Aug 2022, Richard Miller wrote:
>> So bad maxpkt... etherusb check for maxpkt < 512 which is the max for
>> bulk transfers in usb2 high speed. Using 1024 instead fixes the problem.
>
> Thanks, the code was correct when written but the world has moved on!
> Is your adapter transmitting packets successfully now?

It's working really well. I'm using David du Colombier's nat patch
(I'll post to the list another one without the il stuff in case
someone wants to try it) to share internet to a linux machine, so
I made a test from the browser (speedtest.net) and almost reached
the contracted limit of my fiber connection, although I only have
a crappy 100Mb plan (I don't really need more). So packets are
flowing, but when plan9 is more involved things get worse.

For example with this setup (a small arm linux machine connected
through plan9, using the usb adapter for internet):

linux:
$ wget -O /dev/null https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2
[...]
/dev/null           100%[===================>] 489.88M  10.5MB/s    in 51s

2022-08-02 22:56:44 (9.58 MB/s) - ???/dev/null??? saved [513671168/513671168]


real    0m51.875s
user    0m8.594s
sys     0m5.379s

And the progress bar saw me several times the speed reaching my
limit. So the usb and ethernet drivers seem to be working pretty
good (at least for my bandwidth)

plan9:
; time hget -o /dev/null https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2
couldn't set mtime: permission denied
8.06u 35.25s 211.17r     hget -o /dev/null https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2

See the difference? That's a little more than 19Mb/s, some 2MiB/s almost 50% slower.

By the way I don't use archlinux but I've always use their servers
to make speed tests, usually they are fast.

> Actually the limit on maxpkt in that context wasn't strictly necessary anyway.
> Just remove the check.

Yes, I thought the same.

adr

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M99ba4fa5f9a9f29941fe318d
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-02  9:07     ` Richard Miller
@ 2022-08-03  0:52       ` adr
  2022-08-03  1:08         ` adr
  0 siblings, 1 reply; 23+ messages in thread
From: adr @ 2022-08-03  0:52 UTC (permalink / raw)
  To: 9fans

On Tue, 2 Aug 2022, Richard Miller wrote:
> It should be documented somewhere, but at present it's only implemented
> for the bcm kernel, and raspberry pi doesn't have a plan9.ini.

I think that cmdline.txt should be described in plan9.ini(8)

> (It should be possible to move etherusb.c to /sys/src/9/port and use it on other platforms
> too. But it's a bit of an ugly hack.)

The performance of the usb/ether driver without etherusb is much,
much worse. I think is a good idea to move it to port. Those devices
are very common nowadays, and can save your day if your ethernet
card breaks, or if your new computer has incompatible drivers, etc.

What I would love to see is the kernel creating the '#l' device
when a usb ethernet card is attached, and destroy it when the card
is unplugged. But that's the dream of an ignorant, that would already
been done if it were feasible, I suppose.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M7a30ac10a19a0a86b37f471e
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-03  0:52       ` adr
@ 2022-08-03  1:08         ` adr
  2022-08-03 12:41           ` Richard Miller
  0 siblings, 1 reply; 23+ messages in thread
From: adr @ 2022-08-03  1:08 UTC (permalink / raw)
  To: 9fans

By the way Richard, should I take the last rpi image at 9legacy as
the more recent with your work? Are you going to abandon
https://plan9.io/sources/contrib/miller/?

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M406ab1ba9e7ee9fe37676b4a
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-03  0:34       ` adr
@ 2022-08-03  3:00         ` adr
  2022-08-03 10:57           ` Richard Miller
  0 siblings, 1 reply; 23+ messages in thread
From: adr @ 2022-08-03  3:00 UTC (permalink / raw)
  To: 9fans

On Wed, 3 Aug 2022, adr wrote:
> linux:
> $ wget -O /dev/null 
> https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2
> [...]
> /dev/null           100%[===================>] 489.88M  10.5MB/s    in 51s
>
> 2022-08-02 22:56:44 (9.58 MB/s) - ???/dev/null??? saved [513671168/513671168]
>
>
> real    0m51.875s
> user    0m8.594s
> sys     0m5.379s
>
> And the progress bar saw me several times the speed reaching my
> limit. So the usb and ethernet drivers seem to be working pretty
> good (at least for my bandwidth)
>
> plan9:
> ; time hget -o /dev/null 
> https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2
> couldn't set mtime: permission denied
> 8.06u 35.25s 211.17r   hget -o /dev/null 
> https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2
>
> See the difference? That's a little more than 19Mb/s, some 2MiB/s almost 50% 
> slower.

That was a typo, is almost five times slower (~80% slower). Just
to be clear, it is really worst!

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M34d187a3f75731aef4861b40
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-03  3:00         ` adr
@ 2022-08-03 10:57           ` Richard Miller
  2022-08-03 18:46             ` adr
  0 siblings, 1 reply; 23+ messages in thread
From: Richard Miller @ 2022-08-03 10:57 UTC (permalink / raw)
  To: 9fans

>> See the difference? That's a little more than 19Mb/s, some 2MiB/s almost 50% 
>> slower.
> 
> That was a typo, is almost five times slower (~80% slower). Just
> to be clear, it is really worst!

Unless your "small arm linux machine" is a raspberry pi, you are changing
too many variables to make a meaningful comparison.  Benchmarking i/o across
the internet will also introduce enough variation to suggest an experimental
sample size of more than one attempt.

An interesting set of experiments might be to run both linux and plan 9 on
the pi4, try the built-in ether adapter and the usb dongle, and try fetching
both from http:// and https:// (to see how much of the variation is due to
tls decode speed).


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M421b620beeed0f3199881e28
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-03  1:08         ` adr
@ 2022-08-03 12:41           ` Richard Miller
  2022-08-03 18:47             ` adr
  0 siblings, 1 reply; 23+ messages in thread
From: Richard Miller @ 2022-08-03 12:41 UTC (permalink / raw)
  To: 9fans

> By the way Richard, should I take the last rpi image at 9legacy as
> the more recent with your work? Are you going to abandon
> https://plan9.io/sources/contrib/miller/?

9legacy should always have up to date patches for the Raspberry Pi, and
the 9legacy SD card image will usually be fresher than the 9pi.img.gz
image in contrib/miller. The only value now in the 9pi image is for
historians: I've tried to conserve the metadata so it's possible to see
when source files were most recently changed. On 9legacy, patches are
re-applied when a distribution image is made, so mtime information is
less informative.

I will also continue to maintain the Pi kernel source in contrib/miller/9/bcm,
where a more complete history can be seen:

term% srv -n 9p.io sources
post...
term% history /n/sources/contrib/miller/9/bcm/etherusb.c
Aug  3 13:16:49 BST 2022 /n/sources/contrib/miller/9/bcm/etherusb.c 8872 [miller]
Jul 18 13:26:35 BST 2019 /n/sourcesdump/2022/0803/contrib/miller/9/bcm/etherusb.c 8888 [miller]
Apr  4 17:04:17 BST 2018 /n/sourcesdump/2019/0718/contrib/miller/9/bcm/etherusb.c 8837 [bootes]
Mar  4 21:18:17 GMT 2016 /n/sourcesdump/2018/0404/contrib/miller/9/bcm/etherusb.c 7878 [bootes]
Jul  2 09:00:28 BST 2015 /n/sourcesdump/2016/0304/contrib/miller/9/bcm/etherusb.c 7838 [bootes]


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M67bb6396df945caf4d9c54df
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-03 10:57           ` Richard Miller
@ 2022-08-03 18:46             ` adr
  2022-08-04 10:13               ` Richard Miller
  0 siblings, 1 reply; 23+ messages in thread
From: adr @ 2022-08-03 18:46 UTC (permalink / raw)
  To: 9fans

On Wed, 3 Aug 2022, Richard Miller wrote:
>> That was a typo, is almost five times slower (~80% slower). Just
>> to be clear, it is really worst!
>
> Unless your "small arm linux machine" is a raspberry pi, you are changing
> too many variables to make a meaningful comparison.  Benchmarking i/o across
> the internet will also introduce enough variation to suggest an experimental
> sample size of more than one attempt.
>
> An interesting set of experiments might be to run both linux and plan 9 on
> the pi4, try the built-in ether adapter and the usb dongle, and try fetching
> both from http:// and https:// (to see how much of the variation is due to
> tls decode speed).

Reading my mail it seems like this was something I suddenly was
aware of. It is not, I've been trying to understand what is wrong
for some time now. The pi4 saturates my fiber in linux, there is
no problem with the hardware. But note that the linux machine is
sending and receiving packages through the plan9 machine. If there
was a hardware problem in the pi4 (or the usb adapter), the linux
machine's connection would be affected.

And yes, I have made tests with different sites. As I said before
the archlinux servers usually let me saturate my bandwidth, so they
have become a good target for me.

The thing this setting is telling me is that the usb and ethernet
drivers are working reasonably good (again, at least for my
bandwidth).

I don't know any reliable server with good bandwidth serving without
tls, I could set an http server on the linux machine and make tests,
but I modified the example at

https://golangdocs.com/golang-download-files

and there is no real difference between the native and go tls implementations:

; time ./gget -o /dev/null https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2
Downloaded a file /dev/null with size 513671168
2.25u 1.45s 65.51r       ./gget -o /dev/null https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2

; time hget -o /dev/null https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2
couldn't set mtime: permission denied
9.27u 35.02s 64.62r      hget -o /dev/null https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2

With linux (again, through the plan9 machine) I get:
$ time ./gget -o /dev/null https://arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2
Downloaded a file /dev/null with size 513671168

real    0m48.624s
user    0m14.265s
sys     0m21.648s

So now It is only about 25% slower in plan9, so yes, the factors are variable.

I found this thread in the list's archives:

https://marc.info/?t=145579525000007&r=1&w=2

Maybe is related.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-Mf2898ed1a403f2159dda8d9a
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-03 12:41           ` Richard Miller
@ 2022-08-03 18:47             ` adr
  0 siblings, 0 replies; 23+ messages in thread
From: adr @ 2022-08-03 18:47 UTC (permalink / raw)
  To: 9fans

On Wed, 3 Aug 2022, Richard Miller wrote:
> 9legacy should always have up to date patches for the Raspberry Pi, and
> the 9legacy SD card image will usually be fresher than the 9pi.img.gz
> image in contrib/miller. The only value now in the 9pi image is for
> historians: I've tried to conserve the metadata so it's possible to see
> when source files were most recently changed. On 9legacy, patches are
> re-applied when a distribution image is made, so mtime information is
> less informative.
>
> I will also continue to maintain the Pi kernel source in contrib/miller/9/bcm,
> where a more complete history can be seen:

Ok, thanks Richard.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M92fb37694c2857c72317a3f2
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-03 18:46             ` adr
@ 2022-08-04 10:13               ` Richard Miller
  2022-08-04 11:56                 ` ori
  2022-08-04 19:34                 ` adr
  0 siblings, 2 replies; 23+ messages in thread
From: Richard Miller @ 2022-08-04 10:13 UTC (permalink / raw)
  To: 9fans

> I don't know any reliable server with good bandwidth serving without tls

I am able to connect to your example arch.mirror.constant.com using
both http and https.

Something is going on with usb ethernet and tls which I don't understand.
Could it be as simple as different block sizes interacting with the usb
packet size?

I modified hget -v option to print the number of reads in each second, as
well as the original bytes-so-far and bytes-total. My internet wire speed
is about 40 megabit/sec.

Using the pi4 built-in ethernet, I see this:

term% url=//arch.mirror.constant.com/images/v20220801.71902/Arch-Linux-x86_64-basic-20220801.71902.qcow2term% 5.hget -v -o /dev/null http:$url
1 1074 513671168
154 236298 513671168
1891 4664898 513671168
1885 9351954 513671168
1952 14040462 513671168
1948 18715902 513671168
1908 23411670 513671168
2035 28095822 513671168
1992 32781426 513671168
1995 37459770 513671168
1945 42146826 513671168
...
term% 5.hget -v -o /dev/null https:$url
1 1670 513671168
136 1113734 513671168
571 5791366 513671168
569 10452614 513671168
571 15130246 513671168
568 19783302 513671168
570 24452742 513671168
569 29113990 513671168
573 33808006 513671168
568 38461062 513671168
542 42901126 513671168
...

Using a 100Mb/s usb2 ether adapter on the same pi4, I see this:

term% 5.hget -v -o /dev/null http:$url
1 1074 513671168
662 1261410 513671168
2129 5896194 513671168
2121 10525170 513671168
2259 15167214 513671168
2212 19804902 513671168
2223 24442590 513671168
2195 29112222 513671168
2175 33800730 513671168
2197 38479074 513671168
2244 43164678 513671168
...
term% 5.hget -v -o /dev/null https:$url
1 1670 513671168
75 614022 513671168
132 1695366 513671168
2 1711750 513671168
4 1744518 513671168
128 2793094 513671168
10 2875014 513671168
24 3071622 513671168
98 3874438 513671168
4 3907206 513671168
130 4972166 513671168
...

Needs deeper investigation.


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-Mbdc3913dcd77a8be18193e66
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-04 10:13               ` Richard Miller
@ 2022-08-04 11:56                 ` ori
  2022-08-04 19:34                 ` adr
  1 sibling, 0 replies; 23+ messages in thread
From: ori @ 2022-08-04 11:56 UTC (permalink / raw)
  To: 9fans

Quoth Richard Miller <9fans@hamnavoe.com>:
> I am able to connect to your example arch.mirror.constant.com using
> both http and https.

Same.

> Something is going on with usb ethernet and tls which I don't understand.
> Could it be as simple as different block sizes interacting with the usb
> packet size?
> 
> I modified hget -v option to print the number of reads in each second, as
> well as the original bytes-so-far and bytes-total. My internet wire speed
> is about 40 megabit/sec.

I'm not certain that it's only USB ethernet.

On my (gigabit) ethernet, my CPU server is about 15% the speed of
my Linux work machine, averaging something like 15 MiB/second.
But with different URLs (I picked some OpenBSD mirrors), the results
vary wildly:

        cpu% hget http:$url | tput
        10.98 MB/s
        20.70 MB/s
        17.07 MB/s
        15.59 MB/s
        <snip>
        10.70 MB/s
        10.77 MB/s
        <snip>
        15.17 MB/s
        15.26 MB/s

        $ curl -o /dev/null http:$url                                                                               <
          % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                         Dload  Upload   Total   Spent    Left  Speed
          0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
        100  489M  100  489M    0     0   105M      0  0:00:04  0:00:04 --:--:--  105M

Compare this one, where 9front beats Linux handily:

        cpu% url=//ftp4.usa.openbsd.org/pub/OpenBSD/7.1/amd64/install71.img
        cpu% hget http:$url | tput
        60.20 MB/s
        71.13 MB/s
        70.77 MB/s
        71.21 MB/s
        69.97 MB/s
        70.15 MB/s
        72.19 MB/s
        73.59 MB/s

        $ curl -o /dev/null http:$url 
          % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                         Dload  Upload   Total   Spent    Left  Speed
        100  664M  100  664M    0     0  24.7M      0  0:00:26  0:00:26 --:--:-- 25.1M

And:

        % hget http:$url | tput
        1.26 MB/s
        1.54 MB/s
        1.54 MB/s
        1.45 MB/s
        <snip>
        3.53 MB/s


        $ curl -o /dev/null http:$url                               
          % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                         Dload  Upload   Total   Spent    Left  Speed
         27  664M   27  183M    0     0  22.7M      0  0:00:29  0:00:08  0:00:21 23.4M

and:

        % hget http:$url | tput
        33.00 MB/s
        37.52 MB/s
        36.93 MB/s
        <snip>
        38.97 MB/s
        39.09 MB/s

        $ curl -o /dev/null http:$url                               
          % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                         Dload  Upload   Total   Spent    Left  Speed
        100  664M  100  664M    0     0  90.0M      0  0:00:07  0:00:07 --:--:-- 93.7M


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M6384a056753c4f907745c22c
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-04 10:13               ` Richard Miller
  2022-08-04 11:56                 ` ori
@ 2022-08-04 19:34                 ` adr
  2022-08-05 12:14                   ` ori
  1 sibling, 1 reply; 23+ messages in thread
From: adr @ 2022-08-04 19:34 UTC (permalink / raw)
  To: 9fans

On Thu, 4 Aug 2022, Richard Miller wrote:
>> I don't know any reliable server with good bandwidth serving without tls
>
> I am able to connect to your example arch.mirror.constant.com using
> both http and https.

And now I feel like an idiot! The thing is that other tests I've
made (I'm talking about a long, long time ago) with archlinux's
mirrors allways redirected me to an htpps server, silly of me for
not trying again.

But I don't think it is the usb ethernet or the tls implementation,
I've tryed before with the internal interface and ori's experience
is similar.  I've tryed with other user agent strings, and with
the user agent string hget is using in linux's wget, but that is
not the issue. Maybe is the tcp implementation?

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M353e1347db2203b819e9e672
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4)
  2022-08-04 19:34                 ` adr
@ 2022-08-05 12:14                   ` ori
  0 siblings, 0 replies; 23+ messages in thread
From: ori @ 2022-08-05 12:14 UTC (permalink / raw)
  To: 9fans

Quoth adr <adr@SDF.ORG>:
> On Thu, 4 Aug 2022, Richard Miller wrote:
> >> I don't know any reliable server with good bandwidth serving without tls
> >
> > I am able to connect to your example arch.mirror.constant.com using
> > both http and https.
> 
> And now I feel like an idiot! The thing is that other tests I've
> made (I'm talking about a long, long time ago) with archlinux's
> mirrors allways redirected me to an htpps server, silly of me for
> not trying again.
> 
> But I don't think it is the usb ethernet or the tls implementation,
> I've tryed before with the internal interface and ori's experience
> is similar.  I've tryed with other user agent strings, and with
> the user agent string hget is using in linux's wget, but that is
> not the issue. Maybe is the tcp implementation?

Is it easy enough to write something native which is compatible
with iperf? that may be a good starting point, and a good target
for an initial round of optimizations - even though it's not the
most real world thing out there.


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T668643d11149fab4-M2227ce2573f5460e301c5dff
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2022-08-05 12:15 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-31 17:58 [9fans] USB3 1Gb ethernet card working on 9legacy (rpi 4) adr
2022-07-31 18:08 ` hiro
2022-07-31 18:09   ` adr
2022-07-31 21:35 ` Richard Miller
2022-08-01  9:16 ` Richard Miller
2022-08-01 16:29   ` adr
2022-08-02  8:50     ` Richard Miller
2022-08-03  0:34       ` adr
2022-08-03  3:00         ` adr
2022-08-03 10:57           ` Richard Miller
2022-08-03 18:46             ` adr
2022-08-04 10:13               ` Richard Miller
2022-08-04 11:56                 ` ori
2022-08-04 19:34                 ` adr
2022-08-05 12:14                   ` ori
2022-08-01 16:43   ` adr
2022-08-01 17:06     ` adr
2022-08-02  3:45       ` Lucio De Re
2022-08-02  9:07     ` Richard Miller
2022-08-03  0:52       ` adr
2022-08-03  1:08         ` adr
2022-08-03 12:41           ` Richard Miller
2022-08-03 18:47             ` adr

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).