[9fans] dd(1) takes very long

[9fans] dd(1) takes very long

[Marco Feichtinger]

I setup a mirror, according to the instructions on the 9legacy Doc page.
Booted into fossil from disk3, installed fossil on disk1 and copied the data to disk2 with dd(1).
The mirrored disks are 480GB ssds.

The copying with dd(1) took ~16 hours. 
Is that normal?


-marco


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T0fd6444acb9c7811-M9e259dabc0f573036af7e0e3
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] dd(1) takes very long

[Aleksandar Kuktin]

[-- Attachment #1.1: Type: text/plain, Size: 925 bytes --]

>On Fri, 1 Mar 2024 09:24:15 +0100
>Marco Feichtinger <marco@germteig.com> wrote:
>
> The mirrored disks are 480GB ssds.
> 
> The copying with dd(1) took ~16 hours. 
> Is that normal?
> 
> -marco

Hi, a long-time lurker here. If 480 GiB are transferred over 16 hours,
the implied transfer rate is 70 Mbps. If 480 GB are transferred, the
implied rate is 66 Mbps. Both transfer rates are consistent with network
interfaces that are in common use (100 Mbps Ethernet). They may or may
not be consistent with internal system interfaces (SSDs are supposed to
have much greater bandwidth). If both disks live in the same computer
and the computer isn't a SBC bitty box, the transfer rate is weirdly
low.

-- 
Svi moji e-mailovi su kriptografski potpisani. Proverite ih.
All of my e-mails are cryptographically signed. Verify them.
--
You don't need an AI for a robot uprising.
Humans will do just fine.
--

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 228 bytes --]


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T0fd6444acb9c7811-M9dd4ba41d4b92276d12a1f1f
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] dd(1) takes very long

[Marco Feichtinger]

> and the computer isn't a SBC bitty box, the transfer rate is weirdly
> low.

Well, both disk are on the same machine.
It's a Supermicro X7SPA-H-D525 board.

-marco

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T0fd6444acb9c7811-M60383cb2ae1adb17499f70da
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] dd(1) takes very long

[Aleksandar Kuktin]

[-- Attachment #1.1: Type: text/plain, Size: 1077 bytes --]

>On Fri, 1 Mar 2024 10:08:25 +0100
>Marco Feichtinger <marco@germteig.com> wrote:
>
> > and the computer isn't a SBC bitty box, the transfer rate is weirdly
> > low.
> 
> Well, both disk are on the same machine.
> It's a Supermicro X7SPA-H-D525 board.
> 
> -marco

Well, that's not a bitty box. Wish I bought something like that instead
of my BananaPi. Anyway, someone more knowledgeable on Plan 9 than me is
needed. I can only speculate that the OS and hardware fight. I have
something similar happening on my desktop with modern hardware running
old software. I run GNU/Linux on it. For some reason I can't figure out,
transfers start off normal but then degrade to 10 Mbps or less after a
few GiB are transferred. If I try it with CentOS 7, it runs fine. But
when I use my own homegrown distro it's pathologic. Kernel version
3.16.85, vanilla.

-- 
Svi moji e-mailovi su kriptografski potpisani. Proverite ih.
All of my e-mails are cryptographically signed. Verify them.
--
You don't need an AI for a robot uprising.
Humans will do just fine.
--

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 228 bytes --]


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T0fd6444acb9c7811-Maaa7e17b02d8ea4da65a457e
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] dd(1) takes very long

[Lucio De Re]

Increasing the dd block size (-bs 1024k or as big as the man pages
allow) could make a big difference.


On 3/1/24, Aleksandar Kuktin <ak@triklod.rs> wrote:
>>On Fri, 1 Mar 2024 10:08:25 +0100
>>Marco Feichtinger <marco@germteig.com> wrote:
>>
>> > and the computer isn't a SBC bitty box, the transfer rate is weirdly
>> > low.
>>
>> Well, both disk are on the same machine.
>> It's a Supermicro X7SPA-H-D525 board.
>>
>> -marco
>
> Well, that's not a bitty box. Wish I bought something like that instead
> of my BananaPi. Anyway, someone more knowledgeable on Plan 9 than me is
> needed. I can only speculate that the OS and hardware fight. I have
> something similar happening on my desktop with modern hardware running
> old software. I run GNU/Linux on it. For some reason I can't figure out,
> transfers start off normal but then degrade to 10 Mbps or less after a
> few GiB are transferred. If I try it with CentOS 7, it runs fine. But
> when I use my own homegrown distro it's pathologic. Kernel version
> 3.16.85, vanilla.
>
> --
> Svi moji e-mailovi su kriptografski potpisani. Proverite ih.
> All of my e-mails are cryptographically signed. Verify them.
> --
> You don't need an AI for a robot uprising.
> Humans will do just fine.
> --
>


-- 
Lucio De Re
2 Piet Retief St
Kestell (Eastern Free State)
9860 South Africa

Ph.: +27 58 653 1433
Cell: +27 83 251 5824

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T0fd6444acb9c7811-M3511a4d8d92d12118f55f6bf
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] dd(1) takes very long

[Steve simon]

A larger block size as lucio says, and also try two dd's with a pipe between them,
one reading and one writing. dd(1) is single threaded but you have two asynchronous physical devices.

I have had good success copying sd-cards using fcp -  rsc's  multithreaded cp, already in the distribution,
though I don't know what blocksize it uses offhand.

-Steve

> On 1 Mar 2024, at 09:39, Lucio De Re <lucio.dere@gmail.com> wrote:
> 
> Increasing the dd block size (-bs 1024k or as big as the man pages
> allow) could make a big difference.
> 
> 
> On 3/1/24, Aleksandar Kuktin <ak@triklod.rs> wrote:
>>> On Fri, 1 Mar 2024 10:08:25 +0100
>>> Marco Feichtinger <marco@germteig.com> wrote:
>>> 
>>>> and the computer isn't a SBC bitty box, the transfer rate is weirdly
>>>> low.
>>> 
>>> Well, both disk are on the same machine.
>>> It's a Supermicro X7SPA-H-D525 board.
>>> 
>>> -marco
>> 
>> Well, that's not a bitty box. Wish I bought something like that instead
>> of my BananaPi. Anyway, someone more knowledgeable on Plan 9 than me is
>> needed. I can only speculate that the OS and hardware fight. I have
>> something similar happening on my desktop with modern hardware running
>> old software. I run GNU/Linux on it. For some reason I can't figure out,
>> transfers start off normal but then degrade to 10 Mbps or less after a
>> few GiB are transferred. If I try it with CentOS 7, it runs fine. But
>> when I use my own homegrown distro it's pathologic. Kernel version
>> 3.16.85, vanilla.
>> 
>> --
>> Svi moji e-mailovi su kriptografski potpisani. Proverite ih.
>> All of my e-mails are cryptographically signed. Verify them.
>> --
>> You don't need an AI for a robot uprising.
>> Humans will do just fine.
>> --
>> 
> 
> 
> --
> Lucio De Re
> 2 Piet Retief St
> Kestell (Eastern Free State)
> 9860 South Africa
> 
> Ph.: +27 58 653 1433
> Cell: +27 83 251 5824



------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T0fd6444acb9c7811-Mb5849711471b91fba2a419ff
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] dd(1) takes very long

[sirjofri]

01.03.2024 11:38:06 Steve simon <steve@quintile.net>:

> A larger block size as lucio says, and also try two dd's with a pipe between them,
> one reading and one writing. dd(1) is single threaded but you have two asynchronous physical devices.

You can probably even pipe it through some compressor, but I doubt you'll get much performance out of it in a case like this (local machine). I guess it's more useful for separate machines with slower network connections.

sirjofri

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T0fd6444acb9c7811-M08db6c6b992d9e5efc4ac1a6
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription