* zfs send/recv error
@ 2024-04-08 19:35 Gabriele Bulfon
2024-04-08 19:58 ` [developer] " Joshua M. Clulow
0 siblings, 1 reply; 4+ messages in thread
From: Gabriele Bulfon @ 2024-04-08 19:35 UTC (permalink / raw)
To: illumos-developer
[-- Attachment #1.1: Type: text/plain, Size: 882 bytes --]
Hi,
I've been doing these send/recv for years now, but can't understand what's happening here:
attempting rename databackup/sos01/zones/z000/ROOT to databackup/sos01/zones/z178/ROOT failed - trying rename databackup/sos01/zones/z000/ROOT to databackup/sos01/recv-20383-1
Why the hell it's trying to rename a dataset into another one of the same stream?!
That's basically a normal send/recv:
zfs send -R -i $snapshot_fromsnap $snapshot_tosnap | $rshell $rhost "pfexec /sonicle/bin/mbuffer -q -s 128k -W 600 -m 256M |pfexec /usr/sbin/zfs receive -Fduv $dstdataset"
I tried starting again from full send/recv two times, once I run the incremental it just throws all of these...
Any idea?!
Gabriele
Sonicle S.r.l. : http://www.sonicle.com
Music: http://www.gabrielebulfon.com
eXoplanets : https://gabrielebulfon.bandcamp.com/album/exoplanets
[-- Attachment #1.2: Type: text/html, Size: 1912 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [developer] zfs send/recv error
2024-04-08 19:35 zfs send/recv error Gabriele Bulfon
@ 2024-04-08 19:58 ` Joshua M. Clulow
2024-04-10 16:25 ` Gabriele Bulfon
0 siblings, 1 reply; 4+ messages in thread
From: Joshua M. Clulow @ 2024-04-08 19:58 UTC (permalink / raw)
To: illumos-developer
On Mon, 8 Apr 2024 at 12:35, Gabriele Bulfon via illumos-developer
<developer@lists.illumos.org> wrote:
> I've been doing these send/recv for years now, but can't understand what's happening here:
>
> attempting rename databackup/sos01/zones/z000/ROOT to databackup/sos01/zones/z178/ROOT failed - trying rename databackup/sos01/zones/z000/ROOT to databackup/sos01/recv-20383-1
>
> I tried starting again from full send/recv two times, once I run the incremental it just throws all of these...
>
> Any idea?!
I would probably look at the actual ZFS send stream contents with the
zstreamdump(8) command to see the particulars, and then look at the
code to see why it might be interpreting it that way, and if you feel
it's not correct, then look at the code in the kernel that generates
the streams. While doing that you can probably use zdb(8) to inspect
the objects in the pool as well, to see how they line up with the
observed behaviour.
Cheers.
--
Joshua M. Clulow
http://blog.sysmgr.org
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [developer] zfs send/recv error
2024-04-08 19:58 ` [developer] " Joshua M. Clulow
@ 2024-04-10 16:25 ` Gabriele Bulfon
2024-04-11 14:50 ` Gabriele Bulfon
0 siblings, 1 reply; 4+ messages in thread
From: Gabriele Bulfon @ 2024-04-10 16:25 UTC (permalink / raw)
To: illumos-developer
[-- Attachment #1.1: Type: text/plain, Size: 8208 bytes --]
I really don't understand what's happening here.
I restart from a full send receive and I get these correct replica:
databackup 887G 23.8T 104K /databackup
databackup/sos01 30.3G 23.8T 96K /databackup/sos01
databackup/sos01/zones 30.3G 23.8T 120K /databackup/sos01/zones
databackup/sos01/zones/dmbtest 22.3G 23.8T 104K /databackup/sos01/zones/dmbtest
databackup/sos01/zones/dmbtest/ROOT 3.21G 23.8T 96K legacy
databackup/sos01/zones/dmbtest/ROOT/xstreamos-2023:08:06 3.07G 23.8T 2.44G legacy
databackup/sos01/zones/dmbtest/ROOT/xstreamos-2023:08:06-backup-1 208K 23.8T 2.13G legacy
databackup/sos01/zones/dmbtest/ROOT/xstreamos-2023:08:06-backup-2 240K 23.8T 2.13G legacy
databackup/sos01/zones/dmbtest/ROOT/xstreamos-2023:08:06-backup-3 344K 23.8T 2.19G legacy
databackup/sos01/zones/dmbtest/ROOT/xstreamos-2023:08:06-backup-4 496K 23.8T 2.42G legacy
databackup/sos01/zones/dmbtest/ROOT/zbe 126M 23.8T 2.00G legacy
databackup/sos01/zones/dmbtest/ROOT/zbe-1 16.9M 23.8T 2.00G legacy
databackup/sos01/zones/dmbtest/data 2.72G 23.8T 2.72G /export/home
databackup/sos01/zones/dmbtest/dataold 16.4G 23.8T 16.4G /data/zones/dmbtest/dataold
databackup/sos01/zones/z000 882M 23.8T 104K /databackup/sos01/zones/z000
databackup/sos01/zones/z000/ROOT 862M 23.8T 96K legacy
databackup/sos01/zones/z000/ROOT/zbe 862M 23.8T 848M legacy
databackup/sos01/zones/z000/data 20.4M 23.8T 20.1M /export/home
databackup/sos01/zones/z153 919M 23.8T 104K /databackup/sos01/zones/z153
databackup/sos01/zones/z153/ROOT 898M 23.8T 96K legacy
databackup/sos01/zones/z153/ROOT/zbe 898M 23.8T 849M legacy
databackup/sos01/zones/z153/data 20.4M 23.8T 20.1M /export/home
databackup/sos01/zones/z177 5.36G 23.8T 104K /databackup/sos01/zones/z177
databackup/sos01/zones/z177/ROOT 926M 23.8T 96K legacy
databackup/sos01/zones/z177/ROOT/zbe 926M 23.8T 910M legacy
databackup/sos01/zones/z177/data 4.46G 23.8T 2.42G /export/home
databackup/sos01/zones/z178 914M 23.8T 104K /databackup/sos01/zones/z178
databackup/sos01/zones/z178/ROOT 894M 23.8T 96K legacy
databackup/sos01/zones/z178/ROOT/zbe 894M 23.8T 849M legacy
databackup/sos01/zones/z178/data 20.3M 23.8T 20.1M /export/home
Here you can see z000/data , z000/ROOT and z153/ROOT/zbe are there.
Once I start the incremental manually, I get these first errors:
cannot open 'databackup/sos01/zones/z000/data': dataset does not exist
cannot open 'databackup/sos01/zones/z000/ROOT': dataset does not exist
cannot open 'databackup/sos01/zones/z153/ROOT/zbe': dataset does not exist
and then
attempting rename databackup/sos01/zones/z000 to databackup/sos01/zones/z178
failed - trying rename databackup/sos01/zones/z000 to databackup/sos01/recv-23639-1
attempting rename databackup/sos01/zones/z153/data to databackup/sos01/zones/z178/data
failed - trying rename databackup/sos01/zones/z153/data to databackup/sos01/recv-23639-2
attempting rename databackup/sos01/zones/z153/ROOT to databackup/sos01/zones/z178/ROOT
failed - trying rename databackup/sos01/zones/z153/ROOT to databackup/sos01/recv-23639-3
then it goes on receiving the other datasets.
At the end z000 and z153 datasets are no more on destination, and I find all the renamed versions:
databackup 887G 23.8T 104K /databackup
databackup/sos01 30.3G 23.8T 96K /databackup/sos01
databackup/sos01/recv-23639-1 882M 23.8T 104K /databackup/sos01/recv-23639-1
databackup/sos01/recv-23639-1/ROOT 862M 23.8T 96K legacy
databackup/sos01/recv-23639-1/ROOT/zbe 862M 23.8T 848M legacy
databackup/sos01/recv-23639-1/data 20.4M 23.8T 20.1M /export/home
databackup/sos01/recv-23639-2 20.4M 23.8T 20.1M /export/home
databackup/sos01/recv-23639-3 898M 23.8T 96K legacy
databackup/sos01/recv-23639-3/zbe 898M 23.8T 849M legacy
Nothing happened in between the full and incremental send that could mislead zfs, no in between snapshots or new datasets. Nothing.
Any idea?
The only thing that comes to my mind: source machine is illumos / sparc, latest version possible since it was dismissed, destination machine is illumos / x86 more recent version.
May this be creating the issue? Why only during incremental and not during full send?
Gabriele
Sonicle S.r.l. : http://www.sonicle.com
Music: http://www.gabrielebulfon.com
eXoplanets : https://gabrielebulfon.bandcamp.com/album/exoplanets
----------------------------------------------------------------------------------
Da: Joshua M. Clulow via illumos-developer <developer@lists.illumos.org>
A: illumos-developer <developer@lists.illumos.org>
Data: 8 aprile 2024 21.58.11 CEST
Oggetto: Re: [developer] zfs send/recv error
On Mon, 8 Apr 2024 at 12:35, Gabriele Bulfon via illumos-developer<developer@lists.illumos.org> wrote:> I've been doing these send/recv for years now, but can't understand what's happening here:>> attempting rename databackup/sos01/zones/z000/ROOT to databackup/sos01/zones/z178/ROOT failed - trying rename databackup/sos01/zones/z000/ROOT to databackup/sos01/recv-20383-1>> I tried starting again from full send/recv two times, once I run the incremental it just throws all of these...>> Any idea?!I would probably look at the actual ZFS send stream contents with thezstreamdump(8) command to see the particulars, and then look at thecode to see why it might be interpreting it that way, and if you feelit's not correct, then look at the code in the kernel that generatesthe streams. While doing that you can probably use zdb(8) to inspectthe objects in the pool as well, to see how they line up with theobserved behaviour.Cheers.-- Joshua M. Clulowhttp://blog.sysmgr.org------------------------------------------illumos: illumos-developerPermalink: https://illumos.topicbox.com/groups/developer/Tc459a59332b0d379-Mcb3a44284f5101f9e3e01464Delivery options: https://illumos.topicbox.com/groups/developer/subscription
[-- Attachment #1.2: Type: text/html, Size: 16577 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [developer] zfs send/recv error
2024-04-10 16:25 ` Gabriele Bulfon
@ 2024-04-11 14:50 ` Gabriele Bulfon
0 siblings, 0 replies; 4+ messages in thread
From: Gabriele Bulfon @ 2024-04-11 14:50 UTC (permalink / raw)
To: illumos-developer
[-- Attachment #1.1: Type: text/plain, Size: 8905 bytes --]
I ended up modifying the script to send/recv each zone datasets one by one.
That way I have no mess.
Can't say what was happening doing it from the root pool.
Not even zstreamdump helped in any way.
Gabriele
Sonicle S.r.l. : http://www.sonicle.com
Music: http://www.gabrielebulfon.com
eXoplanets : https://gabrielebulfon.bandcamp.com/album/exoplanets
Da: Gabriele Bulfon via illumos-developer <developer@lists.illumos.org>
A: illumos-developer <developer@lists.illumos.org>
Data: 10 aprile 2024 18.25.16 CEST
Oggetto: Re: [developer] zfs send/recv error
I really don't understand what's happening here.
I restart from a full send receive and I get these correct replica:
databackup 887G 23.8T 104K /databackup
databackup/sos01 30.3G 23.8T 96K /databackup/sos01
databackup/sos01/zones 30.3G 23.8T 120K /databackup/sos01/zones
databackup/sos01/zones/dmbtest 22.3G 23.8T 104K /databackup/sos01/zones/dmbtest
databackup/sos01/zones/dmbtest/ROOT 3.21G 23.8T 96K legacy
databackup/sos01/zones/dmbtest/ROOT/xstreamos-2023:08:06 3.07G 23.8T 2.44G legacy
databackup/sos01/zones/dmbtest/ROOT/xstreamos-2023:08:06-backup-1 208K 23.8T 2.13G legacy
databackup/sos01/zones/dmbtest/ROOT/xstreamos-2023:08:06-backup-2 240K 23.8T 2.13G legacy
databackup/sos01/zones/dmbtest/ROOT/xstreamos-2023:08:06-backup-3 344K 23.8T 2.19G legacy
databackup/sos01/zones/dmbtest/ROOT/xstreamos-2023:08:06-backup-4 496K 23.8T 2.42G legacy
databackup/sos01/zones/dmbtest/ROOT/zbe 126M 23.8T 2.00G legacy
databackup/sos01/zones/dmbtest/ROOT/zbe-1 16.9M 23.8T 2.00G legacy
databackup/sos01/zones/dmbtest/data 2.72G 23.8T 2.72G /export/home
databackup/sos01/zones/dmbtest/dataold 16.4G 23.8T 16.4G /data/zones/dmbtest/dataold
databackup/sos01/zones/z000 882M 23.8T 104K /databackup/sos01/zones/z000
databackup/sos01/zones/z000/ROOT 862M 23.8T 96K legacy
databackup/sos01/zones/z000/ROOT/zbe 862M 23.8T 848M legacy
databackup/sos01/zones/z000/data 20.4M 23.8T 20.1M /export/home
databackup/sos01/zones/z153 919M 23.8T 104K /databackup/sos01/zones/z153
databackup/sos01/zones/z153/ROOT 898M 23.8T 96K legacy
databackup/sos01/zones/z153/ROOT/zbe 898M 23.8T 849M legacy
databackup/sos01/zones/z153/data 20.4M 23.8T 20.1M /export/home
databackup/sos01/zones/z177 5.36G 23.8T 104K /databackup/sos01/zones/z177
databackup/sos01/zones/z177/ROOT 926M 23.8T 96K legacy
databackup/sos01/zones/z177/ROOT/zbe 926M 23.8T 910M legacy
databackup/sos01/zones/z177/data 4.46G 23.8T 2.42G /export/home
databackup/sos01/zones/z178 914M 23.8T 104K /databackup/sos01/zones/z178
databackup/sos01/zones/z178/ROOT 894M 23.8T 96K legacy
databackup/sos01/zones/z178/ROOT/zbe 894M 23.8T 849M legacy
databackup/sos01/zones/z178/data 20.3M 23.8T 20.1M /export/home
Here you can see z000/data , z000/ROOT and z153/ROOT/zbe are there.
Once I start the incremental manually, I get these first errors:
cannot open 'databackup/sos01/zones/z000/data': dataset does not exist
cannot open 'databackup/sos01/zones/z000/ROOT': dataset does not exist
cannot open 'databackup/sos01/zones/z153/ROOT/zbe': dataset does not exist
and then
attempting rename databackup/sos01/zones/z000 to databackup/sos01/zones/z178
failed - trying rename databackup/sos01/zones/z000 to databackup/sos01/recv-23639-1
attempting rename databackup/sos01/zones/z153/data to databackup/sos01/zones/z178/data
failed - trying rename databackup/sos01/zones/z153/data to databackup/sos01/recv-23639-2
attempting rename databackup/sos01/zones/z153/ROOT to databackup/sos01/zones/z178/ROOT
failed - trying rename databackup/sos01/zones/z153/ROOT to databackup/sos01/recv-23639-3
then it goes on receiving the other datasets.
At the end z000 and z153 datasets are no more on destination, and I find all the renamed versions:
databackup 887G 23.8T 104K /databackup
databackup/sos01 30.3G 23.8T 96K /databackup/sos01
databackup/sos01/recv-23639-1 882M 23.8T 104K /databackup/sos01/recv-23639-1
databackup/sos01/recv-23639-1/ROOT 862M 23.8T 96K legacy
databackup/sos01/recv-23639-1/ROOT/zbe 862M 23.8T 848M legacy
databackup/sos01/recv-23639-1/data 20.4M 23.8T 20.1M /export/home
databackup/sos01/recv-23639-2 20.4M 23.8T 20.1M /export/home
databackup/sos01/recv-23639-3 898M 23.8T 96K legacy
databackup/sos01/recv-23639-3/zbe 898M 23.8T 849M legacy
Nothing happened in between the full and incremental send that could mislead zfs, no in between snapshots or new datasets. Nothing.
Any idea?
The only thing that comes to my mind: source machine is illumos / sparc, latest version possible since it was dismissed, destination machine is illumos / x86 more recent version.
May this be creating the issue? Why only during incremental and not during full send?
Gabriele
Sonicle S.r.l. : http://www.sonicle.com
Music: http://www.gabrielebulfon.com
eXoplanets : https://gabrielebulfon.bandcamp.com/album/exoplanets
----------------------------------------------------------------------------------
Da: Joshua M. Clulow via illumos-developer <developer@lists.illumos.org>
A: illumos-developer <developer@lists.illumos.org>
Data: 8 aprile 2024 21.58.11 CEST
Oggetto: Re: [developer] zfs send/recv error
On Mon, 8 Apr 2024 at 12:35, Gabriele Bulfon via illumos-developer<developer@lists.illumos.org> wrote:> I've been doing these send/recv for years now, but can't understand what's happening here:>> attempting rename databackup/sos01/zones/z000/ROOT to databackup/sos01/zones/z178/ROOT failed - trying rename databackup/sos01/zones/z000/ROOT to databackup/sos01/recv-20383-1>> I tried starting again from full send/recv two times, once I run the incremental it just throws all of these...>> Any idea?!I would probably look at the actual ZFS send stream contents with thezstreamdump(8) command to see the particulars, and then look at thecode to see why it might be interpreting it that way, and if you feelit's not correct, then look at the code in the kernel that generatesthe streams. While doing that you can probably use zdb(8) to inspectthe objects in the pool as well, to see how they line up with theobserved behaviour.Cheers.-- Joshua M. Clulowhttp://blog.sysmgr.org------------------------------------------illumos: illumos-developerPermalink: https://illumos.topicbox.com/groups/developer/Tc459a59332b0d379-Mcb3a44284f5101f9e3e01464Delivery options: https://illumos.topicbox.com/groups/developer/subscriptionillumos / illumos-developer / see discussions + participants + delivery options Permalink
[-- Attachment #1.2: Type: text/html, Size: 19638 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2024-04-11 14:50 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-08 19:35 zfs send/recv error Gabriele Bulfon
2024-04-08 19:58 ` [developer] " Joshua M. Clulow
2024-04-10 16:25 ` Gabriele Bulfon
2024-04-11 14:50 ` Gabriele Bulfon
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).