Help wanted tracking down a bug in divergefs code

Help wanted tracking down a bug in divergefs code

[Trevor Higgins]

I have been using divergefs (from the contrib directory) and everything 
looked fine. I have found a problem which I don't know how I track down 
the cause.

I was using it to rebuild the system but diverting all new files to a 
ramdisk. The problem showed up when using mkfs to make the rootfs.paq file.

I have traced through the code and ascertained why it causes an error 
but I don't know how to further debug the system as the problem may be 
started back in libdisk with the rdproto call.

When mkfs trys to rename the temporary file after coping it, it gets and 
error from the  dirfwstat call. Depending on what is the underlying FS 
serving the file the message is something like

         wstat -- attempt to change qid.type

         wstat can't convert between files and directories

The reason for this, is the the actual Dir * entry that gets passed down 
to the FS:wstat call is the entry for the parent directory   and not the 
entry for the file descriptor passed to the dirfwstat call (from mkfs).

I am not familiar enough with the mapping of QID->FID et all to sort out 
where the underlying problem occurs.

If I change the dirfwstat to a dirwstat using the file name, mkfs will 
run (very slowly) and change the file name and modes.

I am wondering if the bug is not somuch in divergefs, but in how the dir 
entries are constructed prior to calling the 9p wstat call.

Could it be stale data in the library functions or is it more likely a 
referencing error in divergefs code.

Any way, any ideas on where I can put more fprints to track down which 
bit of code is in the wrong?



-- 
We need another plan

Re: [9front] Help wanted tracking down a bug in divergefs code

[ori]

> Any way, any ideas on where I can put more fprints to track down which 
> bit of code is in the wrong?

The first step: find as small a repro for this bug as you can. And then
post it (ideally as a script) so that someeone else can test it.

Re: [9front] Help wanted tracking down a bug in divergefs code

[kvik]

Unfortunately I can't help you with divergefs problem; I am
not familiar with the code and I don't feel like dissecting it
at the time.

If I may instead shamelessly plug unionfs(4) for the task at hand:
	git://code.a-b.xyz/unionfs	
or
	http://code.a-b.xyz/unionfs/snapshot/unionfs-master.tar.gz

          ; ramfs
          ; bind /sys/src /mnt/src
          ; unionfs -m /sys/src /tmp /mnt/src
          ; @{cd /sys/src; mk install}

Re: [9front] Help wanted tracking down a bug in divergefs code

[raingloom]

Oh dang, I've been looking for this. Also struggling with divergefs 
weirdness, but this seems so much simpler.

On Mon, Feb 17, 2020 at 16:44, kvik@a-b.xyz wrote:
> Unfortunately I can't help you with divergefs problem; I am
> not familiar with the code and I don't feel like dissecting it
> at the time.
> 
> If I may instead shamelessly plug unionfs(4) for the task at hand:
> 	git://code.a-b.xyz/unionfs
> or
> 	http://code.a-b.xyz/unionfs/snapshot/unionfs-master.tar.gz
> 
>           ; ramfs
>           ; bind /sys/src /mnt/src
>           ; unionfs -m /sys/src /tmp /mnt/src
>           ; @{cd /sys/src; mk install}

Re: [9front] Help wanted tracking down a bug in divergefs code

[raingloom]

Since I'm also struggling with some error, I'll put this here. Maybe 
it'll be helpful to someone.

https://git.sr.ht/~raingloom/divergefs

Anyone else has patches / forks of it?

The bugs I'm working on are documented in the TODO file of 
https://git.sr.ht/~raingloom/package

(sorry for the top reply, my other mailing client borked and this one 
doesn't let me bottom reply)

On Mon, Feb 17, 2020 at 21:03, raingloom <raingloom@riseup.net> wrote:
> Oh dang, I've been looking for this. Also struggling with divergefs 
> weirdness, but this seems so much simpler.
> 
> On Mon, Feb 17, 2020 at 16:44, kvik@a-b.xyz wrote:
>> Unfortunately I can't help you with divergefs problem; I am
>> not familiar with the code and I don't feel like dissecting it
>> at the time.
>> 
>> If I may instead shamelessly plug unionfs(4) for the task at hand:
>> 	git://code.a-b.xyz/unionfs
>> or
>> 	http://code.a-b.xyz/unionfs/snapshot/unionfs-master.tar.gz
>> 
>>           ; ramfs
>>           ; bind /sys/src /mnt/src
>>           ; unionfs -m /sys/src /tmp /mnt/src
>>           ; @{cd /sys/src; mk install}
> 
> 

Re: [9front] Help wanted tracking down a bug in divergefs code [Demonstration code]

[Trevor Higgins]

[-- Attachment #1: Type: text/plain, Size: 459 bytes --]

On 02/17/2020 06:59 PM, ori@eigenstate.org wrote:
>> Any way, any ideas on where I can put more fprints to track down which
>> bit of code is in the wrong?
> The first step: find as small a repro for this bug as you can. And then
> post it (ideally as a script) so that someeone else can test it.
>
I have attached small tar file of test, as I was testing using a config 
file, i have stripped down the test config to bare minimum


-- 
We need another plan


[-- Attachment #2: demo-bug-mkfs+divergefs.tgz --]
[-- Type: application/x-compressed-tar, Size: 872 bytes --]

Re: [9front] Help wanted tracking down a bug in divergefs code [unionfs]

[Trevor Higgins]

Thanks kvik,

Unionfs is definately a keeper. I see why but personally I don't like 
the intermediate mount point requirement.

Unfortunately it will not replace divergefs in my use case. I am not 
saying this is the best approach but I divergefs over the root 
namespace, this precludes any intermediate mount point, as it would 
create a loop.

I have an aversion to historic and obsolete crap and don't appear to 
have any sentimental  attachment  (although I own two pdp11's) to the 
the origins of p9 files and programs.

In keeping with the above, I have moved every file and directory I am 
not partial to into an /attic directory , as well as moving some dirs 
from / into /arch and other changes.

However, I still want to pull changes from 9front.org and I want the 
least hassles and merge conflicts.

This is were divergefs comes in handy. It presents to sysupdate a layout 
of the namespace consistant with the mercurial repository.

The ability to write back to the diverged directory is paramount to its 
success.

If  for example, one of the fortune files is updated (like /lib/theo et 
al), then the changes will be written back to /attic/lib/... and not 
recreated in /lib. So updates don't recruft my namespace but are still 
kept current so I dont get future merge conflicts. But changes to other 
files in /lib will be written back so are kept current.

After I have sysupdated,  I unmount and I am back to a system will less 
cruft.

Before I got sidetracked with this bug , I was on the way to adding in 
some more features to make NS rearranging more seamless.

Thanks for you comments and the code for unionfs.

On 02/18/2020 09:03 AM, raingloom wrote:
> Oh dang, I've been looking for this. Also struggling with divergefs 
> weirdness, but this seems so much simpler.
>
> On Mon, Feb 17, 2020 at 16:44, kvik@a-b.xyz wrote:
>> Unfortunately I can't help you with divergefs problem; I am
>> not familiar with the code and I don't feel like dissecting it
>> at the time.
>>
>> If I may instead shamelessly plug unionfs(4) for the task at hand:
>>     git://code.a-b.xyz/unionfs
>> or
>>     http://code.a-b.xyz/unionfs/snapshot/unionfs-master.tar.gz
>>
>>           ; ramfs
>>           ; bind /sys/src /mnt/src
>>           ; unionfs -m /sys/src /tmp /mnt/src
>>           ; @{cd /sys/src; mk install}
>
>
-- 
We need another plan

Re: [9front] Help wanted tracking down a bug in divergefs code [unionfs]

[kvik]

> I see why but personally I don't like the intermediate mount
> point requirement.

I'm not a fan either; unionfs already detects the loop and
could perform the bind trick behind the scenes instead of
snipping the branch off.

> This is were divergefs comes in handy.  It presents to
> sysupdate a layout of the namespace consistant with the
> mercurial repository.
> 
> The ability to write back to the diverged directory is
> paramount to its success.

Quite neat!

Thanks for sharing this use case of divergefs, I'm always
interested in learning about namespace tactics being
deployed in the wild.