Re: [9front] can our tools handle very large usb gpt fat32 partitions?

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[cinap_lenrek]

ok, the following commit address the issue with 32 bit sectors:

changeset:   6968:6f3b45a79a84
tag:         tip
user:        cinap_lenrek@felloff.net
date:        Sat Dec 22 20:49:24 2018 +0100
summary:     dossrv: use 64 bit vlong for sectors

this might not fix your issue, but gets that one potential problem
out of the way.

what you can also do is try to rule out nudb/disk issues, by reading
the last sector (or some sector >= 2TB) with dd from openbsd and calculate
a checksum like md5 or sha1 (while having the fat unmounted on openbsd),
then do the same on plan9 and see if they differ.

that way we can rule out large device issues with nusb/disk.

--
cinap

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[Stanley Lieber]

Thanks! I will try later today and report back.

sl

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[sl]

belated update:

running latest code, our dossrv is able to handle the two GPT FAT32
1.8TB partitions on my USB drive without error.  I can read and write
the last block of both partitions.

sl

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[cinap_lenrek]

excellent, thanks!

--
cinap

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[Stanley Lieber]

Latest code allows me to ls all the nested directories on the second partition that appeared empty before. I will test writing and reading the last block tomorrow when I’m able to use a real keyboard again.

Thanks!

sl

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[cinap_lenrek]

try latest commit of nusb/disk:

http://code.9front.org/hg/plan9front/rev/c908fd120133

(you might need to rebuild nusb/disk and then the kernel unless
you run the new nusb/disk manually after killing the running one)

--
cinap

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[cinap_lenrek]

ok, i'll have to figure out how to implement 64 bit lba... i'll write
you back when i have some code.

--
cinap

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[cinap_lenrek]

yeahhhhh.... found it. 32 bit lba -> 2TB max.

--
cinap

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[cinap_lenrek]

ok.

--
cinap

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[sl]

> do 512 byte reads at 1TB and 2TB ±512 byte and compare

Doing all this on an iPhone, so bear with me.

Note: Neither partition is full, so the last (many) GB of each
partition is nothing but zeroes.

To recap:

Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/sd1i      1.8T    914G    948G    49%    /n/no1
/dev/sd1j      1.8T    270G    1.6T    15%    /n/no2

I wrote 512 bytes (all ones) to the last block on each partition.
OpenBSD was able to read both back correctly.  9front was able to read
the last block of the first partition correctly, but could not read
the last block of the second partiton correctly.  Instead of all ones
I got garbage characters.

sl

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[cinap_lenrek]

i woner if you could export the dos partition readonly over 9p. then
i could debug dossrv on it.

--
cinap

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[cinap_lenrek]

lets rule out nusb/disk first.

do 512 byte reads at 1TB and 2TB ±512 byte and compare
with 9front.

--
cinap

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[sl]

Under OpenBSD:

/dev/sd1i 1.8T 914G 948G 49% /n/no1 /dev/sd1j 1.8T 270G 1.6T 15%
/n/no2

So, I guess this makes sense.  All of the first partition is still
just under 1TB.

sl

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[sl]

With the latest changes, I can now read the nested directories on the
first partition that previously I could not read.

The second partition still mounts, and I can see some top-level
directories, but some nested directories still appear to be empty, or
print garbage when I run ls:

deckard; ls Carver_County_Electronic_Records games movies deckard; cd
games deckard; ls fi���!.�'d character in file name: '% deckard; cd
../movies deckard; ls deckard;

Both games/ and movies/ contain many GB of files, and are accessible
under OpenBSD.

I'll try your other suggestion as I get time.

sl

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[cinap_lenrek]

hmmm

dossrv uses signed long for sector numbers... i wonder if this is the problem.
according to my calculation, that would give you 1TB maximum filesystem size
instead of your 2TB.

--
cinap

Re: [9front] can our tools handle very large usb gpt fat32 partitions?

[cinap_lenrek]

hm... i'd try to put some debug prints into readdir() in dossrv.

especially the parsing of the Dosdir:

diff -r c605016cef86 sys/src/cmd/dossrv/dossubs.c
--- a/sys/src/cmd/dossrv/dossubs.c	Sat Dec 15 20:39:32 2018 +0100
+++ b/sys/src/cmd/dossrv/dossubs.c	Sat Dec 22 17:27:14 2018 +0100
@@ -704,6 +704,10 @@
 			return -1;
 		for(o=0; o<bp->sectsize; o+=DOSDIRSIZE){
 			d = (Dosdir *)&p->iobuf[o];
+if(chatty){
+	fmtinstall('H', encodefmt);
+	fprint(2, "readdir addr=%d isect=%d o=%d d=%.*H\n", addr, isect, o, DOSDIRSIZE, d);
+}
 			if(d->name[0] == 0x00){
 				putsect(p);
 				return rcnt;

then invoke dossrv manually like:

dossrv -v -f /dev/sdUX/dos dosdebug >[2]/tmp/dosdebug.log
mount /srv/dosdebug /n/d
ls /n/d/path/to/strange/dir

then check out /tmp/dosdebug.log

--
cinap