From mboxrd@z Thu Jan 1 00:00:00 1970 Message-Id: <1491980708.58243.942088184.4DEC0B7B@webmail.messagingengine.com> From: David Arroyo To: 9fans@9fans.net MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: multipart/mixed; boundary="_----------=_1491980708582434" In-Reply-To: <7D62156B-8EFC-4AB4-9FC9-F8315D4706C0@gmail.com> References: <167A2E66-33F8-4909-8FDB-06F92E930D8B@gmail.com> <795C88A6-B6FE-4B29-AD74-50A0E4D80D97@orthanc.ca> <6C339BCD-FF90-4C03-9AC9-D4098E63EDD8@gmail.com> <789A602E-FC47-40AE-96F4-64E2D2A5B375@orthanc.ca> <4B168609-E25A-4D3C-9A80-1457BA0EE830@gmail.com> <20170407234654.40b4e23976d220cea4ab3daf@eigenstate.org> <1491636459.3063398.938223400.7B35C566@webmail.messagingengine.com> <1491642434.3078770.938277904.69229AAA@webmail.messagingengine.com> <01A91DF6-C9BC-46FD-85EE-A978410FDAB1@gmail.com> <7D62156B-8EFC-4AB4-9FC9-F8315D4706C0@gmail.com> Date: Wed, 12 Apr 2017 03:05:08 -0400 Subject: Re: [9fans] p9p: 9 ls /dev Topicbox-Message-UUID: b9d336fa-ead9-11e9-9d60-3106f5b1d025 This is a multi-part message in MIME format. --_----------=_1491980708582434 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The problem is twofold; * The function exits early if it can't read a max-length directory entry, so that entry is "skipped" in subsequent calls to mygetdents. * As you said, we also lose any buffered dirents that haven't been returned from readdir yet. I think these are both fixable problems, but it may not be worth it. Ori's suggestion to use Getdirentries64 on OSX might be better. David On Tue, Apr 11, 2017, at 10:33 PM, arisawa wrote: > I did more test on david code and found a problem. >=20 > -bash$ mk -f mkfile_david > -bash$ o.test_dirread -a /usr/bin |wc > 1084 4336 27266 > -bash$ o.test_dirread /usr/bin |wc > 1084 4336 27266 > -bash$ ls /usr/bin |wc > 1108 1108 9719 >=20 > option -a is for dirreadall. > 1108 - 1084 entries are lost. >=20 > they are: > o.test_dirread /usr/bin | awk '{print $1}' | sort >/tmp/a > ls /usr/bin | sort >/tmp/b > diff /tmp/a /tmp/b >=20 > -bash$ diff /tmp/a /tmp/b > 21a22 > > SplitForks > 240a242,246 > > easy_install-2.6 > > easy_install-2.7 > > efax > > efix > > egrep > 461a468,473 > > kcc > > kdestroy > > kextutil > > keytool > > kgetcred > > kill.d > 675a688,694 > > piconv > > piconv5.16 > > piconv5.18 > > pidpersec.d > > pkgbuild > > pkill > > pl > 880a900,904 > > spfquery5.18 > > splain > > splain5.16 > > splain5.18 > > split >=20 > sorry if I make a mistake, but I suspect readdir_r() has a buffer, > which can make a problem in using dup(). >=20 > Kenji Arisawa. >=20 > > 2017/04/09 13:18=E3=80=81arisawa =E3=81=AE=E3=83= =A1=E3=83=BC=E3=83=AB=EF=BC=9A > >=20 > > thanks david. > >=20 > > using dup() is very nice idea! > > your code works with > > CFLAGS=3D-D__DARWIN_64_BIT_INO_T # manual is wrong > > and a fix: > > // buf =3D ((char*)buf) + d_reclen(buf); > > buf =3D (struct dirent *)(((char*)buf) + d_reclen(buf)); > > and adding > > #define NAME_MAX 256 > > in somewhere. > >=20 > > now /dev is readable. > >=20 > > one problem is left. > >=20 > > my test code: > > fd =3D open(dirname,OREAD); > > if(fd < 0) > > fatal("%s open error",dirname); > > while((n =3D dirread(fd, &db)) > 0){ > > print("#DBG n=3D%d\n",n); > > for(i =3D 0; i < n; i++) > > print("%s %s %s \n", db[i].name, db[i].uid, db[i].gid); > > } > > close(fd); > > shows for dirname=3D$HOME > > ... > > arch root 501=20 > > bin root 501=20 > > ... > > but they should be > > arch arisawa staff=20 > > bin arisawa staff > > this problem comes from _p9dir() that is used in dirpackage(). > >=20 > > Kenji Arisawa > >=20 > >=20 > >> 2017/04/08 18:07=E3=80=81David Arroyo =E3=81=AE=E3= =83=A1=E3=83=BC=E3=83=AB=EF=BC=9A > >>=20 > >> Ignore my previous post, I was tired and forgot about dup(). How about > >> something like this? (attached) > >>=20 > >> I only tested this on Ubuntu, I don't have an OS X machine. I still we= nt > >> with readdir_r because the AIX and Solaris man pages for readdir were > >> vague about its behavior when called from multiple threads (glibc, mus= l, > >> FreeBSD look pretty safe). > >>=20 > >> Cheers, > >> David > >>=20 > >> On Sat, Apr 8, 2017, at 03:27 AM, David Arroyo wrote: > >>> This should be doable with some combination of fdopendir(3) and > >>> readdir(3). I'm not sure how to avoid leaking memory through the > >>> returned DIR pointer and any memory allocated with by readdir(3). > >>> This is usually free'd by closedir(3), which we can't use without > >>> closing the underlying file. > >>>=20 > >>> It should be OK to use free() on the return value of fdopendir, and > >>> stick to the uglier readdir_r(3) interface. I can definitely see why > >>> Russ went with the simpler system-specific interfaces on this. > >>>=20 > >>> David > >>>=20 > >>> On Sat, Apr 8, 2017, at 02:46 AM, Ori Bernstein wrote: > >>>> On Sat, 8 Apr 2017 15:21:47 +0900, arisawa wrot= e: > >>>>=20 > >>>>> but how to? > >>>>>=20 > >>>>> unix doesn=E2=80=99t have something like fdreaddir(int fd). > >>>>> my guess: russ unwillingly used a low level function such as > >>>>> int getdirentries(int fd, char *buf, int nbytes, long *basep). > >>>>>=20 > >>>>> readdirall() might be OK in regular usage. > >>>>=20 > >>>> I don't use OSX regularly, although I do maintain the syscall > >>>> layer for Myrddin on it. > >>>>=20 > >>>> Getdirentries64 exists, and rudimentary testing doesn't show > >>>> any difficulties with using it. > >>>>=20 > >>>> --=20 > >>>> Ori Bernstein > >>>>=20 > >>>=20 > >> > >=20 >=20 >=20 --_----------=_1491980708582434 Content-Disposition: attachment; filename="posix-dirread2.patch" Content-Id: <1491975708.43689.467692adc6cba3f1b7d91be98e3f77e2769b6ee1.7E05FD9D@content.messagingengine.com> Content-Transfer-Encoding: base64 Content-Type: text/plain; name="posix-dirread2.patch" ZGlmZiAtLWdpdCBhL3NyYy9saWI5L2RpcnJlYWQuYyBiL3NyYy9saWI5L2Rp cnJlYWQuYwppbmRleCA0MGZiZTNjLi4zYjcwOTM4IDEwMDY0NAotLS0gYS9z cmMvbGliOS9kaXJyZWFkLmMKKysrIGIvc3JjL2xpYjkvZGlycmVhZC5jCkBA IC02LDQ2ICs2LDYgQEAKIAogZXh0ZXJuIGludCBfcDlkaXIoc3RydWN0IHN0 YXQqLCBzdHJ1Y3Qgc3RhdCosIGNoYXIqLCBEaXIqLCBjaGFyKiosIGNoYXIq KTsKIAotI2lmIGRlZmluZWQoX19saW51eF9fKQotc3RhdGljIGludAotbXln ZXRkZW50cyhpbnQgZmQsIHN0cnVjdCBkaXJlbnQgKmJ1ZiwgaW50IG4pCi17 Ci0Jb2ZmX3Qgb2ZmOwotCWludCBubjsKLQotCS8qIFRoaXMgZG9lc24ndCBt YXRjaCB0aGUgbWFuIHBhZ2UsIGJ1dCBpdCB3b3JrcyBpbiBEZWJpYW4gd2l0 aCBhIDIuMiBrZXJuZWwgKi8KLQlvZmYgPSBwOXNlZWsoZmQsIDAsIDEpOwot CW5uID0gZ2V0ZGlyZW50cmllcyhmZCwgKHZvaWQqKWJ1ZiwgbiwgJm9mZik7 Ci0JcmV0dXJuIG5uOwotfQotI2VsaWYgZGVmaW5lZChfX0FQUExFX18pIAot c3RhdGljIGludAotbXlnZXRkZW50cyhpbnQgZmQsIHN0cnVjdCBkaXJlbnQg KmJ1ZiwgaW50IG4pCi17Ci0JbG9uZyBvZmY7Ci0JcmV0dXJuIGdldGRpcmVu dHJpZXMoZmQsICh2b2lkKilidWYsIG4sICZvZmYpOwotfQotI2VsaWYgZGVm aW5lZChfX0ZyZWVCU0RfXykgfHwgZGVmaW5lZChfX0RyYWdvbkZseV9fKQot c3RhdGljIGludAotbXlnZXRkZW50cyhpbnQgZmQsIHN0cnVjdCBkaXJlbnQg KmJ1ZiwgaW50IG4pCi17Ci0Jb2ZmX3Qgb2ZmOwotCXJldHVybiBnZXRkaXJl bnRyaWVzKGZkLCAodm9pZCopYnVmLCBuLCAmb2ZmKTsKLX0KLSNlbGlmIGRl ZmluZWQoX19zdW5fXykgfHwgZGVmaW5lZChfX05ldEJTRF9fKSB8fCBkZWZp bmVkKF9fT3BlbkJTRF9fKQotc3RhdGljIGludAotbXlnZXRkZW50cyhpbnQg ZmQsIHN0cnVjdCBkaXJlbnQgKmJ1ZiwgaW50IG4pCi17Ci0JcmV0dXJuIGdl dGRlbnRzKGZkLCAodm9pZCopYnVmLCBuKTsKLX0KLSNlbGlmIGRlZmluZWQo X19BSVhfXykKLXN0YXRpYyBpbnQKLW15Z2V0ZGVudHMoaW50IGZkLCBzdHJ1 Y3QgZGlyZW50ICpidWYsIGludCBuKQotewotCXJldHVybiBnZXRkaXJlbnQo ZmQsICh2b2lkKilidWYsIG4pOwotfQotI2VuZGlmCi0KICNpZiBkZWZpbmVk KF9fRHJhZ29uRmx5X18pCiBzdGF0aWMgaW5saW5lIGludCBkX3JlY2xlbihz dHJ1Y3QgZGlyZW50ICpkZSkgeyByZXR1cm4gX0RJUkVOVF9ESVJTSVooZGUp OyB9CiAjZWxzZQpAQCAtNTMsNiArMTMsMzEgQEAgc3RhdGljIGlubGluZSBp bnQgZF9yZWNsZW4oc3RydWN0IGRpcmVudCAqZGUpIHsgcmV0dXJuIGRlLT5k X3JlY2xlbjsgfQogI2VuZGlmCiAKIHN0YXRpYyBpbnQKK215Z2V0ZGVudHMo aW50IGZkLCBzdHJ1Y3QgZGlyZW50ICpidWYsIGludCBuKQoreworCWludCB0 bXBmZDsKKwljaGFyICpwOworCURJUiAqZGlycDsKKwlzdHJ1Y3QgZGlyZW50 ICplbnRyeTsKKwkKKwlpZiAoKHRtcGZkID0gZHVwKGZkKSkgPT0gLTEpCisJ CXJldHVybiAtMTsKKwlpZiAoKGRpcnAgPSBmZG9wZW5kaXIodG1wZmQpKSA9 PSBuaWwpCisJCXJldHVybiAtMTsKKwkKKwlwID0gKGNoYXIqKWJ1ZjsKKwl3 aGlsZSgoZW50cnkgPSByZWFkZGlyKGRpcnApKSAhPSBuaWwpIHsKKwkJaWYg KGRfcmVjbGVuKGVudHJ5KSA+IG4pCisJCQlicmVhazsKKworCQltZW1jcHko cCwgZW50cnksIGRfcmVjbGVuKGVudHJ5KSk7CisJCXAgKz0gZF9yZWNsZW4o ZW50cnkpOworCQluIC09IGRfcmVjbGVuKGVudHJ5KTsKKwl9CisJcmV0dXJu IChpbnQpKHAgLSAoY2hhciopYnVmKTsKK30KKworc3RhdGljIGludAogY291 bnRkZShjaGFyICpwLCBpbnQgbikKIHsKIAljaGFyICplOwo= --_----------=_1491980708582434--