From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/7653 Path: news.gmane.org!not-for-mail From: Kevin Bowling Newsgroups: gmane.linux.lib.musl.general Subject: Re: fgetgrent_a questions/review Date: Sun, 17 May 2015 00:02:14 -0700 Message-ID: References: <20150507164938.GT17573@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a1140289c402d9c051641a8ed X-Trace: ger.gmane.org 1431846154 30064 80.91.229.3 (17 May 2015 07:02:34 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 17 May 2015 07:02:34 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-7665-gllmg-musl=m.gmane.org@lists.openwall.com Sun May 17 09:02:30 2015 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1Ytsai-0008O6-Qz for gllmg-musl@m.gmane.org; Sun, 17 May 2015 09:02:28 +0200 Original-Received: (qmail 15783 invoked by uid 550); 17 May 2015 07:02:27 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 15762 invoked from network); 17 May 2015 07:02:26 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=BLzo7nNpHyLGOKKOJSrY3tkxPDj3bUDra5FRPVuLgJg=; b=I4m5qfLems4skR9l1Zd5KZl9rNUsWwutiFoUC7L9hw5WTPWz908ToLhHcLCT0cbidr opoMEJlKyZXCKi7yKZ8qOSc7D8Kx4fdE9Wcao2mWN/UOGfucYPEU8K9A4WNp3G4SKEMm Aks0wZ2sM7xTAYERl0uOFh14oel6OIA0hVoL4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=BLzo7nNpHyLGOKKOJSrY3tkxPDj3bUDra5FRPVuLgJg=; b=jSh2PAtrsgO9JOD4/2KXRjlAu4N5ryFA7Z3TUcqYn0ZNWuDSYJVqlx0ztauknNfvrh m9PBwxwuCrEll1Y+mp0BQF5bXP5QY9iySfCgCZD14LL1oZUu4xeyfvDTG5RhWBgRbsf+ QGn+iBfE7SgEmF4EePwHOXTWnhzsPIzcuGu0KeSlhOQagtGDjbuV74DaMuLH6kdvH+lw dCXJ3C3kuRn5+z8fvw1fWa855MmEVbf5810z4AbJ3JBcVIXTt/I8zAFUy60RvoC5RWqr wmlqn61BotGMCFZRy9HxFqbWkQ6DMBBYmHHw58W2WyfT2kaVMiEJlP3z7RBnVIMe8/eb hAsg== X-Gm-Message-State: ALoCoQmJx4B+DBYHZjhZ3XQwnFS3g4vH8KeEHcs42nvuOok3/l0e7XaePxXdhEk2cJgV0WEx9Xae X-Received: by 10.107.150.14 with SMTP id y14mr20058224iod.55.1431846135008; Sun, 17 May 2015 00:02:15 -0700 (PDT) In-Reply-To: <20150507164938.GT17573@brightrain.aerifal.cx> Xref: news.gmane.org gmane.linux.lib.musl.general:7653 Archived-At: --001a1140289c402d9c051641a8ed Content-Type: text/plain; charset=UTF-8 I studied the musl code enough to understand it, and understand the underlying memory contract of nsswitch from glibc and FreeBSD. I came up with this: https://github.com/google/libnss-cache/blob/master/compat/getgrent_r.c I use the line buffer to also store the group members. We can ERANGE to get a bigger buffer to store everything if necessary. nsswitch on these platforms will rewind the file pointer when that happens. Thanks for the succinct implementation to start from! Regards, Kevin On Thu, May 7, 2015 at 9:49 AM, Rich Felker wrote: > On Thu, May 07, 2015 at 05:53:43AM -0700, Kevin Bowling wrote: > > Hi, > > > > I borrowed the fget*ent_a functions to port libnss-cache to FreeBSD > because > > the fgetent family of functions are not standard and the musl > > implementations looked clean and compact with a good license. > > > > > https://github.com/google/libnss-cache/pull/1/files#diff-800bf143f84497855c6338a07c19b4af > > > > I had to make a few changes which may be suitable for musl. First, > getline > > seems to resize the buffer as it pleases but this causes problems since > the > > glibc implementation uses fgets and generally something higher up the > call > > stack handles the resizing. Second, the current musl implementation > > doesn't return ERANGE which was necessary to get the caller's (nsswitch) > > code to do the right thing to the buffer. > > The way the code is factored in musl, with the internal function > (getgrent_a) being allocating/using getline, is very intentional. > Implementing getgrnam/getgrgid in terms of their *_r versions and > retrying on ERANGE would be possible (albeit inefficient), but > getgrent cannot be implemented that way because it will have messed up > the iterator state already. Minimal code duplication and runtime > efficiency is achieved by having the internal function always succeed > (except on resource errors like ENOMEM or EMFILE/ENFILE) and the > wrapper functions for the *_r interfaces attempt to copy the data into > the caller-provided buffer and report ERANGE if the data doesn't fit. > > If a different factoring works better for your project, by all means > use it, but I don't think it would work for what musl needs. > > > Finally, I wasn't quite sure what to do with mem and nmem in this case. > I > > made mem static, and pass nmem in in a wrapper function.. but I do not > know > > if these are allocated and freed correctly used standalone like this? > > I don't know what libnss-cache's API is like, but this sounds > unsuitable. It's definitely not thread-safe or library-safe > (multiple-caller-safe). > > Rich > --001a1140289c402d9c051641a8ed Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I studied the musl code enough to understand it,= and understand the underlying memory contract of nsswitch from glibc and F= reeBSD.
I came up with this:=C2=A0 https://github.com/google/lib= nss-cache/blob/master/compat/getgrent_r.c

I use the l= ine buffer to also store the group members.=C2=A0 We can ERANGE to get a bi= gger buffer to store everything if necessary.=C2=A0 nsswitch on these platf= orms will rewind the file pointer when that happens.

Thanks for the succinct implementation to start from!
=
Regards,
Kevin

<= div class=3D"gmail_quote">On Thu, May 7, 2015 at 9:49 AM, Rich Felker <dalia= s@libc.org> wrote:
On Thu, May 07, 2015 at 05:53:43AM -0700, Kevin Bowling wrote:
> Hi,
>
> I borrowed the fget*ent_a functions to port libnss-cache to FreeBSD be= cause
> the fgetent family of functions are not standard and the musl
> implementations looked clean and compact with a good license.
>
> https://github.com/google= /libnss-cache/pull/1/files#diff-800bf143f84497855c6338a07c19b4af
>
> I had to make a few changes which may be suitable for musl.=C2=A0 Firs= t, getline
> seems to resize the buffer as it pleases but this causes problems sinc= e the
> glibc implementation uses fgets and generally something higher up the = call
> stack handles the resizing.=C2=A0 Second, the current musl implementat= ion
> doesn't return ERANGE which was necessary to get the caller's = (nsswitch)
> code to do the right thing to the buffer.

The way the code is factored in musl, with the internal function
(getgrent_a) being allocating/using getline, is very intentional.
Implementing getgrnam/getgrgid in terms of their *_r versions and
retrying on ERANGE would be possible (albeit inefficient), but
getgrent cannot be implemented that way because it will have messed up
the iterator state already. Minimal code duplication and runtime
efficiency is achieved by having the internal function always succeed
(except on resource errors like ENOMEM or EMFILE/ENFILE) and the
wrapper functions for the *_r interfaces attempt to copy the data into
the caller-provided buffer and report ERANGE if the data doesn't fit.
If a different factoring works better for your project, by all means
use it, but I don't think it would work for what musl needs.

> Finally, I wasn't quite sure what to do with mem and nmem in this = case.=C2=A0 I
> made mem static, and pass nmem in in a wrapper function.. but I do not= know
> if these are allocated and freed correctly used standalone like this?<= br>
I don't know what libnss-cache's API is like, but this sound= s
unsuitable. It's definitely not thread-safe or library-safe
(multiple-caller-safe).

Rich

--001a1140289c402d9c051641a8ed--