fgetgrent_a questions/review

fgetgrent_a questions/review

[Kevin Bowling]

[-- Attachment #1: Type: text/plain, Size: 946 bytes --]

Hi,

I borrowed the fget*ent_a functions to port libnss-cache to FreeBSD because
the fgetent family of functions are not standard and the musl
implementations looked clean and compact with a good license.

https://github.com/google/libnss-cache/pull/1/files#diff-800bf143f84497855c6338a07c19b4af

I had to make a few changes which may be suitable for musl.  First, getline
seems to resize the buffer as it pleases but this causes problems since the
glibc implementation uses fgets and generally something higher up the call
stack handles the resizing.  Second, the current musl implementation
doesn't return ERANGE which was necessary to get the caller's (nsswitch)
code to do the right thing to the buffer.

Finally, I wasn't quite sure what to do with mem and nmem in this case.  I
made mem static, and pass nmem in in a wrapper function.. but I do not know
if these are allocated and freed correctly used standalone like this?

Regards,
Kevin

[-- Attachment #2: Type: text/html, Size: 1190 bytes --]

Re: fgetgrent_a questions/review

[Rich Felker]

On Thu, May 07, 2015 at 05:53:43AM -0700, Kevin Bowling wrote:
> Hi,
> 
> I borrowed the fget*ent_a functions to port libnss-cache to FreeBSD because
> the fgetent family of functions are not standard and the musl
> implementations looked clean and compact with a good license.
> 
> https://github.com/google/libnss-cache/pull/1/files#diff-800bf143f84497855c6338a07c19b4af
> 
> I had to make a few changes which may be suitable for musl.  First, getline
> seems to resize the buffer as it pleases but this causes problems since the
> glibc implementation uses fgets and generally something higher up the call
> stack handles the resizing.  Second, the current musl implementation
> doesn't return ERANGE which was necessary to get the caller's (nsswitch)
> code to do the right thing to the buffer.

The way the code is factored in musl, with the internal function
(getgrent_a) being allocating/using getline, is very intentional.
Implementing getgrnam/getgrgid in terms of their *_r versions and
retrying on ERANGE would be possible (albeit inefficient), but
getgrent cannot be implemented that way because it will have messed up
the iterator state already. Minimal code duplication and runtime
efficiency is achieved by having the internal function always succeed
(except on resource errors like ENOMEM or EMFILE/ENFILE) and the
wrapper functions for the *_r interfaces attempt to copy the data into
the caller-provided buffer and report ERANGE if the data doesn't fit.

If a different factoring works better for your project, by all means
use it, but I don't think it would work for what musl needs.

> Finally, I wasn't quite sure what to do with mem and nmem in this case.  I
> made mem static, and pass nmem in in a wrapper function.. but I do not know
> if these are allocated and freed correctly used standalone like this?

I don't know what libnss-cache's API is like, but this sounds
unsuitable. It's definitely not thread-safe or library-safe
(multiple-caller-safe).

Rich

Re: fgetgrent_a questions/review

[Kevin Bowling]

[-- Attachment #1: Type: text/plain, Size: 2639 bytes --]

I studied the musl code enough to understand it, and understand the
underlying memory contract of nsswitch from glibc and FreeBSD.
I came up with this:
https://github.com/google/libnss-cache/blob/master/compat/getgrent_r.c

I use the line buffer to also store the group members.  We can ERANGE to
get a bigger buffer to store everything if necessary.  nsswitch on these
platforms will rewind the file pointer when that happens.

Thanks for the succinct implementation to start from!

Regards,
Kevin

On Thu, May 7, 2015 at 9:49 AM, Rich Felker <dalias@libc.org> wrote:

> On Thu, May 07, 2015 at 05:53:43AM -0700, Kevin Bowling wrote:
> > Hi,
> >
> > I borrowed the fget*ent_a functions to port libnss-cache to FreeBSD
> because
> > the fgetent family of functions are not standard and the musl
> > implementations looked clean and compact with a good license.
> >
> >
> https://github.com/google/libnss-cache/pull/1/files#diff-800bf143f84497855c6338a07c19b4af
> >
> > I had to make a few changes which may be suitable for musl.  First,
> getline
> > seems to resize the buffer as it pleases but this causes problems since
> the
> > glibc implementation uses fgets and generally something higher up the
> call
> > stack handles the resizing.  Second, the current musl implementation
> > doesn't return ERANGE which was necessary to get the caller's (nsswitch)
> > code to do the right thing to the buffer.
>
> The way the code is factored in musl, with the internal function
> (getgrent_a) being allocating/using getline, is very intentional.
> Implementing getgrnam/getgrgid in terms of their *_r versions and
> retrying on ERANGE would be possible (albeit inefficient), but
> getgrent cannot be implemented that way because it will have messed up
> the iterator state already. Minimal code duplication and runtime
> efficiency is achieved by having the internal function always succeed
> (except on resource errors like ENOMEM or EMFILE/ENFILE) and the
> wrapper functions for the *_r interfaces attempt to copy the data into
> the caller-provided buffer and report ERANGE if the data doesn't fit.
>
> If a different factoring works better for your project, by all means
> use it, but I don't think it would work for what musl needs.
>
> > Finally, I wasn't quite sure what to do with mem and nmem in this case.
> I
> > made mem static, and pass nmem in in a wrapper function.. but I do not
> know
> > if these are allocated and freed correctly used standalone like this?
>
> I don't know what libnss-cache's API is like, but this sounds
> unsuitable. It's definitely not thread-safe or library-safe
> (multiple-caller-safe).
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 3526 bytes --]