_PATH_LASTLOG

_PATH_LASTLOG

[Raphael Cohn]

[-- Attachment #1: Type: text/plain, Size: 384 bytes --]

Hi,

I'm trying to compile linux-pam 1.1.8 using musl-cross, and I've hit a
compilation error in  modules/pam_lastlog/pam_lastlog.c

Essentially, this code includes the clib utmp.h (based on HAVE_UTMP_H) and
then assumes _PATH_LASTLOG is defined.

utmp.h doesn't define this macro, but does define _PATH_UTMP and
_PATH_WTMP. Should it? (And why are they set to /dev/null/xxx )?

Raph

[-- Attachment #2: Type: text/html, Size: 541 bytes --]

Re: _PATH_LASTLOG

[Szabolcs Nagy]

* Raphael Cohn <raphael.cohn@stormmq.com> [2013-12-03 17:44:04 +0000]:
> I'm trying to compile linux-pam 1.1.8 using musl-cross, and I've hit a
> compilation error in  modules/pam_lastlog/pam_lastlog.c
> 
> Essentially, this code includes the clib utmp.h (based on HAVE_UTMP_H) and
> then assumes _PATH_LASTLOG is defined.
> 
> utmp.h doesn't define this macro, but does define _PATH_UTMP and
> _PATH_WTMP. Should it? (And why are they set to /dev/null/xxx )?

the utmp business should be added to the faq..
and probably to the differences compared to glibc page as well

utmp.h was never part of any standard so it can contain anything
it could include paths.h (all the macros there are in the reserved
name space anyway), glibc does include paths.h there and the bsds
define _PATH_LASTLOG

the musl paths.h has _PATH_LASTLOG defined twice

the "/dev/null/xxx" is a file that's guaranteed to fail to get
opened or unlinked

Re: _PATH_LASTLOG

[Raphael Cohn]

[-- Attachment #1: Type: text/plain, Size: 1439 bytes --]

Ta.

Would it be possible to have the "/dev/null/xxx" paths' values as an option
to ./configure?

Actually, it would be very useful to be able to ./configure all the other
hard coded paths in musl, eg the default dynamlic linker search path. When
running with a setup like Nixos, or the like, these paths need to be
different. Of course, one can patch, but that's not sustainable in the long
run.

Please?


On 3 December 2013 18:42, Szabolcs Nagy <nsz@port70.net> wrote:

> * Raphael Cohn <raphael.cohn@stormmq.com> [2013-12-03 17:44:04 +0000]:
> > I'm trying to compile linux-pam 1.1.8 using musl-cross, and I've hit a
> > compilation error in  modules/pam_lastlog/pam_lastlog.c
> >
> > Essentially, this code includes the clib utmp.h (based on HAVE_UTMP_H)
> and
> > then assumes _PATH_LASTLOG is defined.
> >
> > utmp.h doesn't define this macro, but does define _PATH_UTMP and
> > _PATH_WTMP. Should it? (And why are they set to /dev/null/xxx )?
>
> the utmp business should be added to the faq..
> and probably to the differences compared to glibc page as well
>
> utmp.h was never part of any standard so it can contain anything
> it could include paths.h (all the macros there are in the reserved
> name space anyway), glibc does include paths.h there and the bsds
> define _PATH_LASTLOG
>
> the musl paths.h has _PATH_LASTLOG defined twice
>
> the "/dev/null/xxx" is a file that's guaranteed to fail to get
> opened or unlinked
>

[-- Attachment #2: Type: text/html, Size: 2067 bytes --]

Re: _PATH_LASTLOG

[Rich Felker]

On Tue, Dec 03, 2013 at 05:44:04PM +0000, Raphael Cohn wrote:
> Hi,
> 
> I'm trying to compile linux-pam 1.1.8 using musl-cross, and I've hit a
> compilation error in  modules/pam_lastlog/pam_lastlog.c
> 
> Essentially, this code includes the clib utmp.h (based on HAVE_UTMP_H) and
> then assumes _PATH_LASTLOG is defined.

This is a bug in pam. It should be testing #ifdef _PATH_LASTLOG.
_PATH_LASTLOG is not specified to be defined anywhere, much less
utmp.h; its presence is a glibc "feature". At present musl has some of
these paths that are fairly universally-agreed-upon in paths.h, but
it's really bad design for libc to be imposing policy for things that
have nothing to do with libc itself through the headers it installs.
(In fact, if I'm not mistaken, most distros patch glibc's paths.h to
conform to their FS layours...)

> utmp.h doesn't define this macro, but does define _PATH_UTMP and
> _PATH_WTMP. Should it? (And why are they set to /dev/null/xxx )?

These are all very good questions. At present musl does not support
storing anything to utmp, and uses /dev/null/xxx as a pathname that
will fail to open and fail to unlink (since /dev/null is required by
POSIX to exist as a device, i.e. not a directory). (Using /dev/null
would be dangerous since some broken programs unlink the utmp file and
make a new one.)

Rich

Re: _PATH_LASTLOG

[Raphael Cohn]

[-- Attachment #1: Type: text/plain, Size: 3054 bytes --]

Fair enough. I agree coding absolute paths inside a libc is a bad design.
Frankly, the more I see of the gnu toolchain et al, esp glibc, the less I
like. If this was a minor language, I'd say it wasn't fit for real-world
use (;-). In practice, it's just showing its age. Thank you for writing
musl, and shining a light into some of the horrors.

I've hit another roadblock with PAM - innetgr is not defined. It seems
there's a configure check for it, which is then not consistently used. Any
ideas for a dirty workaround - even if it means some pam modules aren't
useful? What would a dummy stub do?

PS I'm using musl to compile my own distro set up to get a handle on why
the traditionally gnu-based c-compiled world of Linux programs, esp when
cross-compiled, is so brittle and bug-prone... Everything I've seen so far,
including the GNU toolchain, autoconf, m4, make, libtool (uggh) and its ilk
has only confirmed my prejudices - too much ego and high theory, not enough
real-world pragmatism. Indeed, most of it seems to be more about copying
what was already done without really understanding why it was done the way
it was, rather than innovating...


Raphael Cohn
Chief Architect, stormmq
Co-Chair, OASIS MQTT Standard
Secretary, OASIS AMQP Standard
raphael.cohn@stormmq.com
+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United
Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number
07175657
StormMQ.com


On 3 December 2013 19:34, Rich Felker <dalias@aerifal.cx> wrote:

> On Tue, Dec 03, 2013 at 05:44:04PM +0000, Raphael Cohn wrote:
> > Hi,
> >
> > I'm trying to compile linux-pam 1.1.8 using musl-cross, and I've hit a
> > compilation error in  modules/pam_lastlog/pam_lastlog.c
> >
> > Essentially, this code includes the clib utmp.h (based on HAVE_UTMP_H)
> and
> > then assumes _PATH_LASTLOG is defined.
>
> This is a bug in pam. It should be testing #ifdef _PATH_LASTLOG.
> _PATH_LASTLOG is not specified to be defined anywhere, much less
> utmp.h; its presence is a glibc "feature". At present musl has some of
> these paths that are fairly universally-agreed-upon in paths.h, but
> it's really bad design for libc to be imposing policy for things that
> have nothing to do with libc itself through the headers it installs.
> (In fact, if I'm not mistaken, most distros patch glibc's paths.h to
> conform to their FS layours...)
>
> > utmp.h doesn't define this macro, but does define _PATH_UTMP and
> > _PATH_WTMP. Should it? (And why are they set to /dev/null/xxx )?
>
> These are all very good questions. At present musl does not support
> storing anything to utmp, and uses /dev/null/xxx as a pathname that
> will fail to open and fail to unlink (since /dev/null is required by
> POSIX to exist as a device, i.e. not a directory). (Using /dev/null
> would be dangerous since some broken programs unlink the utmp file and
> make a new one.)
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 3913 bytes --]

Re: _PATH_LASTLOG

[Rich Felker]

On Tue, Dec 03, 2013 at 07:09:05PM +0000, Raphael Cohn wrote:
> Ta.
> 
> Would it be possible to have the "/dev/null/xxx" paths' values as an option
> to ./configure?
> 
> Actually, it would be very useful to be able to ./configure all the other
> hard coded paths in musl, eg the default dynamlic linker search path. When
> running with a setup like Nixos, or the like, these paths need to be
> different. Of course, one can patch, but that's not sustainable in the long
> run.

The dynamic linker searches for its path file relative to its own
location, which should cover this kind of usage. It's only in the case
where no path file exists that the hard-coded /lib, /usr/lib, etc.
would get searched.

> Please?

I think such a request should be accompanied by explanations of what
you're trying to achieve that's difficult or impossible with the
current scheme.

Most of the hard-coded paths in musl are hard-coded because there's a
standard pathname either required by the standards or that was
universal in all historical systems, and because musl aims to be
useful for producing "run anywhere" static binaries. Gratuitously
changing paths defeats this goal. Of course musl attempts to minimize
the number of hard-coded pathnames anyway; here's a list from the
current documentation draft which you could review to determine which
are problematic to your intended usage cases:

----------------------------------------------------------------------
* `/dev/null` - device node, required by POSIX

* `/dev/tty` - device node, required by POSIX

* `/tmp` - required by POSIX to exist as a directory, and used by
  various temporary file creation functions.

* `/bin/sh` - an executable file providing a POSIX-conforming shell

* `/proc` - must be a mount point for Linux procfs or a symlink to
  such. Several functions such as realpath, fexecve, and a number of
  the "at" functions added in POSIX 2008 need access to /proc to
  function correctly.

While some programs may operate correctly even without some or all of
the above, musl's behavior in their absence is unspecified.

### Additional Pathnames Used

* `/dev/log` - a UNIX domain socket to which the `syslog()` interface
  sends log messages. If absent or inaccessible, log messages will be
  discarded.

* `/dev/shm` - a directory; should have permissions 01777. If absent,
  POSIX shared memory and named semaphore interfaces will fail;
  programs not using these features will be unaffected.

* `/dev/ptmx` and `/dev/pts` - device node and devpts filesystem mount
  point, respectively. If absent or inaccessible, `posix_openpt()` and
  `openpty()` will fail.

* `/etc/passwd` and `/etc/group` - text files containing the user and
  group databases, mappings between names and numeric ids, and group
  membership lists, in the standard traditional format. If absent,
  user and/or group lookups will fail.

* `/etc/shadow` - text file containing shadow password hashes for some
  or all users.

* `/etc/resolv.conf` - text file providing addresses of nameservers to
  be used for DNS lookups. If absent, DNS requests will be sent to the
  loopback address and will fail unless the host has its own
  nameserver.

* `/etc/hosts` - text file mapping hostnames to IP addresses.

* `/etc/services` - text file mapping network service names to port
  numbers.

* `/usr/share/zoneinfo`, `/share/zoneinfo`, and `/etc/zoneinfo` -
  directories searched for time zone files when the `TZ` environment
  variable is set to a relative pathname.

* `../etc/ld-musl-$(ARCH).path`, taken relative to the location of the
  "program interpreter" specified in the program's headers - if
  present, this will be processed as a text file containing the shared
  library search path, with components delimited by newlines or
  colons. If absent, a default path of
  `"/lib:/usr/local/lib:/usr/lib"` will be used. Not used by
  static-linked programs.
----------------------------------------------------------------------

Let me know. This may end up being an ugly issue but it's something we
should look at, in any case...

Rich

Re: _PATH_LASTLOG

[Laurent Bercot]

> Everything I've seen so far, including the GNU toolchain, autoconf, m4, make, libtool (uggh) and its ilk has only confirmed my prejudices
>  - too much ego and high theory, not enough real-world pragmatism.
> Indeed, most of it seems to be more about copying what was already done without really understanding why it was done the way it was, rather than innovating...

  You have to remember that GNU started as, and remained, a political project, not a
technical project. The point of GNU was to rewrite patent-encumbered Unix as free
software: the point was not to innovate or produce quality code. You should view
GNU as a proof-of-concept project, which it really started as; unfortunately, a lot
of today's applications are still relying on a proof-of-concept implementation.

  Grats on making the switch, and welcome to the world of real, technically-focused
implementations. ;)

-- 
  Laurent

Re: _PATH_LASTLOG

[Rich Felker]

On Tue, Dec 03, 2013 at 07:50:21PM +0000, Raphael Cohn wrote:
> Fair enough. I agree coding absolute paths inside a libc is a bad design.

For what it's worth, there are two issues here:

1. Hard-coding absolute paths inside libc.

2. Providing header files with libc that hard-code paths for
   application usage even though libc does not use these paths for
   anything and they are purely an application-ecosystem policy
   matter.

The latter is what I was mildly complaining about glibc doing.

> Frankly, the more I see of the gnu toolchain et al, esp glibc, the less I
> like. If this was a minor language, I'd say it wasn't fit for real-world
> use (;-). In practice, it's just showing its age. Thank you for writing
> musl, and shining a light into some of the horrors.

:)

For what it's worth, the glibc part of the gnu toolchain is getting
"less bad". The new maintainership is a lot more friendly and
responsive to bug reports and willing to go back and fix major cruft,
but the latter of course requires finding people qualified to
understand and fix it... :-)

On the other hand, GCC has become a mixed-up mess of C and C++ and
patchwork of random libraries it outsources its math and optimization
work to. Sadly this seems to have been kicked off by competition from
LLVM/clang... libfirm/cparser looks really promising as an alternative
to both but still has a long ways to go.

> I've hit another roadblock with PAM - innetgr is not defined. It seems
> there's a configure check for it, which is then not consistently used. Any
> ideas for a dirty workaround - even if it means some pam modules aren't
> useful? What would a dummy stub do?

I recall someone else running into this problem. I think he solved it
by writing a dummy implementation of these functions that just always
returns NULL. It might be possible to just #if 0 out the code. You
might ask on IRC or search the mail archives and see if you can find
anything about it.

Rich

Re: _PATH_LASTLOG

[Raphael Cohn]

[-- Attachment #1: Type: text/plain, Size: 6418 bytes --]

Thanks for that - just having a list is an useful place to start. I think
the default file names are quite sensible - especially for a common
run-anywhere use case. And some - where mandated by POSIX - probably should
never change. What would be nice might be to be able to define the prefix
for /etc to something else (so we can use atomic symlink changes to flip
configs).

I'd like to have more of a think about the other paths. We're only a short
way into our project, so our ideas might change. What we're looking at is a
Nixos-like linux, where we rebuild only packages because other packages
have changed. We want to keep every package isolated, so we can apply PATH
controls, fine-grained capability permissions, chattr -ai, etc. Part of
doing this means we don't want paths 'hanging around' inside libraries that
are used if present - as these allow an attacker (or more likely, a duff
package) to accidentally stop itself working, ie if there's no /usr/lib on
system, then nothing should be able to stick itself in /usr/lib and
override the system setup.


PS As an aside, I've always wanted /etc/hosts to also have a parallel
/etc/hosts.d/. It'd make maintaining things without a DNS server extremely
easy - think dynamically adding and removing VMs in most cloud providers,
especially those where multicast DNS doesn't work... like Azure. (Yes, I
had a client that insisted on using it with Linux). Likewise it'd be nice
to be able to add and remove DNS servers with a /etc/resolv.conf.d. Makes
automated config and change management and audit that bit easier. (Debian
do this using run-parts for lots of things for those sorts of reasons).


Raphael Cohn
Chief Architect, stormmq
Co-Chair, OASIS MQTT Standard
Secretary, OASIS AMQP Standard
raphael.cohn@stormmq.com
+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United
Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number
07175657
StormMQ.com


On 3 December 2013 19:54, Rich Felker <dalias@aerifal.cx> wrote:

> On Tue, Dec 03, 2013 at 07:09:05PM +0000, Raphael Cohn wrote:
> > Ta.
> >
> > Would it be possible to have the "/dev/null/xxx" paths' values as an
> option
> > to ./configure?
> >
> > Actually, it would be very useful to be able to ./configure all the other
> > hard coded paths in musl, eg the default dynamlic linker search path.
> When
> > running with a setup like Nixos, or the like, these paths need to be
> > different. Of course, one can patch, but that's not sustainable in the
> long
> > run.
>
> The dynamic linker searches for its path file relative to its own
> location, which should cover this kind of usage. It's only in the case
> where no path file exists that the hard-coded /lib, /usr/lib, etc.
> would get searched.
>
> > Please?
>
> I think such a request should be accompanied by explanations of what
> you're trying to achieve that's difficult or impossible with the
> current scheme.
>
> Most of the hard-coded paths in musl are hard-coded because there's a
> standard pathname either required by the standards or that was
> universal in all historical systems, and because musl aims to be
> useful for producing "run anywhere" static binaries. Gratuitously
> changing paths defeats this goal. Of course musl attempts to minimize
> the number of hard-coded pathnames anyway; here's a list from the
> current documentation draft which you could review to determine which
> are problematic to your intended usage cases:
>
> ----------------------------------------------------------------------
> * `/dev/null` - device node, required by POSIX
>
> * `/dev/tty` - device node, required by POSIX
>
> * `/tmp` - required by POSIX to exist as a directory, and used by
>   various temporary file creation functions.
>
> * `/bin/sh` - an executable file providing a POSIX-conforming shell
>
> * `/proc` - must be a mount point for Linux procfs or a symlink to
>   such. Several functions such as realpath, fexecve, and a number of
>   the "at" functions added in POSIX 2008 need access to /proc to
>   function correctly.
>
> While some programs may operate correctly even without some or all of
> the above, musl's behavior in their absence is unspecified.
>
> ### Additional Pathnames Used
>
> * `/dev/log` - a UNIX domain socket to which the `syslog()` interface
>   sends log messages. If absent or inaccessible, log messages will be
>   discarded.
>
> * `/dev/shm` - a directory; should have permissions 01777. If absent,
>   POSIX shared memory and named semaphore interfaces will fail;
>   programs not using these features will be unaffected.
>
> * `/dev/ptmx` and `/dev/pts` - device node and devpts filesystem mount
>   point, respectively. If absent or inaccessible, `posix_openpt()` and
>   `openpty()` will fail.
>
> * `/etc/passwd` and `/etc/group` - text files containing the user and
>   group databases, mappings between names and numeric ids, and group
>   membership lists, in the standard traditional format. If absent,
>   user and/or group lookups will fail.
>
> * `/etc/shadow` - text file containing shadow password hashes for some
>   or all users.
>
> * `/etc/resolv.conf` - text file providing addresses of nameservers to
>   be used for DNS lookups. If absent, DNS requests will be sent to the
>   loopback address and will fail unless the host has its own
>   nameserver.
>
> * `/etc/hosts` - text file mapping hostnames to IP addresses.
>
> * `/etc/services` - text file mapping network service names to port
>   numbers.
>
> * `/usr/share/zoneinfo`, `/share/zoneinfo`, and `/etc/zoneinfo` -
>   directories searched for time zone files when the `TZ` environment
>   variable is set to a relative pathname.
>
> * `../etc/ld-musl-$(ARCH).path`, taken relative to the location of the
>   "program interpreter" specified in the program's headers - if
>   present, this will be processed as a text file containing the shared
>   library search path, with components delimited by newlines or
>   colons. If absent, a default path of
>   `"/lib:/usr/local/lib:/usr/lib"` will be used. Not used by
>   static-linked programs.
> ----------------------------------------------------------------------
>
> Let me know. This may end up being an ugly issue but it's something we
> should look at, in any case...
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 7600 bytes --]

Re: _PATH_LASTLOG

[Raphael Cohn]

[-- Attachment #1: Type: text/plain, Size: 1579 bytes --]

Yes, I very much accept that. Email isn't the best way for me of
communicating a lot of thought in a few words!

Raphael Cohn
Chief Architect, stormmq
Co-Chair, OASIS MQTT Standard
Secretary, OASIS AMQP Standard
raphael.cohn@stormmq.com
+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United
Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number
07175657
StormMQ.com


On 3 December 2013 20:03, Laurent Bercot <ska-dietlibc@skarnet.org> wrote:

>
>  Everything I've seen so far, including the GNU toolchain, autoconf, m4,
>> make, libtool (uggh) and its ilk has only confirmed my prejudices
>>  - too much ego and high theory, not enough real-world pragmatism.
>> Indeed, most of it seems to be more about copying what was already done
>> without really understanding why it was done the way it was, rather than
>> innovating...
>>
>
>  You have to remember that GNU started as, and remained, a political
> project, not a
> technical project. The point of GNU was to rewrite patent-encumbered Unix
> as free
> software: the point was not to innovate or produce quality code. You
> should view
> GNU as a proof-of-concept project, which it really started as;
> unfortunately, a lot
> of today's applications are still relying on a proof-of-concept
> implementation.
>
>  Grats on making the switch, and welcome to the world of real,
> technically-focused
> implementations. ;)
>
> --
>  Laurent
>
>

[-- Attachment #2: Type: text/html, Size: 2364 bytes --]

Re: _PATH_LASTLOG

[Raphael Cohn]

[-- Attachment #1: Type: text/plain, Size: 2686 bytes --]

Thanks for the pointers to doing stuff with PAM. Out of interest, if there
is an interesting answer, is there a good place to write it up?

Raphael Cohn
Chief Architect, stormmq
Co-Chair, OASIS MQTT Standard
Secretary, OASIS AMQP Standard
raphael.cohn@stormmq.com
+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United
Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number
07175657
StormMQ.com


On 3 December 2013 20:06, Rich Felker <dalias@aerifal.cx> wrote:

> On Tue, Dec 03, 2013 at 07:50:21PM +0000, Raphael Cohn wrote:
> > Fair enough. I agree coding absolute paths inside a libc is a bad design.
>
> For what it's worth, there are two issues here:
>
> 1. Hard-coding absolute paths inside libc.
>
> 2. Providing header files with libc that hard-code paths for
>    application usage even though libc does not use these paths for
>    anything and they are purely an application-ecosystem policy
>    matter.
>
> The latter is what I was mildly complaining about glibc doing.
>
> > Frankly, the more I see of the gnu toolchain et al, esp glibc, the less I
> > like. If this was a minor language, I'd say it wasn't fit for real-world
> > use (;-). In practice, it's just showing its age. Thank you for writing
> > musl, and shining a light into some of the horrors.
>
> :)
>
> For what it's worth, the glibc part of the gnu toolchain is getting
> "less bad". The new maintainership is a lot more friendly and
> responsive to bug reports and willing to go back and fix major cruft,
> but the latter of course requires finding people qualified to
> understand and fix it... :-)
>
> On the other hand, GCC has become a mixed-up mess of C and C++ and
> patchwork of random libraries it outsources its math and optimization
> work to. Sadly this seems to have been kicked off by competition from
> LLVM/clang... libfirm/cparser looks really promising as an alternative
> to both but still has a long ways to go.
>
> > I've hit another roadblock with PAM - innetgr is not defined. It seems
> > there's a configure check for it, which is then not consistently used.
> Any
> > ideas for a dirty workaround - even if it means some pam modules aren't
> > useful? What would a dummy stub do?
>
> I recall someone else running into this problem. I think he solved it
> by writing a dummy implementation of these functions that just always
> returns NULL. It might be possible to just #if 0 out the code. You
> might ask on IRC or search the mail archives and see if you can find
> anything about it.
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 3584 bytes --]

Re: _PATH_LASTLOG

[Rich Felker]

On Tue, Dec 03, 2013 at 08:10:56PM +0000, Raphael Cohn wrote:
> Thanks for that - just having a list is an useful place to start. I think
> the default file names are quite sensible - especially for a common
> run-anywhere use case. And some - where mandated by POSIX - probably should
> never change. What would be nice might be to be able to define the prefix
> for /etc to something else (so we can use atomic symlink changes to flip
> configs).

Are you also talking about having separate configuration dirs for each
package? Would each package need its own user database (passwd file),
DNS configuration (which the DHCP client would have to update), etc.?
The intent in musl is that the "config"-like files it reads from /etc
are not things that should be application-local but system-global.
Obviously there are different views on what's local/global, though.
One problem I'd like to solve is making a way for users to override
the system resolv.conf; this would be especially valuable for us (musl
developers) writing test cases for the resolver, in that we could
redirect lookups to a fake DNS daemon running in a separate thread of
the test program serving fixed, possibly-intentionally-malformed
replies.

> I'd like to have more of a think about the other paths. We're only a short
> way into our project, so our ideas might change. What we're looking at is a
> Nixos-like linux, where we rebuild only packages because other packages
> have changed. We want to keep every package isolated, so we can apply PATH
> controls, fine-grained capability permissions, chattr -ai, etc. Part of
> doing this means we don't want paths 'hanging around' inside libraries that
> are used if present - as these allow an attacker (or more likely, a duff
> package) to accidentally stop itself working, ie if there's no /usr/lib on
> system, then nothing should be able to stick itself in /usr/lib and
> override the system setup.

This seems like a good foundation for a package system. I've looked
into Nixos before but never really tried it out, and got the
impression that the concept was very good but it might not be the best
implementation. So something similar to Nixos sounds interesting. :-)

> PS As an aside, I've always wanted /etc/hosts to also have a parallel
> /etc/hosts.d/. It'd make maintaining things without a DNS server extremely
> easy - think dynamically adding and removing VMs in most cloud providers,
> especially those where multicast DNS doesn't work... like Azure. (Yes, I
> had a client that insisted on using it with Linux). Likewise it'd be nice
> to be able to add and remove DNS servers with a /etc/resolv.conf.d. Makes
> automated config and change management and audit that bit easier. (Debian
> do this using run-parts for lots of things for those sorts of reasons).

The philosophy used in musl, which is somewhat different from the sort
of philosophy you might have when designing a new distribution, is not
to invent new policy but to avoid policy and build on existing,
already-widely-accepteed policy when it's unavoidable. There are LOTS
of ways one could extend hostname lookups, ranging from NSS modules to
hosts.d and resolv.d, but rather than trying to support everything
imaginable (result: bloat and serious security considerations) in
libc, the musl approach to hostname lookup is that libc contains the
basics that are suitable for most/all simple systems, and anything
more advances can be provided by an external daemon running on
localhost that speaks DNS protocol and provides whatever lookup
semantics you desire.

Rich

Re: _PATH_LASTLOG

[Rich Felker]

On Tue, Dec 03, 2013 at 08:18:28PM +0000, Raphael Cohn wrote:
> Thanks for the pointers to doing stuff with PAM. Out of interest, if there
> is an interesting answer, is there a good place to write it up?

The wiki is the place for it in general, although I'm not sure where
in the current organization of the wiki would be best to put it. Note
that sabotage and Gregor's pkgsrc already have patches for a number of
packages but I don't think PAM is covered.

Rich

Re: _PATH_LASTLOG

[Laurent Bercot]

> One problem I'd like to solve is making a way for users to override
> the system resolv.conf;

  The s6-dns client library uses the DNSCACHEIP environment variable for
this: if it contains a list of DNS caches, this list will override the
/etc/resolv.conf-provided one. (The idea comes from djbdns, but has been
extended to a full list instead of a single cache address.)
  Same thing with the DNSQUALIFY environment variable, which can have
a list of suffixes that overrides resolv.conf. (djbdns had a complex
rules-rewriting-based qualification mechanism that nobody ever used,
so the simpler approach was easier and better.)

  Maybe musl could use the same approach: environment variables are a
reasonable place for hardcoded-path overrides. But it has to be balanced
against namespace pollution.


> This seems like a good foundation for a package system. I've looked
> into Nixos before but never really tried it out, and got the
> impression that the concept was very good but it might not be the best
> implementation. So something similar to Nixos sounds interesting. :-)

  I've always believed that the filesystem itself should be used as a
packaging system: every package should have its own system user and reside
in its own directory, and /usr/bin and friends should only contain
symlinks. Native isolation via Unix permissions, atomic package replacement,
easy package management. But for some reason, people seem absolutely
reluctant to do this.


> The philosophy used in musl, which is somewhat different from the sort
> of philosophy you might have when designing a new distribution, is not
> to invent new policy but to avoid policy and build on existing,
> already-widely-accepteed policy when it's unavoidable.

  I don't agree with all decisions in musl, but this one I can definitely
stand for.


> There are LOTS of ways one could extend hostname lookups, ranging from  NSS modules to
> hosts.d and resolv.d, but rather than trying to support everything
> imaginable (result: bloat and serious security considerations) in
> libc, the musl approach to hostname lookup is that libc contains the
> basics that are suitable for most/all simple systems, and anything
> more advances can be provided by an external daemon running on
> localhost that speaks DNS protocol and provides whatever lookup
> semantics you desire.

  In the DNS case, the flexible - and best, IMNSHO - approach is to run a
small local DNS cache on localhost indeed; but the problem is that there's
an existing codebase that sometimes insists on clobbering /etc/resolv.conf,
which adds to the packaging burden when your purpose is to create or maintain
a distribution. Having extension mechanisms at the libc level can help in
that situation.

-- 
  Laurent

Re: _PATH_LASTLOG

[Raphael Cohn]

[-- Attachment #1: Type: text/plain, Size: 6262 bytes --]

Q: Are you also talking about having separate configuration dirs...

Actually, not quite, but that also sounds like a very interesting idea with
a lot of potential. I suspect one needs to be careful that it wouldn't then
be possible for a nasty binary to define its own path, to say, password,
and allow it to be overridden. I'm thinking LD_LIBRARY_PATH here.
(Actually, it might be a good idea to be able to disable that inside the
libc for some set-ups).

What I'm thinking about is a system reconfiguration. When it's ready
(packages compiled, etc changes made, and so on), one flips a symbolic link.

Something we are toying with is 'what is config'. Some config really isn't
- once you have a source controlled, rebuildable system, there's no point
having a special /etc any more. In practice, not everything in /etc is
actually config. Some things are actually writable (eg the OATH Toolkit's
PAM does this). Some things never change unless the packages change, And a
few small things need to change occasionally. Such as users. But even then,
with most modern servers that actually only have a few admins, that can be
managed by package rebuilds...

Q: Nixos...
It's clever, but there's too much CompSci. The scripting language is
impressive but hard for sysadmins. What's more interesting is a build
system based around the lowest common denominator that's workable. Bash is
our current choice. We'd like to go ash, but fundamental things (eg support
for reading \0 delimited data for security when parsing file lists) isn't
supported.

Q: Philosophy
That'd go nicely in the wiki or the website! I've not got much problem
using a local daemon. We're probably adopt a variant of dnsmasq for what we
need to do. Personally, I loathe NSS and NIS and that world, but there you
go, it's out there and its used.

The downside is that sloppy programming propagates. Really, most DNS
lookups should be unified and use SRV records (or MX for a specialised use
case) and DNS Sec, but that's hard to do and if the library's not on the
box, it doesn't happen.


Raphael Cohn
Chief Architect, stormmq
Co-Chair, OASIS MQTT Standard
Secretary, OASIS AMQP Standard
raphael.cohn@stormmq.com
+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United
Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number
07175657
StormMQ.com


On 3 December 2013 20:25, Rich Felker <dalias@aerifal.cx> wrote:

> On Tue, Dec 03, 2013 at 08:10:56PM +0000, Raphael Cohn wrote:
> > Thanks for that - just having a list is an useful place to start. I think
> > the default file names are quite sensible - especially for a common
> > run-anywhere use case. And some - where mandated by POSIX - probably
> should
> > never change. What would be nice might be to be able to define the prefix
> > for /etc to something else (so we can use atomic symlink changes to flip
> > configs).
>
> Are you also talking about having separate configuration dirs for each
> package? Would each package need its own user database (passwd file),
> DNS configuration (which the DHCP client would have to update), etc.?
> The intent in musl is that the "config"-like files it reads from /etc
> are not things that should be application-local but system-global.
> Obviously there are different views on what's local/global, though.
> One problem I'd like to solve is making a way for users to override
> the system resolv.conf; this would be especially valuable for us (musl
> developers) writing test cases for the resolver, in that we could
> redirect lookups to a fake DNS daemon running in a separate thread of
> the test program serving fixed, possibly-intentionally-malformed
> replies.
>
> > I'd like to have more of a think about the other paths. We're only a
> short
> > way into our project, so our ideas might change. What we're looking at
> is a
> > Nixos-like linux, where we rebuild only packages because other packages
> > have changed. We want to keep every package isolated, so we can apply
> PATH
> > controls, fine-grained capability permissions, chattr -ai, etc. Part of
> > doing this means we don't want paths 'hanging around' inside libraries
> that
> > are used if present - as these allow an attacker (or more likely, a duff
> > package) to accidentally stop itself working, ie if there's no /usr/lib
> on
> > system, then nothing should be able to stick itself in /usr/lib and
> > override the system setup.
>
> This seems like a good foundation for a package system. I've looked
> into Nixos before but never really tried it out, and got the
> impression that the concept was very good but it might not be the best
> implementation. So something similar to Nixos sounds interesting. :-)
>
> > PS As an aside, I've always wanted /etc/hosts to also have a parallel
> > /etc/hosts.d/. It'd make maintaining things without a DNS server
> extremely
> > easy - think dynamically adding and removing VMs in most cloud providers,
> > especially those where multicast DNS doesn't work... like Azure. (Yes, I
> > had a client that insisted on using it with Linux). Likewise it'd be nice
> > to be able to add and remove DNS servers with a /etc/resolv.conf.d. Makes
> > automated config and change management and audit that bit easier. (Debian
> > do this using run-parts for lots of things for those sorts of reasons).
>
> The philosophy used in musl, which is somewhat different from the sort
> of philosophy you might have when designing a new distribution, is not
> to invent new policy but to avoid policy and build on existing,
> already-widely-accepteed policy when it's unavoidable. There are LOTS
> of ways one could extend hostname lookups, ranging from NSS modules to
> hosts.d and resolv.d, but rather than trying to support everything
> imaginable (result: bloat and serious security considerations) in
> libc, the musl approach to hostname lookup is that libc contains the
> basics that are suitable for most/all simple systems, and anything
> more advances can be provided by an external daemon running on
> localhost that speaks DNS protocol and provides whatever lookup
> semantics you desire.
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 7495 bytes --]

Re: _PATH_LASTLOG

[Laurent Bercot]

> What would be nice might be to be able to define the prefix for /etc to something else (so we can use atomic symlink changes to flip configs).

  Make /etc a symlink to /var/config/etc. Make /var/config a symlink to config-0.
Flip configs by atomically replacing the /var/config symlinks.
  You'll have to move /etc, or part of it, out of the root filesystem anyway, because
you'll want to make your / read-only: so, kill two birds with one stone.


> PS As an aside, I've always wanted /etc/hosts to also have a parallel /etc/hosts.d/.
>  It'd make maintaining things without a DNS server extremely easy - think dynamically
>  adding and removing VMs in most cloud providers, especially those where multicast DNS
>  doesn't work... like Azure.

  Honestly, using djbdns is almost as easy, thanks to the programmatically sane config
file formats; and it will scale better.

-- 
  Laurent

Re: _PATH_LASTLOG

[Raphael Cohn]

[-- Attachment #1: Type: text/plain, Size: 3529 bytes --]

I've always believed that the filesystem itself should be used as a
packaging system...

This is EXACTLY what I want to do. Each package is its own FHS...

Raphael Cohn
Chief Architect, stormmq
Co-Chair, OASIS MQTT Standard
Secretary, OASIS AMQP Standard
raphael.cohn@stormmq.com
+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United
Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number
07175657
StormMQ.com


On 3 December 2013 20:44, Laurent Bercot <ska-dietlibc@skarnet.org> wrote:

>
>  One problem I'd like to solve is making a way for users to override
>> the system resolv.conf;
>>
>
>  The s6-dns client library uses the DNSCACHEIP environment variable for
> this: if it contains a list of DNS caches, this list will override the
> /etc/resolv.conf-provided one. (The idea comes from djbdns, but has been
> extended to a full list instead of a single cache address.)
>  Same thing with the DNSQUALIFY environment variable, which can have
> a list of suffixes that overrides resolv.conf. (djbdns had a complex
> rules-rewriting-based qualification mechanism that nobody ever used,
> so the simpler approach was easier and better.)
>
>  Maybe musl could use the same approach: environment variables are a
> reasonable place for hardcoded-path overrides. But it has to be balanced
> against namespace pollution.
>
>
>
>  This seems like a good foundation for a package system. I've looked
>> into Nixos before but never really tried it out, and got the
>> impression that the concept was very good but it might not be the best
>> implementation. So something similar to Nixos sounds interesting. :-)
>>
>
>  I've always believed that the filesystem itself should be used as a
> packaging system: every package should have its own system user and reside
> in its own directory, and /usr/bin and friends should only contain
> symlinks. Native isolation via Unix permissions, atomic package
> replacement,
> easy package management. But for some reason, people seem absolutely
> reluctant to do this.
>
>
>
>  The philosophy used in musl, which is somewhat different from the sort
>> of philosophy you might have when designing a new distribution, is not
>> to invent new policy but to avoid policy and build on existing,
>> already-widely-accepteed policy when it's unavoidable.
>>
>
>  I don't agree with all decisions in musl, but this one I can definitely
> stand for.
>
>
>
>  There are LOTS of ways one could extend hostname lookups, ranging from
>>  NSS modules to
>> hosts.d and resolv.d, but rather than trying to support everything
>> imaginable (result: bloat and serious security considerations) in
>> libc, the musl approach to hostname lookup is that libc contains the
>> basics that are suitable for most/all simple systems, and anything
>> more advances can be provided by an external daemon running on
>> localhost that speaks DNS protocol and provides whatever lookup
>> semantics you desire.
>>
>
>  In the DNS case, the flexible - and best, IMNSHO - approach is to run a
> small local DNS cache on localhost indeed; but the problem is that there's
> an existing codebase that sometimes insists on clobbering /etc/resolv.conf,
> which adds to the packaging burden when your purpose is to create or
> maintain
> a distribution. Having extension mechanisms at the libc level can help in
> that situation.
>
> --
>  Laurent
>
>

[-- Attachment #2: Type: text/html, Size: 4898 bytes --]

Re: _PATH_LASTLOG

[Raphael Cohn]

[-- Attachment #1: Type: text/plain, Size: 1523 bytes --]

Linking /etc to something in /var was what we were going to do...

Raphael Cohn
Chief Architect, stormmq
Co-Chair, OASIS MQTT Standard
Secretary, OASIS AMQP Standard
raphael.cohn@stormmq.com
+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United
Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number
07175657
StormMQ.com


On 3 December 2013 20:51, Laurent Bercot <ska-dietlibc@skarnet.org> wrote:

>
>  What would be nice might be to be able to define the prefix for /etc to
>> something else (so we can use atomic symlink changes to flip configs).
>>
>
>  Make /etc a symlink to /var/config/etc. Make /var/config a symlink to
> config-0.
> Flip configs by atomically replacing the /var/config symlinks.
>  You'll have to move /etc, or part of it, out of the root filesystem
> anyway, because
> you'll want to make your / read-only: so, kill two birds with one stone.
>
>
>
>  PS As an aside, I've always wanted /etc/hosts to also have a parallel
>> /etc/hosts.d/.
>>  It'd make maintaining things without a DNS server extremely easy - think
>> dynamically
>>  adding and removing VMs in most cloud providers, especially those where
>> multicast DNS
>>  doesn't work... like Azure.
>>
>
>  Honestly, using djbdns is almost as easy, thanks to the programmatically
> sane config
> file formats; and it will scale better.
>
> --
>  Laurent
>
>

[-- Attachment #2: Type: text/html, Size: 2467 bytes --]

Re: _PATH_LASTLOG

[Nathan McSween]

You can implement this with bind mounts and containers as well, maybe 
when Linux gets union mounts within the vfs this can be done cleanly 
and per container.

On Tuesday, December 03, 2013 12:51:07 PM, Raphael Cohn wrote:
> I've always believed that the filesystem itself should be used as a
> packaging system...
>
> This is EXACTLY what I want to do. Each package is its own FHS...
>
> Raphael Cohn
> Chief Architect, stormmq
> Co-Chair, OASIS MQTT Standard
> Secretary, OASIS AMQP Standard
> raphael.cohn@stormmq.com
> +44 7590 675 756
>
> UK Office:
> Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United
> Kingdom
> Telephone: +44 845 3712 567
>
> Registered office:
> 16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
> StormMQ Limited is Registered in England and Wales under Company Number
> 07175657
> StormMQ.com
>
>
> On 3 December 2013 20:44, Laurent Bercot <ska-dietlibc@skarnet.org> wrote:
>
>>
>>  One problem I'd like to solve is making a way for users to override
>>> the system resolv.conf;
>>>
>>
>>  The s6-dns client library uses the DNSCACHEIP environment variable for
>> this: if it contains a list of DNS caches, this list will override the
>> /etc/resolv.conf-provided one. (The idea comes from djbdns, but has been
>> extended to a full list instead of a single cache address.)
>>  Same thing with the DNSQUALIFY environment variable, which can have
>> a list of suffixes that overrides resolv.conf. (djbdns had a complex
>> rules-rewriting-based qualification mechanism that nobody ever used,
>> so the simpler approach was easier and better.)
>>
>>  Maybe musl could use the same approach: environment variables are a
>> reasonable place for hardcoded-path overrides. But it has to be balanced
>> against namespace pollution.
>>
>>
>>
>>  This seems like a good foundation for a package system. I've looked
>>> into Nixos before but never really tried it out, and got the
>>> impression that the concept was very good but it might not be the best
>>> implementation. So something similar to Nixos sounds interesting. :-)
>>>
>>
>>  I've always believed that the filesystem itself should be used as a
>> packaging system: every package should have its own system user and reside
>> in its own directory, and /usr/bin and friends should only contain
>> symlinks. Native isolation via Unix permissions, atomic package
>> replacement,
>> easy package management. But for some reason, people seem absolutely
>> reluctant to do this.
>>
>>
>>
>>  The philosophy used in musl, which is somewhat different from the sort
>>> of philosophy you might have when designing a new distribution, is not
>>> to invent new policy but to avoid policy and build on existing,
>>> already-widely-accepteed policy when it's unavoidable.
>>>
>>
>>  I don't agree with all decisions in musl, but this one I can definitely
>> stand for.
>>
>>
>>
>>  There are LOTS of ways one could extend hostname lookups, ranging from
>>>  NSS modules to
>>> hosts.d and resolv.d, but rather than trying to support everything
>>> imaginable (result: bloat and serious security considerations) in
>>> libc, the musl approach to hostname lookup is that libc contains the
>>> basics that are suitable for most/all simple systems, and anything
>>> more advances can be provided by an external daemon running on
>>> localhost that speaks DNS protocol and provides whatever lookup
>>> semantics you desire.
>>>
>>
>>  In the DNS case, the flexible - and best, IMNSHO - approach is to run a
>> small local DNS cache on localhost indeed; but the problem is that there's
>> an existing codebase that sometimes insists on clobbering /etc/resolv.conf,
>> which adds to the packaging burden when your purpose is to create or
>> maintain
>> a distribution. Having extension mechanisms at the libc level can help in
>> that situation.
>>
>> --
>>  Laurent
>>
>>
>

Re: _PATH_LASTLOG

[Raphael Cohn]

[-- Attachment #1: Type: text/plain, Size: 5222 bytes --]

Yes, we can - the only problem being that a lot of the systems I have
access to have either broken implementations or lack the necessary stuff in
the kernel.UnionFS' code base seems very complex. And bind mounts are very
hard to track by 'just looking at the file system'. We're very keen on
something that has good visual audit qualities, ie a sysadmin can look at
it and go 'that doesn't like right...' We'd also need a lot of bind
mounts...

The performance argument from the few tests I did does seem moot - symlink
look up is almost instantaneous on a SSD. On rust it might be different.
But rust isn't where we're going...


Raphael Cohn
Chief Architect, stormmq
Co-Chair, OASIS MQTT Standard
Secretary, OASIS AMQP Standard
raphael.cohn@stormmq.com
+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United
Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number
07175657
StormMQ.com


On 3 December 2013 21:00, Nathan McSween <nwmcsween@gmail.com> wrote:

> You can implement this with bind mounts and containers as well, maybe
> when Linux gets union mounts within the vfs this can be done cleanly
> and per container.
>
> On Tuesday, December 03, 2013 12:51:07 PM, Raphael Cohn wrote:
> > I've always believed that the filesystem itself should be used as a
> > packaging system...
> >
> > This is EXACTLY what I want to do. Each package is its own FHS...
> >
> > Raphael Cohn
> > Chief Architect, stormmq
> > Co-Chair, OASIS MQTT Standard
> > Secretary, OASIS AMQP Standard
> > raphael.cohn@stormmq.com
> > +44 7590 675 756
> >
> > UK Office:
> > Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United
> > Kingdom
> > Telephone: +44 845 3712 567
> >
> > Registered office:
> > 16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
> > StormMQ Limited is Registered in England and Wales under Company Number
> > 07175657
> > StormMQ.com
> >
> >
> > On 3 December 2013 20:44, Laurent Bercot <ska-dietlibc@skarnet.org>
> wrote:
> >
> >>
> >>  One problem I'd like to solve is making a way for users to override
> >>> the system resolv.conf;
> >>>
> >>
> >>  The s6-dns client library uses the DNSCACHEIP environment variable for
> >> this: if it contains a list of DNS caches, this list will override the
> >> /etc/resolv.conf-provided one. (The idea comes from djbdns, but has been
> >> extended to a full list instead of a single cache address.)
> >>  Same thing with the DNSQUALIFY environment variable, which can have
> >> a list of suffixes that overrides resolv.conf. (djbdns had a complex
> >> rules-rewriting-based qualification mechanism that nobody ever used,
> >> so the simpler approach was easier and better.)
> >>
> >>  Maybe musl could use the same approach: environment variables are a
> >> reasonable place for hardcoded-path overrides. But it has to be balanced
> >> against namespace pollution.
> >>
> >>
> >>
> >>  This seems like a good foundation for a package system. I've looked
> >>> into Nixos before but never really tried it out, and got the
> >>> impression that the concept was very good but it might not be the best
> >>> implementation. So something similar to Nixos sounds interesting. :-)
> >>>
> >>
> >>  I've always believed that the filesystem itself should be used as a
> >> packaging system: every package should have its own system user and
> reside
> >> in its own directory, and /usr/bin and friends should only contain
> >> symlinks. Native isolation via Unix permissions, atomic package
> >> replacement,
> >> easy package management. But for some reason, people seem absolutely
> >> reluctant to do this.
> >>
> >>
> >>
> >>  The philosophy used in musl, which is somewhat different from the sort
> >>> of philosophy you might have when designing a new distribution, is not
> >>> to invent new policy but to avoid policy and build on existing,
> >>> already-widely-accepteed policy when it's unavoidable.
> >>>
> >>
> >>  I don't agree with all decisions in musl, but this one I can definitely
> >> stand for.
> >>
> >>
> >>
> >>  There are LOTS of ways one could extend hostname lookups, ranging from
> >>>  NSS modules to
> >>> hosts.d and resolv.d, but rather than trying to support everything
> >>> imaginable (result: bloat and serious security considerations) in
> >>> libc, the musl approach to hostname lookup is that libc contains the
> >>> basics that are suitable for most/all simple systems, and anything
> >>> more advances can be provided by an external daemon running on
> >>> localhost that speaks DNS protocol and provides whatever lookup
> >>> semantics you desire.
> >>>
> >>
> >>  In the DNS case, the flexible - and best, IMNSHO - approach is to run a
> >> small local DNS cache on localhost indeed; but the problem is that
> there's
> >> an existing codebase that sometimes insists on clobbering
> /etc/resolv.conf,
> >> which adds to the packaging burden when your purpose is to create or
> >> maintain
> >> a distribution. Having extension mechanisms at the libc level can help
> in
> >> that situation.
> >>
> >> --
> >>  Laurent
> >>
> >>
> >
>

[-- Attachment #2: Type: text/html, Size: 6991 bytes --]

Re: _PATH_LASTLOG

[Rich Felker]

On Tue, Dec 03, 2013 at 08:44:01PM +0000, Laurent Bercot wrote:
> 
> >One problem I'd like to solve is making a way for users to override
> >the system resolv.conf;
> 
>  The s6-dns client library uses the DNSCACHEIP environment variable for
> this: if it contains a list of DNS caches, this list will override the
> /etc/resolv.conf-provided one. (The idea comes from djbdns, but has been
> extended to a full list instead of a single cache address.)
>  Same thing with the DNSQUALIFY environment variable, which can have
> a list of suffixes that overrides resolv.conf. (djbdns had a complex
> rules-rewriting-based qualification mechanism that nobody ever used,
> so the simpler approach was easier and better.)
> 
>  Maybe musl could use the same approach: environment variables are a
> reasonable place for hardcoded-path overrides. But it has to be balanced
> against namespace pollution.

Indeed. If we go with environment variables, I'd like to aim to avoid
reinventing the wheel and use something already supported elsewhere.
But as far as I can tell, other libcs don't offer such a feature, and
it's questionable whether we should be using variables that were
"intended" (by a third party library) to affect that library and not
libc. This is an area we really need to do more research in to
determine a safe solution. (BTW, whatever we do, suid programs
(libc.secure != 0) should always ignore the custom setting.)

> >There are LOTS of ways one could extend hostname lookups, ranging from  NSS modules to
> >hosts.d and resolv.d, but rather than trying to support everything
> >imaginable (result: bloat and serious security considerations) in
> >libc, the musl approach to hostname lookup is that libc contains the
> >basics that are suitable for most/all simple systems, and anything
> >more advances can be provided by an external daemon running on
> >localhost that speaks DNS protocol and provides whatever lookup
> >semantics you desire.
> 
>  In the DNS case, the flexible - and best, IMNSHO - approach is to run a
> small local DNS cache on localhost indeed; but the problem is that there's
> an existing codebase that sometimes insists on clobbering /etc/resolv.conf,
> which adds to the packaging burden when your purpose is to create or maintain
> a distribution. Having extension mechanisms at the libc level can help in
> that situation.

chattr +i is your friend. :-)

Rich

Re: _PATH_LASTLOG

[Rich Felker]

On Tue, Dec 03, 2013 at 01:00:26PM -0800, Nathan McSween wrote:
> You can implement this with bind mounts and containers as well, maybe 
> when Linux gets union mounts within the vfs this can be done cleanly 
> and per container.

I'm not holding my breath for union mounts. The likely candidate to go
in, overlayfs, has HORRIBLY WRONG semantics with respect to inodes and
file identity. For example if you have an open file descriptor to a
given file for read and another process opens the file for write, the
dev/inode changes and you don't see any of the changes in your
existing open file. It's as if open (for write) also performs unlink
and O_CREAT behind your back.

Making a union mount filesystem with proper semantics would require
some fairly heavy machinery to give consistent inode behavior.

Rich