R/GNU S: up with a couple hitches...

R/GNU S: up with a couple hitches...

[idunham]

Cleaning up my Drafts folder, I found this:

I decided to give my gcc-3.4 build a workout (including the g77 fortran
compiler) by building R (a/k/a GNU S).
R is a fairly large stats package that uses C and Fortran.
Here are the issues I encountered:
1. iconv() needs UCS-* support: I used
LD_PRELOAD=preloadable_libiconv.so
2. __libc_stack_end is *apparently* assumed present on linux, rather than
properly checked for. (This breaks build in src/unix/system.c) So you need
to change (in src/include/config.h)
#define HAVE_LIBC_STACK_END 1
to
#undef HAVE_LIBC_STACK_END

There is fallback code, but currently it seems to just be guessing the
location of the stack end (if I understand properly).

After these, R builds.  It runs, but does not pass tests.

Isaac Dunham

Re: R/GNU S: up with a couple hitches...

[Rich Felker]

On Thu, Sep 13, 2012 at 12:29:07AM -0400, idunham@lavabit.com wrote:
> Cleaning up my Drafts folder, I found this:
> 
> I decided to give my gcc-3.4 build a workout (including the g77 fortran
> compiler) by building R (a/k/a GNU S).
> R is a fairly large stats package that uses C and Fortran.
> Here are the issues I encountered:
> 1. iconv() needs UCS-* support: I used
> LD_PRELOAD=preloadable_libiconv.so

Hmm, which UCS-* is musl missing? It's supposed to be there.

> 2. __libc_stack_end is *apparently* assumed present on linux, rather than
> properly checked for. (This breaks build in src/unix/system.c) So you need
> to change (in src/include/config.h)
> #define HAVE_LIBC_STACK_END 1
> to
> #undef HAVE_LIBC_STACK_END

I think this is another case of improper use of an internal symbol. In
case we do want to add support for it though, do you know if it's
supposed to be a symbol whose address is the end of the stack, or a
pointer variable pointing to the end of the stack? My guess would be
the latter since the former isn't really possible if the kernel can
randomize the stack location.

> There is fallback code, but currently it seems to just be guessing the
> location of the stack end (if I understand properly).

Basically if the stack grows downward, any address >= an automatic
variable's address is also a stack address. This is true on any
platform I know of.

> After these, R builds.  It runs, but does not pass tests.

Any idea why?

Rich

Re: R/GNU S: up with a couple hitches...

[Daniel Cegiełka]

R requires Sun's rpc (xdr) and musl doesn't support it now, but we can
use a external library:

http://sourceforge.net/projects/libtirpc/

Daniel

Re: R/GNU S: up with a couple hitches...

[Isaac Dunham]

On Thu, 13 Sep 2012 09:08:17 +0200
Daniel Cegiełka <daniel.cegielka@gmail.com> wrote:

> R requires Sun's rpc (xdr) 
Somehow that didn't seem to show up (it built "successfully")...how did you discover this?
> and musl doesn't support it now, but we can
> use a external library:
> 
> http://sourceforge.net/projects/libtirpc/
Not yet, unless you're using the g_hack branch and don't need rpc to actually _work_. libtirpc needs getifaddrs.
I planned to see if "Apple RPC" (IIRC, based on Novell's update of the Open Group's dce-rpc 1.2.2, which latter released as LGPL in 2005) built with musl. Unfortunately, that has to wait till the laptop I've been using for musl experimentation gets a new AC adapter.

-- 
Isaac Dunham <idunham@lavabit.com>

Re: R/GNU S: up with a couple hitches...

[Rich Felker]

On Fri, Sep 14, 2012 at 08:21:07PM -0700, Isaac Dunham wrote:
> On Thu, 13 Sep 2012 09:08:17 +0200
> Daniel Cegiełka <daniel.cegielka@gmail.com> wrote:
> 
> > R requires Sun's rpc (xdr) 
> Somehow that didn't seem to show up (it built "successfully")...how did you discover this?
> > and musl doesn't support it now, but we can
> > use a external library:
> > 
> > http://sourceforge.net/projects/libtirpc/
> Not yet, unless you're using the g_hack branch and don't need rpc to
> actually _work_. libtirpc needs getifaddrs.

Is the getifaddrs code there license-compatible with musl?

Rich

Re: R/GNU S: up with a couple hitches...

[Isaac Dunham]

On Fri, 14 Sep 2012 23:51:10 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> On Fri, Sep 14, 2012 at 08:21:07PM -0700, Isaac Dunham wrote:

> > > and musl doesn't support it now, but we can
> > > use a external library:
> > > 
> > > http://sourceforge.net/projects/libtirpc/
> > Not yet, unless you're using the g_hack branch and don't need rpc to
> > actually _work_. libtirpc needs getifaddrs.
> 
> Is the getifaddrs code there license-compatible with musl?
Yes but it's only a stub I wrote (hence the comment about "don't need rpc to actually _work_"). It will almost certainly leave any software that uses getifaddrs crippled. 
And no, I' haven't tested what builds with it. Probably nothing in its present state (I don't have a proper header yet).
Also, I have mentioned once or twice that getifaddrs, when implemented, will require (BSD) <sys/queue.h> TAILQ_* macros. If any header contains copyrightable material, it's this one: hundreds of LOCs, implementing linked lists in macros.

HTH,
Isaac Dunham <idunham@lavabit.com>

Re: R/GNU S: up with a couple hitches...

[idunham]

> On Fri, 14 Sep 2012 23:51:10 -0400
> Rich Felker <dalias@aerifal.cx> wrote:
>
>> On Fri, Sep 14, 2012 at 08:21:07PM -0700, Isaac Dunham wrote:
...
>> > Not yet, unless you're using the g_hack branch and don't need rpc to
>> > actually _work_. libtirpc needs getifaddrs.
>>
>> Is the getifaddrs code there license-compatible with musl?
> Yes but it's only a stub I wrote (hence the comment about "don't need rpc
> to actually _work_"). It will almost certainly leave any software that
> uses getifaddrs crippled.
> And no, I' haven't tested what builds with it. Probably nothing in its
> present state (I don't have a proper header yet).

I have since gone looking and found that newlib has a BSDi-based version,
which looks like it might be usable on some arches with some minor (?)
changes and cleanup.  There's a note about one codepath not being LP64
friendly, though, as well as one abort() that should be turned into a
retry or failure. (Apple used the same base for implementing getifaddrs,
FWIW)
Refer to newlib/libc/sys/linux/net/getifaddrs.c in the newlib source tree.
Not sure if
http://www.kegel.com/dkftpbench/dkftpbench-0.45/getifaddrs.c
ftp://ftp.inet.no/pub/barefoot/extracted/libscompat/getifaddrs.c
(elinks) src/osdep/getifaddrs.c
are worthwhile, though they are compatibly licensed (BSDi or 3-clause
BSDish).

> Also, I have mentioned once or twice that getifaddrs, when implemented,
> will require (BSD) <sys/queue.h> TAILQ_* macros. If any header contains
> copyrightable material, it's this one: hundreds of LOCs, implementing
> linked lists in macros.
<sys/queue.h> will only be needed for source compatibility only, not for
implementing it.

Re: R/GNU S: up with a couple hitches...

[Daniel Cegiełka]

2012/9/15 Isaac Dunham <idunham@lavabit.com>:
> On Thu, 13 Sep 2012 09:08:17 +0200
> Daniel Cegiełka <daniel.cegielka@gmail.com> wrote:
>
>> R requires Sun's rpc (xdr)
> Somehow that didn't seem to show up (it built "successfully")...how did you discover this?

it wasn't so difficult :)

http://cran.r-project.org/doc/manuals/R-admin.html#Essential-programs-and-libraries

"An implementation of XDR is required. This is part of RPC and
historically has been part of libc on a Unix-alike: however some
builds26 of glibc 2.14 hide it. The intention seems to be that the
TI-RPC library be used instead, in which case libtirpc (and its
development version) needs to be installed, and its headers need to be
on the C include path (and configure tries /usr/include/tirpc if the
headers are not found on the standard include path). The R sources
contain a simple implementation of XDR which in recent versions can be
used on platforms with 32-bit or 64-bit long (and earlier ones will
fail to compile unless long is 32-bit)."

Daniel

Re: R/GNU S: up with a couple hitches...

[idunham]

> 2012/9/15 Isaac Dunham <idunham@lavabit.com>:
>> On Thu, 13 Sep 2012 09:08:17 +0200
>> Daniel Cegiełka <daniel.cegielka@gmail.com> wrote:
>>
>>> R requires Sun's rpc (xdr)
>> Somehow that didn't seem to show up (it built "successfully")...how did
>> you discover this?
>
> it wasn't so difficult :)
>
> http://cran.r-project.org/doc/manuals/R-admin.html#Essential-programs-and-libraries

That doesn't indicate what you said:
> "*The R sources contain a simple implementation of XDR*
> which in recent versions can be
> used on platforms with 32-bit or 64-bit long (and earlier ones will
> fail to compile unless long is 32-bit)."
>

But I've discovered that there is also bsd-xdr on googlecode, which I
might add to muslin down the road.

BTW, Rich, here's what that page says about iconv:
| The R usage requires iconv to be able to translate between "latin1"
| and "UTF-8", to recognize "" (as the current encoding) and "ASCII",
| and to translate to and from the Unicode wide-character formats
| "UCS-[24][BL]E"
How much of this should musl support?

Isaac Dunham

Re: R/GNU S: up with a couple hitches...

[Rich Felker]

On Tue, Sep 18, 2012 at 08:19:17PM -0400, idunham@lavabit.com wrote:
> BTW, Rich, here's what that page says about iconv:
> | The R usage requires iconv to be able to translate between "latin1"
> | and "UTF-8", to recognize "" (as the current encoding) and "ASCII",
> | and to translate to and from the Unicode wide-character formats
> | "UCS-[24][BL]E"
> How much of this should musl support?

I've never heard of using "" (blank string) to mean "the current
encoding". If it's documented usage for glibc, it must be buried
somewhere in the docs; it's definitely not a standard usage. The
standard way to get the locale's encoding is nl_langinfo(CODESET); on
any good implementation it will be accepted as an argument to
iconv_open.

As for the others (UTF-8, latin1, UCS-[24][BL]E, and ASCII) they
should all be accepted just fine. But it would probably be smarter to
use the standard name for latin1, ISO-8859-1.

Rich

Re: R/GNU S: up with a couple hitches...

[idunham]

> On Tue, Sep 18, 2012 at 08:19:17PM -0400, idunham@lavabit.com wrote:
>> BTW, Rich, here's what that page says about iconv:
>> | The R usage requires iconv to be able to translate between "latin1"
>> | and "UTF-8", to recognize "" (as the current encoding) and "ASCII",
>> | and to translate to and from the Unicode wide-character formats
>> | "UCS-[24][BL]E"
>> How much of this should musl support?
>
> I've never heard of using "" (blank string) to mean "the current
> encoding". If it's documented usage for glibc, it must be buried
> somewhere in the docs; it's definitely not a standard usage. The
> standard way to get the locale's encoding is nl_langinfo(CODESET); on
> any good implementation it will be accepted as an argument to
> iconv_open.
Here's what the glibc docs say
(http://www.gnu.org/software/libc/manual/html_node/Standard-Locales.html#Standard-Locales):
===
The only locale names you can count on finding on all operating systems
are these three standard ones:

"C"
    This is the standard C locale. The attributes and behavior it provides
are specified in the ISO C standard. When your program starts up, it
initially uses this locale by default.
"POSIX"
    This is the standard POSIX locale. Currently, it is an alias for the
standard C locale.
""
    The empty name says to select a locale based on environment variables.
See Locale Categories.
===
Not that they accurately represent facts outside glibc!

> As for the others (UTF-8, latin1, UCS-[24][BL]E, and ASCII) they
> should all be accepted just fine. But it would probably be smarter to
> use the standard name for latin1, ISO-8859-1.
>
> Rich
>
> ____________________________________________________________________________________
> Have questions about Online? Let Ask Find Your Answers Today!
> http://click.lavabit.com/ma8j8371x6rc36grpdg4csc38eh4wfjauifmaofnbuqiz13tbtfb/
> ____________________________________________________________________________________
>

Re: R/GNU S: up with a couple hitches...

[Rich Felker]

On Wed, Sep 19, 2012 at 12:44:41AM -0400, idunham@lavabit.com wrote:
> > On Tue, Sep 18, 2012 at 08:19:17PM -0400, idunham@lavabit.com wrote:
> >> BTW, Rich, here's what that page says about iconv:
> >> | The R usage requires iconv to be able to translate between "latin1"
> >> | and "UTF-8", to recognize "" (as the current encoding) and "ASCII",
> >> | and to translate to and from the Unicode wide-character formats
> >> | "UCS-[24][BL]E"
> >> How much of this should musl support?
> >
> > I've never heard of using "" (blank string) to mean "the current
> > encoding". If it's documented usage for glibc, it must be buried
> > somewhere in the docs; it's definitely not a standard usage. The
> > standard way to get the locale's encoding is nl_langinfo(CODESET); on
> > any good implementation it will be accepted as an argument to
> > iconv_open.
> Here's what the glibc docs say
> (http://www.gnu.org/software/libc/manual/html_node/Standard-Locales.html#Standard-Locales):
> ===
> The only locale names you can count on finding on all operating systems
> are these three standard ones:
> 
> "C"
>     This is the standard C locale. The attributes and behavior it provides
> are specified in the ISO C standard. When your program starts up, it
> initially uses this locale by default.
> "POSIX"
>     This is the standard POSIX locale. Currently, it is an alias for the
> standard C locale.
> ""
>     The empty name says to select a locale based on environment variables.
> See Locale Categories.
> ===
> Not that they accurately represent facts outside glibc!

Those are _locale_ names. Nothing to do with charset names passed to
iconv.

Rich

Re: R/GNU S: up with a couple hitches...

[Daniel Cegiełka]

2012/9/13  <idunham@lavabit.com>:
> Cleaning up my Drafts folder, I found this:
>

> After these, R builds.  It runs, but does not pass tests.

What exactly is checked during these tests?

Daniel

>
> Isaac Dunham
>