Bug on fstat()?

Bug on fstat()?

[piranna]

I have been able to compile Node.js using musl to create a fully
statically linked executable (SSL disabled, OpenSSL was giving too
much problems...), and although Node.js REPL works, it can't be able
to load script files.

Digging on the code I have got to the point that Node.js do a fstat()
call to know the size of the script file so it can fill a buffer of
the correct size, problem is that fstat() is giving me some crazy
values. I did the same before using glibc and it was able to load
simple script files, so seems to be a problem related to musl (or the
environment I've crafted to be able to compile Node.js using musl).

The simple test file is:

#!./node
while(true)
{
  console.log('Hello NodeOS! :-)')
};

./node is the musl-compiled static Node.js executable. Executing this
file from console give me the next error, the same error is thrown by
dispatching it as parameter of the Node.js executable:

fs.js:505
  var r = binding.read(fd, buffer, offset, length, position);
                  ^
Error: Offset is out of bounds
    at Error (native)
    at Object.fs.readSync (fs.js:505:19)
    at Object.fs.readFileSync (fs.js:346:28)
    at Object.Module._extensions..js (module.js:477:20)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:124:16)
    at node.js:811:3

The values that fstat() is returning are:

{ dev: 2051,
  mode: 33261,
  nlink: 1,
  uid: 1000,
  gid: 1000,
  rdev: 0,
  blksize: 8,
  ino: 584753551390482400,
  size: 17592186044477,
  blocks: 1485010404567043000,
  atime: Sun Aug 24 2014 22:33:06 GMT+0200 (CEST),
  mtime: Sun Aug 24 2014 22:33:06 GMT+0200 (CEST),
  ctime: Thu Jan 01 1970 01:00:00 GMT+0100 (CET),
  birthtime: Thu Jan 01 1970 01:00:00 GMT+0100 (CET) }

As you can see, is giving a size of 164GBs for a file of just 68
characters :-) ino, blocks and blksize have some strange values too,
but the other fields seems to be normal. As I told you, fstat() give a
correct output while compiling Node.js with glibc.

I'm using musl with the musl-gcc wrapper, and since it was giving me
problems related to some includes not available, I've done some (not
very clean, I know) symbolic links to them:

    sudo ln -s /usr/include/linux
/usr/include/i386-linux-musl/linux
    sudo ln -s /usr/include/i386-linux-gnu/asm /usr/include/i386-linux-musl/asm
    sudo ln -s /usr/include/asm-generic
/usr/include/i386-linux-musl/asm-generic

Any advice about what could be happening? How could I be able to solve it?


-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Re: Bug on fstat()?

[Rich Felker]

On Sun, Aug 24, 2014 at 10:51:09PM +0200, piranna@gmail.com wrote:
> The values that fstat() is returning are:
> 
> { dev: 2051,
>   mode: 33261,
>   nlink: 1,
>   uid: 1000,
>   gid: 1000,
>   rdev: 0,
>   blksize: 8,
>   ino: 584753551390482400,
>   size: 17592186044477,
>   blocks: 1485010404567043000,
>   atime: Sun Aug 24 2014 22:33:06 GMT+0200 (CEST),
>   mtime: Sun Aug 24 2014 22:33:06 GMT+0200 (CEST),
>   ctime: Thu Jan 01 1970 01:00:00 GMT+0100 (CET),
>   birthtime: Thu Jan 01 1970 01:00:00 GMT+0100 (CET) }
> 
> As you can see, is giving a size of 164GBs for a file of just 68
> characters :-) ino, blocks and blksize have some strange values too,
> but the other fields seems to be normal. As I told you, fstat() give a
> correct output while compiling Node.js with glibc.
> 
> I'm using musl with the musl-gcc wrapper, and since it was giving me
> problems related to some includes not available, I've done some (not
> very clean, I know) symbolic links to them:
> 
>     sudo ln -s /usr/include/linux
> /usr/include/i386-linux-musl/linux
>     sudo ln -s /usr/include/i386-linux-gnu/asm /usr/include/i386-linux-musl/asm
>     sudo ln -s /usr/include/asm-generic
> /usr/include/i386-linux-musl/asm-generic
> 
> Any advice about what could be happening? How could I be able to solve it?

What prefix is musl-gcc installed in? And have you done anything to
modify it or added additional include paths for it? The above symlinks
don't make sense for a typical out-of-the-box installation. I suspect
you've somehow gotten -I/usr/include added to the CFLAGS during build
(either doing it yourself, or as a result of a broken build script) if
adding symlinks to /usr/include made a difference, in which case you
may be using glibc's headers, not musl's, and thus getting broken
results.

Rich

Re: Bug on fstat()?

[piranna]

> What prefix is musl-gcc installed in?

I'm using the musl packages availables on Ubuntu Unicorn, musl-gcc is
available at /usr/bin/musl-gcc and musl includes are at
/usr/include/i386-linux-musl


> And have you done anything to modify it or added additional include paths for it? The above symlinks don't make sense for a typical out-of-the-box installation.

I've not modified musl source code, only modifications have been to
add that symlinks for linux, asm & asm-generic includes so they are
available to Node.js during compilation. I didn't found a cleaner way
to do it. They are not required by Node.js itself, but instead by
libuv.


> I suspect you've somehow gotten -I/usr/include added to the CFLAGS during build
> (either doing it yourself, or as a result of a broken build script) if
> adding symlinks to /usr/include made a difference, in which case you
> may be using glibc's headers, not musl's, and thus getting broken
> results.

The build scripts are generated by GYP and the Node.js build process,
that I'm generating with

CC=musl-gcc ./configure --fully-static --without-ssl

(--fully-static is a flag that I've patched that adds '-static' to gcc call)

/usr/include should not be referenced, if so, I wouldn't need to add
the symlinks for /usr/include/linux and /usr/include/asm-generic :-/
Anyway, I'll review the full build process.


-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Re: Bug on fstat()?

[Luca Barbato]

On 24/08/14 23:23, piranna@gmail.com wrote:
>> What prefix is musl-gcc installed in?
> 
> I'm using the musl packages availables on Ubuntu Unicorn, musl-gcc is
> available at /usr/bin/musl-gcc and musl includes are at
> /usr/include/i386-linux-musl
> 
> 
>> And have you done anything to modify it or added additional include paths for it? The above symlinks don't make sense for a typical out-of-the-box installation.
> 
> I've not modified musl source code, only modifications have been to
> add that symlinks for linux, asm & asm-generic includes so they are
> available to Node.js during compilation. I didn't found a cleaner way
> to do it. They are not required by Node.js itself, but instead by
> libuv.

Make sure everything picks up the correct compiler and include paths.

lu

Re: Bug on fstat()?

[piranna]

> Make sure everything picks up the correct compiler and include paths.

I've review the compilation output and in the last command, I have
seen it's using g++ for linking and in fact is throwing the same
errors about "Using 'dlopen' in statically linked applications
requires at runtime the shared libraries from the glibc version used
for linking" (I've missed them because since all the compilation was
being done by musl-gcc and exit status was ok, I though everything was
compiled correctly. My fault).

I have reviewed the generated makefiles and found this paragraph
regarding th g++ linking:


# C++ apps need to be linked with g++.
#
# Note: flock is used to seralize linking. Linking is a memory-intensive
# process so running parallel links can often lead to thrashing.  To disable
# the serialization, override LINK via an envrionment variable as follows:
#
#   export LINK=g++
#
# This will allow make to invoke N linker processes as specified in -jN.
LINK ?= flock $(builddir)/linker.lock $(CXX.target)

So, is g++ mandatory for compilation? Is there any solution? I'll try
to set CXX to musl-gcc too, but didn't see anything about this and I'm
not sure if it would work... If it doesn't work, would I need to
create a musl-g++ wrapper?



-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Re: Bug on fstat()?

[piranna]

> So, is g++ mandatory for compilation? Is there any solution? I'll try
> to set CXX to musl-gcc too, but didn't see anything about this and I'm
> not sure if it would work... If it doesn't work, would I need to
> create a musl-g++ wrapper?

Confirmed: doesn't work, it has failed just in the first C++ source
code file it has found:

  musl-gcc '-DV8_TARGET_ARCH_IA32' '-DENABLE_DISASSEMBLER'
'-DV8_USE_DEFAULT_PLATFORM'
'-DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC' -I../deps/v8/src  -pthread
-Wall -Wextra -Wno-unused-parameter -m32 -fno-strict-aliasing -m32 -O3
-ffunction-sections -fdata-sections -fno-tree-vrp
-fno-omit-frame-pointer -fdata-sections -ffunction-sections -O3
-fno-rtti -fno-exceptions -MMD -MF
/home/piranna/Dropbox/Proyectos/NodeOS/NodeOS-QEmu/Layer2-nodejs/deps/node/out/Release/.deps//home/piranna/Dropbox/Proyectos/NodeOS/NodeOS-QEmu/Layer2-nodejs/deps/node/out/Release/obj.target/v8_base.ia32/deps/v8/src/accessors.o.d.raw
 -c -o /home/piranna/Dropbox/Proyectos/NodeOS/NodeOS-QEmu/Layer2-nodejs/deps/node/out/Release/obj.target/v8_base.ia32/deps/v8/src/accessors.o
../deps/v8/src/accessors.cc
In file included from ../deps/v8/src/platform.h:28:0,
                 from ../deps/v8/src/assert-scope.h:9,
                 from ../deps/v8/src/v8.h:34,
                 from ../deps/v8/src/accessors.cc:5:
../deps/v8/src/vector.h:9:21: fatal error: algorithm: No existe el
archivo o el directorio
 #include <algorithm>
                     ^
compilation terminated.

I'll try to hack a musl-gcc wrapper to see if there's better luck...


-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Re: Bug on fstat()?

[Szabolcs Nagy]

* piranna@gmail.com <piranna@gmail.com> [2014-08-25 11:31:28 +0200]:
> > So, is g++ mandatory for compilation? Is there any solution? I'll try
> > to set CXX to musl-gcc too, but didn't see anything about this and I'm
> > not sure if it would work... If it doesn't work, would I need to
> > create a musl-g++ wrapper?
> 
> Confirmed: doesn't work, it has failed just in the first C++ source
> code file it has found:
> 

musl-gcc is just a thin gcc wrapper (uses a custom spec file
to locate the standard headers and other libc files)

eg. it does not have a libstdc++ linked against musl so it
cannot build c++ code

use a musl cross compiler for your target with c++ support
(eg http://musl.codu.org/ has prebuilt ones but you can build
musl-cross yourself) or build on a musl based system (sabotage
or alpinelinux, they probably already have node.js packaged)

Re: Bug on fstat()?

[Rich Felker]

On Mon, Aug 25, 2014 at 11:49:03AM +0200, Szabolcs Nagy wrote:
> * piranna@gmail.com <piranna@gmail.com> [2014-08-25 11:31:28 +0200]:
> > > So, is g++ mandatory for compilation? Is there any solution? I'll try
> > > to set CXX to musl-gcc too, but didn't see anything about this and I'm
> > > not sure if it would work... If it doesn't work, would I need to
> > > create a musl-g++ wrapper?
> > 
> > Confirmed: doesn't work, it has failed just in the first C++ source
> > code file it has found:
> > 
> 
> musl-gcc is just a thin gcc wrapper (uses a custom spec file
> to locate the standard headers and other libc files)
> 
> eg. it does not have a libstdc++ linked against musl so it
> cannot build c++ code

In principle it should be possible to make it work. libstdc++.so
actually works fine with musl due to the ABI-compat; the problem is
the nasty precompiled header files that have already pulled in junk
from glibc's headers. If there's a way around that, it may be possible
to get musl-gcc working with C++. Nobody has tried it yet as far as I
know, though.

> use a musl cross compiler for your target with c++ support
> (eg http://musl.codu.org/ has prebuilt ones but you can build
> musl-cross yourself) or build on a musl based system (sabotage
> or alpinelinux, they probably already have node.js packaged)

Yes, this is the best option. You might need to tweak their builds to
get a static binary though.

Rich

Re: Bug on fstat()?

[piranna]

Sorry for the delay. Ok, I must to say it: the cross-compiler did the
trick :-D I downloaded one of the pre-compiled ones and didn't
understand too much how to use it, so I clone the repository and build
and install it by hand thinking it would help me to understand how it
worked, but didn't :-( Lucklily, I found a Node.js install script from
alpinelinux (http://git.alpinelinux.org/cgit/aports/tree/main/nodejs/APKBUILD)
and a cross-compiler helper script for Parrot AR drone
(https://github.com/felixge/node-cross-compiler/blob/master/platforms/ardrone2.sh)
that gave me some inspiration and I was able to finally compile the
static build :-) I needed to disable OpenSSL support because it gave a
strange error:


  /opt/cross/i686-linux-musl/bin/i686-linux-musl-g++
'-DOPENSSL_NO_SSL2=1' '-DV8_TARGET_ARCH_IA32' '-DENABLE_DISASSEMBLER'
'-DV8_USE_DEFAULT_PLATFORM' -I../deps/v8/src  -pthread -Wall -Wextra
-Wno-unused-parameter -m32 -fno-strict-aliasing -m32 -O3
-ffunction-sections -fdata-sections -fno-tree-vrp
-fno-omit-frame-pointer -fdata-sections -ffunction-sections -O3
-fno-rtti -fno-exceptions -MMD -MF
/home/piranna/Dropbox/Proyectos/NodeOS/NodeOS-QEmu/Layer2-nodejs/deps/node/out/Release/.deps//home/piranna/Dropbox/Proyectos/NodeOS/NodeOS-QEmu/Layer2-nodejs/deps/node/out/Release/obj.target/mksnapshot.ia32/deps/v8/src/mksnapshot.o.d.raw
 -c -o /home/piranna/Dropbox/Proyectos/NodeOS/NodeOS-QEmu/Layer2-nodejs/deps/node/out/Release/obj.target/mksnapshot.ia32/deps/v8/src/mksnapshot.o
../deps/v8/src/mksnapshot.cc
/home/piranna/Dropbox/Proyectos/NodeOS/NodeOS-QEmu/Layer2-nodejs/deps/node/out/Release/obj.target/deps/openssl/libopenssl.a(eng_all.o):
En la función `ENGINE_load_builtin_engines':
eng_all.c:(.text.ENGINE_load_builtin_engines+0x46): referencia a
`ENGINE_load_padlock' sin definir
eng_all.c:(.text.ENGINE_load_builtin_engines+0x4b): referencia a
`ENGINE_load_gost' sin definir
collect2: error: ld returned 1 exit status
deps/openssl/openssl-cli.target.mk:193: recipe for target
'/home/piranna/Dropbox/Proyectos/NodeOS/NodeOS-QEmu/Layer2-nodejs/deps/node/out/Release/openssl-cli'
failed


Seems to be that OpenSSL command line executable is trying to find
this references. I have tried to disable the generation of that target
from GYP but seems more difficult to do that what I though. I know
OpenSSL has some issues with musl, but anyone knows how to fix this?
Seems OpenSSL support is required so NPM can work and to be able to
install packages... :-/

Otherwise, the Node.js static build is working flawlessly and the
problem with fstat() has been fixed with the cross-compiler :-D I'm
having some issues with compiled modules, but probably they are dues
to compile them outside the custom environment (with the current
Node.js version and glibc instead of the development one and musl),
I'll try to make it work from inside them :-)


-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux