size of executable

size of executable

[Jorge Almeida]

I think my executables are way too large. They used to be much
smaller, not so long ago (e.g. 18 months ago). I'm using musl 1.1.23
with gcc-8.3.0 on x86_64. Could someone check the following hello.c
and see what size you get?
I get 16768 bytes (not stripped) and 12324 (stripped).
Note that I'm not even using printf.

Thanks

#include <unistd.h>
int main(int argc, char* argv[]){
 write(1, "Hello, bloated world...\n", 24);
}

Re: size of executable

[Jorge Almeida]

[-- Attachment #1: Type: text/plain, Size: 496 bytes --]

On Mon, Aug 12, 2019 at 5:48 PM Jorge Almeida <jjalmeida@gmail.com> wrote:
>

> with gcc-8.3.0 on x86_64. Could someone check the following hello.c
> and see what size you get?
> I get 16768 bytes (not stripped) and 12324 (stripped).
> Note that I'm not even using printf.
>
> Thanks
>
> #include <unistd.h>
> int main(int argc, char* argv[]){
>  write(1, "Hello, bloated world...\n", 24);
> }

Sorry, I forgot to add the compile options. In attachement, due to
Gmail interface being what it is.

[-- Attachment #2: hello.txt --]
[-- Type: text/plain, Size: 736 bytes --]

gcc -static -Os -march=native -fomit-frame-pointer -pipe -Wall  -Werror=pedantic  -Werror=implicit-function-declaration -pedantic -pedantic-errors -mpreferred-stack-boundary=4 -falign-functions=1 -falign-jumps=1 -falign-loops=1 -fno-unwind-tables -fdata-sections -ffunction-sections -Wl,--gc-sections -fno-asynchronous-unwind-tables -fstrict-aliasing -Wstrict-aliasing=2 -Wno-unused-function -std=c11 hello.c

Stripping:

strip -s a.out
strip -S --strip-unneeded --remove-section=.note.gnu.gold-version --remove-section=.comment --remove-section=.note --remove-section=.note.gnu.build-id --remove-section=.note.ABI-tag --remove-section=.eh_frame --remove-section=.eh_frame_ptr -R .note -R .comment -R .note.GNU-stack a.out
sstrip a.out

Re: Re: size of executable

[Rich Felker]

On Mon, Aug 12, 2019 at 05:55:28PM +0100, Jorge Almeida wrote:
> On Mon, Aug 12, 2019 at 5:48 PM Jorge Almeida <jjalmeida@gmail.com> wrote:
> >
> 
> > with gcc-8.3.0 on x86_64. Could someone check the following hello.c
> > and see what size you get?
> > I get 16768 bytes (not stripped) and 12324 (stripped).
> > Note that I'm not even using printf.
> >
> > Thanks
> >
> > #include <unistd.h>
> > int main(int argc, char* argv[]){
> >  write(1, "Hello, bloated world...\n", 24);
> > }
> 
> Sorry, I forgot to add the compile options. In attachement, due to
> Gmail interface being what it is.

> gcc -static -Os -march=native -fomit-frame-pointer -pipe -Wall  -Werror=pedantic  -Werror=implicit-function-declaration -pedantic -pedantic-errors -mpreferred-stack-boundary=4 -falign-functions=1 -falign-jumps=1 -falign-loops=1 -fno-unwind-tables -fdata-sections -ffunction-sections -Wl,--gc-sections -fno-asynchronous-unwind-tables -fstrict-aliasing -Wstrict-aliasing=2 -Wno-unused-function -std=c11 hello.c
> 
> Stripping:
> 
> strip -s a.out
> strip -S --strip-unneeded --remove-section=.note.gnu.gold-version --remove-section=.comment --remove-section=.note --remove-section=.note.gnu.build-id --remove-section=.note.ABI-tag --remove-section=.eh_frame --remove-section=.eh_frame_ptr -R .note -R .comment -R .note.GNU-stack a.out
> sstrip a.out

This is a binutils regression from a dubious anti-ROP feature, -z
separate-code. Add -Wl,-z,noseparate-code and it will go away.

FYI musl-cross-make has been updated to change the default back in
toolchains it builds.

Rich

Re: Re: size of executable

[Jorge Almeida]

On Mon, Aug 12, 2019 at 6:19 PM Rich Felker <dalias@libc.org> wrote:
>
> On Mon, Aug 12, 2019 at 05:55:28PM +0100, Jorge Almeida wrote:

> > gcc -static -Os -march=native -fomit-frame-pointer -pipe -Wall  -Werror=pedantic  -Werror=implicit-function-declaration -pedantic -pedantic-errors -mpreferred-stack-boundary=4 -falign-functions=1 -falign-jumps=1 -falign-loops=1 -fno-unwind-tables -fdata-sections -ffunction-sections -Wl,--gc-sections -fno-asynchronous-unwind-tables -fstrict-aliasing -Wstrict-aliasing=2 -Wno-unused-function -std=c11 hello.c
> >
> > Stripping:
> >
> > strip -s a.out
> > strip -S --strip-unneeded --remove-section=.note.gnu.gold-version --remove-section=.comment --remove-section=.note --remove-section=.note.gnu.build-id --remove-section=.note.ABI-tag --remove-section=.eh_frame --remove-section=.eh_frame_ptr -R .note -R .comment -R .note.GNU-stack a.out
> > sstrip a.out
>
> This is a binutils regression from a dubious anti-ROP feature, -z
> separate-code. Add -Wl,-z,noseparate-code and it will go away.
>
> FYI musl-cross-make has been updated to change the default back in
> toolchains it builds.
>
Aha!

8472 not stripped and 4132 stripped.

Thanks!

Jorge

Re: Re: size of executable

[Khem Raj]

On Mon, Aug 12, 2019 at 10:19 AM Rich Felker <dalias@libc.org> wrote:
>
> On Mon, Aug 12, 2019 at 05:55:28PM +0100, Jorge Almeida wrote:
> > On Mon, Aug 12, 2019 at 5:48 PM Jorge Almeida <jjalmeida@gmail.com> wrote:
> > >
> >
> > > with gcc-8.3.0 on x86_64. Could someone check the following hello.c
> > > and see what size you get?
> > > I get 16768 bytes (not stripped) and 12324 (stripped).
> > > Note that I'm not even using printf.
> > >
> > > Thanks
> > >
> > > #include <unistd.h>
> > > int main(int argc, char* argv[]){
> > >  write(1, "Hello, bloated world...\n", 24);
> > > }
> >
> > Sorry, I forgot to add the compile options. In attachement, due to
> > Gmail interface being what it is.
>
> > gcc -static -Os -march=native -fomit-frame-pointer -pipe -Wall  -Werror=pedantic  -Werror=implicit-function-declaration -pedantic -pedantic-errors -mpreferred-stack-boundary=4 -falign-functions=1 -falign-jumps=1 -falign-loops=1 -fno-unwind-tables -fdata-sections -ffunction-sections -Wl,--gc-sections -fno-asynchronous-unwind-tables -fstrict-aliasing -Wstrict-aliasing=2 -Wno-unused-function -std=c11 hello.c
> >
> > Stripping:
> >
> > strip -s a.out
> > strip -S --strip-unneeded --remove-section=.note.gnu.gold-version --remove-section=.comment --remove-section=.note --remove-section=.note.gnu.build-id --remove-section=.note.ABI-tag --remove-section=.eh_frame --remove-section=.eh_frame_ptr -R .note -R .comment -R .note.GNU-stack a.out
> > sstrip a.out
>
> This is a binutils regression from a dubious anti-ROP feature, -z
> separate-code. Add -Wl,-z,noseparate-code and it will go away.
>

is this still so with latest release as well.

> FYI musl-cross-make has been updated to change the default back in
> toolchains it builds.
>
> Rich

Re: Re: size of executable

[Rich Felker]

On Mon, Aug 12, 2019 at 11:16:39AM -0700, Khem Raj wrote:
> On Mon, Aug 12, 2019 at 10:19 AM Rich Felker <dalias@libc.org> wrote:
> >
> > On Mon, Aug 12, 2019 at 05:55:28PM +0100, Jorge Almeida wrote:
> > > On Mon, Aug 12, 2019 at 5:48 PM Jorge Almeida <jjalmeida@gmail.com> wrote:
> > > >
> > >
> > > > with gcc-8.3.0 on x86_64. Could someone check the following hello.c
> > > > and see what size you get?
> > > > I get 16768 bytes (not stripped) and 12324 (stripped).
> > > > Note that I'm not even using printf.
> > > >
> > > > Thanks
> > > >
> > > > #include <unistd.h>
> > > > int main(int argc, char* argv[]){
> > > >  write(1, "Hello, bloated world...\n", 24);
> > > > }
> > >
> > > Sorry, I forgot to add the compile options. In attachement, due to
> > > Gmail interface being what it is.
> >
> > > gcc -static -Os -march=native -fomit-frame-pointer -pipe -Wall  -Werror=pedantic  -Werror=implicit-function-declaration -pedantic -pedantic-errors -mpreferred-stack-boundary=4 -falign-functions=1 -falign-jumps=1 -falign-loops=1 -fno-unwind-tables -fdata-sections -ffunction-sections -Wl,--gc-sections -fno-asynchronous-unwind-tables -fstrict-aliasing -Wstrict-aliasing=2 -Wno-unused-function -std=c11 hello.c
> > >
> > > Stripping:
> > >
> > > strip -s a.out
> > > strip -S --strip-unneeded --remove-section=.note.gnu.gold-version --remove-section=.comment --remove-section=.note --remove-section=.note.gnu.build-id --remove-section=.note.ABI-tag --remove-section=.eh_frame --remove-section=.eh_frame_ptr -R .note -R .comment -R .note.GNU-stack a.out
> > > sstrip a.out
> >
> > This is a binutils regression from a dubious anti-ROP feature, -z
> > separate-code. Add -Wl,-z,noseparate-code and it will go away.
> >
> 
> is this still so with latest release as well.

The breakage that caused separate-code to crash at runtime was fixed
between 2.31 and 2.32, but the size and performance regression
remains. With separate-code, a couple extra pages of memory and disk
are needed, with corresponding runtime cost to mmap them properly.

All to avoid ROP gadgets, when every single dynamic-linked program has
a nice ROP gadget named "system" (among many others) in it...

Rich

Re: Re: size of executable

[Micha Nelissen]

On 12-08-2019 20:23, Rich Felker wrote:
> On Mon, Aug 12, 2019 at 11:16:39AM -0700, Khem Raj wrote:
>> On Mon, Aug 12, 2019 at 10:19 AM Rich Felker <dalias@libc.org> wrote:
>>> On Mon, Aug 12, 2019 at 05:55:28PM +0100, Jorge Almeida wrote:
>>>> On Mon, Aug 12, 2019 at 5:48 PM Jorge Almeida <jjalmeida@gmail.com> wrote:
>>>>> I get 16768 bytes (not stripped) and 12324 (stripped).
>>>
>>> This is a binutils regression from a dubious anti-ROP feature, -z
>>> separate-code. Add -Wl,-z,noseparate-code and it will go away.
>>
>> is this still so with latest release as well.
> 
> The breakage that caused separate-code to crash at runtime was fixed
> between 2.31 and 2.32, but the size and performance regression
> remains. With separate-code, a couple extra pages of memory and disk
> are needed, with corresponding runtime cost to mmap them properly.
> 
> All to avoid ROP gadgets, when every single dynamic-linked program has
> a nice ROP gadget named "system" (among many others) in it...

I'm curious. Jorge reports that the executable goes from 12k to 4k. That 
suggests two pages saved? But if I look at documentation for this 
separate-code option, then it says to allocate a separate code PT_LOAD 
segment. (PT_LOAD just means loadable?) That would suggest up to 4k more 
usage, not 8k right? One extra page necessary. Are by default rodata and 
code combined but with separate-code those are separated? Or something 
more happening?

Thanks, Micha

Re: Re: size of executable

[Fangrui Song]

On 2019-08-13, Micha Nelissen wrote:
>On 12-08-2019 20:23, Rich Felker wrote:
>>On Mon, Aug 12, 2019 at 11:16:39AM -0700, Khem Raj wrote:
>>>On Mon, Aug 12, 2019 at 10:19 AM Rich Felker <dalias@libc.org> wrote:
>>>>On Mon, Aug 12, 2019 at 05:55:28PM +0100, Jorge Almeida wrote:
>>>>>On Mon, Aug 12, 2019 at 5:48 PM Jorge Almeida <jjalmeida@gmail.com> wrote:
>>>>>>I get 16768 bytes (not stripped) and 12324 (stripped).
>>>>
>>>>This is a binutils regression from a dubious anti-ROP feature, -z
>>>>separate-code. Add -Wl,-z,noseparate-code and it will go away.
>>>
>>>is this still so with latest release as well.
>>
>>The breakage that caused separate-code to crash at runtime was fixed
>>between 2.31 and 2.32, but the size and performance regression
>>remains. With separate-code, a couple extra pages of memory and disk
>>are needed, with corresponding runtime cost to mmap them properly.
>>
>>All to avoid ROP gadgets, when every single dynamic-linked program has
>>a nice ROP gadget named "system" (among many others) in it...
>
>I'm curious. Jorge reports that the executable goes from 12k to 4k. 
>That suggests two pages saved? But if I look at documentation for this 
>separate-code option, then it says to allocate a separate code PT_LOAD 
>segment. (PT_LOAD just means loadable?) That would suggest up to 4k 
>more usage, not 8k right? One extra page necessary. Are by default 
>rodata and code combined but with separate-code those are separated? 
>Or something more happening?

binutils 2.31 includes a change "ld: Add --enable-separate-code". This is enabled by default for Linux x86.

You can compile a trivial program with -z separate-code and -z noseparate-code, run readelf -l and compare the results.

-z noseparate-code

  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x0007d8 0x0007d8 R E 0x1000
  LOAD           0x000e18 0x0000000000001e18 0x0000000000001e18 0x000210 0x000218 RW  0x1000

-z separate-code

  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x000530 0x000530 R   0x1000
  LOAD           0x001000 0x0000000000001000 0x0000000000001000 0x0001cd 0x0001cd R E 0x1000
  LOAD           0x002000 0x0000000000002000 0x0000000000002000 0x000148 0x000148 R   0x1000
  LOAD           0x002e18 0x0000000000003e18 0x0000000000003e18 0x000210 0x000218 RW  0x1000

-z separate-code has two more PT_LOAD segments. What is bad is that the two PT_LOAD segments have aligned p_offset:

diff -u =(ld.bfd --verbose -z noseparate-code) =(ld.bfd --verbose -z separate-code)

+  . = ALIGN(CONSTANT (MAXPAGESIZE));

+  . = ALIGN(CONSTANT (MAXPAGESIZE));
+  /* Adjust the address for the rodata segment.  We want to adjust up to                                      
+     the same address within the page on the next page up.  */                                                
+  . = SEGMENT_START("rodata-segment", ALIGN(CONSTANT (MAXPAGESIZE)) + (. & (CONSTANT (MAXPAGESIZE) - 1)));

This disables an important size optimization (I have some description in https://reviews.llvm.org/D64906)