From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/7940 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: [PATCH v8] Build process uses script to add CFI directives to x86 asm Date: Sun, 14 Jun 2015 23:26:53 -0400 Message-ID: <20150615032653.GB1173@brightrain.aerifal.cx> References: <1433493558-9027-1-git-send-email-alexinbeijing@gmail.com> <20150614043725.GL17573@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1434338831 23318 80.91.229.3 (15 Jun 2015 03:27:11 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 15 Jun 2015 03:27:11 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-7953-gllmg-musl=m.gmane.org@lists.openwall.com Mon Jun 15 05:27:10 2015 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1Z4L3E-0000FJ-UL for gllmg-musl@m.gmane.org; Mon, 15 Jun 2015 05:27:09 +0200 Original-Received: (qmail 13726 invoked by uid 550); 15 Jun 2015 03:27:07 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 13708 invoked from network); 15 Jun 2015 03:27:06 -0000 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:7940 Archived-At: On Sun, Jun 14, 2015 at 09:06:16PM +0200, Alex wrote: > Thanks for the reply! Comments below: > > On Sun, Jun 14, 2015 at 6:37 AM, Rich Felker wrote: > > > On Fri, Jun 05, 2015 at 10:39:18AM +0200, Alex Dowad wrote: > > > diff --git a/Makefile b/Makefile > > > index 2eb7b30..9b55fd8 100644 > > > --- a/Makefile > > > +++ b/Makefile > > > @@ -120,7 +120,11 @@ $(foreach s,$(wildcard src/*/$(ARCH)*/*.s),$(eval > > $(call mkasmdep,$(s)))) > > > $(CC) $(CFLAGS_ALL_STATIC) -c -o $@ $(dir $<)$(shell cat $<) > > > > > > %.o: $(ARCH)/%.s > > > - $(CC) $(CFLAGS_ALL_STATIC) -c -o $@ $< > > > +ifeq ($(ADD_CFI),yes) > > > + LC_ALL=C awk -f tools/add-cfi.$(ARCH).awk $< | $(CC) $(ASFLAGS) -x > > assembler -c -o $@ - > > > +else > > > + $(CC) $(ASFLAGS) -c -o $@ $< > > > +endif > > > > Removing $(CFLAGS_STATIC_ALL) here is a regression. -Wa,--noexecstack > > is necessary to prevent the kernel from giving us an executable stack > > when asm files are linked. We could move it to a separate ASFLAGS, but > > the patch doesn't do this, and unless there's a real need to avoid > > passing CFLAGS, I'd rather not add more vars. (In this case, needing > > the new var would be a silent security regression for anyone building > > without re-running configure.) > > > > The reason for not passing CFLAGS is because clang chokes on "-g" when > assembling code with CFI directives. I also thought that ASFLAGS might be a > useful customization point for people who want to edit config.mak to create > a custom build. But you are the judge of that. > > Since it seems that CFLAGS is needed, would it be acceptable to bypass the > issue by saying that clang users simply won't be able to do debug builds of > musl until their compiler is fixed? The current state of LLVM's CFI > generation is so bad that debug builds probably won't be useful anyways. Could you elaborate on what happens? I'm not opposed to this approach as long as either (1) the configure test successfully determines that CFI gen doesn't work on clang, or (2) the 'choking' just produces bad CFI, but doesn't break the build. > If that is a sticking point, I might put together a patch for LLVM and see > if they want it. Unfortunately, I have already discovered a bunch of other > problems with LLVM which would be nice to fix, but time for developing and > polishing patches is limited... Why is -g even being processes for asm? Are they trying to auto-generate CFI when it's not present? I think this really needs to be fixed in any case since there are plenty of .s files that _do_ have CFI and build systems that use -g. All this points to clang's internal assembler being not-widely-tested and not ready for serious use... :( But another option would be just having the Makefile remove "-g*" for asm. Obviously this is hard to make robust because technically "-g" could be an argument to "-o" or something stupid, but our Makefile doesn't need to be robust against arbitrary ridiculous filenames and such... It's not like spaces work in pathnames in Makefiles anyway... ;-) > As an aside, I admire the fact that you picked up on that subtle > regression. The standard of code quality and attention to detail on this > project is very high, as compared to other open-source projects I have > worked on. Kudos to all the contributors! Thanks! > As for the naming (tools/add-cfi.$(ARCH).awk), I'm not opposed to this > > and the configure test for it is nice, but I wonder if there will be > > significant code duplication between versions of this script for > > different archs that would make it preferable to take the arch as an > > argument. What do you think? Or does awk have an easy #include-like > > mechanism? > > > > I'm not an AWKer, but from what I have read, apparently "awk -f script1.awk > -f script2.awk" is the equivalent of concatenating "script1.awk" and > "script2.awk", so shared functions can easily be put in a common file. > > It seems that the amount of shared code will be small, however. Actually, > the entire script for x86-32 is already fairly small. I feel that anything > more sophisticated than picking a script based on arch would just be > complicating matters for little benefit. > > If it turns out that I am wrong, the commonalities can be abstracted out > later. At that time, with several such preprocessing scripts available to > look at, it will be clearer what and how to abstract. OK, this sounds fine. I just wanted to hear your opinion on it. Apologies if you already stated it earlier and I missed it; I was rather focused on other things at the time most of the discussion and review happened. > > > +# Preprocess asm files to add extra debugging information if debug is > > > +# enabled, our assembler supports the needed directives, and the > > > +# preprocessing script has been written for our architecture. > > > +# > > > +printf "checking whether we should preprocess assembly to add debugging > > information... " > > > +if fnmatch '-g*|*\ -g*' "$CFLAGS_AUTO" && > > > + test -f "tools/add-cfi.$ARCH.awk" && > > > + echo ".cfi_startproc > > > +.cfi_endproc" | $CC -x assembler -c -o /dev/null - > > > +then > > > + ADD_CFI=yes > > > +else > > > + ADD_CFI=no > > > +fi > > > +printf "%s\n" "$ADD_CFI" > > > + > > > +# > > > > This test looks nice and robust. I'd mildly prefer: > > > > printf '.cfi_startproc\n.cfi_endproc\n' > > > > to avoid the multi-line string with echo, but that's a tiny detail. > > > > OK. It was written like this because "echo '.cfi_startproc\n.cfi_endproc'" > didn't work on BusyBox ash. But it seems that printf is fine. Will revise. Yes, also musl's configure redefines echo as a shell function in terms of printf, since echo varies widely in behavior and the standard's text on echo is contrary to most real-world implementations... So using printf is preferred in general anyway for non-trivial usage. Rich