From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/7243 Path: news.gmane.org!not-for-mail From: Szabolcs Nagy Newsgroups: gmane.linux.lib.musl.general Subject: Re: buffer overflow in regcomp and a way to find more of those Date: Sat, 21 Mar 2015 14:28:10 +0100 Message-ID: <20150321132810.GI16260@port70.net> References: <20150321004637.GQ23507@brightrain.aerifal.cx> <20150321010043.GR23507@brightrain.aerifal.cx> <20150321013225.GT23507@brightrain.aerifal.cx> <20150321015619.GU23507@brightrain.aerifal.cx> <20150321022023.GW23507@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="7SrMUQONj8Rl9QNG" X-Trace: ger.gmane.org 1426944523 21521 80.91.229.3 (21 Mar 2015 13:28:43 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 21 Mar 2015 13:28:43 +0000 (UTC) Cc: Rich Felker , musl@lists.openwall.com To: Konstantin Serebryany Original-X-From: musl-return-7256-gllmg-musl=m.gmane.org@lists.openwall.com Sat Mar 21 14:28:30 2015 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1YZJRz-00086p-Be for gllmg-musl@m.gmane.org; Sat, 21 Mar 2015 14:28:27 +0100 Original-Received: (qmail 7497 invoked by uid 550); 21 Mar 2015 13:28:23 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 7476 invoked from network); 21 Mar 2015 13:28:22 -0000 Mail-Followup-To: Konstantin Serebryany , Rich Felker , musl@lists.openwall.com Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Xref: news.gmane.org gmane.linux.lib.musl.general:7243 Archived-At: --7SrMUQONj8Rl9QNG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline * Konstantin Serebryany [2015-03-20 23:05:13 -0700]: > On Fri, Mar 20, 2015 at 7:20 PM, Rich Felker wrote: > > On Fri, Mar 20, 2015 at 07:14:33PM -0700, Konstantin Serebryany wrote: > >> If you build the source with "-fsanitize=leak -fsanitize-coverage=4 > >> -O1" the compiler will not insert any of the asan instrumentation > >> and only insert calls to a couple of functions needed for coverage. > >> Then, instead of linking with the full asan+coverage run-time, you > >> will need a very simple re-implementation of coverage-only runtime. > > > > Could the existing runtime be used, just stripped down? > > Yes, but for the basic functionality needed by the fuzzer it's simpler > to write it from scratch, see below: > > ======================================================== > svn co http://llvm.org/svn/llvm-project/llvm/trunk/lib/Fuzzer > cat <cov-minimal-rt.c > static long counter; > void __sanitizer_cov_with_check(int *guard) { > if (*guard == 0) { > counter++; > *guard=1; > } > } > long __sanitizer_get_total_unique_coverage() { return counter; } > void __sanitizer_cov_module_init() {} > void __sanitizer_reset_coverage(){} > void __sanitizer_get_coverage_guards(){} > void __sanitizer_get_number_of_counters(){} > void __sanitizer_update_counter_bitset_and_clear_counters(){} > void __sanitizer_set_death_callback(){} > EOF > > clang -std=c++11 -c Fuzzer/Fuzzer*.cpp -I Fuzzer > clang -std=c++11 -fsanitize=leak -fsanitize-coverage=3 -mllvm > -sanitizer-coverage-block-threshold=0 Fuzzer/test/SimpleTest.cpp -c > clang -c cov-minimal-rt.c > clang++ *.o > ./a.out > ======================================================== with this i could run the fuzzer against libc.a it's a bit more work to link to libc.a than adding a -L so i attached the scripts i used (and an example) so others can reproduce it c++ headers cannot be used in the test (that would require cleaning up the libstdc++ header mess) but i think there is no reason to use c++ for these libc api tests anyway you may need to adjust the directories the scripts use (the linking may need to change when compiler-rt is used instead of libgcc) usage: cd workdir ./buildfuzz.sh ./buildmusl.sh ./fuzzcompile.sh reg.c ./fuzzlink.sh reg.o ./a.out of course to make it useful the malloc magic is needed for more likely crashes > The recently added afl-style counters > (https://code.google.com/p/address-sanitizer/wiki/AsanCoverage#Coverage_counters) > are a bit more involved, but the basic bool-per-edge is quite enough > in most cases. > ok > The fuzzer itself is written in C++ and uses STL (probably, not the > best idea, but it makes the experiments simpler). > Can't tell if it will be a problem with musl, but after all the fuzzer > itself is also trivial (as well as the entire concept) > c++ happens to work because musl is (almost) abi compatible with glibc on x86 so we can just link to the glibc linked libstdc++ (this can eg fail when the c++ thread local storage destructor abi is used, that is not implemented in musl yet) so yes c++ makes things more painful: you need to recompile the entire toolchain to make it work reliably (and then both gcc and clang have broken assumptions about the libc so you have to patch them) which is too much work for running tests > > Well static linking with musl does not impose any constraint on > > redefining functions, so you could easily use a debugging malloc that > > lines up each allocation to end on a page boundary with a guard page > > after it. > > Yea... This will slowdown fuzzing and guard pages only protect you > from overflow in one direction (ether left, of right, but not both). > But this is better than nothing. > you can run the tests twice (for left and right) :) > > This would of course be slow and use lots of memory but > > would catch all heap overflows. And -fstack-protector-all would catch > > most stack-based overflows. > > Only stack-overflow-write by a small amount, but yes, better than nothing. > > BTW, writing a minimalistic asan run-time as part of musl should be a > matter of a couple of hours. > Probably much faster than making the current monster work with static linking. > I'd be happy to help with such. > how would this look? compile the tests and libc with asan, but instead of linking the asan runtime from clang use a musl specific one? i assume for that we still need to change the libc startup code, malloc functions and may be some things around thread stacks --7SrMUQONj8Rl9QNG Content-Type: application/x-sh Content-Disposition: attachment; filename="buildfuzz.sh" Content-Transfer-Encoding: quoted-printable #!/bin/sh=0A=0A# http://www.openwall.com/lists/musl/2015/03/21/17=0A=0Aset = -xue=0A=0Afuzzdir=3Dfuzz=0A=0Amkdir -p $fuzzdir=0Acd $fuzzdir=0A[ -e libfuz= z.a ] && exit 0=0A=0Asvn co http://llvm.org/svn/llvm-project/llvm/trunk/lib= /Fuzzer=0Acat <cov-minimal-rt.c=0Astatic long counter;=0Avoid __sanit= izer_cov_with_check(int *guard) {=0Aif (*guard =3D=3D 0) {=0Acounter++;=0A*= guard=3D1;=0A}=0A}=0Along __sanitizer_get_total_unique_coverage() { return = counter; }=0Avoid __sanitizer_cov_module_init() {}=0Avoid __sanitizer_reset= _coverage(){}=0Avoid __sanitizer_get_coverage_guards(){}=0Avoid __sanitizer= _get_number_of_counters(){}=0Avoid __sanitizer_update_counter_bitset_and_cl= ear_counters(){}=0Avoid __sanitizer_set_death_callback(){}=0AEOF=0A=0Aclang= -std=3Dc++11 -c Fuzzer/Fuzzer*.cpp -I Fuzzer=0Aclang -c cov-minimal-rt.c= =0Aar rc libfuzz.a *.o=0Aranlib libfuzz.a=0A --7SrMUQONj8Rl9QNG Content-Type: application/x-sh Content-Disposition: attachment; filename="buildmusl.sh" Content-Transfer-Encoding: quoted-printable #!/bin/sh=0A=0Aset -xue=0A=0Amusldir=3Dmusl=0A=0A[ -e $musldir ] || git clo= ne git://git.musl-libc.org/musl $musldir=0Acd $musldir=0A=0ACFLAGS=3D'-O1 -= fsanitize=3Dleak -fsanitize-coverage=3D3 -mllvm -sanitizer-coverage-block-t= hreshold=3D0'=0ACC=3Dclang CFLAGS=3D"$CFLAGS" ./configure --disable-shared= =0Amake clean=0Amake -j=0A --7SrMUQONj8Rl9QNG Content-Type: application/x-sh Content-Disposition: attachment; filename="fuzzcompile.sh" Content-Transfer-Encoding: quoted-printable #!/bin/sh=0A=0Amusldir=3Dmusl=0A=0Aclang -nostdinc -isystem $musldir/includ= e \=0A -fsanitize=3Dleak -fsanitize-coverage=3D3 -mllvm \=0A -sanitiz= er-coverage-block-threshold=3D0 -c "$@"=0A --7SrMUQONj8Rl9QNG Content-Type: application/x-sh Content-Disposition: attachment; filename="fuzzlink.sh" Content-Transfer-Encoding: quoted-printable #!/bin/sh=0A=0Afuzzdir=3Dfuzz=0Amusldir=3Dmusl=0A=0Accdir=3D$(clang --print= -file-name=3Dlibgcc.a)=0Accdir=3D${ccdir%/*}=0A=0ASTART=3D"$musldir/lib/crt= i.o $musldir/lib/crt1.o $ccdir/crtbeginT.o"=0AEND=3D"-L$fuzzdir -lfuzz -L$c= cdir -lstdc++ -lgcc -lgcc_eh -L$musldir/lib -lc $ccdir/crtend.o $musldir/li= b/crtn.o"=0Aclang -nostdlib -static $START "$@" $END=0A --7SrMUQONj8Rl9QNG Content-Type: text/x-csrc; charset=us-ascii Content-Disposition: attachment; filename="reg.c" #include #include #include void TestOneInput(const uint8_t *p, size_t n) { regex_t preg; regmatch_t pmatch[2]; char *s = strndup((char*)p, n); if (!regcomp(&preg, s, REG_EXTENDED)) { regexec(&preg, s, 0, pmatch, 0); regfree(&preg); } } --7SrMUQONj8Rl9QNG--