* tcmalloc compatibility
@ 2018-04-10 10:22 Denis V. Razumovsky
2018-04-10 14:33 ` Rich Felker
0 siblings, 1 reply; 16+ messages in thread
From: Denis V. Razumovsky @ 2018-04-10 10:22 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 755 bytes --]
Dear Sirs,
Could you please be so kind to fix muls compatibility with tcmalloc
(libtcmalloc_minimal in particular). Seems that tcmalloc developers
can't fix all issues without corresponding changes of musl.
Small cite from tcmalloc bug-report:
>> Doesn't look like musl will ever support replacing their malloc for
dynamically linked musl.
All appropriate details you can find at the page:
https://github.com/gperftools/gperftools/issues/693
Because of this many important software can't be ported properly to
Alpine linux because they depends from libtcmalloc_minimal.
My personal competencies are not enough for that fixes. Therefore, I
appeal in support of both libraries.
Thank you in advance,
Dennis.
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3587 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-10 10:22 tcmalloc compatibility Denis V. Razumovsky @ 2018-04-10 14:33 ` Rich Felker 2018-04-10 14:45 ` Bobby Powers [not found] ` <878t9vlzh1.fsf@mid.deneb.enyo.de> 0 siblings, 2 replies; 16+ messages in thread From: Rich Felker @ 2018-04-10 14:33 UTC (permalink / raw) To: musl On Tue, Apr 10, 2018 at 01:22:54PM +0300, Denis V. Razumovsky wrote: > Dear Sirs, > > Could you please be so kind to fix muls compatibility with tcmalloc > (libtcmalloc_minimal in particular). Seems that tcmalloc developers > can't fix all issues without corresponding changes of musl. malloc interposition is undefined behavior (as is any interposition of standard functions), and is very difficult to actually support as an extension in a way that doesn't have lots of serious problems. This has been discussed before but I don't have links handy. I'll try to dig them up later. The glibc folks are also aware that it's broken (on glibc, it only works if you get lucky and follow unwritten rules) and would probably like a fix but it's not easy on their side either. > Small cite from tcmalloc bug-report: > > > >> Doesn't look like musl will ever support replacing their malloc for > dynamically linked musl. This claim doesn't seem to be well-justified. Myself and members of our community have written a lot on why existing malloc interposition hacks are broken, but there's also an interest in what would take to make it work, and I particularly am interested in this from a standpoint that musl's malloc is not very good, and that being able to dynamically interpose it would facilitate developing and testing a replacement. Note however that if malloc interposition is supported at some point, there will be a specification for constraints on the malloc implementation including what functions you can call from it (e.g. something like AS-safety), and bug reports for implementations that do things outside this spec (and thereby inherently can't work safely or reliably) will not be considered bugs. > All appropriate details you can find at the page: > > https://github.com/gperftools/gperftools/issues/693 > > > Because of this many important software can't be ported properly to > Alpine linux because they depends from libtcmalloc_minimal. This seems false. You can just remove tcmalloc from them and use the system malloc and they will work fine (possibly after patching out unnecessary calls into the internals of the malloc replacement, e.g. for debug/stats/etc.). Rich ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-10 14:33 ` Rich Felker @ 2018-04-10 14:45 ` Bobby Powers 2018-04-10 15:35 ` Rich Felker [not found] ` <878t9vlzh1.fsf@mid.deneb.enyo.de> 1 sibling, 1 reply; 16+ messages in thread From: Bobby Powers @ 2018-04-10 14:45 UTC (permalink / raw) To: musl On Tue, Apr 10, 2018 at 2:34 PM Rich Felker <dalias@libc.org> wrote: > This claim doesn't seem to be well-justified. Myself and members of > our community have written a lot on why existing malloc interposition > hacks are broken, but there's also an interest in what would take to > make it work, and I particularly am interested in this from a > standpoint that musl's malloc is not very good, and that being able to > dynamically interpose it would facilitate developing and testing a > replacement. This sounds super interesting -- what needs to happen to make progress on this? I would love to help out. > Note however that if malloc interposition is supported at some point, > there will be a specification for constraints on the malloc > implementation including what functions you can call from it (e.g. > something like AS-safety), and bug reports for implementations that do > things outside this spec (and thereby inherently can't work safely or > reliably) will not be considered bugs. That sounds reasonable. Some existing software (like Hoard) goes out of its way to interpose on all functions that might call into malloc to ensure the system allocator isn't called indirectly: https://github.com/emeryberger/Heap-Layers/blob/master/wrappers/wrapper.cpp yours, Bobby ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-10 14:45 ` Bobby Powers @ 2018-04-10 15:35 ` Rich Felker 0 siblings, 0 replies; 16+ messages in thread From: Rich Felker @ 2018-04-10 15:35 UTC (permalink / raw) To: musl On Tue, Apr 10, 2018 at 10:45:03AM -0400, Bobby Powers wrote: > On Tue, Apr 10, 2018 at 2:34 PM Rich Felker <dalias@libc.org> wrote: > > This claim doesn't seem to be well-justified. Myself and members of > > our community have written a lot on why existing malloc interposition > > hacks are broken, but there's also an interest in what would take to > > make it work, and I particularly am interested in this from a > > standpoint that musl's malloc is not very good, and that being able to > > dynamically interpose it would facilitate developing and testing a > > replacement. > > This sounds super interesting -- what needs to happen to make progress > on this? I would love to help out. For allowing interposition, it's mainly working out policy so that it's clear what can and can't be supported and we don't get stuck seemingly promising something impossible. The actual code changes are fairly small. We'd need to switch from -Wl,-Bsymbolic-functions when linking to -Wl,--dynamic-list in order to exclude the malloc functions from being bound at link time, and some changes might be necessary in the dynamic linker in how it deals with donating gaps to malloc and early allocations before the interposed malloc is available. There's also a question of whether the dynamic linker should have code to detect and refuse to run with incorrect malloc interposition (some but not all functions interposed). But back to the point, the main issue is specifying the constraints on the interposing functions. > > Note however that if malloc interposition is supported at some point, > > there will be a specification for constraints on the malloc > > implementation including what functions you can call from it (e.g. > > something like AS-safety), and bug reports for implementations that do > > things outside this spec (and thereby inherently can't work safely or > > reliably) will not be considered bugs. > > That sounds reasonable. Some existing software (like Hoard) goes out > of its way to interpose on all functions that might call into malloc > to ensure the system allocator isn't called indirectly: > > https://github.com/emeryberger/Heap-Layers/blob/master/wrappers/wrapper.cpp This is really impossible to do correctly, for multiple reasons: 1. Some such functions are fundamentally not replacable, like the dynamic linker functions (dlopen). 2. There is no specification for which libc functions call into malloc; this is an implementation detail. The only related things that are parts of the public contract are whether they return memory "as if by malloc" and whether they're AS-safe or AC-safe (in which case it's not formally correct, but it's fairly reasonable to assume they don't call malloc). For example on glibc, qsort and printf call malloc (but qsort has, and necessarily has to have, a fallback for when it fails since qsort cant' fail). 3. Some functions which use malloc are sufficiently heavy that you'd be replacing (and possibly changing or reducing functionality in) whole major libc components if you wanted to replace them. For example, getaddrinfo (the whole resolver infrastructure), iconv, regex, ... Note that, unless the malloc replacement and the system malloc somehow step on each other's state, there's no harm in having both present and getting called as long as the libc functions that return memory "as if by malloc" (thus that's permissible to pass to realloc and free) use the interposed malloc replacement. This would just be things like strdup. So if you only replace those, it's a more managable task. But I still think it's a wrong approach. If musl does add support for malloc interposition, I'm strongly leaning towards using the interposed malloc everywhere so that this kind of issue does not matter. Otherwise there are too many opportunities for subtle errors. The main argument for not calling the interposed malloc from libc except when you have to is that you don't have to deal with reentrancy & inconsistent state issues that could be prone to incorrect usage, but getline() inherently has to return memory as if by malloc, and thus you're already stuck with at least one function that has to call the interposed malloc with stdio locks held (or else work in a temp buffer managed by internal malloc, then only move to a public-malloc-allocated buffer after finishing, but that's an awful hack, and imposing libc implementation constraints like that around the allowance for interposing malloc is exactly the type of nasty situation I don't want to get into). Rich ^ permalink raw reply [flat|nested] 16+ messages in thread
[parent not found: <878t9vlzh1.fsf@mid.deneb.enyo.de>]
* Re: tcmalloc compatibility [not found] ` <878t9vlzh1.fsf@mid.deneb.enyo.de> @ 2018-04-10 18:39 ` Rich Felker 2018-04-10 20:33 ` Szabolcs Nagy 1 sibling, 0 replies; 16+ messages in thread From: Rich Felker @ 2018-04-10 18:39 UTC (permalink / raw) To: musl On Tue, Apr 10, 2018 at 07:53:46PM +0200, Florian Weimer wrote: > * Rich Felker: > > > malloc interposition is undefined behavior (as is any interposition of > > standard functions), and is very difficult to actually support as an > > extension in a way that doesn't have lots of serious problems. This > > has been discussed before but I don't have links handy. I'll try to > > dig them up later. The glibc folks are also aware that it's broken (on > > glibc, it only works if you get lucky and follow unwritten rules) > > We have some documentation nowadays: > > <https://www.gnu.org/software/libc/manual/html_node/Replacing-malloc.html> Thanks. This is useful information. I'm a bit concerned about the status of memalign, etc. though as described there. If malloc is replaced but the memalign family is not, calls to whichever function is not replaced will either use the system malloc (producing a result that's unsafe to pass to free) or will call the interposed malloc then assume they can doctor its book-keeping structures as if they matched the system malloc. Neither of these is safe. In musl, what we should probably do is have extra weak+hidden aliases, so that memalign can do something like: if (malloc != __internal_malloc) return 0; Maybe something similar would be appropriate in glibc? > The remaining undocumented aspects concern cyclic dependencies, such > as the suitability of certain TLS models for implementing a custom > malloc, or using memory-allocating glibc functions such as fopen or > backtrace from the allocator itself. > > In practice, The "in practice" here imposes strong constraints on implementation internals -- things could stop working at any time in the future if changes are made to ways/places in which malloc can be called. This could especially bite glibc if further alloca/large stack usage is replaced with malloc. > malloc interposition works extremely well and causes few > issues due to interposition itself. Obviously, there are bugs, but > most of them would still be bugs if the allocator was non-interposing. > (Examples are lots of initial-exec TLS data, and incorrect alignment > for allocations.) > > I believe musl uses less malloc internally, so it should be even more > compatible with an interposing malloc implementation than glibc. Indeed. But while musl uses less malloc, it also tries to be more opaque and provide fewer guarantees about implementation internals, leaving more free to change. There are some functions that musl makes AS-safe, beyond what POSIX requires, where I intended for the safety to actually be a public property, and I think those would be a good basis for what's allowed to be called from a malloc implementation, if we actually take the effort to document them. This would include a lot of useful things like dprintf, snprintf, mutex lock and (non-robust-type) unlock, semaphore wait (post is already AS-safe by POSIX), ... Unfortunately it doesn't even include mmap/munmap right now, due to tricks we're doing to work around the unsafety (inherent race conditions) in Linux's robust mutex support (I think glibc bug #14485 is related). I don't think this actually affects malloc though, and mmap/munmap could just be explicitly allowed for use in malloc even without documenting them as AS-safe. I think allowing any use of stdio (meaning functions that actually operate on FILEs, not counting snprintf, etc.) from malloc is a bad idea. Some functions (getline/getdelim) use malloc and some FILE types will necessarily (memstream) or potentially (cookie) involve malloc in potentially complex ways. A more interesting question is whether pthread functions, especially pthread_create (e.g. background thread managing migration between heaps) are permissible. They seem useful but also hard to guarantee safe to use in this context. Rich ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility [not found] ` <878t9vlzh1.fsf@mid.deneb.enyo.de> 2018-04-10 18:39 ` Rich Felker @ 2018-04-10 20:33 ` Szabolcs Nagy 2018-04-10 20:44 ` Rich Felker 2018-04-18 20:50 ` Florian Weimer 1 sibling, 2 replies; 16+ messages in thread From: Szabolcs Nagy @ 2018-04-10 20:33 UTC (permalink / raw) To: musl; +Cc: Rich Felker, Florian Weimer * Florian Weimer <fw@deneb.enyo.de> [2018-04-10 19:53:46 +0200]: > We have some documentation nowadays: > > <https://www.gnu.org/software/libc/manual/html_node/Replacing-malloc.html> > > The remaining undocumented aspects concern cyclic dependencies, such > as the suitability of certain TLS models for implementing a custom > malloc, or using memory-allocating glibc functions such as fopen or > backtrace from the allocator itself. > it's not documented what the interposed malloc may or may not do. cyclic dependency (because malloc calls a memory-allocating libc function) is one issue, but in general the libc can be in an inconsistent state at the point of an internal malloc call (e.g. some lock is held that should never be held when user code is running) and thus certain other libc functions may not operate as expected in an interposer. documenting the set of libc apis that work as expected in an interposer is difficult. > In practice, malloc interposition works extremely well and causes few > issues due to interposition itself. Obviously, there are bugs, but > most of them would still be bugs if the allocator was non-interposing. > (Examples are lots of initial-exec TLS data, and incorrect alignment > for allocations.) > even with the very small set of libc apis that a malloc interposer may want to use, there known (but undocumented) caveats: - glibc dlsym calls calloc (bites anybody who tries to wrap calloc in the usual way) - general dynamic tls may call malloc and uses global dl lock so using tls can cause recursion or deadlock. - using stdio for logging is problematic because of stdio locks and recursive malloc calls. - malloc may get called before the first ctor and after the last dtor. - malloc may be called on a thread with small stack (e.g. gai helper thread) so it should not have excessive stack usage. - glibc fork tries to reset the malloc state before calling the child atfork handler in an attempt to support non-as-safe fork handlers, code that relies on this can get broken with malloc interposition. interposition only seems to work extremely well because users already fixed their code and things dont change much in this space. > I believe musl uses less malloc internally, so it should be even more > compatible with an interposing malloc implementation than glibc. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-10 20:33 ` Szabolcs Nagy @ 2018-04-10 20:44 ` Rich Felker 2018-04-10 21:17 ` Szabolcs Nagy 2018-04-18 20:50 ` Florian Weimer 1 sibling, 1 reply; 16+ messages in thread From: Rich Felker @ 2018-04-10 20:44 UTC (permalink / raw) To: musl; +Cc: Florian Weimer On Tue, Apr 10, 2018 at 10:33:55PM +0200, Szabolcs Nagy wrote: > * Florian Weimer <fw@deneb.enyo.de> [2018-04-10 19:53:46 +0200]: > > We have some documentation nowadays: > > > > <https://www.gnu.org/software/libc/manual/html_node/Replacing-malloc.html> > > > > The remaining undocumented aspects concern cyclic dependencies, such > > as the suitability of certain TLS models for implementing a custom > > malloc, or using memory-allocating glibc functions such as fopen or > > backtrace from the allocator itself. > > > > it's not documented what the interposed malloc may or may not do. > > cyclic dependency (because malloc calls a memory-allocating libc > function) is one issue, but in general the libc can be in an > inconsistent state at the point of an internal malloc call (e.g. > some lock is held that should never be held when user code is > running) and thus certain other libc functions may not operate > as expected in an interposer. > > documenting the set of libc apis that work as expected in an > interposer is difficult. I think limiting to AS-safe plus a small list of additional allowed functions is reasonably easy to guarantee works. > > In practice, malloc interposition works extremely well and causes few > > issues due to interposition itself. Obviously, there are bugs, but > > most of them would still be bugs if the allocator was non-interposing. > > (Examples are lots of initial-exec TLS data, and incorrect alignment > > for allocations.) > > even with the very small set of libc apis that a malloc > interposer may want to use, there known (but undocumented) > caveats: > > - glibc dlsym calls calloc (bites anybody who tries to wrap > calloc in the usual way) Yes. While I think it's unreasonable that glibc calls calloc from dlsym, I also think it's unreasonable to be calling dlsym from a malloc replacement. I don't think I would include dlsym or other heavy, AS-unsafe functions in an allowed-list for musl. > - general dynamic tls may call malloc and uses global dl > lock so using tls can cause recursion or deadlock. Yes but this is a known glibc bug that's hopefully on the way to being fixed. A robust/correct implementation (where TLS works in signal handlers and without possibility of OOM-crashing) can't allocate at access time. > - using stdio for logging is problematic because of stdio > locks and recursive malloc calls. Yes. > - malloc may get called before the first ctor and after > the last dtor. An implementation could avoid this, but yes, I agree, there should be no contract that it won't. > - malloc may be called on a thread with small stack > (e.g. gai helper thread) so it should not have excessive > stack usage. Certainly increasing the default thread stack size could be a requirement to use malloc replacements that have large stack usage, but I tend to doubt that would happen. If malloc has large stack usage, it's likely to be very slow. > - glibc fork tries to reset the malloc state before > calling the child atfork handler in an attempt to support > non-as-safe fork handlers, code that relies on this can > get broken with malloc interposition. Interposed mallocs probably try to support this with pthread_atfork, but state after MT-fork is permanently an AS context anyway. > interposition only seems to work extremely well because > users already fixed their code and things dont change > much in this space. Maybe. :) Rich ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-10 20:44 ` Rich Felker @ 2018-04-10 21:17 ` Szabolcs Nagy 2018-04-15 11:52 ` ardi 0 siblings, 1 reply; 16+ messages in thread From: Szabolcs Nagy @ 2018-04-10 21:17 UTC (permalink / raw) To: musl; +Cc: Florian Weimer * Rich Felker <dalias@libc.org> [2018-04-10 16:44:11 -0400]: > On Tue, Apr 10, 2018 at 10:33:55PM +0200, Szabolcs Nagy wrote: > > - glibc dlsym calls calloc (bites anybody who tries to wrap > > calloc in the usual way) > > Yes. While I think it's unreasonable that glibc calls calloc from > dlsym, I also think it's unreasonable to be calling dlsym from a > malloc replacement. I don't think I would include dlsym or other > heavy, AS-unsafe functions in an allowed-list for musl. then the wrappers with dlsym(RTLD_NEXT,sym) would not work. (malloc checkers, valgrind, sanitizers etc all do it) in glibc there is __libc_calloc so you can get the address without dlsym, but in general you cannot wrap libc functions for instrumentation if dlsym is disallowed. i guess one could run dlsym early to initialize global pointers but then the interceptor still has to handle the case when it gets called before the appropriate dlsym is done, so users need to know what dlsym might call and how to do the initialization early enough. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-10 21:17 ` Szabolcs Nagy @ 2018-04-15 11:52 ` ardi 2018-04-16 4:19 ` Markus Wichmann 0 siblings, 1 reply; 16+ messages in thread From: ardi @ 2018-04-15 11:52 UTC (permalink / raw) To: musl On Tue, Apr 10, 2018 at 11:17 PM, Szabolcs Nagy <nsz@port70.net> wrote: > > then the wrappers with dlsym(RTLD_NEXT,sym) would not work. > (malloc checkers, valgrind, sanitizers etc all do it) I've been using ElectricFence, as my only memory debugger since 1996 or so; mostly with the libc of commercial Unices, but also with glibc in Linux, and with the OSX libc. I never considered I could run into the issues commented in this thread, and in fact I never faced these issues and it always worked as expected (however, I must admit I only use multithreading for accelerating clearly isolated math-intensive loops that don't call malloc-related functions from inside the loop). Said this, when I'm linking with ElectricFence, my brain has the "hack mode flag" ON (I mean, I always had the feeling that I'm working with a temporary hack that can fail whenever my link line contains -lefence , and I'm aware that things can go wrong --I didn't consider thread safety, but anyway I know ElectricFence can fail if the OS syscalls that allocate protected memory at buffer ends change their behaviour in newer versions, or if there's some OS/CPU-dependent subtlety with alignment, etc...) I've not tried to use ElectricFence with musl yet... but reading this, can I suppose it won't work? Is there any "hack mode ON" procedure (yet easy) that would allow to use ElectricFence (assuming non-threaded code, which is always my case). I agree with your commitment to correctness, and I'm not asking for a safe and guaranteed implementation of function interposition, just that sometimes I need to break my binaries to make them crash hard as soon as pointer accesses a byte it shouldn't access. Cheers, ardi ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-15 11:52 ` ardi @ 2018-04-16 4:19 ` Markus Wichmann 2018-04-16 4:40 ` Rich Felker 0 siblings, 1 reply; 16+ messages in thread From: Markus Wichmann @ 2018-04-16 4:19 UTC (permalink / raw) To: musl On Sun, Apr 15, 2018 at 01:52:10PM +0200, ardi wrote: > On Tue, Apr 10, 2018 at 11:17 PM, Szabolcs Nagy <nsz@port70.net> wrote: > > > > then the wrappers with dlsym(RTLD_NEXT,sym) would not work. > > (malloc checkers, valgrind, sanitizers etc all do it) > > I've been using ElectricFence, as my only memory debugger since 1996 > or so; mostly with the libc of commercial Unices, but also with glibc > in Linux, and with the OSX libc. I never considered I could run into > the issues commented in this thread, and in fact I never faced these > issues and it always worked as expected (however, I must admit I only > use multithreading for accelerating clearly isolated math-intensive > loops that don't call malloc-related functions from inside the loop). > > Said this, when I'm linking with ElectricFence, my brain has the "hack > mode flag" ON (I mean, I always had the feeling that I'm working with > a temporary hack that can fail whenever my link line contains -lefence > , and I'm aware that things can go wrong --I didn't consider thread > safety, but anyway I know ElectricFence can fail if the OS syscalls > that allocate protected memory at buffer ends change their behaviour > in newer versions, or if there's some OS/CPU-dependent subtlety with > alignment, etc...) > > I've not tried to use ElectricFence with musl yet... but reading this, > can I suppose it won't work? Is there any "hack mode ON" procedure > (yet easy) that would allow to use ElectricFence (assuming > non-threaded code, which is always my case). > > I agree with your commitment to correctness, and I'm not asking for a > safe and guaranteed implementation of function interposition, just > that sometimes I need to break my binaries to make them crash hard as > soon as pointer accesses a byte it shouldn't access. > > Cheers, > > ardi So long as you refrain from using dynamic linking (because of the memory donation) and calloc() and memalign() (and posix_memalign()) are unused or overloaded, you should be fine. Both of these functions use the internal bookkeeping of musl's malloc. calloc() uses it to figure out if a chunk was mmapped (in which case no initialization is necessary), and memalign() uses it to construct a second chunk header to cause the returned pointer to be aligned. Most of the questioning here arose from that first part. Those are the two big problems, actually, we need an interface to donate memory to the malloc implementation, and the malloc implementation needs to provide all of the hairier functions like memalign(). And we currently have no way of enforcing either of these. Ciao, Markus ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-16 4:19 ` Markus Wichmann @ 2018-04-16 4:40 ` Rich Felker 2018-04-18 20:35 ` Rich Felker 0 siblings, 1 reply; 16+ messages in thread From: Rich Felker @ 2018-04-16 4:40 UTC (permalink / raw) To: musl On Mon, Apr 16, 2018 at 06:19:24AM +0200, Markus Wichmann wrote: > On Sun, Apr 15, 2018 at 01:52:10PM +0200, ardi wrote: > > On Tue, Apr 10, 2018 at 11:17 PM, Szabolcs Nagy <nsz@port70.net> wrote: > > > > > > then the wrappers with dlsym(RTLD_NEXT,sym) would not work. > > > (malloc checkers, valgrind, sanitizers etc all do it) > > > > I've been using ElectricFence, as my only memory debugger since 1996 > > or so; mostly with the libc of commercial Unices, but also with glibc > > in Linux, and with the OSX libc. I never considered I could run into > > the issues commented in this thread, and in fact I never faced these > > issues and it always worked as expected (however, I must admit I only > > use multithreading for accelerating clearly isolated math-intensive > > loops that don't call malloc-related functions from inside the loop). > > > > Said this, when I'm linking with ElectricFence, my brain has the "hack > > mode flag" ON (I mean, I always had the feeling that I'm working with > > a temporary hack that can fail whenever my link line contains -lefence > > , and I'm aware that things can go wrong --I didn't consider thread > > safety, but anyway I know ElectricFence can fail if the OS syscalls > > that allocate protected memory at buffer ends change their behaviour > > in newer versions, or if there's some OS/CPU-dependent subtlety with > > alignment, etc...) > > > > I've not tried to use ElectricFence with musl yet... but reading this, > > can I suppose it won't work? Is there any "hack mode ON" procedure > > (yet easy) that would allow to use ElectricFence (assuming > > non-threaded code, which is always my case). > > > > I agree with your commitment to correctness, and I'm not asking for a > > safe and guaranteed implementation of function interposition, just > > that sometimes I need to break my binaries to make them crash hard as > > soon as pointer accesses a byte it shouldn't access. > > > > Cheers, > > > > ardi > > So long as you refrain from using dynamic linking (because of the memory > donation) This is only a small part of the reason you can't use dynamic linking. The other big part is that references in libc.so are bound at libc.so link time, so functions like getline, open_memstream, strdup, etc. will return pointers that won't be valid for you to free. > and calloc() and memalign() (and posix_memalign()) are unused > or overloaded, you should be fine. Both of these functions use the > internal bookkeeping of musl's malloc. calloc() uses it to figure out if > a chunk was mmapped (in which case no initialization is necessary), and > memalign() uses it to construct a second chunk header to cause the > returned pointer to be aligned. Yes, but this rule always applies for interposing, with any implementation. It's not musl-specific. > Most of the questioning here arose from that first part. Those are the > two big problems, actually, we need an interface to donate memory to the > malloc implementation, This isn't needed. It's fine for donation to donate to the internal (unused) implementation if malloc is interposed, or for donation not to happen at all. I don't think it's a good idea to create a public interposable API for donation. The big thing that does need to happen is getting rid of the call to free() to do the donation, which is unsafe/incorrect if it's interposed. Alexander Monakov's patch (which looks ok to commit with minor changes described in the thread) should fix that. > and the malloc implementation needs to provide > all of the hairier functions like memalign(). And we currently have no > way of enforcing either of these. A way to enforce this was discussed earlier in the thread, so it looks doable. Rich ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-16 4:40 ` Rich Felker @ 2018-04-18 20:35 ` Rich Felker 2018-04-19 18:27 ` Szabolcs Nagy 0 siblings, 1 reply; 16+ messages in thread From: Rich Felker @ 2018-04-18 20:35 UTC (permalink / raw) To: musl On Mon, Apr 16, 2018 at 12:40:06AM -0400, Rich Felker wrote: > On Mon, Apr 16, 2018 at 06:19:24AM +0200, Markus Wichmann wrote: > > On Sun, Apr 15, 2018 at 01:52:10PM +0200, ardi wrote: > > [...] > > > > So long as you refrain from using dynamic linking (because of the memory > > donation) > > This is only a small part of the reason you can't use dynamic linking. > The other big part is that references in libc.so are bound at libc.so > link time, so functions like getline, open_memstream, strdup, etc. > will return pointers that won't be valid for you to free. > > > and calloc() and memalign() (and posix_memalign()) are unused > > or overloaded, you should be fine. Both of these functions use the > > internal bookkeeping of musl's malloc. calloc() uses it to figure out if > > a chunk was mmapped (in which case no initialization is necessary), and > > memalign() uses it to construct a second chunk header to cause the > > returned pointer to be aligned. > > Yes, but this rule always applies for interposing, with any > implementation. It's not musl-specific. > > > Most of the questioning here arose from that first part. Those are the > > two big problems, actually, we need an interface to donate memory to the > > malloc implementation, > > This isn't needed. It's fine for donation to donate to the internal > (unused) implementation if malloc is interposed, or for donation not > to happen at all. I don't think it's a good idea to create a public > interposable API for donation. > > The big thing that does need to happen is getting rid of the call to > free() to do the donation, which is unsafe/incorrect if it's > interposed. Alexander Monakov's patch (which looks ok to commit with > minor changes described in the thread) should fix that. > > > and the malloc implementation needs to provide > > all of the hairier functions like memalign(). And we currently have no > > way of enforcing either of these. > > A way to enforce this was discussed earlier in the thread, so it looks > doable. Today I pushed changes which should make malloc replacement/interposition work reliably as long as you only use AS-safe functions. If you try this, please let us know how it turns out and if you run into any unexpected problems. Rich ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-18 20:35 ` Rich Felker @ 2018-04-19 18:27 ` Szabolcs Nagy 2018-04-19 19:11 ` Rich Felker 0 siblings, 1 reply; 16+ messages in thread From: Szabolcs Nagy @ 2018-04-19 18:27 UTC (permalink / raw) To: musl * Rich Felker <dalias@libc.org> [2018-04-18 16:35:56 -0400]: > Today I pushed changes which should make malloc > replacement/interposition work reliably as long as you only use > AS-safe functions. If you try this, please let us know how it turns > out and if you run into any unexpected problems. there was a report on irc that this change in memalign has falsepositives: - if (len > SIZE_MAX - align) { + if (len > SIZE_MAX - align || free != __internal_free) { in particular &free can point to a plt entry in the main executable while &__internal_free is in libc.so so they compare unequal. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-19 18:27 ` Szabolcs Nagy @ 2018-04-19 19:11 ` Rich Felker 0 siblings, 0 replies; 16+ messages in thread From: Rich Felker @ 2018-04-19 19:11 UTC (permalink / raw) To: musl On Thu, Apr 19, 2018 at 08:27:44PM +0200, Szabolcs Nagy wrote: > * Rich Felker <dalias@libc.org> [2018-04-18 16:35:56 -0400]: > > Today I pushed changes which should make malloc > > replacement/interposition work reliably as long as you only use > > AS-safe functions. If you try this, please let us know how it turns > > out and if you run into any unexpected problems. > > there was a report on irc that this change in memalign has falsepositives: > > - if (len > SIZE_MAX - align) { > + if (len > SIZE_MAX - align || free != __internal_free) { > > in particular &free can point to a plt entry in the main executable > while &__internal_free is in libc.so so they compare unequal. Indeed, this will need to be reworked. I'll probably remove these checks for now so that git master is not broken. Rich ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-10 20:33 ` Szabolcs Nagy 2018-04-10 20:44 ` Rich Felker @ 2018-04-18 20:50 ` Florian Weimer 2018-04-18 21:06 ` Rich Felker 1 sibling, 1 reply; 16+ messages in thread From: Florian Weimer @ 2018-04-18 20:50 UTC (permalink / raw) To: musl; +Cc: Rich Felker * Szabolcs Nagy: > even with the very small set of libc apis that a malloc > interposer may want to use, there known (but undocumented) > caveats: > > - glibc dlsym calls calloc (bites anybody who tries to wrap > calloc in the usual way) That's not what I meant with an interposed malloc. Any such partial interposition is going to be very tricky indeed. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: tcmalloc compatibility 2018-04-18 20:50 ` Florian Weimer @ 2018-04-18 21:06 ` Rich Felker 0 siblings, 0 replies; 16+ messages in thread From: Rich Felker @ 2018-04-18 21:06 UTC (permalink / raw) To: musl On Wed, Apr 18, 2018 at 10:50:24PM +0200, Florian Weimer wrote: > * Szabolcs Nagy: > > > even with the very small set of libc apis that a malloc > > interposer may want to use, there known (but undocumented) > > caveats: > > > > - glibc dlsym calls calloc (bites anybody who tries to wrap > > calloc in the usual way) > > That's not what I meant with an interposed malloc. Any such partial > interposition is going to be very tricky indeed. I don't understand. By "partial interposition" do you mean partial in the sense of "wrapping and calling the original via dlsym" or in the sense of "replacing some but not all allocator functions"? Rich ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2018-04-19 19:11 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-04-10 10:22 tcmalloc compatibility Denis V. Razumovsky 2018-04-10 14:33 ` Rich Felker 2018-04-10 14:45 ` Bobby Powers 2018-04-10 15:35 ` Rich Felker [not found] ` <878t9vlzh1.fsf@mid.deneb.enyo.de> 2018-04-10 18:39 ` Rich Felker 2018-04-10 20:33 ` Szabolcs Nagy 2018-04-10 20:44 ` Rich Felker 2018-04-10 21:17 ` Szabolcs Nagy 2018-04-15 11:52 ` ardi 2018-04-16 4:19 ` Markus Wichmann 2018-04-16 4:40 ` Rich Felker 2018-04-18 20:35 ` Rich Felker 2018-04-19 18:27 ` Szabolcs Nagy 2018-04-19 19:11 ` Rich Felker 2018-04-18 20:50 ` Florian Weimer 2018-04-18 21:06 ` Rich Felker
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).