* Todo for release? @ 2012-08-13 18:53 Rich Felker 2012-08-13 21:31 ` Szabolcs Nagy ` (2 more replies) 0 siblings, 3 replies; 38+ messages in thread From: Rich Felker @ 2012-08-13 18:53 UTC (permalink / raw) To: musl Starting a new thread since the old one got too long and OT.. :-) New stuff we have so far: - Blowfish crypt - MIPS dynamic linker - Major MIPS bug fixes - ARM hard float support - BSD fgetln function - Major bug fix in wcsstr - Optimized memcpy for i386 and x86_64 - Public exposure for getdents - Added significand function to math lib The requested stuff that's still pending (with person who requested or is working on it): - Exception behavior in i386/x86_64 exponential asm (nsz). - Finer-grained _XOPEN_SOURCE (Luca) - Support for __progname (Daniel) - MD5 and SHA crypt (nsz?) In addition, there are the GNU hash and dladdr patches (Boris) pending, which I don't want to overlook, but I think unless I manage to get a discussion of simplifying them finished really soon and find some time to thoroughly test them, it's best to hold off on this until just after the next release so we have plenty of time to test and tune it before another release. Anything else I missed? Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-13 18:53 Todo for release? Rich Felker @ 2012-08-13 21:31 ` Szabolcs Nagy 2012-08-13 21:53 ` Rich Felker 2012-08-13 22:20 ` Solar Designer 2012-08-15 4:08 ` Rich Felker 2012-08-15 22:44 ` boris brezillon 2 siblings, 2 replies; 38+ messages in thread From: Szabolcs Nagy @ 2012-08-13 21:31 UTC (permalink / raw) To: musl * Rich Felker <dalias@aerifal.cx> [2012-08-13 14:53:30 -0400]: > - MD5 and SHA crypt (nsz?) i only have code for the hashes, not crypt it seems these crypt schemes are fairly ugly i don't mind if their implementation is delayed md5 based crypt is not recommended anymore http://phk.freebsd.dk/sagas/md5crypt_eol.html the sha2 based crypt seems to be designed recently and the spec has a public domain implementation http://www.akkadia.org/drepper/SHA-crypt.txt ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-13 21:31 ` Szabolcs Nagy @ 2012-08-13 21:53 ` Rich Felker 2012-08-13 22:06 ` Solar Designer 2012-08-13 22:20 ` Solar Designer 1 sibling, 1 reply; 38+ messages in thread From: Rich Felker @ 2012-08-13 21:53 UTC (permalink / raw) To: musl On Mon, Aug 13, 2012 at 11:31:54PM +0200, Szabolcs Nagy wrote: > * Rich Felker <dalias@aerifal.cx> [2012-08-13 14:53:30 -0400]: > > - MD5 and SHA crypt (nsz?) > > i only have code for the hashes, not crypt > > it seems these crypt schemes are fairly ugly > i don't mind if their implementation is delayed It looks like the API the hash functions provide matches closely what the BSD crypt functions expect, so I think we could potentially just use or adapt one of them.. > md5 based crypt is not recommended anymore > http://phk.freebsd.dk/sagas/md5crypt_eol.html Indeed. But is it used in existing Linux user databases on any significant scale? If not, I agree we can just drop it. > the sha2 based crypt seems to be designed recently > and the spec has a public domain implementation > http://www.akkadia.org/drepper/SHA-crypt.txt I'm confused by all the SHA names (1/2/256/512)... Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-13 21:53 ` Rich Felker @ 2012-08-13 22:06 ` Solar Designer 2012-08-14 15:02 ` Szabolcs Nagy 0 siblings, 1 reply; 38+ messages in thread From: Solar Designer @ 2012-08-13 22:06 UTC (permalink / raw) To: musl On Mon, Aug 13, 2012 at 05:53:44PM -0400, Rich Felker wrote: > On Mon, Aug 13, 2012 at 11:31:54PM +0200, Szabolcs Nagy wrote: > > md5 based crypt is not recommended anymore > > http://phk.freebsd.dk/sagas/md5crypt_eol.html > > Indeed. But is it used in existing Linux user databases on any > significant scale? It is. > If not, I agree we can just drop it. We should support it. Maybe use my MD5 code, but for md5crypt write new code to avoid the beerware license (I would be happy to buy phk a beer, but having to mention another license for a component in musl's license is not nice). > I'm confused by all the SHA names (1/2/256/512)... You need sha512crypt and sha256crypt. SHA-1 is irrelevant (not used in any common crypt(3) flavor). SHA-2 is a common name for the SHA-224/256/384/512 primitives (although these are actually different). Of sha512crypt and sha256crypt, only the former is commonly used, but you may choose to support both anyway (systems generally support both). The high-level structure of md5crypt, sha512crypt, and sha256crypt is similar, but it'd be tricky/unreasonable to exploit that for reduced code size as you'd likely increase source code complexity and make the code slower (important in case of sha512crypt and sha256crypt, which support variable iteration counts). Alexander ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-13 22:06 ` Solar Designer @ 2012-08-14 15:02 ` Szabolcs Nagy 2012-08-15 0:30 ` Szabolcs Nagy 0 siblings, 1 reply; 38+ messages in thread From: Szabolcs Nagy @ 2012-08-14 15:02 UTC (permalink / raw) To: musl * Solar Designer <solar@openwall.com> [2012-08-14 02:06:01 +0400]: > Maybe use my MD5 code, but for md5crypt write new code to avoid the > beerware license (I would be happy to buy phk a beer, but having to > mention another license for a component in musl's license is not nice). > the license seems to be changed to 2clause bsd in 2003 http://svnweb.FreeBSD.org/base/head/lib/libcrypt/crypt-md5.c?view=log (openbsd still use the beerware version though) ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-14 15:02 ` Szabolcs Nagy @ 2012-08-15 0:30 ` Szabolcs Nagy 0 siblings, 0 replies; 38+ messages in thread From: Szabolcs Nagy @ 2012-08-15 0:30 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 595 bytes --] * Szabolcs Nagy <nsz@port70.net> [2012-08-14 17:02:37 +0200]: > * Solar Designer <solar@openwall.com> [2012-08-14 02:06:01 +0400]: > > Maybe use my MD5 code, but for md5crypt write new code to avoid the > > beerware license (I would be happy to buy phk a beer, but having to > > mention another license for a component in musl's license is not nice). > > > > the license seems to be changed to 2clause bsd in 2003 > http://svnweb.FreeBSD.org/base/head/lib/libcrypt/crypt-md5.c?view=log > > (openbsd still use the beerware version though) i attached a crypt_md5 based on this (not complete) [-- Attachment #2: crypt_md5.c --] [-- Type: text/x-csrc, Size: 4148 bytes --] /* * origin: FreeBSD /usr/src/lib/libcrypt/crypt-md5.c * This is a heavily modified version for musl libc. */ /*- * Copyright (c) 2003 Poul-Henning Kamp * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include <string.h> #include <stdint.h> #include "md5.h" static unsigned char itoa64[] = "./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"; static char *to64(char *s, unsigned int v, int n) { while (--n >= 0) { *s++ = itoa64[v & 0x3f]; v >>= 6; } return s; } static char *md5crypt(const char *key, const char *setting, char *output) { MD5_CTX ctx; unsigned char md[MD5_DIGEST_LENGTH]; unsigned int i; unsigned int klen = strlen(key); // TODO: key limit: fit into unsigned int unsigned int slen; const char *salt; char *p; // TODO: missing $1$ or >8byte salt? /* check setting: $1$ followed by max 8 bytes salt until '$' or '\0' */ if (setting[0] != '$' || setting[1] != '1' || setting[2] != '$') return 0; salt = setting + 3; for (i = 0; i < 8 && salt[i] && salt[i] != '$'; i++); slen = i; /* md5(key salt key) */ MD5_Init(&ctx); MD5_Update(&ctx, key, klen); MD5_Update(&ctx, salt, slen); MD5_Update(&ctx, key, klen); MD5_Final(md, &ctx); /* md5(key $1$ salt repeated-md weird-key[0]-0) */ MD5_Init(&ctx); MD5_Update(&ctx, key, klen); MD5_Update(&ctx, setting, 3 + slen); for (i = klen; i > sizeof md; i -= sizeof md) MD5_Update(&ctx, md, sizeof md); MD5_Update(&ctx, md, i); md[0] = 0; for (i = klen; i; i >>= 1) if (i & 1) MD5_Update(&ctx, md, 1); else MD5_Update(&ctx, key, 1); MD5_Final(md, &ctx); /* On a 60 MHz Pentium this takes 34 ms */ for (i = 0; i < 1000; i++) { MD5_Init(&ctx); if (i % 2) MD5_Update(&ctx, key, klen); else MD5_Update(&ctx, md, sizeof md); if (i % 3) MD5_Update(&ctx, salt, slen); if (i % 7) MD5_Update(&ctx, key, klen); if (i % 2) MD5_Update(&ctx, md, sizeof md); else MD5_Update(&ctx, key, klen); MD5_Final(md, &ctx); } /* output is $1$<salt>$<hash> */ memcpy(output, setting, 3 + slen); p = output + 3 + slen; *p++ = '$'; // TODO: assumes >24 bit int p = to64(p, (md[0]<<16)|(md[6]<<8)|md[12], 4); p = to64(p, (md[1]<<16)|(md[7]<<8)|md[13], 4); p = to64(p, (md[2]<<16)|(md[8]<<8)|md[14], 4); p = to64(p, (md[3]<<16)|(md[9]<<8)|md[15], 4); p = to64(p, (md[4]<<16)|(md[10]<<8)|md[5], 4); p = to64(p, md[11], 2); *p = 0; return output; } char *__crypt_md5(const char *key, const char *setting, char *output) { static const char testkey[] = "Xy01@#\x01\x02\x80\x7f\xff\r\n\x81\t !"; static const char testsetting[] = "$1$abcd0123$"; static const char testhash[] = "$1$abcd0123$9Qcg8DyviekV3tDGMZynJ1"; char testbuf[64]; char *p, *q; p = md5crypt(key, setting, output); q = md5crypt(testkey, testsetting, testbuf); if (!p || q != testbuf || memcmp(testbuf, testhash, sizeof testhash)) return "*"; return p; } ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-13 21:31 ` Szabolcs Nagy 2012-08-13 21:53 ` Rich Felker @ 2012-08-13 22:20 ` Solar Designer 2012-08-14 1:46 ` Rich Felker 1 sibling, 1 reply; 38+ messages in thread From: Solar Designer @ 2012-08-13 22:20 UTC (permalink / raw) To: musl On Mon, Aug 13, 2012 at 11:31:54PM +0200, Szabolcs Nagy wrote: > the sha2 based crypt seems to be designed recently > and the spec has a public domain implementation > http://www.akkadia.org/drepper/SHA-crypt.txt Unfortunately, the reference implementation uses alloca() on both salt and key strings. glibc has recently fixed that by using malloc() and returning NULL on its failure, but that's not great. Also, if potentially unreasonably long running time is a concern, it should be noted that for md5crypt and sha*crypt it is roughly proportional to password length (modulo block size of the underlying primitive). So e.g. a 1 million char password (which may realistically be passed to libc's crypt() e.g. via a scripting language) may take thousands of times longer to be hashed than the sysadmin had intended by tuning the iteration count. I'm not sure whether and how a libc should deal with that. In a sense, it is similar to the issue of high iteration counts, but it's worse in that the input that may trigger the issue very often comes from a remote system. For the extended DES-based crypt() hashes that we now support, this issue mostly does not arise since the password (even if very long, which is supported) is passed through just one instance of DES block-by-block, which is quick. The multiple iterations loop is then applied to the "compressed" version of the password. For bcrypt hashes, the issue does not arise because they truncate passwords at 72 characters (not great, but that's how they're defined, and it's good enough for practical purposes so far). Alexander ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-13 22:20 ` Solar Designer @ 2012-08-14 1:46 ` Rich Felker 2012-08-14 2:13 ` Solar Designer 0 siblings, 1 reply; 38+ messages in thread From: Rich Felker @ 2012-08-14 1:46 UTC (permalink / raw) To: musl On Tue, Aug 14, 2012 at 02:20:58AM +0400, Solar Designer wrote: > On Mon, Aug 13, 2012 at 11:31:54PM +0200, Szabolcs Nagy wrote: > > the sha2 based crypt seems to be designed recently > > and the spec has a public domain implementation > > http://www.akkadia.org/drepper/SHA-crypt.txt > > Unfortunately, the reference implementation uses alloca() on both salt > and key strings. Why? Does it need working space proportional to the input length? > glibc has recently fixed that by using malloc() and > returning NULL on its failure, but that's not great. Hmm, and as you've pointed out several times, that means the daemon will almost surely crash, since few of them check for failure... > Also, if potentially unreasonably long running time is a concern, it > should be noted that for md5crypt and sha*crypt it is roughly > proportional to password length (modulo block size of the underlying > primitive). So e.g. a 1 million char password (which may realistically > be passed to libc's crypt() e.g. via a scripting language) may take > thousands of times longer to be hashed than the sysadmin had intended by > tuning the iteration count. > > I'm not sure whether and how a libc should deal with that. In a sense, > it is similar to the issue of high iteration counts, but it's worse in > that the input that may trigger the issue very often comes from a remote > system. In light of both the alloca issue and the way runtime scales with key length, I think we should just put an arbitrary limit on the key length and return failure for longer keys. This should not affect any real-world authentication systems, since the daemon you're attempting to login to will also be placing a (probably much lower) limit on the input buffer size for passwords (if it's not, you can trivially DoS the server by sending gigabyte-long passwords for random users). Something like 128-256 bytes would probably be a very generous limit. > For the extended DES-based crypt() hashes that we now support, this > issue mostly does not arise since the password (even if very long, which > is supported) is passed through just one instance of DES block-by-block, > which is quick. The multiple iterations loop is then applied to the > "compressed" version of the password. > > For bcrypt hashes, the issue does not arise because they truncate > passwords at 72 characters (not great, but that's how they're defined, > and it's good enough for practical purposes so far). Thanks for explaining the issues well. Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-14 1:46 ` Rich Felker @ 2012-08-14 2:13 ` Solar Designer 2012-08-14 2:35 ` Rich Felker 0 siblings, 1 reply; 38+ messages in thread From: Solar Designer @ 2012-08-14 2:13 UTC (permalink / raw) To: musl On Mon, Aug 13, 2012 at 09:46:53PM -0400, Rich Felker wrote: > On Tue, Aug 14, 2012 at 02:20:58AM +0400, Solar Designer wrote: > > On Mon, Aug 13, 2012 at 11:31:54PM +0200, Szabolcs Nagy wrote: > > > the sha2 based crypt seems to be designed recently > > > and the spec has a public domain implementation > > > http://www.akkadia.org/drepper/SHA-crypt.txt > > > > Unfortunately, the reference implementation uses alloca() on both salt > > and key strings. > > Why? Does it need working space proportional to the input length? It uses implementations of SHA-512 and SHA-256 that assume alignment, so it provides such alignment by copying the inputs to aligned buffers if the inputs to crypt() don't happen to be already aligned. The same applies to glibc's md5crypt (but we're not going to use that implementation of md5crypt anyway). > In light of both the alloca issue and the way runtime scales with key > length, I think we should just put an arbitrary limit on the key > length and return failure for longer keys. This should not affect any > real-world authentication systems, since the daemon you're attempting > to login to will also be placing a (probably much lower) limit on the > input buffer size for passwords (if it's not, you can trivially DoS > the server by sending gigabyte-long passwords for random users). > > Something like 128-256 bytes would probably be a very generous limit. Yes, but the failure should be indicated in the way we discussed - those "*0" and "*1" strings, not NULL. Some real-world authentication systems may be affected; it is not unrealistic even for a C program to use a buffer several kilobytes large. Alexander ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-14 2:13 ` Solar Designer @ 2012-08-14 2:35 ` Rich Felker 2012-08-14 2:49 ` Solar Designer 0 siblings, 1 reply; 38+ messages in thread From: Rich Felker @ 2012-08-14 2:35 UTC (permalink / raw) To: musl On Tue, Aug 14, 2012 at 06:13:17AM +0400, Solar Designer wrote: > On Mon, Aug 13, 2012 at 09:46:53PM -0400, Rich Felker wrote: > > On Tue, Aug 14, 2012 at 02:20:58AM +0400, Solar Designer wrote: > > > On Mon, Aug 13, 2012 at 11:31:54PM +0200, Szabolcs Nagy wrote: > > > > the sha2 based crypt seems to be designed recently > > > > and the spec has a public domain implementation > > > > http://www.akkadia.org/drepper/SHA-crypt.txt > > > > > > Unfortunately, the reference implementation uses alloca() on both salt > > > and key strings. > > > > Why? Does it need working space proportional to the input length? > > It uses implementations of SHA-512 and SHA-256 that assume alignment, so > it provides such alignment by copying the inputs to aligned buffers if > the inputs to crypt() don't happen to be already aligned. This could be solved by doing the copy a block at a time, and submitting the blocks to the encryption code a block at a time. Failure to do so is just laziness, and it's the same type of laziness that's all over glibc. If this issue is as simple to solve as it sounds, it might make sense to allow arbitrary key sizes. After all, programs that could be DoS'd by long keys are already going to be limiting key length themselves. If time grew superlinearly in key length, I'd say it should definitely be limited, but since the growth is just linear (the expected growth rate for any interface that takes a string argument), I think it's less clear what should be done. > > In light of both the alloca issue and the way runtime scales with key > > length, I think we should just put an arbitrary limit on the key > > length and return failure for longer keys. This should not affect any > > real-world authentication systems, since the daemon you're attempting > > to login to will also be placing a (probably much lower) limit on the > > input buffer size for passwords (if it's not, you can trivially DoS > > the server by sending gigabyte-long passwords for random users). > > > > Something like 128-256 bytes would probably be a very generous limit. > > Yes, but the failure should be indicated in the way we discussed - those > "*0" and "*1" strings, not NULL. Yes, it should be treated like any other invalid input and hashed to something that can never match. By the way, would you agree that all programs that generate new password hashes should do so by calling crypt twice, the second time using the output of the first as the setting/salt, and verify that the results match? This seems to be the only safe/portable way to make sure you got a valid hash and not an error. Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-14 2:35 ` Rich Felker @ 2012-08-14 2:49 ` Solar Designer 2012-08-14 2:58 ` Rich Felker 0 siblings, 1 reply; 38+ messages in thread From: Solar Designer @ 2012-08-14 2:49 UTC (permalink / raw) To: musl On Mon, Aug 13, 2012 at 10:35:08PM -0400, Rich Felker wrote: > On Tue, Aug 14, 2012 at 06:13:17AM +0400, Solar Designer wrote: > > On Mon, Aug 13, 2012 at 09:46:53PM -0400, Rich Felker wrote: > > > On Tue, Aug 14, 2012 at 02:20:58AM +0400, Solar Designer wrote: > > > > On Mon, Aug 13, 2012 at 11:31:54PM +0200, Szabolcs Nagy wrote: > > > > > http://www.akkadia.org/drepper/SHA-crypt.txt > > > > > > > > Unfortunately, the reference implementation uses alloca() on both salt > > > > and key strings. > > > > > > Why? Does it need working space proportional to the input length? > > > > It uses implementations of SHA-512 and SHA-256 that assume alignment, so > > it provides such alignment by copying the inputs to aligned buffers if > > the inputs to crypt() don't happen to be already aligned. > > This could be solved by doing the copy a block at a time, and > submitting the blocks to the encryption code a block at a time. Note that this moves the copying inside the many-iterations loop, so there will be a difference in speed. That said, yes, e.g. my public domain implementation of MD5 avoids the alignment requirement by possibly copying the block the first time it's used. There may be some performance hit from using it to implement md5crypt as compared to using glibc's implementation of MD5 (because the copying will in fact be in the 1000 iterations loop), but not much - and overall my implementation might be faster (for other reasons), especially on x86, where the copying may be avoided altogether in favor of possibly unaligned accesses. > If this issue is as simple to solve as it sounds, it might make sense > to allow arbitrary key sizes. After all, programs that could be DoS'd > by long keys are already going to be limiting key length themselves. That's wishful thinking. > If time grew superlinearly in key length, I'd say it should definitely > be limited, but since the growth is just linear (the expected growth > rate for any interface that takes a string argument), I think it's > less clear what should be done. Yes, it is not clear what should be done. > > Yes, but the failure should be indicated in the way we discussed - those > > "*0" and "*1" strings, not NULL. > > Yes, it should be treated like any other invalid input and hashed to > something that can never match. By the way, would you agree that all > programs that generate new password hashes should do so by calling > crypt twice, the second time using the output of the first as the > setting/salt, and verify that the results match? This seems to be the > only safe/portable way to make sure you got a valid hash and not an > error. This makes sense, yet it sounds overkill to me. I'd simply check for hash && strlen(hash) >= 13. Alexander ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-14 2:49 ` Solar Designer @ 2012-08-14 2:58 ` Rich Felker 2012-08-14 3:35 ` Solar Designer 0 siblings, 1 reply; 38+ messages in thread From: Rich Felker @ 2012-08-14 2:58 UTC (permalink / raw) To: musl On Tue, Aug 14, 2012 at 06:49:03AM +0400, Solar Designer wrote: > > If this issue is as simple to solve as it sounds, it might make sense > > to allow arbitrary key sizes. After all, programs that could be DoS'd > > by long keys are already going to be limiting key length themselves. > > That's wishful thinking. Are you sure? I haven't read the code lately, but I can't imagine any login daemon is going to be calling realloc() in a loop to read an arbitrarily long password before authentication. That just sounds gratuitously broken (i.e. someone went out of their way to write painful code that does nothing useful and makes their daemon susceptible to DoS). > > If time grew superlinearly in key length, I'd say it should definitely > > be limited, but since the growth is just linear (the expected growth > > rate for any interface that takes a string argument), I think it's > > less clear what should be done. > > Yes, it is not clear what should be done. If it's a toss-up on whether we should limit key length for runtime considerations, I might just go on a basis of how it affects code complexity for handling long keys and thus still limit them. > > something that can never match. By the way, would you agree that all > > programs that generate new password hashes should do so by calling > > crypt twice, the second time using the output of the first as the > > setting/salt, and verify that the results match? This seems to be the > > only safe/portable way to make sure you got a valid hash and not an > > error. > > This makes sense, yet it sounds overkill to me. I'd simply check for > hash && strlen(hash) >= 13. Indeed, strlen(hash)>=13 is certainly a necessary condition, but is it sufficient? I could imagine a hypothetical crypt implementation that puts error messages in the unmatchable hash as a debugging aid to why generating the hash failed, but I agree they probably don't exist. Still, changing your password should not be a frequent action, so it might make the most sense to do the check the way I suggested. Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-14 2:58 ` Rich Felker @ 2012-08-14 3:35 ` Solar Designer 2012-08-14 4:49 ` Rich Felker 0 siblings, 1 reply; 38+ messages in thread From: Solar Designer @ 2012-08-14 3:35 UTC (permalink / raw) To: musl On Mon, Aug 13, 2012 at 10:58:05PM -0400, Rich Felker wrote: > On Tue, Aug 14, 2012 at 06:49:03AM +0400, Solar Designer wrote: > > > If this issue is as simple to solve as it sounds, it might make sense > > > to allow arbitrary key sizes. After all, programs that could be DoS'd > > > by long keys are already going to be limiting key length themselves. > > > > That's wishful thinking. > > Are you sure? I haven't read the code lately, but I can't imagine any > login daemon is going to be calling realloc() in a loop to read an > arbitrarily long password before authentication. That just sounds > gratuitously broken (i.e. someone went out of their way to write > painful code that does nothing useful and makes their daemon > susceptible to DoS). I guess many daemons written in C limit the length at a few kilobytes - which may allow for about 100 times greater than intended (by sysadmin) crypt() running time. For md5crypt and sha*crypt, the first slowdown occurs between length 15 and 16. Then, it does not take explicit realloc() for just the password string to support arbitrarily long passwords. The daemon may be using an abstraction layer for all strings - e.g., qmail, Postfix, and vsftpd have such dynamic string libraries of their own, and overall this is good (it avoids buffer overflows and artificial limits in other places). I don't know if vsftpd would in fact pass arbitrarily long passwords to crypt() - this is worth checking. Finally, some services are written in languages that support dynamically allocated strings natively. I recall that OpenStack's Python code was patched to impose a limit of 4096 chars on passwords recently, specifically in response to risks like what we're discussing here. (And 4096 is still a lot - may allow for some attacks.) > > > something that can never match. By the way, would you agree that all > > > programs that generate new password hashes should do so by calling > > > crypt twice, the second time using the output of the first as the > > > setting/salt, and verify that the results match? This seems to be the > > > only safe/portable way to make sure you got a valid hash and not an > > > error. > > > > This makes sense, yet it sounds overkill to me. I'd simply check for > > hash && strlen(hash) >= 13. > > Indeed, strlen(hash)>=13 is certainly a necessary condition, but is it > sufficient? I could imagine a hypothetical crypt implementation that > puts error messages in the unmatchable hash as a debugging aid to why > generating the hash failed, but I agree they probably don't exist. > Still, changing your password should not be a frequent action, so it > might make the most sense to do the check the way I suggested. An aspect to consider is that if you call crypt() twice for just one request from the user (to set a password), you effectively double the iteration count as it relates to potential CPU time exhaustion DoS attacks (twice more CPU time is consumed per request). So if these attacks are the limiting factor in a sysadmin's ability to set a higher iteration count, this will result in the iteration count being twice lower (and accordingly weaker hashes being used on the system). (If the protocol is such that the same request contains the old password to be checked first, then the difference may be 3/2.) While each individual sysadmin is somewhat unlikely to apply this reasoning and consider password setting as the worst DoS attack vector, I think that overall this may in fact result in somewhat lower iteration counts being used (average across many systems). Alexander ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-14 3:35 ` Solar Designer @ 2012-08-14 4:49 ` Rich Felker 0 siblings, 0 replies; 38+ messages in thread From: Rich Felker @ 2012-08-14 4:49 UTC (permalink / raw) To: musl On Tue, Aug 14, 2012 at 07:35:14AM +0400, Solar Designer wrote: > I guess many daemons written in C limit the length at a few kilobytes - > which may allow for about 100 times greater than intended (by sysadmin) > crypt() running time. For md5crypt and sha*crypt, the first slowdown > occurs between length 15 and 16. > > Then, it does not take explicit realloc() for just the password string > to support arbitrarily long passwords. The daemon may be using an > abstraction layer for all strings - e.g., qmail, Postfix, and vsftpd > have such dynamic string libraries of their own, and overall this is > good (it avoids buffer overflows and artificial limits in other places). I disagree. Avoiding artificial limits almost always means creating difficult-to-debug corner cases when resources are exhausted. It was a popular mantra of the GNU folks in the 80s and 90s, when they boasted how superior their system was to Unix with its hard-coded limits. Of course traditional Unices did have very bad, very low arbitrary limits, and this is what allowed the GNU philosophy to look good, but on a conceptual level, the difference was that the traditional tools with arbitrary limits were able to promise that they would ALWAYS work on conforming input (e.g. text files that met the line-length limit), whereas the GNU utilities would work, well, whenever they didn't run out of memory. Back to the point about logins and daemons that run as root prior to authentication: I consider it a moderate-level security bug for any such program to allow unbounded resource allocation by an unauthenticated client or prior to dropping privileges to the authenticated uses. I'm also pretty cold to the idea of "safe string libraries". Just recently I got to looking inside the MaraDNS code, and its string library, promoted as being extremely secure, just fails to handle allocation failures at all. Sadly this kind of attitude seems to be common. My idea of a safe string library is snprintf. If you use plain C strings, most of them in fixed-size buffers, and never use any function but snprintf (with the correct length argument) to write to them, you're not going to have string overflow exploits. And your code is going to be a lot simpler and more robust than code that's trying to emulate Python/JavaScript/etc.-style strings in C... > I don't know if vsftpd would in fact pass arbitrarily long passwords to > crypt() - this is worth checking. I actually just checked DropBear, and had a hard time finding if/how it limits the length, but it turned out to be a simple 35000-byte limit on packet size in the packet reception code. Presumably the vast majority of that can be the password, if an attacker so desired. > Finally, some services are written in languages that support dynamically > allocated strings natively. I recall that OpenStack's Python code was > patched to impose a limit of 4096 chars on passwords recently, > specifically in response to risks like what we're discussing here. > (And 4096 is still a lot - may allow for some attacks.) This is a great example of why the idea that higher-level languages are more secure than C is such a fallacy... Different language idioms just lead to different things that are easy to get wrong in security-critical ways if you're not careful... Perhaps an ideal language without security issues could be designed, but it would require scrapping the idea that you can pretend resources are infinite and that the runtime magically manages object lifetimes for you. Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-13 18:53 Todo for release? Rich Felker 2012-08-13 21:31 ` Szabolcs Nagy @ 2012-08-15 4:08 ` Rich Felker 2012-08-15 8:55 ` Daniel Cegiełka ` (2 more replies) 2012-08-15 22:44 ` boris brezillon 2 siblings, 3 replies; 38+ messages in thread From: Rich Felker @ 2012-08-15 4:08 UTC (permalink / raw) To: musl On Mon, Aug 13, 2012 at 02:53:30PM -0400, Rich Felker wrote: > The requested stuff that's still pending (with person who requested or > is working on it): > > - Exception behavior in i386/x86_64 exponential asm (nsz). Committed. > - Finer-grained _XOPEN_SOURCE (Luca) On hold pending details on what the real problem is; unlikely to make it in for next release. Luca, if you still want this, please provide details on what issues you're facing that could be solved. I don't want to target old versions of standards unless there's a concrete practical goal. I mentioned one possible approach (using old versions only as a way to reenable stuff that was removed from the standard, not a way to get the entire outdated-standard behavior that would also require removing new symbols) but I haven't heard back on whether that would meet your needs. > - Support for __progname (Daniel) Daniel, any more thoughts on this? Are there lots of programs that want it that can't easily be patched to simply use argv[0] themselves? > - MD5 and SHA crypt (nsz?) Thanks nsz for the further work on MD5. It's looking like this won't be ready in the next few days, but I'd be happy to be proven wrong. > Anything else I missed? I got Landley's request for $CROSS_COMPILER prefix support into configure too. I'd forgotten about that long-standing one. Anything else that's been overlooked for a while? One thing that comes to my mind is updating credits. I actually realized just after the last release I forgot to add rdp and Solar to the COPYRIGHT file for their contributions, and credit for rdp is actually missing from most (all?) of the source files too - I felt really bad about this omission. Please accept my apologies, and let me know if there are any other credits I've forgotten to add. Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 4:08 ` Rich Felker @ 2012-08-15 8:55 ` Daniel Cegiełka 2012-08-15 10:20 ` Szabolcs Nagy 2012-08-15 12:36 ` Rich Felker 2012-08-15 12:57 ` Luca Barbato 2012-08-15 13:27 ` Richard Pennington 2 siblings, 2 replies; 38+ messages in thread From: Daniel Cegiełka @ 2012-08-15 8:55 UTC (permalink / raw) To: musl > >> - Support for __progname (Daniel) > > Daniel, any more thoughts on this? Are there lots of programs that > want it that can't easily be patched to simply use argv[0] themselves? This is not something that is absolutely necessary. __progname quite often is used on *BSD and less on Linux (eg. Owl's msulogin, popa3d)... but __progname is always easy to fix. Here we have the OpenBSD repo and content /bin directories: http://www.openbsd.org/cgi-bin/cvsweb/src/bin/ And here's a list of programs (from /bin/) that require __progname (70% of all): mv systrace md5 cp chmod cat rmail kill sleep rmdir mkdir extern ps df rcp ln date chio domainname stty rm pwd hostname For __progname we probably need to modify (asm) files in the musl/crt/. Best regards, Daniel ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 8:55 ` Daniel Cegiełka @ 2012-08-15 10:20 ` Szabolcs Nagy 2012-08-15 10:53 ` Daniel Cegiełka 2012-08-15 13:32 ` Szabolcs Nagy 2012-08-15 12:36 ` Rich Felker 1 sibling, 2 replies; 38+ messages in thread From: Szabolcs Nagy @ 2012-08-15 10:20 UTC (permalink / raw) To: musl * Daniel Cegie?ka <daniel.cegielka@gmail.com> [2012-08-15 10:55:06 +0200]: > >> - Support for __progname (Daniel) > > > > Daniel, any more thoughts on this? Are there lots of programs that > > want it that can't easily be patched to simply use argv[0] themselves? > > This is not something that is absolutely necessary. __progname quite > often is used on *BSD and less on Linux (eg. Owl's msulogin, > popa3d)... but __progname is always easy to fix. > i think the fact that *bsd uses it is not enough justification openbsd uses it because it's part of their style guide for whatever reason "The __progname string may be used instead of hard-coding the program name." http://www.openbsd.org/cgi-bin/man.cgi?query=style&sektion=9 but we don't support many things from there (like sys/queue.h) i don't think many linux tools uses it as it's not part of the lsb and glibc has its own silly program_invocation_name and program_invocation_short_name (which are aliases to __progname and __progname_full) the main justification i see is that we already support bsd err and warn apis which are required to print the __progname as well (currently they don't and actually a simple warn("hi"); segfaults here with musl but i havent investigated it) ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 10:20 ` Szabolcs Nagy @ 2012-08-15 10:53 ` Daniel Cegiełka 2012-08-15 13:10 ` John Spencer 2012-08-15 13:32 ` Szabolcs Nagy 1 sibling, 1 reply; 38+ messages in thread From: Daniel Cegiełka @ 2012-08-15 10:53 UTC (permalink / raw) To: musl 2012/8/15 Szabolcs Nagy <nsz@port70.net>: > * Daniel Cegie?ka <daniel.cegielka@gmail.com> [2012-08-15 10:55:06 +0200]: >> >> - Support for __progname (Daniel) >> > >> > Daniel, any more thoughts on this? Are there lots of programs that >> > want it that can't easily be patched to simply use argv[0] themselves? >> >> This is not something that is absolutely necessary. __progname quite >> often is used on *BSD and less on Linux (eg. Owl's msulogin, >> popa3d)... but __progname is always easy to fix. >> > > i think the fact that *bsd uses it > is not enough justification > > openbsd uses it because it's part of > their style guide for whatever reason > > "The __progname string may be used instead > of hard-coding the program name." > http://www.openbsd.org/cgi-bin/man.cgi?query=style&sektion=9 > > but we don't support many things from > there (like sys/queue.h) > > > i don't think many linux tools uses it > as it's not part of the lsb and glibc > has its own silly > program_invocation_name and > program_invocation_short_name > (which are aliases to __progname and > __progname_full) > > the main justification i see is that > we already support bsd err and warn > apis which are required to print > the __progname as well > (currently they don't and actually > a simple warn("hi"); segfaults here > with musl but i havent investigated > it) I understand that and that's why my first sentence was: This is not something that is absolutely necessary. We often say that we don't want to reproduce 'ugly stuff from glibc etc.' (eg. __progname). This does not change the fact that a lot of code will require patches to fix __progname problem. If Rich has taken the effort to rewrite/fix libc, we can fix __progname... if needed (it's really small patch.. discussed on the list). Daniel ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 10:53 ` Daniel Cegiełka @ 2012-08-15 13:10 ` John Spencer 2012-08-15 13:23 ` Daniel Cegiełka 0 siblings, 1 reply; 38+ messages in thread From: John Spencer @ 2012-08-15 13:10 UTC (permalink / raw) To: musl On 08/15/2012 12:53 PM, Daniel Cegiełka wrote: > 2012/8/15 Szabolcs Nagy<nsz@port70.net>: >> * Daniel Cegie?ka<daniel.cegielka@gmail.com> [2012-08-15 10:55:06 +0200]: >>>>> - Support for __progname (Daniel) >> > We often say that we don't want to reproduce 'ugly stuff from glibc > etc.' (eg. __progname). This does not change the fact that a lot of > code will require patches to fix __progname problem. i don't think that "a lot" of programs need it. actually i never encountered it in the 200 packages i ported to sabotage, and i even doubt that it is in pkgsrc as i never heard anything about it in the irc channel. since this feature would add bloat to every program, i am strongly opposed to adding it. instead you can just do a quick sed over BSD coreutils that use it. > If Rich has taken > the effort to rewrite/fix libc, we can fix __progname... if needed > (it's really small patch.. discussed on the list). > > Daniel > ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 13:10 ` John Spencer @ 2012-08-15 13:23 ` Daniel Cegiełka 0 siblings, 0 replies; 38+ messages in thread From: Daniel Cegiełka @ 2012-08-15 13:23 UTC (permalink / raw) To: musl 2012/8/15 John Spencer <maillist-musl@barfooze.de>: > On 08/15/2012 12:53 PM, Daniel Cegiełka wrote: > > i don't think that "a lot" of programs need it. actually i never encountered > it in the 200 packages i ported to sabotage, > and i even doubt that it is in pkgsrc as i never heard anything about it in > the irc channel. I gave a list of packages from src/bin from OpenBSD repo (grep -r progname src/bin). > since this feature would add bloat to every program, i am strongly opposed > to adding it. > instead you can just do a quick sed over BSD coreutils that use it. And this may be the best solution... I also dislike this __progname. Daniel ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 10:20 ` Szabolcs Nagy 2012-08-15 10:53 ` Daniel Cegiełka @ 2012-08-15 13:32 ` Szabolcs Nagy 2012-08-15 14:36 ` Rich Felker 1 sibling, 1 reply; 38+ messages in thread From: Szabolcs Nagy @ 2012-08-15 13:32 UTC (permalink / raw) To: musl * Szabolcs Nagy <nsz@port70.net> [2012-08-15 12:20:29 +0200]: > the main justification i see is that > we already support bsd err and warn > apis which are required to print > the __progname as well > (currently they don't and actually > a simple warn("hi"); segfaults here > with musl but i havent investigated > it) it seems warn(0) and err(1,0) segfault (they should handle fmt==0 before passing it to vfprintf) and they do not print the ': ' nor the __progname (perror works correctly) test program: #include <stdio.h> #include <err.h> int main() { warn("warntest"); warn(0); perror("perrortest"); perror(""); perror(0); } $ gcc err.c && ./a.out a.out: warntest: Success a.out: Success perrortest: Success Success Success $ musl-gcc err.c && ./a.out warntestNo error information Segmentation fault (core dumped) ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 13:32 ` Szabolcs Nagy @ 2012-08-15 14:36 ` Rich Felker 2012-08-17 9:49 ` Szabolcs Nagy 0 siblings, 1 reply; 38+ messages in thread From: Rich Felker @ 2012-08-15 14:36 UTC (permalink / raw) To: musl On Wed, Aug 15, 2012 at 03:32:57PM +0200, Szabolcs Nagy wrote: > * Szabolcs Nagy <nsz@port70.net> [2012-08-15 12:20:29 +0200]: > > the main justification i see is that > > we already support bsd err and warn > > apis which are required to print > > the __progname as well > > (currently they don't and actually > > a simple warn("hi"); segfaults here > > with musl but i havent investigated > > it) > > it seems warn(0) and err(1,0) segfault > (they should handle fmt==0 before passing > it to vfprintf) > and they do not print the ': ' nor the > __progname Thanks for the report. This should be easy to fix. By the way, is it worth making these functions take a lock on the file for the whole operation (to make it atomic)? I'm leaning towards no, since they seem to only be used in legacy junk that's all single-threaded anyway. Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 14:36 ` Rich Felker @ 2012-08-17 9:49 ` Szabolcs Nagy 2012-08-17 12:10 ` Rich Felker 0 siblings, 1 reply; 38+ messages in thread From: Szabolcs Nagy @ 2012-08-17 9:49 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 723 bytes --] * Rich Felker <dalias@aerifal.cx> [2012-08-15 10:36:40 -0400]: > On Wed, Aug 15, 2012 at 03:32:57PM +0200, Szabolcs Nagy wrote: > > it seems warn(0) and err(1,0) segfault > > (they should handle fmt==0 before passing > > it to vfprintf) > > and they do not print the ': ' nor the > > __progname > > Thanks for the report. This should be easy to fix. By the way, is it > worth making these functions take a lock on the file for the whole > operation (to make it atomic)? I'm leaning towards no, since they seem > to only be used in legacy junk that's all single-threaded anyway. i dont mind if the file is not locked although it would not itroduce too much overhead.. the ": " is not fixed, i think that should be added [-- Attachment #2: err.diff --] [-- Type: text/x-diff, Size: 443 bytes --] diff --git a/src/linux/err.c b/src/linux/err.c index c291136..bc8a928 100644 --- a/src/linux/err.c +++ b/src/linux/err.c @@ -3,9 +3,15 @@ #include <stdarg.h> #include <stdlib.h> +/* basename(argv[0]) is not printed and stderr is not locked */ + void vwarn(const char *fmt, va_list ap) { - if (fmt) vfprintf(stderr, fmt, ap); + if (fmt) { + vfprintf(stderr, fmt, ap); + fputc(':', stderr); + fputc(' ', stderr); + } perror(""); } ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-17 9:49 ` Szabolcs Nagy @ 2012-08-17 12:10 ` Rich Felker 2012-08-22 17:45 ` Daniel Cegiełka 0 siblings, 1 reply; 38+ messages in thread From: Rich Felker @ 2012-08-17 12:10 UTC (permalink / raw) To: musl On Fri, Aug 17, 2012 at 11:49:01AM +0200, Szabolcs Nagy wrote: > * Rich Felker <dalias@aerifal.cx> [2012-08-15 10:36:40 -0400]: > > On Wed, Aug 15, 2012 at 03:32:57PM +0200, Szabolcs Nagy wrote: > > > it seems warn(0) and err(1,0) segfault > > > (they should handle fmt==0 before passing > > > it to vfprintf) > > > and they do not print the ': ' nor the > > > __progname > > > > Thanks for the report. This should be easy to fix. By the way, is it > > worth making these functions take a lock on the file for the whole > > operation (to make it atomic)? I'm leaning towards no, since they seem > > to only be used in legacy junk that's all single-threaded anyway. > > i dont mind if the file is not locked although it would not > itroduce too much overhead.. > > the ": " is not fixed, i think that should be added Oh, my intent was for perror to do the printing of it. I forgot perror special cases not only NULL but also ""... Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-17 12:10 ` Rich Felker @ 2012-08-22 17:45 ` Daniel Cegiełka 2012-08-22 18:57 ` Rich Felker 0 siblings, 1 reply; 38+ messages in thread From: Daniel Cegiełka @ 2012-08-22 17:45 UTC (permalink / raw) To: musl Hi, Very helpful would be support for fts.h in musl: http://www.kernel.org/doc/man-pages/online/pages/man3/fts.3.html Daniel ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-22 17:45 ` Daniel Cegiełka @ 2012-08-22 18:57 ` Rich Felker 2012-08-22 19:15 ` Daniel Cegiełka 2012-08-22 20:24 ` Richard Pennington 0 siblings, 2 replies; 38+ messages in thread From: Rich Felker @ 2012-08-22 18:57 UTC (permalink / raw) To: musl On Wed, Aug 22, 2012 at 07:45:01PM +0200, Daniel Cegiełka wrote: > Hi, > Very helpful would be support for fts.h in musl: > > http://www.kernel.org/doc/man-pages/online/pages/man3/fts.3.html Can these functions be implemented purely as library functionality, without any hooks into libc internals? I was thinking it might be nice to isolate all such code into a single subtree in musl, so that folks wanting to use it outside musl (either directly in application source trees, or in libraries) could easily find and use it. Of course there's also the option of putting such code in a separate library outside of libc (i.e. not including it in libc) which some people might prefer, but if it's small and historically in libcs and doesn't create maintenance burden, I'm not opposed to having it in libc.. Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-22 18:57 ` Rich Felker @ 2012-08-22 19:15 ` Daniel Cegiełka 2012-08-22 20:24 ` Richard Pennington 1 sibling, 0 replies; 38+ messages in thread From: Daniel Cegiełka @ 2012-08-22 19:15 UTC (permalink / raw) To: musl 2012/8/22 Rich Felker <dalias@aerifal.cx>: > On Wed, Aug 22, 2012 at 07:45:01PM +0200, Daniel Cegiełka wrote: >> Hi, >> Very helpful would be support for fts.h in musl: >> >> http://www.kernel.org/doc/man-pages/online/pages/man3/fts.3.html > > Can these functions be implemented purely as library functionality, > without any hooks into libc internals? I was thinking it might be nice > to isolate all such code into a single subtree in musl, so that folks > wanting to use it outside musl (either directly in application source > trees, or in libraries) could easily find and use it. This is a very good idea! Daniel > Of course there's also the option of putting such code in a separate > library outside of libc (i.e. not including it in libc) which some > people might prefer, but if it's small and historically in libcs and > doesn't create maintenance burden, I'm not opposed to having it in > libc.. > > Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-22 18:57 ` Rich Felker 2012-08-22 19:15 ` Daniel Cegiełka @ 2012-08-22 20:24 ` Richard Pennington 2012-08-22 22:44 ` Rich Felker 1 sibling, 1 reply; 38+ messages in thread From: Richard Pennington @ 2012-08-22 20:24 UTC (permalink / raw) To: musl; +Cc: Rich Felker On Wednesday, August 22, 2012 02:57:09 PM Rich Felker wrote: > On Wed, Aug 22, 2012 at 07:45:01PM +0200, Daniel Cegiełka wrote: > > Hi, > > Very helpful would be support for fts.h in musl: > > > > http://www.kernel.org/doc/man-pages/online/pages/man3/fts.3.html > > Can these functions be implemented purely as library functionality, > without any hooks into libc internals? I was thinking it might be nice > to isolate all such code into a single subtree in musl, so that folks > wanting to use it outside musl (either directly in application source > trees, or in libraries) could easily find and use it. > > Of course there's also the option of putting such code in a separate > library outside of libc (i.e. not including it in libc) which some > people might prefer, but if it's small and historically in libcs and > doesn't create maintenance burden, I'm not opposed to having it in > libc.. > > Rich I used the NetBSD version and it built with musl just fine with no internal stuff, as I recall. http://ellcc.org/viewvc/svn/ellcc/trunk/libecc/src/musl/src/bsd/ -Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-22 20:24 ` Richard Pennington @ 2012-08-22 22:44 ` Rich Felker 0 siblings, 0 replies; 38+ messages in thread From: Rich Felker @ 2012-08-22 22:44 UTC (permalink / raw) To: musl On Wed, Aug 22, 2012 at 03:24:13PM -0500, Richard Pennington wrote: > On Wednesday, August 22, 2012 02:57:09 PM Rich Felker wrote: > > On Wed, Aug 22, 2012 at 07:45:01PM +0200, Daniel Cegiełka wrote: > > > Hi, > > > Very helpful would be support for fts.h in musl: > > > > > > http://www.kernel.org/doc/man-pages/online/pages/man3/fts.3.html > > > > Can these functions be implemented purely as library functionality, > > without any hooks into libc internals? I was thinking it might be nice > > to isolate all such code into a single subtree in musl, so that folks > > wanting to use it outside musl (either directly in application source > > trees, or in libraries) could easily find and use it. > > > > Of course there's also the option of putting such code in a separate > > library outside of libc (i.e. not including it in libc) which some > > people might prefer, but if it's small and historically in libcs and > > doesn't create maintenance burden, I'm not opposed to having it in > > libc.. > > > > Rich > > I used the NetBSD version and it built with musl just fine with no internal > stuff, as I recall. > > http://ellcc.org/viewvc/svn/ellcc/trunk/libecc/src/musl/src/bsd/ How big is it? I looked at it casually earlier today and didn't see anything horribly offensive in the code; it looks to even get allocation failure error cases right. Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 8:55 ` Daniel Cegiełka 2012-08-15 10:20 ` Szabolcs Nagy @ 2012-08-15 12:36 ` Rich Felker 1 sibling, 0 replies; 38+ messages in thread From: Rich Felker @ 2012-08-15 12:36 UTC (permalink / raw) To: musl On Wed, Aug 15, 2012 at 10:55:06AM +0200, Daniel Cegiełka wrote: > > > >> - Support for __progname (Daniel) > > > > Daniel, any more thoughts on this? Are there lots of programs that > > want it that can't easily be patched to simply use argv[0] themselves? > > This is not something that is absolutely necessary. __progname quite > often is used on *BSD and less on Linux (eg. Owl's msulogin, > popa3d)... but __progname is always easy to fix. My leaning is to omit it at least for now then. > Here we have the OpenBSD repo and content /bin directories: > > http://www.openbsd.org/cgi-bin/cvsweb/src/bin/ > > And here's a list of programs (from /bin/) that require __progname (70% of all): This might be a little bit inflated if it includes programs which detected the presence of __progname at build time and only used it because of that. > For __progname we probably need to modify (asm) files in the musl/crt/. No, the only thing that belongs there is the minimum code to get argc, argv, the addresses of main, _init, _fini, and jump to __libc_start_main. The latter is responsible for things like __progname. If the code were put in crt1.o, all programs would have a reference to __progname encoded into them, which is not something desirable; it would also increase the amount of per-arch code that must be maintained. Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 4:08 ` Rich Felker 2012-08-15 8:55 ` Daniel Cegiełka @ 2012-08-15 12:57 ` Luca Barbato 2012-08-15 14:34 ` Rich Felker 2012-08-15 13:27 ` Richard Pennington 2 siblings, 1 reply; 38+ messages in thread From: Luca Barbato @ 2012-08-15 12:57 UTC (permalink / raw) To: musl On 8/15/12 6:08 AM, Rich Felker wrote: > On hold pending details on what the real problem is; unlikely to make > it in for next release. > > Luca, if you still want this, please provide details on what issues > you're facing that could be solved. I don't want to target old > versions of standards unless there's a concrete practical goal. I > mentioned one possible approach (using old versions only as a way to > reenable stuff that was removed from the standard, not a way to get > the entire outdated-standard behavior that would also require removing > new symbols) but I haven't heard back on whether that would meet your > needs. Your approach would be fine even making something wrong like exposing all the symbols would be ok for my specific purposes. Alternatively forcing one of the two SOURCES would work as well. lu ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 12:57 ` Luca Barbato @ 2012-08-15 14:34 ` Rich Felker 2012-08-15 18:28 ` Luca Barbato 0 siblings, 1 reply; 38+ messages in thread From: Rich Felker @ 2012-08-15 14:34 UTC (permalink / raw) To: musl On Wed, Aug 15, 2012 at 02:57:26PM +0200, Luca Barbato wrote: > On 8/15/12 6:08 AM, Rich Felker wrote: > >On hold pending details on what the real problem is; unlikely to make > >it in for next release. > > > >Luca, if you still want this, please provide details on what issues > >you're facing that could be solved. I don't want to target old > >versions of standards unless there's a concrete practical goal. I > >mentioned one possible approach (using old versions only as a way to > >reenable stuff that was removed from the standard, not a way to get > >the entire outdated-standard behavior that would also require removing > >new symbols) but I haven't heard back on whether that would meet your > >needs. > > Your approach would be fine even making something wrong like > exposing all the symbols would be ok for my specific purposes. Could you explain a little bit what the problem is, like "I'm trying to build X and function Y is undeclared" or similar? Before trying to address the issue, I'd like to know what problem it's solving. :-) Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 14:34 ` Rich Felker @ 2012-08-15 18:28 ` Luca Barbato 2012-08-15 18:35 ` Rich Felker 2012-08-15 21:25 ` Rich Felker 0 siblings, 2 replies; 38+ messages in thread From: Luca Barbato @ 2012-08-15 18:28 UTC (permalink / raw) To: musl On 8/15/12 4:34 PM, Rich Felker wrote: > Could you explain a little bit what the problem is, like "I'm trying > to build X and function Y is undeclared" or similar? Before trying to > address the issue, I'd like to know what problem it's solving. :-) One is libav, I need to complete my gentoo chroot to name exactly other programs relying on XOPEN but I recall that there are others. lu ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 18:28 ` Luca Barbato @ 2012-08-15 18:35 ` Rich Felker 2012-08-15 21:25 ` Rich Felker 1 sibling, 0 replies; 38+ messages in thread From: Rich Felker @ 2012-08-15 18:35 UTC (permalink / raw) To: musl On Wed, Aug 15, 2012 at 08:28:18PM +0200, Luca Barbato wrote: > On 8/15/12 4:34 PM, Rich Felker wrote: > >Could you explain a little bit what the problem is, like "I'm trying > >to build X and function Y is undeclared" or similar? Before trying to > >address the issue, I'd like to know what problem it's solving. :-) > > One is libav, I need to complete my gentoo chroot to name exactly > other programs relying on XOPEN but I recall that there are others. I'm sorry, I think we're not communicating well. :-( What happens when you try to build libav against musl as-is, with -D_XOPEN_SOURCE=700 or 600 or whatever you're using? Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 18:28 ` Luca Barbato 2012-08-15 18:35 ` Rich Felker @ 2012-08-15 21:25 ` Rich Felker 2012-08-16 17:11 ` Luca Barbato 1 sibling, 1 reply; 38+ messages in thread From: Rich Felker @ 2012-08-15 21:25 UTC (permalink / raw) To: musl On Wed, Aug 15, 2012 at 08:28:18PM +0200, Luca Barbato wrote: > On 8/15/12 4:34 PM, Rich Felker wrote: > >Could you explain a little bit what the problem is, like "I'm trying > >to build X and function Y is undeclared" or similar? Before trying to > >address the issue, I'd like to know what problem it's solving. :-) > > One is libav, I need to complete my gentoo chroot to name exactly > other programs relying on XOPEN but I recall that there are others. Does the latest git solve your problem? Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 21:25 ` Rich Felker @ 2012-08-16 17:11 ` Luca Barbato 0 siblings, 0 replies; 38+ messages in thread From: Luca Barbato @ 2012-08-16 17:11 UTC (permalink / raw) To: musl On 08/15/2012 11:25 PM, Rich Felker wrote: > On Wed, Aug 15, 2012 at 08:28:18PM +0200, Luca Barbato wrote: >> On 8/15/12 4:34 PM, Rich Felker wrote: >>> Could you explain a little bit what the problem is, like "I'm trying >>> to build X and function Y is undeclared" or similar? Before trying to >>> address the issue, I'd like to know what problem it's solving. :-) >> >> One is libav, I need to complete my gentoo chroot to name exactly >> other programs relying on XOPEN but I recall that there are others. > > Does the latest git solve your problem? > Yes it does. Thank you. ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-15 4:08 ` Rich Felker 2012-08-15 8:55 ` Daniel Cegiełka 2012-08-15 12:57 ` Luca Barbato @ 2012-08-15 13:27 ` Richard Pennington 2 siblings, 0 replies; 38+ messages in thread From: Richard Pennington @ 2012-08-15 13:27 UTC (permalink / raw) To: musl; +Cc: Rich Felker On Wednesday, August 15, 2012 12:08:37 AM Rich Felker wrote: > Anything else that's been overlooked for a while? One thing that comes > to my mind is updating credits. I actually realized just after the > last release I forgot to add rdp and Solar to the COPYRIGHT file for > their contributions, and credit for rdp is actually missing from most > (all?) of the source files too - I felt really bad about this > omission. Please accept my apologies, and let me know if there are any > other credits I've forgotten to add. No apology required. I'm sorry that I haven't been more engaged. Unfortunately, my day job is in 24/7 mode (working on a new product release) and I haven't been able to focus on the important stuff as much as I'd like. Plus, I got way more from musl than I contributed. ;-) -Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: Todo for release? 2012-08-13 18:53 Todo for release? Rich Felker 2012-08-13 21:31 ` Szabolcs Nagy 2012-08-15 4:08 ` Rich Felker @ 2012-08-15 22:44 ` boris brezillon 2 siblings, 0 replies; 38+ messages in thread From: boris brezillon @ 2012-08-15 22:44 UTC (permalink / raw) To: musl 2012/8/13 Rich Felker <dalias@aerifal.cx>: > Starting a new thread since the old one got too long and OT.. :-) > > New stuff we have so far: > > - Blowfish crypt > - MIPS dynamic linker > - Major MIPS bug fixes > - ARM hard float support > - BSD fgetln function > - Major bug fix in wcsstr > - Optimized memcpy for i386 and x86_64 > - Public exposure for getdents > - Added significand function to math lib > > The requested stuff that's still pending (with person who requested or > is working on it): > > - Exception behavior in i386/x86_64 exponential asm (nsz). > - Finer-grained _XOPEN_SOURCE (Luca) > - Support for __progname (Daniel) > - MD5 and SHA crypt (nsz?) > > In addition, there are the GNU hash and dladdr patches (Boris) > pending, which I don't want to overlook, but I think unless I manage > to get a discussion of simplifying them finished really soon and find > some time to thoroughly test them, it's best to hold off on this until > just after the next release so we have plenty of time to test and tune > it before another release. I agree. We should wait for the next release. > > Anything else I missed? > > Rich ^ permalink raw reply [flat|nested] 38+ messages in thread
end of thread, other threads:[~2012-08-22 22:44 UTC | newest] Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-08-13 18:53 Todo for release? Rich Felker 2012-08-13 21:31 ` Szabolcs Nagy 2012-08-13 21:53 ` Rich Felker 2012-08-13 22:06 ` Solar Designer 2012-08-14 15:02 ` Szabolcs Nagy 2012-08-15 0:30 ` Szabolcs Nagy 2012-08-13 22:20 ` Solar Designer 2012-08-14 1:46 ` Rich Felker 2012-08-14 2:13 ` Solar Designer 2012-08-14 2:35 ` Rich Felker 2012-08-14 2:49 ` Solar Designer 2012-08-14 2:58 ` Rich Felker 2012-08-14 3:35 ` Solar Designer 2012-08-14 4:49 ` Rich Felker 2012-08-15 4:08 ` Rich Felker 2012-08-15 8:55 ` Daniel Cegiełka 2012-08-15 10:20 ` Szabolcs Nagy 2012-08-15 10:53 ` Daniel Cegiełka 2012-08-15 13:10 ` John Spencer 2012-08-15 13:23 ` Daniel Cegiełka 2012-08-15 13:32 ` Szabolcs Nagy 2012-08-15 14:36 ` Rich Felker 2012-08-17 9:49 ` Szabolcs Nagy 2012-08-17 12:10 ` Rich Felker 2012-08-22 17:45 ` Daniel Cegiełka 2012-08-22 18:57 ` Rich Felker 2012-08-22 19:15 ` Daniel Cegiełka 2012-08-22 20:24 ` Richard Pennington 2012-08-22 22:44 ` Rich Felker 2012-08-15 12:36 ` Rich Felker 2012-08-15 12:57 ` Luca Barbato 2012-08-15 14:34 ` Rich Felker 2012-08-15 18:28 ` Luca Barbato 2012-08-15 18:35 ` Rich Felker 2012-08-15 21:25 ` Rich Felker 2012-08-16 17:11 ` Luca Barbato 2012-08-15 13:27 ` Richard Pennington 2012-08-15 22:44 ` boris brezillon
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).