* Re: High-priority library replacements?
2013-04-25 4:15 Rich Felker
@ 2013-04-25 5:05 ` Daniel Cegiełka
2013-04-25 5:21 ` Rich Felker
2013-04-25 5:55 ` Kurt H Maier
` (5 subsequent siblings)
6 siblings, 1 reply; 76+ messages in thread
From: Daniel Cegiełka @ 2013-04-25 5:05 UTC (permalink / raw)
To: musl
2013/4/25 Rich Felker <dalias@aerifal.cx>:
For a list of core libraries I would add basic, but high-priority
tools: ssh, pam (passwd, login, su).
ssh - dropbear?
pam - openpam?
These key software we can also support (static linking etc.).
btw. SSL - instead libcrypto clone maybe it's better to use
crypto-algorithms from linux kernel?
http://cryptodev-linux.org/
Daniel
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: High-priority library replacements?
2013-04-25 5:05 ` Daniel Cegiełka
@ 2013-04-25 5:21 ` Rich Felker
0 siblings, 0 replies; 76+ messages in thread
From: Rich Felker @ 2013-04-25 5:21 UTC (permalink / raw)
To: musl
On Thu, Apr 25, 2013 at 07:05:12AM +0200, Daniel Cegiełka wrote:
> 2013/4/25 Rich Felker <dalias@aerifal.cx>:
>
> For a list of core libraries I would add basic, but high-priority
> tools: ssh, pam (passwd, login, su).
>
> ssh - dropbear?
I think dropbear fully covers the needs of most non-"enterprise" usage
and maybe that too. It could however use some hardening. I don't think
it's terribly insecure, but I'd like to see a robust privilege model
that would make it safe even in the event of bugs that would otherwise
result in a compromise.
> pam - openpam?
I would say pam is less critical. I've had my pamlite in limbo for a
long time but haven't gotten around to making it do anything... Still
a good one for the list though.
> These key software we can also support (static linking etc.).
>
> btw. SSL - instead libcrypto clone maybe it's better to use
> crypto-algorithms from linux kernel?
>
> http://cryptodev-linux.org/
I'm not sure what the advantage would be; the disadvantage is
certainly being Linux-specific and dependent on the host system
configuration (last I checked, crypto in the kernel is optional; maybe
this has changed..?) to work. There's also the issue that it's not
fail-proof; it requires allocating resources. IMO supporting hardware
crypto devices is not really relevant for most users of SSL. Yes, a
high volume web server might need to be tuned for performance, but it
doesn't matter for most network client applications like wget, chat
clients, mail clients, etc.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 4:15 Rich Felker
2013-04-25 5:05 ` Daniel Cegiełka
@ 2013-04-25 5:55 ` Kurt H Maier
2013-04-25 7:34 ` Jens Staal
2013-04-25 6:43 ` Gregor Pintar
` (4 subsequent siblings)
6 siblings, 1 reply; 76+ messages in thread
From: Kurt H Maier @ 2013-04-25 5:55 UTC (permalink / raw)
To: musl
On Thu, Apr 25, 2013 at 12:15:53AM -0400, Rich Felker wrote:
>
> 1. A light, C, UTF-8-only Unicode library.
The only one I know of that might not already be common knowledge is
Connor Lane Smith's libutf: https://github.com/cls/libutf
khm
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: High-priority library replacements?
2013-04-25 5:55 ` Kurt H Maier
@ 2013-04-25 7:34 ` Jens Staal
2013-04-25 12:18 ` Rich Felker
2013-04-25 13:54 ` Kurt H Maier
0 siblings, 2 replies; 76+ messages in thread
From: Jens Staal @ 2013-04-25 7:34 UTC (permalink / raw)
To: musl
On 2013-04-25 07:55, Kurt H Maier wrote:
> On Thu, Apr 25, 2013 at 12:15:53AM -0400, Rich Felker wrote:
>>
>> 1. A light, C, UTF-8-only Unicode library.
>
> The only one I know of that might not already be common knowledge is
> Connor Lane Smith's libutf: https://github.com/cls/libutf
>
> khm
>
Does it work as a drop-in replacement for unixy utf/wchar
implementations or is it the Plan9 variant (reference to Runes implies
the later)? I found myself needing at first libwchar from heirloom and
later the (old) GNU libutf8 under APE to provide some wchar-related
functions expected by various applications.
It felt really silly to "devolve" like that.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 7:34 ` Jens Staal
@ 2013-04-25 12:18 ` Rich Felker
2013-04-25 13:54 ` Kurt H Maier
1 sibling, 0 replies; 76+ messages in thread
From: Rich Felker @ 2013-04-25 12:18 UTC (permalink / raw)
To: musl
On Thu, Apr 25, 2013 at 09:34:35AM +0200, Jens Staal wrote:
> On 2013-04-25 07:55, Kurt H Maier wrote:
> >On Thu, Apr 25, 2013 at 12:15:53AM -0400, Rich Felker wrote:
> >>
> >>1. A light, C, UTF-8-only Unicode library.
> >
> >The only one I know of that might not already be common knowledge is
> >Connor Lane Smith's libutf: https://github.com/cls/libutf
>
> Does it work as a drop-in replacement for unixy utf/wchar
> implementations
This is exactly what libutf does. It's not a Unicode library in the
form I described; it's just a workaround for broken systems lacking
UTF-8 locales, for which it replaces all the important functions
(mb*towc and wc*tomb, etc.) with versions that convert UTF-8. I
believe it also replaces the is* and to* functions and maybe a few
others. But it doesn't help with any Unicode processing needs the
application might have outside the scope of C or POSIX, and it's not
meant to be used on systems that already have proper UTF-8 support at
the system library level.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 7:34 ` Jens Staal
2013-04-25 12:18 ` Rich Felker
@ 2013-04-25 13:54 ` Kurt H Maier
1 sibling, 0 replies; 76+ messages in thread
From: Kurt H Maier @ 2013-04-25 13:54 UTC (permalink / raw)
To: musl
On Thu, Apr 25, 2013 at 09:34:35AM +0200, Jens Staal wrote:
> On 2013-04-25 07:55, Kurt H Maier wrote:
>
> Does it work as a drop-in replacement for unixy utf/wchar
> implementations or is it the Plan9 variant (reference to Runes implies
> the later)? I found myself needing at first libwchar from heirloom and
> later the (old) GNU libutf8 under APE to provide some wchar-related
> functions expected by various applications.
>
> It felt really silly to "devolve" like that.
It's the Plan 9 variant -- wchar is a thing that needs to go away...
maybe one day we can get there. :(
khm
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 4:15 Rich Felker
2013-04-25 5:05 ` Daniel Cegiełka
2013-04-25 5:55 ` Kurt H Maier
@ 2013-04-25 6:43 ` Gregor Pintar
2013-04-26 0:55 ` idunham
` (2 more replies)
2013-04-25 7:21 ` Hal Clark
` (3 subsequent siblings)
6 siblings, 3 replies; 76+ messages in thread
From: Gregor Pintar @ 2013-04-25 6:43 UTC (permalink / raw)
To: musl
Hello.
2013/4/25, Rich Felker <dalias@aerifal.cx>:
> 2. SSL. The modern internet basically requires using SSL everywhere.
> We should be aiming/expecting to transition to a world where even
> non-login-based sites are 100% https; the threats of malicious http
> injection attacks from rogue or advertising-based access points has
> gotten too great. Unfortunately, all of the existing SSL
> implementations are bloated, buggy, and fail even the most basic
> robustness requirements. A good solution would be based on tomcrypt
> and would expose a minimal, simple API suited for event-loop-based or
> threaded use. It may also be useful to have an optional wrapper layer
> to expose an API that mimics openssl or gnutls. It should also be able
> to keep up with the changing demands of how to determine which
> certificate authorities are to be trusted.
I am working on cryptographic library. It's far from being finished.
I would be very glad, if someone could look at it.
Currently I have problems with API design so help would be welcome.
It isn't in git yet.
You can get it here:
https://dl.dropboxusercontent.com/u/83450675/kripto/kripto.tar.gz
I have plans for SSL library on top of it, but it could take years.
I think best way is not to trust any certificate authority.
Maybe some certificate p2p protocol could be done?
> All of these libraries should:
>
> - Avoid namespace pollution. Only external symbols should be the
> public API and internal-use stuff prefixed with an ugly prefix
> that's extremely unlikely to clash with anything.
All external symbols have "kripto_" prefix.
> - Avoid unnecessary allocation. Use caller-provided objects where
> possible or provide both options.
I am trying to do least malloc()s possible.
> - Have absolutely zero global state.
There is no global state and there won't be any.
Gregor
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: High-priority library replacements?
2013-04-25 6:43 ` Gregor Pintar
@ 2013-04-26 0:55 ` idunham
2013-04-26 1:51 ` Rich Felker
2013-04-26 8:11 ` Gregor Pintar
2013-04-30 18:47 ` Nicolas Braud-Santoni
2013-05-26 20:09 ` Daniel Cegiełka
2 siblings, 2 replies; 76+ messages in thread
From: idunham @ 2013-04-26 0:55 UTC (permalink / raw)
To: musl
On Thu, Apr 25, 2013 at 08:43:25AM +0200, Gregor Pintar wrote:
> Hello.
>
> 2013/4/25, Rich Felker <dalias@aerifal.cx>:
> > 2. SSL. The modern internet basically requires using SSL everywhere.
> > We should be aiming/expecting to transition to a world where even
> > non-login-based sites are 100% https; the threats of malicious http
> > injection attacks from rogue or advertising-based access points has
> > gotten too great. Unfortunately, all of the existing SSL
> > implementations are bloated, buggy, and fail even the most basic
> > robustness requirements. A good solution would be based on tomcrypt
> > and would expose a minimal, simple API suited for event-loop-based or
> > threaded use. It may also be useful to have an optional wrapper layer
> > to expose an API that mimics openssl or gnutls. It should also be able
> > to keep up with the changing demands of how to determine which
> > certificate authorities are to be trusted.
> I am working on cryptographic library. It's far from being finished.
> I would be very glad, if someone could look at it.
> Currently I have problems with API design so help would be welcome.
> It isn't in git yet.
> You can get it here:
> https://dl.dropboxusercontent.com/u/83450675/kripto/kripto.tar.gz
> I have plans for SSL library on top of it, but it could take years.
I hate to be the one who says this, but...
Why another crypto library?
There are at least 6 I can think of off the top of my head
(openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
and I know that's not even half of them.
tomcrypt is already good (as Rich mentioned), so code quality isn't
a reason.
While writing your own "xyz" may be a good learning experience and fun
and so on, a crypto library faces some restrictions:
-You will need to fix bugs promptly until you hand over maintainership.
(Otherwise, you become responsible when there's a vulnerability that
stays unfixed.)
-You have to finish it.
Consider the case of gcrypt, which is a GNU project with multiple
contributors, but still doesn't implement enough to make LDAP over SSL
work in all cases (I don't know the exact issue, but it's a standing bug
in Debian.)
And you have to write a good API. When someone else has a library with
that, you may be better off using it.
What Rich asked about was an SSL lib based on an existing crypto lib,
namely tomcrypt. And that is likely to be a quicker path to results.
> I think best way is not to trust any certificate authority.
> Maybe some certificate p2p protocol could be done?
Not really an option for regular SSL, I'm afraid.
(just due to the standards, and the need for getting it going...)
But something along the lines of OpenBGP (I think that's the name; it's
a BSD-licensed PGP library) would be one place to start if you wanted
that.
It would be interesting if the library transparently handles both SSL
and something along those lines, so someone gets the certificate p2p
stuff for free by using the library.
> > All of these libraries should:
> >
> > - Avoid namespace pollution. Only external symbols should be the
> > public API and internal-use stuff prefixed with an ugly prefix
> > that's extremely unlikely to clash with anything.
> All external symbols have "kripto_" prefix.
>
> > - Avoid unnecessary allocation. Use caller-provided objects where
> > possible or provide both options.
> I am trying to do least malloc()s possible.
>
> > - Have absolutely zero global state.
> There is no global state and there won't be any.
Sounding good, and looking good from what I saw. (Although I tend to
dislike "magic number tables", but that's mainly personal taste...
Also, don't consider me an authority on C or programming. I know some,
but far too little.)
Thanks,
Isaac Dunham
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-26 0:55 ` idunham
@ 2013-04-26 1:51 ` Rich Felker
2013-04-26 8:11 ` Gregor Pintar
1 sibling, 0 replies; 76+ messages in thread
From: Rich Felker @ 2013-04-26 1:51 UTC (permalink / raw)
To: musl
On Thu, Apr 25, 2013 at 05:55:45PM -0700, idunham@lavabit.com wrote:
> > I am working on cryptographic library. It's far from being finished.
> > I would be very glad, if someone could look at it.
> > Currently I have problems with API design so help would be welcome.
> > It isn't in git yet.
> > You can get it here:
> > https://dl.dropboxusercontent.com/u/83450675/kripto/kripto.tar.gz
> > I have plans for SSL library on top of it, but it could take years.
>
> I hate to be the one who says this, but...
> Why another crypto library?
> There are at least 6 I can think of off the top of my head
> (openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
> and I know that's not even half of them.
> tomcrypt is already good (as Rich mentioned), so code quality isn't
> a reason.
> While writing your own "xyz" may be a good learning experience and fun
> and so on, a crypto library faces some restrictions:
> -You will need to fix bugs promptly until you hand over maintainership.
> (Otherwise, you become responsible when there's a vulnerability that
> stays unfixed.)
> -You have to finish it.
> Consider the case of gcrypt, which is a GNU project with multiple
> contributors, but still doesn't implement enough to make LDAP over SSL
> work in all cases (I don't know the exact issue, but it's a standing bug
> in Debian.)
>
> And you have to write a good API. When someone else has a library with
> that, you may be better off using it.
> What Rich asked about was an SSL lib based on an existing crypto lib,
> namely tomcrypt. And that is likely to be a quicker path to results.
Yes. Crypto is hard to get right and when there's an implementation
with high-quality public domain code, I think you'd need a really
strong justification for using a new crypto implementation in an SSL
library.
> > I think best way is not to trust any certificate authority.
> > Maybe some certificate p2p protocol could be done?
>
> Not really an option for regular SSL, I'm afraid.
> (just due to the standards, and the need for getting it going...)
> But something along the lines of OpenBGP (I think that's the name; it's
> a BSD-licensed PGP library) would be one place to start if you wanted
> that.
> It would be interesting if the library transparently handles both SSL
> and something along those lines, so someone gets the certificate p2p
> stuff for free by using the library.
Yes, I don't really know what the future directions will be, I just
think a library should be prepared to adapt to changing needs for how
we determine trust.
> > > - Have absolutely zero global state.
> > There is no global state and there won't be any.
>
> Sounding good, and looking good from what I saw. (Although I tend to
> dislike "magic number tables", but that's mainly personal taste...
> Also, don't consider me an authority on C or programming. I know some,
> but far too little.)
If you dislike magic number tables, you can always include a script in
the build process to generate them. Using tens or hundreds of kb per
process and tons of startup time however is not an acceptable solution
to "I dislike magic number tables". Like them or not, magic number
tables are the only non-bloated solution.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-26 0:55 ` idunham
2013-04-26 1:51 ` Rich Felker
@ 2013-04-26 8:11 ` Gregor Pintar
2013-04-26 15:47 ` Rich Felker
2013-04-29 10:16 ` Szabolcs Nagy
1 sibling, 2 replies; 76+ messages in thread
From: Gregor Pintar @ 2013-04-26 8:11 UTC (permalink / raw)
To: musl
2013/4/26, idunham@lavabit.com <idunham@lavabit.com>:
> I hate to be the one who says this, but...
> Why another crypto library?
> There are at least 6 I can think of off the top of my head
> (openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
> and I know that's not even half of them.
> tomcrypt is already good (as Rich mentioned), so code quality isn't
> a reason.
Most of them are realy bad (support only few ciphers, ugly API,
inflexible, license).
tomcrypt is good, but it has some global states (ltc_cipher_descriptor, ...)
and I want even more flexible library (variable rounds, no global state).
I would like API that allows replacing ciphers very easy.
For example: allways use kripto_stream_encrypt(),
if you use any block cipher in any mode or stream cipher directly.
Also my hash API supports variable length output (great for sponge
constructions).
> While writing your own "xyz" may be a good learning experience and fun
> and so on, a crypto library faces some restrictions:
> -You will need to fix bugs promptly until you hand over maintainership.
> (Otherwise, you become responsible when there's a vulnerability that
> stays unfixed.)
Not really a problem for me.
BTW, latest official stable tomcrypt release was released in 2007.
> -You have to finish it.
> Consider the case of gcrypt, which is a GNU project with multiple
> contributors, but still doesn't implement enough to make LDAP over SSL
> work in all cases (I don't know the exact issue, but it's a standing bug
> in Debian.)
I am trying to do it very extensible.
> And you have to write a good API. When someone else has a library with
> that, you may be better off using it.
That is the hardest. API better than tomcrypts is my goal.
I also plan stable ABI.
> What Rich asked about was an SSL lib based on an existing crypto lib,
> namely tomcrypt. And that is likely to be a quicker path to results.
tomcrypt is definitely quicker path.
>> I think best way is not to trust any certificate authority.
>> Maybe some certificate p2p protocol could be done?
>
> Not really an option for regular SSL, I'm afraid.
> (just due to the standards, and the need for getting it going...)
> But something along the lines of OpenBGP (I think that's the name; it's
> a BSD-licensed PGP library) would be one place to start if you wanted
> that.
> It would be interesting if the library transparently handles both SSL
> and something along those lines, so someone gets the certificate p2p
> stuff for free by using the library.
I know, it was more like an long-term idea.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-26 8:11 ` Gregor Pintar
@ 2013-04-26 15:47 ` Rich Felker
2013-04-26 17:24 ` Gregor Pintar
2013-04-28 21:43 ` Rob Landley
2013-04-29 10:16 ` Szabolcs Nagy
1 sibling, 2 replies; 76+ messages in thread
From: Rich Felker @ 2013-04-26 15:47 UTC (permalink / raw)
To: musl
On Fri, Apr 26, 2013 at 10:11:32AM +0200, Gregor Pintar wrote:
> 2013/4/26, idunham@lavabit.com <idunham@lavabit.com>:
> > I hate to be the one who says this, but...
> > Why another crypto library?
> > There are at least 6 I can think of off the top of my head
> > (openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
> > and I know that's not even half of them.
> > tomcrypt is already good (as Rich mentioned), so code quality isn't
> > a reason.
> Most of them are realy bad (support only few ciphers, ugly API,
> inflexible, license).
> tomcrypt is good, but it has some global states (ltc_cipher_descriptor, ...)
Yes, this is stupid, but I don't think there's any reason to use it.
Can't you just use the extern descriptor for the cipher you want
directly? In any case, I agree that when using tomcrypt the register
and unregister functions should be nop'd out and the cipher_descriptor
array replaced with a const version thereof. There may be more changes
needed to fix it too, but that sounds like most of it.
> > While writing your own "xyz" may be a good learning experience and fun
> > and so on, a crypto library faces some restrictions:
> > -You will need to fix bugs promptly until you hand over maintainership.
> > (Otherwise, you become responsible when there's a vulnerability that
> > stays unfixed.)
> Not really a problem for me.
> BTW, latest official stable tomcrypt release was released in 2007.
Yes, that's because it's already very mature.. :) BTW, a big plus of
that is that it would be safe to fork tomcrypt and fix any issues in
it that aren't going to get fixed upstream, like global state, since
maintaining a fork of a mature but clean codebase is almost no work.
> > What Rich asked about was an SSL lib based on an existing crypto lib,
> > namely tomcrypt. And that is likely to be a quicker path to results.
> tomcrypt is definitely quicker path.
Note that there's no reason the crypto implementation couldn't later
be replaced if there's an option better than tomcrypt.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-26 15:47 ` Rich Felker
@ 2013-04-26 17:24 ` Gregor Pintar
2013-04-28 21:43 ` Rob Landley
1 sibling, 0 replies; 76+ messages in thread
From: Gregor Pintar @ 2013-04-26 17:24 UTC (permalink / raw)
To: musl
2013/4/26, Rich Felker <dalias@aerifal.cx>:
> On Fri, Apr 26, 2013 at 10:11:32AM +0200, Gregor Pintar wrote:
>> 2013/4/26, idunham@lavabit.com <idunham@lavabit.com>:
>> > I hate to be the one who says this, but...
>> > Why another crypto library?
>> > There are at least 6 I can think of off the top of my head
>> > (openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
>> > and I know that's not even half of them.
>> > tomcrypt is already good (as Rich mentioned), so code quality isn't
>> > a reason.
>> Most of them are realy bad (support only few ciphers, ugly API,
>> inflexible, license).
>> tomcrypt is good, but it has some global states (ltc_cipher_descriptor,
>> ...)
>
> Yes, this is stupid, but I don't think there's any reason to use it.
> Can't you just use the extern descriptor for the cipher you want
> directly?
You can't, because modes read from global descriptor.
See: https://github.com/libtom/libtomcrypt/blob/master/src/modes/ctr/ctr_start.c
> In any case, I agree that when using tomcrypt the register
> and unregister functions should be nop'd out and the cipher_descriptor
> array replaced with a const version thereof. There may be more changes
> needed to fix it too, but that sounds like most of it.
But that would break API anyway.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-26 15:47 ` Rich Felker
2013-04-26 17:24 ` Gregor Pintar
@ 2013-04-28 21:43 ` Rob Landley
1 sibling, 0 replies; 76+ messages in thread
From: Rob Landley @ 2013-04-28 21:43 UTC (permalink / raw)
To: musl; +Cc: musl
On 04/26/2013 10:47:29 AM, Rich Felker wrote:
> > > While writing your own "xyz" may be a good learning experience
> and fun
> > > and so on, a crypto library faces some restrictions:
> > > -You will need to fix bugs promptly until you hand over
> maintainership.
> > > (Otherwise, you become responsible when there's a vulnerability
> that
> > > stays unfixed.)
> > Not really a problem for me.
> > BTW, latest official stable tomcrypt release was released in 2007.
>
> Yes, that's because it's already very mature.. :) BTW, a big plus of
> that is that it would be safe to fork tomcrypt and fix any issues in
> it that aren't going to get fixed upstream, like global state, since
> maintaining a fork of a mature but clean codebase is almost no work.
I note that dropbear has been maintaining a de-factor fork of
libtomcrypt all that time. Might want to coordinate with him if you're
going to bother.
Rob
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-26 8:11 ` Gregor Pintar
2013-04-26 15:47 ` Rich Felker
@ 2013-04-29 10:16 ` Szabolcs Nagy
2013-04-29 12:09 ` Rich Felker
2013-04-29 17:35 ` Gregor Pintar
1 sibling, 2 replies; 76+ messages in thread
From: Szabolcs Nagy @ 2013-04-29 10:16 UTC (permalink / raw)
To: musl
* Gregor Pintar <grpintar@gmail.com> [2013-04-26 10:11:32 +0200]:
> tomcrypt is good, but it has some global states (ltc_cipher_descriptor, ...)
> and I want even more flexible library (variable rounds, no global state).
> I would like API that allows replacing ciphers very easy.
> For example: allways use kripto_stream_encrypt(),
> if you use any block cipher in any mode or stream cipher directly.
> Also my hash API supports variable length output (great for sponge
> constructions).
>
i'm not sure about the flexibility part
it is good when you want to have the same api
for a wide range of algorithms and you want
to be able to fiddle with their internal settings
but in my opinion this adds many failure modes
which is bad
i never liked that in tomcrypt, openssl etc trivial
functions have error code return values which
should never fail
(eg hash_update(ctx, buf, len) should never fail
even if there is a counter in ctx that can overflow
every 2^64th bit of input, documenting the behaviour
for longer inputs is better, it would be even better
if the apropriate standards were more careful about
failures)
the way it is done in most crypt apis make most of the
code that use them broken:
they don't check the return value so if the function
may fail the code is broken, or they check but never
test the failure path so the code is broken for a
different reason
my preference would be to have a well documented, clean
highlevel api with minimal amount of failure modes
(no error code return value), and a flexible api with
ugly names for those who like to fiddle with internals
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-29 10:16 ` Szabolcs Nagy
@ 2013-04-29 12:09 ` Rich Felker
2013-04-29 17:35 ` Gregor Pintar
1 sibling, 0 replies; 76+ messages in thread
From: Rich Felker @ 2013-04-29 12:09 UTC (permalink / raw)
To: musl
On Mon, Apr 29, 2013 at 12:16:20PM +0200, Szabolcs Nagy wrote:
> * Gregor Pintar <grpintar@gmail.com> [2013-04-26 10:11:32 +0200]:
> > tomcrypt is good, but it has some global states (ltc_cipher_descriptor, ...)
> > and I want even more flexible library (variable rounds, no global state).
> > I would like API that allows replacing ciphers very easy.
> > For example: allways use kripto_stream_encrypt(),
> > if you use any block cipher in any mode or stream cipher directly.
> > Also my hash API supports variable length output (great for sponge
> > constructions).
> >
>
> i'm not sure about the flexibility part
>
> it is good when you want to have the same api
> for a wide range of algorithms and you want
> to be able to fiddle with their internal settings
>
> but in my opinion this adds many failure modes
> which is bad
>
> i never liked that in tomcrypt, openssl etc trivial
> functions have error code return values which
> should never fail
>
> (eg hash_update(ctx, buf, len) should never fail
> even if there is a counter in ctx that can overflow
> every 2^64th bit of input, documenting the behaviour
> for longer inputs is better, it would be even better
> if the apropriate standards were more careful about
> failures)
This is a really good point. Hashing and encryption should
fundamentally be able to deal with arbitrary inputs and should not
require resource allocation to do their job. As such, they should have
no failure cases. Decryption of course has the failure case where the
input is malformed, but should not have any other failure cases.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-29 10:16 ` Szabolcs Nagy
2013-04-29 12:09 ` Rich Felker
@ 2013-04-29 17:35 ` Gregor Pintar
2013-04-29 21:55 ` Szabolcs Nagy
1 sibling, 1 reply; 76+ messages in thread
From: Gregor Pintar @ 2013-04-29 17:35 UTC (permalink / raw)
To: musl
2013/4/29, Szabolcs Nagy <nsz@port70.net>:
> * Gregor Pintar <grpintar@gmail.com> [2013-04-26 10:11:32 +0200]:
>> tomcrypt is good, but it has some global states (ltc_cipher_descriptor,
>> ...)
>> and I want even more flexible library (variable rounds, no global state).
>> I would like API that allows replacing ciphers very easy.
>> For example: allways use kripto_stream_encrypt(),
>> if you use any block cipher in any mode or stream cipher directly.
>> Also my hash API supports variable length output (great for sponge
>> constructions).
>>
>
> i'm not sure about the flexibility part
>
> it is good when you want to have the same api
> for a wide range of algorithms and you want
> to be able to fiddle with their internal settings
>
> but in my opinion this adds many failure modes
> which is bad
>
> i never liked that in tomcrypt, openssl etc trivial
> functions have error code return values which
> should never fail
>
> (eg hash_update(ctx, buf, len) should never fail
> even if there is a counter in ctx that can overflow
> every 2^64th bit of input, documenting the behaviour
> for longer inputs is better, it would be even better
> if the apropriate standards were more careful about
> failures)
Devs that ignore return values would probably ignore documentation too.
> the way it is done in most crypt apis make most of the
> code that use them broken:
> they don't check the return value so if the function
> may fail the code is broken, or they check but never
> test the failure path so the code is broken for a
> different reason
Encrypting more than 2^(block size) / 2 of blocks is broken too.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-29 17:35 ` Gregor Pintar
@ 2013-04-29 21:55 ` Szabolcs Nagy
2013-04-30 2:10 ` Rich Felker
0 siblings, 1 reply; 76+ messages in thread
From: Szabolcs Nagy @ 2013-04-29 21:55 UTC (permalink / raw)
To: musl
* Gregor Pintar <grpintar@gmail.com> [2013-04-29 19:35:03 +0200]:
> > (eg hash_update(ctx, buf, len) should never fail
> > even if there is a counter in ctx that can overflow
> > every 2^64th bit of input, documenting the behaviour
> > for longer inputs is better, it would be even better
> > if the apropriate standards were more careful about
> > failures)
> Devs that ignore return values would probably ignore documentation too.
i think the problem is not ignorance, but the combinatorial
explosion of code paths which increases bug probability
in my view the purpose of error codes is to indicate runtime
errors (eg allocation or io failures) and not to prevent bugs
in the program logic (there are other tools for that, most
notably the type checker, adding extra error handling paths
for bug prevention usually makes things worse)
returning an error code when some inconsistency is detected
seems simple, but error semantics are usually not trivial:
is it enough to retry, or retry with different parameters
or treat the failure as fatal, should all the related states
be cleaned if they might contain sensitive data or would the
library do this for us, how to report the error on the ui etc
without documentation the user should prepare for the worst and
that can easily turn into a harder problem than implementing
sha1 from scratch
> Encrypting more than 2^(block size) / 2 of blocks is broken too.
that's why cryptographers should choose the block size
wisely so the user does not have to worry about it
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-29 21:55 ` Szabolcs Nagy
@ 2013-04-30 2:10 ` Rich Felker
2013-04-30 6:32 ` Gregor Pintar
0 siblings, 1 reply; 76+ messages in thread
From: Rich Felker @ 2013-04-30 2:10 UTC (permalink / raw)
To: musl
On Mon, Apr 29, 2013 at 11:55:00PM +0200, Szabolcs Nagy wrote:
> * Gregor Pintar <grpintar@gmail.com> [2013-04-29 19:35:03 +0200]:
> > > (eg hash_update(ctx, buf, len) should never fail
> > > even if there is a counter in ctx that can overflow
> > > every 2^64th bit of input, documenting the behaviour
> > > for longer inputs is better, it would be even better
> > > if the apropriate standards were more careful about
> > > failures)
> > Devs that ignore return values would probably ignore documentation too.
>
> i think the problem is not ignorance, but the combinatorial
> explosion of code paths which increases bug probability
>
> in my view the purpose of error codes is to indicate runtime
> errors (eg allocation or io failures) and not to prevent bugs
> in the program logic (there are other tools for that, most
> notably the type checker, adding extra error handling paths
> for bug prevention usually makes things worse)
Agreed. You've expressed the issue very concisely and effectively. The
way I tend to think of it is that error codes should express a
condition that can occur in a correct program. If the condition can't
occur in a correct program, no effort should be spent at runtime
detecting it. If a program is going to go about detecting error codes
that couldn't occur without a bug in the program, though, it should do
so via assert() rather than complex error-propagation logic.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-30 2:10 ` Rich Felker
@ 2013-04-30 6:32 ` Gregor Pintar
2013-04-30 8:35 ` Szabolcs Nagy
0 siblings, 1 reply; 76+ messages in thread
From: Gregor Pintar @ 2013-04-30 6:32 UTC (permalink / raw)
To: musl
2013/4/30, Rich Felker <dalias@aerifal.cx>:
> On Mon, Apr 29, 2013 at 11:55:00PM +0200, Szabolcs Nagy wrote:
>> * Gregor Pintar <grpintar@gmail.com> [2013-04-29 19:35:03 +0200]:
>> > > (eg hash_update(ctx, buf, len) should never fail
>> > > even if there is a counter in ctx that can overflow
>> > > every 2^64th bit of input, documenting the behaviour
>> > > for longer inputs is better, it would be even better
>> > > if the apropriate standards were more careful about
>> > > failures)
>> > Devs that ignore return values would probably ignore documentation too.
>>
>> i think the problem is not ignorance, but the combinatorial
>> explosion of code paths which increases bug probability
>>
>> in my view the purpose of error codes is to indicate runtime
>> errors (eg allocation or io failures) and not to prevent bugs
>> in the program logic (there are other tools for that, most
>> notably the type checker, adding extra error handling paths
>> for bug prevention usually makes things worse)
>
> Agreed. You've expressed the issue very concisely and effectively. The
> way I tend to think of it is that error codes should express a
> condition that can occur in a correct program. If the condition can't
> occur in a correct program, no effort should be spent at runtime
> detecting it. If a program is going to go about detecting error codes
> that couldn't occur without a bug in the program, though, it should do
> so via assert() rather than complex error-propagation logic.
>
My idea was that program would be correct, if it inputs too much data
to hash function. It is very cheap to implement in most algorithms
(detect counter overflow). Otherwise program has to count it himself.
However, in most cases it is too late to handle correctly when it
already fails. Not returning error on too much input or output could
silently cause incorrect usage. Using assert might be better idea
since at that point it is fatal anyway, but that would cause data
leak, unless program would catch SIGABRT (it should others like SIGINT
anyway) and wipe data, but that requires all sensitive data in program
is global.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-30 6:32 ` Gregor Pintar
@ 2013-04-30 8:35 ` Szabolcs Nagy
2013-04-30 9:58 ` Gregor Pintar
0 siblings, 1 reply; 76+ messages in thread
From: Szabolcs Nagy @ 2013-04-30 8:35 UTC (permalink / raw)
To: musl
* Gregor Pintar <grpintar@gmail.com> [2013-04-30 08:32:26 +0200]:
> My idea was that program would be correct, if it inputs too much data
> to hash function. It is very cheap to implement in most algorithms
> (detect counter overflow). Otherwise program has to count it himself.
i dont think the program has to count
eg in case of sha1 if you know that the throughput is less than
10gbps then it takes more than 50years to overflow
in theory there might be use-cases where the overflow could occure
in which case reporting error makes sense, but it seems to me that
can be avoided by the proper choice of algorithm or reasonable
application design
of course if you allow configurability of the block size and rounds
etc then the overflow becomes a practical concern so the error is
justified, which is why i said that flexibility is not necessarily
a good thing (general interface requires general error handling)
> However, in most cases it is too late to handle correctly when it
> already fails. Not returning error on too much input or output could
> silently cause incorrect usage. Using assert might be better idea
> since at that point it is fatal anyway, but that would cause data
> leak, unless program would catch SIGABRT (it should others like SIGINT
> anyway) and wipe data, but that requires all sensitive data in program
> is global.
i'd let the user do the bad thing (silently overflow)
if it matters it can be designed around in the application
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-30 8:35 ` Szabolcs Nagy
@ 2013-04-30 9:58 ` Gregor Pintar
2013-04-30 11:30 ` Szabolcs Nagy
2013-05-01 7:26 ` Gregor Pintar
0 siblings, 2 replies; 76+ messages in thread
From: Gregor Pintar @ 2013-04-30 9:58 UTC (permalink / raw)
To: musl
2013/4/30, Szabolcs Nagy <nsz@port70.net>:
> * Gregor Pintar <grpintar@gmail.com> [2013-04-30 08:32:26 +0200]:
>> My idea was that program would be correct, if it inputs too much data
>> to hash function. It is very cheap to implement in most algorithms
>> (detect counter overflow). Otherwise program has to count it himself.
>
> i dont think the program has to count
>
> eg in case of sha1 if you know that the throughput is less than
> 10gbps then it takes more than 50years to overflow
>
Blowfish can encrypt max 128GB.
> in theory there might be use-cases where the overflow could occure
> in which case reporting error makes sense, but it seems to me that
> can be avoided by the proper choice of algorithm or reasonable
> application design
>
Choice of algorithm is not mine.
> of course if you allow configurability of the block size and rounds
> etc then the overflow becomes a practical concern so the error is
> justified, which is why i said that flexibility is not necessarily
> a good thing (general interface requires general error handling)
>
Block size isn't configurable (but algorithms have different block
size). Rounds can cause error only on *_create() (so does key).
Flexibility usually increases complexy.
>> However, in most cases it is too late to handle correctly when it
>> already fails. Not returning error on too much input or output could
>> silently cause incorrect usage. Using assert might be better idea
>> since at that point it is fatal anyway, but that would cause data
>> leak, unless program would catch SIGABRT (it should others like SIGINT
>> anyway) and wipe data, but that requires all sensitive data in program
>> is global.
>
> i'd let the user do the bad thing (silently overflow)
> if it matters it can be designed around in the application
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-30 9:58 ` Gregor Pintar
@ 2013-04-30 11:30 ` Szabolcs Nagy
2013-04-30 14:11 ` Gregor Pintar
2013-05-01 7:26 ` Gregor Pintar
1 sibling, 1 reply; 76+ messages in thread
From: Szabolcs Nagy @ 2013-04-30 11:30 UTC (permalink / raw)
To: musl
* Gregor Pintar <grpintar@gmail.com> [2013-04-30 11:58:26 +0200]:
> 2013/4/30, Szabolcs Nagy <nsz@port70.net>:
> > * Gregor Pintar <grpintar@gmail.com> [2013-04-30 08:32:26 +0200]:
> >> My idea was that program would be correct, if it inputs too much data
> >> to hash function. It is very cheap to implement in most algorithms
> >> (detect counter overflow). Otherwise program has to count it himself.
> >
> > i dont think the program has to count
> >
> > eg in case of sha1 if you know that the throughput is less than
> > 10gbps then it takes more than 50years to overflow
> >
> Blowfish can encrypt max 128GB.
>
i see, i didn't know this
i wonder how often do ppl check the return value
(eg you don't seem to do it in your tests)
> > in theory there might be use-cases where the overflow could occure
> > in which case reporting error makes sense, but it seems to me that
> > can be avoided by the proper choice of algorithm or reasonable
> > application design
> >
> Choice of algorithm is not mine.
i meant that if a tool is used for something that it was not
designed for then that's a user error not a tool error
the lib looks fine otherwise, but the hard parts are missing for tls
(rsa,dsa,ecdsa require a bignum library, tls requires a parser for a
lot of complex formats)
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-30 11:30 ` Szabolcs Nagy
@ 2013-04-30 14:11 ` Gregor Pintar
0 siblings, 0 replies; 76+ messages in thread
From: Gregor Pintar @ 2013-04-30 14:11 UTC (permalink / raw)
To: musl
2013/4/30, Szabolcs Nagy <nsz@port70.net>:
> * Gregor Pintar <grpintar@gmail.com> [2013-04-30 11:58:26 +0200]:
>> 2013/4/30, Szabolcs Nagy <nsz@port70.net>:
>> > * Gregor Pintar <grpintar@gmail.com> [2013-04-30 08:32:26 +0200]:
>> >> My idea was that program would be correct, if it inputs too much data
>> >> to hash function. It is very cheap to implement in most algorithms
>> >> (detect counter overflow). Otherwise program has to count it himself.
>> >
>> > i dont think the program has to count
>> >
>> > eg in case of sha1 if you know that the throughput is less than
>> > 10gbps then it takes more than 50years to overflow
>> >
>> Blowfish can encrypt max 128GB.
>>
>
> i see, i didn't know this
>
> i wonder how often do ppl check the return value
> (eg you don't seem to do it in your tests)
>
I wrote tests only to see if implementation is close to correct. Most
tests doesn't even work (they were not updated when api changed). So
tests are not even close to finished.
>> > in theory there might be use-cases where the overflow could occure
>> > in which case reporting error makes sense, but it seems to me that
>> > can be avoided by the proper choice of algorithm or reasonable
>> > application design
>> >
>> Choice of algorithm is not mine.
>
> i meant that if a tool is used for something that it was not
> designed for then that's a user error not a tool error
>
>
> the lib looks fine otherwise, but the hard parts are missing for tls
> (rsa,dsa,ecdsa require a bignum library, tls requires a parser for a
> lot of complex formats)
>
Yes, I know I will probably use libtommath (for start) and public key
crypto will probably be ported from libtomcrypt (twofish is also form
libtomcrypt). tls/ssl will probably be separate.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-30 9:58 ` Gregor Pintar
2013-04-30 11:30 ` Szabolcs Nagy
@ 2013-05-01 7:26 ` Gregor Pintar
2013-05-08 21:37 ` Daniel Cegiełka
1 sibling, 1 reply; 76+ messages in thread
From: Gregor Pintar @ 2013-05-01 7:26 UTC (permalink / raw)
To: musl
2013/4/30, Gregor Pintar <grpintar@gmail.com>:
> Blowfish can encrypt max 128GB.
>
Sorry, it's 32GB --> (2^(block size in bits / 2)) * (block size in
bytes) = (2^32) * 8
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-01 7:26 ` Gregor Pintar
@ 2013-05-08 21:37 ` Daniel Cegiełka
2013-05-08 23:00 ` idunham
0 siblings, 1 reply; 76+ messages in thread
From: Daniel Cegiełka @ 2013-05-08 21:37 UTC (permalink / raw)
To: musl
and tropicssl?
http://gitorious.org/tropicssl
http://teholabs.com/tag/ssl/
Daniel
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-08 21:37 ` Daniel Cegiełka
@ 2013-05-08 23:00 ` idunham
2013-05-09 7:36 ` Daniel Cegiełka
0 siblings, 1 reply; 76+ messages in thread
From: idunham @ 2013-05-08 23:00 UTC (permalink / raw)
To: musl
On Wed, May 08, 2013 at 11:37:47PM +0200, Daniel Cegie?ka wrote:
> and tropicssl?
>
> http://gitorious.org/tropicssl
>
> http://teholabs.com/tag/ssl/
Tropicssl is a fork of polarssl, but as far as I can tell they haven't
fixed CVEs 2013-0169, 2013-1621, and 2013-1622 yet.
While I agree with their view of the license switch, the activity (or
lack thereof) does not seem promising.
--
Isaac Dunham
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-08 23:00 ` idunham
@ 2013-05-09 7:36 ` Daniel Cegiełka
2013-05-09 9:03 ` Daniel Cegiełka
` (2 more replies)
0 siblings, 3 replies; 76+ messages in thread
From: Daniel Cegiełka @ 2013-05-09 7:36 UTC (permalink / raw)
To: musl
2013/5/9 <idunham@lavabit.com>:
> Tropicssl is a fork of polarssl, but as far as I can tell they haven't
> fixed CVEs 2013-0169, 2013-1621, and 2013-1622 yet.
>
> While I agree with their view of the license switch, the activity (or
> lack thereof) does not seem promising.
The problem is that we have the choice to either accept the GPL
(CyaSSL, PolarSSL) or write own implementation of a ssl from
scratch... or use eg tropicssl as a ssl base.
btw. license issue - can we legally link noxcuse (GPL) with musl (MIT)? :)
Daniel
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: High-priority library replacements?
2013-05-09 7:36 ` Daniel Cegiełka
@ 2013-05-09 9:03 ` Daniel Cegiełka
2013-05-09 11:10 ` LM
2013-05-09 14:08 ` Rich Felker
2 siblings, 0 replies; 76+ messages in thread
From: Daniel Cegiełka @ 2013-05-09 9:03 UTC (permalink / raw)
To: musl
> 2013/5/9 <idunham@lavabit.com>:
>
> While I agree with their view of the license switch, the activity (or
> lack thereof) does not seem promising.
FOSS License Exception - it might not be as bad as we described earlier.
PolarSSL:
https://polarssl.org/foss-license-exception
CyaSSL:
http://www.yassh.com/yaSSL/License.html
Daniel
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-09 7:36 ` Daniel Cegiełka
2013-05-09 9:03 ` Daniel Cegiełka
@ 2013-05-09 11:10 ` LM
2013-05-09 14:08 ` Rich Felker
2 siblings, 0 replies; 76+ messages in thread
From: LM @ 2013-05-09 11:10 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 590 bytes --]
On Thu, May 9, 2013 at 3:36 AM, Daniel Cegiełka
<daniel.cegielka@gmail.com>wrote:
>
> btw. license issue - can we legally link noxcuse (GPL) with musl (MIT)? :)
>
>
That's a nice collection. Wasn't aware it was available. Minix (check
simple directory for lightweight alternatives not ported from BSD systems)
and sbase also offer some alternatives to GNU tools, busybox, toybox,
Heirloom Project, et al.
I'm currently look at code for various implementations of common utilities
and checking how well it will port to other platforms besides Linux.
Sincerely,
Laura
[-- Attachment #2: Type: text/html, Size: 894 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-09 7:36 ` Daniel Cegiełka
2013-05-09 9:03 ` Daniel Cegiełka
2013-05-09 11:10 ` LM
@ 2013-05-09 14:08 ` Rich Felker
2013-05-09 14:40 ` Daniel Cegiełka
2 siblings, 1 reply; 76+ messages in thread
From: Rich Felker @ 2013-05-09 14:08 UTC (permalink / raw)
To: musl
On Thu, May 09, 2013 at 09:36:18AM +0200, Daniel Cegiełka wrote:
> 2013/5/9 <idunham@lavabit.com>:
>
> > Tropicssl is a fork of polarssl, but as far as I can tell they haven't
> > fixed CVEs 2013-0169, 2013-1621, and 2013-1622 yet.
> >
> > While I agree with their view of the license switch, the activity (or
> > lack thereof) does not seem promising.
>
> The problem is that we have the choice to either accept the GPL
> (CyaSSL, PolarSSL) or write own implementation of a ssl from
> scratch... or use eg tropicssl as a ssl base.
>
> btw. license issue - can we legally link noxcuse (GPL) with musl (MIT)? :)
Of course; otherwise musl would be rather useless since there's a huge
corpus of GPL code you want to link and run. The point of a permissive
license is that it does not restrict you from doing things like this.
BTW, I'll probably relicense noxcuse anyway. I just never got around
to doing much/anything with it after initially publishing it. The main
app in it that's really useful right now though is iconv, since
Busybox lacks iconv and glibc iconv is glibc-specific.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-09 14:08 ` Rich Felker
@ 2013-05-09 14:40 ` Daniel Cegiełka
2013-05-09 14:45 ` Rich Felker
2013-05-09 16:40 ` LM
0 siblings, 2 replies; 76+ messages in thread
From: Daniel Cegiełka @ 2013-05-09 14:40 UTC (permalink / raw)
To: musl
2013/5/9 Rich Felker <dalias@aerifal.cx>:
>> btw. license issue - can we legally link noxcuse (GPL) with musl (MIT)? :)
>
> Of course; otherwise musl would be rather useless since there's a huge
> corpus of GPL code you want to link and run. The point of a permissive
> license is that it does not restrict you from doing things like this.
My question was a joke, which was to show how absurd case we have now
- instead of software freedom we have concerns.
> BTW, I'll probably relicense noxcuse anyway.
This is the result of frustration:
http://en.wikipedia.org/wiki/WTFPL
http://www.wtfpl.net/about/
btw. OpenSSL.org published a small version of openssl-fips. It
contains only FIPS 140-2 validated code.
http://www.openssl.org/source/
http://www.openssl.org/docs/fips/fipsnotes.html
OpeSSL-fips vs CyaSSL (musl)
# ls -lh /var/tmp/ports/cyassl/image/usr/lib/
-rw-r--r-- 1 root root 866.9k May 9 09:13 libcyassl.a
-rwxr-xr-x 1 root root 659.6k May 9 09:13 libcyassl.so.5.0.0
# ls -lh /usr/local/ssl/fips-2.0/lib/
-r--r--r-- 1 root root 685.7k May 9 12:34 fipscanister.o
thx,
Daniel
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-09 14:40 ` Daniel Cegiełka
@ 2013-05-09 14:45 ` Rich Felker
2013-05-12 22:42 ` Brad Conroy
2013-05-09 16:40 ` LM
1 sibling, 1 reply; 76+ messages in thread
From: Rich Felker @ 2013-05-09 14:45 UTC (permalink / raw)
To: musl
On Thu, May 09, 2013 at 04:40:03PM +0200, Daniel Cegiełka wrote:
> btw. OpenSSL.org published a small version of openssl-fips. It
> contains only FIPS 140-2 validated code.
>
> http://www.openssl.org/source/
> http://www.openssl.org/docs/fips/fipsnotes.html
>
> OpeSSL-fips vs CyaSSL (musl)
>
> # ls -lh /var/tmp/ports/cyassl/image/usr/lib/
> -rw-r--r-- 1 root root 866.9k May 9 09:13 libcyassl.a
> -rwxr-xr-x 1 root root 659.6k May 9 09:13 libcyassl.so.5.0.0
>
> # ls -lh /usr/local/ssl/fips-2.0/lib/
> -r--r--r-- 1 root root 685.7k May 9 12:34 fipscanister.o
Wow, putting the whole thing in a single .o file. That's classy. 685k
even for programs that just need a few functions from it.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-09 14:45 ` Rich Felker
@ 2013-05-12 22:42 ` Brad Conroy
2013-05-15 20:17 ` Rich Felker
0 siblings, 1 reply; 76+ messages in thread
From: Brad Conroy @ 2013-05-12 22:42 UTC (permalink / raw)
To: musl
IIRC, Axtls was <100kb and has an openSSL API and has a stunnel (axtlswrap) and httpd with ssl (axhttpd) that are <100kb static builds
> > OpeSSL-fips vs CyaSSL (musl)
> >
> > # ls -lh /var/tmp/ports/cyassl/image/usr/lib/
> > -rw-r--r-- 1 root
> root 866.9k
> May 9 09:13 libcyassl.a
> > -rwxr-xr-x 1 root
> root 659.6k
> May 9 09:13 libcyassl.so.5.0.0
> >
> > # ls -lh /usr/local/ssl/fips-2.0/lib/
> > -r--r--r-- 1 root
> root 685.7k
> May 9 12:34 fipscanister.o
>
In case I don't get around to fully researching it, the kernel spews out major and minor numbers in /sys that are greater than 255, so the makedev in /sys/sysmacros.h is insufficient (also major() and minor()???) I noticed because I wrote a shell script that populates /dev from /sys in <0.2 seconds but failed for large major/minor numbers when I built busybox with musl. At first glance, it looks like dev_t needs to be uint and have the extra bits from major and minor prepended as in the kernel's dev_t?
R,
Brad Conroy
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-12 22:42 ` Brad Conroy
@ 2013-05-15 20:17 ` Rich Felker
2013-05-16 16:12 ` Justin Cormack
0 siblings, 1 reply; 76+ messages in thread
From: Rich Felker @ 2013-05-15 20:17 UTC (permalink / raw)
To: musl
On Sun, May 12, 2013 at 03:42:23PM -0700, Brad Conroy wrote:
> In case I don't get around to fully researching it, the kernel spews
> out major and minor numbers in /sys that are greater than 255, so
> the makedev in /sys/sysmacros.h is insufficient (also major() and
> minor()???) I noticed because I wrote a shell script that populates
> /dev from /sys in <0.2 seconds but failed for large major/minor
> numbers when I built busybox with musl. At first glance, it looks
> like dev_t needs to be uint and have the extra bits from major and
> minor prepended as in the kernel's dev_t?
Should be fixed in git. Let me know if it doesn't work right.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-15 20:17 ` Rich Felker
@ 2013-05-16 16:12 ` Justin Cormack
2013-05-17 1:56 ` Rich Felker
0 siblings, 1 reply; 76+ messages in thread
From: Justin Cormack @ 2013-05-16 16:12 UTC (permalink / raw)
To: musl
On Wed, May 15, 2013 at 9:17 PM, Rich Felker <dalias@aerifal.cx> wrote:
> On Sun, May 12, 2013 at 03:42:23PM -0700, Brad Conroy wrote:
>> In case I don't get around to fully researching it, the kernel spews
>> out major and minor numbers in /sys that are greater than 255, so
>> the makedev in /sys/sysmacros.h is insufficient (also major() and
>> minor()???) I noticed because I wrote a shell script that populates
>> /dev from /sys in <0.2 seconds but failed for large major/minor
>> numbers when I built busybox with musl. At first glance, it looks
>> like dev_t needs to be uint and have the extra bits from major and
>> minor prepended as in the kernel's dev_t?
>
> Should be fixed in git. Let me know if it doesn't work right.
mknod/mknodat still does return syscall(SYS_mknod, path, mode, dev &
0xffff); - I believe it should pass the full 32 bits.
Also the top 32 bits are only for compatibility with glibc I think,
not sure the macros should touch or use them, as the kernel has 32 bit
dev_t and no plans to change.
Justin
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-16 16:12 ` Justin Cormack
@ 2013-05-17 1:56 ` Rich Felker
2013-05-17 7:28 ` Justin Cormack
0 siblings, 1 reply; 76+ messages in thread
From: Rich Felker @ 2013-05-17 1:56 UTC (permalink / raw)
To: musl
On Thu, May 16, 2013 at 05:12:04PM +0100, Justin Cormack wrote:
> On Wed, May 15, 2013 at 9:17 PM, Rich Felker <dalias@aerifal.cx> wrote:
> > On Sun, May 12, 2013 at 03:42:23PM -0700, Brad Conroy wrote:
> >> In case I don't get around to fully researching it, the kernel spews
> >> out major and minor numbers in /sys that are greater than 255, so
> >> the makedev in /sys/sysmacros.h is insufficient (also major() and
> >> minor()???) I noticed because I wrote a shell script that populates
> >> /dev from /sys in <0.2 seconds but failed for large major/minor
> >> numbers when I built busybox with musl. At first glance, it looks
> >> like dev_t needs to be uint and have the extra bits from major and
> >> minor prepended as in the kernel's dev_t?
> >
> > Should be fixed in git. Let me know if it doesn't work right.
>
> mknod/mknodat still does return syscall(SYS_mknod, path, mode, dev &
> 0xffff); - I believe it should pass the full 32 bits.
>
> Also the top 32 bits are only for compatibility with glibc I think,
> not sure the macros should touch or use them, as the kernel has 32 bit
> dev_t and no plans to change.
mknod and mknodat should be fixed now. Anything else? I don't see much
harm in having the macros match what glibc does, and it could be
useful if the kernel allows 64bit device numbers on 64-bit archs. But
if not, there's also an argument to be made for reducing code size by
dropping the part of the macros for the high bits. What do you think?
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-17 1:56 ` Rich Felker
@ 2013-05-17 7:28 ` Justin Cormack
0 siblings, 0 replies; 76+ messages in thread
From: Justin Cormack @ 2013-05-17 7:28 UTC (permalink / raw)
To: musl
On Fri, May 17, 2013 at 2:56 AM, Rich Felker <dalias@aerifal.cx> wrote:
> On Thu, May 16, 2013 at 05:12:04PM +0100, Justin Cormack wrote:
>> On Wed, May 15, 2013 at 9:17 PM, Rich Felker <dalias@aerifal.cx> wrote:
>> > On Sun, May 12, 2013 at 03:42:23PM -0700, Brad Conroy wrote:
>> >> In case I don't get around to fully researching it, the kernel spews
>> >> out major and minor numbers in /sys that are greater than 255, so
>> >> the makedev in /sys/sysmacros.h is insufficient (also major() and
>> >> minor()???) I noticed because I wrote a shell script that populates
>> >> /dev from /sys in <0.2 seconds but failed for large major/minor
>> >> numbers when I built busybox with musl. At first glance, it looks
>> >> like dev_t needs to be uint and have the extra bits from major and
>> >> minor prepended as in the kernel's dev_t?
>> >
>> > Should be fixed in git. Let me know if it doesn't work right.
>>
>> mknod/mknodat still does return syscall(SYS_mknod, path, mode, dev &
>> 0xffff); - I believe it should pass the full 32 bits.
>>
>> Also the top 32 bits are only for compatibility with glibc I think,
>> not sure the macros should touch or use them, as the kernel has 32 bit
>> dev_t and no plans to change.
>
> mknod and mknodat should be fixed now. Anything else? I don't see much
> harm in having the macros match what glibc does, and it could be
> useful if the kernel allows 64bit device numbers on 64-bit archs. But
> if not, there's also an argument to be made for reducing code size by
> dropping the part of the macros for the high bits. What do you think?
Given that the struct stat64 the kernel uses specifically defines
st_rdev as unsigned long long, it probably makes sense to leave dev_t
as 64 bits. I would be inclined to not do anything with the extra bits
in the macros though, as the meaning of those bits is undefined, and
might not be defined as glibc does (seems to be based on what NetBSD
does), and changing the macros does not break anything.
Justin
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-09 14:40 ` Daniel Cegiełka
2013-05-09 14:45 ` Rich Felker
@ 2013-05-09 16:40 ` LM
1 sibling, 0 replies; 76+ messages in thread
From: LM @ 2013-05-09 16:40 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 779 bytes --]
On Thu, May 9, 2013 at 10:40 AM, Daniel Cegiełka
<daniel.cegielka@gmail.com>wrote:
>
> This is the result of frustration:
> http://en.wikipedia.org/wiki/WTFPL
> http://www.wtfpl.net/about/
>
> I personally like this one:
http://creativecommons.org/publicdomain/zero/1.0/
However, you're very right about the frustration. I think the average user
just ignores it and copies things without worrying about the legalities.
However, the whole point of Open Source was to have licenses that gave
certain freedoms that would allow you to safely reuse works. Out of
respect, if someone goes to the trouble to release code with an Open Source
license, it's nice to at least attempt to comply with the conditions of the
license if one can figure out what those are.
[-- Attachment #2: Type: text/html, Size: 1248 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 6:43 ` Gregor Pintar
2013-04-26 0:55 ` idunham
@ 2013-04-30 18:47 ` Nicolas Braud-Santoni
2013-04-30 19:18 ` Gregor Pintar
2013-05-26 20:09 ` Daniel Cegiełka
2 siblings, 1 reply; 76+ messages in thread
From: Nicolas Braud-Santoni @ 2013-04-30 18:47 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 599 bytes --]
On 25/04/2013 08:43, Gregor Pintar wrote:
> Hello.
> [...]
>
> I think best way is not to trust any certificate authority.
> Maybe some certificate p2p protocol could be done?
Hello,
Are you aware of DANE (RFC6698, https://en.wikipedia.org/wiki/DANE) ?
It is a RFC which suggests holding certificates fingerprints in special
DNS records.
Since DNSSEC allows us to establish trust of these records, this is a
simple and robust alternative to CA-based trust models.
However, and AFAIK, it doesn't cope with entities that aren't accessed
through a hostname.
Have a good day,
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-30 18:47 ` Nicolas Braud-Santoni
@ 2013-04-30 19:18 ` Gregor Pintar
0 siblings, 0 replies; 76+ messages in thread
From: Gregor Pintar @ 2013-04-30 19:18 UTC (permalink / raw)
To: musl
2013/4/30, Nicolas Braud-Santoni <nicolas.braudsantoni@gmail.com>:
> On 25/04/2013 08:43, Gregor Pintar wrote:
>> Hello.
>> [...]
>>
>> I think best way is not to trust any certificate authority.
>> Maybe some certificate p2p protocol could be done?
>
> Hello,
>
> Are you aware of DANE (RFC6698, https://en.wikipedia.org/wiki/DANE) ?
> It is a RFC which suggests holding certificates fingerprints in special
> DNS records.
> Since DNSSEC allows us to establish trust of these records, this is a
> simple and robust alternative to CA-based trust models.
>
> However, and AFAIK, it doesn't cope with entities that aren't accessed
> through a hostname.
>
>
> Have a good day,
>
>
Thanks, I was not aware of it.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 6:43 ` Gregor Pintar
2013-04-26 0:55 ` idunham
2013-04-30 18:47 ` Nicolas Braud-Santoni
@ 2013-05-26 20:09 ` Daniel Cegiełka
2013-05-27 15:53 ` Gregor Pintar
2013-05-28 13:11 ` LM
2 siblings, 2 replies; 76+ messages in thread
From: Daniel Cegiełka @ 2013-05-26 20:09 UTC (permalink / raw)
To: musl
2013/4/25 Gregor Pintar <grpintar@gmail.com>:
> Hello.
> I am working on cryptographic library. It's far from being finished.
> I would be very glad, if someone could look at it.
> Currently I have problems with API design so help would be welcome.
> It isn't in git yet.
> You can get it here:
> https://dl.dropboxusercontent.com/u/83450675/kripto/kripto.tar.gz
> I have plans for SSL library on top of it, but it could take years.
@Gregor Pintar
What do you think about using tropicssl on libkripto (aka libssl and
libcrypto in openssl)?
I looked at the difference between PolarSSL and tropicssl and
differences are often purely cosmetic. I think we can apply the
following changes in PolarSSL to tropicssl and thereby gain a small
SSL library under a BSD license.
http://gitorious.org/tropicssl
Does anyone have a better suggestion?
Daniel
> I think best way is not to trust any certificate authority.
> Maybe some certificate p2p protocol could be done?
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-26 20:09 ` Daniel Cegiełka
@ 2013-05-27 15:53 ` Gregor Pintar
2013-05-28 9:27 ` Daniel Cegiełka
2013-05-28 13:11 ` LM
1 sibling, 1 reply; 76+ messages in thread
From: Gregor Pintar @ 2013-05-27 15:53 UTC (permalink / raw)
To: musl
2013/5/26, Daniel Cegiełka <daniel.cegielka@gmail.com>:
> @Gregor Pintar
> What do you think about using tropicssl on libkripto (aka libssl and
> libcrypto in openssl)?
>
TropicSSL has quite nice API, but I think it would require API changes
(cipher suites) to get full advantage of "dynamic construction" of
algorithms (block cipher + mode + mac = authenticated stream cipher)
and kripto API is not finished yet so I can't really say.
> I looked at the difference between PolarSSL and tropicssl and
> differences are often purely cosmetic. I think we can apply the
> following changes in PolarSSL to tropicssl and thereby gain a small
> SSL library under a BSD license.
>
> http://gitorious.org/tropicssl
>
> Does anyone have a better suggestion?
>
Unless someone really takes a deep look (vulnerabilities, TLS 1.2)
into tropicssl, I think it's better to use PolarSSL for now despite
GPL.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-27 15:53 ` Gregor Pintar
@ 2013-05-28 9:27 ` Daniel Cegiełka
2013-05-28 17:30 ` Gregor Pintar
0 siblings, 1 reply; 76+ messages in thread
From: Daniel Cegiełka @ 2013-05-28 9:27 UTC (permalink / raw)
To: musl
2013/5/27 Gregor Pintar <grpintar@gmail.com>:
> Unless someone really takes a deep look (vulnerabilities, TLS 1.2)
> into tropicssl,
I think that in a few weeks, I will have free time to do so. That's
why I care about the opinion, if you are interested to use tropicssl's
BSD code as ssl layer for kripto.
Daniel
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-28 9:27 ` Daniel Cegiełka
@ 2013-05-28 17:30 ` Gregor Pintar
0 siblings, 0 replies; 76+ messages in thread
From: Gregor Pintar @ 2013-05-28 17:30 UTC (permalink / raw)
To: musl
2013/5/28, Daniel Cegiełka <daniel.cegielka@gmail.com>:
> 2013/5/27 Gregor Pintar <grpintar@gmail.com>:
>
>> Unless someone really takes a deep look (vulnerabilities, TLS 1.2)
>> into tropicssl,
>
> I think that in a few weeks, I will have free time to do so. That's
> why I care about the opinion, if you are interested to use tropicssl's
> BSD code as ssl layer for kripto.
>
> Daniel
>
Yes, I think it would be good, but API could maybe be little
cleaner/simpler. I would suggest to use prefix (library name) on all
external symbols. PolarSSL and TropicSSL expose too much in header.
I'm think tropicssl compatibility doesn't mean anything as it is not
very popular, but we could add PolarSSL and OpenSSL compatibility
header. I would also look at axTLS's API
(http://axtls.sourceforge.net/dox/group__c__api.html). API is very
important, all other things can be fixed.
Currently I'm quite busy, but maybe I could help as I have problems
with kripto's API. I might get some ideas for it.
Gregor
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-26 20:09 ` Daniel Cegiełka
2013-05-27 15:53 ` Gregor Pintar
@ 2013-05-28 13:11 ` LM
2013-05-28 21:38 ` Rob Landley
1 sibling, 1 reply; 76+ messages in thread
From: LM @ 2013-05-28 13:11 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 230 bytes --]
Some of you have probably seen this before, but here's another interesting
list of lightweight libraries:
http://suckless.org/rocks
The site also mentions this list with some other alternatives:
http://harmful.cat-v.org/software/
[-- Attachment #2: Type: text/html, Size: 341 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-28 13:11 ` LM
@ 2013-05-28 21:38 ` Rob Landley
2013-05-31 11:13 ` LM
0 siblings, 1 reply; 76+ messages in thread
From: Rob Landley @ 2013-05-28 21:38 UTC (permalink / raw)
To: musl; +Cc: LM
On 05/28/2013 08:11:34 AM, LM wrote:
> Some of you have probably seen this before, but here's another
> interesting
> list of lightweight libraries:
> http://suckless.org/rocks
> The site also mentions this list with some other alternatives:
> http://harmful.cat-v.org/software/
Did anybody ever do a wiki page from the mailing list thread?
Rob
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-28 21:38 ` Rob Landley
@ 2013-05-31 11:13 ` LM
2013-05-31 11:36 ` LM
0 siblings, 1 reply; 76+ messages in thread
From: LM @ 2013-05-31 11:13 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 276 bytes --]
On Tue, May 28, 2013 at 5:38 PM, Rob Landley <rob@landley.net> wrote:
> Did anybody ever do a wiki page from the mailing list thread?
>
If nobody's volunteered, I can put the links we've mentioned on the list
plus some others up on a page at the wiki when I have some time.
[-- Attachment #2: Type: text/html, Size: 580 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-05-31 11:13 ` LM
@ 2013-05-31 11:36 ` LM
0 siblings, 0 replies; 76+ messages in thread
From: LM @ 2013-05-31 11:36 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 487 bytes --]
Forgot to mention, I'd need permissions to create/edit a page first. I can
login to the wiki, but I can't add anything at this point.
On Fri, May 31, 2013 at 7:13 AM, LM <lmemsm@gmail.com> wrote:
> On Tue, May 28, 2013 at 5:38 PM, Rob Landley <rob@landley.net> wrote:
>
>> Did anybody ever do a wiki page from the mailing list thread?
>>
>
> If nobody's volunteered, I can put the links we've mentioned on the list
> plus some others up on a page at the wiki when I have some time.
>
[-- Attachment #2: Type: text/html, Size: 1021 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 4:15 Rich Felker
` (2 preceding siblings ...)
2013-04-25 6:43 ` Gregor Pintar
@ 2013-04-25 7:21 ` Hal Clark
2013-04-25 10:58 ` Igmar Palsenberg
2013-04-25 12:28 ` Rich Felker
2013-04-25 11:44 ` LM
` (2 subsequent siblings)
6 siblings, 2 replies; 76+ messages in thread
From: Hal Clark @ 2013-04-25 7:21 UTC (permalink / raw)
To: musl
On 4/24/13, Rich Felker <dalias@aerifal.cx> wrote:
> Hi all,
>
> The recent thread "Best place to discuss other lightweight libraries"
> had me thinking we should really put together a list of high-priority
> library replacements that need to be done. I don't think I'll be the
> one to do them, but I wouldn't mind overseeing/mentoring someone
> working on them, or just getting somebody else interested in these
> projects.
>
> A few initial ideas:
>
> 1. A light, C, UTF-8-only Unicode library. The most important things
> it should implement are things needed by any application that presents
> text to the user, specifically line-breaking (UAX#14), bidi (UAX#9),
> identifying grapheme clusters, etc. Things like case- and
> normalization-insensitive comparison, application of Unicode-format
> collation rules, etc. would also possibly be useful.
>
> 2. SSL. The modern internet basically requires using SSL everywhere.
> We should be aiming/expecting to transition to a world where even
> non-login-based sites are 100% https; the threats of malicious http
> injection attacks from rogue or advertising-based access points has
> gotten too great. Unfortunately, all of the existing SSL
> implementations are bloated, buggy, and fail even the most basic
> robustness requirements. A good solution would be based on tomcrypt
> and would expose a minimal, simple API suited for event-loop-based or
> threaded use. It may also be useful to have an optional wrapper layer
> to expose an API that mimics openssl or gnutls. It should also be able
> to keep up with the changing demands of how to determine which
> certificate authorities are to be trusted.
>
I saw earlier that Cyassl was briefly mentioned. Now, in general I am
interested in your idea of a "minimal, simple API", but I've had great
experiences with Cyassl. IMO it is exactly what you're looking for.
Could you comment on what parts of it are unsuitable for your intended
purposes? I'm curious what would need to be changed or replaced.
(I'm not affiliated with Cyassl - I have just had (really) good
experiences with it after trying the GnuTLS/OpenSSL route.)
> 3. Image format and compression (libpng, zlib, etc.). The existing
> libraries are full of atrociously bad error-handling practices, and
> quite bloated. These libraries should be well-factored so that
> programs that just want to read (the majority of use cases) don't have
> to pull in code for writing. Unfortunately the interfaces might not be
> so simple, since an important usage case is progressive rendering of
> partially-loaded images, both for the purpose of loading over slow
> network connections and for rendering thumbnails quickly.
>
> All of these libraries should:
>
> - Avoid namespace pollution. Only external symbols should be the
> public API and internal-use stuff prefixed with an ugly prefix
> that's extremely unlikely to clash with anything.
> - Avoid unnecessary allocation. Use caller-provided objects where
> possible or provide both options.
> - Have absolutely zero global state.
>
> That's all I can think of at the moment but I'm sure there are other
> needs I've come across and forgotten. Please feel free to supplement
> this list.
>
> Rich
>
-hal clark
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: High-priority library replacements?
2013-04-25 7:21 ` Hal Clark
@ 2013-04-25 10:58 ` Igmar Palsenberg
2013-04-25 12:28 ` Rich Felker
1 sibling, 0 replies; 76+ messages in thread
From: Igmar Palsenberg @ 2013-04-25 10:58 UTC (permalink / raw)
To: musl
> I saw earlier that Cyassl was briefly mentioned. Now, in general I am
> interested in your idea of a "minimal, simple API", but I've had great
> experiences with Cyassl. IMO it is exactly what you're looking for.
>
> Could you comment on what parts of it are unsuitable for your intended
> purposes? I'm curious what would need to be changed or replaced.
>
> (I'm not affiliated with Cyassl - I have just had (really) good
> experiences with it after trying the GnuTLS/OpenSSL route.)
It's also GPL. I consider that a huge disadvantage, and also makes inclusion in Musl a no-go.
Igmar
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 7:21 ` Hal Clark
2013-04-25 10:58 ` Igmar Palsenberg
@ 2013-04-25 12:28 ` Rich Felker
1 sibling, 0 replies; 76+ messages in thread
From: Rich Felker @ 2013-04-25 12:28 UTC (permalink / raw)
To: musl
On Thu, Apr 25, 2013 at 12:21:12AM -0700, Hal Clark wrote:
> I saw earlier that Cyassl was briefly mentioned. Now, in general I am
> interested in your idea of a "minimal, simple API", but I've had great
> experiences with Cyassl. IMO it is exactly what you're looking for.
Minimal, simple API basically means I wouldn't want to see anything
like the mess of OpenSSL's "BIO" layer. If I didn't care for making
users of the library happy, I would say it should just have one
function, an open function, that would hand over the socket to the
library and give you a new socket (from socketpair) to use in its
place as an unencrypted socket. But then everybody would whine about
threads. So I think there need to be some additional interfaces, but
not more than a few.
> Could you comment on what parts of it are unsuitable for your intended
> purposes? I'm curious what would need to be changed or replaced.
So far, based on what I've seen/read, I find it rather suitable. Lots
of people find the license (GPL) unacceptable however.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 4:15 Rich Felker
` (3 preceding siblings ...)
2013-04-25 7:21 ` Hal Clark
@ 2013-04-25 11:44 ` LM
2013-04-25 12:51 ` Rich Felker
2013-04-26 4:19 ` Isaac Dunham
2013-04-26 4:32 ` nwmcsween
6 siblings, 1 reply; 76+ messages in thread
From: LM @ 2013-04-25 11:44 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 5668 bytes --]
On Thu, Apr 25, 2013 at 12:15 AM, Rich Felker <dalias@aerifal.cx> wrote:
> The recent thread "Best place to discuss other lightweight libraries"
> had me thinking we should really put together a list of high-priority
> library replacements that need to be done.
>
Sounds like a great idea.
> 1. A light, C, UTF-8-only Unicode library. The most important things
> it should implement are things needed by any application that presents
> text to the user, specifically line-breaking (UAX#14), bidi (UAX#9),
> identifying grapheme clusters, etc. Things like case- and
> normalization-insensitive comparison, application of Unicode-format
> collation rules, etc. would also possibly be useful.
>
I have something I'm working on that I'm using for my sdcv port. It mainly
covers things like switching from utf8 to uint32_t and back and utf8
equivalents of isspace, isdigit, toupper, tolower, etc. I really like the
Plan 9 rune implementation. I didn't need all the functionality of the
Plan 9 library, so I'm creating a small library with just the minimal
functions I need. If you check Ohloh, I've also seen some examples of
internationalized versions of standard functions like isdigit, isspace
there.
>
> 2. SSL.
...
> Unfortunately, all of the existing SSL
> implementations are bloated, buggy, and fail even the most basic
> robustness requirements. A good solution would be based on tomcrypt
> and would expose a minimal, simple API suited for event-loop-based or
> threaded use. It may also be useful to have an optional wrapper layer
> to expose an API that mimics openssl or gnutls.
I think an openssl wrapper and possibly a gnutls one would be essential if
you want to have it work with all the Open Source programs already out
there.
There are three main options I currently know about: openssl, gnutls, and
nss. Each has a different license. So depending on how a program is
licensed, you may not legally be able to use some of these libraries with
the program. Any GNU licensed program requires an optional waiver to link
with openssl. You find many Open Source programs provide the optional
waiver. One really good example of the license issue concerns lynx. You
can check their mailing list for a detailed discussion. The highlights are
that lynx is GPLv2 and the project can't contact all the authors who worked
on it to change the license or get a waiver for linking to other types of
incompatible licenses. The openssl library can't be used with a GNU
program unless there's a waiver for it because one of the clauses in the
openssl license goes against the GNU license principles. The gnutls
openssl compatibility wrapper is GPL v3, so it can't be used unless the
program is relicensed at GPL v3 (
http://www.gnu.org/licenses/gpl-faq.html#AllCompatibility ). That just
leaves nss licensed under MPL, LGPL and/or GPL. If it's LGPL, lynx can use
it.
Moral of the story, if anything's going to be done to replace these
libraries, it would have to be under appropriate license(s) or one would
not be able to use the results with existing Open Source software already
out there without violating copyright.
3. Image format and compression (libpng, zlib, etc.).
>
A good SVG library is needed. To get SVG into Tuxpaint, it added libcroco,
librsvg2, libcairo and other GTK+ library dependencies. Tuxpaint is based
on SDL, so would prefer not to see all the added code for two GUI
libraries. Someone was recently asking on the SDL list about SVG library
options that are actively supported. The only one with active support
that I could locate (besides librsvg2) was the one used by netsurf (
http://www.netsurf-browser.org/projects/libsvgtiny/ ).
If you're looking at graphics formats. You might as well add audio formats
to the list too. There are a wide variety of libraries for the different
audio formats and they range in quality (and license types).
> That's all I can think of at the moment but I'm sure there are other
> needs I've come across and forgotten. Please feel free to supplement
> this list.
>
>
As already mentioned, libintl is on my list of things to replace if
possible. pkg-config has a nice replacement in pkgconf. (If a list is
created, might be helpful to list possible replacements already out
there.) Would like to see some of the pieces that are essential parts of
the GNU build system/autotools replaced with some more efficient
alternatives. It's easier to keep using the GNU autotools system than to
completely replace it with a new design. Replacing it with another system
completely would mean switching over any program or library one wants to
build from the many Open Source programs already out there that currently
use it. The CoApp project is attempting to do that on Windows (switch
other build systems over to the one used by Visual C++). Seems like a lot
of unnecessary work to port applications. Would rather see the current
build systems already used (autotools, cmake, etc.) streamlined or see drop
in replacements that are better designed.
One other area that can be troublesome is when there are multiple libraries
with similar functionality and some of the programs you want to build use
one while the rest use another. The openssl/gnutls/nss example is an
issue. It's typically solved by supplying wrapper interfaces. The case I
keep running into at the moment is libxml2/expat. libxml2 is newer and
works in conjunction with libraries like libxslt. Some Open Source
programs let you pick one or the other. Unfortunately, there are some that
insist on only one. Makes it hard to simplify one's system down to just
using one XML library.
Sincerely,
Laura
[-- Attachment #2: Type: text/html, Size: 6927 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: High-priority library replacements?
2013-04-25 11:44 ` LM
@ 2013-04-25 12:51 ` Rich Felker
2013-04-25 15:30 ` Jens Staal
2013-04-25 16:51 ` Zvi Gilboa
0 siblings, 2 replies; 76+ messages in thread
From: Rich Felker @ 2013-04-25 12:51 UTC (permalink / raw)
To: musl
On Thu, Apr 25, 2013 at 07:44:39AM -0400, LM wrote:
> I have something I'm working on that I'm using for my sdcv port. It mainly
> covers things like switching from utf8 to uint32_t and back and utf8
> equivalents of isspace, isdigit, toupper, tolower, etc. I really like the
> Plan 9 rune implementation. I didn't need all the functionality of the
> Plan 9 library, so I'm creating a small library with just the minimal
> functions I need. If you check Ohloh, I've also seen some examples of
> internationalized versions of standard functions like isdigit, isspace
> there.
This is somewhat different from what I was thinking of. You're looking
more at replacing the standard C api functions for mb/wc stuff so you
can work with UTF-8 regardless of the locale encoding. I was talking
about importane Unicode functionality outside what the C API can
provide.
> I think an openssl wrapper and possibly a gnutls one would be essential if
> you want to have it work with all the Open Source programs already out
> there.
Agreed. The big question, I guess, is whether the aim would be to have
big existing programs adopt it, or to provide just a new option for
lighter programs. But since I think we'd want to be able to use many
existing programs without dealing with the trouble of openssl and
gnutls, I would lean towards the former as a goal.
> incompatible licenses. The openssl library can't be used with a GNU
> program unless there's a waiver for it because one of the clauses in the
> openssl license goes against the GNU license principles. The gnutls
Not _used_ but _distributed_. The GPL does not restrict use
whatsoever (and takes the position that it legally can't do so) so
it's fine to use OpenSSL with GPL programs as long as you don't
distribute the resulting binary. This is of course a problem for
package maintainers/distributions, and distributing both openssl and
the GNU program and a script to link them together might even be seen
as an infringing activity.
> Moral of the story, if anything's going to be done to replace these
> libraries, it would have to be under appropriate license(s) or one would
> not be able to use the results with existing Open Source software already
> out there without violating copyright.
Agreed. I should have mentioned that all of these libraries should aim
for permissive license (MIT/BSD) or at the very least, LGPL 2.1+,
which while not as permissive, is at least compatible with all
relevant foss licenses.
> 3. Image format and compression (libpng, zlib, etc.).
>
> A good SVG library is needed. To get SVG into Tuxpaint, it added libcroco,
> librsvg2, libcairo and other GTK+ library dependencies. Tuxpaint is based
> on SDL, so would prefer not to see all the added code for two GUI
Yes, I completely forgot about SVG, but really all modern UI and
website graphics should be SVG, since raster images look hideous on
modern 300+ dpi display devices (assuming they're interpreted as
96dpi, which they always are).
> libraries. Someone was recently asking on the SDL list about SVG library
> options that are actively supported. The only one with active support
> that I could locate (besides librsvg2) was the one used by netsurf (
> http://www.netsurf-browser.org/projects/libsvgtiny/ ).
How complete is it? SVG rendering is rather difficult if you want to
support all the features, and if you don't, lots of files will come
out looking wrong... In particular if you want to support text, you
need a font and layout engine, and you may need clones of "standard"
fonts that people use assuming you'll have them because they're
included with Windows...
BTW, SVG is XML-based, but I don't think a good SVG library should
need a heavy XML library. Rather, it should take shortcuts to
interpret just SVG content in a less-bloated way.
> If you're looking at graphics formats. You might as well add audio formats
> to the list too. There are a wide variety of libraries for the different
> audio formats and they range in quality (and license types).
Yes and no. Audio and video are both generally a good bit more
difficult than images, and fewer programs need these things. Images
are needed by almost every interactive program these days both for
interface elements and displaying content (which everyone assumes can
display images), but audio is really only widely used for
notifications, games, media players, and voice-chat apps.
> As already mentioned, libintl is on my list of things to replace if
> possible.
Yes, gettext..
> pkg-config has a nice replacement in pkgconf. (If a list is
> created, might be helpful to list possible replacements already out
> there.) Would like to see some of the pieces that are essential parts of
> the GNU build system/autotools replaced with some more efficient
> [...]
> of unnecessary work to port applications. Would rather see the current
> build systems already used (autotools, cmake, etc.) streamlined or see drop
> in replacements that are better designed.
I seem to recall an effort somewhere to do exactly this, but I can't
remember where I saw it..
> One other area that can be troublesome is when there are multiple libraries
> with similar functionality and some of the programs you want to build use
> one while the rest use another. The openssl/gnutls/nss example is an
> issue. It's typically solved by supplying wrapper interfaces. The case I
> keep running into at the moment is libxml2/expat. libxml2 is newer and
> works in conjunction with libraries like libxslt. Some Open Source
> programs let you pick one or the other. Unfortunately, there are some that
> insist on only one. Makes it hard to simplify one's system down to just
> using one XML library.
In theory it shouldn't matter if more than one is linked, except for
some additional bloat. In reality, there are sometime symbol clashes,
especially when one is a forked version of the other...
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 12:51 ` Rich Felker
@ 2013-04-25 15:30 ` Jens Staal
2013-04-25 16:51 ` Zvi Gilboa
1 sibling, 0 replies; 76+ messages in thread
From: Jens Staal @ 2013-04-25 15:30 UTC (permalink / raw)
To: musl
On 2013-04-25 14:51, Rich Felker wrote:
>> >libraries. Someone was recently asking on the SDL list about SVG library
>> >options that are actively supported. The only one with active support
>> >that I could locate (besides librsvg2) was the one used by netsurf (
>> >http://www.netsurf-browser.org/projects/libsvgtiny/ ).
> How complete is it? SVG rendering is rather difficult if you want to
> support all the features, and if you don't, lots of files will come
> out looking wrong... In particular if you want to support text, you
> need a font and layout engine, and you may need clones of "standard"
> fonts that people use assuming you'll have them because they're
> included with Windows...
>
> BTW, SVG is XML-based, but I don't think a good SVG library should
> need a heavy XML library. Rather, it should take shortcuts to
> interpret just SVG content in a less-bloated way.
>
There is also an efl-related svg implementation under
http://code.google.com/p/enesim/wiki/GettingStarted
no idea how complete it is though.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 12:51 ` Rich Felker
2013-04-25 15:30 ` Jens Staal
@ 2013-04-25 16:51 ` Zvi Gilboa
2013-04-25 16:57 ` Justin Cormack
2013-04-26 6:11 ` Igmar Palsenberg
1 sibling, 2 replies; 76+ messages in thread
From: Zvi Gilboa @ 2013-04-25 16:51 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 2231 bytes --]
On 04/25/2013 08:51 AM, Rich Felker wrote:
> On Thu, Apr 25, 2013 at 07:44:39AM -0400, LM wrote:
>> incompatible licenses. The openssl library can't be used with a GNU
>> program unless there's a waiver for it because one of the clauses in the
>> openssl license goes against the GNU license principles. The gnutls
> Not _used_ but _distributed_. The GPL does not restrict use
> whatsoever (and takes the position that it legally can't do so) so
> it's fine to use OpenSSL with GPL programs as long as you don't
> distribute the resulting binary. This is of course a problem for
> package maintainers/distributions, and distributing both openssl and
> the GNU program and a script to link them together might even be seen
> as an infringing activity.
What about explicitly loading the library at run-time using uselib(2) in
a plug-in like fashion? Is that also considered problematic from a GNU
perspective?
>> pkg-config has a nice replacement in pkgconf. (If a list is
>> created, might be helpful to list possible replacements already out
>> there.) Would like to see some of the pieces that are essential parts of
>> the GNU build system/autotools replaced with some more efficient
>> [...]
>> of unnecessary work to port applications. Would rather see the current
>> build systems already used (autotools, cmake, etc.) streamlined or see drop
>> in replacements that are better designed.
> I seem to recall an effort somewhere to do exactly this, but I can't
> remember where I saw it..
Here too, one option would be to have all of a library's options,
dependencies, and build steps represented in a /real/ database (my
preference would be PostgreSql), and then have the actual build
script(s) generated using command-line utilities. This is something
that I have been doing in a rather miniature scale in a few
research-related projects, and found to be both effective and
expandable. Arguably, for all projects beyond a certain degree of
complexity, using a database to manage the build is the most feasible
path to follow (no need to name projects that demonstrate what happens
if you don't...), yet switching an existing build system to one that is
database-driven could become quite a challenge.
[-- Attachment #2: Type: text/html, Size: 3081 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 16:51 ` Zvi Gilboa
@ 2013-04-25 16:57 ` Justin Cormack
2013-04-25 17:53 ` Zvi Gilboa
2013-04-26 6:11 ` Igmar Palsenberg
1 sibling, 1 reply; 76+ messages in thread
From: Justin Cormack @ 2013-04-25 16:57 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 1161 bytes --]
On 25 Apr 2013 17:52, "Zvi Gilboa" <zg7s@eservices.virginia.edu> wrote:
>
> On 04/25/2013 08:51 AM, Rich Felker wrote:
>>
>> On Thu, Apr 25, 2013 at 07:44:39AM -0400, LM wrote:
>>>
>>> incompatible licenses. The openssl library can't be used with a GNU
>>> program unless there's a waiver for it because one of the clauses in the
>>> openssl license goes against the GNU license principles. The gnutls
>>
>> Not _used_ but _distributed_. The GPL does not restrict use
>> whatsoever (and takes the position that it legally can't do so) so
>> it's fine to use OpenSSL with GPL programs as long as you don't
>> distribute the resulting binary. This is of course a problem for
>> package maintainers/distributions, and distributing both openssl and
>> the GNU program and a script to link them together might even be seen
>> as an infringing activity.
>
>
> What about explicitly loading the library at run-time using uselib(2) in
a plug-in like fashion? Is that also considered problematic from a GNU
perspective?
There is some disagreement about this and it depends what you distribute.
See here http://lwn.net/SubscriberLink/548216/731f7ad0abe52f40/
Justin
[-- Attachment #2: Type: text/html, Size: 1594 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 16:57 ` Justin Cormack
@ 2013-04-25 17:53 ` Zvi Gilboa
2013-04-27 5:45 ` Rob Landley
0 siblings, 1 reply; 76+ messages in thread
From: Zvi Gilboa @ 2013-04-25 17:53 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 1991 bytes --]
On 04/25/2013 12:57 PM, Justin Cormack wrote:
>
>
> On 25 Apr 2013 17:52, "Zvi Gilboa" <zg7s@eservices.virginia.edu
> <mailto:zg7s@eservices.virginia.edu>> wrote:
> >
> > On 04/25/2013 08:51 AM, Rich Felker wrote:
> >>
> >> On Thu, Apr 25, 2013 at 07:44:39AM -0400, LM wrote:
> >>>
> >>> incompatible licenses. The openssl library can't be used with a GNU
> >>> program unless there's a waiver for it because one of the clauses
> in the
> >>> openssl license goes against the GNU license principles. The gnutls
> >>
> >> Not _used_ but _distributed_. The GPL does not restrict use
> >> whatsoever (and takes the position that it legally can't do so) so
> >> it's fine to use OpenSSL with GPL programs as long as you don't
> >> distribute the resulting binary. This is of course a problem for
> >> package maintainers/distributions, and distributing both openssl and
> >> the GNU program and a script to link them together might even be seen
> >> as an infringing activity.
> >
> >
> > What about explicitly loading the library at run-time using
> uselib(2) in a plug-in like fashion? Is that also considered
> problematic from a GNU perspective?
>
> There is some disagreement about this and it depends what you
> distribute. See here
> http://lwn.net/SubscriberLink/548216/731f7ad0abe52f40/
>
> Justin
>
Thank you for pointing to this excellent article. I initially
considered the plug-in-like case easier to assess -- specifically since
the loaded library will never be listed as one of the loading object's
dependencies -- yet with notions such as interdependency and
"collectivity," one can never be on safe ground when loading a GPL'ed
library from within a differently-licensed program, open-source or not.
Ironically, much of the current thread is about the need to create
alternatives to commonly-used GPL'ed libraries, which in itself reminds
of past (and present) efforts to create open source alternatives to
proprietary libraries and software products.
Zvi
[-- Attachment #2: Type: text/html, Size: 3100 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 17:53 ` Zvi Gilboa
@ 2013-04-27 5:45 ` Rob Landley
2013-04-27 8:13 ` Luca Barbato
2013-04-27 13:05 ` Zvi Gilboa
0 siblings, 2 replies; 76+ messages in thread
From: Rob Landley @ 2013-04-27 5:45 UTC (permalink / raw)
To: musl; +Cc: musl
On 04/25/2013 12:53:53 PM, Zvi Gilboa wrote:
> Ironically, much of the current thread is about the need to create
> alternatives to commonly-used GPL'ed libraries, which in itself
> reminds
> of past (and present) efforts to create open source alternatives to
> proprietary libraries and software products.
Unfortunately the FSF poisoned copyleft in 2006. The GPL was a category
killer synonymous with copyleft... until GPLv3 came out and there was
no longer any such thing as "The GPL".
Today Linux and Samba can't share code even though they implement two
ends of the same protocol. QEMU is caught between wanting Linux driver
code to implement devices and gdb/binutils code to implement processors
and it can't have both. Licensing code "GPLv2 or later" just makes it
worse: you can donate code to both but can't accept code from either
one.
Programmers are not lawyers, we're not _good_ at how licenses interact.
The GPL was a terminal node in a directed graph of license
convertibily, where all a programmer had to care was "is this
convertible to The GPL or not"? If it is, treat it as GPL, if not avoid
it. There was no interaction, there was just The GPL. The FSF destroyed
that, leaving a fragmented incompatible pool, and people's attempts to
_fix_ it with Affero GPL or GPL-Next or other viral licenses just
fragments it further.
Copyleft only worked with a category killer license creating one big
pool. With multiple incompatible copyleft licenses, copyleft _prevents_
code sharing because you can't re-use it or combine it in new projects.
In the absence of a universal receiver license, non-lawyer programmers
looking for someting simple and understandable are switching to
universal donor licenses. BSD/MIT or outright public domain.
This shows the GPL falling from 72% market share in 2009 to 58 in 2013:
https://lwn.net/Articles/547400/
Eben Moglen, author of GPLv2 and GPLv3 (Stallman is not a lawyer,
Moglen is) recently lamented the decline of copyleft but doesn't seem
to understand why:
https://lwn.net/Articles/547379/
As far as I can tell the FSF has alienated all the young programmers.
The most popular license on Github is not specifying a license at all,
taking the Napster approach of civil disobedience and waiting for the
intellectual property system to collapse. Ten years ago the GPL would
have appealed to them, but since GPLv3 shattered the ecosystem it does
not.
That's why Android's "no GPL in userspace" policy (if you add GPL code
to your userspace, you can't use the Android trademark advertising your
product) actually makes _sense_.
> Zvi
Rob
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-27 5:45 ` Rob Landley
@ 2013-04-27 8:13 ` Luca Barbato
2013-04-27 13:05 ` Zvi Gilboa
1 sibling, 0 replies; 76+ messages in thread
From: Luca Barbato @ 2013-04-27 8:13 UTC (permalink / raw)
To: musl
On 04/27/2013 07:45 AM, Rob Landley wrote:
> On 04/25/2013 12:53:53 PM, Zvi Gilboa wrote:
>> Ironically, much of the current thread is about the need to create
>> alternatives to commonly-used GPL'ed libraries, which in itself reminds
>> of past (and present) efforts to create open source alternatives to
>> proprietary libraries and software products.
>
> Unfortunately the FSF poisoned copyleft in 2006. The GPL was a category
> killer synonymous with copyleft... until GPLv3 came out and there was no
> longer any such thing as "The GPL".
For my personal usage I'm still quite fond of LGPL, (even for programs).
I do not see how to fix it beside picking a license I like and tracking
contributors in case I need to switch to another.
lu
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-27 5:45 ` Rob Landley
2013-04-27 8:13 ` Luca Barbato
@ 2013-04-27 13:05 ` Zvi Gilboa
1 sibling, 0 replies; 76+ messages in thread
From: Zvi Gilboa @ 2013-04-27 13:05 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 3130 bytes --]
On 04/27/2013 01:45 AM, Rob Landley wrote:
> On 04/25/2013 12:53:53 PM, Zvi Gilboa wrote:
>> Ironically, much of the current thread is about the need to create
>> alternatives to commonly-used GPL'ed libraries, which in itself reminds
>> of past (and present) efforts to create open source alternatives to
>> proprietary libraries and software products.
>
> Unfortunately the FSF poisoned copyleft in 2006. The GPL was a
> category killer synonymous with copyleft... until GPLv3 came out and
> there was no longer any such thing as "The GPL".
>
> Today Linux and Samba can't share code even though they implement two
> ends of the same protocol. QEMU is caught between wanting Linux driver
> code to implement devices and gdb/binutils code to implement
> processors and it can't have both. Licensing code "GPLv2 or later"
> just makes it worse: you can donate code to both but can't accept code
> from either one.
>
> Programmers are not lawyers, we're not _good_ at how licenses
> interact. The GPL was a terminal node in a directed graph of license
> convertibily, where all a programmer had to care was "is this
> convertible to The GPL or not"? If it is, treat it as GPL, if not
> avoid it. There was no interaction, there was just The GPL. The FSF
> destroyed that, leaving a fragmented incompatible pool, and people's
> attempts to _fix_ it with Affero GPL or GPL-Next or other viral
> licenses just fragments it further.
>
> Copyleft only worked with a category killer license creating one big
> pool. With multiple incompatible copyleft licenses, copyleft
> _prevents_ code sharing because you can't re-use it or combine it in
> new projects.
... which apparently is the source of widespread confusion. To
illustrate this, go to google.com (with auto-complete enabled), and type
/including bsd/...
> In the absence of a universal receiver license, non-lawyer programmers
> looking for someting simple and understandable are switching to
> universal donor licenses. BSD/MIT or outright public domain.
Another balanced license is the LaTeX Project Public License (LPPL),
which also effectively governs cases in which a copyright holder has
abandoned his/her project, or can no longer be reached.
>
> This shows the GPL falling from 72% market share in 2009 to 58 in 2013:
>
> https://lwn.net/Articles/547400/
>
> Eben Moglen, author of GPLv2 and GPLv3 (Stallman is not a lawyer,
> Moglen is) recently lamented the decline of copyleft but doesn't seem
> to understand why:
>
> https://lwn.net/Articles/547379/
>
> As far as I can tell the FSF has alienated all the young programmers.
> The most popular license on Github is not specifying a license at all,
> taking the Napster approach of civil disobedience and waiting for the
> intellectual property system to collapse. Ten years ago the GPL would
> have appealed to them, but since GPLv3 shattered the ecosystem it does
> not.
>
> That's why Android's "no GPL in userspace" policy (if you add GPL code
> to your userspace, you can't use the Android trademark advertising
> your product) actually makes _sense_.
>
>> Zvi
>
> Rob
[-- Attachment #2: Type: text/html, Size: 4458 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 16:51 ` Zvi Gilboa
2013-04-25 16:57 ` Justin Cormack
@ 2013-04-26 6:11 ` Igmar Palsenberg
1 sibling, 0 replies; 76+ messages in thread
From: Igmar Palsenberg @ 2013-04-26 6:11 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 2774 bytes --]
>>> incompatible licenses. The openssl library can't be used with a GNU
>>> program unless there's a waiver for it because one of the clauses in the
>>> openssl license goes against the GNU license principles. The gnutls
>> Not _used_ but _distributed_. The GPL does not restrict use
>> whatsoever (and takes the position that it legally can't do so) so
>> it's fine to use OpenSSL with GPL programs as long as you don't
>> distribute the resulting binary. This is of course a problem for
>> package maintainers/distributions, and distributing both openssl and
>> the GNU program and a script to link them together might even be seen
>> as an infringing activity.
>
> What about explicitly loading the library at run-time using uselib(2) in a plug-in like fashion? Is that also considered problematic from a GNU perspective?
I consider this a grey area. I personally don't thing it is considered a problem, but there are a number of interesting (theoretical) scenario's :
- You have a a.so that dynamically links GPL library b.so. Program C loads a.so, and uses it's API, does it need to be GPL ? a.so can alternatively also load d.so instead of b.so, which is BSD licensed.
Using / linking a GPL lib, and making the whole work as such GPL is a clear violation. I personally avoid GPL libs where I can.
>>> pkg-config has a nice replacement in pkgconf. (If a list is
>>> created, might be helpful to list possible replacements already out
>>> there.) Would like to see some of the pieces that are essential parts of
>>> the GNU build system/autotools replaced with some more efficient
>>> [...]
>>> of unnecessary work to port applications. Would rather see the current
>>> build systems already used (autotools, cmake, etc.) streamlined or see drop
>>> in replacements that are better designed.
>> I seem to recall an effort somewhere to do exactly this, but I can't
>> remember where I saw it..
>
> Here too, one option would be to have all of a library's options, dependencies, and build steps represented in a real database (my preference would be PostgreSql), and then have the actual build script(s) generated using command-line utilities. This is something that I have been doing in a rather miniature scale in a few research-related projects, and found to be both effective and expandable. Arguably, for all projects beyond a certain degree of complexity, using a database to manage the build is the most feasible path to follow (no need to name projects that demonstrate what happens if you don't...), yet switching an existing build system to one that is database-driven could become quite a challenge.
I've done nice stuff with SQLIte. Make it also easy to transfer stuff to a different system.
Igmar
[-- Attachment #2: Type: text/html, Size: 3765 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 4:15 Rich Felker
` (4 preceding siblings ...)
2013-04-25 11:44 ` LM
@ 2013-04-26 4:19 ` Isaac Dunham
2013-04-26 11:41 ` LM
2013-04-26 4:32 ` nwmcsween
6 siblings, 1 reply; 76+ messages in thread
From: Isaac Dunham @ 2013-04-26 4:19 UTC (permalink / raw)
To: musl
On Thu, 25 Apr 2013 00:15:53 -0400
Rich Felker <dalias@aerifal.cx> wrote:
> 3. Image format and compression (libpng, zlib, etc.). The existing
> libraries are full of atrociously bad error-handling practices, and
> quite bloated. These libraries should be well-factored so that
> programs that just want to read (the majority of use cases) don't have
> to pull in code for writing. Unfortunately the interfaces might not be
> so simple, since an important usage case is progressive rendering of
> partially-loaded images, both for the purpose of loading over slow
> network connections and for rendering thumbnails quickly.
Have you perchance seen libpnglite?
http://sourceforge.net/projects/pnglite/files/
Yes, I know it's NOT well-factored; the author wanted something where
a single file and header could be added to a project and there's PNG support.
Uses the png_ namespace.
AFAICT, it accepts either a buffer to use for the png or a null pointer, in which case it malloc's what is necessary.
License is BSD-ish.
It includes a reference to "zlite.h", which seems to be a zip library with a z_ namespace.
Also, I ran across stb_image_write.h, but...I hesitate to _recommend_ something where the header contains all the function source. stb_image.c is another "small, but I don't think that's good enough" thing...
> All of these libraries should:
>
> - Avoid namespace pollution. Only external symbols should be the
> public API and internal-use stuff prefixed with an ugly prefix
> that's extremely unlikely to clash with anything.
> - Avoid unnecessary allocation. Use caller-provided objects where
> possible or provide both options.
> - Have absolutely zero global state.
>
> That's all I can think of at the moment but I'm sure there are other
> needs I've come across and forgotten. Please feel free to supplement
> this list.
A links that may be helpful, perhaps...
http://bashismal.blogspot.com/2011/10/unbloated-resources-in-c.html
--
Isaac Dunham <idunham@lavabit.com>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: High-priority library replacements?
2013-04-26 4:19 ` Isaac Dunham
@ 2013-04-26 11:41 ` LM
2013-04-26 12:57 ` Muhammad Sumyandityo Noor
0 siblings, 1 reply; 76+ messages in thread
From: LM @ 2013-04-26 11:41 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 1127 bytes --]
On Fri, Apr 26, 2013 at 12:19 AM, Isaac Dunham <idunham@lavabit.com> wrote:
> A links that may be helpful, perhaps...
> http://bashismal.blogspot.com/2011/10/unbloated-resources-in-c.html
>
>
Nice resource. Thanks for sharing it.
If anyone's interested in the OpenGL replacement mentioned on that page,
there are 2 other candidates similar to tinyGL. There are picogl and
sdl-tinygl. The Syllable forum (
http://forum.syllable.org/viewforum.php?f=1 ) has some interesting
information about all three of these and makes some comparisons. If I
remember, one of them has a subset of the glut functionality included. I
believe some porting is typically needed to use any of them as a
replacement for OpenGL. Speaking of glut, freeglut (
http://freeglut.sourceforge.net/ ) seems more portable and up-to-date than
glut.
Another interesting resource is picobsd (
http://code.google.com/p/freebsd-head/source/browse/release/?r=bbfa6f219c41b6850ef0e7699f439ad5488435ae#release%2Fpicobsd).
Has some small utility programs. I don't think all of them port to
other operating systems, but some of it does work.
Sincerely,
Laura
[-- Attachment #2: Type: text/html, Size: 1785 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-26 11:41 ` LM
@ 2013-04-26 12:57 ` Muhammad Sumyandityo Noor
2013-04-26 15:53 ` Rich Felker
2013-04-26 16:52 ` LM
0 siblings, 2 replies; 76+ messages in thread
From: Muhammad Sumyandityo Noor @ 2013-04-26 12:57 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 1431 bytes --]
You meant replacement for Mesa? Because TinyGL is software renderer.
It's unlikely people will utilize software renderer. As for embedded
system, each SoCs provides their own userland to utilize its hardware
accelerator.
LM wrote:
>
> On Fri, Apr 26, 2013 at 12:19 AM, Isaac Dunham <idunham@lavabit.com
> <mailto:idunham@lavabit.com>> wrote:
>
> A links that may be helpful, perhaps...
> http://bashismal.blogspot.com/2011/10/unbloated-resources-in-c.html
>
>
> Nice resource. Thanks for sharing it.
>
> If anyone's interested in the OpenGL replacement mentioned on that
> page, there are 2 other candidates similar to tinyGL. There are
> picogl and sdl-tinygl. The Syllable forum (
> http://forum.syllable.org/viewforum.php?f=1 ) has some interesting
> information about all three of these and makes some comparisons. If I
> remember, one of them has a subset of the glut functionality
> included. I believe some porting is typically needed to use any of
> them as a replacement for OpenGL. Speaking of glut, freeglut (
> http://freeglut.sourceforge.net/ ) seems more portable and up-to-date
> than glut.
>
> Another interesting resource is picobsd (
> http://code.google.com/p/freebsd-head/source/browse/release/?r=bbfa6f219c41b6850ef0e7699f439ad5488435ae#release%2Fpicobsd
>
> ). Has some small utility programs. I don't think all of them port
> to other operating systems, but some of it does work.
>
> Sincerely,
> Laura
[-- Attachment #2: Type: text/html, Size: 1742 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-26 12:57 ` Muhammad Sumyandityo Noor
@ 2013-04-26 15:53 ` Rich Felker
2013-04-28 6:53 ` Muhammad Sumyandityo Noor
2013-04-26 16:52 ` LM
1 sibling, 1 reply; 76+ messages in thread
From: Rich Felker @ 2013-04-26 15:53 UTC (permalink / raw)
To: musl
On Fri, Apr 26, 2013 at 07:57:36PM +0700, Muhammad Sumyandityo Noor wrote:
> You meant replacement for Mesa? Because TinyGL is software renderer.
> It's unlikely people will utilize software renderer. As for embedded
> system, each SoCs provides their own userland to utilize its
> hardware accelerator.
I don't really understand the GL architecture presently in use well
enough to know the right solutions, but if I'm not mistaken, it
involves loading dynamic modules, possibly even binaryware
hardware-vendor-provided ones, into the address space of your
application. This seems like a recipe for disaster.
Software-rendered GL is definitely not the solution, but I wonder if
it would be reasonable to create a library that provides the OpenGL
API with NO namespace pollution or introduction of dangerous code into
the application's address space, by cooperating with a separate
process via shared memory. On modern Linux, this separate process
could use Linux namespaces to completely throw away all privileges
except to the graphics device. Then the nasty legacy OpenGL code could
run in a sandbox.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-26 15:53 ` Rich Felker
@ 2013-04-28 6:53 ` Muhammad Sumyandityo Noor
2013-04-28 17:46 ` Rich Felker
0 siblings, 1 reply; 76+ messages in thread
From: Muhammad Sumyandityo Noor @ 2013-04-28 6:53 UTC (permalink / raw)
To: musl
Rich Felker wrote:
> On Fri, Apr 26, 2013 at 07:57:36PM +0700, Muhammad Sumyandityo Noor wrote:
>> You meant replacement for Mesa? Because TinyGL is software renderer.
>> It's unlikely people will utilize software renderer. As for embedded
>> system, each SoCs provides their own userland to utilize its
>> hardware accelerator.
>
> I don't really understand the GL architecture presently in use well
> enough to know the right solutions, but if I'm not mistaken, it
> involves loading dynamic modules, possibly even binaryware
> hardware-vendor-provided ones, into the address space of your
> application. This seems like a recipe for disaster.
OpenGL support varies from platforms to platforms. On Windows, for
example, M$ will use software renderer via OpenGL32.dll if the ICD
(Installable Client Driver) is not present. If the driver is installed
(e.g. from nVidia or ATI), the call to OpenGL32.dll will be redirected
to the ICD libraries automatically.
On Linux, especially on embedded space. The SoC manufacturer usually
ships open source kernel driver to create a device nodes and communicate
with it. I'll take PowerVR SGX by imagination as an example. They have
open source driver to create /dev/pvrsvrkm device node. The bulk of
graphics operation however, lies in userland (non-open source) library
(libEGL.so and libGLES**.so). This library communicates with the device
nodes by uploading some (another proprietary) 'firmware' to the GPU, and
then communicates with device nodes via ioctl with protocol that is
proprietary too.
So, as you have said it is a big mess.
(http://www.phoronix.com/scan.php?page=news_item&px=ODk0NA) the article
published in 2010 and still true even for today.
>
> Software-rendered GL is definitely not the solution, but I wonder if
> it would be reasonable to create a library that provides the OpenGL
> API with NO namespace pollution or introduction of dangerous code into
> the application's address space, by cooperating with a separate
> process via shared memory. On modern Linux, this separate process
> could use Linux namespaces to completely throw away all privileges
> except to the graphics device. Then the nasty legacy OpenGL code could
> run in a sandbox.
Pardon for my limited understanding of this matter. But if you mean
replacing manufacturer-provided libEGL.so and libGLES**.so, then I don't
think it's possible, as many parts of it are hidden. And to separate
that to another process, I don't think OpenGL Context can be shared to
another process or created without referencing to libEGL.
> Rich
I'm very new in linux system programming. It's based of my (very)
limited knowledge of how OpenGL ES works in embedded device.
Thanks,
Muhammad Sumyandityo Noor (Didiet)
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-28 6:53 ` Muhammad Sumyandityo Noor
@ 2013-04-28 17:46 ` Rich Felker
0 siblings, 0 replies; 76+ messages in thread
From: Rich Felker @ 2013-04-28 17:46 UTC (permalink / raw)
To: musl
On Sun, Apr 28, 2013 at 02:53:44PM +0800, Muhammad Sumyandityo Noor wrote:
> >Software-rendered GL is definitely not the solution, but I wonder if
> >it would be reasonable to create a library that provides the OpenGL
> >API with NO namespace pollution or introduction of dangerous code into
> >the application's address space, by cooperating with a separate
> >process via shared memory. On modern Linux, this separate process
> >could use Linux namespaces to completely throw away all privileges
> >except to the graphics device. Then the nasty legacy OpenGL code could
> >run in a sandbox.
>
> Pardon for my limited understanding of this matter. But if you mean
> replacing manufacturer-provided libEGL.so and libGLES**.so, then I
> don't think it's possible, as many parts of it are hidden. And to
Not replacing, wrapping. With the wrapped code being in another
sandboxed process.
> separate that to another process, I don't think OpenGL Context can
> be shared to another process or created without referencing to
> libEGL.
Sure it can. This is always fundamentally possible.
Rich
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-26 12:57 ` Muhammad Sumyandityo Noor
2013-04-26 15:53 ` Rich Felker
@ 2013-04-26 16:52 ` LM
1 sibling, 0 replies; 76+ messages in thread
From: LM @ 2013-04-26 16:52 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 897 bytes --]
On Fri, Apr 26, 2013 at 8:57 AM, Muhammad Sumyandityo Noor <
lynxluna@gmail.com> wrote:
> You meant replacement for Mesa? Because TinyGL is software renderer. It's
> unlikely people will utilize software renderer. As for embedded system,
> each SoCs provides their own userland to utilize its hardware accelerator.
>
> TinyGL and the other two libraries mentioned can be used as a replacement
for Mesa. Typically, Windows developers will work with the Windows
implementation of OpenGL not Mesa and TinyGL can be used as a replacement
for that as well. I'm not saying these libraries provide hardware
acceleration like OpenGL. They just try to provide the major parts of the
API. Syllable Desktop does not have access to working OpenGL or Mesa
libraries, so they've been using these to at least allow some programs
written for the OpenGL API to port, render and run on their operating
system.
[-- Attachment #2: Type: text/html, Size: 1177 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: High-priority library replacements?
2013-04-25 4:15 Rich Felker
` (5 preceding siblings ...)
2013-04-26 4:19 ` Isaac Dunham
@ 2013-04-26 4:32 ` nwmcsween
6 siblings, 0 replies; 76+ messages in thread
From: nwmcsween @ 2013-04-26 4:32 UTC (permalink / raw)
To: musl
[-- Attachment #1: Type: text/plain, Size: 89 bytes --]
Thoughts on https://github.com/jedisct1/libsodium ? I haven't dug too much into the code.
[-- Attachment #2: Type: text/html, Size: 268 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread