High-priority library replacements?

High-priority library replacements?

[Rich Felker]

Hi all,

The recent thread "Best place to discuss other lightweight libraries"
had me thinking we should really put together a list of high-priority
library replacements that need to be done. I don't think I'll be the
one to do them, but I wouldn't mind overseeing/mentoring someone
working on them, or just getting somebody else interested in these
projects.

A few initial ideas:

1. A light, C, UTF-8-only Unicode library. The most important things
it should implement are things needed by any application that presents
text to the user, specifically line-breaking (UAX#14), bidi (UAX#9),
identifying grapheme clusters, etc. Things like case- and
normalization-insensitive comparison, application of Unicode-format
collation rules, etc. would also possibly be useful.

2. SSL. The modern internet basically requires using SSL everywhere.
We should be aiming/expecting to transition to a world where even
non-login-based sites are 100% https; the threats of malicious http
injection attacks from rogue or advertising-based access points has
gotten too great. Unfortunately, all of the existing SSL
implementations are bloated, buggy, and fail even the most basic
robustness requirements. A good solution would be based on tomcrypt
and would expose a minimal, simple API suited for event-loop-based or
threaded use. It may also be useful to have an optional wrapper layer
to expose an API that mimics openssl or gnutls. It should also be able
to keep up with the changing demands of how to determine which
certificate authorities are to be trusted.

3. Image format and compression (libpng, zlib, etc.). The existing
libraries are full of atrociously bad error-handling practices, and
quite bloated. These libraries should be well-factored so that
programs that just want to read (the majority of use cases) don't have
to pull in code for writing. Unfortunately the interfaces might not be
so simple, since an important usage case is progressive rendering of
partially-loaded images, both for the purpose of loading over slow
network connections and for rendering thumbnails quickly.

All of these libraries should:

- Avoid namespace pollution. Only external symbols should be the
  public API and internal-use stuff prefixed with an ugly prefix
  that's extremely unlikely to clash with anything.
- Avoid unnecessary allocation. Use caller-provided objects where
  possible or provide both options.
- Have absolutely zero global state.

That's all I can think of at the moment but I'm sure there are other
needs I've come across and forgotten. Please feel free to supplement
this list.

Rich

Re: High-priority library replacements?

[Daniel Cegiełka]

2013/4/25 Rich Felker <dalias@aerifal.cx>:

For a list of core libraries I would add basic, but high-priority
tools: ssh, pam (passwd, login, su).

ssh - dropbear?
pam - openpam?

These key software we can also support (static linking etc.).

btw. SSL - instead libcrypto clone maybe it's better to use
crypto-algorithms from linux kernel?

http://cryptodev-linux.org/

Daniel

Re: High-priority library replacements?

[Rich Felker]

On Thu, Apr 25, 2013 at 07:05:12AM +0200, Daniel Cegiełka wrote:
> 2013/4/25 Rich Felker <dalias@aerifal.cx>:
> 
> For a list of core libraries I would add basic, but high-priority
> tools: ssh, pam (passwd, login, su).
> 
> ssh - dropbear?

I think dropbear fully covers the needs of most non-"enterprise" usage
and maybe that too. It could however use some hardening. I don't think
it's terribly insecure, but I'd like to see a robust privilege model
that would make it safe even in the event of bugs that would otherwise
result in a compromise.

> pam - openpam?

I would say pam is less critical. I've had my pamlite in limbo for a
long time but haven't gotten around to making it do anything... Still
a good one for the list though.

> These key software we can also support (static linking etc.).
> 
> btw. SSL - instead libcrypto clone maybe it's better to use
> crypto-algorithms from linux kernel?
> 
> http://cryptodev-linux.org/

I'm not sure what the advantage would be; the disadvantage is
certainly being Linux-specific and dependent on the host system
configuration (last I checked, crypto in the kernel is optional; maybe
this has changed..?) to work. There's also the issue that it's not
fail-proof; it requires allocating resources. IMO supporting hardware
crypto devices is not really relevant for most users of SSL. Yes, a
high volume web server might need to be tuned for performance, but it
doesn't matter for most network client applications like wget, chat
clients, mail clients, etc.

Rich

Re: High-priority library replacements?

[Kurt H Maier]

On Thu, Apr 25, 2013 at 12:15:53AM -0400, Rich Felker wrote:
> 
> 1. A light, C, UTF-8-only Unicode library.

The only one I know of that might not already be common knowledge is
Connor Lane Smith's libutf: https://github.com/cls/libutf

khm

Re: High-priority library replacements?

[Gregor Pintar]

Hello.

2013/4/25, Rich Felker <dalias@aerifal.cx>:
> 2. SSL. The modern internet basically requires using SSL everywhere.
> We should be aiming/expecting to transition to a world where even
> non-login-based sites are 100% https; the threats of malicious http
> injection attacks from rogue or advertising-based access points has
> gotten too great. Unfortunately, all of the existing SSL
> implementations are bloated, buggy, and fail even the most basic
> robustness requirements. A good solution would be based on tomcrypt
> and would expose a minimal, simple API suited for event-loop-based or
> threaded use. It may also be useful to have an optional wrapper layer
> to expose an API that mimics openssl or gnutls. It should also be able
> to keep up with the changing demands of how to determine which
> certificate authorities are to be trusted.
I am working on cryptographic library. It's far from being finished.
I would be very glad, if someone could look at it.
Currently I have problems with API design so help would be welcome.
It isn't in git yet.
You can get it here:
https://dl.dropboxusercontent.com/u/83450675/kripto/kripto.tar.gz
I have plans for SSL library on top of it, but it could take years.

I think best way is not to trust any certificate authority.
Maybe some certificate p2p protocol could be done?

> All of these libraries should:
>
> - Avoid namespace pollution. Only external symbols should be the
>   public API and internal-use stuff prefixed with an ugly prefix
>   that's extremely unlikely to clash with anything.
All external symbols have "kripto_" prefix.

> - Avoid unnecessary allocation. Use caller-provided objects where
>   possible or provide both options.
I am trying to do least malloc()s possible.

> - Have absolutely zero global state.
There is no global state and there won't be any.

Gregor

Re: High-priority library replacements?

[Hal Clark]

On 4/24/13, Rich Felker <dalias@aerifal.cx> wrote:
> Hi all,
>
> The recent thread "Best place to discuss other lightweight libraries"
> had me thinking we should really put together a list of high-priority
> library replacements that need to be done. I don't think I'll be the
> one to do them, but I wouldn't mind overseeing/mentoring someone
> working on them, or just getting somebody else interested in these
> projects.
>
> A few initial ideas:
>
> 1. A light, C, UTF-8-only Unicode library. The most important things
> it should implement are things needed by any application that presents
> text to the user, specifically line-breaking (UAX#14), bidi (UAX#9),
> identifying grapheme clusters, etc. Things like case- and
> normalization-insensitive comparison, application of Unicode-format
> collation rules, etc. would also possibly be useful.
>
> 2. SSL. The modern internet basically requires using SSL everywhere.
> We should be aiming/expecting to transition to a world where even
> non-login-based sites are 100% https; the threats of malicious http
> injection attacks from rogue or advertising-based access points has
> gotten too great. Unfortunately, all of the existing SSL
> implementations are bloated, buggy, and fail even the most basic
> robustness requirements. A good solution would be based on tomcrypt
> and would expose a minimal, simple API suited for event-loop-based or
> threaded use. It may also be useful to have an optional wrapper layer
> to expose an API that mimics openssl or gnutls. It should also be able
> to keep up with the changing demands of how to determine which
> certificate authorities are to be trusted.
>

I saw earlier that Cyassl was briefly mentioned. Now, in general I am
interested in your idea of a "minimal, simple API", but I've had great
experiences with Cyassl. IMO it is exactly what you're looking for.

Could you comment on what parts of it are unsuitable for your intended
purposes? I'm curious what would need to be changed or replaced.

(I'm not affiliated with Cyassl - I have just had (really) good
experiences with it after trying the GnuTLS/OpenSSL route.)


> 3. Image format and compression (libpng, zlib, etc.). The existing
> libraries are full of atrociously bad error-handling practices, and
> quite bloated. These libraries should be well-factored so that
> programs that just want to read (the majority of use cases) don't have
> to pull in code for writing. Unfortunately the interfaces might not be
> so simple, since an important usage case is progressive rendering of
> partially-loaded images, both for the purpose of loading over slow
> network connections and for rendering thumbnails quickly.
>
> All of these libraries should:
>
> - Avoid namespace pollution. Only external symbols should be the
>   public API and internal-use stuff prefixed with an ugly prefix
>   that's extremely unlikely to clash with anything.
> - Avoid unnecessary allocation. Use caller-provided objects where
>   possible or provide both options.
> - Have absolutely zero global state.
>
> That's all I can think of at the moment but I'm sure there are other
> needs I've come across and forgotten. Please feel free to supplement
> this list.
>
> Rich
>


-hal clark

Re: High-priority library replacements?

[Jens Staal]

On 2013-04-25 07:55, Kurt H Maier wrote:
> On Thu, Apr 25, 2013 at 12:15:53AM -0400, Rich Felker wrote:
>>
>> 1. A light, C, UTF-8-only Unicode library.
>
> The only one I know of that might not already be common knowledge is
> Connor Lane Smith's libutf: https://github.com/cls/libutf
>
> khm
>


Does it work as a drop-in replacement for unixy utf/wchar 
implementations or is it the Plan9 variant (reference to Runes implies 
the later)? I found myself needing at first libwchar from heirloom and 
later the (old) GNU libutf8 under APE to provide some wchar-related 
functions expected by various applications.

It felt really silly to "devolve" like that.

Re: High-priority library replacements?

[Igmar Palsenberg]

> I saw earlier that Cyassl was briefly mentioned. Now, in general I am
> interested in your idea of a "minimal, simple API", but I've had great
> experiences with Cyassl. IMO it is exactly what you're looking for.
> 
> Could you comment on what parts of it are unsuitable for your intended
> purposes? I'm curious what would need to be changed or replaced.
> 
> (I'm not affiliated with Cyassl - I have just had (really) good
> experiences with it after trying the GnuTLS/OpenSSL route.)

It's also GPL. I consider that a huge disadvantage, and also makes inclusion in Musl a no-go.



	Igmar

Re: High-priority library replacements?

[LM]

[-- Attachment #1: Type: text/plain, Size: 5668 bytes --]

On Thu, Apr 25, 2013 at 12:15 AM, Rich Felker <dalias@aerifal.cx> wrote:

> The recent thread "Best place to discuss other lightweight libraries"
> had me thinking we should really put together a list of high-priority
> library replacements that need to be done.
>

Sounds like a great idea.


> 1. A light, C, UTF-8-only Unicode library. The most important things
> it should implement are things needed by any application that presents
> text to the user, specifically line-breaking (UAX#14), bidi (UAX#9),
> identifying grapheme clusters, etc. Things like case- and
> normalization-insensitive comparison, application of Unicode-format
> collation rules, etc. would also possibly be useful.
>

I have something I'm working on that I'm using for my sdcv port.  It mainly
covers things like switching from utf8 to uint32_t and back and utf8
equivalents of isspace, isdigit, toupper, tolower, etc.  I really like the
Plan 9 rune implementation.  I didn't need all the functionality of the
Plan 9 library, so I'm creating a small library with just the minimal
functions I need.  If you check Ohloh, I've also seen some examples of
internationalized versions of standard functions like isdigit, isspace
there.


>
> 2. SSL.

...

> Unfortunately, all of the existing SSL
> implementations are bloated, buggy, and fail even the most basic
> robustness requirements. A good solution would be based on tomcrypt
> and would expose a minimal, simple API suited for event-loop-based or
> threaded use. It may also be useful to have an optional wrapper layer
> to expose an API that mimics openssl or gnutls.


I think an openssl wrapper and possibly a gnutls one would be essential if
you want to have it work with all the Open Source programs already out
there.

There are three main options I currently know about: openssl, gnutls, and
nss.  Each has a different license.  So depending on how a program is
licensed, you may not legally be able to use some of these libraries with
the program.  Any GNU licensed program requires an optional waiver to link
with openssl.  You find many Open Source programs provide the optional
waiver.  One really good example of the license issue concerns lynx.  You
can check their mailing list for a detailed discussion.  The highlights are
that lynx is GPLv2 and the project can't contact all the authors who worked
on it to change the license or get a waiver for linking to other types of
incompatible licenses.  The openssl library can't be used with a GNU
program unless there's a waiver for it because one of the clauses in the
openssl license goes against the GNU license principles.  The gnutls
openssl compatibility wrapper is GPL v3, so it can't be used unless the
program is relicensed at GPL v3 (
http://www.gnu.org/licenses/gpl-faq.html#AllCompatibility ).  That just
leaves nss licensed under MPL, LGPL and/or GPL.  If it's LGPL, lynx can use
it.

Moral of the story, if anything's going to be done to replace these
libraries, it would have to be under appropriate license(s) or one would
not be able to use the results with existing Open Source software already
out there without violating copyright.

 3. Image format and compression (libpng, zlib, etc.).
>

A good SVG library is needed.  To get SVG into Tuxpaint, it added libcroco,
librsvg2, libcairo and other GTK+ library dependencies.  Tuxpaint is based
on SDL, so would prefer not to see all the added code for two GUI
libraries.  Someone was recently asking on the SDL list about SVG library
options that are actively supported.  The only one with active  support
that I could locate (besides librsvg2) was the one used by netsurf (
http://www.netsurf-browser.org/projects/libsvgtiny/ ).

If you're looking at graphics formats.  You might as well add audio formats
to the list too.  There are a wide variety of libraries for the different
audio formats and they range in quality (and license types).


> That's all I can think of at the moment but I'm sure there are other
> needs I've come across and forgotten. Please feel free to supplement
> this list.
>
>
As already mentioned, libintl is on my list of things to replace if
possible.  pkg-config has a nice replacement in pkgconf.  (If a list is
created, might be helpful to list possible replacements already out
there.)  Would like to see some of the pieces that are essential parts of
the GNU build system/autotools replaced with some more efficient
alternatives.  It's easier to keep using the GNU autotools system than to
completely replace it with a new design.  Replacing it with another system
completely would mean switching over any program or library one wants to
build from the many Open Source programs already out there that currently
use it.  The CoApp project is attempting to do that on Windows (switch
other build systems over to the one used by Visual C++).  Seems like a lot
of unnecessary work to port applications.  Would rather see the current
build systems already used (autotools, cmake, etc.) streamlined or see drop
in replacements that are better designed.

One other area that can be troublesome is when there are multiple libraries
with similar functionality and some of the programs you want to build use
one while the rest use another.  The openssl/gnutls/nss example is an
issue.  It's typically solved by supplying wrapper interfaces.  The case I
keep running into at the moment is libxml2/expat.  libxml2 is newer and
works in conjunction with libraries like libxslt.  Some Open Source
programs let you pick one or the other.  Unfortunately, there are some that
insist on only one.  Makes it hard to simplify one's system down to just
using one XML library.

Sincerely,
Laura

[-- Attachment #2: Type: text/html, Size: 6927 bytes --]

Re: High-priority library replacements?

[Rich Felker]

On Thu, Apr 25, 2013 at 09:34:35AM +0200, Jens Staal wrote:
> On 2013-04-25 07:55, Kurt H Maier wrote:
> >On Thu, Apr 25, 2013 at 12:15:53AM -0400, Rich Felker wrote:
> >>
> >>1. A light, C, UTF-8-only Unicode library.
> >
> >The only one I know of that might not already be common knowledge is
> >Connor Lane Smith's libutf: https://github.com/cls/libutf
> 
> Does it work as a drop-in replacement for unixy utf/wchar
> implementations

This is exactly what libutf does. It's not a Unicode library in the
form I described; it's just a workaround for broken systems lacking
UTF-8 locales, for which it replaces all the important functions
(mb*towc and wc*tomb, etc.) with versions that convert UTF-8. I
believe it also replaces the is* and to* functions and maybe a few
others. But it doesn't help with any Unicode processing needs the
application might have outside the scope of C or POSIX, and it's not
meant to be used on systems that already have proper UTF-8 support at
the system library level.

Rich

Re: High-priority library replacements?

[Rich Felker]

On Thu, Apr 25, 2013 at 12:21:12AM -0700, Hal Clark wrote:
> I saw earlier that Cyassl was briefly mentioned. Now, in general I am
> interested in your idea of a "minimal, simple API", but I've had great
> experiences with Cyassl. IMO it is exactly what you're looking for.

Minimal, simple API basically means I wouldn't want to see anything
like the mess of OpenSSL's "BIO" layer. If I didn't care for making
users of the library happy, I would say it should just have one
function, an open function, that would hand over the socket to the
library and give you a new socket (from socketpair) to use in its
place as an unencrypted socket. But then everybody would whine about
threads. So I think there need to be some additional interfaces, but
not more than a few.

> Could you comment on what parts of it are unsuitable for your intended
> purposes? I'm curious what would need to be changed or replaced.

So far, based on what I've seen/read, I find it rather suitable. Lots
of people find the license (GPL) unacceptable however.

Rich

Re: High-priority library replacements?

[Rich Felker]

On Thu, Apr 25, 2013 at 07:44:39AM -0400, LM wrote:
> I have something I'm working on that I'm using for my sdcv port.  It mainly
> covers things like switching from utf8 to uint32_t and back and utf8
> equivalents of isspace, isdigit, toupper, tolower, etc.  I really like the
> Plan 9 rune implementation.  I didn't need all the functionality of the
> Plan 9 library, so I'm creating a small library with just the minimal
> functions I need.  If you check Ohloh, I've also seen some examples of
> internationalized versions of standard functions like isdigit, isspace
> there.

This is somewhat different from what I was thinking of. You're looking
more at replacing the standard C api functions for mb/wc stuff so you
can work with UTF-8 regardless of the locale encoding. I was talking
about importane Unicode functionality outside what the C API can
provide.

> I think an openssl wrapper and possibly a gnutls one would be essential if
> you want to have it work with all the Open Source programs already out
> there.

Agreed. The big question, I guess, is whether the aim would be to have
big existing programs adopt it, or to provide just a new option for
lighter programs. But since I think we'd want to be able to use many
existing programs without dealing with the trouble of openssl and
gnutls, I would lean towards the former as a goal.

> incompatible licenses.  The openssl library can't be used with a GNU
> program unless there's a waiver for it because one of the clauses in the
> openssl license goes against the GNU license principles.  The gnutls

Not _used_ but _distributed_. The GPL does not restrict use
whatsoever (and takes the position that it legally can't do so) so
it's fine to use OpenSSL with GPL programs as long as you don't
distribute the resulting binary. This is of course a problem for
package maintainers/distributions, and distributing both openssl and
the GNU program and a script to link them together might even be seen
as an infringing activity.

> Moral of the story, if anything's going to be done to replace these
> libraries, it would have to be under appropriate license(s) or one would
> not be able to use the results with existing Open Source software already
> out there without violating copyright.

Agreed. I should have mentioned that all of these libraries should aim
for permissive license (MIT/BSD) or at the very least, LGPL 2.1+,
which while not as permissive, is at least compatible with all
relevant foss licenses.

>  3. Image format and compression (libpng, zlib, etc.).
> 
> A good SVG library is needed.  To get SVG into Tuxpaint, it added libcroco,
> librsvg2, libcairo and other GTK+ library dependencies.  Tuxpaint is based
> on SDL, so would prefer not to see all the added code for two GUI

Yes, I completely forgot about SVG, but really all modern UI and
website graphics should be SVG, since raster images look hideous on
modern 300+ dpi display devices (assuming they're interpreted as
96dpi, which they always are).

> libraries.  Someone was recently asking on the SDL list about SVG library
> options that are actively supported.  The only one with active  support
> that I could locate (besides librsvg2) was the one used by netsurf (
> http://www.netsurf-browser.org/projects/libsvgtiny/ ).

How complete is it? SVG rendering is rather difficult if you want to
support all the features, and if you don't, lots of files will come
out looking wrong... In particular if you want to support text, you
need a font and layout engine, and you may need clones of "standard"
fonts that people use assuming you'll have them because they're
included with Windows...

BTW, SVG is XML-based, but I don't think a good SVG library should
need a heavy XML library. Rather, it should take shortcuts to
interpret just SVG content in a less-bloated way.

> If you're looking at graphics formats.  You might as well add audio formats
> to the list too.  There are a wide variety of libraries for the different
> audio formats and they range in quality (and license types).

Yes and no. Audio and video are both generally a good bit more
difficult than images, and fewer programs need these things. Images
are needed by almost every interactive program these days both for
interface elements and displaying content (which everyone assumes can
display images), but audio is really only widely used for
notifications, games, media players, and voice-chat apps.

> As already mentioned, libintl is on my list of things to replace if
> possible.

Yes, gettext..

> pkg-config has a nice replacement in pkgconf.  (If a list is
> created, might be helpful to list possible replacements already out
> there.)  Would like to see some of the pieces that are essential parts of
> the GNU build system/autotools replaced with some more efficient
> [...]
> of unnecessary work to port applications.  Would rather see the current
> build systems already used (autotools, cmake, etc.) streamlined or see drop
> in replacements that are better designed.

I seem to recall an effort somewhere to do exactly this, but I can't
remember where I saw it..

> One other area that can be troublesome is when there are multiple libraries
> with similar functionality and some of the programs you want to build use
> one while the rest use another.  The openssl/gnutls/nss example is an
> issue.  It's typically solved by supplying wrapper interfaces.  The case I
> keep running into at the moment is libxml2/expat.  libxml2 is newer and
> works in conjunction with libraries like libxslt.  Some Open Source
> programs let you pick one or the other.  Unfortunately, there are some that
> insist on only one.  Makes it hard to simplify one's system down to just
> using one XML library.

In theory it shouldn't matter if more than one is linked, except for
some additional bloat. In reality, there are sometime symbol clashes,
especially when one is a forked version of the other...

Rich

Re: High-priority library replacements?

[Kurt H Maier]

On Thu, Apr 25, 2013 at 09:34:35AM +0200, Jens Staal wrote:
> On 2013-04-25 07:55, Kurt H Maier wrote:
> 
> Does it work as a drop-in replacement for unixy utf/wchar 
> implementations or is it the Plan9 variant (reference to Runes implies 
> the later)? I found myself needing at first libwchar from heirloom and 
> later the (old) GNU libutf8 under APE to provide some wchar-related 
> functions expected by various applications.
> 
> It felt really silly to "devolve" like that.

It's the Plan 9 variant -- wchar is a thing that needs to go away...
maybe one day we can get there.  :(

khm

Re: High-priority library replacements?

[Jens Staal]

On 2013-04-25 14:51, Rich Felker wrote:
>> >libraries.  Someone was recently asking on the SDL list about SVG library
>> >options that are actively supported.  The only one with active  support
>> >that I could locate (besides librsvg2) was the one used by netsurf (
>> >http://www.netsurf-browser.org/projects/libsvgtiny/  ).
> How complete is it? SVG rendering is rather difficult if you want to
> support all the features, and if you don't, lots of files will come
> out looking wrong... In particular if you want to support text, you
> need a font and layout engine, and you may need clones of "standard"
> fonts that people use assuming you'll have them because they're
> included with Windows...
>
> BTW, SVG is XML-based, but I don't think a good SVG library should
> need a heavy XML library. Rather, it should take shortcuts to
> interpret just SVG content in a less-bloated way.
>

There is also an efl-related svg implementation under
http://code.google.com/p/enesim/wiki/GettingStarted

no idea how complete it is though.

Re: High-priority library replacements?

[Zvi Gilboa]

[-- Attachment #1: Type: text/plain, Size: 2231 bytes --]

On 04/25/2013 08:51 AM, Rich Felker wrote:
> On Thu, Apr 25, 2013 at 07:44:39AM -0400, LM wrote:
>> incompatible licenses.  The openssl library can't be used with a GNU
>> program unless there's a waiver for it because one of the clauses in the
>> openssl license goes against the GNU license principles.  The gnutls
> Not _used_ but _distributed_. The GPL does not restrict use
> whatsoever (and takes the position that it legally can't do so) so
> it's fine to use OpenSSL with GPL programs as long as you don't
> distribute the resulting binary. This is of course a problem for
> package maintainers/distributions, and distributing both openssl and
> the GNU program and a script to link them together might even be seen
> as an infringing activity.

What about explicitly loading the library at run-time using uselib(2) in 
a plug-in like fashion?  Is that also considered problematic from a GNU 
perspective?

>> pkg-config has a nice replacement in pkgconf.  (If a list is
>> created, might be helpful to list possible replacements already out
>> there.)  Would like to see some of the pieces that are essential parts of
>> the GNU build system/autotools replaced with some more efficient
>> [...]
>> of unnecessary work to port applications.  Would rather see the current
>> build systems already used (autotools, cmake, etc.) streamlined or see drop
>> in replacements that are better designed.
> I seem to recall an effort somewhere to do exactly this, but I can't
> remember where I saw it..

Here too, one option would be to have all of a library's options, 
dependencies, and build steps represented in a /real/ database (my 
preference would be PostgreSql), and then have the actual build 
script(s) generated using command-line utilities.  This is something 
that I have been doing in a rather miniature scale in a few 
research-related projects, and found to be both effective and 
expandable.  Arguably, for all projects beyond a certain degree of 
complexity, using a database to manage the build is the most feasible 
path to follow (no need to name projects that demonstrate what happens 
if you don't...), yet switching an existing build system to one that is 
database-driven could become quite a challenge.



[-- Attachment #2: Type: text/html, Size: 3081 bytes --]

Re: High-priority library replacements?

[Justin Cormack]

[-- Attachment #1: Type: text/plain, Size: 1161 bytes --]

On 25 Apr 2013 17:52, "Zvi Gilboa" <zg7s@eservices.virginia.edu> wrote:
>
> On 04/25/2013 08:51 AM, Rich Felker wrote:
>>
>> On Thu, Apr 25, 2013 at 07:44:39AM -0400, LM wrote:
>>>
>>> incompatible licenses.  The openssl library can't be used with a GNU
>>> program unless there's a waiver for it because one of the clauses in the
>>> openssl license goes against the GNU license principles.  The gnutls
>>
>> Not _used_ but _distributed_. The GPL does not restrict use
>> whatsoever (and takes the position that it legally can't do so) so
>> it's fine to use OpenSSL with GPL programs as long as you don't
>> distribute the resulting binary. This is of course a problem for
>> package maintainers/distributions, and distributing both openssl and
>> the GNU program and a script to link them together might even be seen
>> as an infringing activity.
>
>
> What about explicitly loading the library at run-time using uselib(2) in
a plug-in like fashion?  Is that also considered problematic from a GNU
perspective?

There is some disagreement about this and it depends what you distribute.
See here http://lwn.net/SubscriberLink/548216/731f7ad0abe52f40/

Justin

[-- Attachment #2: Type: text/html, Size: 1594 bytes --]

Re: High-priority library replacements?

[Zvi Gilboa]

[-- Attachment #1: Type: text/plain, Size: 1991 bytes --]

On 04/25/2013 12:57 PM, Justin Cormack wrote:
>
>
> On 25 Apr 2013 17:52, "Zvi Gilboa" <zg7s@eservices.virginia.edu 
> <mailto:zg7s@eservices.virginia.edu>> wrote:
> >
> > On 04/25/2013 08:51 AM, Rich Felker wrote:
> >>
> >> On Thu, Apr 25, 2013 at 07:44:39AM -0400, LM wrote:
> >>>
> >>> incompatible licenses.  The openssl library can't be used with a GNU
> >>> program unless there's a waiver for it because one of the clauses 
> in the
> >>> openssl license goes against the GNU license principles.  The gnutls
> >>
> >> Not _used_ but _distributed_. The GPL does not restrict use
> >> whatsoever (and takes the position that it legally can't do so) so
> >> it's fine to use OpenSSL with GPL programs as long as you don't
> >> distribute the resulting binary. This is of course a problem for
> >> package maintainers/distributions, and distributing both openssl and
> >> the GNU program and a script to link them together might even be seen
> >> as an infringing activity.
> >
> >
> > What about explicitly loading the library at run-time using 
> uselib(2) in a plug-in like fashion?  Is that also considered 
> problematic from a GNU perspective?
>
> There is some disagreement about this and it depends what you 
> distribute. See here 
> http://lwn.net/SubscriberLink/548216/731f7ad0abe52f40/
>
> Justin
>

Thank you for pointing to this excellent article.  I initially 
considered the plug-in-like case easier to assess -- specifically since 
the loaded library will never be listed as one of the loading object's 
dependencies -- yet with notions such as interdependency and 
"collectivity," one can never be on safe ground when loading a GPL'ed 
library from within a differently-licensed program, open-source or not.  
Ironically, much of the current thread is about the need to create 
alternatives to commonly-used GPL'ed libraries, which in itself reminds 
of past (and present) efforts to create open source alternatives to 
proprietary libraries and software products.

Zvi

[-- Attachment #2: Type: text/html, Size: 3100 bytes --]

Re: High-priority library replacements?

[idunham]

On Thu, Apr 25, 2013 at 08:43:25AM +0200, Gregor Pintar wrote:
> Hello.
> 
> 2013/4/25, Rich Felker <dalias@aerifal.cx>:
> > 2. SSL. The modern internet basically requires using SSL everywhere.
> > We should be aiming/expecting to transition to a world where even
> > non-login-based sites are 100% https; the threats of malicious http
> > injection attacks from rogue or advertising-based access points has
> > gotten too great. Unfortunately, all of the existing SSL
> > implementations are bloated, buggy, and fail even the most basic
> > robustness requirements. A good solution would be based on tomcrypt
> > and would expose a minimal, simple API suited for event-loop-based or
> > threaded use. It may also be useful to have an optional wrapper layer
> > to expose an API that mimics openssl or gnutls. It should also be able
> > to keep up with the changing demands of how to determine which
> > certificate authorities are to be trusted.

> I am working on cryptographic library. It's far from being finished.
> I would be very glad, if someone could look at it.
> Currently I have problems with API design so help would be welcome.
> It isn't in git yet.
> You can get it here:
> https://dl.dropboxusercontent.com/u/83450675/kripto/kripto.tar.gz
> I have plans for SSL library on top of it, but it could take years.

I hate to be the one who says this, but...
Why another crypto library?
There are at least 6 I can think of off the top of my head
(openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
and I know that's not even half of them.
tomcrypt is already good (as Rich mentioned), so code quality isn't
a reason.
While writing your own "xyz" may be a good learning experience and fun
and so on, a crypto library faces some restrictions:
-You will need to fix bugs promptly until you hand over maintainership.
(Otherwise, you become responsible when there's a vulnerability that
stays unfixed.)
-You have to finish it.
Consider the case of gcrypt, which is a GNU project with multiple
contributors, but still doesn't implement enough to make LDAP over SSL
work in all cases (I don't know the exact issue, but it's a standing bug
in Debian.)

And you have to write a good API.  When someone else has a library with
that, you may be better off using it.
What Rich asked about was an SSL lib based on an existing crypto lib,
namely tomcrypt. And that is likely to be a quicker path to results.

> I think best way is not to trust any certificate authority.
> Maybe some certificate p2p protocol could be done?

Not really an option for regular SSL, I'm afraid.
(just due to the standards, and the need for getting it going...)
But something along the lines of OpenBGP (I think that's the name; it's
a BSD-licensed PGP library) would be one place to start if you wanted
that.
It would be interesting if the library transparently handles both SSL
and something along those lines, so someone gets the certificate p2p
stuff for free by using the library.

> > All of these libraries should:
> >
> > - Avoid namespace pollution. Only external symbols should be the
> >   public API and internal-use stuff prefixed with an ugly prefix
> >   that's extremely unlikely to clash with anything.
> All external symbols have "kripto_" prefix.
> 
> > - Avoid unnecessary allocation. Use caller-provided objects where
> >   possible or provide both options.
> I am trying to do least malloc()s possible.
> 
> > - Have absolutely zero global state.
> There is no global state and there won't be any.

Sounding good, and looking good from what I saw. (Although I tend to
dislike "magic number tables", but that's mainly personal taste...
Also, don't consider me an authority on C or programming. I know some, 
but far too little.)

Thanks,
Isaac Dunham

Re: crypto libraries

[idunham]

On Thu, Apr 25, 2013 at 05:55:45PM -0700, idunham@lavabit.com wrote:
> There are at least 6 I can think of off the top of my head
> (openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
> and I know that's not even half of them.
> Not really an option for regular SSL, I'm afraid.
<snip>
> (just due to the standards, and the need for getting it going...)
> But something along the lines of OpenBGP (I think that's the name; it's
> a BSD-licensed PGP library) would be one place to start if you wanted
> that.
> It would be interesting if the library transparently handles both SSL
> and something along those lines, so someone gets the certificate p2p
> stuff for free by using the library.

Sorry, all instances of openbgp should be "netpgp":
http://www.netpgp.com/faq.html

> Thanks,
> Isaac Dunham
> 

Re: High-priority library replacements?

[Rich Felker]

On Thu, Apr 25, 2013 at 05:55:45PM -0700, idunham@lavabit.com wrote:
> > I am working on cryptographic library. It's far from being finished.
> > I would be very glad, if someone could look at it.
> > Currently I have problems with API design so help would be welcome.
> > It isn't in git yet.
> > You can get it here:
> > https://dl.dropboxusercontent.com/u/83450675/kripto/kripto.tar.gz
> > I have plans for SSL library on top of it, but it could take years.
> 
> I hate to be the one who says this, but...
> Why another crypto library?
> There are at least 6 I can think of off the top of my head
> (openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
> and I know that's not even half of them.
> tomcrypt is already good (as Rich mentioned), so code quality isn't
> a reason.
> While writing your own "xyz" may be a good learning experience and fun
> and so on, a crypto library faces some restrictions:
> -You will need to fix bugs promptly until you hand over maintainership.
> (Otherwise, you become responsible when there's a vulnerability that
> stays unfixed.)
> -You have to finish it.
> Consider the case of gcrypt, which is a GNU project with multiple
> contributors, but still doesn't implement enough to make LDAP over SSL
> work in all cases (I don't know the exact issue, but it's a standing bug
> in Debian.)
> 
> And you have to write a good API.  When someone else has a library with
> that, you may be better off using it.
> What Rich asked about was an SSL lib based on an existing crypto lib,
> namely tomcrypt. And that is likely to be a quicker path to results.

Yes. Crypto is hard to get right and when there's an implementation
with high-quality public domain code, I think you'd need a really
strong justification for using a new crypto implementation in an SSL
library.

> > I think best way is not to trust any certificate authority.
> > Maybe some certificate p2p protocol could be done?
> 
> Not really an option for regular SSL, I'm afraid.
> (just due to the standards, and the need for getting it going...)
> But something along the lines of OpenBGP (I think that's the name; it's
> a BSD-licensed PGP library) would be one place to start if you wanted
> that.
> It would be interesting if the library transparently handles both SSL
> and something along those lines, so someone gets the certificate p2p
> stuff for free by using the library.

Yes, I don't really know what the future directions will be, I just
think a library should be prepared to adapt to changing needs for how
we determine trust.

> > > - Have absolutely zero global state.
> > There is no global state and there won't be any.
> 
> Sounding good, and looking good from what I saw. (Although I tend to
> dislike "magic number tables", but that's mainly personal taste...
> Also, don't consider me an authority on C or programming. I know some, 
> but far too little.)

If you dislike magic number tables, you can always include a script in
the build process to generate them. Using tens or hundreds of kb per
process and tons of startup time however is not an acceptable solution
to "I dislike magic number tables". Like them or not, magic number
tables are the only non-bloated solution.

Rich

Re: High-priority library replacements?

[Isaac Dunham]

On Thu, 25 Apr 2013 00:15:53 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> 3. Image format and compression (libpng, zlib, etc.). The existing
> libraries are full of atrociously bad error-handling practices, and
> quite bloated. These libraries should be well-factored so that
> programs that just want to read (the majority of use cases) don't have
> to pull in code for writing. Unfortunately the interfaces might not be
> so simple, since an important usage case is progressive rendering of
> partially-loaded images, both for the purpose of loading over slow
> network connections and for rendering thumbnails quickly.

Have you perchance seen libpnglite?
http://sourceforge.net/projects/pnglite/files/

Yes, I know it's NOT well-factored; the author wanted something where
a single file and header could be added to a project and there's PNG support.
Uses the png_ namespace.
AFAICT, it accepts either a buffer to use for the png or a null pointer, in which case it malloc's what is necessary. 
License is BSD-ish.
It includes a reference to "zlite.h", which seems to be a zip library with a z_ namespace.

Also, I ran across stb_image_write.h, but...I hesitate to _recommend_ something where the header contains all the function source. stb_image.c is another "small, but I don't think that's good enough" thing... 

> All of these libraries should:
> 
> - Avoid namespace pollution. Only external symbols should be the
>   public API and internal-use stuff prefixed with an ugly prefix
>   that's extremely unlikely to clash with anything.
> - Avoid unnecessary allocation. Use caller-provided objects where
>   possible or provide both options.
> - Have absolutely zero global state.
> 
> That's all I can think of at the moment but I'm sure there are other
> needs I've come across and forgotten. Please feel free to supplement
> this list.

A links that may be helpful, perhaps...
http://bashismal.blogspot.com/2011/10/unbloated-resources-in-c.html

-- 
Isaac Dunham <idunham@lavabit.com>

Re: High-priority library replacements?

[nwmcsween]

[-- Attachment #1: Type: text/plain, Size: 89 bytes --]

Thoughts on https://github.com/jedisct1/libsodium ? I haven't dug too much into the code.

[-- Attachment #2: Type: text/html, Size: 268 bytes --]

Re: High-priority library replacements?

[Igmar Palsenberg]

[-- Attachment #1: Type: text/plain, Size: 2774 bytes --]


>>> incompatible licenses.  The openssl library can't be used with a GNU
>>> program unless there's a waiver for it because one of the clauses in the
>>> openssl license goes against the GNU license principles.  The gnutls
>> Not _used_ but _distributed_. The GPL does not restrict use
>> whatsoever (and takes the position that it legally can't do so) so
>> it's fine to use OpenSSL with GPL programs as long as you don't
>> distribute the resulting binary. This is of course a problem for
>> package maintainers/distributions, and distributing both openssl and
>> the GNU program and a script to link them together might even be seen
>> as an infringing activity.
> 
> What about explicitly loading the library at run-time using uselib(2) in a plug-in like fashion?  Is that also considered problematic from a GNU perspective?

I consider this a grey area. I personally don't thing it is considered a problem, but there are a number of interesting (theoretical) scenario's : 

- You have a a.so that dynamically links GPL library b.so. Program C loads a.so, and uses it's API, does it need to be GPL ? a.so can alternatively also load d.so instead of b.so, which is BSD licensed.
 
Using / linking a GPL lib, and making the whole work as such GPL is a clear violation. I personally avoid GPL libs where I can.

>>> pkg-config has a nice replacement in pkgconf.  (If a list is
>>> created, might be helpful to list possible replacements already out
>>> there.)  Would like to see some of the pieces that are essential parts of
>>> the GNU build system/autotools replaced with some more efficient
>>> [...]
>>> of unnecessary work to port applications.  Would rather see the current
>>> build systems already used (autotools, cmake, etc.) streamlined or see drop
>>> in replacements that are better designed.
>> I seem to recall an effort somewhere to do exactly this, but I can't
>> remember where I saw it..
> 
> Here too, one option would be to have all of a library's options, dependencies, and build steps represented in a real database (my preference would be PostgreSql), and then have the actual build script(s) generated using command-line utilities.  This is something that I have been doing in a rather miniature scale in a few research-related projects, and found to be both effective and expandable.  Arguably, for all projects beyond a certain degree of complexity, using a database to manage the build is the most feasible path to follow (no need to name projects that demonstrate what happens if you don't...), yet switching an existing build system to one that is database-driven could become quite a challenge.

I've done nice stuff with SQLIte. Make it also easy to transfer stuff to a different system.



	Igmar


[-- Attachment #2: Type: text/html, Size: 3765 bytes --]

Re: crypto libraries

[Daniel Cegiełka]

2013/4/26  <idunham@lavabit.com>:

>> It would be interesting if the library transparently handles both SSL
>> and something along those lines, so someone gets the certificate p2p
>> stuff for free by using the library.
>
> Sorry, all instances of openbgp should be "netpgp":
> http://www.netpgp.com/faq.html

netpgp require openssl.

Daniel

Re: High-priority library replacements?

[Gregor Pintar]

2013/4/26, idunham@lavabit.com <idunham@lavabit.com>:
> I hate to be the one who says this, but...
> Why another crypto library?
> There are at least 6 I can think of off the top of my head
> (openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
> and I know that's not even half of them.
> tomcrypt is already good (as Rich mentioned), so code quality isn't
> a reason.
Most of them are realy bad (support only few ciphers, ugly API,
inflexible, license).
tomcrypt is good, but it has some global states (ltc_cipher_descriptor, ...)
and I want even more flexible library (variable rounds, no global state).
I would like API that allows replacing ciphers very easy.
For example: allways use kripto_stream_encrypt(),
if you use any block cipher in any mode or stream cipher directly.
Also my hash API supports variable length output (great for sponge
constructions).

> While writing your own "xyz" may be a good learning experience and fun
> and so on, a crypto library faces some restrictions:
> -You will need to fix bugs promptly until you hand over maintainership.
> (Otherwise, you become responsible when there's a vulnerability that
> stays unfixed.)
Not really a problem for me.
BTW, latest official stable tomcrypt release was released in 2007.

> -You have to finish it.
> Consider the case of gcrypt, which is a GNU project with multiple
> contributors, but still doesn't implement enough to make LDAP over SSL
> work in all cases (I don't know the exact issue, but it's a standing bug
> in Debian.)
I am trying to do it very extensible.

> And you have to write a good API.  When someone else has a library with
> that, you may be better off using it.
That is the hardest. API better than tomcrypts is my goal.
I also plan stable ABI.

> What Rich asked about was an SSL lib based on an existing crypto lib,
> namely tomcrypt. And that is likely to be a quicker path to results.
tomcrypt is definitely quicker path.

>> I think best way is not to trust any certificate authority.
>> Maybe some certificate p2p protocol could be done?
>
> Not really an option for regular SSL, I'm afraid.
> (just due to the standards, and the need for getting it going...)
> But something along the lines of OpenBGP (I think that's the name; it's
> a BSD-licensed PGP library) would be one place to start if you wanted
> that.
> It would be interesting if the library transparently handles both SSL
> and something along those lines, so someone gets the certificate p2p
> stuff for free by using the library.
I know, it was more like an long-term idea.

Re: High-priority library replacements?

[LM]

[-- Attachment #1: Type: text/plain, Size: 1127 bytes --]

On Fri, Apr 26, 2013 at 12:19 AM, Isaac Dunham <idunham@lavabit.com> wrote:

> A links that may be helpful, perhaps...
> http://bashismal.blogspot.com/2011/10/unbloated-resources-in-c.html
>
>
Nice resource.  Thanks for sharing it.

If anyone's interested in the OpenGL replacement mentioned on that page,
there are 2 other candidates similar to tinyGL.  There are picogl and
sdl-tinygl.  The Syllable forum (
http://forum.syllable.org/viewforum.php?f=1 ) has some interesting
information about all three of these and makes some comparisons.  If I
remember, one of them has a subset of the glut functionality included.  I
believe some porting is typically needed to use any of them as a
replacement for OpenGL.  Speaking of glut, freeglut (
http://freeglut.sourceforge.net/ ) seems more portable and up-to-date than
glut.

Another interesting resource is picobsd (
http://code.google.com/p/freebsd-head/source/browse/release/?r=bbfa6f219c41b6850ef0e7699f439ad5488435ae#release%2Fpicobsd).
 Has some small utility programs.  I don't think all of them port to
other operating systems, but some of it does work.

Sincerely,
Laura

[-- Attachment #2: Type: text/html, Size: 1785 bytes --]

Re: High-priority library replacements?

[Muhammad Sumyandityo Noor]

[-- Attachment #1: Type: text/plain, Size: 1431 bytes --]

You meant replacement for Mesa? Because TinyGL is software renderer. 
It's unlikely people will utilize software renderer. As for embedded 
system, each SoCs provides their own userland to utilize its hardware 
accelerator.

LM wrote:
>
> On Fri, Apr 26, 2013 at 12:19 AM, Isaac Dunham <idunham@lavabit.com
> <mailto:idunham@lavabit.com>> wrote:
>
> A links that may be helpful, perhaps...
> http://bashismal.blogspot.com/2011/10/unbloated-resources-in-c.html
>
>
> Nice resource. Thanks for sharing it.
>
> If anyone's interested in the OpenGL replacement mentioned on that
> page, there are 2 other candidates similar to tinyGL. There are
> picogl and sdl-tinygl. The Syllable forum (
> http://forum.syllable.org/viewforum.php?f=1 ) has some interesting
> information about all three of these and makes some comparisons. If I
> remember, one of them has a subset of the glut functionality
> included. I believe some porting is typically needed to use any of
> them as a replacement for OpenGL. Speaking of glut, freeglut (
> http://freeglut.sourceforge.net/ ) seems more portable and up-to-date
> than glut.
>
> Another interesting resource is picobsd (
> http://code.google.com/p/freebsd-head/source/browse/release/?r=bbfa6f219c41b6850ef0e7699f439ad5488435ae#release%2Fpicobsd 
>
> ). Has some small utility programs. I don't think all of them port
> to other operating systems, but some of it does work.
>
> Sincerely,
> Laura

[-- Attachment #2: Type: text/html, Size: 1742 bytes --]

Re: High-priority library replacements?

[Rich Felker]

On Fri, Apr 26, 2013 at 10:11:32AM +0200, Gregor Pintar wrote:
> 2013/4/26, idunham@lavabit.com <idunham@lavabit.com>:
> > I hate to be the one who says this, but...
> > Why another crypto library?
> > There are at least 6 I can think of off the top of my head
> > (openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
> > and I know that's not even half of them.
> > tomcrypt is already good (as Rich mentioned), so code quality isn't
> > a reason.
> Most of them are realy bad (support only few ciphers, ugly API,
> inflexible, license).
> tomcrypt is good, but it has some global states (ltc_cipher_descriptor, ...)

Yes, this is stupid, but I don't think there's any reason to use it.
Can't you just use the extern descriptor for the cipher you want
directly? In any case, I agree that when using tomcrypt the register
and unregister functions should be nop'd out and the cipher_descriptor
array replaced with a const version thereof. There may be more changes
needed to fix it too, but that sounds like most of it.

> > While writing your own "xyz" may be a good learning experience and fun
> > and so on, a crypto library faces some restrictions:
> > -You will need to fix bugs promptly until you hand over maintainership.
> > (Otherwise, you become responsible when there's a vulnerability that
> > stays unfixed.)
> Not really a problem for me.
> BTW, latest official stable tomcrypt release was released in 2007.

Yes, that's because it's already very mature.. :) BTW, a big plus of
that is that it would be safe to fork tomcrypt and fix any issues in
it that aren't going to get fixed upstream, like global state, since
maintaining a fork of a mature but clean codebase is almost no work.

> > What Rich asked about was an SSL lib based on an existing crypto lib,
> > namely tomcrypt. And that is likely to be a quicker path to results.
> tomcrypt is definitely quicker path.

Note that there's no reason the crypto implementation couldn't later
be replaced if there's an option better than tomcrypt.

Rich

Re: High-priority library replacements?

[Rich Felker]

On Fri, Apr 26, 2013 at 07:57:36PM +0700, Muhammad Sumyandityo Noor wrote:
> You meant replacement for Mesa? Because TinyGL is software renderer.
> It's unlikely people will utilize software renderer. As for embedded
> system, each SoCs provides their own userland to utilize its
> hardware accelerator.

I don't really understand the GL architecture presently in use well
enough to know the right solutions, but if I'm not mistaken, it
involves loading dynamic modules, possibly even binaryware
hardware-vendor-provided ones, into the address space of your
application. This seems like a recipe for disaster.

Software-rendered GL is definitely not the solution, but I wonder if
it would be reasonable to create a library that provides the OpenGL
API with NO namespace pollution or introduction of dangerous code into
the application's address space, by cooperating with a separate
process via shared memory. On modern Linux, this separate process
could use Linux namespaces to completely throw away all privileges
except to the graphics device. Then the nasty legacy OpenGL code could
run in a sandbox.

Rich

Re: High-priority library replacements?

[LM]

[-- Attachment #1: Type: text/plain, Size: 897 bytes --]

On Fri, Apr 26, 2013 at 8:57 AM, Muhammad Sumyandityo Noor <
lynxluna@gmail.com> wrote:

> You meant replacement for Mesa? Because TinyGL is software renderer. It's
> unlikely people will utilize software renderer. As for embedded system,
> each SoCs provides their own userland to utilize its hardware accelerator.
>
> TinyGL and the other two libraries mentioned can be used as a replacement
for Mesa.  Typically, Windows developers will work with the Windows
implementation of OpenGL not Mesa and TinyGL can be used as a replacement
for that as well.  I'm not saying these libraries provide hardware
acceleration like OpenGL.  They just try to provide the major parts of the
API.  Syllable Desktop does not have access to working OpenGL or Mesa
libraries, so they've been using these to at least allow some programs
written for the OpenGL API to port, render and run on their operating
system.

[-- Attachment #2: Type: text/html, Size: 1177 bytes --]

Re: High-priority library replacements?

[Gregor Pintar]

2013/4/26, Rich Felker <dalias@aerifal.cx>:
> On Fri, Apr 26, 2013 at 10:11:32AM +0200, Gregor Pintar wrote:
>> 2013/4/26, idunham@lavabit.com <idunham@lavabit.com>:
>> > I hate to be the one who says this, but...
>> > Why another crypto library?
>> > There are at least 6 I can think of off the top of my head
>> > (openssl crypto, gcrypt, nettle, tomcrypt, gpg, openbgp)
>> > and I know that's not even half of them.
>> > tomcrypt is already good (as Rich mentioned), so code quality isn't
>> > a reason.
>> Most of them are realy bad (support only few ciphers, ugly API,
>> inflexible, license).
>> tomcrypt is good, but it has some global states (ltc_cipher_descriptor,
>> ...)
>
> Yes, this is stupid, but I don't think there's any reason to use it.
> Can't you just use the extern descriptor for the cipher you want
> directly?
You can't, because modes read from global descriptor.
See: https://github.com/libtom/libtomcrypt/blob/master/src/modes/ctr/ctr_start.c

> In any case, I agree that when using tomcrypt the register
> and unregister functions should be nop'd out and the cipher_descriptor
> array replaced with a const version thereof. There may be more changes
> needed to fix it too, but that sounds like most of it.
But that would break API anyway.

Re: High-priority library replacements?

[Rob Landley]

On 04/25/2013 12:53:53 PM, Zvi Gilboa wrote:
> Ironically, much of the current thread is about the need to create
> alternatives to commonly-used GPL'ed libraries, which in itself  
> reminds
> of past (and present) efforts to create open source alternatives to
> proprietary libraries and software products.

Unfortunately the FSF poisoned copyleft in 2006. The GPL was a category  
killer synonymous with copyleft... until GPLv3 came out and there was  
no longer any such thing as "The GPL".

Today Linux and Samba can't share code even though they implement two  
ends of the same protocol. QEMU is caught between wanting Linux driver  
code to implement devices and gdb/binutils code to implement processors  
and it can't have both. Licensing code "GPLv2 or later" just makes it  
worse: you can donate code to both but can't accept code from either  
one.

Programmers are not lawyers, we're not _good_ at how licenses interact.  
The GPL was a terminal node in a directed graph of license  
convertibily, where all a programmer had to care was "is this  
convertible to The GPL or not"? If it is, treat it as GPL, if not avoid  
it. There was no interaction, there was just The GPL. The FSF destroyed  
that, leaving a fragmented incompatible pool, and people's attempts to  
_fix_ it with Affero GPL or GPL-Next or other viral licenses just  
fragments it further.

Copyleft only worked with a category killer license creating one big  
pool. With multiple incompatible copyleft licenses, copyleft _prevents_  
code sharing because you can't re-use it or combine it in new projects.

In the absence of a universal receiver license, non-lawyer programmers  
looking for someting simple and understandable are switching to  
universal donor licenses. BSD/MIT or outright public domain.

This shows the GPL falling from 72% market share in 2009 to 58 in 2013:

   https://lwn.net/Articles/547400/

Eben Moglen, author of GPLv2 and GPLv3 (Stallman is not a lawyer,  
Moglen is) recently lamented the decline of copyleft but doesn't seem  
to understand why:

   https://lwn.net/Articles/547379/

As far as I can tell the FSF has alienated all the young programmers.  
The most popular license on Github is not specifying a license at all,  
taking the Napster approach of civil disobedience and waiting for the  
intellectual property system to collapse. Ten years ago the GPL would  
have appealed to them, but since GPLv3 shattered the ecosystem it does  
not.

That's why Android's "no GPL in userspace" policy (if you add GPL code  
to your userspace, you can't use the Android trademark advertising your  
product) actually makes _sense_.

> Zvi

Rob

Re: High-priority library replacements?

[Luca Barbato]

On 04/27/2013 07:45 AM, Rob Landley wrote:
> On 04/25/2013 12:53:53 PM, Zvi Gilboa wrote:
>> Ironically, much of the current thread is about the need to create
>> alternatives to commonly-used GPL'ed libraries, which in itself reminds
>> of past (and present) efforts to create open source alternatives to
>> proprietary libraries and software products.
> 
> Unfortunately the FSF poisoned copyleft in 2006. The GPL was a category
> killer synonymous with copyleft... until GPLv3 came out and there was no
> longer any such thing as "The GPL".

For my personal usage I'm still quite fond of LGPL, (even for programs).

I do not see how to fix it beside picking a license I like and tracking
contributors in case I need to switch to another.

lu

Re: High-priority library replacements?

[Zvi Gilboa]

[-- Attachment #1: Type: text/plain, Size: 3130 bytes --]

On 04/27/2013 01:45 AM, Rob Landley wrote:
> On 04/25/2013 12:53:53 PM, Zvi Gilboa wrote:
>> Ironically, much of the current thread is about the need to create
>> alternatives to commonly-used GPL'ed libraries, which in itself reminds
>> of past (and present) efforts to create open source alternatives to
>> proprietary libraries and software products.
>
> Unfortunately the FSF poisoned copyleft in 2006. The GPL was a 
> category killer synonymous with copyleft... until GPLv3 came out and 
> there was no longer any such thing as "The GPL".
>
> Today Linux and Samba can't share code even though they implement two 
> ends of the same protocol. QEMU is caught between wanting Linux driver 
> code to implement devices and gdb/binutils code to implement 
> processors and it can't have both. Licensing code "GPLv2 or later" 
> just makes it worse: you can donate code to both but can't accept code 
> from either one.
>
> Programmers are not lawyers, we're not _good_ at how licenses 
> interact. The GPL was a terminal node in a directed graph of license 
> convertibily, where all a programmer had to care was "is this 
> convertible to The GPL or not"? If it is, treat it as GPL, if not 
> avoid it. There was no interaction, there was just The GPL. The FSF 
> destroyed that, leaving a fragmented incompatible pool, and people's 
> attempts to _fix_ it with Affero GPL or GPL-Next or other viral 
> licenses just fragments it further.
>
> Copyleft only worked with a category killer license creating one big 
> pool. With multiple incompatible copyleft licenses, copyleft 
> _prevents_ code sharing because you can't re-use it or combine it in 
> new projects.

... which apparently is the source of widespread confusion.  To 
illustrate this, go to google.com (with auto-complete enabled), and type 
/including bsd/...

> In the absence of a universal receiver license, non-lawyer programmers 
> looking for someting simple and understandable are switching to 
> universal donor licenses. BSD/MIT or outright public domain.

Another balanced license is the LaTeX Project Public License (LPPL), 
which also effectively governs cases in which a copyright holder has 
abandoned his/her project, or can no longer be reached.


>
> This shows the GPL falling from 72% market share in 2009 to 58 in 2013:
>
>   https://lwn.net/Articles/547400/
>
> Eben Moglen, author of GPLv2 and GPLv3 (Stallman is not a lawyer, 
> Moglen is) recently lamented the decline of copyleft but doesn't seem 
> to understand why:
>
>   https://lwn.net/Articles/547379/
>
> As far as I can tell the FSF has alienated all the young programmers. 
> The most popular license on Github is not specifying a license at all, 
> taking the Napster approach of civil disobedience and waiting for the 
> intellectual property system to collapse. Ten years ago the GPL would 
> have appealed to them, but since GPLv3 shattered the ecosystem it does 
> not.
>
> That's why Android's "no GPL in userspace" policy (if you add GPL code 
> to your userspace, you can't use the Android trademark advertising 
> your product) actually makes _sense_.
>
>> Zvi
>
> Rob


[-- Attachment #2: Type: text/html, Size: 4458 bytes --]

Re: High-priority library replacements?

[Muhammad Sumyandityo Noor]

Rich Felker wrote:
> On Fri, Apr 26, 2013 at 07:57:36PM +0700, Muhammad Sumyandityo Noor wrote:
>> You meant replacement for Mesa? Because TinyGL is software renderer.
>> It's unlikely people will utilize software renderer. As for embedded
>> system, each SoCs provides their own userland to utilize its
>> hardware accelerator.
>
> I don't really understand the GL architecture presently in use well
> enough to know the right solutions, but if I'm not mistaken, it
> involves loading dynamic modules, possibly even binaryware
> hardware-vendor-provided ones, into the address space of your
> application. This seems like a recipe for disaster.

OpenGL support varies from platforms to platforms. On Windows, for 
example, M$ will use software renderer via OpenGL32.dll if the ICD 
(Installable Client Driver) is not present. If the driver is installed 
(e.g. from nVidia or ATI), the call to OpenGL32.dll will be redirected 
to the ICD libraries automatically.

On Linux, especially on embedded space. The SoC manufacturer usually 
ships open source kernel driver to create a device nodes and communicate 
with it. I'll take PowerVR SGX by imagination as an example. They have 
open source driver to create /dev/pvrsvrkm device node. The bulk of 
graphics operation however, lies in userland (non-open source) library 
(libEGL.so and libGLES**.so). This library communicates with the device 
nodes by uploading some (another proprietary) 'firmware' to the GPU, and 
then communicates with device nodes via ioctl with protocol that is 
proprietary too.

So, as you have said it is a big mess. 
(http://www.phoronix.com/scan.php?page=news_item&px=ODk0NA) the article 
published in 2010 and still true even for today.

>
> Software-rendered GL is definitely not the solution, but I wonder if
> it would be reasonable to create a library that provides the OpenGL
> API with NO namespace pollution or introduction of dangerous code into
> the application's address space, by cooperating with a separate
> process via shared memory. On modern Linux, this separate process
> could use Linux namespaces to completely throw away all privileges
> except to the graphics device. Then the nasty legacy OpenGL code could
> run in a sandbox.

Pardon for my limited understanding of this matter. But if you mean 
replacing manufacturer-provided libEGL.so and libGLES**.so, then I don't 
think it's possible, as many parts of it are hidden. And to separate 
that to another process, I don't think OpenGL Context can be shared to 
another process or created without referencing to libEGL.

> Rich

I'm very new in linux system programming. It's based of my (very) 
limited knowledge of how OpenGL ES works in embedded device.

Thanks,
Muhammad Sumyandityo Noor (Didiet)

Re: High-priority library replacements?

[Rich Felker]

On Sun, Apr 28, 2013 at 02:53:44PM +0800, Muhammad Sumyandityo Noor wrote:
> >Software-rendered GL is definitely not the solution, but I wonder if
> >it would be reasonable to create a library that provides the OpenGL
> >API with NO namespace pollution or introduction of dangerous code into
> >the application's address space, by cooperating with a separate
> >process via shared memory. On modern Linux, this separate process
> >could use Linux namespaces to completely throw away all privileges
> >except to the graphics device. Then the nasty legacy OpenGL code could
> >run in a sandbox.
> 
> Pardon for my limited understanding of this matter. But if you mean
> replacing manufacturer-provided libEGL.so and libGLES**.so, then I
> don't think it's possible, as many parts of it are hidden. And to

Not replacing, wrapping. With the wrapped code being in another
sandboxed process.

> separate that to another process, I don't think OpenGL Context can
> be shared to another process or created without referencing to
> libEGL.

Sure it can. This is always fundamentally possible.

Rich

Licensing.

[Rob Landley]

On 04/26/2013 01:11:07 AM, Igmar Palsenberg wrote:
> 
> >>> incompatible licenses.  The openssl library can't be used with a  
> GNU
> >>> program unless there's a waiver for it because one of the clauses  
> in the
> >>> openssl license goes against the GNU license principles.  The  
> gnutls
> >> Not _used_ but _distributed_. The GPL does not restrict use
...
> > What about explicitly loading the library at run-time using  
> uselib(2) in a plug-in like fashion?  Is that also considered  
> problematic from a GNU perspective?
> 
> I consider this a grey area. I personally don't thing it is  
> considered a problem,
> but there are a number of interesting (theoretical) scenario's :

Um, back up:

You know how cryptographers point and laugh at non-cryptographers  
trying to figure out whether something's breakable?

You know how professional security auditors find most programmers' code  
appallingly insecure, and the best of us have to put out regular  
updates to fix exploits that we didn't personally find?

Now imagine what lawyers think of programmers' legal theories.

To write secure code you need a deep understanding of your operating  
system. To license code securely, you have to understand your legal  
jurisdiction. It's fundamentally not enough to know what the  
code/license says in isolation.

Programming-side example: the /tmp dir has the sticky bit set other  
users running inotify to spot new files being created don't immediately  
delete them and replace with a symlink so your mknod/open pair is now  
accessing the wrong file. What your code is doing worked fine, but the  
context it was running in made it insecure. Now imagine telling a  
lawyer that your license usage is unexploitable in all jurisdictions,  
and you know this because you read the license text and you're sure  
you're using it ok. (The best a lawyer or security professional can  
EVER say is "I can't spot where you screwed up".)

Imagine that GPLv2 code has to run as root, so as soon as part of your  
project is running as root you might as well treat the whole thing like  
it is from a security standpoint. Your question above about mixing GPL  
and non-GPL code is like asking "when is it safe to set the sudo bit on  
this binary while leaving the rest non-root". Suddenly you need to know  
a LOT more about your system's configuration and permissions and use  
cases, and the problematic parts are written in a language (legalese)  
that you don't code in, and the correct answer varies by distro.

If GPLv2 is where everything is root, then you just don't mix untrusted  
code in with that, period. GPLv3 has to run as Windows Administrator,  
an incompatible type of root you can't mix with what GPLv2 needs  
(unless you write dual-mode code full of #ifdefs from day one). Other  
licenses are like other userids each with its own security connotations  
when you have to run code _as_ them and not you; again, it's the mixing  
in the same program that's most problematic.

BSD/MIT/PD code requests no special permissions; legally speaking it  
runs as your login user. It can run as anything else you need it to,  
but doesn't _require_ it. It is not itself opening that can of worms.

It's not a perfect analogy, but it should get the "here there be  
dragons" aspect across. Viral and non-viral licensing can each be made  
to work. Mixing them while keeping them distinct is a MINEFIELD, and  
coming to a local consensus with non-experts doesn't help.

Rob

P.S. Trolls are the legal equivalent of script kiddies, trying to  
figure out where to hit a company to make money come out. Patent  
trolls, copyright trolls, trademark trolls, contract trolls... we've  
even seen trade secret trolling against decss and such. Somebody who  
comes along with an obscure legal interpretation and sends you a "pay  
us now or we'll cost your $50,000 and several months of your life  
defending yourself in court EVEN IF YOU WIN, so give us $20k to go  
away" letter. The fact people have started doing this on behalf of the  
GPL is one of the things that turned me against that license.

P.P.S. I could explain how the real legal issue you were trying to  
tackle above is whether or not something can be considered a "derived  
work" under copyright law of the virally licensed external material,  
but this would be like me explaining cryptography or security to you.  
It's not the same as an audit for any specific usage for  
exploitability, AND I'm not the domain expert you'd need to do that  
audit anyway. I know enough to know I _don't_ know enough to reliably  
answer this question.

Re: High-priority library replacements?

[Rob Landley]

On 04/26/2013 10:47:29 AM, Rich Felker wrote:
> > > While writing your own "xyz" may be a good learning experience  
> and fun
> > > and so on, a crypto library faces some restrictions:
> > > -You will need to fix bugs promptly until you hand over  
> maintainership.
> > > (Otherwise, you become responsible when there's a vulnerability  
> that
> > > stays unfixed.)
> > Not really a problem for me.
> > BTW, latest official stable tomcrypt release was released in 2007.
> 
> Yes, that's because it's already very mature.. :) BTW, a big plus of
> that is that it would be safe to fork tomcrypt and fix any issues in
> it that aren't going to get fixed upstream, like global state, since
> maintaining a fork of a mature but clean codebase is almost no work.

I note that dropbear has been maintaining a de-factor fork of  
libtomcrypt all that time. Might want to coordinate with him if you're  
going to bother.

Rob

Re: High-priority library replacements?

[Szabolcs Nagy]

* Gregor Pintar <grpintar@gmail.com> [2013-04-26 10:11:32 +0200]:
> tomcrypt is good, but it has some global states (ltc_cipher_descriptor, ...)
> and I want even more flexible library (variable rounds, no global state).
> I would like API that allows replacing ciphers very easy.
> For example: allways use kripto_stream_encrypt(),
> if you use any block cipher in any mode or stream cipher directly.
> Also my hash API supports variable length output (great for sponge
> constructions).
> 

i'm not sure about the flexibility part

it is good when you want to have the same api
for a wide range of algorithms and you want
to be able to fiddle with their internal settings

but in my opinion this adds many failure modes
which is bad

i never liked that in tomcrypt, openssl etc trivial
functions have error code return values which
should never fail

(eg hash_update(ctx, buf, len) should never fail
even if there is a counter in ctx that can overflow
every 2^64th bit of input, documenting the behaviour
for longer inputs is better, it would be even better
if the apropriate standards were more careful about
failures)

the way it is done in most crypt apis make most of the
code that use them broken:
they don't check the return value so if the function
may fail the code is broken, or they check but never
test the failure path so the code is broken for a
different reason

my preference would be to have a well documented, clean
highlevel api with minimal amount of failure modes
(no error code return value), and a flexible api with
ugly names for those who like to fiddle with internals

Re: High-priority library replacements?

[Rich Felker]

On Mon, Apr 29, 2013 at 12:16:20PM +0200, Szabolcs Nagy wrote:
> * Gregor Pintar <grpintar@gmail.com> [2013-04-26 10:11:32 +0200]:
> > tomcrypt is good, but it has some global states (ltc_cipher_descriptor, ...)
> > and I want even more flexible library (variable rounds, no global state).
> > I would like API that allows replacing ciphers very easy.
> > For example: allways use kripto_stream_encrypt(),
> > if you use any block cipher in any mode or stream cipher directly.
> > Also my hash API supports variable length output (great for sponge
> > constructions).
> > 
> 
> i'm not sure about the flexibility part
> 
> it is good when you want to have the same api
> for a wide range of algorithms and you want
> to be able to fiddle with their internal settings
> 
> but in my opinion this adds many failure modes
> which is bad
> 
> i never liked that in tomcrypt, openssl etc trivial
> functions have error code return values which
> should never fail
> 
> (eg hash_update(ctx, buf, len) should never fail
> even if there is a counter in ctx that can overflow
> every 2^64th bit of input, documenting the behaviour
> for longer inputs is better, it would be even better
> if the apropriate standards were more careful about
> failures)

This is a really good point. Hashing and encryption should
fundamentally be able to deal with arbitrary inputs and should not
require resource allocation to do their job. As such, they should have
no failure cases. Decryption of course has the failure case where the
input is malformed, but should not have any other failure cases.

Rich

Re: High-priority library replacements?

[Gregor Pintar]

2013/4/29, Szabolcs Nagy <nsz@port70.net>:
> * Gregor Pintar <grpintar@gmail.com> [2013-04-26 10:11:32 +0200]:
>> tomcrypt is good, but it has some global states (ltc_cipher_descriptor,
>> ...)
>> and I want even more flexible library (variable rounds, no global state).
>> I would like API that allows replacing ciphers very easy.
>> For example: allways use kripto_stream_encrypt(),
>> if you use any block cipher in any mode or stream cipher directly.
>> Also my hash API supports variable length output (great for sponge
>> constructions).
>>
>
> i'm not sure about the flexibility part
>
> it is good when you want to have the same api
> for a wide range of algorithms and you want
> to be able to fiddle with their internal settings
>
> but in my opinion this adds many failure modes
> which is bad
>
> i never liked that in tomcrypt, openssl etc trivial
> functions have error code return values which
> should never fail
>
> (eg hash_update(ctx, buf, len) should never fail
> even if there is a counter in ctx that can overflow
> every 2^64th bit of input, documenting the behaviour
> for longer inputs is better, it would be even better
> if the apropriate standards were more careful about
> failures)
Devs that ignore return values would probably ignore documentation too.

> the way it is done in most crypt apis make most of the
> code that use them broken:
> they don't check the return value so if the function
> may fail the code is broken, or they check but never
> test the failure path so the code is broken for a
> different reason
Encrypting more than 2^(block size) / 2 of blocks is broken too.

Re: Licensing.

[Rich Felker]

On Sun, Apr 28, 2013 at 04:34:38PM -0500, Rob Landley wrote:
> On 04/26/2013 01:11:07 AM, Igmar Palsenberg wrote:
> >
> >>>> incompatible licenses.  The openssl library can't be used with
> >a GNU
> >>>> program unless there's a waiver for it because one of the
> >clauses in the
> >>>> openssl license goes against the GNU license principles.  The
> >gnutls
> >>> Not _used_ but _distributed_. The GPL does not restrict use
> ....
> >> What about explicitly loading the library at run-time using
> >uselib(2) in a plug-in like fashion?  Is that also considered
> >problematic from a GNU perspective?
> >
> >I consider this a grey area. I personally don't thing it is
> >considered a problem,
> >but there are a number of interesting (theoretical) scenario's :
> 
> Um, back up:
> 
> You know how cryptographers point and laugh at non-cryptographers
> trying to figure out whether something's breakable?
> 
> You know how professional security auditors find most programmers'
> code appallingly insecure, and the best of us have to put out
> regular updates to fix exploits that we didn't personally find?
> 
> Now imagine what lawyers think of programmers' legal theories.

Your analogy would hold more water if the majority of lawyers doing
software licenses had any clue about the law, but they don't. Nearly
everything ever written in a proprietary software license has no basis
in law; legally, such documents are not even licenses (a license gives
you permission to do something that would otherwise not be permitted
under the law; it doesn't take away your rights) but one-sided
contracts that are never signed and that offer the victim nothing of
value in exchange for surrendering their rights.

> Programming-side example: the /tmp dir has the sticky bit set other
> users running inotify to spot new files being created don't
> immediately delete them and replace with a symlink so your
> mknod/open pair is now accessing the wrong file. What your code is
> doing worked fine, but the context it was running in made it
> insecure.

I don't follow. Unless you do idiotic things (like omit O_EXCL) there
is nothing unsafe about properly-configured public temp directories.

> Now imagine telling a lawyer that your license usage is
> unexploitable in all jurisdictions, and you know this because you
> read the license text and you're sure you're using it ok. (The best
> a lawyer or security professional can EVER say is "I can't spot
> where you screwed up".)

Even someone with no security skills at all can tell you there is no
vulnerability *in your code* if your code is:

int main() {}

Similarly, a non-lawyer can tell you there's no vulnerability in a
"0-clause" BSD license.

The potential for vulnerability is only introduced with complexity.

Anyway, this thread has gotten fairly off-topic. Legal theories and
the merits of different licenses from legal standpoints are fairly
out-of-scope here. The only place license is relevant to the library
discussion is when it affects whether or not parties who want to use
the functionality of the library are able to use it.

Rich

Re: Licensing.

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 836 bytes --]

Am Montag, den 29.04.2013, 16:47 -0400 schrieb Rich Felker:
> > Now imagine what lawyers think of programmers' legal theories.
> 
> Your analogy would hold more water if the majority of lawyers doing
> software licenses had any clue about the law, but they don't.

so the majority of self proclaimed C programmers actually know C ?-)

Your subsequent mail seems to indicate that there are a lot that don't.

sorry, seeing your two mails, just one after the other, couldn't
resist :)

Jens


-- 
:: INRIA Nancy Grand Est :: http://www.loria.fr/~gustedt/   ::
:: AlGorille ::::::::::::::: office Nancy : +33 383593090   ::
:: ICube :::::::::::::: office Strasbourg : +33 368854536   ::
:: ::::::::::::::::::::::::::: gsm France : +33 651400183   ::
:: :::::::::::::::::::: gsm international : +49 15737185122 ::



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: High-priority library replacements?

[Szabolcs Nagy]

* Gregor Pintar <grpintar@gmail.com> [2013-04-29 19:35:03 +0200]:
> > (eg hash_update(ctx, buf, len) should never fail
> > even if there is a counter in ctx that can overflow
> > every 2^64th bit of input, documenting the behaviour
> > for longer inputs is better, it would be even better
> > if the apropriate standards were more careful about
> > failures)
> Devs that ignore return values would probably ignore documentation too.

i think the problem is not ignorance, but the combinatorial
explosion of code paths which increases bug probability

in my view the purpose of error codes is to indicate runtime
errors (eg allocation or io failures) and not to prevent bugs
in the program logic (there are other tools for that, most
notably the type checker, adding extra error handling paths
for bug prevention usually makes things worse)

returning an error code when some inconsistency is detected
seems simple, but error semantics are usually not trivial:
is it enough to retry, or retry with different parameters
or treat the failure as fatal, should all the related states
be cleaned if they might contain sensitive data or would the
library do this for us, how to report the error on the ui etc

without documentation the user should prepare for the worst and
that can easily turn into a harder problem than implementing
sha1 from scratch

> Encrypting more than 2^(block size) / 2 of blocks is broken too.

that's why cryptographers should choose the block size
wisely so the user does not have to worry about it

Re: Licensing.

[Kurt H Maier]

On Mon, Apr 29, 2013 at 11:10:59PM +0200, Jens Gustedt wrote:
> Am Montag, den 29.04.2013, 16:47 -0400 schrieb Rich Felker:
> > > Now imagine what lawyers think of programmers' legal theories.
> > 
> > Your analogy would hold more water if the majority of lawyers doing
> > software licenses had any clue about the law, but they don't.
> 
> so the majority of self proclaimed C programmers actually know C ?-)
> 
> Your subsequent mail seems to indicate that there are a lot that don't.
> 
> sorry, seeing your two mails, just one after the other, couldn't
> resist :)

This phenomenon is not limited to C.

khm

Re: Licensing.

[Rob Landley]

On 04/29/2013 03:47:31 PM, Rich Felker wrote:
> On Sun, Apr 28, 2013 at 04:34:38PM -0500, Rob Landley wrote:
> > On 04/26/2013 01:11:07 AM, Igmar Palsenberg wrote:
> > >
> > >>>> incompatible licenses.  The openssl library can't be used with
> > >a GNU
> > >>>> program unless there's a waiver for it because one of the
> > >clauses in the
> > >>>> openssl license goes against the GNU license principles.  The
> > >gnutls
> > >>> Not _used_ but _distributed_. The GPL does not restrict use
> > ....
> > >> What about explicitly loading the library at run-time using
> > >uselib(2) in a plug-in like fashion?  Is that also considered
> > >problematic from a GNU perspective?
> > >
> > >I consider this a grey area. I personally don't thing it is
> > >considered a problem,
> > >but there are a number of interesting (theoretical) scenario's :
> >
> > Um, back up:
> >
> > You know how cryptographers point and laugh at non-cryptographers
> > trying to figure out whether something's breakable?
> >
> > You know how professional security auditors find most programmers'
> > code appallingly insecure, and the best of us have to put out
> > regular updates to fix exploits that we didn't personally find?
> >
> > Now imagine what lawyers think of programmers' legal theories.
> 
> Your analogy would hold more water if the majority of lawyers doing
> software licenses had any clue about the law, but they don't. Nearly
> everything ever written in a proprietary software license has no basis
> in law; legally, such documents are not even licenses (a license gives
> you permission to do something that would otherwise not be permitted
> under the law; it doesn't take away your rights) but one-sided
> contracts that are never signed and that offer the victim nothing of
> value in exchange for surrendering their rights.

Sturgeon's Law applies everywhere, yes. Lots of self-proclaimed domain  
experts aren't. (And an MCSE doesn't qualify you do secure posix code.)

> > Programming-side example: the /tmp dir has the sticky bit set other
> > users running inotify to spot new files being created don't
> > immediately delete them and replace with a symlink so your
> > mknod/open pair is now accessing the wrong file. What your code is
> > doing worked fine, but the context it was running in made it
> > insecure.
> 
> I don't follow. Unless you do idiotic things (like omit O_EXCL) there
> is nothing unsafe about properly-configured public temp directories.

Agreed, but most programmers had somebody else set it up for them and  
often don't know why they did it that way. (And then there's the  
"predictable names" fun where "they can't stomp you" doesn't mean "you  
can't me tricked into stomping them". Sure it's managable, but the  
issue does exist.)

The reason for that digression was it's a fairly simple, easily  
explained example of "you don't just have to understand what _you_ did,  
you have to understand the entire environment you did it in".

> > Now imagine telling a lawyer that your license usage is
> > unexploitable in all jurisdictions, and you know this because you
> > read the license text and you're sure you're using it ok. (The best
> > a lawyer or security professional can EVER say is "I can't spot
> > where you screwed up".)
> 
> Even someone with no security skills at all can tell you there is no
> vulnerability *in your code* if your code is:
> 
> int main() {}
> 
> Similarly, a non-lawyer can tell you there's no vulnerability in a
> "0-clause" BSD license.

I agree in principle, sure.

(My friend Lamont "Piggy" Yaroll used an old unix system, beta of NeXT  
I think, where /bin/true was a 0 byte file with the executable bit set.  
That got interpreted as a shell script, does nothing, returns true.  
Clever, eh? Until he used true in his /etc/profile, which loaded  
/bin/sh, which parsed /etc/profile, which loaded /bin/sh... filled up  
all memory, brought down the system. He filed a bug. Their bug  
reporting system tried to calculate defect density in the file and  
threw a division by zero exception. I mean, I break stuff all the time,  
but I bow down to the master.)

(The point of that digression was that I've hung around enough system  
engineers and enough lawyers to wince when anybody says "there's no  
vulnerability in X". Reflex at this point, sorry. "I don't see how  
you'd exploit that" and "sucks less than the alternatives" are the best  
I can do...)

(The legal side of that includes software patents, disclaimer of  
liability, the author's "moral rights" in Germany that they _couldn't_  
disclaim until the law changed in december 2007, the not QUITE fully  
repealed US crypto export regulations, DMCA anti-circumvention  
nonsense... A 0-clause BSD does indeed suck least as a clear attempt to  
opt out, but IP law is inherently horrible for the forseeable future.)

> The potential for vulnerability is only introduced with complexity.

Complexity such as mixing multiple licenses in the same program, unless  
they're all fully convertible to a single license.

I've learned enough of this game that I don't want to play it anymore...

> Anyway, this thread has gotten fairly off-topic.

Agreed.

Rob

Re: High-priority library replacements?

[Rich Felker]

On Mon, Apr 29, 2013 at 11:55:00PM +0200, Szabolcs Nagy wrote:
> * Gregor Pintar <grpintar@gmail.com> [2013-04-29 19:35:03 +0200]:
> > > (eg hash_update(ctx, buf, len) should never fail
> > > even if there is a counter in ctx that can overflow
> > > every 2^64th bit of input, documenting the behaviour
> > > for longer inputs is better, it would be even better
> > > if the apropriate standards were more careful about
> > > failures)
> > Devs that ignore return values would probably ignore documentation too.
> 
> i think the problem is not ignorance, but the combinatorial
> explosion of code paths which increases bug probability
> 
> in my view the purpose of error codes is to indicate runtime
> errors (eg allocation or io failures) and not to prevent bugs
> in the program logic (there are other tools for that, most
> notably the type checker, adding extra error handling paths
> for bug prevention usually makes things worse)

Agreed. You've expressed the issue very concisely and effectively. The
way I tend to think of it is that error codes should express a
condition that can occur in a correct program. If the condition can't
occur in a correct program, no effort should be spent at runtime
detecting it. If a program is going to go about detecting error codes
that couldn't occur without a bug in the program, though, it should do
so via assert() rather than complex error-propagation logic.

Rich

Re: High-priority library replacements?

[Gregor Pintar]

2013/4/30, Rich Felker <dalias@aerifal.cx>:
> On Mon, Apr 29, 2013 at 11:55:00PM +0200, Szabolcs Nagy wrote:
>> * Gregor Pintar <grpintar@gmail.com> [2013-04-29 19:35:03 +0200]:
>> > > (eg hash_update(ctx, buf, len) should never fail
>> > > even if there is a counter in ctx that can overflow
>> > > every 2^64th bit of input, documenting the behaviour
>> > > for longer inputs is better, it would be even better
>> > > if the apropriate standards were more careful about
>> > > failures)
>> > Devs that ignore return values would probably ignore documentation too.
>>
>> i think the problem is not ignorance, but the combinatorial
>> explosion of code paths which increases bug probability
>>
>> in my view the purpose of error codes is to indicate runtime
>> errors (eg allocation or io failures) and not to prevent bugs
>> in the program logic (there are other tools for that, most
>> notably the type checker, adding extra error handling paths
>> for bug prevention usually makes things worse)
>
> Agreed. You've expressed the issue very concisely and effectively. The
> way I tend to think of it is that error codes should express a
> condition that can occur in a correct program. If the condition can't
> occur in a correct program, no effort should be spent at runtime
> detecting it. If a program is going to go about detecting error codes
> that couldn't occur without a bug in the program, though, it should do
> so via assert() rather than complex error-propagation logic.
>
My idea was that program would be correct, if it inputs too much data
to hash function. It is very cheap to implement in most algorithms
(detect counter overflow). Otherwise program has to count it himself.
However, in most cases it is too late to handle correctly when it
already fails. Not returning error on too much input or output could
silently cause incorrect usage. Using assert might be better idea
since at that point it is fatal anyway, but that would cause data
leak, unless program would catch SIGABRT (it should others like SIGINT
anyway) and wipe data, but that requires all sensitive data in program
is global.

Re: High-priority library replacements?

[Szabolcs Nagy]

* Gregor Pintar <grpintar@gmail.com> [2013-04-30 08:32:26 +0200]:
> My idea was that program would be correct, if it inputs too much data
> to hash function. It is very cheap to implement in most algorithms
> (detect counter overflow). Otherwise program has to count it himself.

i dont think the program has to count

eg in case of sha1 if you know that the throughput is less than
10gbps then it takes more than 50years to overflow

in theory there might be use-cases where the overflow could occure
in which case reporting error makes sense, but it seems to me that
can be avoided by the proper choice of algorithm or reasonable
application design

of course if you allow configurability of the block size and rounds
etc then the overflow becomes a practical concern so the error is
justified, which is why i said that flexibility is not necessarily
a good thing (general interface requires general error handling)

> However, in most cases it is too late to handle correctly when it
> already fails. Not returning error on too much input or output could
> silently cause incorrect usage. Using assert might be better idea
> since at that point it is fatal anyway, but that would cause data
> leak, unless program would catch SIGABRT (it should others like SIGINT
> anyway) and wipe data, but that requires all sensitive data in program
> is global.

i'd let the user do the bad thing (silently overflow)
if it matters it can be designed around in the application

Re: High-priority library replacements?

[Gregor Pintar]

2013/4/30, Szabolcs Nagy <nsz@port70.net>:
> * Gregor Pintar <grpintar@gmail.com> [2013-04-30 08:32:26 +0200]:
>> My idea was that program would be correct, if it inputs too much data
>> to hash function. It is very cheap to implement in most algorithms
>> (detect counter overflow). Otherwise program has to count it himself.
>
> i dont think the program has to count
>
> eg in case of sha1 if you know that the throughput is less than
> 10gbps then it takes more than 50years to overflow
>
Blowfish can encrypt max 128GB.

> in theory there might be use-cases where the overflow could occure
> in which case reporting error makes sense, but it seems to me that
> can be avoided by the proper choice of algorithm or reasonable
> application design
>
Choice of algorithm is not mine.

> of course if you allow configurability of the block size and rounds
> etc then the overflow becomes a practical concern so the error is
> justified, which is why i said that flexibility is not necessarily
> a good thing (general interface requires general error handling)
>
Block size isn't configurable (but algorithms have different block
size). Rounds can cause error only on *_create() (so does key).
Flexibility usually increases complexy.

>> However, in most cases it is too late to handle correctly when it
>> already fails. Not returning error on too much input or output could
>> silently cause incorrect usage. Using assert might be better idea
>> since at that point it is fatal anyway, but that would cause data
>> leak, unless program would catch SIGABRT (it should others like SIGINT
>> anyway) and wipe data, but that requires all sensitive data in program
>> is global.
>
> i'd let the user do the bad thing (silently overflow)
> if it matters it can be designed around in the application
>

Re: High-priority library replacements?

[Szabolcs Nagy]

* Gregor Pintar <grpintar@gmail.com> [2013-04-30 11:58:26 +0200]:
> 2013/4/30, Szabolcs Nagy <nsz@port70.net>:
> > * Gregor Pintar <grpintar@gmail.com> [2013-04-30 08:32:26 +0200]:
> >> My idea was that program would be correct, if it inputs too much data
> >> to hash function. It is very cheap to implement in most algorithms
> >> (detect counter overflow). Otherwise program has to count it himself.
> >
> > i dont think the program has to count
> >
> > eg in case of sha1 if you know that the throughput is less than
> > 10gbps then it takes more than 50years to overflow
> >
> Blowfish can encrypt max 128GB.
> 

i see, i didn't know this

i wonder how often do ppl check the return value
(eg you don't seem to do it in your tests)

> > in theory there might be use-cases where the overflow could occure
> > in which case reporting error makes sense, but it seems to me that
> > can be avoided by the proper choice of algorithm or reasonable
> > application design
> >
> Choice of algorithm is not mine.

i meant that if a tool is used for something that it was not
designed for then that's a user error not a tool error


the lib looks fine otherwise, but the hard parts are missing for tls
(rsa,dsa,ecdsa require a bignum library, tls requires a parser for a
lot of complex formats)

Re: Licensing.

[LM]

[-- Attachment #1: Type: text/plain, Size: 2330 bytes --]

While the license subject has been brought up, would appreciate any
clarification on how licenses/copyrights might apply to runtime libraries
like musl and distributing executables.

Here's what I've dug up trying to read through the 'legalese'.  gcc
provides a runtime exception (
http://www.gnu.org/licenses/gcc-exception-faq.html ), so according to it,
you should be able to use the license of your choice with your executable
and don't have to use the compiler's license.  Linux kernel developers
don't consider the API an issue and have a note (
https://www.kernel.org/pub/linux/kernel/COPYING ) about not copyrighting
kernel services through normal system calls.  Zvi and I were discussing
copyright and he mentioned the case of Oracle against Google and that APIs
were found not copyrightable.  So, if you build a program with gcc and musl
as the runtime library, do you need to distribute any kind of licensing
information other than whatever you decide to license your own program
under?

I was reading some information on mingw-w64.  It uses a runtime made up of
a variety of licenses (from BSD to LGPL).
There's mention of distributing a runtime license with executables created
by the compiler:
http://www.math.washington.edu/tex-archive/systems/texlive/tlnet/tlpkg/installer/COPYING.MinGW-runtime.txt
With mingw (the original from www.mingw.org), the runtime has always been
in the public domain in the past, so needing to supply the end user with a
runtime license when distributing executables was never necessary.  Was
surprised to read the recommendation that one needed a runtime license to
distribute programs with mingw-w64.  My understanding was that one doesn't
need this sort of thing for gcc/glibc on standard Linux systems.  I'm
wondering what applies to musl and what doesn't.  Does one need to
distribute the musl license when distributing executables built with it?
Does the gcc compiler runtime exception cover the issue or is there
something else that applies?  Also can't help wondering how it applies to
llvm (http://llvm.org/docs/DeveloperPolicy.html#license) and executables
built with it.

I agree with the previous comment posted about the more licenses involved
the more confusing it can get.  Any further information or clarifications
would be greatly appreciated.  Thanks.

Sincerely,
Laura

[-- Attachment #2: Type: text/html, Size: 2817 bytes --]

Re: High-priority library replacements?

[Gregor Pintar]

2013/4/30, Szabolcs Nagy <nsz@port70.net>:
> * Gregor Pintar <grpintar@gmail.com> [2013-04-30 11:58:26 +0200]:
>> 2013/4/30, Szabolcs Nagy <nsz@port70.net>:
>> > * Gregor Pintar <grpintar@gmail.com> [2013-04-30 08:32:26 +0200]:
>> >> My idea was that program would be correct, if it inputs too much data
>> >> to hash function. It is very cheap to implement in most algorithms
>> >> (detect counter overflow). Otherwise program has to count it himself.
>> >
>> > i dont think the program has to count
>> >
>> > eg in case of sha1 if you know that the throughput is less than
>> > 10gbps then it takes more than 50years to overflow
>> >
>> Blowfish can encrypt max 128GB.
>>
>
> i see, i didn't know this
>
> i wonder how often do ppl check the return value
> (eg you don't seem to do it in your tests)
>
I wrote tests only to see if implementation is close to correct. Most
tests doesn't even work (they were not updated when api changed). So
tests are not even close to finished.

>> > in theory there might be use-cases where the overflow could occure
>> > in which case reporting error makes sense, but it seems to me that
>> > can be avoided by the proper choice of algorithm or reasonable
>> > application design
>> >
>> Choice of algorithm is not mine.
>
> i meant that if a tool is used for something that it was not
> designed for then that's a user error not a tool error
>
>
> the lib looks fine otherwise, but the hard parts are missing for tls
> (rsa,dsa,ecdsa require a bignum library, tls requires a parser for a
> lot of complex formats)
>
Yes, I know I will probably use libtommath (for start) and public key
crypto will probably be ported from libtomcrypt (twofish is also form
libtomcrypt). tls/ssl will probably be separate.

Re: High-priority library replacements?

[Nicolas Braud-Santoni]

[-- Attachment #1: Type: text/plain, Size: 599 bytes --]

On 25/04/2013 08:43, Gregor Pintar wrote:
> Hello.
> [...]
>
> I think best way is not to trust any certificate authority.
> Maybe some certificate p2p protocol could be done?

Hello,

Are you aware of DANE (RFC6698, https://en.wikipedia.org/wiki/DANE) ?
It is a RFC which suggests holding certificates fingerprints in special
DNS records.
Since DNSSEC allows us to establish trust of these records, this is a
simple and robust alternative to CA-based trust models.

However, and AFAIK, it doesn't cope with entities that aren't accessed
through a hostname.


Have a good day,


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: High-priority library replacements?

[Gregor Pintar]

2013/4/30, Nicolas Braud-Santoni <nicolas.braudsantoni@gmail.com>:
> On 25/04/2013 08:43, Gregor Pintar wrote:
>> Hello.
>> [...]
>>
>> I think best way is not to trust any certificate authority.
>> Maybe some certificate p2p protocol could be done?
>
> Hello,
>
> Are you aware of DANE (RFC6698, https://en.wikipedia.org/wiki/DANE) ?
> It is a RFC which suggests holding certificates fingerprints in special
> DNS records.
> Since DNSSEC allows us to establish trust of these records, this is a
> simple and robust alternative to CA-based trust models.
>
> However, and AFAIK, it doesn't cope with entities that aren't accessed
> through a hostname.
>
>
> Have a good day,
>
>
Thanks, I was not aware of it.

Re: High-priority library replacements?

[Gregor Pintar]

2013/4/30, Gregor Pintar <grpintar@gmail.com>:
> Blowfish can encrypt max 128GB.
>
Sorry, it's 32GB --> (2^(block size in bits / 2)) * (block size in
bytes) = (2^32) * 8

Re: High-priority library replacements?

[Daniel Cegiełka]

and tropicssl?

http://gitorious.org/tropicssl

http://teholabs.com/tag/ssl/

Daniel

Re: High-priority library replacements?

[idunham]

On Wed, May 08, 2013 at 11:37:47PM +0200, Daniel Cegie?ka wrote:
> and tropicssl?
> 
> http://gitorious.org/tropicssl
> 
> http://teholabs.com/tag/ssl/

Tropicssl is a fork of polarssl, but as far as I can tell they haven't
fixed CVEs 2013-0169, 2013-1621, and 2013-1622 yet.

While I agree with their view of the license switch, the activity (or
lack thereof) does not seem promising.

--
Isaac Dunham

Re: High-priority library replacements?

[Daniel Cegiełka]

2013/5/9  <idunham@lavabit.com>:

> Tropicssl is a fork of polarssl, but as far as I can tell they haven't
> fixed CVEs 2013-0169, 2013-1621, and 2013-1622 yet.
>
> While I agree with their view of the license switch, the activity (or
> lack thereof) does not seem promising.

The problem is that we have the choice to either accept the GPL
(CyaSSL, PolarSSL) or write own implementation of a ssl from
scratch... or use eg tropicssl as a ssl base.

btw.  license issue - can we legally link noxcuse (GPL) with musl (MIT)? :)

Daniel

Re: High-priority library replacements?

[Daniel Cegiełka]

> 2013/5/9  <idunham@lavabit.com>:
>

> While I agree with their view of the license switch, the activity (or
> lack thereof) does not seem promising.

FOSS License Exception - it might not be as bad as we described earlier.

PolarSSL:
https://polarssl.org/foss-license-exception

CyaSSL:
http://www.yassh.com/yaSSL/License.html

Daniel

Re: High-priority library replacements?

[LM]

[-- Attachment #1: Type: text/plain, Size: 590 bytes --]

On Thu, May 9, 2013 at 3:36 AM, Daniel Cegiełka
<daniel.cegielka@gmail.com>wrote:

>
> btw.  license issue - can we legally link noxcuse (GPL) with musl (MIT)? :)
>
>
That's a nice collection.  Wasn't aware it was available.  Minix (check
simple directory for lightweight alternatives not ported from BSD systems)
and sbase also offer some alternatives to GNU tools, busybox, toybox,
Heirloom Project, et al.

I'm currently look at code for various implementations of common utilities
and checking how well it will port to other platforms besides Linux.

Sincerely,
Laura

[-- Attachment #2: Type: text/html, Size: 894 bytes --]

Re: High-priority library replacements?

[Rich Felker]

On Thu, May 09, 2013 at 09:36:18AM +0200, Daniel Cegiełka wrote:
> 2013/5/9  <idunham@lavabit.com>:
> 
> > Tropicssl is a fork of polarssl, but as far as I can tell they haven't
> > fixed CVEs 2013-0169, 2013-1621, and 2013-1622 yet.
> >
> > While I agree with their view of the license switch, the activity (or
> > lack thereof) does not seem promising.
> 
> The problem is that we have the choice to either accept the GPL
> (CyaSSL, PolarSSL) or write own implementation of a ssl from
> scratch... or use eg tropicssl as a ssl base.
> 
> btw.  license issue - can we legally link noxcuse (GPL) with musl (MIT)? :)

Of course; otherwise musl would be rather useless since there's a huge
corpus of GPL code you want to link and run. The point of a permissive
license is that it does not restrict you from doing things like this.

BTW, I'll probably relicense noxcuse anyway. I just never got around
to doing much/anything with it after initially publishing it. The main
app in it that's really useful right now though is iconv, since
Busybox lacks iconv and glibc iconv is glibc-specific.

Rich

Re: High-priority library replacements?

[Daniel Cegiełka]

2013/5/9 Rich Felker <dalias@aerifal.cx>:

>> btw.  license issue - can we legally link noxcuse (GPL) with musl (MIT)? :)
>
> Of course; otherwise musl would be rather useless since there's a huge
> corpus of GPL code you want to link and run. The point of a permissive
> license is that it does not restrict you from doing things like this.

My question was a joke, which was to show how absurd case we have now
- instead of software freedom we have concerns.

> BTW, I'll probably relicense noxcuse anyway.

This is the result of frustration:
http://en.wikipedia.org/wiki/WTFPL
http://www.wtfpl.net/about/


btw. OpenSSL.org published a small version of openssl-fips. It
contains only FIPS 140-2 validated code.

http://www.openssl.org/source/
http://www.openssl.org/docs/fips/fipsnotes.html

OpeSSL-fips vs CyaSSL (musl)

# ls -lh /var/tmp/ports/cyassl/image/usr/lib/
-rw-r--r--    1 root     root       866.9k May  9 09:13 libcyassl.a
-rwxr-xr-x    1 root     root       659.6k May  9 09:13 libcyassl.so.5.0.0

# ls -lh /usr/local/ssl/fips-2.0/lib/
-r--r--r--    1 root     root       685.7k May  9 12:34 fipscanister.o

thx,
Daniel

Re: High-priority library replacements?

[Rich Felker]

On Thu, May 09, 2013 at 04:40:03PM +0200, Daniel Cegiełka wrote:
> btw. OpenSSL.org published a small version of openssl-fips. It
> contains only FIPS 140-2 validated code.
> 
> http://www.openssl.org/source/
> http://www.openssl.org/docs/fips/fipsnotes.html
> 
> OpeSSL-fips vs CyaSSL (musl)
> 
> # ls -lh /var/tmp/ports/cyassl/image/usr/lib/
> -rw-r--r--    1 root     root       866.9k May  9 09:13 libcyassl.a
> -rwxr-xr-x    1 root     root       659.6k May  9 09:13 libcyassl.so.5.0.0
> 
> # ls -lh /usr/local/ssl/fips-2.0/lib/
> -r--r--r--    1 root     root       685.7k May  9 12:34 fipscanister.o

Wow, putting the whole thing in a single .o file. That's classy. 685k
even for programs that just need a few functions from it.

Rich

Re: High-priority library replacements?

[LM]

[-- Attachment #1: Type: text/plain, Size: 779 bytes --]

On Thu, May 9, 2013 at 10:40 AM, Daniel Cegiełka
<daniel.cegielka@gmail.com>wrote:

>
> This is the result of frustration:
> http://en.wikipedia.org/wiki/WTFPL
> http://www.wtfpl.net/about/
>
> I personally like this one:
http://creativecommons.org/publicdomain/zero/1.0/

However, you're very right about the frustration.  I think the average user
just ignores it and copies things without worrying about the legalities.
However, the whole point of Open Source was to have licenses that gave
certain freedoms that would allow you to safely reuse works.  Out of
respect, if someone goes to the trouble to release code with an Open Source
license, it's nice to at least attempt to comply with the conditions of the
license if one can figure out what those are.

[-- Attachment #2: Type: text/html, Size: 1248 bytes --]

Re: High-priority library replacements?

[Brad Conroy]

IIRC, Axtls was <100kb and has an openSSL API and has a stunnel (axtlswrap) and httpd with ssl (axhttpd) that are <100kb static builds

> > OpeSSL-fips vs CyaSSL (musl)
> > 
> > # ls -lh /var/tmp/ports/cyassl/image/usr/lib/
> > -rw-r--r--    1 root 
>    root       866.9k
> May  9 09:13 libcyassl.a
> > -rwxr-xr-x    1 root 
>    root       659.6k
> May  9 09:13 libcyassl.so.5.0.0
> > 
> > # ls -lh /usr/local/ssl/fips-2.0/lib/
> > -r--r--r--    1 root 
>    root       685.7k
> May  9 12:34 fipscanister.o
> 

In case I don't get around to fully researching it, the kernel spews out major and minor numbers in /sys that are greater than 255, so the makedev in /sys/sysmacros.h is insufficient (also major() and minor()???)  I noticed because I wrote a shell script that populates /dev from /sys in <0.2 seconds but failed for large major/minor numbers when I built busybox with musl.  At first glance, it looks like dev_t needs to be uint and have the extra bits from major and minor prepended as in the kernel's dev_t?

R,
Brad Conroy

Re: High-priority library replacements?

[Rich Felker]

On Sun, May 12, 2013 at 03:42:23PM -0700, Brad Conroy wrote:
> In case I don't get around to fully researching it, the kernel spews
> out major and minor numbers in /sys that are greater than 255, so
> the makedev in /sys/sysmacros.h is insufficient (also major() and
> minor()???) I noticed because I wrote a shell script that populates
> /dev from /sys in <0.2 seconds but failed for large major/minor
> numbers when I built busybox with musl. At first glance, it looks
> like dev_t needs to be uint and have the extra bits from major and
> minor prepended as in the kernel's dev_t?

Should be fixed in git. Let me know if it doesn't work right.

Rich

Re: High-priority library replacements?

[Justin Cormack]

On Wed, May 15, 2013 at 9:17 PM, Rich Felker <dalias@aerifal.cx> wrote:
> On Sun, May 12, 2013 at 03:42:23PM -0700, Brad Conroy wrote:
>> In case I don't get around to fully researching it, the kernel spews
>> out major and minor numbers in /sys that are greater than 255, so
>> the makedev in /sys/sysmacros.h is insufficient (also major() and
>> minor()???) I noticed because I wrote a shell script that populates
>> /dev from /sys in <0.2 seconds but failed for large major/minor
>> numbers when I built busybox with musl. At first glance, it looks
>> like dev_t needs to be uint and have the extra bits from major and
>> minor prepended as in the kernel's dev_t?
>
> Should be fixed in git. Let me know if it doesn't work right.

mknod/mknodat still does return syscall(SYS_mknod, path, mode, dev &
0xffff); - I believe it should pass the full 32 bits.

Also the top 32 bits are only for compatibility with glibc I think,
not sure the macros should touch or use them, as the kernel has 32 bit
dev_t and no plans to change.

Justin

Re: High-priority library replacements?

[Rich Felker]

On Thu, May 16, 2013 at 05:12:04PM +0100, Justin Cormack wrote:
> On Wed, May 15, 2013 at 9:17 PM, Rich Felker <dalias@aerifal.cx> wrote:
> > On Sun, May 12, 2013 at 03:42:23PM -0700, Brad Conroy wrote:
> >> In case I don't get around to fully researching it, the kernel spews
> >> out major and minor numbers in /sys that are greater than 255, so
> >> the makedev in /sys/sysmacros.h is insufficient (also major() and
> >> minor()???) I noticed because I wrote a shell script that populates
> >> /dev from /sys in <0.2 seconds but failed for large major/minor
> >> numbers when I built busybox with musl. At first glance, it looks
> >> like dev_t needs to be uint and have the extra bits from major and
> >> minor prepended as in the kernel's dev_t?
> >
> > Should be fixed in git. Let me know if it doesn't work right.
> 
> mknod/mknodat still does return syscall(SYS_mknod, path, mode, dev &
> 0xffff); - I believe it should pass the full 32 bits.
> 
> Also the top 32 bits are only for compatibility with glibc I think,
> not sure the macros should touch or use them, as the kernel has 32 bit
> dev_t and no plans to change.

mknod and mknodat should be fixed now. Anything else? I don't see much
harm in having the macros match what glibc does, and it could be
useful if the kernel allows 64bit device numbers on 64-bit archs. But
if not, there's also an argument to be made for reducing code size by
dropping the part of the macros for the high bits. What do you think?

Rich

Re: High-priority library replacements?

[Justin Cormack]

On Fri, May 17, 2013 at 2:56 AM, Rich Felker <dalias@aerifal.cx> wrote:
> On Thu, May 16, 2013 at 05:12:04PM +0100, Justin Cormack wrote:
>> On Wed, May 15, 2013 at 9:17 PM, Rich Felker <dalias@aerifal.cx> wrote:
>> > On Sun, May 12, 2013 at 03:42:23PM -0700, Brad Conroy wrote:
>> >> In case I don't get around to fully researching it, the kernel spews
>> >> out major and minor numbers in /sys that are greater than 255, so
>> >> the makedev in /sys/sysmacros.h is insufficient (also major() and
>> >> minor()???) I noticed because I wrote a shell script that populates
>> >> /dev from /sys in <0.2 seconds but failed for large major/minor
>> >> numbers when I built busybox with musl. At first glance, it looks
>> >> like dev_t needs to be uint and have the extra bits from major and
>> >> minor prepended as in the kernel's dev_t?
>> >
>> > Should be fixed in git. Let me know if it doesn't work right.
>>
>> mknod/mknodat still does return syscall(SYS_mknod, path, mode, dev &
>> 0xffff); - I believe it should pass the full 32 bits.
>>
>> Also the top 32 bits are only for compatibility with glibc I think,
>> not sure the macros should touch or use them, as the kernel has 32 bit
>> dev_t and no plans to change.
>
> mknod and mknodat should be fixed now. Anything else? I don't see much
> harm in having the macros match what glibc does, and it could be
> useful if the kernel allows 64bit device numbers on 64-bit archs. But
> if not, there's also an argument to be made for reducing code size by
> dropping the part of the macros for the high bits. What do you think?

Given that the struct stat64 the kernel uses specifically defines
st_rdev as unsigned long long, it probably makes sense to leave dev_t
as 64 bits. I would be inclined to not do anything with the extra bits
in the macros though, as the meaning of those bits is undefined, and
might not be defined as glibc does (seems to be based on what NetBSD
does), and changing the macros does not break anything.

Justin

Re: High-priority library replacements?

[Daniel Cegiełka]

2013/4/25 Gregor Pintar <grpintar@gmail.com>:
> Hello.

> I am working on cryptographic library. It's far from being finished.
> I would be very glad, if someone could look at it.
> Currently I have problems with API design so help would be welcome.
> It isn't in git yet.
> You can get it here:
> https://dl.dropboxusercontent.com/u/83450675/kripto/kripto.tar.gz
> I have plans for SSL library on top of it, but it could take years.

@Gregor Pintar
What do you think about using tropicssl on libkripto (aka libssl and
libcrypto in openssl)?

I looked at the difference between PolarSSL and tropicssl and
differences are often purely cosmetic. I think we can apply the
following changes in PolarSSL to tropicssl and thereby gain a small
SSL library under a BSD license.

http://gitorious.org/tropicssl

Does anyone have a better suggestion?

Daniel


> I think best way is not to trust any certificate authority.
> Maybe some certificate p2p protocol could be done?

Re: High-priority library replacements?

[Gregor Pintar]

2013/5/26, Daniel Cegiełka <daniel.cegielka@gmail.com>:
> @Gregor Pintar
> What do you think about using tropicssl on libkripto (aka libssl and
> libcrypto in openssl)?
>
TropicSSL has quite nice API, but I think it would require API changes
(cipher suites) to get full advantage of "dynamic construction" of
algorithms (block cipher + mode + mac = authenticated stream cipher)
and kripto API is not finished yet so I can't really say.

> I looked at the difference between PolarSSL and tropicssl and
> differences are often purely cosmetic. I think we can apply the
> following changes in PolarSSL to tropicssl and thereby gain a small
> SSL library under a BSD license.
>
> http://gitorious.org/tropicssl
>
> Does anyone have a better suggestion?
>
Unless someone really takes a deep look (vulnerabilities, TLS 1.2)
into tropicssl, I think it's better to use PolarSSL for now despite
GPL.

Re: High-priority library replacements?

[Daniel Cegiełka]

2013/5/27 Gregor Pintar <grpintar@gmail.com>:

> Unless someone really takes a deep look (vulnerabilities, TLS 1.2)
> into tropicssl,

I think that in a few weeks, I will have free time to do so. That's
why I care about the opinion, if you are interested to use tropicssl's
BSD code as ssl layer for kripto.

Daniel

Re: High-priority library replacements?

[LM]

[-- Attachment #1: Type: text/plain, Size: 230 bytes --]

Some of you have probably seen this before, but here's another interesting
list of lightweight libraries:
http://suckless.org/rocks
The site also mentions this list with some other alternatives:
http://harmful.cat-v.org/software/

[-- Attachment #2: Type: text/html, Size: 341 bytes --]

Re: High-priority library replacements?

[Gregor Pintar]

2013/5/28, Daniel Cegiełka <daniel.cegielka@gmail.com>:
> 2013/5/27 Gregor Pintar <grpintar@gmail.com>:
>
>> Unless someone really takes a deep look (vulnerabilities, TLS 1.2)
>> into tropicssl,
>
> I think that in a few weeks, I will have free time to do so. That's
> why I care about the opinion, if you are interested to use tropicssl's
> BSD code as ssl layer for kripto.
>
> Daniel
>
Yes, I think it would be good, but API could maybe be little
cleaner/simpler. I would suggest to use prefix (library name) on all
external symbols. PolarSSL and TropicSSL expose too much in header.
I'm think tropicssl compatibility doesn't mean anything as it is not
very popular, but we could add PolarSSL and OpenSSL compatibility
header. I would also look at axTLS's API
(http://axtls.sourceforge.net/dox/group__c__api.html). API is very
important, all other things can be fixed.

Currently I'm quite busy, but maybe I could help as I have problems
with kripto's API. I might get some ideas for it.


Gregor

Re: High-priority library replacements?

[Rob Landley]

On 05/28/2013 08:11:34 AM, LM wrote:
> Some of you have probably seen this before, but here's another  
> interesting
> list of lightweight libraries:
> http://suckless.org/rocks
> The site also mentions this list with some other alternatives:
> http://harmful.cat-v.org/software/

Did anybody ever do a wiki page from the mailing list thread?

Rob

Re: High-priority library replacements?

[LM]

[-- Attachment #1: Type: text/plain, Size: 276 bytes --]

On Tue, May 28, 2013 at 5:38 PM, Rob Landley <rob@landley.net> wrote:

> Did anybody ever do a wiki page from the mailing list thread?
>

If nobody's volunteered, I can put the links we've mentioned on the list
plus some others up on a page at the wiki when I have some time.

[-- Attachment #2: Type: text/html, Size: 580 bytes --]

Re: High-priority library replacements?

[LM]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

Forgot to mention, I'd need permissions to create/edit a page first.  I can
login to the wiki, but I can't add anything at this point.

On Fri, May 31, 2013 at 7:13 AM, LM <lmemsm@gmail.com> wrote:

> On Tue, May 28, 2013 at 5:38 PM, Rob Landley <rob@landley.net> wrote:
>
>> Did anybody ever do a wiki page from the mailing list thread?
>>
>
> If nobody's volunteered, I can put the links we've mentioned on the list
> plus some others up on a page at the wiki when I have some time.
>

[-- Attachment #2: Type: text/html, Size: 1021 bytes --]