New articles on ewontfix

New articles on ewontfix

[Rich Felker]

Hi all,

I've posted a few new articles on ewontfix.com based on experience
from problems encountered by people using musl:

  Breakincludes
  http://ewontfix.com/12

  NULL considered harmful
  http://ewontfix.com/11

The latter seems to be something Jens has covered in the past, too. :)

Rich

Re: New articles on ewontfix

[Jeremy Huntwork]

[-- Attachment #1: Type: text/plain, Size: 332 bytes --]

On Thursday, July 4, 2013 at 12:48 PM, Rich Felker wrote:
> Hi all,
> 
> I've posted a few new articles on ewontfix.com based on experience
> from problems encountered by people using musl:
> 
> Breakincludes
> http://ewontfix.com/12
> 
> NULL considered harmful
> http://ewontfix.com/11
> 


I really enjoyed these, thanks!

JH  


[-- Attachment #2: Type: text/html, Size: 804 bytes --]

Re: New articles on ewontfix

[Rich Felker]

On Thu, Jul 04, 2013 at 01:50:15PM -0400, Jeremy Huntwork wrote:
> On Thursday, July 4, 2013 at 12:48 PM, Rich Felker wrote:
> > Hi all,
> > 
> > I've posted a few new articles on ewontfix.com based on experience
> > from problems encountered by people using musl:
> > 
> > Breakincludes
> > http://ewontfix.com/12
> > 
> > NULL considered harmful
> > http://ewontfix.com/11
> 
> I really enjoyed these, thanks!

Thanks for the feedback. BTW, I think it's really nice how the
widespread adoption of version control systems and web interfaces for
them has made it possible to cite source so easily, in such a way that
readers can go directly to the code I'm talking about and see for
themselves what it's doing.

Rich

Re: New articles on ewontfix

[Jeremy Huntwork]

On Jul 4, 2013, at 1:58 PM, Rich Felker <dalias@aerifal.cx> wrote:

>> I really enjoyed these, thanks!
> 
> Thanks for the feedback. BTW, I think it's really nice how the
> widespread adoption of version control systems and web interfaces for
> them has made it possible to cite source so easily, in such a way that
> readers can go directly to the code I'm talking about and see for
> themselves what it's doing.

Yep, it's at the point now where you can pretty much take it for granted. Only after you commented did I recall what it was like a few years back when typically the best you had was grabbing a cvs snapshot for yourself and examining the code locally.

Rich, you have the rare ability of being able communicate technical details in a way that is easy to understand and not horribly dry to read. If you ever have time and motivation, I would love to read an article from you about what makes good software.  For example, pitfalls to avoid in C (esp. ones that even experienced devs fall into). Or good design decisions on the Unix/Linux platform vs poor design decisions - writing good code that is both efficient and scalable. 

I know you're busy and many of these things one can find piecemeal on the internets. (E.g., Rob Landley has some good thoughts on his toybox list about cleaning up code contributions to toybox.) Still, having one good article that is a joy to read makes a huge difference.

JH

Re: New articles on ewontfix

[Ivan Kanakarakis]

[-- Attachment #1: Type: text/plain, Size: 1648 bytes --]

I am waiting for the "library-safe" article ;)




On 5 July 2013 18:31, Jeremy Huntwork <jhuntwork@lightcubesolutions.com>wrote:

> On Jul 4, 2013, at 1:58 PM, Rich Felker <dalias@aerifal.cx> wrote:
>
> >> I really enjoyed these, thanks!
> >
> > Thanks for the feedback. BTW, I think it's really nice how the
> > widespread adoption of version control systems and web interfaces for
> > them has made it possible to cite source so easily, in such a way that
> > readers can go directly to the code I'm talking about and see for
> > themselves what it's doing.
>
> Yep, it's at the point now where you can pretty much take it for granted.
> Only after you commented did I recall what it was like a few years back
> when typically the best you had was grabbing a cvs snapshot for yourself
> and examining the code locally.
>
> Rich, you have the rare ability of being able communicate technical
> details in a way that is easy to understand and not horribly dry to read.
> If you ever have time and motivation, I would love to read an article from
> you about what makes good software.  For example, pitfalls to avoid in C
> (esp. ones that even experienced devs fall into). Or good design decisions
> on the Unix/Linux platform vs poor design decisions - writing good code
> that is both efficient and scalable.
>
> I know you're busy and many of these things one can find piecemeal on the
> internets. (E.g., Rob Landley has some good thoughts on his toybox list
> about cleaning up code contributions to toybox.) Still, having one good
> article that is a joy to read makes a huge difference.
>
> JH




-- 
*Ivan c00kiemon5ter Kanakarakis*  >:3

[-- Attachment #2: Type: text/html, Size: 2521 bytes --]

Re: New articles on ewontfix

[Rich Felker]

On Fri, Jul 05, 2013 at 06:45:27PM +0300, Ivan Kanakarakis wrote:
> I am waiting for the "library-safe" article ;)

I'm still trying to determine how to work out a formal definition of
library-safe. My thought is that it would be based on the property of
being able to combine two programs with well-defined behavior, both
using the library code, into a single program where each original
program runs starting with its own initial thread, such that the
combined program does not invoke UB and the two sub-programs match
their behavior before being combined. However there are lots of ugly
issues that have to be considered.

With that done, the interesting part would be covering common failures
of libraries to be library-safe.

Rich

Re: New articles on ewontfix

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2013-07-05 11:54:11 -0400]:
> I'm still trying to determine how to work out a formal definition of
> library-safe. My thought is that it would be based on the property of
> being able to combine two programs with well-defined behavior, both
> using the library code, into a single program where each original
> program runs starting with its own initial thread, such that the
> combined program does not invoke UB and the two sub-programs match
> their behavior before being combined. However there are lots of ugly
> issues that have to be considered.
> 
> With that done, the interesting part would be covering common failures
> of libraries to be library-safe.

i'm not sure if you can derive all the interesting failures from a
single definition

this definition covers multi-thread issues

i think library safety should also cover single thread issues
(abort in lib, unbounded resource usage, trusting input sources
unjustifiably, strong assumtions about the environment..)

and source code level issues
(abi, visibility, namespace pollution, clean headers..)

Re: New articles on ewontfix

[Rich Felker]

On Sun, Jul 07, 2013 at 02:20:15PM +0200, Szabolcs Nagy wrote:
> * Rich Felker <dalias@aerifal.cx> [2013-07-05 11:54:11 -0400]:
> > I'm still trying to determine how to work out a formal definition of
> > library-safe. My thought is that it would be based on the property of
> > being able to combine two programs with well-defined behavior, both
> > using the library code, into a single program where each original
> > program runs starting with its own initial thread, such that the
> > combined program does not invoke UB and the two sub-programs match
> > their behavior before being combined. However there are lots of ugly
> > issues that have to be considered.
> > 
> > With that done, the interesting part would be covering common failures
> > of libraries to be library-safe.
> 
> i'm not sure if you can derive all the interesting failures from a
> single definition
> 
> this definition covers multi-thread issues
> 
> i think library safety should also cover single thread issues

It attempts to. Having the "test" be with threads automatically covers
all cases of using the library separately from multiple modules.

> (abort in lib,

This is covered. Just have program A cause the library to abort, and
program B run successfully. Then the combined program wrongly aborts
part B, and thus it's non-library-safe.

> unbounded resource usage,

I don't see how this can be quantified correctly, but in some sense,
it is by the proposed definition. If part A consumes so many resources
that part B can't run, that would be a failure of the test. However
I'm reluctant to call that a failure since it could make any library
fail. This is why the definition is difficult to get right.

> trusting input sources
> unjustifiably,

I think that's more an issue of security, but the proposed definition
precludes UB anyway.

> strong assumtions about the environment..)

Could you elaborate?

> and source code level issues
> (abi, visibility, namespace pollution, clean headers..)

Yes, those are possibly issues that should be covered.

Rich

Re: New articles on ewontfix

[Daniel Cegiełka]

2013/7/7 Rich Felker <dalias@aerifal.cx>:
> On Sun, Jul 07, 2013 at 02:20:15PM +0200, Szabolcs Nagy wrote:
>> * Rich Felker <dalias@aerifal.cx> [2013-07-05 11:54:11 -0400]:
>> > I'm still trying to determine how to work out a formal definition of
>> > library-safe. My thought is that it would be based on the property of
>> > being able to combine two programs with well-defined behavior, both
>> > using the library code, into a single program where each original
>> > program runs starting with its own initial thread, such that the
>> > combined program does not invoke UB and the two sub-programs match
>> > their behavior before being combined. However there are lots of ugly
>> > issues that have to be considered.
>> >
>> > With that done, the interesting part would be covering common failures
>> > of libraries to be library-safe.
>>
>> i'm not sure if you can derive all the interesting failures from a
>> single definition
>>
>> this definition covers multi-thread issues

Lukasz Sowa (blowfish diff for musl) has prepared a very good tool for
debugging multi-threaded applications.

https://github.com/luksow/Coconut

I hope that it will be useful in testing...

Daniel

Re: New articles on ewontfix

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2013-07-07 11:27:40 -0400]:
> On Sun, Jul 07, 2013 at 02:20:15PM +0200, Szabolcs Nagy wrote:
> > * Rich Felker <dalias@aerifal.cx> [2013-07-05 11:54:11 -0400]:
> > > My thought is that it would be based on the property of
> > > being able to combine two programs with well-defined behavior, both
> > > using the library code, into a single program where each original
> > > program runs starting with its own initial thread, such that the
> > > combined program does not invoke UB and the two sub-programs match
> > > their behavior before being combined.
> > 
> > i think library safety should also cover single thread issues
> 
> It attempts to. Having the "test" be with threads automatically covers
> all cases of using the library separately from multiple modules.
> 

ah ok

but then "program with well-defined behavior" is hard
to specify

(i thought you assume working programs and only require
that their combination does not break)

if well-defined can be any program that the language and
the library documentation allows in a single-threaded
execution then the program itself may invoke ub in
multi-threaded case

and a library interface can require a callback that
does impossible things so no program is well-defined

> > unbounded resource usage,
> 
> I don't see how this can be quantified correctly, but in some sense,
> it is by the proposed definition. If part A consumes so many resources
> that part B can't run, that would be a failure of the test. However
> I'm reluctant to call that a failure since it could make any library
> fail. This is why the definition is difficult to get right.

if the library documents its resource usage then it can pass
the strong test

(and there are per-thread resources: stack)

we also want that low resources or runtime failures are
handled and don't cause ub: so the runtime environment
should be part of the definition in some way

by unbounded resources i originally meant resource leaks,
but "resource safety" seems to be hard to specify in general

> > strong assumtions about the environment..)
> 
> Could you elaborate?

by environment i meant the system surrounding the program

and strong assumption is anything that is not guaranteed

eg if a library tries to connect to some webserver to
get some information that is present locally as well,
then it assumes internet connection unjustifiably

Re: New articles on ewontfix

[Rich Felker]

On Sun, Jul 07, 2013 at 07:12:03PM +0200, Szabolcs Nagy wrote:
> > It attempts to. Having the "test" be with threads automatically covers
> > all cases of using the library separately from multiple modules.
> > 
> 
> ah ok
> 
> but then "program with well-defined behavior" is hard
> to specify

:)

> (i thought you assume working programs and only require
> that their combination does not break)

The idea is that a program is non-library-safe if there exists a pair
of programs, subject to certain conditions on the programs that remain
difficult to specify exactly, that do not individually invoke UB but
which invoke UB or whose outputs change for some interleaving.

> if well-defined can be any program that the language and
> the library documentation allows in a single-threaded
> execution then the program itself may invoke ub in
> multi-threaded case

Definitely not any program. For instance the programs must be
restricted not to define symbols that conflict with one another, or to
access the filesystem in ways that could conflict with one another,
etc. They should probably have no input at all. The difficulty is
putting these restrictions in place in such a way that we don't
gratuitously exclude libraries that have legitimate needs to access
the filesystem or network and which do so in safe ways.

> and a library interface can require a callback that
> does impossible things so no program is well-defined

I'm not sure what you mean.

> > > unbounded resource usage,
> > 
> > I don't see how this can be quantified correctly, but in some sense,
> > it is by the proposed definition. If part A consumes so many resources
> > that part B can't run, that would be a failure of the test. However
> > I'm reluctant to call that a failure since it could make any library
> > fail. This is why the definition is difficult to get right.
> 
> if the library documents its resource usage then it can pass
> the strong test
> 
> (and there are per-thread resources: stack)

Yes, the threads for A and B would be assumed to start with the same
amount of stack space they would have if they were the main thread.

> we also want that low resources or runtime failures are
> handled and don't cause ub: so the runtime environment
> should be part of the definition in some way
> 
> by unbounded resources i originally meant resource leaks,
> but "resource safety" seems to be hard to specify in general

Yes, and in some sense I think it's a separate issue.

> > > strong assumtions about the environment..)
> > 
> > Could you elaborate?
> 
> by environment i meant the system surrounding the program
> 
> and strong assumption is anything that is not guaranteed
> 
> eg if a library tries to connect to some webserver to
> get some information that is present locally as well,
> then it assumes internet connection unjustifiably

Yes, these are the most difficult since the behavior of the program is
really not self-contained. I think any formal treatment would have to
assume the network has been dummied-out and replaced with a fixed set
of responses to everything the program sends.

Rich

Re: New articles on ewontfix

[LM]

[-- Attachment #1: Type: text/plain, Size: 541 bytes --]

On Thu, Jul 4, 2013 at 12:48 PM, Rich Felker <dalias@aerifal.cx> wrote:

> I've posted a few new articles on ewontfix.com based on experience
> from problems encountered by people using musl:
>

That is a really clever idea for a web site.  I've run into several bugs
and issues that Open Source projects refuse to fix and don't want to have
anything to do with.  Hope it inspires some people to do something about
the situation even if it means switching to other Open Source software that
doesn't have the same issues.

Best wishes.
Laura

[-- Attachment #2: Type: text/html, Size: 866 bytes --]