getcontext etc was Re: [musl] Re: go support (was: Best place to discuss other lightweight libraries?)

getcontext etc was Re: [musl] Re: go support (was: Best place to discuss other lightweight libraries?)

[Justin Cormack]

On Wed, Apr 24, 2013 at 2:42 PM, Rich Felker <dalias@aerifal.cx> wrote:
>> i tried to build gcc 4.7.2 with go support (--enable-languages=c,c++,go)
>> and that fails due to a lack of set/getcontext().
>> (see pkg/gcc472 in sabotage)
>>
>> according to rich, adding that to musl requires a non-trivial amount
>> of arch specific asm.
>
> Yes, but it is a wanted feature, so I wouldn't mind it getting done.
> It was even part of the standard prior to POSIX 2008, and the reason
> for removing it was stupid. (The reason was that the makecontext
> function's calling convention is bogus and impossible to support
> properly, but they could have fixed this by deprecating the use of the
> variadic arguments in any way except passing a single void* argument,
> rather than deprecating the whole set of interfaces.)

Just to revive this post 1.0, it would be very nice indeed to have
set,get,swapcontext.

Justin

Re: getcontext etc was Re: [musl] Re: go support (was: Best place to discuss other lightweight libraries?)

[Rich Felker]

On Mon, Jun 23, 2014 at 10:09:29PM +0100, Justin Cormack wrote:
> On Wed, Apr 24, 2013 at 2:42 PM, Rich Felker <dalias@aerifal.cx> wrote:
> >> i tried to build gcc 4.7.2 with go support (--enable-languages=c,c++,go)
> >> and that fails due to a lack of set/getcontext().
> >> (see pkg/gcc472 in sabotage)
> >>
> >> according to rich, adding that to musl requires a non-trivial amount
> >> of arch specific asm.
> >
> > Yes, but it is a wanted feature, so I wouldn't mind it getting done.
> > It was even part of the standard prior to POSIX 2008, and the reason
> > for removing it was stupid. (The reason was that the makecontext
> > function's calling convention is bogus and impossible to support
> > properly, but they could have fixed this by deprecating the use of the
> > variadic arguments in any way except passing a single void* argument,
> > rather than deprecating the whole set of interfaces.)
> 
> Just to revive this post 1.0, it would be very nice indeed to have
> set,get,swapcontext.

One approach to setcontext would be using rt_sigreturn, but I've heard
there are issues using it for setcontext involving the alternate
signal stack. It's the only way to do resuming _async_ (i.e. the
ucontext received by a signal handler for the state it interrupted)
contexts though, so maybe there's some way to make it work. This issue
was discussed recently on libc-alpha (the glibc list).

Otherwise, if async contexts don't need to be supported, these
functions are essentially just mechanical transformations of
sigsetjmp/siglongjmp. I think setcontext could be implemented just as
a transformation of the mcontext_t register values into a jmp_buf,
followed by a call to siglongjmp. For getcontext, you can't just wrap
sigsetjmp since wrapping returns-twice functions is impossible. But
maybe there's some way to make it work without code duplication.

Rich

Re: getcontext etc was Re: [musl] Re: go support (was: Best place to discuss other lightweight libraries?)

[Justin Cormack]

[-- Attachment #1: Type: text/plain, Size: 1261 bytes --]

On Jun 23, 2014 11:32 PM, "Rich Felker" <dalias@libc.org> wrote:
> > Just to revive this post 1.0, it would be very nice indeed to have
> > set,get,swapcontext.
>
> One approach to setcontext would be using rt_sigreturn, but I've heard
> there are issues using it for setcontext involving the alternate
> signal stack. It's the only way to do resuming _async_ (i.e. the
> ucontext received by a signal handler for the state it interrupted)
> contexts though, so maybe there's some way to make it work. This issue
> was discussed recently on libc-alpha (the glibc list).

I am not interested in the async case not sure if other users are. If I
read the glibc discussion correctly they are not supporting this any more?

> Otherwise, if async contexts don't need to be supported, these
> functions are essentially just mechanical transformations of
> sigsetjmp/siglongjmp. I think setcontext could be implemented just as
> a transformation of the mcontext_t register values into a jmp_buf,
> followed by a call to siglongjmp. For getcontext, you can't just wrap
> sigsetjmp since wrapping returns-twice functions is impossible. But
> maybe there's some way to make it work without code duplication.
>

This sounds promising. Will take a look at the code.

Justin

[-- Attachment #2: Type: text/html, Size: 1545 bytes --]

Re: getcontext etc was Re: [musl] Re: go support (was: Best place to discuss other lightweight libraries?)

[Timo Teras]

On Tue, 24 Jun 2014 11:05:01 +0100
Justin Cormack <justin@specialbusservice.com> wrote:

> On Jun 23, 2014 11:32 PM, "Rich Felker" <dalias@libc.org> wrote:
> > > Just to revive this post 1.0, it would be very nice indeed to have
> > > set,get,swapcontext.
> >
> > One approach to setcontext would be using rt_sigreturn, but I've
> > heard there are issues using it for setcontext involving the
> > alternate signal stack. It's the only way to do resuming _async_
> > (i.e. the ucontext received by a signal handler for the state it
> > interrupted) contexts though, so maybe there's some way to make it
> > work. This issue was discussed recently on libc-alpha (the glibc
> > list).
> 
> I am not interested in the async case not sure if other users are. If
> I read the glibc discussion correctly they are not supporting this
> any more?

I think that's intended. The whole idea of *context is to be able to do
user-space stack switching to implement co-operative threading (or
"fibers" or "green threads" or whet ever you want to call them). Being
async safe is not requirement for this.

setcontext should be fast or the whole point of it is lost. So it
should not do syscalls (signal mask twiddling).

/Timo

Re: getcontext etc was Re: [musl] Re: go support (was: Best place to discuss other lightweight libraries?)

[Rich Felker]

On Tue, Jun 24, 2014 at 01:12:44PM +0300, Timo Teras wrote:
> On Tue, 24 Jun 2014 11:05:01 +0100
> Justin Cormack <justin@specialbusservice.com> wrote:
> 
> > On Jun 23, 2014 11:32 PM, "Rich Felker" <dalias@libc.org> wrote:
> > > > Just to revive this post 1.0, it would be very nice indeed to have
> > > > set,get,swapcontext.
> > >
> > > One approach to setcontext would be using rt_sigreturn, but I've
> > > heard there are issues using it for setcontext involving the
> > > alternate signal stack. It's the only way to do resuming _async_
> > > (i.e. the ucontext received by a signal handler for the state it
> > > interrupted) contexts though, so maybe there's some way to make it
> > > work. This issue was discussed recently on libc-alpha (the glibc
> > > list).
> > 
> > I am not interested in the async case not sure if other users are. If
> > I read the glibc discussion correctly they are not supporting this
> > any more?
> 
> I think that's intended. The whole idea of *context is to be able to do
> user-space stack switching to implement co-operative threading (or
> "fibers" or "green threads" or whet ever you want to call them). Being
> async safe is not requirement for this.
> 
> setcontext should be fast or the whole point of it is lost. So it
> should not do syscalls (signal mask twiddling).

Saving and restoring the signal mask is required anyway; this has
nothing to do with switching between async signal contexts, but simply
the fact that individual contexts can have their signal masks changed
manually. One benefit of using rt_sigreturn, if we could, is that it
would restore the signal mask automatically along with doing the
register context restoration.

Rich

Re: getcontext etc was Re: [musl] Re: go support (was: Best place to discuss other lightweight libraries?)

[Andy Lutomirski]

On 06/23/2014 03:32 PM, Rich Felker wrote:
> On Mon, Jun 23, 2014 at 10:09:29PM +0100, Justin Cormack wrote:
>> On Wed, Apr 24, 2013 at 2:42 PM, Rich Felker <dalias@aerifal.cx> wrote:
>>>> i tried to build gcc 4.7.2 with go support (--enable-languages=c,c++,go)
>>>> and that fails due to a lack of set/getcontext().
>>>> (see pkg/gcc472 in sabotage)
>>>>
>>>> according to rich, adding that to musl requires a non-trivial amount
>>>> of arch specific asm.
>>>
>>> Yes, but it is a wanted feature, so I wouldn't mind it getting done.
>>> It was even part of the standard prior to POSIX 2008, and the reason
>>> for removing it was stupid. (The reason was that the makecontext
>>> function's calling convention is bogus and impossible to support
>>> properly, but they could have fixed this by deprecating the use of the
>>> variadic arguments in any way except passing a single void* argument,
>>> rather than deprecating the whole set of interfaces.)
>>
>> Just to revive this post 1.0, it would be very nice indeed to have
>> set,get,swapcontext.
> 
> One approach to setcontext would be using rt_sigreturn, but I've heard
> there are issues using it for setcontext involving the alternate
> signal stack. It's the only way to do resuming _async_ (i.e. the
> ucontext received by a signal handler for the state it interrupted)
> contexts though, so maybe there's some way to make it work. This issue
> was discussed recently on libc-alpha (the glibc list).

What's the async issue?

I think that using rt_sigreturn for this directly (rather than sending
yourself a signal, adjusting the context, and returning) is a bad idea:
there are patches floating around to mitigate SROP (sigreturn-oriented
programming) by adding signal frame canaries, and you'll go boom if that
happens.

--Andy

Re: Re: getcontext etc was Re: [musl] Re: go support (was: Best place to discuss other lightweight libraries?)

[Rich Felker]

On Fri, Jun 27, 2014 at 12:41:11PM -0700, Andy Lutomirski wrote:
> On 06/23/2014 03:32 PM, Rich Felker wrote:
> > On Mon, Jun 23, 2014 at 10:09:29PM +0100, Justin Cormack wrote:
> >> On Wed, Apr 24, 2013 at 2:42 PM, Rich Felker <dalias@aerifal.cx> wrote:
> >>>> i tried to build gcc 4.7.2 with go support (--enable-languages=c,c++,go)
> >>>> and that fails due to a lack of set/getcontext().
> >>>> (see pkg/gcc472 in sabotage)
> >>>>
> >>>> according to rich, adding that to musl requires a non-trivial amount
> >>>> of arch specific asm.
> >>>
> >>> Yes, but it is a wanted feature, so I wouldn't mind it getting done.
> >>> It was even part of the standard prior to POSIX 2008, and the reason
> >>> for removing it was stupid. (The reason was that the makecontext
> >>> function's calling convention is bogus and impossible to support
> >>> properly, but they could have fixed this by deprecating the use of the
> >>> variadic arguments in any way except passing a single void* argument,
> >>> rather than deprecating the whole set of interfaces.)
> >>
> >> Just to revive this post 1.0, it would be very nice indeed to have
> >> set,get,swapcontext.
> > 
> > One approach to setcontext would be using rt_sigreturn, but I've heard
> > there are issues using it for setcontext involving the alternate
> > signal stack. It's the only way to do resuming _async_ (i.e. the
> > ucontext received by a signal handler for the state it interrupted)
> > contexts though, so maybe there's some way to make it work. This issue
> > was discussed recently on libc-alpha (the glibc list).
> 
> What's the async issue?

It's impossible to do setcontext to an async context (given to you via
the kernel as the ucontext_t argument to a signal handler) without
rt_sigreturn since the context may store cpu-model-specific
information in the context beyond what userspace is aware of. For
example, if libc is only built for the baseline ISA but the
application is using some SIMD extensions, the kernel may save the
SIMD state at the end of the context in some form that's not a public
API, and libc has no way to restore that.

> I think that using rt_sigreturn for this directly (rather than sending
> yourself a signal, adjusting the context, and returning) is a bad idea:
> there are patches floating around to mitigate SROP (sigreturn-oriented
> programming) by adding signal frame canaries, and you'll go boom if that
> happens.

Can your provide me with links to threads on the topic? Depending on
how they work, there's a small chance they could break cancellation
behavior on musl (and make it impossible to do safe cancellation), and
if so I want to make sure that issue is raised and fixed before the
problem gets integrated anywhere. If it's just a canary it's no
problem, but if they did some kind of checksum of the context, that
would be a big problem.

Also, it's possible that we could support kernel-provided and
userspace-generated contexts separately as long as there's space to
store which type a given context is.

Rich

Re: getcontext etc was Re: [musl] Re: go support (was: Best place to discuss other lightweight libraries?)

[Andy Lutomirski]

On 06/27/2014 01:36 PM, Rich Felker wrote:
> On Fri, Jun 27, 2014 at 12:41:11PM -0700, Andy Lutomirski wrote:
>> On 06/23/2014 03:32 PM, Rich Felker wrote:
>>> On Mon, Jun 23, 2014 at 10:09:29PM +0100, Justin Cormack wrote:
>>>> On Wed, Apr 24, 2013 at 2:42 PM, Rich Felker <dalias@aerifal.cx> wrote:
>>>>>> i tried to build gcc 4.7.2 with go support (--enable-languages=c,c++,go)
>>>>>> and that fails due to a lack of set/getcontext().
>>>>>> (see pkg/gcc472 in sabotage)
>>>>>>
>>>>>> according to rich, adding that to musl requires a non-trivial amount
>>>>>> of arch specific asm.
>>>>>
>>>>> Yes, but it is a wanted feature, so I wouldn't mind it getting done.
>>>>> It was even part of the standard prior to POSIX 2008, and the reason
>>>>> for removing it was stupid. (The reason was that the makecontext
>>>>> function's calling convention is bogus and impossible to support
>>>>> properly, but they could have fixed this by deprecating the use of the
>>>>> variadic arguments in any way except passing a single void* argument,
>>>>> rather than deprecating the whole set of interfaces.)
>>>>
>>>> Just to revive this post 1.0, it would be very nice indeed to have
>>>> set,get,swapcontext.
>>>
>>> One approach to setcontext would be using rt_sigreturn, but I've heard
>>> there are issues using it for setcontext involving the alternate
>>> signal stack. It's the only way to do resuming _async_ (i.e. the
>>> ucontext received by a signal handler for the state it interrupted)
>>> contexts though, so maybe there's some way to make it work. This issue
>>> was discussed recently on libc-alpha (the glibc list).
>>
>> What's the async issue?
> 
> It's impossible to do setcontext to an async context (given to you via
> the kernel as the ucontext_t argument to a signal handler) without
> rt_sigreturn since the context may store cpu-model-specific
> information in the context beyond what userspace is aware of. For
> example, if libc is only built for the baseline ISA but the
> application is using some SIMD extensions, the kernel may save the
> SIMD state at the end of the context in some form that's not a public
> API, and libc has no way to restore that.

Right.

> 
>> I think that using rt_sigreturn for this directly (rather than sending
>> yourself a signal, adjusting the context, and returning) is a bad idea:
>> there are patches floating around to mitigate SROP (sigreturn-oriented
>> programming) by adding signal frame canaries, and you'll go boom if that
>> happens.
> 
> Can your provide me with links to threads on the topic? Depending on
> how they work, there's a small chance they could break cancellation
> behavior on musl (and make it impossible to do safe cancellation), and
> if so I want to make sure that issue is raised and fixed before the
> problem gets integrated anywhere. If it's just a canary it's no
> problem, but if they did some kind of checksum of the context, that
> would be a big problem.

http://lkml.kernel.org/g/20140515210914.GA9536@pizzadoos.com

I haven't spotted any more on the subject, though.

--Andy