* libhybris and musl? @ 2014-09-01 8:55 Jens Staal 2014-09-01 21:08 ` Rich Felker 0 siblings, 1 reply; 9+ messages in thread From: Jens Staal @ 2014-09-01 8:55 UTC (permalink / raw) To: musl Hi Has anyone tried if libhybris will emulate the bionic libc properly on top of musl libc? Alternatively, are there plans for bionic binary/source compatibility in musl like the glibc compatibility (or would this contribute to bloat?). I was mostly curious to look into the possiblility of building AOSP (or Mer) with musl instead of bionic (or glibc in the case of Mer), but for Android app (those built with NDK) compatibility, one would then need libhybris. right now just thinking aloud - no real plans at the moment :) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: libhybris and musl? 2014-09-01 8:55 libhybris and musl? Jens Staal @ 2014-09-01 21:08 ` Rich Felker 2014-09-02 7:11 ` Wermut 2014-09-03 22:59 ` Rich Felker 0 siblings, 2 replies; 9+ messages in thread From: Rich Felker @ 2014-09-01 21:08 UTC (permalink / raw) To: musl On Mon, Sep 01, 2014 at 10:55:26AM +0200, Jens Staal wrote: > Hi > > Has anyone tried if libhybris will emulate the bionic libc properly on top of > musl libc? Alternatively, are there plans for bionic binary/source > compatibility in musl like the glibc compatibility (or would this contribute > to bloat?). > > I was mostly curious to look into the possiblility of building AOSP (or Mer) > with musl instead of bionic (or glibc in the case of Mer), but for Android app > (those built with NDK) compatibility, one would then need libhybris. > > right now just thinking aloud - no real plans at the moment :) While I haven't look at the details for how it works, I'm fairly doubtful that using libhybris would be practical. My guess is that you'll find it fairly glibc-specific, but perhaps more importantly I'm not sure what benefit you would get from having apps built against Bionic using musl. If you have particular aims in mind, let me know, but my feeling is that it would be a lot more practical to just fix up the things Bionic is doing that are gratuitously different/broken to get to a point where programs using Bionic run cleanly side by side with musl. But if the issue is binary apps using native NDK code with the Bionic ABI, but where you want the non-native code to run on a musl-linked Dalvik rather than a Bionic-linked one, and if this all takes place in the same process context, then maybe something like libhybris really is needed. I'm quite interested in this project as a prototype for how Android could be moved from Bionic to musl, and (optionally, but I think this is the right direction) steered back from its gratuitous incompatibilities in filesystem layout, handling of dns, user/group lookups, etc. and re-unified with Linux. Please keep us informed if you make any interesting progress! Rich ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: libhybris and musl? 2014-09-01 21:08 ` Rich Felker @ 2014-09-02 7:11 ` Wermut 2014-09-02 8:34 ` Rich Felker 2014-09-03 22:59 ` Rich Felker 1 sibling, 1 reply; 9+ messages in thread From: Wermut @ 2014-09-02 7:11 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 3183 bytes --] We used libhybris for one single thing. Getting Android/bionic linked graphic drivers up and running under embedded glibc linux. Since all the hardware vendors startet to only deliver their drivers in binary form and exclusivly for Android, these is usually the only way to get GPU up and running. libhybris was originally developed for Sailfish OS that is using Android drivers of all kind under a glibc enviroment. The graphic Stack is Android OpenGL driver > libhybris > wayland. Other projects like the Intel/Samsung "mer" project is doing exactly the same. If you are interessted how to install and use it, there is a rudimentary how to: http://www.cnx-software.com/2013/04/21/how-to-use-libhybris-and-android-gpu-libraries-with-mer-linux-on-the-cubieboard/ This thing is a therible hack, but unfortunatly this is often the only way to get things like OpenGL, hw video decoding etc. working on embedded arm linux. There are no signs that companies like Qualcom or ARM themself with their Mali GPU will deliver open source drivers. Linking your own apps to bionic/libhybris makes absolutly no sense. The reason for libhybris is to use bionic binaries (drivers) within glibc based enviroments. On Mon, Sep 1, 2014 at 11:08 PM, Rich Felker <dalias@libc.org> wrote: > On Mon, Sep 01, 2014 at 10:55:26AM +0200, Jens Staal wrote: > > Hi > > > > Has anyone tried if libhybris will emulate the bionic libc properly on > top of > > musl libc? Alternatively, are there plans for bionic binary/source > > compatibility in musl like the glibc compatibility (or would this > contribute > > to bloat?). > > > > I was mostly curious to look into the possiblility of building AOSP (or > Mer) > > with musl instead of bionic (or glibc in the case of Mer), but for > Android app > > (those built with NDK) compatibility, one would then need libhybris. > > > > right now just thinking aloud - no real plans at the moment :) > > While I haven't look at the details for how it works, I'm fairly > doubtful that using libhybris would be practical. My guess is that > you'll find it fairly glibc-specific, but perhaps more importantly I'm > not sure what benefit you would get from having apps built against > Bionic using musl. If you have particular aims in mind, let me know, > but my feeling is that it would be a lot more practical to just fix up > the things Bionic is doing that are gratuitously different/broken to > get to a point where programs using Bionic run cleanly side by side > with musl. But if the issue is binary apps using native NDK code with > the Bionic ABI, but where you want the non-native code to run on a > musl-linked Dalvik rather than a Bionic-linked one, and if this all > takes place in the same process context, then maybe something like > libhybris really is needed. > > I'm quite interested in this project as a prototype for how Android > could be moved from Bionic to musl, and (optionally, but I think this > is the right direction) steered back from its gratuitous > incompatibilities in filesystem layout, handling of dns, user/group > lookups, etc. and re-unified with Linux. Please keep us informed if > you make any interesting progress! > > Rich > [-- Attachment #2: Type: text/html, Size: 3997 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: libhybris and musl? 2014-09-02 7:11 ` Wermut @ 2014-09-02 8:34 ` Rich Felker 2014-09-02 9:25 ` Wermut 0 siblings, 1 reply; 9+ messages in thread From: Rich Felker @ 2014-09-02 8:34 UTC (permalink / raw) To: musl On Tue, Sep 02, 2014 at 09:11:33AM +0200, Wermut wrote: > We used libhybris for one single thing. Getting Android/bionic linked > graphic drivers up and running under embedded glibc linux. Since all the > hardware vendors startet to only deliver their drivers in binary form and > exclusivly for Android, these is usually the only way to get GPU up and > running. > > libhybris was originally developed for Sailfish OS that is using Android > drivers of all kind under a glibc enviroment. The graphic Stack is Android > OpenGL driver > libhybris > wayland. Other projects like the Intel/Samsung > "mer" project is doing exactly the same. > > If you are interessted how to install and use it, there is a rudimentary > how to: > http://www.cnx-software.com/2013/04/21/how-to-use-libhybris-and-android-gpu-libraries-with-mer-linux-on-the-cubieboard/ > > This thing is a therible hack, but unfortunatly this is often the only way > to get things like OpenGL, hw video decoding etc. working on embedded arm > linux. There are no signs that companies like Qualcom or ARM themself with > their Mali GPU will deliver open source drivers. > > Linking your own apps to bionic/libhybris makes absolutly no sense. The > reason for libhybris is to use bionic binaries (drivers) within glibc based > enviroments. How does this work when the binary driver is accessing the same symbol names (mainly standard functions, but not necessarily) as the program the driver is loaded into, but the latter needs the real versions in libc and only the former should be routed through a hack layer? Rich ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: libhybris and musl? 2014-09-02 8:34 ` Rich Felker @ 2014-09-02 9:25 ` Wermut 2014-09-03 22:35 ` Rich Felker 0 siblings, 1 reply; 9+ messages in thread From: Wermut @ 2014-09-02 9:25 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 2237 bytes --] I think the following blog post explains the concept nicely: http://groleo.wordpress.com/2013/07/22/brief-intro-to-libhybris/ I simply overrites the bionic symbols and provides glibc compatible calls. This solution works very nicely, but can have problems if threading is involved. Also good references from the original creator of libhybris: http://mer-project.blogspot.fi/2013/04/wayland-utilizing-android-gpu-drivers.html http://mer-project.blogspot.fi/2013/05/wayland-utilizing-android-gpu-drivers.html On Tue, Sep 2, 2014 at 10:34 AM, Rich Felker <dalias@libc.org> wrote: > On Tue, Sep 02, 2014 at 09:11:33AM +0200, Wermut wrote: > > We used libhybris for one single thing. Getting Android/bionic linked > > graphic drivers up and running under embedded glibc linux. Since all the > > hardware vendors startet to only deliver their drivers in binary form and > > exclusivly for Android, these is usually the only way to get GPU up and > > running. > > > > libhybris was originally developed for Sailfish OS that is using Android > > drivers of all kind under a glibc enviroment. The graphic Stack is > Android > > OpenGL driver > libhybris > wayland. Other projects like the > Intel/Samsung > > "mer" project is doing exactly the same. > > > > If you are interessted how to install and use it, there is a rudimentary > > how to: > > > http://www.cnx-software.com/2013/04/21/how-to-use-libhybris-and-android-gpu-libraries-with-mer-linux-on-the-cubieboard/ > > > > This thing is a therible hack, but unfortunatly this is often the only > way > > to get things like OpenGL, hw video decoding etc. working on embedded arm > > linux. There are no signs that companies like Qualcom or ARM themself > with > > their Mali GPU will deliver open source drivers. > > > > Linking your own apps to bionic/libhybris makes absolutly no sense. The > > reason for libhybris is to use bionic binaries (drivers) within glibc > based > > enviroments. > > How does this work when the binary driver is accessing the same symbol > names (mainly standard functions, but not necessarily) as the program > the driver is loaded into, but the latter needs the real versions in > libc and only the former should be routed through a hack layer? > > Rich > [-- Attachment #2: Type: text/html, Size: 3295 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: libhybris and musl? 2014-09-02 9:25 ` Wermut @ 2014-09-03 22:35 ` Rich Felker 0 siblings, 0 replies; 9+ messages in thread From: Rich Felker @ 2014-09-03 22:35 UTC (permalink / raw) To: musl On Tue, Sep 02, 2014 at 11:25:20AM +0200, Wermut wrote: > I think the following blog post explains the concept nicely: > http://groleo.wordpress.com/2013/07/22/brief-intro-to-libhybris/ > > I simply overrites the bionic symbols and provides glibc compatible calls. > This solution works very nicely, but can have problems if threading is > involved. > > Also good references from the original creator of libhybris: > http://mer-project.blogspot.fi/2013/04/wayland-utilizing-android-gpu-drivers.html > http://mer-project.blogspot.fi/2013/05/wayland-utilizing-android-gpu-drivers.html I didn't find any of those links terribly informative for understanding the aspects of _how it works_ that I'm unclear on -- particularly, how it deals with the fact that the symbols it wants to override for bionic-linked DSOs have the same names as existing symbols from the system libc that need to be left visible. Rich ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: libhybris and musl? 2014-09-01 21:08 ` Rich Felker 2014-09-02 7:11 ` Wermut @ 2014-09-03 22:59 ` Rich Felker 2014-09-04 15:17 ` Isaac Dunham 1 sibling, 1 reply; 9+ messages in thread From: Rich Felker @ 2014-09-03 22:59 UTC (permalink / raw) To: musl On Mon, Sep 01, 2014 at 05:08:23PM -0400, Rich Felker wrote: > On Mon, Sep 01, 2014 at 10:55:26AM +0200, Jens Staal wrote: > > Hi > > > > Has anyone tried if libhybris will emulate the bionic libc properly on top of > > musl libc? Alternatively, are there plans for bionic binary/source > > compatibility in musl like the glibc compatibility (or would this contribute > > to bloat?). > > > > I was mostly curious to look into the possiblility of building AOSP (or Mer) > > with musl instead of bionic (or glibc in the case of Mer), but for Android app > > (those built with NDK) compatibility, one would then need libhybris. > > > > right now just thinking aloud - no real plans at the moment :) > > While I haven't look at the details for how it works, I'm fairly > doubtful that using libhybris would be practical. My guess is that After writing this, I had some more ideas based on recent discussions on #musl that were not particularly related to Bionic drivers, but rather the problems with existing FOSS and binary-ware drivers for non-mobile hardware, such as the nvidia drivers, which some users are presently working on using with musl's glibc-ABI-compatibility. Basically, my view, as expressed many times on #musl, is that all of the existing GL drivers, but especially the non-free ones, are full of way too much bad code to be safe to load into your program's address space. Any process that's loaded them should be treated as potentially crashing or aborting at any time, and possibly also has serious namespace pollution from random libs getting pulled in. The way I'd like to see this solved for our "new platform vision" is to move the actual GL implementation out of the address space of the application using it, and instead provide a universal libGL for applications to link (even statically, if desired) that marshals all GL operations over shared-memory-based IPC to a separate process which has loaded the actual driver for the target hardware you want to render to. As long as the IPC tools used don't depend on a particular libc's ABI at all, this should make it trivial to solve the problem libhybris aimed to solve at the same time: you simply use Bionic in the GL driver process, and your preferred libc with the application side libGL. Rich ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: libhybris and musl? 2014-09-03 22:59 ` Rich Felker @ 2014-09-04 15:17 ` Isaac Dunham 2014-09-04 15:55 ` Rich Felker 0 siblings, 1 reply; 9+ messages in thread From: Isaac Dunham @ 2014-09-04 15:17 UTC (permalink / raw) To: musl On Wed, Sep 03, 2014 at 06:59:17PM -0400, Rich Felker wrote: > Basically, my view, as expressed many times on #musl, is that all of > the existing GL drivers, but especially the non-free ones, are full of > way too much bad code to be safe to load into your program's address > space. Any process that's loaded them should be treated as potentially > crashing or aborting at any time, and possibly also has serious > namespace pollution from random libs getting pulled in. > > The way I'd like to see this solved for our "new platform vision" is > to move the actual GL implementation out of the address space of the > application using it, and instead provide a universal libGL for > applications to link (even statically, if desired) that marshals all > GL operations over shared-memory-based IPC to a separate process which > has loaded the actual driver for the target hardware you want to > render to. As long as the IPC tools used don't depend on a particular > libc's ABI at all, this should make it trivial to solve the problem > libhybris aimed to solve at the same time: you simply use Bionic in > the GL driver process, and your preferred libc with the application > side libGL. I saw an implementation of GL based on this design or something very similar recently. The point the developer had was to make a GL that could be statically linked and handle remote rendering. Ah yes, there it is: https://github.com/msharov/gleri "Network protocol, service, and API for using OpenGL remotely." HTH, Isaac Dunham ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: libhybris and musl? 2014-09-04 15:17 ` Isaac Dunham @ 2014-09-04 15:55 ` Rich Felker 0 siblings, 0 replies; 9+ messages in thread From: Rich Felker @ 2014-09-04 15:55 UTC (permalink / raw) To: musl On Thu, Sep 04, 2014 at 08:17:09AM -0700, Isaac Dunham wrote: > On Wed, Sep 03, 2014 at 06:59:17PM -0400, Rich Felker wrote: > > Basically, my view, as expressed many times on #musl, is that all of > > the existing GL drivers, but especially the non-free ones, are full of > > way too much bad code to be safe to load into your program's address > > space. Any process that's loaded them should be treated as potentially > > crashing or aborting at any time, and possibly also has serious > > namespace pollution from random libs getting pulled in. > > > > The way I'd like to see this solved for our "new platform vision" is > > to move the actual GL implementation out of the address space of the > > application using it, and instead provide a universal libGL for > > applications to link (even statically, if desired) that marshals all > > GL operations over shared-memory-based IPC to a separate process which > > has loaded the actual driver for the target hardware you want to > > render to. As long as the IPC tools used don't depend on a particular > > libc's ABI at all, this should make it trivial to solve the problem > > libhybris aimed to solve at the same time: you simply use Bionic in > > the GL driver process, and your preferred libc with the application > > side libGL. > > I saw an implementation of GL based on this design or something very > similar recently. > The point the developer had was to make a GL that could be statically > linked and handle remote rendering. > > Ah yes, there it is: > https://github.com/msharov/gleri > "Network protocol, service, and API for using OpenGL remotely." While interesting, it could potentially require a lot of work to adapt this to something practical. My intent is for the performance to be as close as possible to current performance with the buggy, insecure design people are using, because the closer it is, the better chance it has of displacing the utterly idiotic system people are using now. For example, IPC via shared memory should be as the primary mechanism (rather than sockets) for all large or low-latency transfers, and I also want to be able to pass the fd used for mapping GPU buffers across a socket to the application to allow it to directly map this buffer, assuming my information is correct that nothing there needs to be validated (my understanding is that it contains data to be processed by shaders on the GPU which run without privileges to do any harm). Of course compiling shaders should take place on the driver process side, so that applications cannot bypass the shader compiler and submit their own potentially malicious compiled code which would be difficult to validate. These issues were discussed a lot more on IRC. I admit freely to not being an expert on current graphics technology, so I may have misconceptions on some details. But independent of this, it's obvious that the current architecture of loading drivers into applications is an utter disaster from a security and robustness standpoint. My hope is that it can be fixed at a cost that's not noticable to most users, but it really needs to be fixed at any cost. Rich ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2014-09-04 15:55 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-09-01 8:55 libhybris and musl? Jens Staal 2014-09-01 21:08 ` Rich Felker 2014-09-02 7:11 ` Wermut 2014-09-02 8:34 ` Rich Felker 2014-09-02 9:25 ` Wermut 2014-09-03 22:35 ` Rich Felker 2014-09-03 22:59 ` Rich Felker 2014-09-04 15:17 ` Isaac Dunham 2014-09-04 15:55 ` Rich Felker
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).