Github messages for voidlinux
 help / color / mirror / Atom feed
* [ISSUE] Expand using Profile-Guided Optimization (PGO) and Post Link Optimization (PLO) in the Void Linux packages
@ 2023-11-13 20:17 zamazan4ik
  2023-11-15 13:52 ` dmarto
                   ` (6 more replies)
  0 siblings, 7 replies; 8+ messages in thread
From: zamazan4ik @ 2023-11-13 20:17 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 5162 bytes --]

New issue by zamazan4ik on void-packages repository

https://github.com/void-linux/void-packages/issues/47215

Description:
Hello.

Now I am investigating PGO and PLO effects on different kinds of software - all my current results are available at https://github.com/zamazan4ik/awesome-pgo. According to these results, enabling PGO and PLO can help with achieving better overall performance in many cases. I think expanding PGO and PLO usage for Void Linux packages would be a good idea.

PGO is already a well-known technique. All currently known PGO effects on performance can be found at https://github.com/zamazan4ik/awesome-pgo#pgo-showcases . Several OS distros already enabled PGO for some packages like GCC, Rustc, Chromium, Firefox, and others (it depends on each OS distro, of course).

I think we can try to expand PGO usage across Void Linux packages. I think we can start by enabling PGO at least for the following projects:

* Clang
* GCC
* Rust (`rustc`)
* Firefox (however, I am not sure is it PGO-optimized right now or not in Void Linux)

I am sure it's possible to find more packages with upstream PGO support since some packages Void Linux packages like CPython already [support](https://github.com/void-linux/void-packages/pull/43791) building with PGO.

As an example, PGO is already used in other Linux distributions for these packages:

* GCC in Solus: [link](https://github.com/getsolus/packages/blob/main/packages/g/gcc/package.yml#L164)
* Clang in Solus: [link](https://github.com/getsolus/packages/blob/main/packages/l/llvm/package.yml#L116)
* Firefox in Fedora: [link](https://src.fedoraproject.org/rpms/firefox/blob/rawhide/f/firefox.spec)
* Rust in Fedora: [link](https://src.fedoraproject.org/rpms/rust/blob/rawhide/f/rust.spec#_772)

Regarding Post Link Optimization (PLO), right now there are two main tools - LLVM BOLT and Google Propeller.

According to the Facebook Research Paper (https://research.facebook.com/publications/bolt-a-practical-binary-optimizer-for-data-centers-and-beyond/), LLVM BOLT (https://github.com/llvm/llvm-project/blob/main/bolt/README.md) helps with achieving better performance for various packages like compilers and interpreters. I think it would be a good idea to enable LLVM BOLT for some packages to deliver faster binaries for users (since Propeller is less stable right now).

Here I got some examples of how LLVM BOLT is already integrated into other projects:

* Rustc: https://github.com/rust-lang/rust/pull/116352
* CPython: https://github.com/python/cpython/pull/95908
* Pyston:
  - https://github.com/pyston/pyston#building
  - https://github.com/pyston/pyston/blob/pyston_main/Makefile#L200
* Clang: https://github.com/llvm/llvm-project/blob/main/clang/cmake/caches/BOLT.cmake

So at least for the projects above LLVM BOLT effects are tested and some preparations are already done in the upstream projects. In this case, it should be easier to enable BOLT for these packages.

For some projects right now there is ongoing work on integrating LLVM BOLT into the build scripts:

* Chromium: https://bugs.chromium.org/p/chromium/issues/detail?id=1163978
* Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1789087
  - The same for Propeller (a LLVM BOLT alternative): https://bugzilla.mozilla.org/show_bug.cgi?id=1509314
* NodeJS: https://github.com/nodejs/node/issues/50379
* LDC: https://github.com/ldc-developers/ldc/issues/4228
* GCC: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112492

More about LLVM BOLT performance results for other projects can be found in:

* Rustc:
  - https://github.com/rust-lang/rust/pull/116352
  - https://www.reddit.com/r/rust/comments/y4w2kr/llvm_used_by_rustc_is_now_optimized_with_bolt_on/
* CPython: https://github.com/python/cpython/pull/95908
* YDB: https://github.com/ydb-platform/ydb/issues/140
* Clang:
  - [Slides](https://llvm.org/devmtg/2022-11/slides/Lightning15-OptimizingClangWithBOLTUsingCMake.pdf)
  - [Results on building Clang](https://github.com/ptr1337/llvm-bolt-scripts/blob/master/results.md)
  - [Linaro results](https://android-review.linaro.org/plugins/gitiles/toolchain/llvm_android/+/f36c64eeddf531b7b1a144c40f61d6c9a78eee7a)
  - [on AMD 7950X3D](https://github.com/llvm/llvm-project/issues/65010#issuecomment-1701255347)
* LDC: https://github.com/ldc-developers/ldc/issues/4228#issuecomment-1334499428
* NodeJS: https://aaupov.github.io/blog/2020/10/08/bolt-nodejs
* Chromium: https://aaupov.github.io/blog/2022/11/12/bolt-chromium
* MySQL, MongoDB, memcached, Verilator: https://people.ucsc.edu/~hlitz/papers/ocolos.pdf

Some OS already using LLVM BOLT in their build scripts - [check](https://github.com/getsolus/packages/blob/main/packages/l/llvm/package.yml#L116) Clang recipe in Solus.

I don't create an issue per project (like "Enable BOLT for Clang", "Enable PGO for GCC", etc.) since I think first we need to discuss the approach. If we agree with enabling BOLT or expanding PGO usage for some packages, then we can create an additional issue (and use this issue as a meta issue). In this issue, we can discuss approaches regarding PGO and PLO.

^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2024-03-16  1:44 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-11-13 20:17 [ISSUE] Expand using Profile-Guided Optimization (PGO) and Post Link Optimization (PLO) in the Void Linux packages zamazan4ik
2023-11-15 13:52 ` dmarto
2023-11-15 20:36 ` zamazan4ik
2023-11-15 20:39 ` classabbyamp
2023-12-03  1:05 ` ahesford
2023-12-03  1:13 ` zamazan4ik
2024-03-02  1:43 ` github-actions
2024-03-16  1:44 ` [ISSUE] [CLOSED] " github-actions

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).