[PR PATCH] xbps-src: default XBPS_MAKEJOBS to `nproc`

[PR PATCH] xbps-src: default XBPS_MAKEJOBS to `nproc`

[classabbyamp]

[-- Attachment #1: Type: text/plain, Size: 564 bytes --]

There is a new pull request by classabbyamp against master on the void-packages repository

https://github.com/classabbyamp/void-packages xbps-src/j-auto
https://github.com/void-linux/void-packages/pull/40331

xbps-src: default XBPS_MAKEJOBS to `nproc`
previously this defaulted to 1. In some rare cases, nproc is not desirable, but that can be easily overriden in the standard ways, etc/conf or -jN.


#### Testing the changes
- I tested the changes in this PR: **YES**



A patch file from https://github.com/void-linux/void-packages/pull/40331.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-xbps-src/j-auto-40331.patch --]
[-- Type: text/x-diff, Size: 2324 bytes --]

From c59238e760803f1ace5f3952a3045a60807d7f9b Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Sat, 5 Nov 2022 02:27:50 -0400
Subject: [PATCH 1/2] xbps-src: default XBPS_MAKEJOBS to `nproc`

previously this defaulted to 1. In some rare cases, nproc is not
desirable, but that can be easily overriden in the standard ways,
etc/conf or -jN.
---
 common/xbps-src/shutils/common.sh | 1 -
 xbps-src                          | 3 +++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/common/xbps-src/shutils/common.sh b/common/xbps-src/shutils/common.sh
index a640528f4d6a..d523b97d1e29 100644
--- a/common/xbps-src/shutils/common.sh
+++ b/common/xbps-src/shutils/common.sh
@@ -482,7 +482,6 @@ setup_pkg() {
     DESTDIR=$XBPS_DESTDIR/$XBPS_CROSS_TRIPLET/${sourcepkg}-${version}
     PKGDESTDIR=$XBPS_DESTDIR/$XBPS_CROSS_TRIPLET/${pkg}-${version}
 
-    : ${XBPS_MAKEJOBS:=1}
     export XBPS_ORIG_MAKEJOBS=${XBPS_ORIG_MAKEJOBS:=$XBPS_MAKEJOBS}
     if [ -n "$disable_parallel_build" ]; then
         XBPS_MAKEJOBS=1
diff --git a/xbps-src b/xbps-src
index 47681826196b..2fc16f56a247 100755
--- a/xbps-src
+++ b/xbps-src
@@ -489,6 +489,9 @@ fi
 [ -n "$XBPS_ARG_CHECK_PKGS" ] && XBPS_CHECK_PKGS="$XBPS_ARG_CHECK_PKGS"
 [ -n "$XBPS_ARG_MAKEJOBS" ] && XBPS_MAKEJOBS="$XBPS_ARG_MAKEJOBS"
 
+# Default to nproc for XBPS_MAKEJOBS if not set
+: ${XBPS_MAKEJOBS:=$(nproc)}
+
 export XBPS_BUILD_ONLY_ONE_PKG XBPS_SKIP_REMOTEREPOS XBPS_BUILD_FORCEMODE \
        XBPS_INFORMATIVE_RUN XBPS_TEMP_MASTERDIR XBPS_BINPKG_EXISTS \
        XBPS_USE_GIT_REVS XBPS_CHECK_PKGS XBPS_DEBUG_PKGS XBPS_SKIP_DEPS \

From 21cee01defd7a05bd076f3e07535ea14346512e0 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Sat, 5 Nov 2022 02:34:02 -0400
Subject: [PATCH 2/2] etc/defaults.conf: document nproc XBPS_MAKEJOBS default

---
 etc/defaults.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/etc/defaults.conf b/etc/defaults.conf
index d69b4a454087..02399b7f91a6 100644
--- a/etc/defaults.conf
+++ b/etc/defaults.conf
@@ -64,7 +64,7 @@ XBPS_SUCMD="sudo /bin/sh -c"
 
 # [OPTIONAL]
 # Number of parallel jobs to execute when building packages that
-# use make(1) or alike commands.
+# use make(1) or alike commands. Defaults to the result of nproc(1).
 #
 #XBPS_MAKEJOBS=4
 

Re: [PR PATCH] [Updated] xbps-src: default XBPS_MAKEJOBS to `nproc`

[classabbyamp]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

There is an updated pull request by classabbyamp against master on the void-packages repository

https://github.com/classabbyamp/void-packages xbps-src/j-auto
https://github.com/void-linux/void-packages/pull/40331

xbps-src: default XBPS_MAKEJOBS to `nproc`
previously this defaulted to 1. In some rare cases, nproc is not desirable, but that can be easily overriden in the standard ways, etc/conf or -jN.


#### Testing the changes
- I tested the changes in this PR: **YES**



A patch file from https://github.com/void-linux/void-packages/pull/40331.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-xbps-src/j-auto-40331.patch --]
[-- Type: text/x-diff, Size: 2482 bytes --]

From b93a53b52019f39ae4fe970868b9b06876c933ad Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Sat, 5 Nov 2022 02:27:50 -0400
Subject: [PATCH 1/2] xbps-src: default XBPS_MAKEJOBS to `nproc`

previously this defaulted to 1. In some rare cases, nproc is not
desirable, but that can be easily overriden in the standard ways,
etc/conf or -jN.
---
 common/xbps-src/shutils/common.sh | 1 -
 xbps-src                          | 7 +++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/common/xbps-src/shutils/common.sh b/common/xbps-src/shutils/common.sh
index a640528f4d6a..d523b97d1e29 100644
--- a/common/xbps-src/shutils/common.sh
+++ b/common/xbps-src/shutils/common.sh
@@ -482,7 +482,6 @@ setup_pkg() {
     DESTDIR=$XBPS_DESTDIR/$XBPS_CROSS_TRIPLET/${sourcepkg}-${version}
     PKGDESTDIR=$XBPS_DESTDIR/$XBPS_CROSS_TRIPLET/${pkg}-${version}
 
-    : ${XBPS_MAKEJOBS:=1}
     export XBPS_ORIG_MAKEJOBS=${XBPS_ORIG_MAKEJOBS:=$XBPS_MAKEJOBS}
     if [ -n "$disable_parallel_build" ]; then
         XBPS_MAKEJOBS=1
diff --git a/xbps-src b/xbps-src
index 47681826196b..a3831d8e4876 100755
--- a/xbps-src
+++ b/xbps-src
@@ -489,6 +489,13 @@ fi
 [ -n "$XBPS_ARG_CHECK_PKGS" ] && XBPS_CHECK_PKGS="$XBPS_ARG_CHECK_PKGS"
 [ -n "$XBPS_ARG_MAKEJOBS" ] && XBPS_MAKEJOBS="$XBPS_ARG_MAKEJOBS"
 
+# Default to nproc for XBPS_MAKEJOBS if not set and nproc is available
+if command -v nproc >/dev/null ; then
+    : ${XBPS_MAKEJOBS:=$(nproc)}
+else
+    : ${XBPS_MAKEJOBS:=1}
+fi
+
 export XBPS_BUILD_ONLY_ONE_PKG XBPS_SKIP_REMOTEREPOS XBPS_BUILD_FORCEMODE \
        XBPS_INFORMATIVE_RUN XBPS_TEMP_MASTERDIR XBPS_BINPKG_EXISTS \
        XBPS_USE_GIT_REVS XBPS_CHECK_PKGS XBPS_DEBUG_PKGS XBPS_SKIP_DEPS \

From 3422301cc19af8ff197cec41f0f9aed87b309aae Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Sat, 5 Nov 2022 02:34:02 -0400
Subject: [PATCH 2/2] etc/defaults.conf: document nproc XBPS_MAKEJOBS default

---
 etc/defaults.conf | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/etc/defaults.conf b/etc/defaults.conf
index d69b4a454087..f70e4ea3d006 100644
--- a/etc/defaults.conf
+++ b/etc/defaults.conf
@@ -64,7 +64,8 @@ XBPS_SUCMD="sudo /bin/sh -c"
 
 # [OPTIONAL]
 # Number of parallel jobs to execute when building packages that
-# use make(1) or alike commands.
+# use make(1) or alike commands. Defaults to the result of nproc(1).
+# If nproc(1) is not available, defaults to 1.
 #
 #XBPS_MAKEJOBS=4
 

Re: xbps-src: default XBPS_MAKEJOBS to `nproc`

[abenson]

[-- Attachment #1: Type: text/plain, Size: 251 bytes --]

New comment by abenson on void-packages repository

https://github.com/void-linux/void-packages/pull/40331#issuecomment-1304428112

Comment:
I kind of support this.  I just wonder what the default should be; all cores, some cores, or specified cores.

Re: [PR PATCH] [Updated] xbps-src: default XBPS_MAKEJOBS to `nproc`

[classabbyamp]

[-- Attachment #1: Type: text/plain, Size: 660 bytes --]

There is an updated pull request by classabbyamp against master on the void-packages repository

https://github.com/classabbyamp/void-packages xbps-src/j-auto
https://github.com/void-linux/void-packages/pull/40331

xbps-src: default XBPS_MAKEJOBS to `nproc`
previously this defaulted to 1. In some rare cases, nproc is not desirable, but that can be easily overriden in the standard ways, etc/conf or -jN.

For compatibility with other distros, it defaults to 1 when nproc is not available.

#### Testing the changes
- I tested the changes in this PR: **YES**



A patch file from https://github.com/void-linux/void-packages/pull/40331.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-xbps-src/j-auto-40331.patch --]
[-- Type: text/x-diff, Size: 2567 bytes --]

From 85947fb4e3c09bda4aedd2d3071f94b99943efc0 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Sat, 5 Nov 2022 02:27:50 -0400
Subject: [PATCH 1/2] xbps-src: default XBPS_MAKEJOBS to `nproc`

previously this defaulted to 1. In some rare cases, nproc is not
desirable, but that can be easily overriden in the standard ways,
etc/conf or -jN.

For compatibility with other distros, it defaults to 1 when nproc
is not available.
---
 common/xbps-src/shutils/common.sh | 1 -
 xbps-src                          | 7 +++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/common/xbps-src/shutils/common.sh b/common/xbps-src/shutils/common.sh
index a640528f4d6a..d523b97d1e29 100644
--- a/common/xbps-src/shutils/common.sh
+++ b/common/xbps-src/shutils/common.sh
@@ -482,7 +482,6 @@ setup_pkg() {
     DESTDIR=$XBPS_DESTDIR/$XBPS_CROSS_TRIPLET/${sourcepkg}-${version}
     PKGDESTDIR=$XBPS_DESTDIR/$XBPS_CROSS_TRIPLET/${pkg}-${version}
 
-    : ${XBPS_MAKEJOBS:=1}
     export XBPS_ORIG_MAKEJOBS=${XBPS_ORIG_MAKEJOBS:=$XBPS_MAKEJOBS}
     if [ -n "$disable_parallel_build" ]; then
         XBPS_MAKEJOBS=1
diff --git a/xbps-src b/xbps-src
index 47681826196b..a3831d8e4876 100755
--- a/xbps-src
+++ b/xbps-src
@@ -489,6 +489,13 @@ fi
 [ -n "$XBPS_ARG_CHECK_PKGS" ] && XBPS_CHECK_PKGS="$XBPS_ARG_CHECK_PKGS"
 [ -n "$XBPS_ARG_MAKEJOBS" ] && XBPS_MAKEJOBS="$XBPS_ARG_MAKEJOBS"
 
+# Default to nproc for XBPS_MAKEJOBS if not set and nproc is available
+if command -v nproc >/dev/null ; then
+    : ${XBPS_MAKEJOBS:=$(nproc)}
+else
+    : ${XBPS_MAKEJOBS:=1}
+fi
+
 export XBPS_BUILD_ONLY_ONE_PKG XBPS_SKIP_REMOTEREPOS XBPS_BUILD_FORCEMODE \
        XBPS_INFORMATIVE_RUN XBPS_TEMP_MASTERDIR XBPS_BINPKG_EXISTS \
        XBPS_USE_GIT_REVS XBPS_CHECK_PKGS XBPS_DEBUG_PKGS XBPS_SKIP_DEPS \

From 5855e89d801f7177bf4887e83384607496e1f8a2 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Sat, 5 Nov 2022 02:34:02 -0400
Subject: [PATCH 2/2] etc/defaults.conf: document nproc XBPS_MAKEJOBS default

---
 etc/defaults.conf | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/etc/defaults.conf b/etc/defaults.conf
index d69b4a454087..f70e4ea3d006 100644
--- a/etc/defaults.conf
+++ b/etc/defaults.conf
@@ -64,7 +64,8 @@ XBPS_SUCMD="sudo /bin/sh -c"
 
 # [OPTIONAL]
 # Number of parallel jobs to execute when building packages that
-# use make(1) or alike commands.
+# use make(1) or alike commands. Defaults to the result of nproc(1).
+# If nproc(1) is not available, defaults to 1.
 #
 #XBPS_MAKEJOBS=4
 

Re: xbps-src: default XBPS_MAKEJOBS to `nproc`

[dataCobra]

[-- Attachment #1: Type: text/plain, Size: 275 bytes --]

New comment by dataCobra on void-packages repository

https://github.com/void-linux/void-packages/pull/40331#issuecomment-1304458177

Comment:
Could we use the amount of cores divided by two?

I support this also, but I feel like defaulting to all cores is a bit too much?

Re: xbps-src: default XBPS_MAKEJOBS to `nproc`

[jcgruenhage]

[-- Attachment #1: Type: text/plain, Size: 427 bytes --]

New comment by jcgruenhage on void-packages repository

https://github.com/void-linux/void-packages/pull/40331#issuecomment-1304500506

Comment:
I think `nproc` would be a good default. I usually run with `-j 16` on my laptop, and it's rarely a problem. The only times when I run into trouble is not due to CPU load either, but because I'm exhausting memory (which has been solved by going 16->64gig) or exhausting disk space.

Re: [RFC] xbps-src: default XBPS_MAKEJOBS to `nproc`

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 382 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/40331#issuecomment-1304692700

Comment:
I've seen plenty of historical recommendation for spawning twice as many jobs as CPUs with make, so defaulting to nproc seems reasonable. People with SMT CPUs will have the "right" value, and those without get a reduced "compromise" value.

Re: [RFC] xbps-src: default XBPS_MAKEJOBS to `nproc`

[dataCobra]

[-- Attachment #1: Type: text/plain, Size: 371 bytes --]

New comment by dataCobra on void-packages repository

https://github.com/void-linux/void-packages/pull/40331#issuecomment-1317057921

Comment:
Most of the time I'm building stuff while doing other things on my machines. As I mentioned above I still feel like defaulting to all cores is a bit too much.

On some of my machines multi tasking would no longer be possible.

Re: [RFC] xbps-src: default XBPS_MAKEJOBS to `nproc`

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 439 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/40331#issuecomment-1317150032

Comment:
Compilation is I/O bound, which is why the usual recommendation is to use twice as many processes as you have CPUs. There's no need to try to get clever setting this. You can always override the parallelism if you find it too intensive, just as you have to override it now to get any parallelism.

Re: [RFC] xbps-src: default XBPS_MAKEJOBS to `nproc`

[vincele]

[-- Attachment #1: Type: text/plain, Size: 231 bytes --]

New comment by vincele on void-packages repository

https://github.com/void-linux/void-packages/pull/40331#issuecomment-1363292887

Comment:
Was wondering the other day why this defaults to no parallelism (void newbie here).
LGTM

Re: [PR PATCH] [Merged]: [RFC] xbps-src: default XBPS_MAKEJOBS to `nproc`

[classabbyamp]

[-- Attachment #1: Type: text/plain, Size: 492 bytes --]

There's a merged pull request on the void-packages repository

[RFC] xbps-src: default XBPS_MAKEJOBS to `nproc`
https://github.com/void-linux/void-packages/pull/40331

Description:
previously this defaulted to 1. In some rare cases, nproc is not desirable, but that can be easily overriden in the standard ways, etc/conf or -jN.

For compatibility with other distros, it defaults to 1 when nproc is not available.

#### Testing the changes
- I tested the changes in this PR: **YES**