[PR PATCH] common/hooks: speed up generate-runtime-deps.

[PR PATCH] common/hooks: speed up generate-runtime-deps.

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 1836 bytes --]

There is a new pull request by ericonr against master on the void-packages repository

https://github.com/ericonr/void-packages speedelf
https://github.com/void-linux/void-packages/pull/31364

common/hooks: speed up generate-runtime-deps.
Instead of using file(1) to check for ELF files, just read bytes
directly from the file and check if they are the ELF magic bytes.

Should probably be factored out into a common function that can be used
in other places.

<!-- Mark items with [x] where applicable -->

#### General
- [ ] This is a new package and it conforms to the [quality requirements](https://github.com/void-linux/void-packages/blob/master/Manual.md#quality-requirements)

#### Have the results of the proposed changes been tested?
- [ ] I use the packages affected by the proposed changes on a regular basis and confirm this PR works for me
- [ ] I generally don't use the affected packages but briefly tested this PR

<!--
If GitHub CI cannot be used to validate the build result (for example, if the
build is likely to take several hours), make sure to
[skip CI](https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#continuous-integration).
When skipping CI, uncomment and fill out the following section.
Note: for builds that are likely to complete in less than 2 hours, it is not
acceptable to skip CI.
-->
<!-- 
#### Does it build and run successfully? 
(Please choose at least one native build and, if supported, at least one cross build. More are better.)
- [ ] I built this PR locally for my native architecture, (ARCH-LIBC)
- [ ] I built this PR locally for these architectures (if supported. mark crossbuilds):
  - [ ] aarch64-musl
  - [ ] armv7l
  - [ ] armv6l-musl
-->


A patch file from https://github.com/void-linux/void-packages/pull/31364.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-speedelf-31364.patch --]
[-- Type: text/x-diff, Size: 2394 bytes --]

From cc10261f60551f7149dc7693dcbcfc411fa9ecd4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=89rico=20Nogueira?= <erico.erc@gmail.com>
Date: Tue, 8 Jun 2021 11:43:00 -0300
Subject: [PATCH] common/hooks: speed up generate-runtime-deps.

Instead of using file(1) to check for ELF files, just read bytes
directly from the file and check if they are the ELF magic bytes.

Should probably be factored out into a common function that can be used
in other places.
---
 .../hooks/pre-pkg/04-generate-runtime-deps.sh | 23 +++++++++----------
 1 file changed, 11 insertions(+), 12 deletions(-)

diff --git a/common/hooks/pre-pkg/04-generate-runtime-deps.sh b/common/hooks/pre-pkg/04-generate-runtime-deps.sh
index 8ffd3a71d8eb..0b1a6d0a6025 100644
--- a/common/hooks/pre-pkg/04-generate-runtime-deps.sh
+++ b/common/hooks/pre-pkg/04-generate-runtime-deps.sh
@@ -47,7 +47,7 @@ store_pkgdestdir_rundeps() {
 }
 
 hook() {
-    local depsftmp f lf j mapshlibs sorequires _curdep
+    local depsftmp f lf j mapshlibs sorequires _curdep elfmagic
 
     # Disable trap on ERR, xbps-uhelper cmd might return error... but not something
     # to be worried about because if there are broken shlibs this hook returns
@@ -72,18 +72,17 @@ hook() {
 		    msg_normal "Skipping dependency scan for ${lf}\n"
 		    continue
 	    fi
-        case "$(file -bi "$f")" in
-            application/x-*executable*|application/x-sharedlib*)
-                for nlib in $($OBJDUMP -p "$f"|grep NEEDED|awk '{print $2}'); do
-                    [ -z "$verify_deps" ] && verify_deps="$nlib" && continue
-                    found=0
-                    for j in ${verify_deps}; do
-                        [[ $j == $nlib ]] && found=1 && break
-                    done
-                    [[ $found -eq 0 ]] && verify_deps="$verify_deps $nlib"
+        read -n4 elfmagic < "$f"
+        if [ "$elfmagic" = $'\177ELF' ]; then
+            for nlib in $($OBJDUMP -p "$f"|grep NEEDED|awk '{print $2}'); do
+                [ -z "$verify_deps" ] && verify_deps="$nlib" && continue
+                found=0
+                for j in ${verify_deps}; do
+                    [[ $j == $nlib ]] && found=1 && break
                 done
-                ;;
-        esac
+                [[ $found -eq 0 ]] && verify_deps="$verify_deps $nlib"
+            done
+        fi
     done
     exec 0<&3 # restore stdin
     rm -f $depsftmp

Re: [PR REVIEW] common/hooks: speed up generate-runtime-deps.

[sgn]

[-- Attachment #1: Type: text/plain, Size: 279 bytes --]

New review comment by sgn on void-packages repository

https://github.com/void-linux/void-packages/pull/31364#discussion_r647531315

Comment:
Maybe replace `grep NEEDED | awk '{print $2}'` with `awk '/NEEDED/{print $2}'` to save some `fork(2)`, `execve(2)`, `pipe(2)` and `dup2`

Re: [PR REVIEW] common/hooks: speed up generate-runtime-deps.

[Piraty]

[-- Attachment #1: Type: text/plain, Size: 149 bytes --]

New review comment by Piraty on void-packages repository

https://github.com/void-linux/void-packages/pull/31364#discussion_r647649806

Comment:
`[`

Re: [PR REVIEW] common/hooks: speed up generate-runtime-deps.

[Piraty]

[-- Attachment #1: Type: text/plain, Size: 168 bytes --]

New review comment by Piraty on void-packages repository

https://github.com/void-linux/void-packages/pull/31364#discussion_r647649806

Comment:
`[ "$found" = "0" ]` ?

Re: [PR REVIEW] common/hooks: speed up generate-runtime-deps.

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 213 bytes --]

New review comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/31364#discussion_r647683039

Comment:
We had been using that construct for a long while now, why change?

Re: [PR REVIEW] common/hooks: speed up generate-runtime-deps.

[Piraty]

[-- Attachment #1: Type: text/plain, Size: 236 bytes --]

New review comment by Piraty on void-packages repository

https://github.com/void-linux/void-packages/pull/31364#discussion_r650781462

Comment:
i prefer to not use `[[`  if that bashism can be avoided, but consistency is a valid point

Re: [PR REVIEW] common/hooks: speed up generate-runtime-deps.

[Piraty]

[-- Attachment #1: Type: text/plain, Size: 243 bytes --]

New review comment by Piraty on void-packages repository

https://github.com/void-linux/void-packages/pull/31364#discussion_r650781462

Comment:
i prefer to not use `[[` if the performed test is not a bashism, but consistency is a valid point

Re: [PR PATCH] [Updated] common/hooks: speed up generate-runtime-deps.

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 1841 bytes --]

There is an updated pull request by ericonr against master on the void-packages repository

https://github.com/ericonr/void-packages speedelf
https://github.com/void-linux/void-packages/pull/31364

common/hooks: speed up generate-runtime-deps.
Instead of using file(1) to check for ELF files, just read bytes
directly from the file and check if they are the ELF magic bytes.

Should probably be factored out into a common function that can be used
in other places.

<!-- Mark items with [x] where applicable -->

#### General
- [ ] This is a new package and it conforms to the [quality requirements](https://github.com/void-linux/void-packages/blob/master/Manual.md#quality-requirements)

#### Have the results of the proposed changes been tested?
- [ ] I use the packages affected by the proposed changes on a regular basis and confirm this PR works for me
- [ ] I generally don't use the affected packages but briefly tested this PR

<!--
If GitHub CI cannot be used to validate the build result (for example, if the
build is likely to take several hours), make sure to
[skip CI](https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#continuous-integration).
When skipping CI, uncomment and fill out the following section.
Note: for builds that are likely to complete in less than 2 hours, it is not
acceptable to skip CI.
-->
<!-- 
#### Does it build and run successfully? 
(Please choose at least one native build and, if supported, at least one cross build. More are better.)
- [ ] I built this PR locally for my native architecture, (ARCH-LIBC)
- [ ] I built this PR locally for these architectures (if supported. mark crossbuilds):
  - [ ] aarch64-musl
  - [ ] armv7l
  - [ ] armv6l-musl
-->


A patch file from https://github.com/void-linux/void-packages/pull/31364.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-speedelf-31364.patch --]
[-- Type: text/x-diff, Size: 2464 bytes --]

From d7a69d7aff05e6d22369f249d2f566d146e459e0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=89rico=20Nogueira?= <erico.erc@gmail.com>
Date: Tue, 8 Jun 2021 11:43:00 -0300
Subject: [PATCH] common/hooks: speed up generate-runtime-deps.

Instead of using file(1) to check for ELF files, just read bytes
directly from the file and check if they are the ELF magic bytes.

Should probably be factored out into a common function that can be used
in other places, if necessary.

Also use awk instead of "grep|awk" for some minor speedup.
---
 .../hooks/pre-pkg/04-generate-runtime-deps.sh | 23 +++++++++----------
 1 file changed, 11 insertions(+), 12 deletions(-)

diff --git a/common/hooks/pre-pkg/04-generate-runtime-deps.sh b/common/hooks/pre-pkg/04-generate-runtime-deps.sh
index 8ffd3a71d8eb..c8f8c04d4884 100644
--- a/common/hooks/pre-pkg/04-generate-runtime-deps.sh
+++ b/common/hooks/pre-pkg/04-generate-runtime-deps.sh
@@ -47,7 +47,7 @@ store_pkgdestdir_rundeps() {
 }
 
 hook() {
-    local depsftmp f lf j mapshlibs sorequires _curdep
+    local depsftmp f lf j mapshlibs sorequires _curdep elfmagic
 
     # Disable trap on ERR, xbps-uhelper cmd might return error... but not something
     # to be worried about because if there are broken shlibs this hook returns
@@ -72,18 +72,17 @@ hook() {
 		    msg_normal "Skipping dependency scan for ${lf}\n"
 		    continue
 	    fi
-        case "$(file -bi "$f")" in
-            application/x-*executable*|application/x-sharedlib*)
-                for nlib in $($OBJDUMP -p "$f"|grep NEEDED|awk '{print $2}'); do
-                    [ -z "$verify_deps" ] && verify_deps="$nlib" && continue
-                    found=0
-                    for j in ${verify_deps}; do
-                        [[ $j == $nlib ]] && found=1 && break
-                    done
-                    [[ $found -eq 0 ]] && verify_deps="$verify_deps $nlib"
+        read -n4 elfmagic < "$f"
+        if [ "$elfmagic" = $'\177ELF' ]; then
+            for nlib in $($OBJDUMP -p "$f"|awk '/NEEDED/{print $2}'); do
+                [ -z "$verify_deps" ] && verify_deps="$nlib" && continue
+                found=0
+                for j in ${verify_deps}; do
+                    [[ $j == $nlib ]] && found=1 && break
                 done
-                ;;
-        esac
+                [[ $found -eq 0 ]] && verify_deps="$verify_deps $nlib"
+            done
+        fi
     done
     exec 0<&3 # restore stdin
     rm -f $depsftmp

Re: [PR REVIEW] common/hooks: speed up generate-runtime-deps.

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 176 bytes --]

New review comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/31364#discussion_r659229977

Comment:
Done and seems to be working!

Re: [PR PATCH] [Merged]: common/hooks: speed up generate-runtime-deps.

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 1684 bytes --]

There's a merged pull request on the void-packages repository

common/hooks: speed up generate-runtime-deps.
https://github.com/void-linux/void-packages/pull/31364

Description:
Instead of using file(1) to check for ELF files, just read bytes
directly from the file and check if they are the ELF magic bytes.

Should probably be factored out into a common function that can be used
in other places.

<!-- Mark items with [x] where applicable -->

#### General
- [ ] This is a new package and it conforms to the [quality requirements](https://github.com/void-linux/void-packages/blob/master/Manual.md#quality-requirements)

#### Have the results of the proposed changes been tested?
- [ ] I use the packages affected by the proposed changes on a regular basis and confirm this PR works for me
- [ ] I generally don't use the affected packages but briefly tested this PR

<!--
If GitHub CI cannot be used to validate the build result (for example, if the
build is likely to take several hours), make sure to
[skip CI](https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#continuous-integration).
When skipping CI, uncomment and fill out the following section.
Note: for builds that are likely to complete in less than 2 hours, it is not
acceptable to skip CI.
-->
<!-- 
#### Does it build and run successfully? 
(Please choose at least one native build and, if supported, at least one cross build. More are better.)
- [ ] I built this PR locally for my native architecture, (ARCH-LIBC)
- [ ] I built this PR locally for these architectures (if supported. mark crossbuilds):
  - [ ] aarch64-musl
  - [ ] armv7l
  - [ ] armv6l-musl
-->