Re: [PR PATCH] [Updated] ci: use treeless clone instead of actions/checkout

[classabbyamp <classabbyamp@users.noreply.github.com>]

[-- Attachment #1: Type: text/plain, Size: 1731 bytes --]

There is an updated pull request by classabbyamp against master on the void-packages repository

https://github.com/classabbyamp/void-packages ci/treeless-clone
https://github.com/void-linux/void-packages/pull/41130

ci: use treeless clone instead of actions/checkout
Summary:

- [use treeless fetches instead of `--depth`](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/)
  - this is not supported by `actions/checkout`, so it was implemented as a short script (staying with `actions/checkout@v1` does not seem like a good idea because of its age, and updating to `actions/checkout@v3` would require adding `libstdc++` to any CI workflows that run in void containers because of node's requirements)
- This applies to the build and cycles CI workflows, and some additional cleanups were applied to the cycles workflow
  - it can't be implemented as a script in `common/travis/` because the repo doesn't exist on the runner (yet!)
- `changed_templates.sh` had a small logic change, because we now have the complete commit history instead of just `--depth=200`. This preserves the 200 commit limit with a slightly-enhanced error message (wording clarified and it now shows up as an error annotation)
- `fetch_upstream.sh` is no longer needed now that the checkout script fetches the entire commit history of master and the PR branch

#### Testing the changes
- I tested the changes in this PR: **YES**
  - successful run: https://github.com/void-linux/void-packages/actions/runs/3712923390
  - run with too many commits: https://github.com/void-linux/void-packages/actions/runs/3712884082



A patch file from https://github.com/void-linux/void-packages/pull/41130.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/treeless-clone-41130.patch --]
[-- Type: text/x-diff, Size: 9252 bytes --]

From 40e82759d853209f65aaf4cead81cd9c4538f815 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Fri, 16 Dec 2022 05:00:52 -0500
Subject: [PATCH 1/3] .github/workflows: use checkout script instead of action

uses treeless fetches instead of `--depth`. This is not supported by
`actions/checkout`, so it was implemented as a short script (staying
with `actions/checkout@v1` does not seem like a good idea because of
its age, and updating to `actions/checkout@v3` would require adding
`libstdc++` to any CI workflows that run in void containers because
of node's requirements)

see also: https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/

this can't be implemented as a script in `common/travis/` because the
repo doesn't exist on the runner (yet!)

also fix indentation in cycle check
---
 .github/workflows/build.yaml | 42 ++++++++++++++++++++++------
 .github/workflows/cycles.yml | 53 +++++++++++++++++++++++-------------
 2 files changed, 68 insertions(+), 27 deletions(-)

diff --git a/.github/workflows/build.yaml b/.github/workflows/build.yaml
index 95f1cd88e338..ddb2c4e78f2e 100644
--- a/.github/workflows/build.yaml
+++ b/.github/workflows/build.yaml
@@ -26,10 +26,23 @@ jobs:
       LICENSE_LIST: common/travis/license.lst
 
     steps:
-      - uses: actions/checkout@v1
-        with:
-          fetch-depth: 200
-      - run: common/travis/fetch_upstream.sh
+      - name: Clone and checkout
+        run: |
+          REMOTE_REF=$(echo ${{ github.ref }} | sed 's;refs/;refs/remotes/;')
+          /bin/echo -e '::group::\x1b[32mCloning repository...\x1b[0m'
+          git clone --progress --no-checkout --filter=tree:0 \
+              "${{ github.server_url }}/${{ github.repository }}" "$PWD"
+          git config --global --add gc.auto 0
+          git config --global --add safe.directory "$PWD"
+          echo "::endgroup::"
+          /bin/echo -e '::group::\x1b[32mFetching repository refs...\x1b[0m'
+          git fetch --prune --progress --filter=tree:0 origin \
+              +refs/heads/*:refs/remotes/origin/* \
+              +${{ github.ref }}:"${REMOTE_REF}"
+          echo "::endgroup::"
+          /bin/echo -e '::group::\x1b[32mChecking out repository...\x1b[0m'
+          git checkout --progress --force "${REMOTE_REF}"
+          echo "::endgroup::"
       - run: common/travis/changed_templates.sh
       - run: common/travis/fetch-xbps.sh
       - run: common/travis/fetch-xtools.sh
@@ -73,15 +86,28 @@ jobs:
           # Upgrade again (in case there was a xbps update)
           xbps-install -yu
 
-      - uses: actions/checkout@v1
-        with:
-          fetch-depth: 200
+      - name: Clone and checkout
+        run: |
+          REMOTE_REF=$(echo ${{ github.ref }} | sed 's;refs/;refs/remotes/;')
+          /bin/echo -e '::group::\x1b[32mCloning repository...\x1b[0m'
+          git clone --progress --no-checkout --filter=tree:0 \
+              "${{ github.server_url }}/${{ github.repository }}" "$PWD"
+          git config --global --add gc.auto 0
+          git config --global --add safe.directory "$PWD"
+          echo "::endgroup::"
+          /bin/echo -e '::group::\x1b[32mFetching repository refs...\x1b[0m'
+          git fetch --prune --progress --filter=tree:0 origin \
+              +refs/heads/*:refs/remotes/origin/* \
+              +${{ github.ref }}:"${REMOTE_REF}"
+          echo "::endgroup::"
+          /bin/echo -e '::group::\x1b[32mChecking out repository...\x1b[0m'
+          git checkout --progress --force "${REMOTE_REF}"
+          echo "::endgroup::"
       - name: Create hostrepo and prepare masterdir
         run: |
          ln -s "$(pwd)" /hostrepo &&
          common/travis/set_mirror.sh &&
          common/travis/prepare.sh &&
-         common/travis/fetch_upstream.sh &&
          common/travis/fetch-xtools.sh
       - run: common/travis/changed_templates.sh
 
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 7d781076dbe9..f0631abc899d 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -10,7 +10,9 @@ jobs:
     permissions:
       issues: write
     container:
-        image: 'ghcr.io/void-linux/xbps-src-masterdir:20220527RC01-x86_64-musl'
+      image: 'ghcr.io/void-linux/xbps-src-masterdir:20220527RC01-x86_64-musl'
+      env:
+        PATH: '/usr/libexec/chroot-git:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin:/usr/local/bin:/tmp/bin'
     steps:
       - name: Prepare container
         run: |
@@ -23,26 +25,39 @@ jobs:
           xbps-install -yu
           # Install script dependencies
           xbps-install -y python3-networkx github-cli
-      - uses: actions/checkout@v1
-        with:
-          fetch-depth: 1
+
+      - name: Clone and checkout
+        run: |
+          REMOTE_REF=$(echo ${{ github.ref }} | sed 's;refs/;refs/remotes/;')
+          /bin/echo -e '::group::\x1b[32mCloning repository...\x1b[0m'
+          git clone --progress --no-checkout --filter=tree:0 \
+              "${{ github.server_url }}/${{ github.repository }}" "$PWD"
+          git config --global --add gc.auto 0
+          git config --global --add safe.directory "$PWD"
+          echo "::endgroup::"
+          /bin/echo -e '::group::\x1b[32mFetching repository refs...\x1b[0m'
+          git fetch --prune --progress --filter=tree:0 origin \
+              +refs/heads/*:refs/remotes/origin/* \
+              +${{ github.ref }}:"${REMOTE_REF}"
+          echo "::endgroup::"
+          /bin/echo -e '::group::\x1b[32mChecking out repository...\x1b[0m'
+          git checkout --progress --force "${REMOTE_REF}"
+          echo "::endgroup::"
+
       - name: Create hostrepo and prepare masterdir
         run: |
-         ln -s "$(pwd)" /hostrepo &&
-         common/travis/set_mirror.sh &&
-         common/travis/prepare.sh
+          ln -s "$(pwd)" /hostrepo &&
+          common/travis/set_mirror.sh &&
+          common/travis/prepare.sh
       - name: Find cycles and open issues
         run: |
-         PATH="/usr/libexec/chroot-git:$PATH"
-         # required by git 2.35.2+
-         git config --global --add safe.directory "$PWD"
-         common/scripts/xbps-cycles.py | tee cycles
-         grep 'Cycle:' cycles | while read -r line; do
-             if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
-                 printf "Issue on '%s' already exists.\n" "$line"
-             else
-                 gh issue create -R "$GITHUB_REPOSITORY" -b '' -t "$line"
-             fi
-         done
+          common/scripts/xbps-cycles.py | tee cycles
+          grep 'Cycle:' cycles | while read -r line; do
+              if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
+                  printf "Issue on '%s' already exists.\n" "$line"
+              else
+                  gh issue create -R "$GITHUB_REPOSITORY" -b '' -t "$line"
+              fi
+          done
         env:
-            GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}

From cf2e7397c4f05e257a8a5dba0918144eb3700964 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Fri, 16 Dec 2022 04:59:55 -0500
Subject: [PATCH 2/3] common/travis/fetch_upstream.sh: remove now-unnecessary
 script

no longer needed now that the checkout script fetches the entire
commit history of master and the PR branch
---
 common/travis/fetch_upstream.sh | 9 ---------
 1 file changed, 9 deletions(-)
 delete mode 100755 common/travis/fetch_upstream.sh

diff --git a/common/travis/fetch_upstream.sh b/common/travis/fetch_upstream.sh
deleted file mode 100755
index e5ec5979493b..000000000000
--- a/common/travis/fetch_upstream.sh
+++ /dev/null
@@ -1,9 +0,0 @@
-#!/bin/sh
-#
-# changed_templates.sh
-
-# required by git 2.35.2+
-git config --global --add safe.directory "$PWD"
-
-/bin/echo -e '\x1b[32mFetching upstream...\x1b[0m'
-git fetch --depth 200 https://github.com/void-linux/void-packages.git master

From 17961db670fa3dc18cea6c0b9824c22ec52b69ce Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Fri, 16 Dec 2022 06:36:15 -0500
Subject: [PATCH 3/3] common/travis/changed_templates.sh: change branch age
 check for treeless

small logic change because we now have the complete commit history instead
of just `--depth=200`. This preserves the 200 commit limit with a slightly-
enhanced error message (wording clarified and it now shows up as an error
annotation)
---
 common/travis/changed_templates.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/common/travis/changed_templates.sh b/common/travis/changed_templates.sh
index 2bed2250ae71..e6a4328e3c49 100755
--- a/common/travis/changed_templates.sh
+++ b/common/travis/changed_templates.sh
@@ -10,9 +10,10 @@ case "$tip" in
 	*)         tip="${tip%% *}" ;;
 esac
 
-base="$(git merge-base FETCH_HEAD "$tip")" || {
-	echo "Your branches is based on too old copy."
-	echo "Please rebase to newest copy."
+base="$(git merge-base FETCH_HEAD "$tip")"
+
+[ $(git rev-list --count "$tip" "^$base") -lt 200 ] || {
+	echo "::error title=Branch out of date::Your branch is too out of date. Please rebase on upstream and force-push."
 	exit 1
 }