From: classabbyamp <classabbyamp@users.noreply.github.com>
To: ml@inbox.vuxu.org
Subject: Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
Date: Thu, 07 Sep 2023 18:44:06 +0200 [thread overview]
Message-ID: <20230907164406.zFKlbWXEYo5RwQyRyyH8NFkaQ6fqIkYXbqaIeYwT_xM@z> (raw)
In-Reply-To: <gh-mailinglist-notifications-41a7ca26-5023-4802-975b-f1789d68868e-void-packages-45933@inbox.vuxu.org>
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 13842 bytes --]
From 154742a9b2da90d057e7bbd592a575139a1011b6 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/8] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 3f637be0bb5691b836411e0d4fa555eacdb16c1d Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/8] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..c85d789be2d56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v2
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v2
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 6784ec1a7c25845e8a4166c3cd7b765448cb4261 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/8] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index 315131953a149..4a4a8e2073027 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.39.1
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 108ea32f2178939430089a79523791567483093c Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:12:06 -0400
Subject: [PATCH 4/8] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index c85d789be2d56..20a69589736aa 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,7 +43,7 @@ jobs:
common/travis/prepare.sh
- name: Load cached dependencies
id: cache-restore
- uses: actions/cache/restore@v2
+ uses: actions/cache@v2
with:
key: xbps-cycles
path: /xbps-cycles
@@ -51,12 +51,6 @@ jobs:
run: |
mkdir -p /xbps-cycles
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- - name: Save cached dependencies
- if: ${{ github.event_name == 'schedule' }}
- uses: actions/cache/save@v2
- with:
- key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- path: /xbps-cycles
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
From cf950623b8ab047a4fef770e4b656343076c5ea4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:27:43 -0400
Subject: [PATCH 5/8] fixup! fixup! .github/workflows/cycles.yaml: run on PRs,
don't run on forks
---
.github/workflows/cycles.yml | 35 ++++++++++++++++++++++-------------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 20a69589736aa..cb73af6ca7e55 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,6 +3,7 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ workflow_dispatch:
pull_request:
paths:
- 'srcpkgs/**'
@@ -10,11 +11,8 @@ on:
jobs:
cycles:
runs-on: ubuntu-latest
- # run only if on the main repo or on pull requests
- if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
+ if: ${{ github.event_name == 'pull_request' || ( github.event_name == 'schedule' && github.repository_owner == 'void-linux' ) }}
permissions:
- # this will only apply to scheduled runs
- # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -31,7 +29,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx nodejs
+ xbps-install -y python3-networkx github-cli
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -41,20 +39,30 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Load cached dependencies
- id: cache-restore
- uses: actions/cache@v2
- with:
- key: xbps-cycles
- path: /xbps-cycles
- - name: Find cycles
+
+ - name: Restore dependency cache
run: |
mkdir -p /xbps-cycles
+
+ run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
+ if [ -n "$run" ]; then
+ gh run download "$run" -D /xbps-cycles -n dependency-cache
+ fi
+
+ - name: Find cycles
+ run: |
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+
+ - name: Save dependency cache
+ uses: actions/upload-artifact@v3
+ with:
+ name: dependency-cache
+ path: /xbps-cycles
+ retention-days: 5
+
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
- xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -64,6 +72,7 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+
- name: Summary
run: |
if grep -q '^Cycle:' cycles; then
From d469b60ab8b37dc850bb032b1570d9b0bbb9e9a9 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:29:43 -0400
Subject: [PATCH 6/8] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index cb73af6ca7e55..f09d769c47f56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -48,6 +48,8 @@ jobs:
if [ -n "$run" ]; then
gh run download "$run" -D /xbps-cycles -n dependency-cache
fi
+ env:
+ GH_TOKEN: ${{ github.token }}
- name: Find cycles
run: |
From fb81dee5c33d1dcf0649d7a739da672a6a14eae8 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:31:43 -0400
Subject: [PATCH 7/8] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index f09d769c47f56..d5ea25f2bb2fb 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -42,11 +42,12 @@ jobs:
- name: Restore dependency cache
run: |
+ set -x
mkdir -p /xbps-cycles
run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
if [ -n "$run" ]; then
- gh run download "$run" -D /xbps-cycles -n dependency-cache
+ gh run download "$run" -D /xbps-cycles -n dependency-cache || true
fi
env:
GH_TOKEN: ${{ github.token }}
From 5f3d80db66e752ca5810c4f53720f0d5078a7941 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:43:58 -0400
Subject: [PATCH 8/8] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index d5ea25f2bb2fb..3f56b072eaf9c 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -49,6 +49,7 @@ jobs:
if [ -n "$run" ]; then
gh run download "$run" -D /xbps-cycles -n dependency-cache || true
fi
+ ls -l /xbps-cycles
env:
GH_TOKEN: ${{ github.token }}
@@ -57,7 +58,7 @@ jobs:
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- name: Save dependency cache
- uses: actions/upload-artifact@v3
+ uses: actions/upload-artifact@v1
with:
name: dependency-cache
path: /xbps-cycles
next prev parent reply other threads:[~2023-09-07 16:44 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-06 1:02 [PR PATCH] " classabbyamp
2023-09-06 1:04 ` [PR PATCH] [Updated] " classabbyamp
2023-09-06 1:16 ` classabbyamp
2023-09-06 3:02 ` classabbyamp
2023-09-06 3:03 ` classabbyamp
2023-09-06 3:13 ` classabbyamp
2023-09-06 3:14 ` classabbyamp
2023-09-07 16:08 ` [PR PATCH] [Updated] " classabbyamp
2023-09-07 16:09 ` classabbyamp
2023-09-07 16:12 ` classabbyamp
2023-09-07 16:27 ` classabbyamp
2023-09-07 16:29 ` classabbyamp
2023-09-07 16:31 ` classabbyamp
2023-09-07 16:44 ` classabbyamp [this message]
2023-09-07 16:53 ` Chocimier
2023-09-07 16:58 ` [PR PATCH] [Updated] " classabbyamp
2023-09-07 17:22 ` classabbyamp
2023-12-07 1:46 ` github-actions
2023-12-07 3:13 ` [PR PATCH] [Closed]: " classabbyamp
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230907164406.zFKlbWXEYo5RwQyRyyH8NFkaQ6fqIkYXbqaIeYwT_xM@z \
--to=classabbyamp@users.noreply.github.com \
--cc=ml@inbox.vuxu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).