* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
@ 2023-09-06 1:04 ` classabbyamp
2023-09-06 1:16 ` classabbyamp
` (16 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-06 1:04 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 3375 bytes --]
From cce3db1994d9aaf214399bf0ac78f968dc60b39f Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 1/2] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
---
.github/workflows/cycles.yml | 23 +++++++++++++++++++++--
1 file changed, 21 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..e80b97eb723e9 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,13 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Find cycles
run: |
common/scripts/xbps-cycles.py | tee cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +57,11 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - run: |
+ if [ -s cycles ]; then
+ echo "Build cycles found:"
+ cat cycles
+ exit 42
+ else
+ echo "No cycles found"
+ fi
From 5ab5fb17bf8a757ab3b60e20d9a8d3e878d9d5c4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 2/2] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index f046f669df41c..7ecf6f84aef19 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.38.0
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
2023-09-06 1:04 ` [PR PATCH] [Updated] " classabbyamp
@ 2023-09-06 1:16 ` classabbyamp
2023-09-06 3:02 ` classabbyamp
` (15 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-06 1:16 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 4416 bytes --]
From cce3db1994d9aaf214399bf0ac78f968dc60b39f Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 1/3] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
---
.github/workflows/cycles.yml | 23 +++++++++++++++++++++--
1 file changed, 21 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..e80b97eb723e9 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,13 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Find cycles
run: |
common/scripts/xbps-cycles.py | tee cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +57,11 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - run: |
+ if [ -s cycles ]; then
+ echo "Build cycles found:"
+ cat cycles
+ exit 42
+ else
+ echo "No cycles found"
+ fi
From 5ab5fb17bf8a757ab3b60e20d9a8d3e878d9d5c4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 2/3] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index f046f669df41c..7ecf6f84aef19 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.38.0
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 59b9ca5fa0c20072f8da6b9983ebe698360914ad Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:16:30 -0400
Subject: [PATCH 3/3] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index e80b97eb723e9..608cd10d17677 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -57,11 +57,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
- - run: |
- if [ -s cycles ]; then
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
echo "Build cycles found:"
- cat cycles
- exit 42
+ rv=1
else
- echo "No cycles found"
+ echo "No cycles found:"
+ rv=0
fi
+ cat cycles
+ exit $rv
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
2023-09-06 1:04 ` [PR PATCH] [Updated] " classabbyamp
2023-09-06 1:16 ` classabbyamp
@ 2023-09-06 3:02 ` classabbyamp
2023-09-06 3:03 ` classabbyamp
` (14 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-06 3:02 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 6406 bytes --]
From 47b9d0735df69adc6422dfa07b30346386edd233 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/3] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From e14f9abd092478223c58f2c633c758386e496b19 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/3] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..7091d26abbf2d 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v3
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v3
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From c9b07bd55214c27fd0660ed2221282faa92fb9e3 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/3] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index f046f669df41c..7ecf6f84aef19 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.38.0
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (2 preceding siblings ...)
2023-09-06 3:02 ` classabbyamp
@ 2023-09-06 3:03 ` classabbyamp
2023-09-06 3:13 ` classabbyamp
` (13 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-06 3:03 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 7266 bytes --]
From 47b9d0735df69adc6422dfa07b30346386edd233 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/4] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From e14f9abd092478223c58f2c633c758386e496b19 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/4] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..7091d26abbf2d 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v3
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v3
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From c9b07bd55214c27fd0660ed2221282faa92fb9e3 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/4] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index f046f669df41c..7ecf6f84aef19 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.38.0
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 6b286d88cc4827673606d7264654c956f1b29cf2 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 23:03:41 -0400
Subject: [PATCH 4/4] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 7091d26abbf2d..31075cccdf887 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -31,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (3 preceding siblings ...)
2023-09-06 3:03 ` classabbyamp
@ 2023-09-06 3:13 ` classabbyamp
2023-09-06 3:14 ` classabbyamp
` (12 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-06 3:13 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 6413 bytes --]
From 47b9d0735df69adc6422dfa07b30346386edd233 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/3] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 74b475df9bf348c29cc706ea0b50e1558b14608b Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/3] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..31075cccdf887 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v3
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v3
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 0f76076198a2f59a8ae5cd13a0fa86f33931c189 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/3] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index f046f669df41c..7ecf6f84aef19 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.38.0
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (4 preceding siblings ...)
2023-09-06 3:13 ` classabbyamp
@ 2023-09-06 3:14 ` classabbyamp
2023-09-07 16:08 ` [PR PATCH] [Updated] " classabbyamp
` (11 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-06 3:14 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 286 bytes --]
New comment by classabbyamp on void-packages repository
https://github.com/void-linux/void-packages/pull/45933#issuecomment-1707592401
Comment:
seems like running the cache save/restore actions in the container within ci isn't supported: https://github.com/actions/runner/issues/2115
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (5 preceding siblings ...)
2023-09-06 3:14 ` classabbyamp
@ 2023-09-07 16:08 ` classabbyamp
2023-09-07 16:09 ` classabbyamp
` (10 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-07 16:08 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 7590 bytes --]
From 47b9d0735df69adc6422dfa07b30346386edd233 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/4] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 74b475df9bf348c29cc706ea0b50e1558b14608b Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/4] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..31075cccdf887 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v3
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v3
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 0f76076198a2f59a8ae5cd13a0fa86f33931c189 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/4] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index f046f669df41c..7ecf6f84aef19 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.38.0
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 9f85012bfb5ba1ba089ef99e382d27449f5be3a0 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:08:03 -0400
Subject: [PATCH 4/4] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 31075cccdf887..c85d789be2d56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,7 +43,7 @@ jobs:
common/travis/prepare.sh
- name: Load cached dependencies
id: cache-restore
- uses: actions/cache/restore@v3
+ uses: actions/cache/restore@v2
with:
key: xbps-cycles
path: /xbps-cycles
@@ -53,7 +53,7 @@ jobs:
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- name: Save cached dependencies
if: ${{ github.event_name == 'schedule' }}
- uses: actions/cache/save@v3
+ uses: actions/cache/save@v2
with:
key: ${{ steps.cache-restore.outputs.cache-primary-key }}
path: /xbps-cycles
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (6 preceding siblings ...)
2023-09-07 16:08 ` [PR PATCH] [Updated] " classabbyamp
@ 2023-09-07 16:09 ` classabbyamp
2023-09-07 16:12 ` classabbyamp
` (9 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-07 16:09 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 6413 bytes --]
From 154742a9b2da90d057e7bbd592a575139a1011b6 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/3] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 3f637be0bb5691b836411e0d4fa555eacdb16c1d Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/3] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..c85d789be2d56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v2
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v2
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 6784ec1a7c25845e8a4166c3cd7b765448cb4261 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/3] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index 315131953a149..4a4a8e2073027 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.39.1
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (7 preceding siblings ...)
2023-09-07 16:09 ` classabbyamp
@ 2023-09-07 16:12 ` classabbyamp
2023-09-07 16:27 ` classabbyamp
` (8 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-07 16:12 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 7693 bytes --]
From 154742a9b2da90d057e7bbd592a575139a1011b6 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/4] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 3f637be0bb5691b836411e0d4fa555eacdb16c1d Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/4] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..c85d789be2d56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v2
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v2
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 6784ec1a7c25845e8a4166c3cd7b765448cb4261 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/4] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index 315131953a149..4a4a8e2073027 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.39.1
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 108ea32f2178939430089a79523791567483093c Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:12:06 -0400
Subject: [PATCH 4/4] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index c85d789be2d56..20a69589736aa 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,7 +43,7 @@ jobs:
common/travis/prepare.sh
- name: Load cached dependencies
id: cache-restore
- uses: actions/cache/restore@v2
+ uses: actions/cache@v2
with:
key: xbps-cycles
path: /xbps-cycles
@@ -51,12 +51,6 @@ jobs:
run: |
mkdir -p /xbps-cycles
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- - name: Save cached dependencies
- if: ${{ github.event_name == 'schedule' }}
- uses: actions/cache/save@v2
- with:
- key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- path: /xbps-cycles
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (8 preceding siblings ...)
2023-09-07 16:12 ` classabbyamp
@ 2023-09-07 16:27 ` classabbyamp
2023-09-07 16:29 ` classabbyamp
` (7 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-07 16:27 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 10942 bytes --]
From 154742a9b2da90d057e7bbd592a575139a1011b6 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/5] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 3f637be0bb5691b836411e0d4fa555eacdb16c1d Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/5] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..c85d789be2d56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v2
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v2
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 6784ec1a7c25845e8a4166c3cd7b765448cb4261 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/5] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index 315131953a149..4a4a8e2073027 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.39.1
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 108ea32f2178939430089a79523791567483093c Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:12:06 -0400
Subject: [PATCH 4/5] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index c85d789be2d56..20a69589736aa 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,7 +43,7 @@ jobs:
common/travis/prepare.sh
- name: Load cached dependencies
id: cache-restore
- uses: actions/cache/restore@v2
+ uses: actions/cache@v2
with:
key: xbps-cycles
path: /xbps-cycles
@@ -51,12 +51,6 @@ jobs:
run: |
mkdir -p /xbps-cycles
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- - name: Save cached dependencies
- if: ${{ github.event_name == 'schedule' }}
- uses: actions/cache/save@v2
- with:
- key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- path: /xbps-cycles
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
From cf950623b8ab047a4fef770e4b656343076c5ea4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:27:43 -0400
Subject: [PATCH 5/5] fixup! fixup! .github/workflows/cycles.yaml: run on PRs,
don't run on forks
---
.github/workflows/cycles.yml | 35 ++++++++++++++++++++++-------------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 20a69589736aa..cb73af6ca7e55 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,6 +3,7 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ workflow_dispatch:
pull_request:
paths:
- 'srcpkgs/**'
@@ -10,11 +11,8 @@ on:
jobs:
cycles:
runs-on: ubuntu-latest
- # run only if on the main repo or on pull requests
- if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
+ if: ${{ github.event_name == 'pull_request' || ( github.event_name == 'schedule' && github.repository_owner == 'void-linux' ) }}
permissions:
- # this will only apply to scheduled runs
- # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -31,7 +29,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx nodejs
+ xbps-install -y python3-networkx github-cli
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -41,20 +39,30 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Load cached dependencies
- id: cache-restore
- uses: actions/cache@v2
- with:
- key: xbps-cycles
- path: /xbps-cycles
- - name: Find cycles
+
+ - name: Restore dependency cache
run: |
mkdir -p /xbps-cycles
+
+ run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
+ if [ -n "$run" ]; then
+ gh run download "$run" -D /xbps-cycles -n dependency-cache
+ fi
+
+ - name: Find cycles
+ run: |
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+
+ - name: Save dependency cache
+ uses: actions/upload-artifact@v3
+ with:
+ name: dependency-cache
+ path: /xbps-cycles
+ retention-days: 5
+
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
- xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -64,6 +72,7 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+
- name: Summary
run: |
if grep -q '^Cycle:' cycles; then
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (9 preceding siblings ...)
2023-09-07 16:27 ` classabbyamp
@ 2023-09-07 16:29 ` classabbyamp
2023-09-07 16:31 ` classabbyamp
` (6 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-07 16:29 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 11705 bytes --]
From 154742a9b2da90d057e7bbd592a575139a1011b6 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/6] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 3f637be0bb5691b836411e0d4fa555eacdb16c1d Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/6] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..c85d789be2d56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v2
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v2
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 6784ec1a7c25845e8a4166c3cd7b765448cb4261 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/6] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index 315131953a149..4a4a8e2073027 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.39.1
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 108ea32f2178939430089a79523791567483093c Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:12:06 -0400
Subject: [PATCH 4/6] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index c85d789be2d56..20a69589736aa 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,7 +43,7 @@ jobs:
common/travis/prepare.sh
- name: Load cached dependencies
id: cache-restore
- uses: actions/cache/restore@v2
+ uses: actions/cache@v2
with:
key: xbps-cycles
path: /xbps-cycles
@@ -51,12 +51,6 @@ jobs:
run: |
mkdir -p /xbps-cycles
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- - name: Save cached dependencies
- if: ${{ github.event_name == 'schedule' }}
- uses: actions/cache/save@v2
- with:
- key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- path: /xbps-cycles
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
From cf950623b8ab047a4fef770e4b656343076c5ea4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:27:43 -0400
Subject: [PATCH 5/6] fixup! fixup! .github/workflows/cycles.yaml: run on PRs,
don't run on forks
---
.github/workflows/cycles.yml | 35 ++++++++++++++++++++++-------------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 20a69589736aa..cb73af6ca7e55 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,6 +3,7 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ workflow_dispatch:
pull_request:
paths:
- 'srcpkgs/**'
@@ -10,11 +11,8 @@ on:
jobs:
cycles:
runs-on: ubuntu-latest
- # run only if on the main repo or on pull requests
- if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
+ if: ${{ github.event_name == 'pull_request' || ( github.event_name == 'schedule' && github.repository_owner == 'void-linux' ) }}
permissions:
- # this will only apply to scheduled runs
- # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -31,7 +29,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx nodejs
+ xbps-install -y python3-networkx github-cli
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -41,20 +39,30 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Load cached dependencies
- id: cache-restore
- uses: actions/cache@v2
- with:
- key: xbps-cycles
- path: /xbps-cycles
- - name: Find cycles
+
+ - name: Restore dependency cache
run: |
mkdir -p /xbps-cycles
+
+ run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
+ if [ -n "$run" ]; then
+ gh run download "$run" -D /xbps-cycles -n dependency-cache
+ fi
+
+ - name: Find cycles
+ run: |
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+
+ - name: Save dependency cache
+ uses: actions/upload-artifact@v3
+ with:
+ name: dependency-cache
+ path: /xbps-cycles
+ retention-days: 5
+
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
- xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -64,6 +72,7 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+
- name: Summary
run: |
if grep -q '^Cycle:' cycles; then
From d469b60ab8b37dc850bb032b1570d9b0bbb9e9a9 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:29:43 -0400
Subject: [PATCH 6/6] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index cb73af6ca7e55..f09d769c47f56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -48,6 +48,8 @@ jobs:
if [ -n "$run" ]; then
gh run download "$run" -D /xbps-cycles -n dependency-cache
fi
+ env:
+ GH_TOKEN: ${{ github.token }}
- name: Find cycles
run: |
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (10 preceding siblings ...)
2023-09-07 16:29 ` classabbyamp
@ 2023-09-07 16:31 ` classabbyamp
2023-09-07 16:44 ` classabbyamp
` (5 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-07 16:31 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 12772 bytes --]
From 154742a9b2da90d057e7bbd592a575139a1011b6 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/7] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 3f637be0bb5691b836411e0d4fa555eacdb16c1d Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/7] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..c85d789be2d56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v2
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v2
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 6784ec1a7c25845e8a4166c3cd7b765448cb4261 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/7] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index 315131953a149..4a4a8e2073027 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.39.1
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 108ea32f2178939430089a79523791567483093c Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:12:06 -0400
Subject: [PATCH 4/7] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index c85d789be2d56..20a69589736aa 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,7 +43,7 @@ jobs:
common/travis/prepare.sh
- name: Load cached dependencies
id: cache-restore
- uses: actions/cache/restore@v2
+ uses: actions/cache@v2
with:
key: xbps-cycles
path: /xbps-cycles
@@ -51,12 +51,6 @@ jobs:
run: |
mkdir -p /xbps-cycles
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- - name: Save cached dependencies
- if: ${{ github.event_name == 'schedule' }}
- uses: actions/cache/save@v2
- with:
- key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- path: /xbps-cycles
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
From cf950623b8ab047a4fef770e4b656343076c5ea4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:27:43 -0400
Subject: [PATCH 5/7] fixup! fixup! .github/workflows/cycles.yaml: run on PRs,
don't run on forks
---
.github/workflows/cycles.yml | 35 ++++++++++++++++++++++-------------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 20a69589736aa..cb73af6ca7e55 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,6 +3,7 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ workflow_dispatch:
pull_request:
paths:
- 'srcpkgs/**'
@@ -10,11 +11,8 @@ on:
jobs:
cycles:
runs-on: ubuntu-latest
- # run only if on the main repo or on pull requests
- if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
+ if: ${{ github.event_name == 'pull_request' || ( github.event_name == 'schedule' && github.repository_owner == 'void-linux' ) }}
permissions:
- # this will only apply to scheduled runs
- # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -31,7 +29,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx nodejs
+ xbps-install -y python3-networkx github-cli
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -41,20 +39,30 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Load cached dependencies
- id: cache-restore
- uses: actions/cache@v2
- with:
- key: xbps-cycles
- path: /xbps-cycles
- - name: Find cycles
+
+ - name: Restore dependency cache
run: |
mkdir -p /xbps-cycles
+
+ run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
+ if [ -n "$run" ]; then
+ gh run download "$run" -D /xbps-cycles -n dependency-cache
+ fi
+
+ - name: Find cycles
+ run: |
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+
+ - name: Save dependency cache
+ uses: actions/upload-artifact@v3
+ with:
+ name: dependency-cache
+ path: /xbps-cycles
+ retention-days: 5
+
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
- xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -64,6 +72,7 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+
- name: Summary
run: |
if grep -q '^Cycle:' cycles; then
From d469b60ab8b37dc850bb032b1570d9b0bbb9e9a9 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:29:43 -0400
Subject: [PATCH 6/7] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index cb73af6ca7e55..f09d769c47f56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -48,6 +48,8 @@ jobs:
if [ -n "$run" ]; then
gh run download "$run" -D /xbps-cycles -n dependency-cache
fi
+ env:
+ GH_TOKEN: ${{ github.token }}
- name: Find cycles
run: |
From fb81dee5c33d1dcf0649d7a739da672a6a14eae8 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:31:43 -0400
Subject: [PATCH 7/7] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index f09d769c47f56..d5ea25f2bb2fb 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -42,11 +42,12 @@ jobs:
- name: Restore dependency cache
run: |
+ set -x
mkdir -p /xbps-cycles
run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
if [ -n "$run" ]; then
- gh run download "$run" -D /xbps-cycles -n dependency-cache
+ gh run download "$run" -D /xbps-cycles -n dependency-cache || true
fi
env:
GH_TOKEN: ${{ github.token }}
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (11 preceding siblings ...)
2023-09-07 16:31 ` classabbyamp
@ 2023-09-07 16:44 ` classabbyamp
2023-09-07 16:53 ` Chocimier
` (4 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-07 16:44 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 13842 bytes --]
From 154742a9b2da90d057e7bbd592a575139a1011b6 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/8] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 3f637be0bb5691b836411e0d4fa555eacdb16c1d Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/8] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..c85d789be2d56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v2
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v2
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 6784ec1a7c25845e8a4166c3cd7b765448cb4261 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/8] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index 315131953a149..4a4a8e2073027 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.39.1
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 108ea32f2178939430089a79523791567483093c Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:12:06 -0400
Subject: [PATCH 4/8] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index c85d789be2d56..20a69589736aa 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,7 +43,7 @@ jobs:
common/travis/prepare.sh
- name: Load cached dependencies
id: cache-restore
- uses: actions/cache/restore@v2
+ uses: actions/cache@v2
with:
key: xbps-cycles
path: /xbps-cycles
@@ -51,12 +51,6 @@ jobs:
run: |
mkdir -p /xbps-cycles
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- - name: Save cached dependencies
- if: ${{ github.event_name == 'schedule' }}
- uses: actions/cache/save@v2
- with:
- key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- path: /xbps-cycles
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
From cf950623b8ab047a4fef770e4b656343076c5ea4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:27:43 -0400
Subject: [PATCH 5/8] fixup! fixup! .github/workflows/cycles.yaml: run on PRs,
don't run on forks
---
.github/workflows/cycles.yml | 35 ++++++++++++++++++++++-------------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 20a69589736aa..cb73af6ca7e55 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,6 +3,7 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ workflow_dispatch:
pull_request:
paths:
- 'srcpkgs/**'
@@ -10,11 +11,8 @@ on:
jobs:
cycles:
runs-on: ubuntu-latest
- # run only if on the main repo or on pull requests
- if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
+ if: ${{ github.event_name == 'pull_request' || ( github.event_name == 'schedule' && github.repository_owner == 'void-linux' ) }}
permissions:
- # this will only apply to scheduled runs
- # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -31,7 +29,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx nodejs
+ xbps-install -y python3-networkx github-cli
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -41,20 +39,30 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Load cached dependencies
- id: cache-restore
- uses: actions/cache@v2
- with:
- key: xbps-cycles
- path: /xbps-cycles
- - name: Find cycles
+
+ - name: Restore dependency cache
run: |
mkdir -p /xbps-cycles
+
+ run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
+ if [ -n "$run" ]; then
+ gh run download "$run" -D /xbps-cycles -n dependency-cache
+ fi
+
+ - name: Find cycles
+ run: |
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+
+ - name: Save dependency cache
+ uses: actions/upload-artifact@v3
+ with:
+ name: dependency-cache
+ path: /xbps-cycles
+ retention-days: 5
+
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
- xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -64,6 +72,7 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+
- name: Summary
run: |
if grep -q '^Cycle:' cycles; then
From d469b60ab8b37dc850bb032b1570d9b0bbb9e9a9 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:29:43 -0400
Subject: [PATCH 6/8] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index cb73af6ca7e55..f09d769c47f56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -48,6 +48,8 @@ jobs:
if [ -n "$run" ]; then
gh run download "$run" -D /xbps-cycles -n dependency-cache
fi
+ env:
+ GH_TOKEN: ${{ github.token }}
- name: Find cycles
run: |
From fb81dee5c33d1dcf0649d7a739da672a6a14eae8 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:31:43 -0400
Subject: [PATCH 7/8] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index f09d769c47f56..d5ea25f2bb2fb 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -42,11 +42,12 @@ jobs:
- name: Restore dependency cache
run: |
+ set -x
mkdir -p /xbps-cycles
run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
if [ -n "$run" ]; then
- gh run download "$run" -D /xbps-cycles -n dependency-cache
+ gh run download "$run" -D /xbps-cycles -n dependency-cache || true
fi
env:
GH_TOKEN: ${{ github.token }}
From 5f3d80db66e752ca5810c4f53720f0d5078a7941 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:43:58 -0400
Subject: [PATCH 8/8] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index d5ea25f2bb2fb..3f56b072eaf9c 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -49,6 +49,7 @@ jobs:
if [ -n "$run" ]; then
gh run download "$run" -D /xbps-cycles -n dependency-cache || true
fi
+ ls -l /xbps-cycles
env:
GH_TOKEN: ${{ github.token }}
@@ -57,7 +58,7 @@ jobs:
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- name: Save dependency cache
- uses: actions/upload-artifact@v3
+ uses: actions/upload-artifact@v1
with:
name: dependency-cache
path: /xbps-cycles
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (12 preceding siblings ...)
2023-09-07 16:44 ` classabbyamp
@ 2023-09-07 16:53 ` Chocimier
2023-09-07 16:58 ` [PR PATCH] [Updated] " classabbyamp
` (3 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: Chocimier @ 2023-09-07 16:53 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 210 bytes --]
New comment by Chocimier on void-packages repository
https://github.com/void-linux/void-packages/pull/45933#issuecomment-1710489375
Comment:
Merging common/, etc/, xbps-src in should clear and rebuild cache.
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (13 preceding siblings ...)
2023-09-07 16:53 ` Chocimier
@ 2023-09-07 16:58 ` classabbyamp
2023-09-07 17:22 ` classabbyamp
` (2 subsequent siblings)
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-07 16:58 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 15800 bytes --]
From 154742a9b2da90d057e7bbd592a575139a1011b6 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 1/9] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 3f637be0bb5691b836411e0d4fa555eacdb16c1d Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 2/9] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..c85d789be2d56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v2
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v2
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 6784ec1a7c25845e8a4166c3cd7b765448cb4261 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 3/9] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index 315131953a149..4a4a8e2073027 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.39.1
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 108ea32f2178939430089a79523791567483093c Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:12:06 -0400
Subject: [PATCH 4/9] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index c85d789be2d56..20a69589736aa 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,7 +43,7 @@ jobs:
common/travis/prepare.sh
- name: Load cached dependencies
id: cache-restore
- uses: actions/cache/restore@v2
+ uses: actions/cache@v2
with:
key: xbps-cycles
path: /xbps-cycles
@@ -51,12 +51,6 @@ jobs:
run: |
mkdir -p /xbps-cycles
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- - name: Save cached dependencies
- if: ${{ github.event_name == 'schedule' }}
- uses: actions/cache/save@v2
- with:
- key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- path: /xbps-cycles
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
From cf950623b8ab047a4fef770e4b656343076c5ea4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:27:43 -0400
Subject: [PATCH 5/9] fixup! fixup! .github/workflows/cycles.yaml: run on PRs,
don't run on forks
---
.github/workflows/cycles.yml | 35 ++++++++++++++++++++++-------------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 20a69589736aa..cb73af6ca7e55 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,6 +3,7 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ workflow_dispatch:
pull_request:
paths:
- 'srcpkgs/**'
@@ -10,11 +11,8 @@ on:
jobs:
cycles:
runs-on: ubuntu-latest
- # run only if on the main repo or on pull requests
- if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
+ if: ${{ github.event_name == 'pull_request' || ( github.event_name == 'schedule' && github.repository_owner == 'void-linux' ) }}
permissions:
- # this will only apply to scheduled runs
- # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -31,7 +29,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx nodejs
+ xbps-install -y python3-networkx github-cli
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -41,20 +39,30 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Load cached dependencies
- id: cache-restore
- uses: actions/cache@v2
- with:
- key: xbps-cycles
- path: /xbps-cycles
- - name: Find cycles
+
+ - name: Restore dependency cache
run: |
mkdir -p /xbps-cycles
+
+ run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
+ if [ -n "$run" ]; then
+ gh run download "$run" -D /xbps-cycles -n dependency-cache
+ fi
+
+ - name: Find cycles
+ run: |
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+
+ - name: Save dependency cache
+ uses: actions/upload-artifact@v3
+ with:
+ name: dependency-cache
+ path: /xbps-cycles
+ retention-days: 5
+
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
- xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -64,6 +72,7 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+
- name: Summary
run: |
if grep -q '^Cycle:' cycles; then
From d469b60ab8b37dc850bb032b1570d9b0bbb9e9a9 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:29:43 -0400
Subject: [PATCH 6/9] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index cb73af6ca7e55..f09d769c47f56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -48,6 +48,8 @@ jobs:
if [ -n "$run" ]; then
gh run download "$run" -D /xbps-cycles -n dependency-cache
fi
+ env:
+ GH_TOKEN: ${{ github.token }}
- name: Find cycles
run: |
From fb81dee5c33d1dcf0649d7a739da672a6a14eae8 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:31:43 -0400
Subject: [PATCH 7/9] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index f09d769c47f56..d5ea25f2bb2fb 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -42,11 +42,12 @@ jobs:
- name: Restore dependency cache
run: |
+ set -x
mkdir -p /xbps-cycles
run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
if [ -n "$run" ]; then
- gh run download "$run" -D /xbps-cycles -n dependency-cache
+ gh run download "$run" -D /xbps-cycles -n dependency-cache || true
fi
env:
GH_TOKEN: ${{ github.token }}
From 5f3d80db66e752ca5810c4f53720f0d5078a7941 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:43:58 -0400
Subject: [PATCH 8/9] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index d5ea25f2bb2fb..3f56b072eaf9c 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -49,6 +49,7 @@ jobs:
if [ -n "$run" ]; then
gh run download "$run" -D /xbps-cycles -n dependency-cache || true
fi
+ ls -l /xbps-cycles
env:
GH_TOKEN: ${{ github.token }}
@@ -57,7 +58,7 @@ jobs:
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- name: Save dependency cache
- uses: actions/upload-artifact@v3
+ uses: actions/upload-artifact@v1
with:
name: dependency-cache
path: /xbps-cycles
From bd697cedafbf6dff236763ed31d8dbc073f83881 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:58:10 -0400
Subject: [PATCH 9/9] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 21 ++++++++++-----------
1 file changed, 10 insertions(+), 11 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 3f56b072eaf9c..140055147f00e 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,26 +43,19 @@ jobs:
- name: Restore dependency cache
run: |
set -x
- mkdir -p /xbps-cycles
+ mkdir -p "$GITHUB_WORKSPACE/xbps-cycles"
run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
if [ -n "$run" ]; then
- gh run download "$run" -D /xbps-cycles -n dependency-cache || true
+ gh run download "$run" -D "$GITHUB_WORKSPACE/xbps-cycles" -n dependency-cache || true
fi
- ls -l /xbps-cycles
+ ls -l "$GITHUB_WORKSPACE/xbps-cycles"
env:
GH_TOKEN: ${{ github.token }}
- name: Find cycles
run: |
- common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
-
- - name: Save dependency cache
- uses: actions/upload-artifact@v1
- with:
- name: dependency-cache
- path: /xbps-cycles
- retention-days: 5
+ common/scripts/xbps-cycles.py -c "$GITHUB_WORKSPACE/xbps-cycles" | tee cycles
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
@@ -88,3 +81,9 @@ jobs:
fi
cat cycles
exit $rv
+
+ - name: Save dependency cache
+ uses: actions/upload-artifact@v1
+ with:
+ name: dependency-cache
+ path: xbps-cycles
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Updated] .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (14 preceding siblings ...)
2023-09-07 16:58 ` [PR PATCH] [Updated] " classabbyamp
@ 2023-09-07 17:22 ` classabbyamp
2023-12-07 1:46 ` github-actions
2023-12-07 3:13 ` [PR PATCH] [Closed]: " classabbyamp
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-09-07 17:22 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 826 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages ci/cycle-lint
https://github.com/void-linux/void-packages/pull/45933
.github/workflows/cycles.yaml: run on PRs, don't run on forks
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
A patch file from https://github.com/void-linux/void-packages/pull/45933.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-ci/cycle-lint-45933.patch --]
[-- Type: text/x-diff, Size: 17089 bytes --]
From 154742a9b2da90d057e7bbd592a575139a1011b6 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 22:48:55 -0400
Subject: [PATCH 01/10] common/scripts/xbps-cycles.py: improve caching
by storing the hash of the template in the cache, the cache can be invalidated
when the template is changed.
---
common/scripts/xbps-cycles.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/common/scripts/xbps-cycles.py b/common/scripts/xbps-cycles.py
index 7710381eb2925..1ae8f4199a133 100755
--- a/common/scripts/xbps-cycles.py
+++ b/common/scripts/xbps-cycles.py
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
import os
-import sys
import glob
+import hashlib
import subprocess
import multiprocessing
@@ -11,6 +11,17 @@
import networkx as nx
+def hash_template(pkg, xbpsdir):
+ '''
+ Hashes a template with md5 for cache keying
+ '''
+ try:
+ with open(os.path.join(xbpsdir, 'srcpkgs', pkg, 'template'), "rb") as tmpl:
+ return hashlib.file_digest(tmpl, hashlib.md5).hexdigest()
+ except FileNotFoundError:
+ return '0'
+
+
def enum_depends(pkg, xbpsdir, cachedir):
'''
Return a pair (pkg, [dependencies]), where [dependencies] is the list
@@ -21,15 +32,21 @@ def enum_depends(pkg, xbpsdir, cachedir):
<xbpsdir>/xbps-src show-build-deps <pkg>
unless <cachedir>/deps-<pkg> file exist, in that case it is read.
+ To ensure the cache for a package is invalidated when its template changes,
+ the template is hashed and that hash is stored on the first line of the
+ package's cache file.
If the return code of this call nonzero, a message will be printed but
the package will treated as if it has no dependencies.
'''
if cachedir:
cachepath = os.path.join(cachedir, 'deps-' + pkg)
+ newhash = hash_template(pkg, xbpsdir)
try:
with open(cachepath) as f:
- return pkg, [l.strip() for l in f]
+ oldhash = f.readline().strip()
+ if oldhash == newhash:
+ return pkg, [l.strip() for l in f]
except FileNotFoundError:
pass
@@ -44,6 +61,7 @@ def enum_depends(pkg, xbpsdir, cachedir):
deps = [d for d in deps.decode('utf-8').split('\n') if d]
if cachedir:
with open(cachepath, 'w') as f:
+ print(newhash, file=f)
for d in deps:
print(d, file=f)
From 3f637be0bb5691b836411e0d4fa555eacdb16c1d Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 20:10:58 -0400
Subject: [PATCH 02/10] .github/workflows/cycles.yaml: run on PRs, don't run on
forks
- running on forks is kinda useless and just gives people useless error
messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes
the cron cycle check more of a backup/just-in-case for things that
slip past the CI
- speed things up by caching dependencies in github's cache
functionality. this is only written by the scheduled job, not PRs.
---
.github/workflows/cycles.yml | 41 +++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 8eaecdc70287b..c85d789be2d56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,11 +3,18 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ pull_request:
+ paths:
+ - 'srcpkgs/**'
jobs:
cycles:
runs-on: ubuntu-latest
+ # run only if on the main repo or on pull requests
+ if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
permissions:
+ # this will only apply to scheduled runs
+ # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -24,7 +31,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx github-cli
+ xbps-install -y python3-networkx nodejs
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -34,9 +41,26 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Find cycles and open issues
+ - name: Load cached dependencies
+ id: cache-restore
+ uses: actions/cache/restore@v2
+ with:
+ key: xbps-cycles
+ path: /xbps-cycles
+ - name: Find cycles
run: |
- common/scripts/xbps-cycles.py | tee cycles
+ mkdir -p /xbps-cycles
+ common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+ - name: Save cached dependencies
+ if: ${{ github.event_name == 'schedule' }}
+ uses: actions/cache/save@v2
+ with:
+ key: ${{ steps.cache-restore.outputs.cache-primary-key }}
+ path: /xbps-cycles
+ - name: Open issues
+ if: ${{ github.event_name == 'schedule' }}
+ run: |
+ xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -46,3 +70,14 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+ - name: Summary
+ run: |
+ if grep -q '^Cycle:' cycles; then
+ echo "Build cycles found:"
+ rv=1
+ else
+ echo "No cycles found:"
+ rv=0
+ fi
+ cat cycles
+ exit $rv
From 6784ec1a7c25845e8a4166c3cd7b765448cb4261 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 5 Sep 2023 21:04:08 -0400
Subject: [PATCH 03/10] chezmoi:
---
srcpkgs/chezmoi/template | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/srcpkgs/chezmoi/template b/srcpkgs/chezmoi/template
index 315131953a149..4a4a8e2073027 100644
--- a/srcpkgs/chezmoi/template
+++ b/srcpkgs/chezmoi/template
@@ -1,7 +1,7 @@
# Template file for 'chezmoi'
pkgname=chezmoi
version=2.39.1
-revision=1
+revision=2
build_style=go
go_import_path="github.com/twpayne/chezmoi/v2"
go_build_tags="noembeddocs noupgrade"
From 108ea32f2178939430089a79523791567483093c Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:12:06 -0400
Subject: [PATCH 04/10] fixup! .github/workflows/cycles.yaml: run on PRs, don't
run on forks
---
.github/workflows/cycles.yml | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index c85d789be2d56..20a69589736aa 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,7 +43,7 @@ jobs:
common/travis/prepare.sh
- name: Load cached dependencies
id: cache-restore
- uses: actions/cache/restore@v2
+ uses: actions/cache@v2
with:
key: xbps-cycles
path: /xbps-cycles
@@ -51,12 +51,6 @@ jobs:
run: |
mkdir -p /xbps-cycles
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- - name: Save cached dependencies
- if: ${{ github.event_name == 'schedule' }}
- uses: actions/cache/save@v2
- with:
- key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- path: /xbps-cycles
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
From cf950623b8ab047a4fef770e4b656343076c5ea4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:27:43 -0400
Subject: [PATCH 05/10] fixup! fixup! .github/workflows/cycles.yaml: run on
PRs, don't run on forks
---
.github/workflows/cycles.yml | 35 ++++++++++++++++++++++-------------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 20a69589736aa..cb73af6ca7e55 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -3,6 +3,7 @@ name: 'Cycle Check'
on:
schedule:
- cron: '0 18 * * *'
+ workflow_dispatch:
pull_request:
paths:
- 'srcpkgs/**'
@@ -10,11 +11,8 @@ on:
jobs:
cycles:
runs-on: ubuntu-latest
- # run only if on the main repo or on pull requests
- if: ${{ github.event_name == 'pull_request' || github.repository_owner == 'void-linux' }}
+ if: ${{ github.event_name == 'pull_request' || ( github.event_name == 'schedule' && github.repository_owner == 'void-linux' ) }}
permissions:
- # this will only apply to scheduled runs
- # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
issues: write
container:
image: 'ghcr.io/void-linux/void-buildroot-musl:20230904R2'
@@ -31,7 +29,7 @@ jobs:
# Upgrade again (in case there was a xbps update)
xbps-install -yu
# Install script dependencies
- xbps-install -y python3-networkx nodejs
+ xbps-install -y python3-networkx github-cli
- name: Clone and checkout
uses: classabbyamp/treeless-checkout-action@v1
@@ -41,20 +39,30 @@ jobs:
ln -s "$(pwd)" /hostrepo &&
common/travis/set_mirror.sh &&
common/travis/prepare.sh
- - name: Load cached dependencies
- id: cache-restore
- uses: actions/cache@v2
- with:
- key: xbps-cycles
- path: /xbps-cycles
- - name: Find cycles
+
+ - name: Restore dependency cache
run: |
mkdir -p /xbps-cycles
+
+ run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
+ if [ -n "$run" ]; then
+ gh run download "$run" -D /xbps-cycles -n dependency-cache
+ fi
+
+ - name: Find cycles
+ run: |
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
+
+ - name: Save dependency cache
+ uses: actions/upload-artifact@v3
+ with:
+ name: dependency-cache
+ path: /xbps-cycles
+ retention-days: 5
+
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
run: |
- xbps-install -y github-cli
grep 'Cycle:' cycles | while read -r line; do
if gh issue list -R "$GITHUB_REPOSITORY" -S "$line" | grep .; then
printf "Issue on '%s' already exists.\n" "$line"
@@ -64,6 +72,7 @@ jobs:
done
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN }}
+
- name: Summary
run: |
if grep -q '^Cycle:' cycles; then
From d469b60ab8b37dc850bb032b1570d9b0bbb9e9a9 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:29:43 -0400
Subject: [PATCH 06/10] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index cb73af6ca7e55..f09d769c47f56 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -48,6 +48,8 @@ jobs:
if [ -n "$run" ]; then
gh run download "$run" -D /xbps-cycles -n dependency-cache
fi
+ env:
+ GH_TOKEN: ${{ github.token }}
- name: Find cycles
run: |
From fb81dee5c33d1dcf0649d7a739da672a6a14eae8 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:31:43 -0400
Subject: [PATCH 07/10] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index f09d769c47f56..d5ea25f2bb2fb 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -42,11 +42,12 @@ jobs:
- name: Restore dependency cache
run: |
+ set -x
mkdir -p /xbps-cycles
run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
if [ -n "$run" ]; then
- gh run download "$run" -D /xbps-cycles -n dependency-cache
+ gh run download "$run" -D /xbps-cycles -n dependency-cache || true
fi
env:
GH_TOKEN: ${{ github.token }}
From 5f3d80db66e752ca5810c4f53720f0d5078a7941 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:43:58 -0400
Subject: [PATCH 08/10] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index d5ea25f2bb2fb..3f56b072eaf9c 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -49,6 +49,7 @@ jobs:
if [ -n "$run" ]; then
gh run download "$run" -D /xbps-cycles -n dependency-cache || true
fi
+ ls -l /xbps-cycles
env:
GH_TOKEN: ${{ github.token }}
@@ -57,7 +58,7 @@ jobs:
common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
- name: Save dependency cache
- uses: actions/upload-artifact@v3
+ uses: actions/upload-artifact@v1
with:
name: dependency-cache
path: /xbps-cycles
From bd697cedafbf6dff236763ed31d8dbc073f83881 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 12:58:10 -0400
Subject: [PATCH 09/10] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 21 ++++++++++-----------
1 file changed, 10 insertions(+), 11 deletions(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 3f56b072eaf9c..140055147f00e 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -43,26 +43,19 @@ jobs:
- name: Restore dependency cache
run: |
set -x
- mkdir -p /xbps-cycles
+ mkdir -p "$GITHUB_WORKSPACE/xbps-cycles"
run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
if [ -n "$run" ]; then
- gh run download "$run" -D /xbps-cycles -n dependency-cache || true
+ gh run download "$run" -D "$GITHUB_WORKSPACE/xbps-cycles" -n dependency-cache || true
fi
- ls -l /xbps-cycles
+ ls -l "$GITHUB_WORKSPACE/xbps-cycles"
env:
GH_TOKEN: ${{ github.token }}
- name: Find cycles
run: |
- common/scripts/xbps-cycles.py -c /xbps-cycles | tee cycles
-
- - name: Save dependency cache
- uses: actions/upload-artifact@v1
- with:
- name: dependency-cache
- path: /xbps-cycles
- retention-days: 5
+ common/scripts/xbps-cycles.py -c "$GITHUB_WORKSPACE/xbps-cycles" | tee cycles
- name: Open issues
if: ${{ github.event_name == 'schedule' }}
@@ -88,3 +81,9 @@ jobs:
fi
cat cycles
exit $rv
+
+ - name: Save dependency cache
+ uses: actions/upload-artifact@v1
+ with:
+ name: dependency-cache
+ path: xbps-cycles
From 316420a3b6b9f60e57c8d53f41af90548f571fcf Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Thu, 7 Sep 2023 13:22:20 -0400
Subject: [PATCH 10/10] fixup! fixup! fixup! .github/workflows/cycles.yaml: run
on PRs, don't run on forks
---
.github/workflows/cycles.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/cycles.yml b/.github/workflows/cycles.yml
index 140055147f00e..a628e4bd29dc3 100644
--- a/.github/workflows/cycles.yml
+++ b/.github/workflows/cycles.yml
@@ -45,7 +45,7 @@ jobs:
set -x
mkdir -p "$GITHUB_WORKSPACE/xbps-cycles"
- run="$(gh run list --status completed --event schedule --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
+ run="$(gh run list --status completed --workflow cycles.yml --limit 1 --json databaseId --jq '.[].databaseId')"
if [ -n "$run" ]; then
gh run download "$run" -D "$GITHUB_WORKSPACE/xbps-cycles" -n dependency-cache || true
fi
@@ -83,6 +83,7 @@ jobs:
exit $rv
- name: Save dependency cache
+ # if: ${{ github.event_name == 'schedule' }}
uses: actions/upload-artifact@v1
with:
name: dependency-cache
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (15 preceding siblings ...)
2023-09-07 17:22 ` classabbyamp
@ 2023-12-07 1:46 ` github-actions
2023-12-07 3:13 ` [PR PATCH] [Closed]: " classabbyamp
17 siblings, 0 replies; 19+ messages in thread
From: github-actions @ 2023-12-07 1:46 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 305 bytes --]
New comment by github-actions[bot] on void-packages repository
https://github.com/void-linux/void-packages/pull/45933#issuecomment-1844024200
Comment:
Pull Requests become stale 90 days after last activity and are closed 14 days after that. If this pull request is still relevant bump it or assign it.
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PR PATCH] [Closed]: .github/workflows/cycles.yaml: run on PRs, don't run on forks
2023-09-06 1:02 [PR PATCH] .github/workflows/cycles.yaml: run on PRs, don't run on forks classabbyamp
` (16 preceding siblings ...)
2023-12-07 1:46 ` github-actions
@ 2023-12-07 3:13 ` classabbyamp
17 siblings, 0 replies; 19+ messages in thread
From: classabbyamp @ 2023-12-07 3:13 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 654 bytes --]
There's a closed pull request on the void-packages repository
.github/workflows/cycles.yaml: run on PRs, don't run on forks
https://github.com/void-linux/void-packages/pull/45933
Description:
- running on forks is kinda useless and just gives people useless error messages if they sync their master branch at the wrong time
- run cycle check on PRs to catch cycles before they happen. this makes the cron cycle check more of a backup/just-in-case for things that slip past the CI
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**
^ permalink raw reply [flat|nested] 19+ messages in thread