* [PR PATCH] common: retry ftp download during CI
@ 2020-01-29 22:38 voidlinux-github
2020-02-11 22:05 ` [PR PATCH] [Merged]: " voidlinux-github
2020-02-12 18:08 ` voidlinux-github
0 siblings, 2 replies; 3+ messages in thread
From: voidlinux-github @ 2020-01-29 22:38 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 542 bytes --]
There is a new pull request by Chocimier against master on the void-packages repository
https://github.com/Chocimier/void-packages-org file
https://github.com/void-linux/void-packages/pull/18646
common: retry ftp download during CI
Build will [eventually](https://travis-ci.org/Chocimier/void-packages-org/builds/643581137) pass, but may take too long for now, as retry takes 5 minutes. Patch for xbps to customize timeout may decrease needed time.
A patch file from https://github.com/void-linux/void-packages/pull/18646.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-file-18646.patch --]
[-- Type: text/x-diff, Size: 2826 bytes --]
From ac64046408d324262bdc027cdff451364c1c2552 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Piotr=20W=C3=B3jcik?= <chocimier@tlen.pl>
Date: Wed, 29 Jan 2020 23:28:40 +0100
Subject: [PATCH] common: retry ftp download during CI
---
common/hooks/do-fetch/00-distfiles.sh | 19 +++++++++++++++----
common/travis/build.sh | 2 ++
common/xbps-src/shutils/chroot.sh | 1 +
3 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/common/hooks/do-fetch/00-distfiles.sh b/common/hooks/do-fetch/00-distfiles.sh
index 922245fcd21..528eab8896c 100644
--- a/common/hooks/do-fetch/00-distfiles.sh
+++ b/common/hooks/do-fetch/00-distfiles.sh
@@ -198,7 +198,7 @@ try_mirrors() {
hook() {
local srcdir="$XBPS_SRCDISTDIR/$pkgname-$version"
- local dfcount=0 dfgood=0 errors=0
+ local dfcount=0 dfgood=0 errors=0 max_retries
if [ ! -d "$srcdir" ]; then
mkdir -p -m775 "$srcdir"
@@ -266,10 +266,21 @@ hook() {
try_mirrors $curfile $distfile $dfcount $pkgname-$version $f
fi
# If distfile does not exist, download it from the original location.
- if [ ! -f "$distfile" ]; then
- msg_normal "$pkgver: fetching distfile '$curfile'...\n"
- flock "${distfile}.part" $fetch_cmd "$f"
+ if [[ "$FTP_RETRIES" && "${f}" =~ ^ftp:// ]]; then
+ max_retries="$FTP_RETRIES"
+ else
+ max_retries=1
fi
+ for retry in $(seq 1 1 $max_retries); do
+ if [ ! -f "$distfile" ]; then
+ if [ "$retry" == 1 ]; then
+ msg_normal "$pkgver: fetching distfile '$curfile'...\n"
+ else
+ msg_normal "$pkgver: fetch attempt $retry of $max_retries...\n"
+ fi
+ flock "${distfile}.part" $fetch_cmd "$f"
+ fi
+ done
if [ ! -f "$distfile" ]; then
msg_error "$pkgver: failed to fetch $curfile.\n"
fi
diff --git a/common/travis/build.sh b/common/travis/build.sh
index 7732211a59e..1c45910ea69 100755
--- a/common/travis/build.sh
+++ b/common/travis/build.sh
@@ -13,6 +13,8 @@ if [ -r /proc/cpuinfo ]; then
NPROCS=$(grep ^proc /proc/cpuinfo|wc -l)
fi
+export FTP_RETRIES=10
+
for pkg in ${PKGS}; do
/hostrepo/xbps-src -j$NPROCS -H "$HOME"/hostdir $arch pkg "$pkg"
[ $? -eq 1 ] && exit 1
diff --git a/common/xbps-src/shutils/chroot.sh b/common/xbps-src/shutils/chroot.sh
index 228dd2e0399..4f543bf2a8e 100644
--- a/common/xbps-src/shutils/chroot.sh
+++ b/common/xbps-src/shutils/chroot.sh
@@ -173,6 +173,7 @@ chroot_handler() {
${SOCKS_PROXY:+SOCKS_PROXY="${SOCKS_PROXY}"} \
${NO_PROXY:+NO_PROXY="${NO_PROXY}"} \
${HTTP_PROXY_AUTH:+HTTP_PROXY_AUTH="${HTTP_PROXY_AUTH}"} \
+ ${FTP_RETRIES:+FTP_RETRIES="${FTP_RETRIES}"} \
SOURCE_DATE_EPOCH="$SOURCE_DATE_EPOCH" \
XBPS_ALLOW_CHROOT_BREAKOUT="$XBPS_ALLOW_CHROOT_BREAKOUT" \
$XBPS_COMMONDIR/chroot-style/${XBPS_CHROOT_CMD:=uunshare}.sh \
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PR PATCH] [Merged]: common: retry ftp download during CI
2020-01-29 22:38 [PR PATCH] common: retry ftp download during CI voidlinux-github
@ 2020-02-11 22:05 ` voidlinux-github
2020-02-12 18:08 ` voidlinux-github
1 sibling, 0 replies; 3+ messages in thread
From: voidlinux-github @ 2020-02-11 22:05 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 386 bytes --]
There's a merged pull request on the void-packages repository
common: retry ftp download during CI
https://github.com/void-linux/void-packages/pull/18646
Description:
Build will [eventually](https://travis-ci.org/Chocimier/void-packages-org/builds/643581137) pass, but may take too long for now, as retry takes 5 minutes. Patch for xbps to customize timeout may decrease needed time.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: common: retry ftp download during CI
2020-01-29 22:38 [PR PATCH] common: retry ftp download during CI voidlinux-github
2020-02-11 22:05 ` [PR PATCH] [Merged]: " voidlinux-github
@ 2020-02-12 18:08 ` voidlinux-github
1 sibling, 0 replies; 3+ messages in thread
From: voidlinux-github @ 2020-02-12 18:08 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 439 bytes --]
New comment by Chocimier on void-packages repository
https://github.com/void-linux/void-packages/pull/18646#issuecomment-585339505
Comment:
> I see you've ignored my comment...
@xtraeme, which comment?
> Why does it need to be specific to FTP and travis?
This change aims to handle specific configuration of network for travis making ftp connection there poor. https://blog.travis-ci.com/2018-07-23-the-tale-of-ftp-at-travis-ci
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-02-12 18:08 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-29 22:38 [PR PATCH] common: retry ftp download during CI voidlinux-github
2020-02-11 22:05 ` [PR PATCH] [Merged]: " voidlinux-github
2020-02-12 18:08 ` voidlinux-github
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).