From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,NICE_REPLY_A,RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 330 invoked from network); 27 Sep 2023 01:46:08 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 27 Sep 2023 01:46:08 -0000 Received: (qmail 11268 invoked by uid 550); 27 Sep 2023 01:46:04 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 10209 invoked from network); 27 Sep 2023 01:46:03 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=landley-net.20230601.gappssmtp.com; s=20230601; t=1695779151; x=1696383951; darn=lists.openwall.com; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=kWBsmrecXUEkN8ALWJVaKyQn14G4WdrFGh3gcqH9yTM=; b=ayIYIcFbViyCeZTd3/n/BXBqW9fdNv682nudQ2u7PGChCKlVNxvsWJzS/WuDtlvtOW +0W1WrKM5VITUsLGHJWjBFLvwwfWR1n0lkdKjJPunEA6L+UgUvge0B41D7puxTyXj/1E 50/dab4t2PzqmT28l689257FCblWlPkMC0LQDJ1eYkNQ/mAeewHKd/8D8g1jQlqYaSkB JQEV2mYLbAr0DKiesXPVMpRAN7/5xQOeRDz15DdgOqnIz1UQujXWIHMIV+GOXqNJ+U0z mwbL33SUVJcBKY4UI6VvA284RHUHF1kKiSSwYpsvp7AbvYz3HZpRi+CPzn3D68M5oM2w tWhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695779151; x=1696383951; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kWBsmrecXUEkN8ALWJVaKyQn14G4WdrFGh3gcqH9yTM=; b=gtzadcKdhxzQ9c824riNRQ7bDeOOjYm4rQaPmlXl2Jkw8/JTjHHfzwV4CoebkhRpUk MCSFdAMnJnKHJ8MupVusYl0Qa39rQDSE+d7VNq+CituULbBulUFFuNuc6W4o6OMCUhVh fkPJS1U9H6B68I7Y8NkYTj33YelT5lEDRDJ/B4+Vo6KZYZ+PtG9/ec92az5hDmR7ktQ6 zepUT1Ks453vFgw9/w0mv9QTs81JKQURFVIgP84kqsqBrw8tbTB/rU3O44x5MRrouzP+ f/eZKJKOdB2cLQkrMOFAajYo2D5ukWIoKGkHXzKEqBFTlBYuyDJzJZBtLbNajvpshPLj b0Nw== X-Gm-Message-State: AOJu0YxA4gZhoLRSOb3KaRi8H3xaMZhnF1Y0IcifqvLoVxOBkyoJO2lb 7eJQIVf+RzuPq6PuwiI0pO1vzg== X-Google-Smtp-Source: AGHT+IH56ayFKfyQVvTUlskeSqY3qwR/VPCX2deE+mrWSXAfpuFcpBAiZXPyCKVIWnBa3PiljxnmCQ== X-Received: by 2002:a05:6870:612b:b0:1be:fd4e:e36c with SMTP id s43-20020a056870612b00b001befd4ee36cmr881283oae.2.1695779151178; Tue, 26 Sep 2023 18:45:51 -0700 (PDT) Message-ID: <03a1f7cd-12d4-9d53-4997-ebb820bd23c3@landley.net> Date: Tue, 26 Sep 2023 20:49:23 -0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Content-Language: en-US To: Brian Cain , "musl@lists.openwall.com" , "Matheus Bernardino (QUIC)" Cc: Sid Manning , Rich Felker , Fangrui Song , Szabolcs Nagy References: From: Rob Landley In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: [musl] [RFC PATCH 0/5] Add support to Hexagon DSP On 9/26/23 13:13, Brian Cain wrote: >> ./build-toolchain.sh: line 250: TOOLCHAIN_INSTALL: unbound variable >> $ TOOLCHAIN_INSTALL="$PWD"/walrus ./build-toolchain.sh >> ++ date +%Y_%b_%d >> + STAMP=2023_Sep_26 >> + set -euo pipefail >> + set -x >> + set +x >> ./build-toolchain.sh: line 256: ROOT_INSTALL: unbound variable >> > > Ok - sure, let me take a minute to describe the usage of these scripts. Standby. Sigh. It's better, but it's not exactly "./configure; make; make install" either. You're not making it easy for me to wrap your thing in an automated invocation script without installing Docker. +## Usage + +Checkout the required source repos like `llvm-project`, `musl`, etc. Invoke Does not say what they are, still does not provide sample invocation script that would work from a git clone without manual steps to be determined by the user. +`get-src-tarballs.sh` with the corresponding `*_SRC_URL` links to the specific +releases to use (see `Dockerfile` for reference / last-known-good versions). Corresponding. Hmmm... $ grep SRC_URL Dockerfile ENV LLVM_SRC_URL https://github.com/llvm/llvm-project/archive/llvmorg-${VER}.tar.gz ARG QEMU_SRC_URL=https://download.qemu.org/qemu-8.1.0.tar.xz ENV MUSL_SRC_URL https://github.com/quic/musl/archive/7243e0d3a9d7e0f08d21fc194a05749e0bb26725.tar.gz ENV LINUX_SRC_URL https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.4.13.tar.xz #ENV PYTHON_SRC_URL https://www.python.org/ftp/python/3.9.5/Python-3.9.5.tar.xz ENV BUSYBOX_SRC_URL https://busybox.net/downloads/busybox-1.33.1.tar.bz2 One of those has an = and all the others have a space. The LLVM url has $VER in it where the others are explicit version numbers in the URL. I was hoping for something like "env $(sed -n 's/.* \([^ ]*_SRC_URL\)[= ]\(.*\)/\1=\2/p' Dockerfile) ./get-src-tarballs.sh" without having to special case specific variables, but no. (And of course the one with the = is not the one with the $VER.) $ grep -o '{.*SRC_URL}' get-src-tarballs.sh | sort -u {LINUX_SRC_URL} {LLVM_SRC_URL} {MUSL_SRC_URL} {QEMU_SRC_URL} The get-src-tarballs.sh script isn't just downloading and extracting package source, it's also creating package.txt files in $MANIFEST_DIR, which is the first of 2 expected arguments. The script does not check if those arguments are provided, nor does it have help/usage info beyond "read the full source to the end". Are these manifest files important, what does get-src-repos.sh... it calls wait 8 times, when "help wait" says "If ID is not given, waits for all currently active child processes, and the return status is zero" so 7 of those calls should be NOPs? Anyway, dump_checkout_info() is doing "git remote -v" to the same manifest text file names, so maybe that file is consumed by the later script. Although given that "remote -v info" output isn't obviously machine parseable, it looks like the consumer would be catting it into a readme or something at best, so I could just echo "hello" to each of them if I just wanted to make it work quickly... Sigh, any attempt to make a ./build.sh wrapper for this thing is going to be as brittle as my last build script, isn't it? I'll have to re-read all this stuff every git pull to see if it broke my wrapper... +Or instead you can check out the trunk of those projects' repos using +`git` - try invoking `get_src_repos.sh`. Which is nice to know, but "build random git snapshot du jour of everything in a previously never tried by any human combination and hope for the best" is seldom my first choice Let's see... $ bash -c "$(sed -En 's/.* (VER|[^ ]*_SRC_URL)[= ](.*)/\1=\2/p' Dockerfile | xargs) ./get-src-tarballs.sh $PWD/src $PWD/manifest" + get_src_tarballs + cd /home/landley/toybox/toolchain_for_hexagon/src ./get-src-tarballs.sh: line 9: cd: /home/landley/toybox/toolchain_for_hexagon/src: No such file or directory Really? It doesn't make the directories? Sigh... bash -c "mkdir -p src manifest && $(sed -En 's/.* (VER|[^ ]*_SRC_URL)[= ](.*)/\1=\2/p' Dockerfile | xargs) ./get-src-tarballs.sh $PWD/src $PWD/manifest" Your wget has --quiet on it. You _suppressed_ the progress indicator. > * `ARTIFACT_TAG` - the tag from the llvm-project repo with which this release > should be labeled. Does this have to be a tag in the repo, or is it just an arbitrary string? It looks like it's just used to set RESULTS_DIR_ which has a trailing underscore for reasons I do not understand... RESULTS_DIR_=${ARTIFACT_BASE}/${ARTIFACT_TAG} mkdir -p ${RESULTS_DIR_} RESULTS_DIR=$(readlink -f ${RESULTS_DIR_}) Because you didn't want to RESULTS_DIR=$(readlink -f $RESULTS_DIR) You've never tested any of these scripts on paths with spaces in them, have you? > * `TOOLCHAIN_INSTALL` - the path to install the toolchain to. > * `ROOT_INSTALL` - the path to install the rootfs to. Initially this will > only contain the target includes + libraries. > * `ARTIFACT_BASE` - the path to put the tarballs + manifests. > * optional `MAKE_TARBALLS` - if `MAKE_TARBALLS` is set to `1`, it will create > tarballs of the release and purge the intermediate build artifacts. > > Sample usage: > > export ARTIFACT_TAG=17.0.0 > export TOOLCHAIN_INSTALL=$PWD/clang+llvm-${ARTIFACT_TAG}-cross-hexagon-unknown-linux-musl > export ROOT_INSTALL=$PWD/install_rootfs > export ARTIFACT_BASE=$PWD/artifacts > > mkdir -p ${ARTIFACT_BASE} > > ./build-toolchain.sh 2>&1 | tee build_${ARTIFACT_TAG}.log Yay sample usage. Once again the script expects the directory to exist. I personally find prefix assignment useful in these sort of things (lifetime rules are kind of a big deal to me: where does data come from, how long does it last, when is it updated and by who, do the provider and consumer have an obvious relationship), but once again the script expects the directory to exist which makes prefix assignment more awkward here... Hmmm, ROOT_INSTALL is used to set ROOTFS and ROOT_INSTALL_REL, neither of which are used again by the script and not exported either, so I THINK that's just debris in build-toolchain.sh? ./build-toolchain.sh: line 256: ROOT_INSTALL: unbound variable Except, of course, it fails if the unused variable is not set. Let's feed it ROOT_INSTALL=no and... ./build-toolchain.sh: line 276: ccache: command not found $ grep ccache build-toolchain.sh ccache --show-stats ccache --show-stats $ sed -i /ccache/d build-toolchain.sh $ ARTIFACT_TAG=17.0.0 TOOLCHAIN_INSTALL=$PWD/clang+llvm-${ARTIFACT_TAG}-cross-hexagon-unknown-linux-musl ARTIFACT_BASE=$PWD/artifacts ROOT_INSTALL=no ./build-toolchain.sh 2>&1 | tee out.txt ++ date +%Y_%b_%d + STAMP=2023_Sep_26 + set -euo pipefail + set -x + set +x $ And stick some "echo" in there... It silently exited after running "which clang", which is not installed on the host. The interesting part is this one DIDN'T complain about command not found, just silently died. I wonder why? *shrug* I'm making progress, but I think I need to debootstrap a newer root filesystem version than the one I'm using before going much further, since you then call "python3.8" as a command name and this has 3.7.3 and can't apt-get anything newer without a major version update. (I'm still on devuan B and D just dropped, I've skipped the C release entirely. Busy with other things. Sigh, I should bite the bullet...) Still no qemu-system-hexagon I see. When did I last poke Taylor Simpson about that... 2021 it looks like: https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg05062.html Thanks for the help. I'll let you know if I get it working... > -Brian Rob