From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from mail-wr1-f46.google.com ([209.85.221.46]) by ewsd; Tue Sep 18 16:47:25 EDT 2018
Received: by mail-wr1-f46.google.com with SMTP id k5-v6so3495036wre.10
        for <9front@9front.org>; Tue, 18 Sep 2018 13:47:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=offblast-org.20150623.gappssmtp.com; s=20150623;
        h=mime-version:references:in-reply-to:from:date:message-id:subject:to;
        bh=9v+wS/bpoXmZRSCUFfToZTWSYQlRJJsFraLvDdC91uY=;
        b=GYOaqQtse+jYdXzShnx+AdlI6w05Y3utZKzwnVCTbpJOTEZAAuCqL2y0nO0WZkgmXe
         xQe0rvy9vGsA1k5DnwTB9L/LW+XsIeMWETG9+OnsrbcWqWEf5eVBhx6ToQI3UH7zaTzq
         Z/5ljozK+rcAMpwU4J5cxtQoKA043WoGs2vCHhv1I0kHU5t3AuEt1QuHC/RQCoqoNvtM
         Yv8nDFNG8QCLvC1ATNtH+1ZY/nJ0822EXIPY0AQqzDb4L1Q5yU6WLPKElb+EyjdyClmD
         oMRoVqRS5+lSVvBMReRf4ZkgKds+YkhJGW4SU1B/e84qfIcX8ZgmBt6smhAvwMur4Q4a
         RdBw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:mime-version:references:in-reply-to:from:date
         :message-id:subject:to;
        bh=9v+wS/bpoXmZRSCUFfToZTWSYQlRJJsFraLvDdC91uY=;
        b=Mh+rfXPKefmJ9HA/dXwY6xVZH71f1XVt04Rb26mYLqZLeNqskoXJOKoK838H9HzVHl
         DEjDU8J12egXsz0JCMzajKSoAHRBOn+yjlB+6E+5h9VW6rnB1WAYq6cEv/D4CHtjIzlw
         b04N4P4XD8pIHr1yybm4G7kGlrG4Yr1rUq2aoYu4Q+Z5LwFrcmZl4Tl9YvgpEnNraX4V
         iAsTQWB0f1w2s6g4MAwB1DNOIN3B/yNia2p5nNWABlHfVgkktG5f45nfej0zNPSPsAp7
         U5sqSQ4lDO7wvxOZhEgQkMlR62eSWeDnga2sn3BKmQnQBN40CUvwt/wEHGA1ue34apna
         PNXQ==
X-Gm-Message-State: APzg51DuuUV0fl5xkXeeCO/dFsoDqOGTV0iywYK/wL1ApBhQSqCfhOtr
	pwKAXUHROfqy5wzVhY7Uo4DCdIk0Z7hEmUkkFsvuZHFWLnVxRg==
X-Google-Smtp-Source: ANB0Vdasb5N5oBO4JQXcognGSPXIadokxlIsawldr9txPbRi1EGtTl/ZZ5ndJde5dtGmAKSKneaxt+pKt0aIR7pVUdU=
X-Received: by 2002:adf:ab0b:: with SMTP id q11-v6mr26328832wrc.239.1537303640416;
 Tue, 18 Sep 2018 13:47:20 -0700 (PDT)
MIME-Version: 1.0
References: <CAH_zEu60G8Vav9XeRegHt2vcRTrn-mToyemO+x-Zi-wtcZ5uXg@mail.gmail.com>
In-Reply-To: <CAH_zEu60G8Vav9XeRegHt2vcRTrn-mToyemO+x-Zi-wtcZ5uXg@mail.gmail.com>
From: Nick Owens <mischief@offblast.org>
Date: Tue, 18 Sep 2018 13:47:09 -0700
Message-ID: <CAH_zEu7mHpoA+Sy7-Wu57yyfiUK5kCLYcC4DQFyYDEdrc7GgqQ@mail.gmail.com>
Subject: Re: bug: webfs can't download files from sourceforge.net
To: 9front@9front.org
Content-Type: text/plain; charset="UTF-8"
List-ID: <9front.9front.org>
List-Help: <http://lists.9front.org>
X-Glyph: ➈
X-Bullshit: overflow-preventing realtime base-based interface 

after a long and grueling discussion on irc, Referer headers have been
removed from webfs. if anyone encounters a problem with a website as a
result of this, feel free to complain.

fixed in commit 3fe2d53fd3d8.
On Tue, Sep 18, 2018 at 12:31 PM Nick Owens <mischief@offblast.org> wrote:
>
> executing the following command:
>
> echo useragent curl/7.28.1 > /mnt/web/ctl
> hget https://download.sourceforge.net/tinyscheme/tinyscheme-1.41.tar.gz
> > tinyscheme-1.41.tar.gz
>
> will produce a html file.
>
> using actual curl on unix works fine.
>
> the problem appears to be the "Referer" header that webfs forced to be
> attached to the request. removing the Referer header from the webfs
> source and executing the request again produces the expected .tar.gz
> file.
>
> to me the correct solution seems to be to simply drop Referer headers
> from webfs, and add them back to mothra if need be, but the comment
> attached to the Referer-adding code makes me worried about making a
> hasty change.
>
> any thoughts?