From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham autolearn_force=no version=3.4.4 Received: from second.openwall.net (second.openwall.net [193.110.157.125]) by inbox.vuxu.org (Postfix) with SMTP id 037C72F6AA for ; Wed, 18 Sep 2024 07:07:05 +0200 (CEST) Received: (qmail 12033 invoked by uid 550); 18 Sep 2024 05:07:00 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 11994 invoked from network); 18 Sep 2024 05:07:00 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to; s=qcppdkim1; bh=n+HcSgw5GU+DrFo3/fOmi+/2 ZP04rBQQdDkI28WT2/w=; b=B2qGQtT7f6nPpXHJTnHbqry9ucL+UJAXoongmP8+ rHsU0z6WB789xUVRxJtqCFOI6KTRGP9mDVho/8c2lx3PdR05Qt36DXRWHraR6V/k KhZMMnDuSusjcbTGkwenilVQ9e3912HgRBcyl1q77+g94b/nsolsgp9Ut6d0NaBs fTGeQ+oZHiiegoy0ZHIODK921eHzmU/GvJ02e9eQ/BtmEtoNm63VkUIlFKHhPyNj kq/2Df7669eiPO6igYlBUg6cB/TVvYs93IWOatAmg9axUlgFqBRr4v4Gy1ABwUyh coSzhxffTLOEotpO9rcbS0D22eVzYiRRnxGHmrePJA5O5Q== Content-Type: multipart/alternative; boundary="------------bha58WdthCqWEpt5y0uyolmR" Message-ID: Date: Wed, 18 Sep 2024 00:06:44 -0500 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: Markus Wichmann , CC: =?UTF-8?Q?Alex_Benn=C3=A9e?= References: <5179cce6-b63b-4687-8f25-f02a5fd93679@quicinc.com> Content-Language: en-US From: Brian Cain In-Reply-To: X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01c.na.qualcomm.com (10.47.97.35) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: yMEcDbAmM9_TUOhZj0kDi0Ud7c-4yUu8 X-Proofpoint-GUID: yMEcDbAmM9_TUOhZj0kDi0Ud7c-4yUu8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxlogscore=890 mlxscore=0 malwarescore=0 bulkscore=0 priorityscore=1501 clxscore=1011 adultscore=0 suspectscore=0 lowpriorityscore=0 spamscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2408220000 definitions=main-2409180026 Subject: Re: [musl] _GNU_SOURCE and _LARGEFILE_SOURCE --------------bha58WdthCqWEpt5y0uyolmR Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit On 9/17/2024 9:57 PM, Markus Wichmann wrote: > Am Tue, Sep 17, 2024 at 07:29:17PM -0500 schrieb Brian Cain: >> In the "WHATSNEW" text, there's an item from 0.9.2 that states "- make >> _GNU_SOURCE imply _LARGEFILE64_SOURCE".  Is that intended to be the case >> generally?  I ask because in 25e6fee2 (remove LFS64 programming interfaces >> (macro-only) from _GNU_SOURCE, 2022-09-27) it stated "portable software >> should be prepared for them not to exist" and "the intent is that this be a >> very short-term measure and that the macros be removed entirely in the next >> release cycle." >> >> This comes up because in the QEMU project, there's a linux multiarch test >> case that uses readdir64 and it does define _GNU_SOURCE but does not define >> _LARGEFILE64_SOURCE and as such the cross compiler complains that there's no >> declaration of readdir64 before the call site.  I suppose that the test case >> would be more portable if it defined _LARGEFILE64_SOURCE.  But I'd also be >> happy to send a patch to musl that could have _GNU_SOURCE imply >> _LARGEFILE64_SOURCE (again?) if that's desired.  But - I gather that >> defining the macros is not what we want.  Instead of macros I should add >> declarations for readdir64() and its LFS64 friends, but only when >> _LARGEFILE64_SOURCE is defined? >> >> -Brian >> > The LFS64 interfaces are not in POSIX, so you cannot assume they exist. > Rather, you can test for their existance with a configure test, and if > it fails fall back to the portable interface. Or just use the portable > interface in the first place. > > If you want to compile portable code on glibc, then define > _FILE_OFFSET_BITS to 64 and you get the exact same interface! Okay, this is a bit new to me, so let me see if I am following along: musl defines a readdir symbol and no readdir64 symbol, because readdir64 is not specified by POSIX only by Single Unix Specification?  But at one time readdir64 (et al) mappings were provided for _GNU_SOURCE and those now remain but only under _LARGEFILE_SOURCE.  At some future date those mappings might no longer be provided? And musl does not want to take patches that define a readdir64 symbol because that would be beyond the scope of POSIX.  Applications that want to be portable among POSIX targets can define _FILE_OFFSET_BITS to 64 and this will have no effect on musl type definitions nor function declarations.  But when using that define, glibc's type definitions for off_t (et al) will be such that they (1) can be used to represent enough values for large files among 32-bit and 64-bit targets, even when calling readdir-not-readdir64 and (2) are compatible / correspond with musl's type definitions?  The wording of that last part is not-quite-right, but I am capturing some of the essence there? "we'll end up with those same semantics that musl has in the absence of _GNU_SOURCE / _LARGEFILE_SOURCE" maybe? -Brian --------------bha58WdthCqWEpt5y0uyolmR Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: 8bit


On 9/17/2024 9:57 PM, Markus Wichmann wrote:
Am Tue, Sep 17, 2024 at 07:29:17PM -0500 schrieb Brian Cain:

In the "WHATSNEW" text, there's an item from 0.9.2 that states "- make
_GNU_SOURCE imply _LARGEFILE64_SOURCE".  Is that intended to be the case
generally?  I ask because in 25e6fee2 (remove LFS64 programming interfaces
(macro-only) from _GNU_SOURCE, 2022-09-27) it stated "portable software
should be prepared for them not to exist" and "the intent is that this be a
very short-term measure and that the macros be removed entirely in the next
release cycle."

This comes up because in the QEMU project, there's a linux multiarch test
case that uses readdir64 and it does define _GNU_SOURCE but does not define
_LARGEFILE64_SOURCE and as such the cross compiler complains that there's no
declaration of readdir64 before the call site.  I suppose that the test case
would be more portable if it defined _LARGEFILE64_SOURCE.  But I'd also be
happy to send a patch to musl that could have _GNU_SOURCE imply
_LARGEFILE64_SOURCE (again?) if that's desired.  But - I gather that
defining the macros is not what we want.  Instead of macros I should add
declarations for readdir64() and its LFS64 friends, but only when
_LARGEFILE64_SOURCE is defined?

-Brian


The LFS64 interfaces are not in POSIX, so you cannot assume they exist.
Rather, you can test for their existance with a configure test, and if
it fails fall back to the portable interface. Or just use the portable
interface in the first place.

If you want to compile portable code on glibc, then define
_FILE_OFFSET_BITS to 64 and you get the exact same interface!

Okay, this is a bit new to me, so let me see if I am following along:

musl defines a readdir symbol and no readdir64 symbol, because readdir64 is not specified by POSIX only by Single Unix Specification?  But at one time readdir64 (et al) mappings were provided for _GNU_SOURCE and those now remain but only under _LARGEFILE_SOURCE.  At some future date those mappings might no longer be provided? And musl does not want to take patches that define a readdir64 symbol because that would be beyond the scope of POSIX.  Applications that want to be portable among POSIX targets can define _FILE_OFFSET_BITS to 64 and this will have no effect on musl type definitions nor function declarations.  But when using that define, glibc's type definitions for off_t (et al) will be such that they (1) can be used to represent enough values for large files among 32-bit and 64-bit targets, even when calling readdir-not-readdir64 and (2) are compatible / correspond with musl's type definitions?  The wording of that last part is not-quite-right, but I am capturing some of the essence there?  "we'll end up with those same semantics that musl has in the absence of _GNU_SOURCE / _LARGEFILE_SOURCE" maybe?


-Brian

--------------bha58WdthCqWEpt5y0uyolmR--