From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/14340 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Arnd Bergmann Newsgroups: gmane.linux.lib.musl.general Subject: Re: Revisiting 64-bit time_t Date: Mon, 1 Jul 2019 23:12:20 +0200 Message-ID: References: <20190628150659.GD1506@brightrain.aerifal.cx> <20190701153108.GR1506@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="247291"; mail-complaints-to="usenet@blaine.gmane.org" To: musl@lists.openwall.com Original-X-From: musl-return-14356-gllmg-musl=m.gmane.org@lists.openwall.com Mon Jul 01 23:12:54 2019 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.89) (envelope-from ) id 1hi3bY-00129p-2h for gllmg-musl@m.gmane.org; Mon, 01 Jul 2019 23:12:52 +0200 Original-Received: (qmail 21978 invoked by uid 550); 1 Jul 2019 21:12:49 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 21957 invoked from network); 1 Jul 2019 21:12:49 -0000 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=LL/pGQAwK3axH1lDSMvgoXQ8DW5l0oXkHJk4Q7IZWK8=; b=csGzBTUEhpeoaLDwPGULJHOmbLPE3aqEd4qtxqUid/hCgg8+lGWWZB/K4IFOgIdQEb 0jN8r9bLJfnEiXKN4ON3CteT4HQc8hbQ6Q4czH+4fPU9hFoemqL+93tk3tc2Lg/y1vEf 89C9DpD+zs+7PYAcfODI5l3nHTH3A6KrNcBkIIHj1zIv0TolXq1PPcvZIVy7GfBlhB91 z/8U3OS4SI8i2O2n06mRlITq8VyN2G3BMxS5BSjxeJQlbQeJgw/M91FituKZi87fSDxh kx707cOASz+p3trrN8xuH39u5rUE9dezBiJz0RG0PlyoxgVEaPfASbGwmCVCHBhspuM1 od5g== X-Gm-Message-State: APjAAAViyV3NW2t+Njpu4azjJ/L1+eT97n3uDOhupKl1zAYs9pEiuLRk u/7HMIhHaAgcMHyaqBfeg65dYmF3zwI9vjQAtoJdVLL+p+o= X-Google-Smtp-Source: APXvYqxa5DnFJq6kwwbXqL+33GBAK8rk1NHglfZBDg4eahqtsvXOMH+1lFEzfwHaMNCa2Sug1NNNOCGA5wNf2b0w7FM= X-Received: by 2002:a37:76c5:: with SMTP id r188mr22170067qkc.394.1562015557021; Mon, 01 Jul 2019 14:12:37 -0700 (PDT) In-Reply-To: <20190701153108.GR1506@brightrain.aerifal.cx> Xref: news.gmane.org gmane.linux.lib.musl.general:14340 Archived-At: On Mon, Jul 1, 2019 at 5:31 PM Rich Felker wrote: > > On Mon, Jul 01, 2019 at 04:42:29PM +0200, Arnd Bergmann wrote: > > On Fri, Jun 28, 2019 at 5:07 PM Rich Felker wrote: > > > > > > The idea has been that users (like embedded) who don't care > > > much/at-all about an ecosystem of ABI-compatible binaries, but build > > > everything from source with buildroot or yocto or whatever, would > > > switch right away so that their devices don't become Y2038 time bombs, > > > and desktop/server distros that receive constant updates could make > > > the transition at their leisure. > > > > Distros would probably need a varying amount of time to transition, > > right? Would you plan to support both time32 and time64 for a > > transition period, or would a distro that is not yet confident in rebuilding > > everything with time64 be stuck on the last time32 musl release > > before they do? > > > > I suppose the header files could be changed in a musl-1.2 release > > if the times line up, while musl-1.1.x can still get bugfixes? > > ".2" ABI doesn't mean "musl 1.2". It means ld-musl-*.so.2". If taking > this path, the plan was always to support both permanently (modulo > dropping .1 after 2038 ;) -- they'd just be different ABI targets. Yes, I understand that was a different idea. What I was asking about here is how long there would be support for building with the time32 interfaces against a maintained libc after it has become possible to build with time64 when using the same soname. > > > Aside from community feedback, what's needed to make this possible, if > > > it's going to happen, is some good analysis of the scope of breakage. > > > Such analysis would also benefit glibc -- it would help determine how > > > safe their _TIME_BITS=64 option will be and whether it can be turned > > > on safely by default in the presence of old libraries built without > > > it. I've already discussed this casually with a few people and it > > > looks like the right starting point would be getting a Debian system > > > (Debian because their repo is utterly huge) with ALL library packages > > > installed and grepping /usr/include for all headers that involve > > > time_t or any of the derived types. Then, manual analysis would need > > > to be done to determine whether the usage actually has an impact. > > > > Yes, this is also one of the things we eventually plan to do in Linaro, > > but have not actually started. > > Would it be possible to prioritize starting this? It would be a big > help to deciding what direction we should take in musl and make this > move forward a lot quicker, I think. I was thinking we'd have to do it > ourselves or find someone else to convince to do it, but if Linaro > already plans to do this anyway, we could perhaps accelerate things > with no overall increase in effort to be spent. I've had a first look here now, with a scripted search in /usr/include for all packages in debian testing/main: apt-file search /usr/include | cut -f 1 -d: | uniq | while read i ; do mkdir -p ${i} cd ${i} if [ ! -e "${i}_.*.deb" ] ; then apt-get download ${i} fi FILE=${i}_*.deb dpkg-deb --fsys-tarfile ${FILE} | tar xf - ./usr/include cd .. ctags -f - -R ${i}/ | grep '\' > ${i}.tags grep -r '\' ${i}/ > ${i}.files rm -rf ./${i} done There are 4288 packages that provide a file in /usr/include, and out of those, 973 match the regular expression above, see the list at https://pastebin.com/Yu22pLqQ This took a few hours to run and could be done faster by running bits in parallel. I can send you the full output, but it's at 300kb compressed, it's a bit large for the mailing list. The regex also wasn't great, so I'm sure there are lots of false positives and negatives, but its' a start. > > > If there are a significant number of affected libraries and we want to > > > go forward with something like this anyway, there should probably be > > > an optional patch distros can use to make ldso refuse to load certain > > > tagged .so files into a process where any of the 64-bit time symbols > > > have been referenced. This would ensure transitioning users get an > > > error message rather than silent misexecution. > > > > For those distros that build everything from source and generally > > don't update packages independently, another idea would > > be to have a way to leave out all the time32 symbols. This would > > immediately guarantee that they are not mixed. > > Well if you just have the new headers, there's not really any way they > could get mixed except by failing to include the headers and declaring > the symbols yourself -- but then, for almost all cases, you'd be > missing the types needed to declare them, so it'd have to be something > like "long time(long *);" in the source, which is just horribly > broken. So I'm not sure there are any real-world cases where > removing/poisoning the old symbols would help. I was thinking of the case where in theory everything should be built from source, but in reality some binaries made it into system because of sloppy procedures or lack of source code. Arnd