Re: Revisiting 64-bit time_t

[Arnd Bergmann <arnd@arndb.de>]

On Mon, Jul 1, 2019 at 5:31 PM Rich Felker <dalias@libc.org> wrote:
>
> On Mon, Jul 01, 2019 at 04:42:29PM +0200, Arnd Bergmann wrote:
> > On Fri, Jun 28, 2019 at 5:07 PM Rich Felker <dalias@libc.org> wrote:
> > >
> > > The idea has been that users (like embedded) who don't care
> > > much/at-all about an ecosystem of ABI-compatible binaries, but build
> > > everything from source with buildroot or yocto or whatever, would
> > > switch right away so that their devices don't become Y2038 time bombs,
> > > and desktop/server distros that receive constant updates could make
> > > the transition at their leisure.
> >
> > Distros would probably need a varying amount of time to transition,
> > right? Would you plan to support both time32 and time64 for a
> > transition period, or would a distro that is not yet confident in rebuilding
> > everything with time64 be stuck on the last time32 musl release
> > before they do?
> >
> > I suppose the header files could be changed in a musl-1.2 release
> > if the times line up, while musl-1.1.x can still get bugfixes?
>
> ".2" ABI doesn't mean "musl 1.2". It means ld-musl-*.so.2". If taking
> this path, the plan was always to support both permanently (modulo
> dropping .1 after 2038 ;) -- they'd just be different ABI targets.

Yes, I understand that was a different idea. What I was asking about
here is how long there would be support for building with the time32
interfaces against a maintained libc after it has become possible to
build with time64 when using the same soname.


> > > Aside from community feedback, what's needed to make this possible, if
> > > it's going to happen, is some good analysis of the scope of breakage.
> > > Such analysis would also benefit glibc -- it would help determine how
> > > safe their _TIME_BITS=64 option will be and whether it can be turned
> > > on safely by default in the presence of old libraries built without
> > > it. I've already discussed this casually with a few people and it
> > > looks like the right starting point would be getting a Debian system
> > > (Debian because their repo is utterly huge) with ALL library packages
> > > installed and grepping /usr/include for all headers that involve
> > > time_t or any of the derived types. Then, manual analysis would need
> > > to be done to determine whether the usage actually has an impact.
> >
> > Yes, this is also one of the things we eventually plan to do in Linaro,
> > but have not actually started.
>
> Would it be possible to prioritize starting this? It would be a big
> help to deciding what direction we should take in musl and make this
> move forward a lot quicker, I think. I was thinking we'd have to do it
> ourselves or find someone else to convince to do it, but if Linaro
> already plans to do this anyway, we could perhaps accelerate things
> with no overall increase in effort to be spent.

I've had a first look here now, with a scripted search in /usr/include
for all packages in debian testing/main:

apt-file search /usr/include | cut -f 1 -d: | uniq |
while read i ; do
    mkdir -p ${i}
    cd ${i}
    if [ ! -e "${i}_.*.deb" ] ; then
        apt-get download ${i}
    fi
    FILE=${i}_*.deb
    dpkg-deb --fsys-tarfile ${FILE} | tar xf - ./usr/include
    cd ..
    ctags -f - -R ${i}/ | grep
'\<time_t\|timespec\|timeval\|struct.stat\|struct.rusage\>' >
${i}.tags
    grep -r '\<time_t\|timespec\|timeval\|struct.stat\|struct.rusage\>'
${i}/ > ${i}.files
    rm -rf ./${i}
done

There are 4288 packages that provide a file in /usr/include, and out of those,
973 match the regular expression above, see the list at

https://pastebin.com/Yu22pLqQ

This took a few hours to run and could be done faster by running bits
in parallel.
I can send you the full output, but it's at 300kb compressed, it's a bit large
for the mailing list. The regex also wasn't great, so I'm sure there
are lots of false positives and negatives, but its' a start.

> > > If there are a significant number of affected libraries and we want to
> > > go forward with something like this anyway, there should probably be
> > > an optional patch distros can use to make ldso refuse to load certain
> > > tagged .so files into a process where any of the 64-bit time symbols
> > > have been referenced. This would ensure transitioning users get an
> > > error message rather than silent misexecution.
> >
> > For those distros that build everything from source and generally
> > don't update packages independently, another idea would
> > be to have a way to leave out all the time32 symbols. This would
> > immediately guarantee that they are not mixed.
>
> Well if you just have the new headers, there's not really any way they
> could get mixed except by failing to include the headers and declaring
> the symbols yourself -- but then, for almost all cases, you'd be
> missing the types needed to declare them, so it'd have to be something
> like "long time(long *);" in the source, which is just horribly
> broken. So I'm not sure there are any real-world cases where
> removing/poisoning the old symbols would help.

I was thinking of the case where in theory everything should be built
from source, but in reality some binaries made it into system because
of sloppy procedures or lack of source code.

         Arnd