[ISSUE] shlibs checking fail on neovim update

[ISSUE] shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 1107 bytes --]

New issue by Chocimier on void-packages repository

https://github.com/void-linux/void-packages/issues/14856

Description:
Something is wrong with shlibs checking on #14511, but i am not exactly sure what.

Story:

libvterm-0.0.20151229 provides libvterm.so.0, a symlink to libvterm.so.0.0.0
libvterm-0.1.1 provides libvterm.so.0, a symlink to libvterm.so.0.1.1
both provide libvterm.so, a symlink to real libvterm.so.0.?.?
libvterm.so.0.1.1 provides symbols not provided by libvterm.so.0.0.0

neovim-0.4.2 is linked against libvterm.so.0 provided by newer libvterm

xbps-src reads from shlibs that libvterm.so.0 is provided by libvterm>=0.0.20151229
xbps-src generates neovim package with run dep libvterm>=0.0.20151229

xbps don't update libvterm on update of neovim

neovim fails to start because of missing symbols in libvterm.so.0

Question: what is wrong:
- libvterm providing incompatible versions of libvterm.so.0, even if not telling to use them?
- toolchain linking against libvterm.so.0 symlink, not explicitly instructed to do so?
- xbps-src specifying symlink in shlibs?

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 170 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569355976

Comment:
is this still reproducable?

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 451 bytes --]

New comment by Chocimier on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569411152

Comment:
Reproducable: 
1. Build and install neovim and libvterm from commit 8b4f04b75c07016eaa
2. Build them from master, but without depends= set in neovim
3. Update with `xbps-install -u neovim`
4. `nvim --help` fails with `nvim: symbol lookup error: nvim: undefined symbol: vterm_state_convert_color_to_rgb`

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 750 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569419722

Comment:
This is a more general issue, adding new symbols doesn't break the ABI so they don't really have to bump the version.
The same happens with glibc and libressl, where we just bump the the minimum version in `common/shlibs` so packages build against a new `glibc` or `libressl` version that might use a new or in glibcs case a new versioned symbol will not break and depend on the new version.

Not sure what we can do about it, we could just depend on `>=` of the exact package version we linked against instead of `common/shlibs` but then selective/partial updates and installs become a lot less doable. 

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 1032 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569419722

Comment:
This is a more general issue, adding new symbols doesn't break the ABI so they don't really have to bump the version.
The same happens with glibc and libressl, where we just bump the the minimum version in `common/shlibs` so packages build against a new `glibc` or `libressl` version that might use a new or in glibcs case a new versioned symbol will not break and depend on the new version.

Not sure what we can do about it, we could just depend on `>=` of the exact package version we linked against instead of `common/shlibs` but then selective/partial updates and installs become a lot less doable.

Edit:
The best solution would be a way to not just record SONAMEs but also record earch symbol and then check the compiled binaries needed symbols against the record and then take the highest version of the required symbols as minimum version instead of `common/shlibs` version.

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 364 bytes --]

New comment by Chocimier on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569686009

Comment:
> adding new symbols doesn't break the ABI

This answers my question, thanks for explanation. Setting `depends` makes sense.

Symbols record looks unfeasible, but i'll leave thread open if someone want address that.

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 1031 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569419722

Comment:
This is a more general issue, adding new symbols doesn't break the ABI so they don't really have to bump the version.
The same happens with glibc and libressl, where we just bump the the minimum version in `common/shlibs` so packages build against a new `glibc` or `libressl` version that might use a new or in glibcs case a new versioned symbol will not break and depend on the new version.

Not sure what we can do about it, we could just depend on `>=` of the exact package version we linked against instead of `common/shlibs` but then selective/partial updates and installs become a lot less doable.

Edit:
The best solution would be a way to not just record SONAMEs but also record each symbol and then check the compiled binaries needed symbols against the record and then take the highest version of the required symbols as minimum version instead of `common/shlibs` version.

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 337 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569690218

Comment:
In this case (I've been aware of this since day 0), unless we use the Debian way (symbols files) there will always this issue. How to workaround it? Set depends manually to the desired pattern. 

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 264 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569690662

Comment:
To be clear, our common/shlibs file only tracks soname bumps, if they don't bump it we don't know if they changed the ABI

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 210 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569690910

Comment:
Hmm got a new idea, will write this later when I'm with my computer

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569712931

Comment:
My idea is a `pre-pkg` hook in `xbps-src` which generates a list of symbols provided by libraries mapped to the current version of the package, this file would be stored as a new plist inside of packages.
The hook would then use `xbps-query` to get a list of the current list and replace symbols that stayed the same with the version stored in the previous packages list.

This list would evolve over time with the minimum version for each binary, for new bootstraps from source or new packages, it would start with the current version.

I think its a bit hacky, but was the only solution I could come up with without having to add hundreds of file maps like `common/shlibs` for each library.

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569712931

Comment:
My idea is a `pre-pkg` hook in `xbps-src` which generates a list of symbols provided by libraries mapped to the current version of the package, this file would be stored as a new plist inside of packages.
The hook would then use `xbps-query` to get a list of the current list and replace symbols that stayed the same with the version stored in the previous packages list.

This list would evolve over time with the minimum version for each symbol, for new bootstraps from source or new packages, it would start with the current version.

I think its a bit hacky, but was the only solution I could come up with without having to add hundreds of file maps like `common/shlibs` for each library.

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 185 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569713307

Comment:
Haha no, it's not hacky. It's clever rofl 

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 214 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569713584

Comment:
Show me your patch with details and I'll comment. But sounds good to me

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 227 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569713809

Comment:
Surprise. plist2json can convert our plists into valid json... Haha. My Xmas present

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 154 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569713863

Comment:
Coming soon

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 170 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569713940

Comment:
I took the compact json way

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 178 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569714182

Comment:
More details coming soon... ENOBEER

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 1651 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569741159

Comment:
Here is the basic idea: https://github.com/Duncaen/void-packages/commit/4d6e68d0101f2bdda367f47c79d5ba5443520e43, its just the shlib part using the map to get the required version for binaries is not done yet.

`test.sh` produces the following:
```
void-packages$ ./test.sh
ERROR: [reposync] failed to fetch file `https://alpha.de.repo.voidlinux.org/current/musl/x86_64-repodata': Not Found
ERROR: [reposync] failed to fetch file `https://alpha.de.repo.voidlinux.org/current/aarch64/x86_64-repodata': Not Found
symbols for 0.1:
/usr/lib/libtestlib.so.0 foo testlib-0.1
ERROR: [reposync] failed to fetch file `https://alpha.de.repo.voidlinux.org/current/musl/x86_64-repodata': Not Found
ERROR: [reposync] failed to fetch file `https://alpha.de.repo.voidlinux.org/current/aarch64/x86_64-repodata': Not Found
deps for testbin-0.1:
testlib>=0.1_1
glibc>=2.29_1
ERROR: [reposync] failed to fetch file `https://alpha.de.repo.voidlinux.org/current/musl/x86_64-repodata': Not Found
ERROR: [reposync] failed to fetch file `https://alpha.de.repo.voidlinux.org/current/aarch64/x86_64-repodata': Not Found
symbols for 0.2:
/usr/lib/libtestlib.so.0 foo testlib-0.1
/usr/lib/libtestlib.so.0 bar testlib-0.2
ERROR: [reposync] failed to fetch file `https://alpha.de.repo.voidlinux.org/current/musl/x86_64-repodata': Not Found
ERROR: [reposync] failed to fetch file `https://alpha.de.repo.voidlinux.org/current/aarch64/x86_64-repodata': Not Found
deps for testbin-0.2:
testlib>=0.1_1
glibc>=2.29_1
```

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 367 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569741634

Comment:
Its using a `/usr/lib/symbol/$pkgname` file as alternative to adding it to `xbps-{create/query}` maybe its sufficient, we could theoretically provide such a map with the `-devel` package so its only there if actually needed.

Re: shlibs checking fail on neovim update

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 154 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-569746819

Comment:
Haha great 

Re: shlibs checking fail on neovim update

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 222 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-773759714

Comment:
This is still an interesting idea, and would be the ideal form of ABI tracking.

Re: shlibs checking fail on neovim update

[github-actions]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

New comment by github-actions[bot] on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-1099786787

Comment:
Issues become stale 90 days after last activity and are closed 14 days after that.  If this issue is still relevant bump it or assign it.

Re: shlibs checking fail on neovim update

[github-actions]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

New comment by github-actions[bot] on void-packages repository

https://github.com/void-linux/void-packages/issues/14856#issuecomment-1188518512

Comment:
Issues become stale 90 days after last activity and are closed 14 days after that.  If this issue is still relevant bump it or assign it.

Re: [ISSUE] [CLOSED] shlibs checking fail on neovim update

[github-actions]

[-- Attachment #1: Type: text/plain, Size: 1110 bytes --]

Closed issue by Chocimier on void-packages repository

https://github.com/void-linux/void-packages/issues/14856

Description:
Something is wrong with shlibs checking on #14511, but i am not exactly sure what.

Story:

libvterm-0.0.20151229 provides libvterm.so.0, a symlink to libvterm.so.0.0.0
libvterm-0.1.1 provides libvterm.so.0, a symlink to libvterm.so.0.1.1
both provide libvterm.so, a symlink to real libvterm.so.0.?.?
libvterm.so.0.1.1 provides symbols not provided by libvterm.so.0.0.0

neovim-0.4.2 is linked against libvterm.so.0 provided by newer libvterm

xbps-src reads from shlibs that libvterm.so.0 is provided by libvterm>=0.0.20151229
xbps-src generates neovim package with run dep libvterm>=0.0.20151229

xbps don't update libvterm on update of neovim

neovim fails to start because of missing symbols in libvterm.so.0

Question: what is wrong:
- libvterm providing incompatible versions of libvterm.so.0, even if not telling to use them?
- toolchain linking against libvterm.so.0 symlink, not explicitly instructed to do so?
- xbps-src specifying symlink in shlibs?