[ISSUE] User template repositories?

[ISSUE] User template repositories?

[freddylist]

[-- Attachment #1: Type: text/plain, Size: 2356 bytes --]

New issue by freddylist on void-packages repository

https://github.com/void-linux/void-packages/issues/39900

Description:
Something similar has been discussed before in #19830, but I thought I would bring it up again since it's been 2 years and I think it could be nice to have something like Gentoo's overlays or Arch's AUR (a central repository that is more "liberal" in what is accepted) or something for packages that cannot be accepted to the main repositories, such as
 - browsers (brave, librewolf, icecat, etc.)
 - bleeding-edge software (git master branch, unreleased software)
 - binary distributions of software (large software like browsers, software with large outdated build dependencies e.g. [Zig](https://github.com/void-linux/void-packages/pull/37164#issuecomment-1128926851))
 - packages that the original packager isn't intending to maintain thoroughly
 - other minor things (themes and such)
 - and so on...

Right now, the ad-hoc way of building packages distributed outside of `void-packages` is to copy the templates into the `srcpkgs/` directory and building with `xbps-src` from there. This is fine for managing a couple of externally distributed packages, but can quickly become a hassle. One could maybe write a script to help manage packages like this, but this is very error-prone.

`xbps-src` expects all symlinks in the `srcpkgs/` directory to be subpackages, so we can't just symlink packages into `srcpkgs/` or use a symlink farm manager like GNU Stow, which might otherwise be a slightly better ad-hoc solution.

I also randomly tried to symlink to `xbps-src` and its dependencies from an external template repository, and it seems that it won't work without some patching. Copying `xbps-src` and its dependencies also doesn't seem to work without also copying a couple of packages from `void-packages/srcpkgs/`, such as `xbps-triggers` and `base-files`, which isn't ideal.

With all that said, I think it would be good to have a Void-endorsed way to create user template repositories/packages documented somewhere. It almost goes without saying that there are inherent security risks with using these user template repositories, and that should of course be documented as well. So, does there currently exist better way to create user repositories/packages, or must `xbps-src` be updated to support this?

Re: User template repositories?

[paper42]

[-- Attachment #1: Type: text/plain, Size: 634 bytes --]

New comment by paper42 on void-packages repository

https://github.com/void-linux/void-packages/issues/39900#issuecomment-1275293853

Comment:
Nothing changed from 2 years ago. void-packages is by design a monorepo which comes with its advantages and limitations, hacking around them is not supported. `xbps-src` uses the templates of recursive dependencies to create the masterdir, so you need at least those. Changing this would mean entirely changing the design of xbps-src.
If you want to have your own packages, just create a branch and push your changes there. This is not actionable and has been discussed before, so closing.

Re: [ISSUE] [CLOSED] User template repositories?

[paper42]

[-- Attachment #1: Type: text/plain, Size: 2359 bytes --]

Closed issue by freddylist on void-packages repository

https://github.com/void-linux/void-packages/issues/39900

Description:
Something similar has been discussed before in #19830, but I thought I would bring it up again since it's been 2 years and I think it could be nice to have something like Gentoo's overlays or Arch's AUR (a central repository that is more "liberal" in what is accepted) or something for packages that cannot be accepted to the main repositories, such as
 - browsers (brave, librewolf, icecat, etc.)
 - bleeding-edge software (git master branch, unreleased software)
 - binary distributions of software (large software like browsers, software with large outdated build dependencies e.g. [Zig](https://github.com/void-linux/void-packages/pull/37164#issuecomment-1128926851))
 - packages that the original packager isn't intending to maintain thoroughly
 - other minor things (themes and such)
 - and so on...

Right now, the ad-hoc way of building packages distributed outside of `void-packages` is to copy the templates into the `srcpkgs/` directory and building with `xbps-src` from there. This is fine for managing a couple of externally distributed packages, but can quickly become a hassle. One could maybe write a script to help manage packages like this, but this is very error-prone.

`xbps-src` expects all symlinks in the `srcpkgs/` directory to be subpackages, so we can't just symlink packages into `srcpkgs/` or use a symlink farm manager like GNU Stow, which might otherwise be a slightly better ad-hoc solution.

I also randomly tried to symlink to `xbps-src` and its dependencies from an external template repository, and it seems that it won't work without some patching. Copying `xbps-src` and its dependencies also doesn't seem to work without also copying a couple of packages from `void-packages/srcpkgs/`, such as `xbps-triggers` and `base-files`, which isn't ideal.

With all that said, I think it would be good to have a Void-endorsed way to create user template repositories/packages documented somewhere. It almost goes without saying that there are inherent security risks with using these user template repositories, and that should of course be documented as well. So, does there currently exist better way to create user repositories/packages, or must `xbps-src` be updated to support this?

Re: User template repositories?

[paper42]

[-- Attachment #1: Type: text/plain, Size: 638 bytes --]

New comment by paper42 on void-packages repository

https://github.com/void-linux/void-packages/issues/39900#issuecomment-1275293853

Comment:
Nothing changed from 2 years ago. `void-packages` is by design a monorepo which comes with its advantages and limitations, hacking around them is not supported. `xbps-src` uses the templates of recursive dependencies to create the masterdir, so you need at least those. Changing this would mean entirely changing the design of `xbps-src`.
If you want to have your own packages, just create a branch and push your changes there. This is not actionable and has been discussed before, so closing.

Re: User template repositories?

[freddylist]

[-- Attachment #1: Type: text/plain, Size: 2068 bytes --]

New comment by freddylist on void-packages repository

https://github.com/void-linux/void-packages/issues/39900#issuecomment-1277527085

Comment:
> This is not actionable and has been discussed before

I don't know if #19830 should count as a discussion, considering the author was speaking through a translator and was clearly upset by some brush with the Void Linux community...

> `void-packages` is by design a monorepo which comes with its advantages and limitations, hacking around them is not supported.

Sure, but it's still useful to have some semblance of user repositories, especially when you think of all the juicy new contributors that could come from these user repositories. I thought we might discuss some hacks for user repositories, even if they aren't "officially" supported.

For instance, after much contemplation, I came up with [this](https://github.com/freddylist/xbps-src-overlay-demo), which (ab)uses `overlayfs` to trick `xbps-src` into finding templates from multiple directories.

> Changing this would mean entirely changing the design of `xbps-src`.

It was mentioned in the other issue that supporting multiple `srcpkgs` dirs was "[the wrong way of handling the issue](https://github.com/void-linux/void-packages/issues/19830#issuecomment-595679246)" and that it would be "[hard to implement a way how to supply multiple directories](https://github.com/void-linux/void-packages/issues/19830#issuecomment-595695312)", but not *why*. Maybe there is a lot about package management I don't understand, but I don't see how it could be a bad idea to support multiple `srcpkgs` dirs. Just skimming through the `xbps-src` code I think it might be a couple dozen extra lines of Bash to implement multiple `srcpkgs` dirs in a non-breaking way.

Unless I am convinced otherwise or have grossly underestimated how hard it would be to implement, I can maybe have a draft version of `xbps-src` that supports multiple `srcpkgs` dirs, probably configured the same way my hack above is (symlinks in some `repos/` directory or something).

Re: User template repositories?

[paper42]

[-- Attachment #1: Type: text/plain, Size: 543 bytes --]

New comment by paper42 on void-packages repository

https://github.com/void-linux/void-packages/issues/39900#issuecomment-1277564172

Comment:
The supported way of having 3rd party packages is a branch with your changes on your fork, what are you trying to solve?
Adding srcpkgs2 is only going to be confusing and won't fix anything. void-packages is a monorepo, trying to add or modify packages on top of it can easily be done by just doing that and pushing your work to a branch. That way everyone will know it built at that point in time.

Re: User template repositories?

[paper42]

[-- Attachment #1: Type: text/plain, Size: 259 bytes --]

New comment by paper42 on void-packages repository

https://github.com/void-linux/void-packages/issues/39900#issuecomment-1277565681

Comment:
Or you can use proper ways of installing software that's not supported on your distribution like docker or flatpak.

Re: User template repositories?

[freddylist]

[-- Attachment #1: Type: text/plain, Size: 558 bytes --]

New comment by freddylist on void-packages repository

https://github.com/void-linux/void-packages/issues/39900#issuecomment-1491518407

Comment:
I still feel like I was being misunderstood here, but however my suggestion was assimilated definitely sounds like a terrible idea.

In any case, I've made [xbps-src-framework](https://github.com/freddylist/xbps-src-framework) which could help with bulk building custom packages from a variety of sources. There is a list of sources for custom packages in the README there as well.

Thank you for your time!