Github messages for voidlinux
 help / color / mirror / Atom feed
* [ISSUE] Incremental compilation support (xbps-src)?
@ 2022-12-11  6:27 Animeshz
  2022-12-11  6:39 ` Animeshz
                   ` (14 more replies)
  0 siblings, 15 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11  6:27 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 664 bytes --]

New issue by Animeshz on void-packages repository

https://github.com/void-linux/void-packages/issues/41008

Description:
I don't know if this is a valid issue, but compiling big projects like kernel/chromium etc. takes a while, and without cache preservation they take forever to build different versions.

If there could be a incremental compilation support in xbps-src as it already creates a stateful `masterdir`, it could really boost few aspects, such as testing for issues in different versions, and [producing patches](https://www.reddit.com/r/voidlinux/comments/z0mfbh/xbpssrc_creating_patches_for_a_new_package) for them.

Let me know your thoughts!

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
@ 2022-12-11  6:39 ` Animeshz
  2022-12-11  6:41 ` Animeshz
                   ` (13 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11  6:39 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 323 bytes --]

New comment by Animeshz on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345472042

Comment:
One way could be to link `masterdir/pkgname-pkgver/` to a local git repository via a cli-option, which could let it to use the git-tags for the version information and checkouts.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
  2022-12-11  6:39 ` Animeshz
@ 2022-12-11  6:41 ` Animeshz
  2022-12-11  6:42 ` Animeshz
                   ` (12 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11  6:41 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 520 bytes --]

New comment by Animeshz on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345472042

Comment:
One way could be to link `masterdir/pkgname-pkgver/` to a local git repository via a cli-option, which could let it to use the git-tags for the version information and checkouts.

**Edit:** Also, instead of linking we can also do a clone over the local git-repository (`git clone localpath` works) passed via cli-option and pass what (tag) we want to checkout with depth=1.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
  2022-12-11  6:39 ` Animeshz
  2022-12-11  6:41 ` Animeshz
@ 2022-12-11  6:42 ` Animeshz
  2022-12-11  6:43 ` Animeshz
                   ` (11 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11  6:42 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 323 bytes --]

New comment by Animeshz on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345472042

Comment:
One way could be to link `masterdir/pkgname-pkgver/` to a local git repository via a cli-option, which could let it to use the git-tags for the version information and checkouts.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (2 preceding siblings ...)
  2022-12-11  6:42 ` Animeshz
@ 2022-12-11  6:43 ` Animeshz
  2022-12-11  6:56 ` classabbyamp
                   ` (10 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11  6:43 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 332 bytes --]

New comment by Animeshz on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345472042

Comment:
One way could be to link `masterdir/builddir/pkgname-pkgver/` to a local git repository via a cli-option, which could let it to use the git-tags for the version information and checkouts.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (3 preceding siblings ...)
  2022-12-11  6:43 ` Animeshz
@ 2022-12-11  6:56 ` classabbyamp
  2022-12-11  7:14 ` Animeshz
                   ` (9 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: classabbyamp @ 2022-12-11  6:56 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

New comment by classabbyamp on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345474203

Comment:
xbps-src is not designed for incremental compilation, and doing so might take some large architectural changes.

I would recommend setting the following in `etc/conf`:
```
XBPS_MAKEJOBS=4
XBPS_CCACHE=yes
```
(replace the 4 with the number of cores/threads you have) 

they speed things up a lot.

you can also use the `-t` and `-C` flags to avoid rerunning steps like patch or configure over and over while packaging something

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (4 preceding siblings ...)
  2022-12-11  6:56 ` classabbyamp
@ 2022-12-11  7:14 ` Animeshz
  2022-12-11  7:18 ` Animeshz
                   ` (8 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11  7:14 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 404 bytes --]

New comment by Animeshz on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345477155

Comment:
I know we can use multithreading, but applying a few line change patch from internet and building kernel again still takes about 30 minutes or so, while that could take a file compilation and merge final output with a small linking process (a few seconds?)...

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (5 preceding siblings ...)
  2022-12-11  7:14 ` Animeshz
@ 2022-12-11  7:18 ` Animeshz
  2022-12-11 11:28 ` Anachron
                   ` (7 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11  7:18 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 315 bytes --]

New comment by Animeshz on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345477645

Comment:
Compilers are self sufficient to handle incrementals, we just gotta preserve the build files for them somehow. That doesn't looks like a big architectural change to me...

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (6 preceding siblings ...)
  2022-12-11  7:18 ` Animeshz
@ 2022-12-11 11:28 ` Anachron
  2022-12-14  7:04 ` Animeshz
                   ` (6 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Anachron @ 2022-12-11 11:28 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 271 bytes --]

New comment by Anachron on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345528104

Comment:
Did you try it with the settings @classabbyamp suggested?

Because XBPS_CCACHE=yes is responsible for caching build artifacts.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (7 preceding siblings ...)
  2022-12-11 11:28 ` Anachron
@ 2022-12-14  7:04 ` Animeshz
  2022-12-14  7:04 ` Animeshz
                   ` (5 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-14  7:04 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 552 bytes --]

New comment by Animeshz on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1350509816

Comment:
Ah yes, sorry for the late reply, the `-C` actually just keeps the package there at the build directory (I think like `xbps-src install` instead of `xbps-src pkg`), but if you add the patches or do something else it wouldn't rebuild the package. If you really want to build the package introducing a small change you gotta again clean the directory, extract, patch, and finally fully-build (no incremental).

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (8 preceding siblings ...)
  2022-12-14  7:04 ` Animeshz
@ 2022-12-14  7:04 ` Animeshz
  2022-12-14  7:22 ` paper42
                   ` (4 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-14  7:04 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 566 bytes --]

New comment by Animeshz on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1350509816

Comment:
Ah yes, sorry for the late reply, the `-C` actually just keeps the package there at the build directory (I think like `xbps-src install` instead of `xbps-src pkg`), but if you add the patches or do something else it wouldn't rebuild the package. If you really want to build the package introducing a small change you gotta again clean the directory, extract, patch, and finally it'll fully-build (that's not incremental).

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (9 preceding siblings ...)
  2022-12-14  7:04 ` Animeshz
@ 2022-12-14  7:22 ` paper42
  2022-12-14  9:03 ` Anachron
                   ` (3 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: paper42 @ 2022-12-14  7:22 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 436 bytes --]

New comment by paper42 on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1350530620

Comment:
You don't have to do that, any step can be run again with `-f`, but patch and install are not repeatable, so you will have to manually apply the patch/unpatch it (because patching once patched sources obviously won't work) or use -t and remove destdir if you want to run those more than once.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (10 preceding siblings ...)
  2022-12-14  7:22 ` paper42
@ 2022-12-14  9:03 ` Anachron
  2022-12-14  9:20 ` paper42
                   ` (2 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Anachron @ 2022-12-14  9:03 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 331 bytes --]

New comment by Anachron on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1350675700

Comment:
Of course you have to rebuild after patching. But even that should be much faster with ccache because a patch wouldnt touch every file most likely.

How do you suggest this should work? 

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (11 preceding siblings ...)
  2022-12-14  9:03 ` Anachron
@ 2022-12-14  9:20 ` paper42
  2023-03-15  1:57 ` github-actions
  2023-03-29  2:00 ` [ISSUE] [CLOSED] " github-actions
  14 siblings, 0 replies; 16+ messages in thread
From: paper42 @ 2022-12-14  9:20 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 386 bytes --]

New comment by paper42 on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1350703573

Comment:
Running `build` after patching a file should do an incremental build with most regular build systems. It will be faster than ccache, but it won't work if `$version` is updated because then $wrksrc is different, ccache will work in both cases.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (12 preceding siblings ...)
  2022-12-14  9:20 ` paper42
@ 2023-03-15  1:57 ` github-actions
  2023-03-29  2:00 ` [ISSUE] [CLOSED] " github-actions
  14 siblings, 0 replies; 16+ messages in thread
From: github-actions @ 2023-03-15  1:57 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

New comment by github-actions[bot] on void-packages repository

https://github.com/void-linux/void-packages/issues/41008#issuecomment-1469176112

Comment:
Issues become stale 90 days after last activity and are closed 14 days after that.  If this issue is still relevant bump it or assign it.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [ISSUE] [CLOSED] Incremental compilation support (xbps-src)?
  2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
                   ` (13 preceding siblings ...)
  2023-03-15  1:57 ` github-actions
@ 2023-03-29  2:00 ` github-actions
  14 siblings, 0 replies; 16+ messages in thread
From: github-actions @ 2023-03-29  2:00 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 667 bytes --]

Closed issue by Animeshz on void-packages repository

https://github.com/void-linux/void-packages/issues/41008

Description:
I don't know if this is a valid issue, but compiling big projects like kernel/chromium etc. takes a while, and without cache preservation they take forever to build different versions.

If there could be a incremental compilation support in xbps-src as it already creates a stateful `masterdir`, it could really boost few aspects, such as testing for issues in different versions, and [producing patches](https://www.reddit.com/r/voidlinux/comments/z0mfbh/xbpssrc_creating_patches_for_a_new_package) for them.

Let me know your thoughts!

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2023-03-29  2:00 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-11  6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
2022-12-11  6:39 ` Animeshz
2022-12-11  6:41 ` Animeshz
2022-12-11  6:42 ` Animeshz
2022-12-11  6:43 ` Animeshz
2022-12-11  6:56 ` classabbyamp
2022-12-11  7:14 ` Animeshz
2022-12-11  7:18 ` Animeshz
2022-12-11 11:28 ` Anachron
2022-12-14  7:04 ` Animeshz
2022-12-14  7:04 ` Animeshz
2022-12-14  7:22 ` paper42
2022-12-14  9:03 ` Anachron
2022-12-14  9:20 ` paper42
2023-03-15  1:57 ` github-actions
2023-03-29  2:00 ` [ISSUE] [CLOSED] " github-actions

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).