* [ISSUE] Incremental compilation support (xbps-src)?
@ 2022-12-11 6:27 Animeshz
2022-12-11 6:39 ` Animeshz
` (14 more replies)
0 siblings, 15 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11 6:27 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 664 bytes --]
New issue by Animeshz on void-packages repository
https://github.com/void-linux/void-packages/issues/41008
Description:
I don't know if this is a valid issue, but compiling big projects like kernel/chromium etc. takes a while, and without cache preservation they take forever to build different versions.
If there could be a incremental compilation support in xbps-src as it already creates a stateful `masterdir`, it could really boost few aspects, such as testing for issues in different versions, and [producing patches](https://www.reddit.com/r/voidlinux/comments/z0mfbh/xbpssrc_creating_patches_for_a_new_package) for them.
Let me know your thoughts!
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
@ 2022-12-11 6:39 ` Animeshz
2022-12-11 6:41 ` Animeshz
` (13 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11 6:39 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 323 bytes --]
New comment by Animeshz on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345472042
Comment:
One way could be to link `masterdir/pkgname-pkgver/` to a local git repository via a cli-option, which could let it to use the git-tags for the version information and checkouts.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
2022-12-11 6:39 ` Animeshz
@ 2022-12-11 6:41 ` Animeshz
2022-12-11 6:42 ` Animeshz
` (12 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11 6:41 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 520 bytes --]
New comment by Animeshz on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345472042
Comment:
One way could be to link `masterdir/pkgname-pkgver/` to a local git repository via a cli-option, which could let it to use the git-tags for the version information and checkouts.
**Edit:** Also, instead of linking we can also do a clone over the local git-repository (`git clone localpath` works) passed via cli-option and pass what (tag) we want to checkout with depth=1.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
2022-12-11 6:39 ` Animeshz
2022-12-11 6:41 ` Animeshz
@ 2022-12-11 6:42 ` Animeshz
2022-12-11 6:43 ` Animeshz
` (11 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11 6:42 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 323 bytes --]
New comment by Animeshz on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345472042
Comment:
One way could be to link `masterdir/pkgname-pkgver/` to a local git repository via a cli-option, which could let it to use the git-tags for the version information and checkouts.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (2 preceding siblings ...)
2022-12-11 6:42 ` Animeshz
@ 2022-12-11 6:43 ` Animeshz
2022-12-11 6:56 ` classabbyamp
` (10 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11 6:43 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 332 bytes --]
New comment by Animeshz on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345472042
Comment:
One way could be to link `masterdir/builddir/pkgname-pkgver/` to a local git repository via a cli-option, which could let it to use the git-tags for the version information and checkouts.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (3 preceding siblings ...)
2022-12-11 6:43 ` Animeshz
@ 2022-12-11 6:56 ` classabbyamp
2022-12-11 7:14 ` Animeshz
` (9 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: classabbyamp @ 2022-12-11 6:56 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 586 bytes --]
New comment by classabbyamp on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345474203
Comment:
xbps-src is not designed for incremental compilation, and doing so might take some large architectural changes.
I would recommend setting the following in `etc/conf`:
```
XBPS_MAKEJOBS=4
XBPS_CCACHE=yes
```
(replace the 4 with the number of cores/threads you have)
they speed things up a lot.
you can also use the `-t` and `-C` flags to avoid rerunning steps like patch or configure over and over while packaging something
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (4 preceding siblings ...)
2022-12-11 6:56 ` classabbyamp
@ 2022-12-11 7:14 ` Animeshz
2022-12-11 7:18 ` Animeshz
` (8 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11 7:14 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 404 bytes --]
New comment by Animeshz on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345477155
Comment:
I know we can use multithreading, but applying a few line change patch from internet and building kernel again still takes about 30 minutes or so, while that could take a file compilation and merge final output with a small linking process (a few seconds?)...
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (5 preceding siblings ...)
2022-12-11 7:14 ` Animeshz
@ 2022-12-11 7:18 ` Animeshz
2022-12-11 11:28 ` Anachron
` (7 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-11 7:18 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 315 bytes --]
New comment by Animeshz on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345477645
Comment:
Compilers are self sufficient to handle incrementals, we just gotta preserve the build files for them somehow. That doesn't looks like a big architectural change to me...
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (6 preceding siblings ...)
2022-12-11 7:18 ` Animeshz
@ 2022-12-11 11:28 ` Anachron
2022-12-14 7:04 ` Animeshz
` (6 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Anachron @ 2022-12-11 11:28 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 271 bytes --]
New comment by Anachron on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1345528104
Comment:
Did you try it with the settings @classabbyamp suggested?
Because XBPS_CCACHE=yes is responsible for caching build artifacts.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (7 preceding siblings ...)
2022-12-11 11:28 ` Anachron
@ 2022-12-14 7:04 ` Animeshz
2022-12-14 7:04 ` Animeshz
` (5 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-14 7:04 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 552 bytes --]
New comment by Animeshz on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1350509816
Comment:
Ah yes, sorry for the late reply, the `-C` actually just keeps the package there at the build directory (I think like `xbps-src install` instead of `xbps-src pkg`), but if you add the patches or do something else it wouldn't rebuild the package. If you really want to build the package introducing a small change you gotta again clean the directory, extract, patch, and finally fully-build (no incremental).
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (8 preceding siblings ...)
2022-12-14 7:04 ` Animeshz
@ 2022-12-14 7:04 ` Animeshz
2022-12-14 7:22 ` paper42
` (4 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Animeshz @ 2022-12-14 7:04 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 566 bytes --]
New comment by Animeshz on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1350509816
Comment:
Ah yes, sorry for the late reply, the `-C` actually just keeps the package there at the build directory (I think like `xbps-src install` instead of `xbps-src pkg`), but if you add the patches or do something else it wouldn't rebuild the package. If you really want to build the package introducing a small change you gotta again clean the directory, extract, patch, and finally it'll fully-build (that's not incremental).
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (9 preceding siblings ...)
2022-12-14 7:04 ` Animeshz
@ 2022-12-14 7:22 ` paper42
2022-12-14 9:03 ` Anachron
` (3 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: paper42 @ 2022-12-14 7:22 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 436 bytes --]
New comment by paper42 on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1350530620
Comment:
You don't have to do that, any step can be run again with `-f`, but patch and install are not repeatable, so you will have to manually apply the patch/unpatch it (because patching once patched sources obviously won't work) or use -t and remove destdir if you want to run those more than once.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (10 preceding siblings ...)
2022-12-14 7:22 ` paper42
@ 2022-12-14 9:03 ` Anachron
2022-12-14 9:20 ` paper42
` (2 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Anachron @ 2022-12-14 9:03 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 331 bytes --]
New comment by Anachron on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1350675700
Comment:
Of course you have to rebuild after patching. But even that should be much faster with ccache because a patch wouldnt touch every file most likely.
How do you suggest this should work?
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (11 preceding siblings ...)
2022-12-14 9:03 ` Anachron
@ 2022-12-14 9:20 ` paper42
2023-03-15 1:57 ` github-actions
2023-03-29 2:00 ` [ISSUE] [CLOSED] " github-actions
14 siblings, 0 replies; 16+ messages in thread
From: paper42 @ 2022-12-14 9:20 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 386 bytes --]
New comment by paper42 on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1350703573
Comment:
Running `build` after patching a file should do an incremental build with most regular build systems. It will be faster than ccache, but it won't work if `$version` is updated because then $wrksrc is different, ccache will work in both cases.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (12 preceding siblings ...)
2022-12-14 9:20 ` paper42
@ 2023-03-15 1:57 ` github-actions
2023-03-29 2:00 ` [ISSUE] [CLOSED] " github-actions
14 siblings, 0 replies; 16+ messages in thread
From: github-actions @ 2023-03-15 1:57 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 293 bytes --]
New comment by github-actions[bot] on void-packages repository
https://github.com/void-linux/void-packages/issues/41008#issuecomment-1469176112
Comment:
Issues become stale 90 days after last activity and are closed 14 days after that. If this issue is still relevant bump it or assign it.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [ISSUE] [CLOSED] Incremental compilation support (xbps-src)?
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
` (13 preceding siblings ...)
2023-03-15 1:57 ` github-actions
@ 2023-03-29 2:00 ` github-actions
14 siblings, 0 replies; 16+ messages in thread
From: github-actions @ 2023-03-29 2:00 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 667 bytes --]
Closed issue by Animeshz on void-packages repository
https://github.com/void-linux/void-packages/issues/41008
Description:
I don't know if this is a valid issue, but compiling big projects like kernel/chromium etc. takes a while, and without cache preservation they take forever to build different versions.
If there could be a incremental compilation support in xbps-src as it already creates a stateful `masterdir`, it could really boost few aspects, such as testing for issues in different versions, and [producing patches](https://www.reddit.com/r/voidlinux/comments/z0mfbh/xbpssrc_creating_patches_for_a_new_package) for them.
Let me know your thoughts!
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2023-03-29 2:00 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-11 6:27 [ISSUE] Incremental compilation support (xbps-src)? Animeshz
2022-12-11 6:39 ` Animeshz
2022-12-11 6:41 ` Animeshz
2022-12-11 6:42 ` Animeshz
2022-12-11 6:43 ` Animeshz
2022-12-11 6:56 ` classabbyamp
2022-12-11 7:14 ` Animeshz
2022-12-11 7:18 ` Animeshz
2022-12-11 11:28 ` Anachron
2022-12-14 7:04 ` Animeshz
2022-12-14 7:04 ` Animeshz
2022-12-14 7:22 ` paper42
2022-12-14 9:03 ` Anachron
2022-12-14 9:20 ` paper42
2023-03-15 1:57 ` github-actions
2023-03-29 2:00 ` [ISSUE] [CLOSED] " github-actions
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).