Github messages for voidlinux
 help / color / mirror / Atom feed
* [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
@ 2020-02-08 15:01 voidlinux-github
  2020-02-09 15:49 ` voidlinux-github
                   ` (24 more replies)
  0 siblings, 25 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-08 15:01 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 3254 bytes --]

New issue by ahesford on void-packages repository

https://github.com/void-linux/void-packages/issues/18940

Description:
I'm opening this issue to start a discussion about switching the generic kernel config from full PREEMPT to PREEMPT_VOLUNTARY. The kernel configuration documents suggest using PREEMPT for a low-latency desktop and disabling preemption for a server. The suggested use case for PREEMPT_VOLUNTARY is just a "desktop".

PREEMPT_VOLUNTARY adds explicit preemption points in the kernel that allow syscalls to be interrupted [1](https://www.linuxtopia.org/online_books/linux_kernel/kernel_configuration/re152.html). Full PREEMPT takes this a step further by allowing interruption between the explicit points. This comes at the cost of runtime overhead and a corresponding reduction in processing throughput [2](https://www.linuxtopia.org/online_books/linux_kernel/kernel_configuration/re153.html). The desired effect of these options is to make the system appear more responsive by interrupting other tasks more frequently.

There is an objective reason for favoring the switch in the near term; the ZFS kernel module attempts to use preemptible RCU symbols when PREEMPT is configured in the kernel, and these symbols have been marked GPL-only for Linux 5.5. The ZFS modules installed by the Void package will not build against 5.5. When PREEMPT_VOLUNTARY is configured, the kernel uses TREE_RCU instead of PREEMPT_RCU, and the ZFS modules build as expected.

In the long term, I expect the ZoL crowd will fix the compatibility and this motivation will be moot. The remaining question is whether PREEMPT_VOLUNTARY is a more generic option for the stock Void kernel. Attempting to answer this question probably amounts to fighting a holy war. Nevertheless, there may be value in asking it.

I've been asked to provide some benchmarks (e.g., Phoronix Test Suite) to support the proposed switch. To meaningfully measure the tradeoff between latency and throughput as a result of the change in preemption configuration will be difficult or impossible. In any case, the benchmarks will almost certainly favor a particular use case that may or may not be relevant to any specific user. Some people have tried; see [this Raspberry Pi discussion](https://github.com/raspberrypi/linux/issues/1216) that includes some simple timing tests, or [this benchmark using cyclictest and sockperf](https://www.codeblueprint.co.uk/2019/12/23/linux-preemption-latency-throughput.html).

Like Void, Arch Linux uses PREEMPT and a 1000 Hz tick. Ubuntu uses PREEMPT_VOLUNTARY with a 250 Hz tick. Ubuntu also offers alternative kernel packages with different preemption models. The aforementioned Raspberry Pi discussion indicates that their default is PREEMPT_VOLUNTARY with a 100 Hz tick. [Debian](https://packages.debian.org/sid/kernel/linux-config-5.4), like Ubuntu, uses PREEMPT_VOLUNTARY with a 250 Hz tick.

As PREMPT_VOLUNTARY seems to be a tradeoff between throughput and latency, it seems like a prime candidate for generic kernel configs, where the intended audience will have diverse needs and may not be best served by the fully preemptible kernel. Whether the tick rate should be reduced from 1000 Hz to a lower value is a separate issue.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
@ 2020-02-09 15:49 ` voidlinux-github
  2020-02-09 16:28 ` voidlinux-github
                   ` (23 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-09 15:49 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 319 bytes --]

New comment by jrigg on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-583859920

Comment:
Not a dev, but I suspect the number of gamers and multimedia users on Void far exceeds the number of ZFS users. For the former, PREEMPT and 1000 Hz tick have definite advantages.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
  2020-02-09 15:49 ` voidlinux-github
@ 2020-02-09 16:28 ` voidlinux-github
  2020-02-09 16:29 ` voidlinux-github
                   ` (22 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-09 16:28 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-583864913

Comment:
I was never able to see much of a difference in responsiveness between `PREEMPT_VOLUNTARY` and `PREEMPT` tbh.

BTW, the non-x86 kernels are generally configured to use `PREEMPT_VOLUNTARY` (with 250 Hz tick) already. Also when I was configuring the PowerPC kernels, it turned out that using `PREEMPT` results in the kernels not booting at all on a fair amount of machines... that said, I'm not seeing any responsiveness issues on those either.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
  2020-02-09 15:49 ` voidlinux-github
  2020-02-09 16:28 ` voidlinux-github
@ 2020-02-09 16:29 ` voidlinux-github
  2020-02-09 22:53 ` voidlinux-github
                   ` (21 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-09 16:29 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 600 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-583864913

Comment:
I was never able to see much of a difference in responsiveness between `PREEMPT_VOLUNTARY` and `PREEMPT` tbh.

BTW, the non-x86 kernels are generally configured to use `PREEMPT_VOLUNTARY` (with 250 Hz tick, 100 for aarch64) already. Also when I was configuring the PowerPC kernels, it turned out that using `PREEMPT` results in the kernels not booting at all on a fair amount of machines... that said, I'm not seeing any responsiveness issues on those either.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (2 preceding siblings ...)
  2020-02-09 16:29 ` voidlinux-github
@ 2020-02-09 22:53 ` voidlinux-github
  2020-02-09 23:14 ` voidlinux-github
                   ` (20 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-09 22:53 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 1511 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-583903951

Comment:
> Not a dev, but I suspect the number of gamers and multimedia users on Void far exceeds the number of ZFS users. For the former, PREEMPT and 1000 Hz tick have definite advantages.

I agree that using full PREEMPT probably provides benefit to gamers. The multimedia benefits depend on the kind of multimedia work being done. Time-sensitive transcoding operations may actually be helped by switching to PREEMPT_VOLUNTARY (or even PREEMPT_NONE) because the reduced emphasis on low latency will not steal computational time from the transcoding operation. (This is more obvious in resource-constrained environments like the [Raspberry Pi](https://github.com/raspberrypi/linux/issues/1216#issuecomment-163200058).)

The ZFS issue brought my attention to this configuration, but it is not the primary motivator for the change. Like I said, I'm sure ZFS on Linux upstream will work around this latest salvo in the GPL/CDDL wars. The real question is where the center of mass is for Void users. Are most of them gamers that will be sensitive to the latency benefits of PREEMPT, are most of them interested in computational throughput that will be sensitive to the throughput costs of PREEMPT (and, to a lesser extent, PREEMPT_VOLUNTARY), or are the centrally clustered or uniformly distributed? The generic kernel ought to target that center of mass as best as possible.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (3 preceding siblings ...)
  2020-02-09 22:53 ` voidlinux-github
@ 2020-02-09 23:14 ` voidlinux-github
  2020-02-10  7:29 ` voidlinux-github
                   ` (19 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-09 23:14 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 1473 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-583905912

Comment:
> I was never able to see much of a difference in responsiveness between `PREEMPT_VOLUNTARY` and `PREEMPT` tbh.
> 
> BTW, the non-x86 kernels are generally configured to use `PREEMPT_VOLUNTARY` (with 250 Hz tick, 100 for aarch64) already. Also when I was configuring the PowerPC kernels, it turned out that using `PREEMPT` results in the kernels not booting at all on a fair amount of machines... that said, I'm not seeing any responsiveness issues on those either.

This seems to argue in favor of switching to PREEMPT_VOLUNTARY. Anecdotally, on a Ryzen 5 3600 six-core (twelve-thread) system, I've seen no unacceptable latency using a custom 5.5.2_2 config that uses PREEMPT_VOLUNTARY and HZ_250 while watching a (software-decoded) 3840x1920@30fps VP9 video (generally burning around 100% CPU) from YouTube and playing a 256 kbps AAC audio stream through cmus while typing this response and doing some mousing around during a 12-thread [cyclictest]( https://git.kernel.org/pub/scm/linux/kernel/git/clrkwllms/rt-tests.git) latency benchmark. The *maximum* latency measurements on each core during this testing was, in **micro**sec:

    1189, 1113, 1067, 884, 766, 166, 150, 142, 135, 128, 118, 78

while the average latencies were all 1 or 2 microsec.

Again, this is one anecdote. Everybody will have a unique story.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (4 preceding siblings ...)
  2020-02-09 23:14 ` voidlinux-github
@ 2020-02-10  7:29 ` voidlinux-github
  2020-02-10  7:44 ` voidlinux-github
                   ` (18 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10  7:29 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 254 bytes --]

New comment by travankor on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-583990229

Comment:
Clear Linux uses PREEMPT_VOLUNTARY, too.

Also should consider setting CONFIG_NO_HZ_FULL for 64-bit arches.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (5 preceding siblings ...)
  2020-02-10  7:29 ` voidlinux-github
@ 2020-02-10  7:44 ` voidlinux-github
  2020-02-10  8:12 ` voidlinux-github
                   ` (17 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10  7:44 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 208 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-583994186

Comment:
more info: https://www.kernel.org/doc/html/v5.5/timers/no_hz.html

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (6 preceding siblings ...)
  2020-02-10  7:44 ` voidlinux-github
@ 2020-02-10  8:12 ` voidlinux-github
  2020-02-10  8:22 ` voidlinux-github
                   ` (16 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10  8:12 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 199 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584003518

Comment:
@travankor I'm curious, what does clearlinux use for HZ?

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (7 preceding siblings ...)
  2020-02-10  8:12 ` voidlinux-github
@ 2020-02-10  8:22 ` voidlinux-github
  2020-02-10  8:34 ` voidlinux-github
                   ` (15 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10  8:22 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 337 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584006795

Comment:
I don't have any objection against PREEMPTY_VOLUNTARY and HZ=250 if it does not make a reasonable difference.

Why HZ=250? because this would make our kernels use shared options between archs.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (8 preceding siblings ...)
  2020-02-10  8:22 ` voidlinux-github
@ 2020-02-10  8:34 ` voidlinux-github
  2020-02-10  9:21 ` voidlinux-github
                   ` (14 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10  8:34 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 336 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584006795

Comment:
I don't have any objection against PREEMPT_VOLUNTARY and HZ=250 if it does not make a reasonable difference.

Why HZ=250? because this would make our kernels use shared options between archs.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (9 preceding siblings ...)
  2020-02-10  8:34 ` voidlinux-github
@ 2020-02-10  9:21 ` voidlinux-github
  2020-02-10 10:34 ` voidlinux-github
                   ` (13 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10  9:21 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 200 bytes --]

New comment by travankor on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584028231

Comment:
They have it set to 1000HZ but they only support x86_64

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (10 preceding siblings ...)
  2020-02-10  9:21 ` voidlinux-github
@ 2020-02-10 10:34 ` voidlinux-github
  2020-02-10 13:08 ` voidlinux-github
                   ` (12 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10 10:34 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 260 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584057943

Comment:
I've been running a kernel with PREEMPT_VOLUNTARY for some hours already and didn't notice any measurable difference.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (11 preceding siblings ...)
  2020-02-10 10:34 ` voidlinux-github
@ 2020-02-10 13:08 ` voidlinux-github
  2020-02-10 13:09 ` voidlinux-github
                   ` (11 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10 13:08 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 750 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584114139

Comment:
```
1. Never omit scheduling-clock ticks (CONFIG_HZ_PERIODIC=y or CONFIG_NO_HZ=n for older kernels). You normally will -not- want to choose this option.
2. Omit scheduling-clock ticks on idle CPUs (CONFIG_NO_HZ_IDLE=y or CONFIG_NO_HZ=y for older kernels). This is the most common approach, and should be the default.
3. Omit scheduling-clock ticks on CPUs that are either idle or that have only one runnable task (CONFIG_NO_HZ_FULL=y). Unless you are running realtime applications or certain types of HPC workloads, you will normally -not- want this option.
```

Sounds like our config is correct already.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (12 preceding siblings ...)
  2020-02-10 13:08 ` voidlinux-github
@ 2020-02-10 13:09 ` voidlinux-github
  2020-02-10 14:10 ` voidlinux-github
                   ` (10 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10 13:09 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 783 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584114139

Comment:
```
1. Never omit scheduling-clock ticks (CONFIG_HZ_PERIODIC=y or CONFIG_NO_HZ=n for older kernels). You normally will -not- want to choose this option.
2. Omit scheduling-clock ticks on idle CPUs (CONFIG_NO_HZ_IDLE=y or CONFIG_NO_HZ=y for older kernels). This is the most common approach, and should be the default.
3. Omit scheduling-clock ticks on CPUs that are either idle or that have only one runnable task (CONFIG_NO_HZ_FULL=y). Unless you are running realtime applications or certain types of HPC workloads, you will normally -not- want this option.
```

Sounds like our config is correct already, when it comes to `CONFIG_NO_HZ`.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (13 preceding siblings ...)
  2020-02-10 13:09 ` voidlinux-github
@ 2020-02-10 14:10 ` voidlinux-github
  2020-02-10 14:15 ` voidlinux-github
                   ` (9 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10 14:10 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584140017

Comment:
In another uncontrolled and non-rigorous test, I ran PowerTOP on a Dell XPS 15 9575 laptop with the stock configuration (PREEMPT and HZ=1000) and never saw fewer than 700 wakeups per second when doing nothing else on the system. With PREEMPT_VOLUNTARY and HZ=250, the same idle observation generally stays below 700 wakeups per second, and sometimes falls as low as 400/sec.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (14 preceding siblings ...)
  2020-02-10 14:10 ` voidlinux-github
@ 2020-02-10 14:15 ` voidlinux-github
  2020-02-11  9:08 ` voidlinux-github
                   ` (8 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-10 14:15 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 274 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584142560

Comment:
It is known that a soft-realtime kernel with high tick rate will have a negative effect on battery life/power usage.... it's a tradeoff

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (15 preceding siblings ...)
  2020-02-10 14:15 ` voidlinux-github
@ 2020-02-11  9:08 ` voidlinux-github
  2020-02-11  9:17 ` voidlinux-github
                   ` (7 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-11  9:08 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 556 bytes --]

New comment by jrigg on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584536345

Comment:
Just tested 5.4.18 with PREEMPT_VOLUNTARY and 256 Hz tick on my audio workstation (4-core 2.5GHz Athlon II, x86_64 glibc). I get audible dropouts even at maximum jackd latency (21.3 ms at 48kHz sample rate) on a typical multitrack mix session which runs without a problem on the current standard Void kernel.

That said, I'm aware this is a niche application and compiling a custom kernel is not a problem for me.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (16 preceding siblings ...)
  2020-02-11  9:08 ` voidlinux-github
@ 2020-02-11  9:17 ` voidlinux-github
  2020-02-11 11:10 ` voidlinux-github
                   ` (6 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-11  9:17 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 556 bytes --]

New comment by jrigg on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584536345

Comment:
Just tested 5.4.18 with PREEMPT_VOLUNTARY and 250 Hz tick on my audio workstation (4-core 2.5GHz Athlon II, x86_64 glibc). I get audible dropouts even at maximum jackd latency (21.3 ms at 48kHz sample rate) on a typical multitrack mix session which runs without a problem on the current standard Void kernel.

That said, I'm aware this is a niche application and compiling a custom kernel is not a problem for me.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (17 preceding siblings ...)
  2020-02-11  9:17 ` voidlinux-github
@ 2020-02-11 11:10 ` voidlinux-github
  2020-02-11 11:13 ` voidlinux-github
                   ` (5 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-11 11:10 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 173 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584585000

Comment:
@jrigg try with HZ=300 instead

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (18 preceding siblings ...)
  2020-02-11 11:10 ` voidlinux-github
@ 2020-02-11 11:13 ` voidlinux-github
  2020-02-11 16:59 ` voidlinux-github
                   ` (4 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-11 11:13 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 254 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584586155

Comment:
I think the x86 default (HZ=1000) is good enough, just switching to PREEMPT_VOLUNTARY should be enough for zfs.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (19 preceding siblings ...)
  2020-02-11 11:13 ` voidlinux-github
@ 2020-02-11 16:59 ` voidlinux-github
  2020-02-11 17:40 ` voidlinux-github
                   ` (3 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-11 16:59 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 364 bytes --]

New comment by jrigg on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584737628

Comment:
More tests on audio workstation:
PREEMPT_VOLUNTARY with 300 Hz: Fewer dropouts than 250 Hz, but still a lot.
PREEMPT_VOLUNTARY with 1000 Hz: Still a few dropouts, too many to be usable.
PREEMPT with 1000 Hz: No dropouts.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (20 preceding siblings ...)
  2020-02-11 16:59 ` voidlinux-github
@ 2020-02-11 17:40 ` voidlinux-github
  2020-02-11 17:41 ` voidlinux-github
                   ` (2 subsequent siblings)
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-11 17:40 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584760539

Comment:
audio workstation is a niche case that calls for `CONFIG_PREEMPT`... the tick rate won!'t actually make too much of a difference

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (21 preceding siblings ...)
  2020-02-11 17:40 ` voidlinux-github
@ 2020-02-11 17:41 ` voidlinux-github
  2020-02-12 14:14 ` voidlinux-github
  2020-02-12 14:14 ` [ISSUE] [CLOSED] " voidlinux-github
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-11 17:41 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 346 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-584760539

Comment:
audio workstation is a niche case that calls for `CONFIG_PREEMPT` (or ideally even `CONFIG_PREEMPT_RT`, but that was always out of tree patches)... the tick rate won!'t actually make too much of a difference

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (22 preceding siblings ...)
  2020-02-11 17:41 ` voidlinux-github
@ 2020-02-12 14:14 ` voidlinux-github
  2020-02-12 14:14 ` [ISSUE] [CLOSED] " voidlinux-github
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-12 14:14 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 247 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/issues/18940#issuecomment-585224290

Comment:
Closing: PREEMPT_VOLUNTARY has been changed for x86_64 and i386 in linux5.5 and ZFS builds as expected.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [ISSUE] [CLOSED] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY
  2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
                   ` (23 preceding siblings ...)
  2020-02-12 14:14 ` voidlinux-github
@ 2020-02-12 14:14 ` voidlinux-github
  24 siblings, 0 replies; 26+ messages in thread
From: voidlinux-github @ 2020-02-12 14:14 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 3411 bytes --]

Closed issue by ahesford on void-packages repository

https://github.com/void-linux/void-packages/issues/18940

Description:
I'm opening this issue to start a discussion about switching the generic kernel config from full PREEMPT to PREEMPT_VOLUNTARY. The kernel configuration documents suggest using PREEMPT for a low-latency desktop and disabling preemption for a server. The suggested use case for PREEMPT_VOLUNTARY is just a "desktop".

PREEMPT_VOLUNTARY adds explicit preemption points in the kernel that allow syscalls to be interrupted [[1]](https://www.linuxtopia.org/online_books/linux_kernel/kernel_configuration/re152.html). Full PREEMPT takes this a step further by allowing interruption between the explicit points. This comes at the cost of runtime overhead and a corresponding reduction in processing throughput [[2]](https://www.linuxtopia.org/online_books/linux_kernel/kernel_configuration/re153.html). The desired effect of these options is to make the system appear more responsive by interrupting other tasks more frequently.

There is an objective reason for favoring the switch in the near term; the ZFS kernel module attempts to use preemptible RCU symbols when PREEMPT is configured in the kernel, and these symbols have been marked GPL-only for Linux 5.5. The ZFS modules installed by the Void package will not build against 5.5. When PREEMPT_VOLUNTARY is configured, the kernel uses TREE_RCU instead of PREEMPT_RCU, and the ZFS modules build as expected.

In the long term, I expect the ZoL crowd will fix the compatibility and this motivation will be moot. The remaining question is whether PREEMPT_VOLUNTARY is a more generic option for the stock Void kernel. Attempting to answer this question probably amounts to fighting a holy war. Nevertheless, there may be value in asking it.

I've been asked to provide some benchmarks (e.g., Phoronix Test Suite) to support the proposed switch. To meaningfully measure the tradeoff between latency and throughput as a result of the change in preemption configuration will be difficult or impossible. In any case, the benchmarks will almost certainly favor a particular use case that may or may not be relevant to any specific user. Some people have tried; see [this Raspberry Pi discussion](https://github.com/raspberrypi/linux/issues/1216) that includes some simple timing tests, or [this benchmark using cyclictest and sockperf](https://www.codeblueprint.co.uk/2019/12/23/linux-preemption-latency-throughput.html).

Other distributions use a variety of defaults:
1. Like Void, [Arch Linux](https://git.archlinux.org/svntogit/packages.git/tree/trunk/config?h=packages/linux) uses PREEMPT and a 1000 Hz tick.
2. Ubuntu uses PREEMPT_VOLUNTARY with a 250 Hz tick. Ubuntu also offers alternative kernel packages with different preemption models.
3. The aforementioned Raspberry Pi discussion indicates that their default is PREEMPT_VOLUNTARY with a 100 Hz tick.
4. [Debian](https://packages.debian.org/sid/kernel/linux-config-5.4), like Ubuntu, uses PREEMPT_VOLUNTARY with a 250 Hz tick.

As PREEMPT_VOLUNTARY seems to be a tradeoff between throughput and latency, it seems like a prime candidate for generic kernel configs, where the intended audience will have diverse needs and may not be best served by the fully preemptible kernel. Whether the tick rate should be reduced from 1000 Hz to a lower value is a separate issue.

^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2020-02-12 14:14 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-08 15:01 [ISSUE] linux5.5: consider switching from PREEMPT to PREEMPT_VOLUNTARY voidlinux-github
2020-02-09 15:49 ` voidlinux-github
2020-02-09 16:28 ` voidlinux-github
2020-02-09 16:29 ` voidlinux-github
2020-02-09 22:53 ` voidlinux-github
2020-02-09 23:14 ` voidlinux-github
2020-02-10  7:29 ` voidlinux-github
2020-02-10  7:44 ` voidlinux-github
2020-02-10  8:12 ` voidlinux-github
2020-02-10  8:22 ` voidlinux-github
2020-02-10  8:34 ` voidlinux-github
2020-02-10  9:21 ` voidlinux-github
2020-02-10 10:34 ` voidlinux-github
2020-02-10 13:08 ` voidlinux-github
2020-02-10 13:09 ` voidlinux-github
2020-02-10 14:10 ` voidlinux-github
2020-02-10 14:15 ` voidlinux-github
2020-02-11  9:08 ` voidlinux-github
2020-02-11  9:17 ` voidlinux-github
2020-02-11 11:10 ` voidlinux-github
2020-02-11 11:13 ` voidlinux-github
2020-02-11 16:59 ` voidlinux-github
2020-02-11 17:40 ` voidlinux-github
2020-02-11 17:41 ` voidlinux-github
2020-02-12 14:14 ` voidlinux-github
2020-02-12 14:14 ` [ISSUE] [CLOSED] " voidlinux-github

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).