[ISSUE] nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[ISSUE] nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[xAlpharax]

[-- Attachment #1: Type: text/plain, Size: 1306 bytes --]

New issue by xAlpharax on void-packages repository

https://github.com/void-linux/void-packages/issues/50054

Description:
### Is this a new report?

Yes

### System Info

Void 6.8.7_1 x86_64

### Package(s) Affected

nvidia-*

### Does a report exist for this bug with the project's home (upstream) and/or another distro?

_No response_

### Expected behaviour

The NVIDIA driver ships with a service for dynamically changing the power cap the GPU has based on load (crucial for MAX Q GPUs). This service is called nvidia-powerd, and it should be up when the system boots.

### Actual behaviour

In driver versions newer than 535.154.05 (see https://github.com/void-linux/void-packages/commits/master/srcpkgs/nvidia for the versions that I am talking about) the nvidia-powerd service is down and refuses to start, or better said, it starts and immediately crashes/stops (what I believe happens)

### Steps to reproduce

1. Make sure to have a version of nvidia newer than 535.154.05
2. Enable the nvidia-powerd service (on GPUs other than MAX Q variants it won't change the power cap, so there isn't supposed to be an effect of it being up or down but you can still see in the following step whether it is working correctly or not)
3. Run `sudo sv status nvidia-powerd` and see that the service is down

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[classabbyamp]

[-- Attachment #1: Type: text/plain, Size: 158 bytes --]

New comment by classabbyamp on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2080460244

Comment:
@abenson 

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[xAlpharax]

[-- Attachment #1: Type: text/plain, Size: 409 bytes --]

New comment by xAlpharax on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2080611044

Comment:
I made a rollback branch on my fork to version 535.154.05 (https://github.com/xAlpharax/void-packages/tree/nvidia-powerd-rollback), as this fixes the problem for me (and possibly others). However, I'd like to see if we can make the service work in newer versions.

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[abenson]

[-- Attachment #1: Type: text/plain, Size: 225 bytes --]

New comment by abenson on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081175035

Comment:
Yeah, I noticed yesterday it had been crashing on my laptop. I'm looking into it.

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[abenson]

[-- Attachment #1: Type: text/plain, Size: 176 bytes --]

New comment by abenson on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081175390

Comment:
What laptop and GPU model is it?

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[abenson]

[-- Attachment #1: Type: text/plain, Size: 702 bytes --]

New comment by abenson on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081221790

Comment:
It looks like it might be a `dbus` issue:

```
2024-04-27T22:30:52.77406 daemon.info: Apr 27 17:30:52 /usr/bin/nvidia-powerd[5534]: nvidia-powerd version:1.0(build 1)
2024-04-27T22:30:52.77763 daemon.err: Apr 27 17:30:52 /usr/bin/nvidia-powerd[5534]: Error requesting D-Bus name (Connection ":1.529" is not allowed to own the service "nvidia.powerd.server" due to security policies in the configuration file)
2024-04-27T22:30:52.77770 daemon.err: Apr 27 17:30:52 /usr/bin/nvidia-powerd[5534]: Failed to acquire D-Bus name ((null))
```

I was missing 

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[abenson]

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]

New comment by abenson on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081221790

Comment:
It looks like it might be a `dbus` issue:

```
2024-04-27T22:30:52.77406 daemon.info: Apr 27 17:30:52 /usr/bin/nvidia-powerd[5534]: nvidia-powerd version:1.0(build 1)
2024-04-27T22:30:52.77763 daemon.err: Apr 27 17:30:52 /usr/bin/nvidia-powerd[5534]: Error requesting D-Bus name (Connection ":1.529" is not allowed to own the service "nvidia.powerd.server" due to security policies in the configuration file)
2024-04-27T22:30:52.77770 daemon.err: Apr 27 17:30:52 /usr/bin/nvidia-powerd[5534]: Failed to acquire D-Bus name ((null))
```

I was missing the dbus configuration for it. It seems to be working now, and I will be pushing the change in #49897.

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[abenson]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]

New comment by abenson on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081222508

Comment:

```
GPU 00000000:01:00.0
    GPU Power Readings
        Power Draw                        : 8.38 W
        Current Power Limit               : 100.00 W
        Requested Power Limit             : N/A
        Default Power Limit               : 80.00 W
        Min Power Limit                   : 5.00 W
        Max Power Limit                   : 125.00 W
```

At least on my laptop, it's getting 100W instead of sticking at 80W. I'm sure there's something else I'm missing to get it go the full 125W.

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[xAlpharax]

[-- Attachment #1: Type: text/plain, Size: 488 bytes --]

New comment by xAlpharax on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081343114

Comment:
> What laptop and GPU model is it?

Acer Nitro 5 2021 with RTX 3070 MAX Q variant (80 watts to 100 watts). `nvidia-smi` isn't able, at least on this model, to statically change the power limit via `nvidia-smi -pl 100` (https://github.com/NVIDIA/open-gpu-kernel-modules/issues/483) so my only viable solution is with the nvidia-powerd service.

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[xAlpharax]

[-- Attachment #1: Type: text/plain, Size: 856 bytes --]

New comment by xAlpharax on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081344657

Comment:
> It looks like it might be a `dbus` issue:
> 
> ```
> 2024-04-27T22:30:52.77406 daemon.info: Apr 27 17:30:52 /usr/bin/nvidia-powerd[5534]: nvidia-powerd version:1.0(build 1)
> 2024-04-27T22:30:52.77763 daemon.err: Apr 27 17:30:52 /usr/bin/nvidia-powerd[5534]: Error requesting D-Bus name (Connection ":1.529" is not allowed to own the service "nvidia.powerd.server" due to security policies in the configuration file)
> 2024-04-27T22:30:52.77770 daemon.err: Apr 27 17:30:52 /usr/bin/nvidia-powerd[5534]: Failed to acquire D-Bus name ((null))
> ```
> 
> I was missing the dbus configuration for it. It seems to be working now, and I will be pushing the change in #49897.

Thank you for identifying the problem.

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[xAlpharax]

[-- Attachment #1: Type: text/plain, Size: 959 bytes --]

New comment by xAlpharax on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081345564

Comment:
> 
> ```
> GPU 00000000:01:00.0
>     GPU Power Readings
>         Power Draw                        : 8.38 W
>         Current Power Limit               : 100.00 W
>         Requested Power Limit             : N/A
>         Default Power Limit               : 80.00 W
>         Min Power Limit                   : 5.00 W
>         Max Power Limit                   : 125.00 W
> ```
> 
> At least on my laptop, it's getting 100W instead of sticking at 80W. I'm sure there's something else I'm missing to get it go the full 125W.

Hmmm.. Are you sure it has enough load when checking the max power limit? For me, at least, it changes to the max when i get into `Max Performance Mode` in `nvidia-settings` or when actual compute load is happening that signals to the service it needs more power. Check with `nvtop`.

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[xAlpharax]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]

New comment by xAlpharax on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081351034

Comment:
> 
> ```
> GPU 00000000:01:00.0
>     GPU Power Readings
>         Power Draw                        : 8.38 W
>         Current Power Limit               : 100.00 W
>         Requested Power Limit             : N/A
>         Default Power Limit               : 80.00 W
>         Min Power Limit                   : 5.00 W
>         Max Power Limit                   : 125.00 W
> ```
> 
> At least on my laptop, it's getting 100W instead of sticking at 80W. I'm sure there's something else I'm missing to get it go the full 125W.

you may want to look into this, it seems also related to nvidia-smi and nvidia-powerd wattage constraints https://github.com/NVIDIA/open-gpu-kernel-modules/issues/483#issuecomment-1890415595

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[xAlpharax]

[-- Attachment #1: Type: text/plain, Size: 951 bytes --]

New comment by xAlpharax on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081351034

Comment:
> 
> ```
> GPU 00000000:01:00.0
>     GPU Power Readings
>         Power Draw                        : 8.38 W
>         Current Power Limit               : 100.00 W
>         Requested Power Limit             : N/A
>         Default Power Limit               : 80.00 W
>         Min Power Limit                   : 5.00 W
>         Max Power Limit                   : 125.00 W
> ```
> 
> At least on my laptop, it's getting 100W instead of sticking at 80W. I'm sure there's something else I'm missing to get it go the full 125W.

you may want to look into this, it seems also related to nvidia-smi and nvidia-powerd wattage constraints https://github.com/NVIDIA/open-gpu-kernel-modules/issues/483#issuecomment-1890415595 and https://download.nvidia.com/XFree86/Linux-x86_64/550.67/README/dynamicboost.html

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[abenson]

[-- Attachment #1: Type: text/plain, Size: 186 bytes --]

New comment by abenson on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081520408

Comment:
There was no load on the GPU at the time. 

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[xAlpharax]

[-- Attachment #1: Type: text/plain, Size: 276 bytes --]

New comment by xAlpharax on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081594898

Comment:
> There was no load on the GPU at the time. 

Oh, alright, so you reached your 125 Watts. Then, I guess this problem is fixed now?

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[abenson]

[-- Attachment #1: Type: text/plain, Size: 247 bytes --]

New comment by abenson on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2081662507

Comment:
I believe so, I merged the fix with the new version of the driver. Let me know if there are any issues.

Re: nvidia-powerd service is not working on nvidia driver versions newer than 535.154.05

[xAlpharax]

[-- Attachment #1: Type: text/plain, Size: 220 bytes --]

New comment by xAlpharax on void-packages repository

https://github.com/void-linux/void-packages/issues/50054#issuecomment-2082221208

Comment:
Sure, unholding the nvidia package on my machine right now and testing it.