There's a closed pull request on the void-packages repository

pass: add "x11" build option
https://github.com/void-linux/void-packages/pull/15632

Description:
When "x11" option is disabled (enabled by default), "xclip" and
transitive dependencies on X libraries are not pulled by package
installation.

"xclip" still can be installed separately, though, to regain full
functionality.