Re: x11vnc password is always in clear text, and clients are unable to connect with password

[dexgs <dexgs@users.noreply.github.com>]

[-- Attachment #1: Type: text/plain, Size: 2925 bytes --]

New comment by dexgs on void-packages repository

https://github.com/void-linux/void-packages/issues/48007#issuecomment-1953549020

Comment:
Based on the linked commit on the gentoo tracker, It seems like the issue is that x11vnc does not work properly with openssl 3.0.

I tried disabling it by modifying the following templates as follows:
```sh
# Template file for 'x11vnc'
pkgname=x11vnc
version=0.9.16
revision=7
build_style=gnu-configure
configure_args="--without-crypto --without-ssl"
hostmakedepends="automake libtool pkg-config"
makedepends="libXdamage-devel libXinerama-devel libXrandr-devel libXtst-devel
 libjpeg-turbo-devel openssl-devel libvncserver-devel zlib-devel"
short_desc="VNC server for real X displays"
maintainer="Leah Neukirchen <leah@vuxu.org>"
license="GPL-2.0-or-later, x11vnc-openssl-exception"
homepage="http://www.karlrunge.com/x11vnc/"
changelog="https://raw.githubusercontent.com/LibVNC/x11vnc/master/NEWS"
distfiles="https://github.com/LibVNC/x11vnc/archive/${version}.tar.gz"
checksum=885e5b5f5f25eec6f9e4a1e8be3d0ac71a686331ee1cfb442dba391111bd32bd

if [ "$XBPS_TARGET_LIBC" = "glibc" ]; then
	makedepends+=" libgcrypt-devel"
fi

CFLAGS="-fcommon"

pre_configure() {
	autoreconf -fi
}
```
```sh
# Template file for 'libvncserver'
pkgname=libvncserver
version=0.9.14
revision=3
build_style=cmake
confiugre_args="-DWITH_OPENSSL=OFF -DWITH_GNUTLS=ON"
hostmakedepends="pkg-config"
makedepends="zlib-devel libjpeg-turbo-devel libpng-devel libgcrypt-devel"
short_desc="C libraries to easily implement VNC server or client functionality"
maintainer="Orphaned <orphan@voidlinux.org>"
license="GPL-2.0-or-later"
homepage="https://libvnc.github.io/"
distfiles="https://github.com/LibVNC/libvncserver/archive/LibVNCServer-${version}.tar.gz"
checksum=83104e4f7e28b02f8bf6b010d69b626fae591f887e949816305daebae527c9a5

libvncserver-devel_package() {
	depends="${makedepends} ${sourcepkg}-${version}_${revision}"
	short_desc+=" - development files"
	pkg_install() {
		vmove usr/include
		vmove "usr/lib/*.so"
		vmove usr/lib/pkgconfig
	}
}
```
With these changes, password authentication seemed to work as expected (I just tried the `-passwd` command line option, which is broken in the current version of the package). Unfortunately I'm really not interested in fiddling with this anymore. Hopefully this helps someone else make some progress.

The key changes are setting `confiugre_args="-DWITH_OPENSSL=OFF -DWITH_GNUTLS=ON"` for `libvncserver` and setting `
configure_args="--without-crypto --without-ssl"` for `x11vnc`.

EDIT: I also checked using a password file and it works with these changes (connecting works and the password file is not plaintext)

I needed to apply this patch (https://gitweb.gentoo.org/repo/gentoo.git/tree/x11-misc/x11vnc/files/x11vnc-0.9.16-crypto.patch?id=cea7c09c70390799b321479a59b597a25f951f9b) as well.