I finally got it working *cries in lost sanity*...

Turns out I (or nginx or whatever) was not passing the PATH_INFO fastcgi parameter.
Obscure chases through the internet led me to the final nginx.conf
configuration which finally made it work...
Please don't ask me on how this works, it's dangerous forbidden dark arts
of nginx magic. This was my first ever use of nginx, all my other thingies
are on obsd httpd.

Now to see how to work a chroot with nginx and fcgiwrap. #nginx irc users told me that this
is stupid and that this is an OS thing and not an application thing -.- , so that was fun.

Aisha

user nginx nginx;
worker_processes 1;

error_log /var/log/nginx/error_log info;

events {
	worker_connections 1024;
	use epoll;
}

http {
	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	log_format main
		'$remote_addr - $remote_user [$time_local] '
		'"$request" $status $bytes_sent '
		'"$http_referer" "$http_user_agent" '
		'"$gzip_ratio"';

	client_header_timeout 10m;
	client_body_timeout 10m;
	send_timeout 10m;

	connection_pool_size 256;
	client_header_buffer_size 1k;
	large_client_header_buffers 4 2k;
	request_pool_size 4k;

	gzip off;

	output_buffers 1 32k;
	postpone_output 1460;

	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;

	keepalive_timeout 75 20;

	ignore_invalid_headers on;

        server {
		server_name localhost;
		listen 0.0.0.0;
		listen [::];

		location ~ / {
			fastcgi_split_path_info ^(/cgi\-bin/man\.cgi/)(.+)$;
			include fastcgi_params;
			fastcgi_pass 127.0.0.1:12345;
			fastcgi_param SCRIPT_FILENAME /var/www/cgi-bin/man.cgi;
			fastcgi_param PATH_INFO $fastcgi_path_info;
		}
	}
}