Re: problems with QMAILQUEUE and reading stdin

["George Georgalis" <george@galis.org>]

On Mon, May 29, 2006 at 11:49:32AM -0400, George Georgalis wrote:
>On Mon, May 29, 2006 at 10:38:19AM -0400, Charlie Brady wrote:
>>Add a comment about reading stdin if you really think you need it.
>
>yeah. that's what I did. I'll post the new ipsvd/QMAILQUEUE
>program here after some more light mods and testing.

Well here it is, http://galis.org/script/prequeue
not yet added to a version control management yet
and there is not a lot of mail on this Memorial Day,
so testing is incomplete, but this is a rewrite
of the program I've been using for active smtp
filtering, for the past few years, so it may be
okay.  Comments welcome and encouraged.

I tried or looked at qpsmtpd, simscan, qmail-qfilter
and qmail-spp; but for one reason or another stuck
with my own script.

http://smtpd.develooper.com/
http://www.inter7.com/simscan/
http://untroubled.org/qmail-qfilter/
http://qmail-spp.sourceforge.net

I'd really like it if qmail-spp made it easy to make
"entire email" filter plugins, that would probably
be the best way.

Sorry, no doc on entire deployment (that would be a
big project), but maybe if you are on this list you
can figure it out. Besides the djbdns lookups of RBL
lists, I'm using ipsvd (a daemontools replacement)
under runsvdir and maintain a peers.cdb to run qmail
with clamd and spamassassin; and there is an openbsd
spamd front end.

http://smarden.org/ipsvd/
http://www.openbsd.org/papers/bsdcan05-spamd/

I have a patch to make spamd more politically correct,
but it's not even applied to my own production yet,
eventually it will show up somewhere in here
http://galis.org/mkinst/patch/

That's about it. Enjoy. Peace.

// George

#!/bin/sh
#
# $Id$
# $GeorgalisG: prequeue$
#
# This script functions as a QMAILQUEUE program for "in SMTP" (active)
# filtering of email. It accepts stdin from qmail-smtpd, and expects
# associated environmentals. After it tests with clamav, it then tests
# the email with RBLs and spamassassin. Addition or removal of tests is
# a simple mod. Rejected messages are saved in a maildir (easy enough
# to disable).  There may be better way (such as umask and supplementary
# groups), but for now I run the various scanners as user qmaild so all
# programs have the read/write access they need.
#
# As root, run this once, to initialize
# pq="prequeue"
# install -o qmaild -g qmail -m 2770 -d ~qmaild/$pq ~qmaild/$pq/new ~qmaild/$pq/tmp  ~qmaild/$pq/cur
#
# LICENSE: <george@galis.org> wrote this file. As long as you retain this
# notice, you can do anything with it or buy me a beer -- George Georgalis
#
# exit 31 = permanently refuse
# exit 71 = temporarily refuse
#
# TODO deliver failures, with modified header to, and only to, valid users.

set -e # exit on internal error

ptr () { # reverse a dotted quad or subnet
 rev="$(echo "$1" | cut -d\. -f1).$2" ; ip="$(echo "$1" | cut -d\. -f2-)"
 [ "$ip" = "$1" ] && echo "${rev}" || ptr $ip $rev ;}

failforward () { # update ipsvd-instruct(5), cdb regenerated separately
 umask 002 ; echo "$peerm $opinion" >$peerd/$TCPREMOTEIP
 echo "$(basename $0): failforward: $opinion" 1>&2
 rm "$tmp" ; exit 31 ;} # permanently refuse

fail () { # mark the message with failure report and refuse
 formail -f -b -A "$opinion" <"$tmp" | maildir "$pq" >/dev/null # save in maildir for manual delete
 rm "$tmp" ; exit 31 ;} # permanently refuse

drop () {
 echo "$(basename $0): deny: $opinion" 1>&2
 rm "$tmp" ; exit 31 ;} # permanently refuse

warn () { # error, mark the message, save and refuse
 formail -f -b -A "$opinion" <"$tmp" >"${tmp}-$$"
 echo "$(basename $0): warn: $opinion" 1>&2
 mv "${tmp}-$$" ./ && rm "$tmp" # save for review
 # should monitor $PWD, or notify when warn is run...
 exit 71 ;} # temporarily refuse

pass () { # mark it and pass to the regular queue
 formail -f -b -A "$opinion" <"$tmp" | ./bin/qmail-queue ; testexit=$?
 rm "$tmp" ; exit $testexit ;} # return whatever qmail-queue exits as
 # somehow qmail-queue gets descriptor 1 from qmail-smtpd....

cd /var/qmail
host=$(cat control/me)
ptrip=$(ptr ${TCPREMOTEIP})
now="$(date "+%x %r %Z")"
pq="prequeue" # a maildir with qmaild write perms
peerd="supervise/qmail-smtpd/peers" # prepare to update ipsvd-cdb(8) config
peerm='#!/bin/sh\necho  "220 smtp port"\necho  "250 smtp host"\necho  "550'
# $pq/tmp is a tmp for this operation, $pq is tmp for this program
# $pq is also a maildir for messages rejected by this program
tmp="$pq/$(/usr/pkg/bin/safecat $pq/tmp $pq)" || exit 71 # </dev/stdin # put message to disk, if possible

# Check if $ACCEPT is set to tag message and bypass tests
if [ -n "$ACCEPT" ]; then
 opinion="X-ipsvd: $ACCEPT ($now)"
 pass
fi

if [ -n "$DENY" ]; then
 opinion="$DENY ${TCPREMOTEIP}"
 drop
fi
 
score="X-clamav: $(clamdscan --config-file=/usr/local/etc/clamd.conf --no-summary ${tmp})" ; testexit=$?
case $testexit in
 0) true ;; # no virus
 1) opinion="$(echo $score | sed -e "s;${PWD}/${tmp}: ;;") ($now)" ; fail ;; # virus found
 *) opinion="$(echo $score | sed -e "s;${PWD}/${tmp}: ;;") ($now)" ; warn ;; # clamav error
esac

opinion="X-sbl-xbl:$(dnstxt ${ptrip}sbl-xbl.spamhaus.org \
 | sed 's/http/ http/g' | grep http) ($now)" && failforward

# too many major ISP relays added
#opinion="X-sorbs-spam: $(dnstxt ${ptrip}spam.dnsbl.sorbs.net \
# | grep http)" && fail

# blocked yahoo groups... will restore after ACCEPT peers is fortified
# opinion="X-spamcop: $(dnstxt ${ptrip}bl.spamcop.net \
# | grep http) ($now)" && fail

# score upto 300KB with spamd, 250KB default, but no workie -s 307200 
score=$(spamc -x -c <"$tmp") ; testexit=$?
opinion="X-spamc: ${score} ${TCPREMOTEIP}; ${host} ($now)"
case $testexit in
 0) pass ;; # ham
 1) fail ;; # spam 
 *) warn ;; # spamc error 
esac

exit 81 # Internal bug


-- 
George Georgalis, systems architect, administrator <IXOYE><
http://galis.org/ cell:646-331-2027 mailto:george@galis.org