2017 successful install on Debian Ubuntu 16.04 Xenial Xerus

Discussion re sg development. You don't have to be a developer.

2017 successful install on Debian Ubuntu 16.04 Xenial Xerus

Postby Josh Parris » Thu Apr 27, 2017 11:58 am

By standing on the shoulders of the giants who went before me, and some help from the folks at spamgourmet.com, I have done it. Here's how:

Generic actions
I acquired a clean IP address and associated it with my hardened server
I installed a secure Apache webserver to run Perl
I installed exim4 STARTTLS using a free LetsEncrypt certificate. Admittedly, STARTTLS isn't a mandatory part of this process.
I ensured I had trustworthy email by having authentication using exim4, SPF, DKIM and DMARC. Another not strictly necessary part of the process, but ensures a higher level of email delivery.
You're going to need to install some Perl encryption libraries.
Having done the more generic things, it was time to do the more specifically spamgourmet things:

Acquire and customize Spamgourmet
Exchange any instance of example.com with your domain name
Code: Select all
cd ~
sudo apt install imagemagick libdbd-mysql-perl libclass-loader-perl
svn co https://svn.code.sf.net/p/spamgourmet/code/ code
cd code
OLDIFS=$IFS;IFS=$'\n';for f in `grep -rl /path/to/modules .`;do sed -i 's/\/path\/to\/modules/\/usr\/local\/lib\/spamgourmet\/modules/g' $f;done;IFS=$OLDIFS
OLDIFS=$IFS;IFS=$'\n';for f in `grep -rl /path/to/spamgourmet.config .`;do sed -i 's/\/path\/to\/spamgourmet.config/\/etc\/spamgourmet\/spamgourmet.config/g' $f;done;IFS=$OLDIFS
OLDIFS=$IFS;IFS=$'\n';for f in `grep -rl %imagefilename% .`;do sed -i 's/http:\/\/captcha.spamgourmet.com/https:\/\/example.com\/captcha/g' $f;done;IFS=$OLDIFS
OLDIFS=$IFS;IFS=$'\n';for f in `grep -rl /path/to/outbound.log .`;do sed -i 's/\/path\/to\/outbound.log/\/var\/log\/spamgourmet\/outbound.log/g' $f;done;IFS=$OLDIFS
OLDIFS=$IFS;IFS=$'\n';for f in `grep -rl /path/to/debug.txt .`;do sed -i 's/\/path\/to\/debug.txt/\/var\/log\/spamgourmet\/spamgourmet.config/g' $f;done;IFS=$OLDIFS
OLDIFS=$IFS;IFS=$'\n';for f in `grep -rl /path/to/spamgourmet.config .`;do sed -i 's/\/path\/to\/spamgourmet.config/\/etc\/spamgourmet\/spamgourmet.config/g' $f;done;IFS=$OLDIFS
OLDIFS=$IFS;IFS=$'\n';for f in `grep -rl /home/mora/src/spamgourmet/captcha .`;do sed -i 's/\/home\/mora\/src\/spamgourmet\/captcha/\/usr\/local\/lib\/spamgourmet\/captchasrv/g' $f;done;IFS=$OLDIFS
OLDIFS=$IFS;IFS=$'\n';for f in `grep -rl /tmp/sg/captcha .`;do sed -i 's/\/tmp\/sg\/captcha/\/var\/www-spamgourmet\/captcha/g' $f;done;IFS=$OLDIFS
sudo mkdir /usr/local/lib/spamgourmet
sudo mkdir /var/www-spamgourmet
mkdir /var/log/spamgourmet
sudo mkdir /var/www-spamgourmet/captcha
sudo mkdir /etc/spamgourmet
sudo cp -R captchasrv mailhandler modules /usr/local/lib/spamgourmet
sudo cp -R web/graphs.cgi web/html/* web/templates /var/www-spamgourmet
sudo cp conf/spamgourmet.config /etc/spamgourmet

You'll also need to steal the flagmap from the spamgourmet production site. I can't give you the code for that because it's 403 forbidden by bbs.spamgourmet.com

Note that there are various changes necessary to the codebase that we're in subversion at the time this was posted. I'm sure they will be soon.

MySQL
Install MySQL 5.7
Code: Select all
sudo apt install mysql-server-5.7 libdbd-mysql-perl
sudo mysql_secure_installation

Pick a strong password for mysql's root user; I generated a lowercase, words only memorable, strong password using correcthorsebatterystaple.net. Then I wrote that down on a piece of paper, because I only used it four or five times and that was with not knowing what I was doing. You ought to need it twice, when you create the database-user:
Code: Select all
mysqladmin -uroot -p create sguser

(enter that strong password now), and then again when you create the database:
Code: Select all
mysql -uroot -p

(enter that strong password now), and then create a second strong password to use in the mysql interactive prompt:
Code: Select all
create database sg;
grant all privileges on sg.* to sguser identified by 'secondstrongpassword' with grant option;
exit;

Next, back on the unix command line, we create and populate the database schema
Code: Select all
sudo mysql -usguser -psecondstrongpassword -Dsg <db.sql
sudo mysql -usguser -psecondstrongpassword -Dsg <dialogs.sql

Spamgourmet's homegrown Captcha server
Because of the 403 forbiddens, you've got to go off-site to read about making a captcha deamon for spamgourmet installations

Spamgourmet configuration
Code: Select all
sudo nano /etc/spamgourmet/spamgourmet.config

changes:
Code: Select all
%localdomains = (
'example.com' => 1,
'example.org' => 1
);
$admindomain = 'example.com';
$dbstring = 'DBI:mysql:database=sg;host=localhost';
$dbuser = 'sguser';
$dbpassword = 'secondstrongpassword';
$debugfilename = '/var/log/spamgourmet/spamgourmet.log'; # full path to debug output file
$webapproot = '/var/www-spamgourmet/';
$webtemplatedir = '/var/www-spamgourmet/templates/';
$secretphrase = 'thirdstrongpassword';
$mailhost = 'example.com';
$obmailhost = 'ob.example.com';
$adminemail = 'admin@realdomain.com';
$otherdomainemail = 'noreply@example.com';
  $captchagenhost = 'localhost';  #set this to empty ( '' ) to disable captcha
  $captchagenport = 15678;
  $normalURL = "";
  $secureURL = "";

You can now test these files run without crashing:
Code: Select all
perl /usr/local/lib/spamgourmet/mailhandler/outbound
perl /usr/local/lib/spamgourmet/mailhandler/spameater
perl /var/www-spamgourmet/index.pl

Once all is well, we will tie spamgourmet into apache:

Apache serving Spamgourmet
Code: Select all
sudo mv example.com-apache.conf /etc/apache2/sites-available/.
sudo ln -s /etc/apache2/sites-available/example.com-apache.conf /etc/apache2/sites-enabled/example.com-apache.conf

You ought to be able to interact with an eerily familiar site running on your own domain. Create yourself a test account, or for real account you're going to send test emails to.

Exim calling Spamgourmet
Check that you're relaying for outbound email.
Code: Select all
cat /etc/exim4/update-exim4.conf.conf

ought to include
dc_relay_domains='ob.example.win'
the ob can be whatever, but it needs to be consistent with your spamgourmet.config

Spamgourmet uses # in Reply Address Masking, so pull those out:
Code: Select all
sed -i '/^CHECK_RCPT_/ s/\#//g' /etc/exim4/conf.d/main/01_exim4-config_listmacrosdefs


Insert into /etc/exim4/router/00_exim4-config_header after "begin routers"
Code: Select all
spamgourmet_address:
  debug_print = "R: spamgourmet_address $local_part@$domain"
  driver = accept
  transport = sg_to_user_pipe
  domains = example.com
  errors_to = "[me]"

outbound_spamgourmet_address:
  debug_print = "R: outbound_spamgourmet_address $local_part@$domain"
  driver = accept
  transport = sg_from_user_pipe
  domains = ob.example.com


Insert into /etc/exim4/transport/00_exim4-config_header after "begin transports"
Code: Select all
sg_to_user_pipe:
  debug_print = "T: sg_to_user_pipe for $local_part@$domain"
  driver = pipe
  return_output = false
  temp_errors = 69
  ignore_status = true
  environment = REPLYTO=$reply_address:FROM=$header_from::TO=$header_to:
  command = /usr/local/lib/spamgourmet/mailhandler/spameater
  #user = "spamgourmet"
  #group = "spamgourmet"

sg_from_user_pipe:
  debug_print = "T: sg_from_user_pipe for $local_part@$domain"
  driver = pipe
  return_output = true
  environment = REPLYTO=$reply_address:FROM=$header_from:
  command = /usr/local/lib/spamgourmet/mailhandler/outbound
  # user = "spamgourmet"
  #group = "spamgourmet"

The user and group parts are if you've done something sensible and constrained the spamgourmet code to a particular user, which in this installation guide we haven't. Sorry. :(

Restart Exim 4, and you're done. I mustn't tell you how to, it's 403 forbidden. :shock:

If you need debug information from exim, edit /etc/exim4/conf.d/main/90_exim4-config_log_selector
Various parts of the spamgourmet code can have debugging turned on by changing a 0 to a 1, with debug information going to /var/log/spamgourmet/spamgourmet.log

No doubt there are things that I did that I have forgotten to tell you about. Please reply here noting them, so that (at the very least) others don't have to go through your pain.
Josh Parris
 
Posts: 8
Joined: Tue Apr 18, 2017 7:09 am
Location: Melbourne, Australia

Re: 2017 successful install on Debian Ubuntu 16.04 Xenial Xe

Postby josh » Thu Apr 27, 2017 8:42 pm

Awesome. Sorry it hurts so bad to do it
josh
 
Posts: 1371
Joined: Fri Aug 29, 2003 2:28 pm

Re: 2017 successful install on Debian Ubuntu 16.04 Xenial Xe

Postby Josh Parris » Tue May 02, 2017 1:49 am

josh wrote:Awesome. Sorry it hurts so bad to do it

The code was never intended for wide distribution, so the documentation reflects that. It's understandable; the service is the service, the sources are an afterthought. I put in the effort to make it easier for others... such as myself, when I do this again! Or yourself, if you have to restart from scratch for some God-awful reason.

To keep an eye out for updated source code, first do this:
Code: Select all
svn status --show-updates /home/ubuntu/code > /home/ubuntu/spamgourmet.svn.latest

Then periodically run this script:
Code: Select all
#!/bin/bash
#is_spamgourmet_up_to_date.sh
svn status --show-updates /home/ubuntu/code > /home/ubuntu/spamgourmet.svn.query
diff /home/ubuntu/spamgourmet.svn.latest /home/ubuntu/spamgourmet.svn.query
if [ $? -ne 0 ]; then
  {
    echo "The spamgourmet source repository has changed:"
    echo /home/ubuntu/spamgourmet.svn.query
  }
else
  echo "Spamgourmet is up-to-date"
fi
Josh Parris
 
Posts: 8
Joined: Tue Apr 18, 2017 7:09 am
Location: Melbourne, Australia


Return to Developers

Who is online

Users browsing this forum: No registered users and 9 guests