Testing Infrastructure

Discussion re sg development. You don't have to be a developer.

Testing Infrastructure

Postby maratheamit » Sat Oct 18, 2003 3:38 pm

The crisis with spamcop is a good excuse to start a discussion on how we can mitigate such happenings in the future. What happened there was that Mail::Audit didn't preserve the old headers on a resend and since our testing didn't include looking at headers, the change went into production with disruptive effects.

Of course, we will include such checks in our future tests but we should start a test-plan file containing items to test for before release. And we can keep adding to this file over time. This will by no means eliminate buggy releases but at least we can avoid making the same mistake again.

On a related note, we should funnel all code changes through CVS. To that end both the production and test servers should run on a copy checked out from CVS, with only the configuration files being changed from the default CVS values. The production environment may lag the latest version found in CVS but it should not contain any changes not in CVS. We can make use of the branch/tag features in CVS to handle the case where we want to get a hot-fix release out at short notice.
maratheamit
 
Posts: 82
Joined: Fri Aug 29, 2003 2:35 pm

Postby josh » Sat Oct 18, 2003 4:04 pm

I agree. We do have the filename problem with the test environement, but we can work around that. I'll start scouting out another box for testing (which is generally a good idea, anyway, of course).

I ask one thing -- let me get the throttling code in place before we start -- this'll prevent a bunch of crap hard-coded stuff from ever going into CVS and get us on the right track.

Perhaps our test bench should be automated with a script that sends and receives (or doesn't receive) a set of email messages that cover the situations we encounter. I know both of you have included test code with your submissions, so we can get all that in the same place, too, if possible.

Also, contrary to my previous assertion, I did start to see a CPU issue with M::A -- it looked to averaging about 5x the CPU load average of the old. I know this all goes away when we go stateful, but if we distribute the code for stateless operation, it's nice to think about choosing either M::A or the older stuff. I know this could be a real pain in the *ss, and so I'll give it up quickly under pressure :)
josh
 
Posts: 1371
Joined: Fri Aug 29, 2003 2:28 pm

Can you please explain staeful/stateless distinction?

Postby SysKoll » Sun Oct 19, 2003 2:23 am

Do you mean stateful == with a demonized spameater?
-- SysKoll
SysKoll
 
Posts: 893
Joined: Thu Aug 28, 2003 9:24 pm

Postby maratheamit » Sun Oct 19, 2003 3:51 am

Josh, the filename problem can be solved by naming the two links differently. i.e. in /usr/adm/sm.bin execute
ln -s /home/jqh1/spameater spameater-prod
ln -s /home/sgtest/spameater spameater-test

Just as a directory cannot contain two files of the same name, it cannot contain two links of the same name. With my suggestion, the .forward file will have to be different for the production and test environments (if they are hosted on the same machine). But the forwarding code can be identical can live in a file called spameater.
maratheamit
 
Posts: 82
Joined: Fri Aug 29, 2003 2:35 pm

Postby josh » Mon Oct 20, 2003 4:47 pm

yes (by "stateful" I meant daemonized -- I'll avoid that murkiness in the future)

and -- :oops: I'll do the symlink thing here in a minute, too.
josh
 
Posts: 1371
Joined: Fri Aug 29, 2003 2:28 pm


Return to Developers

Who is online

Users browsing this forum: No registered users and 14 guests

cron