Server resources

Discussion re sg development. You don't have to be a developer.

Server resources

Postby maratheamit » Thu Mar 25, 2004 5:42 pm

This is a follow-up to the thread mentioning that SG handles 1GB of spam per day.

Josh observed that the volume of messages is not a problem right now. But there are a couple of things worth considering.
1. avoid launching a sendmail process to send outgoing messages. instead we can use a perl module (Net::SMTP?) to talk directly to the MTA.
2. doing some filtering at the MTA. if we can discard some messages as they come in, it will be more efficient than the spameater processing which comes later. before we can implement this idea, it would help to have some statistics on the incoming spam: what fraction come to expired addresses, what fraction to non-existent users, ...
maratheamit
 
Posts: 82
Joined: Fri Aug 29, 2003 2:35 pm

Postby josh » Mon Mar 29, 2004 10:23 am

I've got some code that opens a socket -- it's written to do http right now, but we could change it to do SMTP -- it doesn't require any modules that aren't in the base distro.

spameater is a little more modular right now -- there's a separate module that sends the mail -- the current one is CommandLineMail.pm (I think that's it).

Amit - the test server is online, but it's not set up all the way yet. I'm overseas (I suppose I don't know where you are, so maybe I'm no longer overseas from your perspective :)), and Syskoll is buried at $DAYJOB. Send me an email if you get a chance and let you know more about it. I think we still need to set up mysql, and I'm torn about whether to go with 3.23 (which is on the production server) or with a newer version.
josh
 
Posts: 1371
Joined: Fri Aug 29, 2003 2:28 pm

Postby maratheamit » Sun Apr 11, 2004 8:31 pm

I installed mysql-3.23, apache, php and perl on the test server. I would also like to get around to playing with postgres and exim at some point. But before that I plan to have the test website up and running.

Josh, is there documentation (however elementary) on the steps that need to be carried out? The SG database needs to be created and populated with the dialogs (for starters). If you send that list to me I will try to get test.spamgourmet.net into a functional state and also flesh out the documentation along the way.
maratheamit
 
Posts: 82
Joined: Fri Aug 29, 2003 2:35 pm

Postby josh » Thu Apr 15, 2004 1:16 pm

Amit,
I jumped the gun and set things up most of the way (only took about a half hour!) , but config is probably still not complete. The documentation is outdated. The reason I did it, rather than use the obvious opportunity to generate good documentation, is so that we can start looking at this multiple spamgourmet recipient problem -- currently the code only handles one received spamgourmet address per message.

I'm looking at a way to configure sendmail to split received messages into one per each recipient, but haven't found that yet. The longer term solution is to properly parse the addresses, and put most of the logic into a for loop that processes all the recipients.

Anyway, I have to stop for now -- I think there are still some config issues, because I didn't receive my confirmation email when I created an account. I'm using the production captchaserv and graph server btw, in case you were wondering.

Hopefully I can look at it again soon -- y'all are welcome to in the meantime. Contact me for the user account info that it's all running under.

Josh
josh
 
Posts: 1371
Joined: Fri Aug 29, 2003 2:28 pm

Postby josh » Thu Apr 15, 2004 1:51 pm

It's all working now -- I had some additional config for sendmail (to enable it on an external port) and I hadn't set up an MX record for the test domain (oops).
josh
 
Posts: 1371
Joined: Fri Aug 29, 2003 2:28 pm


Return to Developers

Who is online

Users browsing this forum: No registered users and 16 guests

cron