Under attack

Discussion of items in the "What's New" log.

Under attack

Postby SysKoll » Tue Mar 18, 2008 6:08 pm

Folks,

Just to keep you posted. The spamgourmet mail server is under Denial-of-Service attack. We see litterally hundreds of thousands of connections and connection attempts, most of them malicious. The machine is heavily loaded. Mail is getting delayed because legit email servers often cannot connect.

Sadly, most of our time is spent babysitting the server instead of improving the service. Thank you, Microsoft, for providing millions of easy-to-Trojan machines to malevolent spammers.

-- SysKoll
-- SysKoll
SysKoll
 
Posts: 893
Joined: Thu Aug 28, 2003 9:24 pm

Postby fmus » Sun Mar 23, 2008 1:35 am

Thanks for continuing the good fight.
fmus
 
Posts: 17
Joined: Tue Dec 06, 2005 8:41 pm
Location: carrollton tx

Postby gourmet » Thu Mar 27, 2008 5:07 pm

Here are a few lame questions and suggestions.

Could it be possible to add more front end servers? It seems (for outsider at least) that all SMTP connections are eaten up. Maybe with stale sessions or with real spam. But result is same. Incoming smtp sessions are rejected / refused.

AFAIK, adding front end servers would help.

What do you guys think about this approach?

What kind of attack it is?

I have been chatting with some other admins (other spamgourmet like services). And they point out at least two favorite attacks.

SYN floods, stale or very slow SMTP sessions.

SYN flood can be rejected using SYN cookies and with stale sessins it helps to set smtp session time out to quite short time. Like 1-2 seconds. It might cause although some problems, if some sending server is darn slow, then it might cause connection to time out. But with properly working services that shouldn't be a problem.

Any comments about that?

I guess you have already take care of these kind of attacks, have you?
gourmet
 
Posts: 124
Joined: Thu Mar 27, 2008 4:46 pm

Postby SysKoll » Thu Mar 27, 2008 7:44 pm

We are seeing exactly these attacks. SYN floods are a pain. We already do cookies and a short SMTP timeout.

What would really help is if ISPs stopped letting individual machines connect to port 25 of a random server (except for the ISP's). Very few individuals run their own SMTP transfer agent and cannot justify port 25 egress connections. Among broadband DLS and cable subscribers, more that 99.9% of the machines conneting to a 3rd-party port 25 are Trojaned PCs running Windows.
-- SysKoll
SysKoll
 
Posts: 893
Joined: Thu Aug 28, 2003 9:24 pm

Postby gourmet » Fri Mar 28, 2008 7:42 am

SysKoll wrote:We are seeing exactly these attacks. SYN floods are a pain. We already do cookies and a short SMTP timeout.

Great.
What would really help is if ISPs stopped letting individual machines connect to port 25 of a random server (except for the ISP's).

Standard protocol in Finland. Even for small and medium businesses. All ISP's do that. And you need pretty good reasoning why you should be able to use SMTP port and usually only for selected IP, not for whole network.

Having open SMTP traffic to internet in some cases causes whole ISP's network block to be blacklisted. And ISP's really try to avoid that. That's why they're also forcing to use ISP's SMTP server.

Among broadband DLS and cable subscribers, more that 99.9% of the machines conneting to a 3rd-party port 25 are Trojaned PCs running Windows.

Yup.

You didn't comment how spamgourmet performance could be improved? If current problems are tcp/smtp related. I would suggest having following configuration.

two or more front-end servers. and one backend server (current gourmet.spamgourmet.com). Gourmet.spamgourmet.com would allow connections only from front end servers. So all syn floods, stale sessions etc. Would be handled by those. Then real message traffic could be efficiently forwarded to main server using esmtp.

Any comments about this suggestion?

Btw. It seems very common that ISP's and other mail service providers have multiple front end servers. And many of those are related to base filtering. Rejecting most of stuff so real mail servers won't be loaded.

Multi server approach stuff, more about costs here:
http://bbs.spamgourmet.com/viewtopic.php?p=5300#5300

So it seems pretty expensive. ;(
gourmet
 
Posts: 124
Joined: Thu Mar 27, 2008 4:46 pm

Postby Paranoid2000 » Sun Jun 01, 2008 9:49 pm

How about trying the Advanced Policy Firewall with an appropriate IP-address blocklist? It also has options for DDoS attacks (Dshield.org blocklist import, Spamhaus "Don't Route" list support, reactive address blocking, etc).

See the Bluetack: advanced policy firewall (apf) for linux servers thread for more details.
Paranoid2000
 
Posts: 71
Joined: Wed Dec 15, 2004 10:48 am

Postby SysKoll » Sun Jun 08, 2008 9:08 pm

We are looking at multi-machine setups, but the expenses for this site would then start to be quite high. We want to be able to continue offering free service.
-- SysKoll
SysKoll
 
Posts: 893
Joined: Thu Aug 28, 2003 9:24 pm

Go colo with the users

Postby miniscus » Tue Jun 10, 2008 9:48 pm

SysKoll wrote: ... but the expenses... continue offering free service.


You cannot have it all, you know. :lol:

This may be off limits for some reason, then please ignore.

Go colo with the users.

Make up a bounty barometer that is very well visible, and ask users to register as reliable supporters for as long as they want to.

It is all about knowledge of reliability for the admins, and maybe the mode of payment. Not Money! Divide the 800 x 13 (per year) that josh mentioned thru 1/170th of all accounts = 1000 and marvel ;-)

I would register with spamgourmet as a reliable supporter with my e-mail address. In the future you guys could remind of me of the barometer - in case necessary.

Admins and users could first watch if enough registered users also register as reliables, via a reg. barometer. If yes, change the barometer to show $ balance, and show the yearly amount needed from each.
At last ask reliables by e-mail to beginn donating, like once a year maybe.

No strings attached. SG would get better, and will last anyway :-)
Arick
miniscus
 
Posts: 48
Joined: Thu Aug 28, 2003 10:05 pm
Location: Wiesbaden, Germany

Postby Paranoid2000 » Thu Jun 19, 2008 6:27 am

SysKoll wrote:We want to be able to continue offering free service.
How about offering a premium option, with extra features for an annual subscription? This could be used to fund SG's expansion and allow the continued supply of the free service.

I'm amazed that SG has been able to continue on a donation basis for so long (kudos to the admins and contributors) but now may be a good point to consider its future. I would certainly be prepared to pay to guarantee the continuation of this service and I suspect many others would too.
Paranoid2000
 
Posts: 71
Joined: Wed Dec 15, 2004 10:48 am

Postby gourmet » Fri Dec 19, 2008 11:26 am

Is this attack still going? My friends web hosting company was just attacked by botnet. They got over 20 Gbit/s bandwidth consumption and attacks came from over 40000 machines.

So it depends a lot, what is "serious" attack and what is not.

Is the problem still the same as it was earlier. All incoming SMTP connections are occupied and no new connections are accepted.
gourmet
 
Posts: 124
Joined: Thu Mar 27, 2008 4:46 pm


Return to What's New

Who is online

Users browsing this forum: No registered users and 25 guests

cron