Page 1 of 4

service outage

PostPosted: Sun Dec 12, 2004 12:37 pm
by info
The affected server is currently down and will be rebuilt soon by our service provider. After that, I'll get the service running again. This is some significant downtime, but hopefully won't result in lost mail, just delayed mail. I've moved the support bbs over to another of our boxes so that it can stay up during this time. I'll post updates to the main site and answer questions here...

PostPosted: Sun Dec 12, 2004 1:06 pm
by basti007
Too bad i'm waiting for some very important emails since this morning... when do you think will the service be available again?


And of course thanks for your great support and good luck...

PostPosted: Sun Dec 12, 2004 5:29 pm
by info
thanks! apparently we need an additional agreement with the service provider, so I'm waiting by the fax machine for that. Then it'll be a couple hours at least before they put the OS on, then we've got some installation and a lot of configuration after that, so we're looking at this evening (US) at the earliest

PostPosted: Sun Dec 12, 2004 7:57 pm
by info
update: server rebuild is currently underway

PostPosted: Sun Dec 12, 2004 10:19 pm
by magik
i'm sorry to hear that you guys got a virus :cry:

but it's great how fast you guys are trying to fix it and get everything back up, best of luck! :D

PostPosted: Sun Dec 12, 2004 10:33 pm
by info
update: new server is up, and mail service is restored. The server is working *very* hard to catch up with the buzillion messages that queued up all over the place while we were down - please be more patient that you thought you would have to be.

I'm still working to restore some of the auxilliary services that will enable new user accounts, for instance.

PostPosted: Sun Dec 12, 2004 10:53 pm
by basti007
Hi,

can confirm that first of several awaited messages reached me. Thanks for the great and fast support!

:-)

PostPosted: Sun Dec 12, 2004 11:36 pm
by info
cool! It's still going to be slow-going for awhile, though.

We're now pretty much fully functional, in that the new user system is back up.

The graph thing on the front page is still down, but i'll get to that later....

Lost mail?

PostPosted: Sun Dec 12, 2004 11:49 pm
by xdcdx
Good to hear everything is up and running again. :)

Did any mail get lost or just delayed?

How much down time would had made the server to start lossing mail?

PostPosted: Mon Dec 13, 2004 12:12 am
by info
most mail system time out at 4 days -- we were down for less than one.

Was every bit of mail delivered (or will it be?)? I really can't answer that -- the system has been processing > 100,000 messages a day from a whole bunch of different servers -- I feel compelled to refer back to the "don't use spamgourmet for important stuff" caveat in the FAQ -- but...

I can say that we were able to use an approach that would cause a TEMPFAIL condition in upstream mail servers if they were RFC compliant, and so we would expect that mail would be delivered (eventually, I suppose -- most keep trying at some interval for 4 days -- some will have to try more than once because the server is so busy catching up -- 3 more days should be plenty, though).

Beyond that, we stopped taking new users and shut down user admin so that whatever gap there was between when we took the database image and when we shut down (just a few seconds) would result in erring on the side of delivering too much mail -- that is, an address that expired during that gap would be unexpired when the new db came back up, and a new address that was created in that gap would be created again on the new db.

PostPosted: Mon Dec 13, 2004 1:03 pm
by basti007
At the moment, emails are bouncing back again from spamgourmet - but i guess, that's just the heavy traffic.

I have approx. 5-6 more mails, that should arrive. 2 have arrived yesterday, since then, nothing is coming through anymore. Guess, we have to wait a little longer - or until the mail server try to send out their stuff again (don't know in which intervals they try)... :)

Re: service outage

PostPosted: Mon Dec 13, 2004 3:15 pm
by uhu
info wrote: After that, I'll get the service running again. This is some significant downtime, but hopefully won't result in lost mail, just delayed mail.


I'm afraid, that there is a loss of mail :-(

The WebPage works again, but I'm missing several mails (CC from discussion board an a self written testmail).

No eaten Spam is shown on my page for today - therefore - mail seems to go to a black hole.


Sincerely yours

"Uhu"

Re: service outage

PostPosted: Mon Dec 13, 2004 3:28 pm
by Guest
uhu wrote:
info wrote: After that, I'll get the service running again. This is some significant downtime, but hopefully won't result in lost mail, just delayed mail.


I'm afraid, that there is a loss of mail :-(

The WebPage works again, but I'm missing several mails (CC from discussion board an a self written testmail).

No eaten Spam is shown on my page for today - therefore - mail seems to go to a black hole.


Sincerely yours

"Uhu"


Probably you just have to wait until the mail server trys it again. I myself am waiting for 2 mailings with voucher codes in it - sent out yesterday and stil nothing received, yet.

PostPosted: Mon Dec 13, 2004 4:38 pm
by Guest
testmails sent, nothing bounced, yet nothing received, there is still hope! :?

PostPosted: Mon Dec 13, 2004 4:54 pm
by SysKoll
Just received a test email. It stayed 2 hours in the queue. The machine is very busy. Give it a chance to catch up.