ITS BACK UP!
 

What happened, why were the forums down?

Re: ITS BACK UP!
 

Hard Drive failure.

Re: ITS BACK UP!
 

Really? That sounds like something that would happend to one of us.... wait, it has happened to many of us.

Can you offer more details Richard?

Re: ITS BACK UP!
 

YAAY! http://forum.shrapnelgames.com/images/icons/icon10.gif http://forum.shrapnelgames.com/images/icons/icon10.gif

Re: ITS BACK UP!
 

Yay! It's just not the same without the forum...

Last night I was about to post something when the whole Shrapnel site went down. Then I kept checking every so often. This morning I saw the note that said that the server failed, and then a bit later the main site came back up but not the forums... Anyway, it's good to have the forum back now! http://forum.shrapnelgames.com/images/icons/icon7.gif

Re: ITS BACK UP!
 

You saw a NOTE? I was this note posted? I looked on the main page but saw nothing.

Re: ITS BACK UP!
 

This morning when I went to the Shrapnel home page, it was just a white screen with a text message that explained that the server failed but you could still order products Online from Gamers Front, with a link to it.

Last night, there was nothing and you just got a page not found error.

Re: ITS BACK UP!
 

Thanks. http://forum.shrapnelgames.com/images/icons/icon7.gif

This is what happens when you let employees use the web sever for browsing of the internet. Someone hit a porn site by mistake, ah huh, and the pop ups overflowed the system buffer, causing a cascade server failure of the main pipe line resulting in a CPU over usage alarm which caused the Hard Drive back up buffer to fail thus shutting down the redundent RAM data and tape back ups causing the Hard drive to loose integrity resulting in Hard Drive system failure.

NOw this could have all been avoid if Shrapnel Games would have invested in the Atrocities 2000, a pop up killer that blocks porno pop ups and redirects them toward competator sites. http://forum.shrapnelgames.com/images/icons/icon10.gif

Re: ITS BACK UP!
 

Ahh thank lord this is back up. As much work i actually got done yesterday as i wasn't constantly here it just doesn't feel the same, are we sad? http://forum.shrapnelgames.com/images/icons/icon10.gif

Hard Drive Failure? Is the harddrive located in canada? if so i'm convinced theres a conspiracy! http://forum.shrapnelgames.com/images/icons/tongue.gif

Re: ITS BACK UP!
 

Ok here is what happened.
Due to new staff on board with technical backgrounds, conversations were restarted involving things like servers, upgrades, and policies such as backups and assigning caps on some services. Someone didnt knock on virtual wood or something because Murphy got wind of our plans. In the middle of the night the system stopped responding. We were notified by the hosting company that pings were failing. reboot authorized. reboot froze. reloading of the OS to the second installed harddrive authorized. system came up (but of course no web pages or anything). scan of the initial harddrive (which takes a LOOoonnngg time) showed over 90% of the files were recoverable. Between backups and files from the old harddrive things should get fully back in swing soon. Lesson learned; never forget to appease Murphy.

Re: ITS BACK UP!
 

<font size="2" face="sans-serif, arial, verdana">Murphy can kiss my a$$!!!! If there was a second drive, where was the raid?

Re: ITS BACK UP!
 

There wasn't a second hard drive before the crash, that was the old drive we were pulling data off of.

But the end result is a pretty minor outage of a day for the forums.

So everythings good http://forum.shrapnelgames.com/images/icons/icon7.gif .

Re: ITS BACK UP!
 

<font size="2" face="sans-serif, arial, verdana">I didnt say the second drive wasnt already being used for something.

Windows and raid? Well that would have been part of the discussions which got murphys attention I guess. Maybe it was included in with the multi-ISP multi-homed multi-pathed load-balancing Sun cluster conversation. (hmmm I think I have most of that sitting in my garage)

[ July 10, 2004, 15:38: Message edited by: Gandalf Parker ]

Re: ITS BACK UP!
 

yea - next time AT give a guy a chance to check the bloody page before signing off!

Re: ITS BACK UP!
 

<font size="2" face="sans-serif, arial, verdana">I didnt say the second drive wasnt already being used for something.

Windows and raid? Well that would have been part of the discussions which got murphys attention I guess. Maybe it was included in with the multi-ISP multi-homed multi-pathed load-balancing Sun cluster conversation. (hmmm I think I have most of that sitting in my garage) </font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">LOL Well, I can't do the SMS stuff but I can do it with IE6 and ISA. Perhaps port it to a dual SQL front end. And then do a hot fail over to a remote site so there are never any outages. Course, we will need to find someone to foot the bill http://forum.shrapnelgames.com/images/icons/shock.gif

Re: ITS BACK UP!
 

<font size="2" face="sans-serif, arial, verdana">That was kindof the point. Everything has its pros and cons.

Ive been with ISPs that went the extreme route on stability and it generally wasnt worth it. Id rather have script routines to zip individual directorys and copy them to other machines. Whenever equipment fails you might as well reconsider, purchase upgrades, rebuild the site with some of what was there and upgrade others. The cost and extra security headaches to get instant recovery or invisible crashes never seemed worth it to me for most sites.

[ July 11, 2004, 04:46: Message edited by: Gandalf Parker ]

Re: ITS BACK UP!
 

<font size="2" face="sans-serif, arial, verdana">I don't know about you all, but I have no life and am a pathetic man who is addicted to this forum like it was crack cocain. So even though everything is good, it was not that good for me. http://forum.shrapnelgames.com/images/icons/icon9.gif I need my daily Shrapnel Forum FIX maaaaaaaaaaannnnnn. Did I mention I have no life?? http://forum.shrapnelgames.com/images/icons/icon12.gif

Re: ITS BACK UP!
 

<font size="2" face="sans-serif, arial, verdana">Honest not my fault. http://forum.shrapnelgames.com/images/icons/icon7.gif I only mentioned this to Mr. Murphy. I had nooooooo idea that he would take it so personal and all. http://forum.shrapnelgames.com/image...s/confused.gif

Re: ITS BACK UP!
 

[quote]Originally posted by Gandalf Parker:

Quote:

Ive been with ISPs that went the extreme route on stability and it generally wasnt worth it. Id rather have script routines to zip individual directorys and copy them to other machines. Whenever equipment fails you might as well reconsider, purchase upgrades, rebuild the site with some of what was there and upgrade others. The cost and extra security headaches to get instant recovery or invisible crashes never seemed worth it to me for most sites.

<font size="2" face="sans-serif, arial, verdana">We do a SQL based web app that 600 user have to have, or no business gets done. So we are up during business hours, 98% 14/6 Last phys. year and it would have been 100% if Verizon would get their ducks all in a row. We were 100% on the LAN, but the WAN had some outages do to dead lines. Servers were 100% even during Version upgrades.

Re: ITS BACK UP!
 

<font size="2" face="sans-serif, arial, verdana">Ive worked for people who achieved such numbers also. I just didnt think it was worth the cost and effort. http://forum.shrapnelgames.com/images/icons/icon7.gif

Re: ITS BACK UP!
 

<font size="2" face="sans-serif, arial, verdana">Honest not my fault. http://forum.shrapnelgames.com/images/icons/icon7.gif I only mentioned this to Mr. Murphy. I had nooooooo idea that he would take it so personal and all. http://forum.shrapnelgames.com/image...s/confused.gif </font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">actualy I was refering to you asking me via msn if I could get to the intel forums then signing off before I could answer.

Re: ITS BACK UP!
 

We had to do the same thing for att.com when I worked at AT&T. The funniest thing is being oncall and then the Super Bowl ad comes on.

"Countdown 5,4,3,2,1"

"Switchover to Server 2"

"Switchover to Server 3"

etc, via pager. You could set your watch to it http://forum.shrapnelgames.com/images/icons/icon12.gif .

<font size="2" face="sans-serif, arial, verdana">Ive worked for people who achieved such numbers also. I just didnt think it was worth the cost and effort. http://forum.shrapnelgames.com/images/icons/icon7.gif </font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">