About the downtime tonight... Democratic Underground failed you, and we are sorry.

[Skinner]

Democratic Underground failed you tonight. We should have been able to provide access to the site, but we did not. I want you to know that the DU Administrators believe that repeated extended downtime like this is not acceptable. We are sorry.

Here's what happened: We lost power AGAIN.

This wasn't supposed to happen. After we had this problem during the VP debate, we had a conference call with our web hosting company to discuss our power situation. They ran power analysis to figure out how much power we were using, and how much we would need to handle a big spike like tonight. They split the servers onto separate circuits to spread the load around this time. But we lost power again anyway. Apparently this time was a different circuit. (For the DU computer geeks: It was the master switch.) But even if it was a different circuit, it shouldn't have happened.

You all have been extremely patient and understanding. Thank you for that. I think blowing a circuit once is understandable. But having it happen *twice* is simply not acceptable.

We are going to do everything we can to make sure this doesn't happen again. We're going to be discussing this issue with our web host tomorrow. If we have to camp a technician next to our servers for the next three weeks, we'll do it.

Thank you for your understanding. Again, we apologize.

David Allen
DU Administrator

[Doityourself]

1. So, it's not the multiple posts then?

[Danger Mouse]

2. I'm sure you guys will do your best to make sure it doesn't happen again.

Hopefully on Election Night we don't lose power!!!

[smiley_glad_hands]

3. No problem. Keep up the good work.

[mvd]

4. Thanks for this post

I know that you will address the issue and that election day will go smoothly. :hi:

[sixmile]

Skinner, you're the Man...

Should we have our tinfoil hats on?
Seems a strange coincidence, maybe the site is getting slammed by outside sources?
Cue mysterious music.

[truebrit71]

5. Thanks for letting us know Skinner..I cannot imagine NOT being able to get my DU on 11/04...

...that would drive me to drink...oh wait...I'll be drinking that night anyway ;-)

[Frisbee]

130. I'm prepared...

Got a bottle of Asti I expect to be opening... :toast:

but plenty of beer just in case I need it to cry in. :beer:

[LynnTheDem]

6. I'm really in the mood for a large juicy cheesieburger...I'd totally forgive you

if you happened to have one handy....

*tummy growl*

[Staph]

7. How much do y'all need

to fix the problem before November? I suspect one or two (thousand) of us would gladly pony up a few bucks to buy a new whatever-you-need-to-fix-it thing!

[emilyg]

105. I'm with you

An upgrade fund drive?

[tekisui]

8. It happens.

No worries, I hope we can stay on Election Night, I want to party with you and everyone else here!

[yardwork]

9. It's fine. I appreciate the fact that the board is up 24/7. It went down briefly - ok.

[mucifer]

10. I do appreciate all of your hard work!

[cynatnite]

11. It's all good, Skinner...

I was a frustrated fool during the debate and posting was not a good idea. Now, that I've cooled off...I'm good to go.

You guys are great and most of us love ya.

[steve2470]

12. Thanks for the fast response ! I know you are doing the best you can ! nt

[Coexist]

13. Please, PLEASE! not on election night.

please.

[deadparrot]

14. Don't worry about it.

Things happen. You guys are always on top of things; I'm not worried.

[SwampG8r]

15. good on you for taking responsibility

i would have started a conspiracy theory myself

[grantcart]

16. suggestion when we go to defcon 3 (or what ever it is) put a 3 thread limit

should solve a lot of problems

[cliffordu]

17. Hang in there, big guy, I'm in your corner no matter what.

hardware ALWAYS follows Murphey's Law....

One of these days I'll tell you about the wedding I photographed with a camera that had a bad focus plane.....

[grantcart]

27. and I am behind you as always

[cliffordu]

63. That tickles....

[SwampG8r]

84. can i get nexties?

[cliffordu]

108. Got a dollah??

[1corona4u]

18. Well, what time did it go down? Because I see that people were posting

during the debate...

[MrScorpio]

19. It's alright

We're back now

It's all good!

[RetroLounge]

20. No worries

thanks for all you do for us...

:hi:

RL

[AwakeAtLast]

21. Hmmmmm, must have happened after I handed over the laptop to my husband

He needed it to look up some things for work.

I'm feeling no frustration here!

:hi:

[KathieG]

22. Thanks for the explanation. I will say...I was having withdrawls when DU was down.

[jjanpundt]

23. Skinner you all do an outstanding job, it's just that we totally

freak when DU goes down.

[krawhitham]

all is forgiven if it is fixed by Nov 4th :)

[percussivemadness]

24. I run a hosting company with 99.9% uptime over the last 5 years, hit me up if you need advice

[AbbeyRoad]

25. Thanks for the explanation. n/t

[nickyt]

26. No problem, Skinner! We know you do everything you can to keep us plugged

in - don't give it a second thought. Seriously. Sh!t happens, no worries!

[gateley]

28. Shit happens. We survived. Don't beat yourselves up. nt

[vanderBeth]

29. No problemo

:grouphug:

[bdamomma]

30. well things happen.

I guess we are a powerful force.

[Ilsa]

31. Hey, you got us about 2/3 or more of the way through. Wonderful, really.

We love everything you do for us here.

[BeyondGeography]

32. Thanks for the note...we love you guys and we love this site

Do a targeted fundraiser if that'll help. We'll pitch in.

[davidpdx]

33. I understand, thanks for letting us know

I hope you read the riot act to your webhosting company and get it fixed for election day. The site is going to get hit hard.

David

[Tatiana]

34. Thanks for the info. I'm glad that you could share what happened with us.

I'm confident you all are working on the problem.

[leftofthedial]

35. well, I guess we'll let it slide this time...

hey. No worries. This is still the best hang on the web.

[Captiosus]

36. You didn't fail us, and I'd take offense to anyone who says you did.

Glitches happen. Web servers are never guaranteed to have 100% uptime.

DU has performed admirably during this election season.

[renate]

37. you guys are so sweet... thank you so much for all you do

:loveya:

:grouphug:

:applause:

[nc4bo]

38. **sniff-sniff** but I missed DU. Nearly had a McFit.

I think we're all mature enough to understand that shi^ happens and know that you guys in charge will get it sorted it out by 11/4 and 5.

:patriot:

[RoccoR5955]

39. Oh well Skinner... I guess that fecal matter happens

when you least need it to happen.

[Jamesm9164]

40. No problem!

I have not logged on for sometime, but just wanted to say - this is still the best damned blog on the internet.

[InvisibleTouch]

41. No worries, it happens.

I didn't watch the debate on TV (too stressful), so I was looking forward to "watching" it by proxy here at DU - but, I can catch up with what I've missed now, so no harm done. I know you guys are doing your best to fix the issue, and there are just some things one has no control over. DU is still the best source of news and info on the web.

[Jennicut]

42. No problem....there is so many of us and we are all super excited.

We overloaded the system. Simple as that. I wonder if Freeptardland can boast that so many wanted to get on that the system crashed. I bet everyone there is demoralized and jumping ship soon.

[geekgrrl]

43. No worries! I kind of went down during the debate too. From laughing at McLame. nt

[gratefultobelib]

44. It happens. I thought it was my computer!!

[WeDidIt]

45. It's Cool, David

I work in IT. The worst shit happens at the worst time.

[five_horizons]

46. No problem. We appreciate everything you do.

[workinclasszero]

47. Please don't let this happen on election night

That would be the suck! I need to celebrate with all you guys that night for sure!:party:

[NC_Nurse]

48. No worries. I'm amazed at how well things usually run!

:yourock:

[CitizenPatriot]

49. you guys rock

and this site rocks.

We know you'll do your best. The only problem is it hurts to go through this angst without the DU!

Thanks for addressing it --

Here's to Victory Nov 4

Woo Hoo!!!! Gear UP, People!

[iconicgnom]

50. congrats on the "growing pains"

O8)

[quakerboy]

51. These things happen

What will it cost to keep it from happening again? Is there some way to separate the lounge out? Every time DU goes down, I imagine a huge server spike at Icanhascheezburger.com, then some sort of evil chain reaction taking down the rest of the internet, one site at a time, as we all shift to the next down on the interest list.

[DefenseLawyer]

52. I appreciate the effort. No complaints here. n/t

[Doremus]

53. PLEASE make sure it doesn't happen election night. I can't imagine not being able

to log on that night!

PLEASE PLEASE PLEASE PLEASE!!!!!!!!!!

[Fire Walk With Me]

54. This is a really good site in so many ways. Downtime happens.

Know that we love the site and that no one would want to see you beating yourself up for issues beyond your direct control. You asked; they failed, it's alright :)

[opiate69]

55. Thanks, Skinner.. for, well.. everything. You, Sir, rock.

[Cha]

56. Definetly addicted but once

again I'm okay:P

[ann_american2004]

57. Phew!

I thought it was my armpits.

So glad you're back!!! I was suffering from withdrawal


edit: spelling

[Dinger]

58. I Guess It's A Good Thing I Didn't Log On TIll Now LOL

We had some friends over and watched it together. They just went home & I log on and just found this out. Looks like I saved myself some stress:)

[mokawanis]

59. We appreciate all you do

An occasional glitch is bound to happen.

[crazylikafox]

60. Me thinks the Web Hosting Co. is run by a bunch of Republicans...

I'm series...

[high density]

61. Wow, we're really getting these machines humming, eh?

I did not know that power demands were so variable between idle and full load.

[Fleshdancer]

62. Shit happens David

Please don't be so hard on yourself, ok?

:hug:

[b2b55]

64. Thanks, However, Can you try really hard to avoid it happening on election night

[likesmountains 52]

69. Want to donate?

[RoadRage]

65. HOW MUCH DO I PAY TO ACCESS THIS SITE AGAIN??

Ohh.. that's right - it's FREE. No complaining about a free service... and anyone who does is certainly able to go elsewhere to find the best dems on earth (good luck).

No worries Skinner.. we appreciate all you guys do!

[BlooInBloo]

71. +1

[tilsammans]

66. Hey, feces occurs.

We'll get over it.

Thanks again for all you do for us. :patriot: :yourock: :toast:

[Arkana]

67. It's all right, brah--it's gotta be a bitch just getting this place going day-to-day.

I'll live without DU for a half-hour at a time.

[Elidor]

68. .

[Gregorian]

70. It saved some of us from great embarrassment.

Some of us had to actually compose ourselves. Sometimes you just need a time-out.

[BlooInBloo]

72. Dude - if that's the worst thing that happens to us this election cycle...

Then we ain't doing too badly.

[likesmountains 52]

73. Thanks..

[DainBramaged]

74. I know it sounds extremely selfish, but I am angry. Angry because

We sailed through 2004 and 2006, and this being the most important election of MY (and I hope your) lifetime, DU is my lifeLINE to politics. After the election, not so much, now, seriously important. I can't post anymore tonight. I missed the best part of the day here and now I need to get some sleep.

[scarletwoman]

75. The feeling of desperation -- like trying to call the boyfriend you've just had a big fight with

so you can apologize and make up. And the phone just keeps ringing and ringing and ringing... No answer... And you're in tears... Oh GOD! Why won't he answer?!?! I'm just going to KILL myself!!!!! *SOB*

:throwing oneself down on the bed and pounding the pillows:

OMG! He's GOT to pick up the phone!!!1!! I CAN'T STAND IT!!1!!1!1 AAAAAUUUUUUGGGGGGHHHHHHH!!!!1!!!1!!!! :great big heaving sobs:

Dialing again... over and over and over and over and over and over and over and over and over and over and over and over and over and over and over...

I'll never get that hour back... *sniff*

[Quantess]

76. It's okay.... I watched Olbermann & Maddow & Tweety give their .02 post-debate

which was interesting.
:)

[CADEMOCRAT7]

77. Thank you Skinner for DU and for all you do, we understand.

[AllentownJake]

78. Skinner

you never fail us! You got this back online as soon as possible. Your the man.

[DinahMoeHum]

79. Heh, I figured you blew another circuit !! ROFL !!

As long as you can fix it so we don't have this problem on the night of November 4, we'll forgive ya!

:hug:

[chieftain]

80. Any angst stems being separated from the community you

provided a home to. Don't beat yourself up. We love and need you. GOBAMA YEA SKINNER

[Cronus Protagonist]

81. My hosting service always had a hot start generator for losing power

And if a breaker flips, someone was always there to flip it back on. I think you could use a better hosting location.

[Faygo Kid]

82. What do you need? This doesn't just belong to you; it's ours. So we're responsible, too.

If you need to upgrade power, let's do it. What do you need? Ask.

We're all family here - right?

[foo_bar]

113. that's what fundraisers are for, right?

The userbase might be a big happy family but the elephant in the room is decidedly an LLC:

And the owners of an LLC can distribute profits in the manner they see fit.

http://www.powerhomebiz.com/vol136/structure.htm

It's hard to ascertain what the operating expense side of the equation here is, but whatever it is clearly isn't enough. Some transparency might be in order if money is going in without highly visible return; the Google ads alone should pay for enough capacity to not resort to Defcon 5s or whatever, but to take donations, go David Lightman*, and subsequently crash for hour(s) at peak usage means not a lot of bang for the buck. I'm sorry, but it has to be said, at least to the network engineers scratching their head on this thread.

* McKittrick: See that sign up here - up here. "Defcon." That indicates our current defense condition. It should read "Defcon 5," which means peace. It's still on 4 because of that little stunt you pulled. Actually, if we hadn't caught it in time, it might have gone to Defcon 1. You know what that means, David?

David Lightman: No. What does that mean?

McKittrick: World War Three.

http://www.imdb.com/character/ch0012893/quotes

[salguine]

83. I think you're being a little hard on yourself.

[CreekDog]

85. It's not your fault

someone in the lounge posted one LOLCAT too many. :hide:

[Oak2004]

86. A circuit failed: you didn't

I know what it's like to manage a server, and sometimes things happen despite one's best efforts.

Thanks for all the work you do for us.

[BlueJac]

87. We are here now, that is what counts!

Love this place!

[garthranzz]

88. Momentary glitches

Sort of like an Obama hesitation - come back stronger.

DU is still the best.

[Oldtimeralso]

89. defecation occurs!

Hope Just Not Nov 4th
Thanks for your work DU is great!

[Oldtimeralso]

90. delete dupe!

[myrna minx]

91. Thanks for all that you guys do for us. n/t

[OKDem08]

92. thank you for the explanation

and your efforts to resolve the problem

[occe]

93. Thank you for telling us.

[DisgustipatedinCA]

94. I'm a network engineer with a hosting company

We host managed and collocated servers. We run two 20- or 30-amp circuits to each cabinet in a one year old data center (this is the DC for managed hosting). As I said, I'm an engineer, not in sales, so I don't stand to gain any profit, but I'd love to see DU hosted where I work, and you'd have my cell # for 24/7 network emergencies. :)

So how much bandwidth, redundancy, and load balancing do you need?

[occe]

95. Thank you for telling us.

[blue sky at night]

96. I just want you to know....

we harbor no bad feelings...how could we, it just amazes me how dependent I have become on my DU....I am telling you I will go off stark raving mad on Election Night if it happens FOR AN EXTENDED PERIOD! Guess we just have to chill and figure that DU is just getting more and more important all the time: AND THAT IS A GOOD THING!!!

THANKS SKINNER, WE LOVE YA MAN.

[Lisa0825]

97. You always do the best you possibly can for DU

and we really appreciate you. Sometimes shit happens, even when you try to plan ahead for it. As others have said here and in the other thread, if there is anything that can be done to really improve this situation, I would be more than willing to throw some extra bucks in toward a designated purchase/project.

Is this something that having a tech on hand at the very moment would have been good? For instance, if someone had been there just for us, could it have been fixed immediately? I'd donate to pay for overtime to have someone designated to be on call on the premises JUST FOR US on election night if that would help keep us up and running.

[quiet.american]

98. Hey, Skinner, luv ya, but please! -- when Obama wins, gotta share that with my fellow DU'ers!

Please... hope this is sorted before Nov. 4!

[slutticus]

99. Don't sweat it...but i've never pressed "reload" so many times in my life...

:D

[zulchzulu]

100. Sorry you had to deal with that crap...

As one who does some IT where I work, it's a thankless gig when things are f*cked up... I was out seeing the debate and not otherwise glued to DU. I'm glad it's back...

[Jersey Devil]

101. Tell your web host to upgrade from knob and tube wiring

and to keep a hefty supply of Nooblin available at all times!

[WillyT]

102. Just Remember Dr. Frankenstein, er... Skinner...

You created this monster, LOL !!! And IT'S ALIVE !!!



:evilgrin:

Ya know Skinner... now that you mention it... I think it would be cool to do a little history (someday after the election) of DU and its growth, upgrades, and expansions. I bet there are a lot of folk here that would be interested in how you guys got this thing up and going, and what it takes to keep it going.

Just a thought...

BTW - As frustrating as it can be to not connect, I always manage a smile because I know what it must take to bring this sucker down... and that's A WHOLE LOT OF PEOPLE payin attention to the DU!!!

Rock on brothers and sisters !!!

:yourock:

[OnionPatch]

103. I'm just always glad that so many people come here.

So when the site is overloaded, I see that as a good sign.

[RFKHumphreyObama]

104. Skinner

As a former moderator, I can testify to the fact that you give more than 1000% of your time and effort to making this site the best that it can be and nothing is spared in terms of your dedication and commitment to this site. The same goes for EarlG and Elad.

If you have a glitch, you have a glitch. These things happen. It doesn't detract from the excellent quality of this site nor the hard work and dedication you put into this site

[NYC_SKP]

106. Lost Power? I knew it would be a power issue! More Hamsters for the generators!!!

I knew it, I knew it would boil down to a power supply issue.

http://www.democraticunderground.com/discuss/duboard.php?az=show_mesg&forum=132&topic_id=7462892&mesg_id=7462892

:patriot:

[SpookyCat]

107. Meh. I thought it was just overloaded.

Had to reboot after patches and figured it was my own damn fault for getting out! LOL!

:hi:

[Lifelong Protester]

109. I'm with the "How much can we donate?" crowd

If you need to buy some thingamabobs to get this doo-hickey running smoothly, don't be afraid to ask, we can all chip in!
And thanks for sweating it out, I am sure that is NO FUN.

[Solon]

110. You are unforgiven.

:P

[PermanentRevolution]

111. Skinner, you KNOW what happens when you don't feed the hamsters...

They can't run on those wheels all night on an empty stomach!

[Blaq Dem]

112. Please don't scare us again...

Tonight is an important night. We NEED our DU fix.

[IsItJustMe]

114. Damn, you mean there are people that take responsibilty. I respect that. That is what we will see

in an Obama administration.

[BrklynLiberal]

115. Thanks for the explanation. I know you guys will ensure it does not happen on election night...

right?????:cry:

[graywarrior]

116. I thought we just were all channeling horror at McCranky and blew a gasket

[Kalifornia.Kid]

117. YouTube videos loading so slooooooow ...

The video's playing directly on DU play/load so slow.
Is that YouTube's servers ar DU's?

[Cooley Hurd]

118. Hi Skinner! Just out of curiosity, aren't the servers connected to a UPS...

...i.e. uninterruptable power supply? I know UPS's are only good for about 10-15 minutes, but wouldn't that buy enough time for the web hosts' tech to get there and reset the circuit?

Nonetheless, thank you (and elad & EarlG) for all you do!:thumbsup:

[intaglio]

119. I'm starting to feel the need for a tinfoil hat about this

I just hope it isn't some Bush apparatchik ordering you off the "tubes"

[LSK]

120. Skinner, dont be too hard on yourself, it happens

You have provided an excellent service for years and I am sure you will continue to do so.

[tedoll78]

121. Thanks for the heads-up.

I consider the debates to be a dry run for the main event. To be honest, if having this kind of thing happen for the debates is what it takes for us to be truly prepared for The Big Night, then it was worth it.

Thanks again! :hi:

[Raine]

122. THANKS for the info

I thought it was just me. You guys do a great job and "stuff" happens sometimes.

[nolabels]

123. I had no problem with it

Did i mention i work work swing-shift :-)

[hobbit709]

124. Tell them to get a couple of these

[mscuedawg]

125. Well if there weren't so many pissed of Democrats out there ..

;) See the good in the bad...thanks for a great site!!!

[BR_Parkway]

126. Refunds and puppies for everyone!!!

[az chela]

127. I am glad to know that it wasnt done by a hacker

I figured there were too many online and in a way thats a good thing

[TooRaLoo]

128. I missed it. Stuff happens, Skinner! No harm no foul. nt

[Yes We Did]

129. Thanks for keeping us informed.

I can speak for all of us when I say we appreciate how you guys & gals keep us up to date on everything.

[Bucky]

131. Democratic Underground has never failed me. Technology has, but DU is more than wires

Nay, more than electrons even

[dhpgetsit]

132. I just want to say this site flat-out ROCKS!

:headbang: :yourock: :headbang:
Thanks for DU!