power outage in San Francisco data center

[rob46]

356 Main Official press release on July 24th

http://365main.net/status_update.html

 

[rbulsara]

This data center was featured in one of high profile conferences. I did not like the arragnement even then. It has no UPSs but Engine-motor-gensets with high speed clutches. I beleive.

 

[itsmoked]

It boggles the mind that someone would even bother to put a data center on top of the San Andreas Fault in a city that was leveled previously by 'nature'.

Keith Cress
Flamin Systems, Inc.-

http://www.flaminsystems.com

 

[ScottyUK]

As the problem is being co-investigated by Hitec, I guess this involves the type of rotary UPS favoured by the high end financial centres in the UK, other vendors being Anton Piller and Eurodiesel. I'm watching this with interest: I've not heard of a multiple failure like this before - maintenance or lack thereof must be a strong candidate for the root cause. These things are normally pretty reliable. The energy store is in the form of a high mass rotor rotating at super-synchronous speed which provides power to the load as the storage rotor slows down. It is an interesting design of machine in its own right.


----------------------------------

Sometimes I only open my mouth to swap feet...

 

[dpc]

These rotary UPS are actually pretty popular with high-tech firms in the US (wafer plants and fabs) It will interesting to learn what went wrong.

 

[gentech1975]

I installed some of these "rotary UPS" systems back in the late '80s for the Canadian Forces at different locations. They were favoured due to the better sine wave from the dedicated generator than from the static UPSs of the day.
They work great and are good ride-through for power fluctuations.
The big question is where were they sensing the power at? Do they have multiple feeders?
Do they switch between feeders rather than start generators?
Do they have a main generator control system for staged starting of the units?
Lots and lots of questions.
Regards.

 

[alehman]

As did rbulsara, I saw a presentation on this facility. As I recall, it has some standard gensets in addition to the Hitec units. The article mentions that the generators "failed to start". That would seem to imply maybe it wasn't the Hitec units. Hard to say.

That together with the problem at CDC in Atlanta, it's been a bad couple of weeks for standby power.

 

[rbulsara]

I must clarify that I was not implying that systems with conventional UPS never fail. Any equipment can see a catastrophic failure. It is a matter of incorporating proper isolation and redundancy. Also through system integration testing or lack thereof could contribute to the end results.

 

[slavag]

Dear Rob.
Are you have some new information?

Could you please send some links about "rotary UPS", it's intresting. I'm never see this and I would like learn about this system.
Thanks.
Slava

 

[ScottyUK]

Hi Slava,

Have a poke around on this web site:

http://www.euro-diesel.com/total_technical.html

Eurodiesel are one of a handful of manufacturers who make this general type of UPS. I like them because they have great fault clearing ability and can start awkward loads like motors without massively oversizing the system. They have their downside too: they are relatively maintenance-heavy, only available in large sizes (I think the smallest is about 250kW) and are expensive per installed kW compared to a static unit of similar rating.


----------------------------------

Sometimes I only open my mouth to swap feet...

 

[MikeHalloran]

With three meetings a day, they may take a while to solve the problem...



Mike Halloran
Pembroke Pines, FL, USA

 

[alehman]

365 Main continues to update their web site. It is greatly appreciated that they are willing to make their process public.

"During over 100 start/stop tests late Saturday night on Unit 1, the investigation team was able to simulate failure. "

Seems like Unit 1 might be about worn about by now.