Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations JAE on being selected by the Eng-Tips community for having the most helpful posts in the forums last week. Way to Go!
Manhattan Partial Blackout 2
- Thread starter Mbrooke
- Start date
- Thread starter
- #2
Governor promising an investigation:
I agree with him and he is right when he says the system is specifically designed to prevent this. In fact the system can handle multiple failures (equipment failure + breaker failure) or failure of any two elements one after the other before load shedding or voltage reduction need to be implemented let alone a complete blackout.
There is also the fact relaying failed during Sandy at 13st st, and at Astoria 6 months ago. Several fires in the past few years at Farragut. Con Ed has some profound protective relaying issue if they are cropping up so often in such magnitude.
I agree with him and he is right when he says the system is specifically designed to prevent this. In fact the system can handle multiple failures (equipment failure + breaker failure) or failure of any two elements one after the other before load shedding or voltage reduction need to be implemented let alone a complete blackout.
There is also the fact relaying failed during Sandy at 13st st, and at Astoria 6 months ago. Several fires in the past few years at Farragut. Con Ed has some profound protective relaying issue if they are cropping up so often in such magnitude.
- Thread starter
- #3
For the discussion, page 15 on the details of West 49th.
The eleventh 138kv goes to Vernon substation and doesn't serve any load directly that I know of.
The eleventh 138kv goes to Vernon substation and doesn't serve any load directly that I know of.
- Thread starter
- #4
- Thread starter
- #6
HamburgerHelper
Electrical
This is what I would bet it is. No one was providing oversight to the engineering because the utility was very short staffed or it was assumed the outside consultant would check their own work to make sure that it was always 100% correct. Problems like this accumulate and since faults are generally infrequent, it can take awhile for someone to notice that the relaying is not set correctly. Whatever the technical problem, in my opinion the causes is really a process and resource problem.
It takes a rare event to verify that your system is coordinated for a rare event. That isn't 100% true and some companies like Omicron have systems that take a system model and bombard the relay with a whole bunch of cases but I don't know of anyone that does that.
------------------------------------------------------------------------------------------
If you can't explain it to a six year old, you don't understand it yourself.
It takes a rare event to verify that your system is coordinated for a rare event. That isn't 100% true and some companies like Omicron have systems that take a system model and bombard the relay with a whole bunch of cases but I don't know of anyone that does that.
------------------------------------------------------------------------------------------
If you can't explain it to a six year old, you don't understand it yourself.
The authority rules out the following:
1) Overload
2) Ciber attack
3) Terrorism.
This site is a 40-year-old first generation of GIS with a large history of issues associated with leaking SF6 gas.
ConEd need to evaluate if this type of old installation with multiple feeders with oil-filled pipe-type cables should be adequate to maintain the reliability of the system expected for this application.
Below is an excerpt with public domain information for the 49th St Substation:
1) Overload
2) Ciber attack
3) Terrorism.
This site is a 40-year-old first generation of GIS with a large history of issues associated with leaking SF6 gas.
ConEd need to evaluate if this type of old installation with multiple feeders with oil-filled pipe-type cables should be adequate to maintain the reliability of the system expected for this application.
Below is an excerpt with public domain information for the 49th St Substation:
- Thread starter
- #9
Yup. That station has me concerned, more so than some older air insulated stations on Con Ed's territory.
Anyway, an update:
I hate to speculate, but could this be a system wide problem involving IEC-61850? This is really no different then the relaying failure in December. Twice primary and backup has failed in 6 months. Probability says this must be rampant.
Anyway, an update:
I hate to speculate, but could this be a system wide problem involving IEC-61850? This is really no different then the relaying failure in December. Twice primary and backup has failed in 6 months. Probability says this must be rampant.
- Thread starter
- #11
Don't think either of these stations have flooded that I am aware of. But yes, lack of maintenance could certainly be it.
Thinking this out it looks like not only did the relaying at 65th fail, but also West 49th in that 49th didn't just cut power to 65th but also 50th and 42nd which are not tied into the 5 138kv feeders serving 46th.
In theory a primary and secondary relaying failure should simply clear a 13.8kv bus section and one transformers (if a circuit switcher is present) at 46th on over current. Worse case scenario two transformers and one 138kv cable get tripped. The remaining load can be served via 4 cables and 8 transformers no problem.
But ok, all 5 138kv feeders trip. How do the reaming 6 coming out of 49th also trip?
There is a lot that went wrong here. If any of this involves microprocessor relaying the industry needs to know.
Thinking this out it looks like not only did the relaying at 65th fail, but also West 49th in that 49th didn't just cut power to 65th but also 50th and 42nd which are not tied into the 5 138kv feeders serving 46th.
In theory a primary and secondary relaying failure should simply clear a 13.8kv bus section and one transformers (if a circuit switcher is present) at 46th on over current. Worse case scenario two transformers and one 138kv cable get tripped. The remaining load can be served via 4 cables and 8 transformers no problem.
But ok, all 5 138kv feeders trip. How do the reaming 6 coming out of 49th also trip?
There is a lot that went wrong here. If any of this involves microprocessor relaying the industry needs to know.
Concerning that there was such a massive failure of the protection system. All relays are digital? Complex relaying schemes are often not adequately tested for functionality - even if the protective element detects the fault, relay programming/configuration issues results in no breaker trip signal. Pointless speculation at this stage, I suppose.
- Thread starter
- #13
I agree that it is somewhat pointless in that we do not have all the details on the relaying, but at the same time such a grand failure deserves a conversation. I know for a fact that if both primary and secondary 13.8kv relaying went at a 115kv-13.8kv substation around here it may (key word) clear the whole substation but not the entire 345-115kv bulk station supplying it.
- Thread starter
- #15
- Thread starter
- #16
Question.
I'm piecing together the original blackout. From all the News reports and amateur video it looks like the Times Square network never went down.
Jump to 10:16 "It is a tale of two Time Squares"
However according to reliability reports, it appears that Time Square Network is fed via 50th st.
Anyone know if it was transferred over to another substation or has feeders from other networks? 50th also has the Hudson network, but it looks to have gone down. I'm beyond words confused.
I'm piecing together the original blackout. From all the News reports and amateur video it looks like the Times Square network never went down.
Jump to 10:16 "It is a tale of two Time Squares"
However according to reliability reports, it appears that Time Square Network is fed via 50th st.
Anyone know if it was transferred over to another substation or has feeders from other networks? 50th also has the Hudson network, but it looks to have gone down. I'm beyond words confused.
- Thread starter
- #17
Interestingly, or inversely, the Empire State building was supposed to be lit but was not. Looks like Con Ed mixed the networks/supply substation around a bit.
Here is Murray Hill supplying Fashion and Empire at the time of Hurricane Sandy.
Networks supplied by Murray Hill in 2015. The number of transformers mentioned in the table are those with PQ monitoring, not the total number in the substation or network.
Murray Hill fed by Vernon at the time of Sandy.
The blue line represents 5 138kv feeders originating from Vernon supplying East 40th st and Murray Hill substation.
If wondering Queens bridge takes 138kv and steps it down to 69kv with 7 69kv feeders supplying East 63rd st.
The Vernon double ring bus, which interconnects generation to the 138kv transmission system in Queens.
38M01, 38M03, 38M04, 38M05, 38M06 feed out to 40th st and Murray Hill.
My guess for the numbering (missing 38M02) is that at one point 7 69kv cables fed out but were upgraded to 5 138kv cables.
If I have it right 38M72 is no longer "capped off" (alive on back-feed) but connected through a phase angle regulator were some power can be exchanged between West 49th 138kv and Vernon.
Here is Murray Hill supplying Fashion and Empire at the time of Hurricane Sandy.
Networks supplied by Murray Hill in 2015. The number of transformers mentioned in the table are those with PQ monitoring, not the total number in the substation or network.
Murray Hill fed by Vernon at the time of Sandy.
The blue line represents 5 138kv feeders originating from Vernon supplying East 40th st and Murray Hill substation.
If wondering Queens bridge takes 138kv and steps it down to 69kv with 7 69kv feeders supplying East 63rd st.
The Vernon double ring bus, which interconnects generation to the 138kv transmission system in Queens.
38M01, 38M03, 38M04, 38M05, 38M06 feed out to 40th st and Murray Hill.
My guess for the numbering (missing 38M02) is that at one point 7 69kv cables fed out but were upgraded to 5 138kv cables.
If I have it right 38M72 is no longer "capped off" (alive on back-feed) but connected through a phase angle regulator were some power can be exchanged between West 49th 138kv and Vernon.
- Thread starter
- #18
FWIW, somewhere in the late 2000s 38M72 was connected to Vernon via a phase angle regulator.
Vernon substation showing the 38M72 phase angle regulator (quadrature phase shifting booster) at the top right.
A 2006 picture of Vernon without the phase angle regulator present on 38M72.
Con Edison did the right thing and replaced the maroon colored breakers with SF6 breakers.
The 3 round cylinder with radiators on some outgoing lines I believe to be iron core current limiting reactors.
However, the flood risk is hellacious with the list of essential equipment disquiet being voluminous.
Vernon substation showing the 38M72 phase angle regulator (quadrature phase shifting booster) at the top right.
A 2006 picture of Vernon without the phase angle regulator present on 38M72.
Con Edison did the right thing and replaced the maroon colored breakers with SF6 breakers.
The 3 round cylinder with radiators on some outgoing lines I believe to be iron core current limiting reactors.
However, the flood risk is hellacious with the list of essential equipment disquiet being voluminous.
- Thread starter
- #19
Update:
Was this the same thing that happened in Astoria? IEC-61850 failure?
Was this the same thing that happened in Astoria? IEC-61850 failure?
- Thread starter
- #20
As I said, I think this is wide spread:
Some relay manufacturer be it ABB or someone else has a hoard of defective equipment.
Some relay manufacturer be it ABB or someone else has a hoard of defective equipment.
- Status
Similar threads
- Question
- Locked
- Question
- Locked
- Question
- Locked
- Question