Ep 68 Sayano-Shushenskaya Hydro Electric Power Plant Failure

Engineering News – Gelatinous Animals Inspiring Underwater Vehicle Design (3:30).

This week's engineering failure is the Sayano-Shushenskaya Hydroelectric Power Plant Failure (7:05).

The power plant dam had significant water pressure and other issues (15:15), which ultimately led to the failure of turbine 2 in 2009 (24:20). The formal investigation (30:15) created more questions than answers, but the turbine hall was eventually rebuilt (36:10) and the power plant is still operating today.

Sources:

Engineering News

Sayano-Shushenskaya Hydroelectric Power Plant Failure


Episode Summary

Hi and welcome to Failurology; a podcast about engineering failures. I’m your host, Nicole

And I’m Brian. And we’re both from Calgary, AB.

Thank you again to our Patreon subscribers! For less than the cost of a 4 boxes of Kraft Dinner, you can hear us talk about more interesting engineering failures! It’s inflation free bonus content!

For those of you out there listening, it will seem like we have been recording continuously because we followed our regular podcast schedule. But what you may not know is that we recorded all the episodes over the holidays in November and took 8 weeks off. It was great! Anyways this is our first episode back after that little break. So if we seem a little rusty, that’s why. And if we seem a little awkward, it's not our fault we are engineers.

This week in the goings on of engineering and other things, gelatinous animals are inspiring underwater vehicle design.

  • Nanomia bijuga, which are related to jellyfish, use little jets on their body to push water backwards and move themselves forward. They can control the jets individually or all at once to prioritize speed and efficiency.

  • Researchers at the University of Oregon are using this discovery to build more robust underwater vehicles.

  • The research found that most underwater animals are either fast or efficient, but not both. But the nanomia bijuga can do both and it doesn't even have a central nervous system.

  • They use 10 to 20 of these jets to get around, each acting like an individual unit, with tentacles for food, reproduction and defense. When all of the jets work together, they move quickly, but when working separately they move slower but more steady.

  • Researchers are hoping to apply this method to underwater vehicles to allow them to have a range of functions.

  • If you want to read more about nanomia bijuga and how they get around, check out the link on the web page for this episode at failurology.ca


Now on to this week’s engineering failure; the Sayano-Shushenskaya hydroelectric power plant.

  • Located on the Yenisei River near Sayanogorsk –which is in southern Russia, about in the middle, near the border of NE Mongolia. Side note, Russia has the second most time zones in the world with 11, right behind France with 12. France has a lot of overseas territories from colonial times.

  • The dam has had 5 significant accidents, we’re going to focus on the 2009 one, but we’ll still tell you a little about the others. But before we get into that, we have to talk about the dam, its construction, how it operates, and some of its limitations.

  • The power plant is the 12th largest hydroelectric power plant in the world, at 6,400 MW.

    • At the top of that list is the Three Gorges Dam in China, which we are going to talk about in a future engineering marvel episode. It has a capacity of 22,500 MW. And the Itaipu Dam on the border of Brazil and Paraguay.

    • Also on the list are a number of power plants in China, Brazil, Russia, and others, including the Robert-Bourassa and Churchill Falls plants in Canada and the Grand Coulee and Bath County plants in the US.

  • The power plant is hydro electric, meaning it uses flowing water to spin a turbine and generate power. It includes a dam to hold back and direct the water to the turbine. The dam type is an arch-gravity dam, which is the same style as the Hoover Dam in the US.

  • There are 10 turbines, each with a capacity of 640 MW. They each sit in cavities in the machinery hall, located at the base of the dam. The pictures of the hall remind me of a nuclear power plant, because you just see the tops of the turbines similar to how you just see the tops of the reactors.

  • The dam is 242m tall, 1,066m long, and 25m wide at the top and 105.7m wide at the base.

  • The dam creates a reservoir, called the Sayano-Shushenskoe reservoir, which has a capacity of 31.3km3 of water and surface area of 621km2. For a comparative reference, the surface area of Calgary is 825.3km2, so this reservoir is about 75% of the area of Calgary.

  • Construction began in 1963 and started the first turbine in 1978 and was fully operational by 1985.

  • Fun fact - more than 70% of the generated electrical power goes to 4 aluminum smelters in Siberia. The smelters are used to heat ore and extract aluminum. Important to note that smelters have rapid and unpredictable load changes, which the plant had to accommodate.

  • The dam was federally operated, but was privatized after the collapse of the USSR in 1993. For the younger millennials and zoomers listening, Russia used to be… not Russia, it was called the USSR and it was much larger, and just as terrible, maybe more terrible. Anywho, it's a fascinating point in history that is not at all relevant to engineering failures or this show, but something I recommend reading up on if you are not familiar.

Water Pressure and Other Issues

  • This next part I think will be interesting for the mechanical and civil engineering folks.

  • The dam is said to “safely” withstand earthquakes up to an 8 on the Richter scale. Although it was recorded into the Guiness Book of Records for being the strongest construction of its type, the word “safely” was in quotations, and it's in Russia, so take that for what you will.

  • There are 30 million tons of water pressure applied to the base of the dam. This is over 3 billion kilopascals and over 67 billion PSI. In my work, which is buildings, I rarely deal with more than 300 psi.

    • I think in oil and gas applications, they are seeing much higher pressures than 300 psi, but still nowhere near 67 billion psi. I worked on the preliminary and front end engineering design for a 2160 psi pipeline a number of years ago. It was one of the most complex projects I have ever been a part of.

  • So this 30 million tons of water pressure applied to the base of the dam is supposed to be split with 60% neutralized by the dams weight and 40% carried to rock on the bank.

  • But in 1998, the Russian Emergency Situations Ministry claimed that the “station construction had dangerously changed” and that if pressures continued to increase during spring floods, the dam might not be able to withstand those higher pressures.

    • Based on the location of the dam, I imagine the river it’s located on sees spring runoff as the snow that had accumulated during the winter starts to melt. We have a similar situation in Calgary and it can be tricky to manage, especially if the melt coincides with a rainy season.

    • If you just look at the water management aspect, ideally you would lower the water levels before the melt and then continue to shed water through a spillway to keep levels at a reasonable place, while looking upstream to see what's coming.

    • But from a power generation perspective, any water sent through the spillway is “wasted” energy potential. From this angle, you want to maximize power generation and ideally don’t want to divert any water out the spillway.

    • So, it's a balance. Also this is my very high level understanding, I imagine in reality it's a bit more complicated and takes years of experience and practice to master.

  • The basement of the dam had weakened between original construction in 1978 and 1998 –which is only 20 years and not ideal (this should have lasted way longer for something “earthquake resistant”. The weakened basement meant that the 30 million tons of water pressure wasn't being divided as intended and it’s estimated that most of the water pressure and some of the dam’s weight were actually being applied to the shore rocks.

  • There were also issues with water infiltration or water passing through the dam's concrete. We have seen this as a common occurrence in earthen dam failures and I’m surprised it didn’t lead to failure here. Luckily in 1993, a french company injected resins into the dam and it worked to reduce infiltration. Russian companies have repeated the exercise over the years as needed. The soil under the dam was also injected with resin to mitigate infiltration.

  • In 2007, the dam was audited and it was determined that 85% of all technological equipment needed to be replaced.

  • In addition to the dam’s structural concerns, the spillway was undersized to handle the spring melt. It could handle up to 7,500 m3/s, but was limited to 5,000 m3/s after it experienced extensive damage during previous spring floods. Which takes us to the earlier failures, before the catastrophic one in 2009 that is the main topic of this episode (and I promise we will get to it shortly).

    • In May 1979 spring flood water entered the machine hall, which is where the turbines were located, and flooded the first working turbine. This happened while the dam was still under construction. Which would have been a great time to correct some of its issues.

    • In 1985 the spring flood destroyed 80% of the concrete spillway's bottom plate. It tore apart 50mm (2in) thick anchor bolts and carved 7m, deep into the bedrock.

    • In 1988 another spring flood destroyed the spillway well.

  • So as you can see, the dam has a number of issues. It appears that it either wasn’t as strong as it was supposed to be, or the designers did not account for the heavy spring melt; reality is likely somewhere in between the two. And the spillway was undersized, which we have seen before and seems to be fairly common in dam failures. Maybe not the capacity of the spillway itself, but one issue or another with the spillway leads to the dam being too full and either collapsing or overtopping. How this dam is still standing, I don't know. In fact, it’s still in operation. Pro-tip, don’t move downstream of it, probably not a great idea.

2009 Turbine Failure

  • With that said, on to the pièce de résistance, the 2009 failure. This was surprisingly not the result of spring floods, water infiltration, or water pressure, at least not directly.

    • On August 17, 2009 turbine 2 catastrophically failed, flooding the turbine hall and collapsing a portion of its roof, damaging or destroying 8 of the other turbines (leaving only one undamaged), and killing 75 people.

  • The turbines have a narrow working band of high efficiency regimes. If the ban is exceeded, the turbines vibrate, cause pulsation of water flow and water strokes and the turbines degrade over time.

  • Turbine 2 had had issues pretty much since it was installed. And in the early 80s there were problems with seals, shaft vibrations, and bearings.

  • In late 2000, they rebuilt the turbine and found cracks 12mm deep and up to 130mm long at the runner which is the part of the turbine that catches water and transfers that rotational force to the generator. These were repaired. And similar degradation was found again in 2005 and repaired.

  • In early 2009, 5-7 months before the failure, turbine 2 was undergoing more repairs and a modernization. It was the only turbine to have an electro-hydraulic regulator of the rotational speed. Also repaired were the turbine blades which were welded to correct cracks and cavities. The runner was not properly balanced after these repairs, which anyone dealing with an unbalanced rotating component knows this is bad news bears.

  • The vibrations that occurred as a result of the unbalanced runner were within specifications at the time, but not ideal for long term use. As the turbine remained in use, the vibrations exceeded specifications in early July and continued to get worse.

  • Between August 16 and 17th, 2009, the turbine vibrations increased substantially and the load was increased and reduced several times, as well as many unsuccessful attempts to stop the turbine.

  • On August 17, the plant's general director was off site to celebrate his anniversary and greet guests, and no one else at the plant had the authority or wanted to make decisions about the turbine and so it was left operating with high vibrations.

  • At the time of the accident, turbine 2’s capacity was about 475 MW and water consumption was about 256 m3/s. The vibration was 4x more than any of the other turbines and greatly exceeded specifications.

  • The turbine was 29 years and 10 months old at the time, with an expected working life from the manufacturer of 30 years.

  • At 8:13am local time on August 17, there was a loud bang from turbine 2. The cover shot up and the 920 ton rotor shot out of its seat. There was nothing to hold back the water and it sprouted from the turbine cavity into the machinery hall, flooding it and all the rooms below it.

  • The power plant went into alarm and output dropped to zero leading to widespread blackouts.

  • The water gates to the turbines were manually closed, which took 25 minutes and was the fastest speed allowed for the operation.

  • An emergency diesel generator was started 3 hours and 19 minutes after the turbine failure to open the spillway gates.

  • One of the survivors said this about the accident:

    • "...I was standing upstairs when I heard some sort of growing noise, then I saw the corrugated turbine cover rise and stand on end. Then I saw the rotor rising from underneath it. It was spinning. I could not believe my eyes. It rose about three meters. Rocks and pieces of metal went flying; we started to dodge them... At that point the corrugated cover was nearly at roof level, and the roof itself had been destroyed... I made a mental calculation: the water is rising, 380 cubic meters per second, so I took to my heels and ran for the turbine 10. I thought that I wouldn't make it. I climbed higher, stopped, looked down, and saw everything getting destroyed, water coming in, people trying to swim... I thought: someone must urgently shut the gates to stop the water, manually... Manually, because there was no power, none of the protection systems had worked…"

Investigation

  • The formal report, released in October 2009 by the Federal Environmental, Technological and Atomic Supervisory Service notes the cause as a result of fatigue damage of the mountings of turbine 2 caused by turbine vibrations. 6 nuts were missing from the bolts securing the cover at the time of the accident. And of the 49 bolts recovered during the investigation, 41 had fatigue cracks and 8 of them had fatigue damage exceeding 90% of the cross sectional area.

  • The report noted that turbine 2, which was controlling the output of the plant; i.e. it would ramp as required to meet the required capacity while the other running turbines operated at a constant output. As mentioned earlier, the plant serviced large aluminum smelters in the area which had rapid and unpredictable load changes that the plant needed to adapt to very quickly. Because turbine 2 was the turbine that was regulating capacity, its output was continually fluctuating and at the time of the accident was in the non-recommended powerband, leading to higher vibrations.

  • Also noted, there was a fire at a different power station that broke communications and automatic driving systems for other power plants in the region. This other plant would typically regulate output for the region, but with it offline, this responsibility was shifted to the Sayano plant.

  • There seems to be a disagreement between the report and the company who designed the automated safety system as to why the turbine water gates didn’t close automatically.

  • The report was removed a few months after it was published and we weren’t able to find a copy, but did find several summaries of the report contents.

  • One article agreed with the fatigue failure of the cover as one of the causes and theorized that a generator short circuit could have been to blame for failure of turbine 2.

  • Other articles suggested that water hammer pressure lifted the turbine cover.

  • Another article that provides an alternate hypothesis of the cause of turbine 2 failure.

    • I thought that a continuous flow of water would pass through the turbine when it was running and a set of gears would control how much of the rotational energy would transfer to the generator. But in reality, there are gates on the turbine called wicket gates that control the volume of water passing through. When the load dropped, the gates closed suddenly, stopping water while the turbine was still spinning, creating what is known as water column separation. This creates a vacuum downstream which then collapses the tubes and as the water equalizes and comes back up the tubes, it collides with the underside of the turbine.

    • This article points out that the formal report from Oct 2009 didn’t address the failure of other turbines at the station (other than to say they flooded), or explain the source of the upward force needed to blow off the cover. It just said that the cover on turbine 2 failed and caused everything else, which seems unlikely.

Aftermath

  • It took about a week to pump water out of the machinery hall and another 4 days to complete the rescue mission.

  • As we mentioned, 9 of the 10 turbines were destroyed.

    • Turbine 6: Flooded

    • Turbine 5: Flooding and electrical damage

    • Turbines 3 and 4: Moderate electrical and mechanical damage. Some damage to the concrete structures around them.

    • Turbines 1, 8, and 10: Severe electrical and mechanical damage. Some damage to the concrete structures around them.

    • Turbines 7 and 9: Completely destroyed, with extreme damage to the concrete structures around them.

    • Turbine 2: Destroyed completely, including the concrete structures around it.

  • Power was fully restored to the area 2 days later on August 19, 2009.

  • An oil spill happened as a result of the accident, releasing 40 tons of transformer oil up to 80 km downstream of the dam.

  • The Russian government paid $31,600 USD to each of the victim’s families and $3,100 USD to each survivor and the power plants owner doubled that. The owner also bought housing for 13 families with underage children, and created programs to help with school and higher education. There was also a program to rebuild the main settlement where power plant workers lived.

  • As of November 2014, the renovations and repairs were completed and the power plant is operational. Only turbines 5 and 6 were repaired in place, the rest were dismantled and rebuilt in a factory or replaced entirely. The estimated repair cost was $1.3 billion USD.

  • By 2017, new control and safety equipment was installed at the plant.

So there you have it, the Sayano-Shushenskaya power plant failure in 2009. An underdesigned structure, erratic load conditions, and operating the system outside of ideal conditions led to catastrophic failure of nearly all of the 10 turbines and a two day blackout.

For photos, sources and an episode summary from this week’s episode head to Failurology.ca. If you’re enjoying what you’re hearing, please rate, review and subscribe to Failurology, so more people can find us. If you want to chat with us, our Twitter handle is @failurology, you can email us thefailurologypodcast@gmail.com, you can connect with us on Linked In or you can message us on our Patreon page. Check out the show notes for links to all of these. Thanks, everyone for listening. And tune in to the next episode where we will talk about the Fernie Arena Ammonia leak in 2017.

Bye everyone, talk soon!