What BP and Lehman Bros. have in common

We humans are pretty useless at assessing risk. Most of the time we get away with it. But then one day, surprise! You're dead.

It should have been good news, but it didn’t feel that way. Fred Bartlit Jr., the lead investigator for the presidential panel looking into BP’s Deepwater Horizon disaster in the Gulf of Mexico, revealed last week that he has found no evidence that the company sacrificed safety for profit.

That will come as a surprise to many, and not a pleasant one. Deep down, we prefer the scandal storyline. We like our villains, our perp walks and our show trials, all the better to turn our fear into anger, find scapegoats, and put things in the past. Bartlit’s report pushes us back toward fear and confusion. If it was just a series of deadly mistakes, it could happen again. The complicated, boring truth of Deepwater Horizon exposes, yet again, a critical weakness in human nature. When it comes to assessing and managing risk, we are next to useless. Our minds play tricks on us, we misread the probabilities and engage in willful self-delusion. Then, one day, surprise! You’re dead.

Nancy Levenson is a professor of aeronautics at MIT and is one of the world’s leading experts on industrial disasters — from space shuttle explosions and plane crashes to collapses and oil spills. At a conference at MIT last month, she was asked about the BP explosion. “They’re all basically the same accident every time,” she sighed.

Indeed, when Levenson begins talking about the dynamics of man-made catastrophe, the consistent profile is amazing. She could just as easily be talking about the sub-prime mortgage meltdown that triggered the ongoing financial crisis. Catastrophes are always the same, and they all look like this: many people working within a highly complex system, all of those people unconsciously reinforcing each other’s optimistic biases, focusing on the opportunities and the rewards, while becoming gradually blinded to mounting risks.

Under these conditions, warning signs go unheeded. As one of Bartlit’s investigators noted, a key pressure test was failed in the hours before the BP blowout, yet work continued. “Why [did] these experienced men on that rig talk themselves into believing that this was a good test that indicated well integrity?” he wonders. “None of them wanted to die or jeopardize their safety. The question is why.”

Levenson knows why. It’s the same reason red flags in the U.S. housing market were ignored and explained away throughout early 2008: human beings have a talent for not seeing things that they don’t want to see. She calls it the “culture of denial” that runs rampant in big operations where large sums of money are on the line — like space programs, oil drilling companies and financial institutions.

It’s a culture that focuses on legal compliance rather than safety: i.e., I may be driving toward a cliff, but I’m not speeding, and I’m wearing a seat belt, so I must be safe. It’s a culture that views risk as a constant, not a variable. So, for instance, it fails to grasp that while you might be a safe driver, after 48 hours without sleep, three beers, on bald tires in a rainstorm, your risk profile looks pretty different. Finally, it confuses long periods without incident as indications of safety. Think of all the companies that put up signs proudly tracking how long it’s been since their last serious accident. This mindset breeds complacency, and an arrogance that is incredibly tough to pierce.

That arrogance remains the defining public image of BP and most of the Wall Street firms that collapsed in 2008. When former BP CEO Tony Hayward said the world overreacted to the spill because the volume of oil released was “tiny in relation to the total water volume of the Ocean,” it was more than just a dumb thing to say. With 11 dead, almost $40 billion lost and untold damage to the ecosystem, he still got the risk calculation wrong. When former Lehman CEO Dick Fuld testified to Congress, in effect, that the bank’s collapse was just a big misunderstanding, it was his culture of denial still talking. They were so enamoured with their ingenuity, even utter failure couldn’t shake their confidence.

That’s the really sobering implication of Bartlit’s report: of course nobody at BP sacrificed safety for profit. The choice never occurred to them. They believed they were being safe. Many of them still believe that.

All this matters because our world is getting infinitely more complex, and the costs of our mistakes are getting bigger. Science is tinkering with the very building blocks of life. Financial markets are so interconnected and intricate that no one can credibly predict the effect of major shocks. Of course, we can’t eliminate risk, nor should we try to. But our business culture has long valued boldness over wisdom, and equated caution with timidity. That is a mindset we can no longer afford. The antidote to our culture of denial is not more fear, but more humility. A lot more.