Death by PowerPoint: the slides that killed seven people

The Rocketry Forum

Help Support The Rocketry Forum:

This site may earn a commission from merchant affiliate links, including eBay, Amazon, and others.

Winston

Lorenzo von Matterhorn
Joined
Jan 31, 2009
Messages
9,560
Reaction score
1,748
Something not mentioned in the article below but which I've read elsewhere is that NASA had no good rescue options anyway which was likely a very powerful influence on their mindset. They wanted to believe everything would come out OK.

April 15, 2019
Death by PowerPoint: the slides that killed seven people

https://mcdreeamiemusings.com/new-blog/2019/4/13/gsux1h6bnt8lqjd7w2t2mtvfg81uhx

[Images of the relevant PowerPoint slides are in the article]

The questions to answer included a very simple one: Why, given that the foam strike had occurred at a force massively out of test conditions had NASA proceeded with re-entry?

Edward Tufte, a Professor at Yale University and expert in communication reviewed the slideshow the Boeing engineers had given NASA, in particular the above slide. His findings were tragically profound.
Firstly. the slide had a misleadingly reassuring title claiming that test data pointed to the tile being able to withstand the foam strike. This was not the case but the presence of the title, centred in the largest font makes this seem the salient, summary point of this slide. This helped Boeing’s message be lost almost immediately.

Secondly, the slide contains four different bullet points with no explanation of what they mean. This means that interpretation is left up to the reader. Is number 1 the main bullet point? Do the bullet points become less important or more? It’s not helped that there’s a change in font sizes as well. In all with bullet points and indents six levels of hierarchy were created. This allowed NASA managers to imply a hierarchy of importance in their head: the writing lower down and in smaller font was ignored. Actually, this had been where the contradictory (and most important) information was placed.

Thirdly, there is a huge amount of text, more than 100 words or figures on one screen. Two words, ‘SOFI’ and ‘ramp’ both mean the same thing: the foam. Vague terms are used. Sufficient is used once, significant or significantly, five times with little or no quantifiable data. As a result this left a lot open to audience interpretation. How much is significant? Is it statistical significance you mean or something else?

Finally the single most important fact, that the foam strike had occurred at forces massively out of test conditions, is hidden at the very bottom. Twelve little words which the audience would have had to wade through more than 100 to get to. If they even managed to keep reading to that point. In the middle it does say that it is possible for the foam to damage the tile. This is in the smallest font, lost.

Edward Tufte’s full report makes for fascinating reading. Since being released in 1987 PowerPoint has grown exponentially to the point where it is now estimated than thirty million PowerPoint presentations are made every day. Yet, PowerPoint is blamed by academics for killing critical thought. Amazon’s CEO Jeff Bezos has banned it from meetings.

Typing text on a screen and reading it out loud does not count as teaching. An audience reading text off the screen does not count as learning. Imagine if the engineers had put up a slide with just: “foam strike more than 600 times bigger than test data.” Maybe NASA would have listened. Maybe they wouldn’t have attempted re-entry. Next time you’re asked to give a talk remember Columbia. Don’t just jump to your laptop and write out slides of text. Think about your message. Don’t let that message be lost amongst text. Death by PowerPoint is a real thing. Sometimes literally.


IMG_1361.jpeg
 
The real tragedy of this was that even if they were fully aware of the tile damage, there was little to no options for rescue. My recollection is that they would have been out of supplies critical to survival before a rescue mission could reach them. I try to never assume that any plan is failure proof. The old saying is that you never see the one that gets you. I guess if we see it coming, we can avoid it. Always a good idea to prepare a 'Plan B'

Jim
 
The author mentioned in this article, Edward Tufte, wrote an excellent book called "Visual Explanations" along this same vein. One of his case histories is the infamous Thiokol/NASA discussion of the SRB o-rings the night before the ill-fated flight of Challenger STS-51L. He points out how the impact of low temperatures on the sealing capability of the o-rings was missed because of the way the data was presented.
 
The other side to this, something I've seen a few times, is that the power point presentation is all encompassing, and all informative. that is, to the ones who are immediately connected to the item / idea / intent. The Boeing people who did the presentation know what they are dealing with, and how they want to represent it, and made some of the noted "mistakes" as they didn't deem them 'critical' as they know what they are talking about & have gone over the points & issues many times themselves & with their immediate people. they all know, so the fact that it's 600x over is just a fact.

They forgot. It's easy to forget that you're' going to talk to people who are fresh, don't know, haven't' a clue or the back-ground preamble to what you're presenting.

"Know your audience" It's easy to make an assumption that it'll be a room full of engineers.. and assume they know the same level you know of the issues.. And, I think this is where they failed. They made some assumptions of their intended audience. And maybe the audience was expecting something different that what was given.

I've worked with a few, who you have to pull information out of them, and even then it's hard to understand.. And some who look at you funny when you ask a question, to get details, clarification..
 
Typing text on a screen and reading it out loud does not count as teaching. An audience reading text off the screen does not count as learning. Imagine if the engineers had put up a slide with just: “foam strike more than 600 times bigger than test data.” Maybe NASA would have listened. Maybe they wouldn’t have attempted re-entry. Next time you’re asked to give a talk remember Columbia. Don’t just jump to your laptop and write out slides of text. Think about your message.

My personal rule when making PowerPoint presentations is that the text can be no smaller than 16 point, and there must be a photo on each page. That keeps the amount of text one the page down to a manageable amount so people can read the text then start paying attention to me again. I also do not use complete sentences in the bulleted lists so that I can't just read the slide. The slide is there more to remind me of the critical points than to be a script

The author mentioned in this article, Edward Tufte, wrote an excellent book called "Visual Explanations" along this same vein. One of his case histories is the infamous Thiokol/NASA discussion of the SRB o-rings the night before the ill-fated flight of Challenger STS-51L. He points out how the impact of low temperatures on the sealing capability of the o-rings was missed because of the way the data was presented.

Tufte's book "The Graphical Display of Quantitative Information" is also really good for producing useful graphs and charts. A probably pirated copy is posted here, but it is well worth buying if you produce a lot of charts and graphs.

And as long as we're on the topic of PowerPoint, there's the classic from 20 years ago, The Gettysburg Powerpoint Presentation. You never knew how bad Abe Lincoln could be until he's in front of a PPT.
 
FYI. I happen to know the engineer that worked 3 yrs with a team to determine the "foam" flaw. Turned out it was quite simple.
Due to EPA's new regulation the manufacturer of the foam changed the agent that makes the liquid "foam" & did not tell NASA.
This was done to meet the ozone not getting destroyed by the gas released during process when it was sprayed on the fuel tank.
This changed the chemical structure making it stiff and not near as adhesive as the old formula.
When finally discovered [3yrs later] shuttle went back into serve with a new foam...rest is history.

I never read anywhere where this info was made public & she could not talk about it till after she retired 2 yrs ago.
 
the point of the article isn't to debate the shuttle's loss, but to point out how powerpoint is being used ineffectively, and how messages are being given that are out of context with the reality of the situation at hand.

in the shuttle case, a simple slide "we have no test data for this situation" would have been more appropriate and sparked real discussion.
Instead, the author chose to confuse the issue with a myriad of non-sequitur data that lead to a very tragic conclusion.

When I worked in corporate, I sat thru many of the second and received complaints when I presented the first. What sparks this? Pride? the inability / unwillingness to state "I don't know?"
 
' snip
When I worked in corporate, I sat thru many of the second and received complaints when I presented the first. What sparks this? Pride? the inability / unwillingness to state "I don't know?"
I frequently have discussions with my bosses who are afraid to tell their bosses "we don't know" I keep saying that is perfectly fine to say, and follow up with "if you will fund a test program, we will get those answers for you." It has been a long time coming, but they have started saying that, and corp has now started saying "Ok". So much wasted time due to managerial egos rather than just stating a simple truth and moving on...*sigh*
 
We are taught to know, that "I don't know" is not an answer, and one that must be corrected. SO, enter into the corporate world, and you're now encouraged to say " you don't know" when you don't know. it's a hard lesson to learn. Doubly so, when you're a NASA / Boeing engineer and you "don't know". the optics can be bad, and the media would spin it till the cows come home.

I've been taught in my working life, and I teach ot to new employees: You have three answers: Yes, No, I don't know.
 
One of my professors once said "If the audience can't absorb and retain 90% of a slide's intent after 20-30 seconds of viewing, it is a bad slide."
 
When doing things that are hard, or that have never been done before, in my experience the 'I don't know' response can usually be followed but 'But I can find out, and this is what it will take'
is an acceptable tack with most management people, and while they may not be technical themselves, they at least understand the process. If not, then you're really screwed.

-Mike
 
https://www.nasa.gov/columbia/caib/html/VOL2.html
2nd volume of the CAIB report.

Sure the presentation of results on that one slide are an easy target for Monday morning quarter backing and second guessing, but to focus on that one thing as responsible for the deaths of seven astronauts is misguided and frankly highly offensive.

The blogger is a doctor and teacher and I am sure will attract more readers to his blog, as well as Tufte will sell more of his books and seminars, with such sensational twisting of events to market themselves, but a larger group of highly educated, experienced, and respected people served on the CAIB and made a more comprehensive assessment of the culture and events that led to this tragic event, in two volumes, and it focused on many contributing factors beyond this one power point slide. This kind of sensational misappropriation of misinformation really does not serve the greater good.
 
Very huge tragedy. We should learn from this mistake. I personally have never seen anyone die from powerpoint, but quite a few have had concussions after falling asleep and faceplanting on the desk.

I think there is an enormous amount of blame-shifting, CYA-ing, and Tufte self-promotion going on in blaming everything on PowerPoint slide. NASA screwed up, royally, in failing to ask the right questions to properly evaluate the risks from the foam strike.

But nowhere do you see naming and shaming of the NASA leads who were responsible for evaluating the tile damage risks. I don't recall reading of anyone getting fired at NASA for gross negligence.

Blaming failure of risk management on a PowerPoint slide, is disingenuous.
Yes, that particular PPT slide is a hot mess, but not outside the norm of something an engineering team would throw together. Marketing and sales people know how to communicate more clearly and to the point, most engineers don't.

Other competent engineers know that, and dig deeper to find the answers to the important questions.
Not at NASA.


[...]to focus on that one thing as responsible for the deaths of seven astronauts is misguided and frankly highly offensive.
[...]This kind of sensational misappropriation of misinformation really does not serve the greater good.

Indeed.

a
 
https://www.nasa.gov/columbia/caib/html/VOL2.html
2nd volume of the CAIB report.

Sure the presentation of results on that one slide are an easy target for Monday morning quarter backing and second guessing, but to focus on that one thing as responsible for the deaths of seven astronauts is misguided and frankly highly offensive.

The blogger is a doctor and teacher and I am sure will attract more readers to his blog, as well as Tufte will sell more of his books and seminars, with such sensational twisting of events to market themselves, but a larger group of highly educated, experienced, and respected people served on the CAIB and made a more comprehensive assessment of the culture and events that led to this tragic event, in two volumes, and it focused on many contributing factors beyond this one power point slide. This kind of sensational misappropriation of misinformation really does not serve the greater good.


I think there is an enormous amount of blame-shifting, CYA-ing, and Tufte self-promotion going on in blaming everything on PowerPoint slide. NASA screwed up, royally, in failing to ask the right questions to properly evaluate the risks from the foam strike.

But nowhere do you see naming and shaming of the NASA leads who were responsible for evaluating the tile damage risks. I don't recall reading of anyone getting fired at NASA for gross negligence.

Blaming failure of risk management on a PowerPoint slide, is disingenuous.
Yes, that particular PPT slide is a hot mess, but not outside the norm of something an engineering team would throw together. Marketing and sales people know how to communicate more clearly and to the point, most engineers don't.

Other competent engineers know that, and dig deeper to find the answers to the important questions.
Not at NASA.




Indeed.

a

If you read the paper Tufte wrote about the slides, I don't think either of you'd make those statements. He is an acknowledged expert in the presentation of technical information, not some casual writer. He's a fellow of the American Statistical Association, and professor emeritus of political science, statistics, and computer science at Yale University. I challenge you to read the actual paper:

https://www.inf.ed.ac.uk/teaching/courses/pi/2016_2017/phil/tufte-powerpoint.pdf
and then tell me that he's missing the mark that the Powerpoint slides were instrumental in the poor decision making. Focusing on the information that helped lead to the decision is not misguided or highly offensive. The whole point is to figure out why and how the decision was reached. If poor communication of highly technical information lead to the wrong decision, that sure seems like a good place to start. The very reason they failed to ask the right questions is very likely due to how the information was presented on those slides.

He's really blaming the usage of a tool like Powerpoint by engineers when trying to communicate incredibly important information in a literal life or death situation. Indeed, the Board (not Tufte) that investigated the accident came to this conclusion:

As information gets passed up an organization hierarchy, from people who do analysis to mid-level managers to high-level leadership, key explanations and supporting information is filtered out. In this context, it is easy to understand how a senior manager might read this PowerPoint slide and not realize that it addresses a life-threatening situation.
At many points during its investigation, the Board was surprised to receive similar presentation slides from NASA officials in place of technical reports. The Board views the endemic use of PowerPoint briefing slides instead of technical papers as an illustration of the problematic methods of technical communication at NASA.

Sure, Tufte may have books to sell, but dismissing him because of that is a classic ad hominem attack. You'd also have to dismiss the conclusions of the entire board. afadeev's suggestion that NASA engineers are not competent is the worst kind of disingenuous misinformation. In the sentence before that statement, he claims most engineers don't know how to communicate. He certainly can't claim that other engineering organizations don't use PowerPoint. But somehow only NASA has incompetent engineers? Exactly the kind of dangerous and disingenuous decision you are railing against.


Tony
 
Last edited:
The real tragedy of this was that even if they were fully aware of the tile damage, there was little to no options for rescue. My recollection is that they would have been out of supplies critical to survival before a rescue mission could reach them. I try to never assume that any plan is failure proof. The old saying is that you never see the one that gets you. I guess if we see it coming, we can avoid it. Always a good idea to prepare a 'Plan B'

Jim
Even worse was that the foam loss area was identified well before the incident and the it was tested and determined that foam didn't even need to be present in that area.
 
Yes, read the whole paper. Fun and informative. (I've reduced it to two bullet items!)

When I was teaching (EE), students expected PowerPoint slides and would panic when I'd just start talking and writing on the blackboard/whiteboard.
 
"

The audacious rescue plan that might have saved space shuttle Columbia

The untold story of the rescue mission that could have been NASA's finest hour."​


https://arstechnica.com/science/201...that-might-have-saved-space-shuttle-columbia/
caib-rdv-lg2-640x426.png


The ENGINEERS were deeply concerned. They used back-channels to arrange for a military SpySat to take images (as had secretly done for STS-1). It was a done deal, was gonna happen!

But a higher-up executive, Linda Ham (senior shuttle manager), got p*ssed about the engineers doing that behind her back, and CANCELLED that. She was never held accountable for that.

Powerpoint did not kill the crew. SHE did! At least as regards a possible rescue.

She deserves the "Lawrence Mulloy" award (for killing Challenger's crew).

Mulloy was a former MSFC shuttle program manager (also not held accountable afterwards). Who on the night before the Challenger accident, upset about Thiokol not recommending the launch due to concerns about the O-rings in cold. His damnable quote: "My God Thiokol, when you you want us to launch, April?".

Totally flipping around 180 degrees, NASA's decades of "Prove it is SAFE", but that night, Thiokol was asked by NASA/Mulloy to "Prove it is NOT safe", which they did not have adequate data for proof (they had "trends" data, from a previous mission launched at about 50 degrees where one O-ring was compromised & burned, and they blamed the low temperature for that. But had not done actual engineering testing in such cold temperatures. Which BTW - was about 17 degrees on the right SRB before liftoff, extra-cold due to the Liquid Hydrogen tank and wind from left to right).

Mulloy's pressure ended up causing Thiokol to "put on their management hats", and not wear their ENGINEERING hats, in approving the launch.

And by January 2003 NASA had slipped RIGHT BACK into the "Prove it is NOT SAFE" mindset. Linda Ham yanked the rug out from under the engineers who had arranged to PROVE exactly that, with the already arranged spysat imaging, that she cancelled.
 
Last edited:
Our rules of thumb from church are still good to apply more broadly:

  • No more than three or four lines of text
  • Text must be easily readable (and/or sung) by an 80 year old in the back row of the sanctuary.
 
Last edited:
If you read the paper Tufte wrote about the slides, I don't think either of you'd make those statements. He is an acknowledged expert in the presentation of technical information, not some casual writer. He's a fellow of the American Statistical Association, and professor emeritus of political science, statistics, and computer science at Yale University. I challenge you to read the actual paper:

https://www.inf.ed.ac.uk/teaching/courses/pi/2016_2017/phil/tufte-powerpoint.pdf
and then tell me that he's missing the mark that the Powerpoint slides were instrumental in the poor decision making. Focusing on the information that helped lead to the decision is not misguided or highly offensive. The whole point is to figure out why and how the decision was reached. If poor communication of highly technical information lead to the wrong decision, that sure seems like a good place to start.The very reason they failed to ask the right questions is very likely due to how the information was presented on those slides.

He's really blaming the usage of a tool like Powerpoint by engineers when trying to communicate incredibly important information in a literal life or death situation. Indeed, the Board (not Tufte) that investigated the accident came to this conclusion:



Sure, Tufte may have books to sell, but dismissing him because of that is a classic ad hominem attack. You'd also have to dismiss the conclusions of the entire board. afadeev's suggestion that NASA engineers are not competent is the worst kind of disingenuous misinformation. In the sentence before that statement, he claims most engineers don't know how to communicate. He certainly can't claim that other engineering organizations don't use PowerPoint. But somehow only NASA has incompetent engineers? Exactly the kind of dangerous and disingenuous decision you are railing against.


Tony
I took Dr. Tufte's class in 2008, it was very interesting and that's why I remember it so well. During his lectures he covered both shuttle disasters and how information display had adversely impacted decision making. He was seriously down on .ppt at communicating complex information, and for complex topics that required some deep thought he recommended actually "gasp" writing (and reading) technical reports. His recommended format, get this, was an old NASA report guide "Clarity in Technical Reporting" published at Langley back 1964. Here is a link: https://archive.org/details/nasa_techdoc_19640016507
 
Yes, read the whole paper. Fun and informative. (I've reduced it to two bullet items!)

When I was teaching (EE), students expected PowerPoint slides and would panic when I'd just start talking and writing on the blackboard/whiteboard.
John - I am absolutely not surprised you had that experience with students. I've lived in the DC area since 1997, since retiring from the USN I've worked in a variety of DoD and DHS acquisition positions. Since moving here I have watched attention spans in the work environment get shorter and shorter and shorter with increasing levels of emphasis on PowerPoint slides combined with an unwillingness to read anything longer than a couple of paragraphs. It was so bad on my last job, where folks pretty much had migrated to teams chat (which is absolutely not good for any somewhat complex information) and I'd have to send a chat to remind them to read an email sent at a certain time with the subject line because they needed to make a decision.

I've come to the conclusion that its a collective combination of intellectual laziness (reading is generally more active mentally than listening to someone briefing briefing bullet points) and actual brain rewiring/plasticity because of "smart" phones and internet browsers. I dont know what the answer is because its now "organizational .ppt culture" that is dumbing everyone down, and senior leaders are just a guilty of it as the rank and file.

1668745864354.png
 
"

The audacious rescue plan that might have saved space shuttle Columbia

The untold story of the rescue mission that could have been NASA's finest hour."​


https://arstechnica.com/science/201...that-might-have-saved-space-shuttle-columbia/
caib-rdv-lg2-640x426.png


The ENGINEERS were deeply concerned. They used back-channels to arrange for a military SpySat to take images (as had secretly done for STS-1). It was a done deal, was gonna happen!

But a higher-up executive, Linda Ham (senior shuttle manager), got p*ssed about the engineers doing that behind her back, and CANCELLED that. She was never held accountable for that.

Powerpoint did not kill the crew. SHE did! At least as regards a possible rescue.

She deserves the "Lawrence Mulloy" award (for killing Challenger's crew).

Mulloy was a former MSFC shuttle program manager (also not held accountable afterwards). Who on the night before the Challenger accident, upset about Thiokol not recommending the launch due to concerns about the O-rings in cold. His damnable quote: "My God Thiokol, when you you want us to launch, April?".

Totally flipping around 180 degrees, NASA's decades of "Prove it is SAFE", but that night, Thiokol was asked by NASA/Mulloy to "Prove it is NOT safe", which they did not have adequate data for proof (they had "trends" data, from a previous mission launched at about 50 degrees where one O-ring was compromised & burned, and they blamed the low temperature for that. But had not done actual engineering testing in such cold temperatures. Which BTW - was about 17 degrees on the right SRB before liftoff, extra-cold due to the Liquid Hydrogen tank and wind from left to right).

Mulloy's pressure ended up causing Thiokol to "put on their management hats", and not wear their ENGINEERING hats, in approving the launch.

And by January 2003 NASA had slipped RIGHT BACK into the "Prove it is NOT SAFE" mindset. Linda Ham yanked the rug out from under the engineers who had arranged to PROVE exactly that, with the already arranged spysat imaging, that she cancelled.
"Truth, Lies, and O-Rings: Inside the Space Shuttle Challenger Disaster" by Allan J McDonald is a really good book written by the SRB man himself (he was the one that blew the whistle at the hearing). Aside from the engineering aspects, he really gets into the politics and decision making foul ups. Without McDonald speaking up it would have been a total white wash. Its a long book that also covers the design defects, the politics (including his job rescue by some senators and him being put in charge of the SRB fixes) and what they did to get the booster design corrected and the shuttle program back on track. The one part of the book that gave me the creeps was, IIRC, his describing the internal inspection on his hands and knees of the entire length of the AP core of the redesigned SRB on the test stand before it was fired.
 
"Truth, Lies, and O-Rings: The one part of the book that gave me the creeps was, IIRC, his describing the internal inspection on his hands and knees of the entire length of the AP core of the redesigned SRB on the test stand before it was fired.
Kewl! I'd love to do that!
 
"Truth, Lies, and O-Rings: Inside the Space Shuttle Challenger Disaster" by Allan J McDonald is a really good book written by the SRB man himself (he was the one that blew the whistle at the hearing). Aside from the engineering aspects, he really gets into the politics and decision making foul ups. Without McDonald speaking up it would have been a total white wash. Its a long book that also covers the design defects, the politics (including his job rescue by some senators and him being put in charge of the SRB fixes) and what they did to get the booster design corrected and the shuttle program back on track. The one part of the book that gave me the creeps was, IIRC, his describing the internal inspection on his hands and knees of the entire length of the AP core of the redesigned SRB on the test stand before it was fired.
I had a (now deceased) friend who worked on the failure analysis. I'm tempted to buy it just to see if he got a mention!
 
Back
Top