The Weekly Reflektion 41/2022
Risk assessments are used extensively in decision making and it is important one understands the dynamics taking place in the assessment process. Obviously, we would expect the participants in the risk assessment process to be objective, the data accurate and the analysis of the data thorough and unambiguous. In the real work this may not always be the case. There are uncertainties that need to be considered and resolved. In the real world, new information may challenge the established views, create doubt, and change peoples’ minds.
Do you understand the dynamics of your risk assessments and how people react to changing information?
In last week’s Reflektion, week 40/2022 we covered the NASA risk assessment process, the Flight Readiness Review (FRR), that was an important part of the space shuttle launch decision making. We also highlighted the culture in NASA with technical rigour, structured approach, extensive quality control, learning and quick response to challenges. The FRR was a structured process with collaborative and adversarial levels. However, when new information emerged the demands for hard and verified data, the adversarial approach affected the dynamics of how this information was assessed.
On the day before the scheduled launch the predicted ambient temperature was -8O C. No launch had taken place at such a low temperature. NASA requested all their contractors to assess their systems and equipment for the low temperature. With the new information, some of the engineers at Morten Thiokol that produced the Solid Rocket Boosters (SRBs) got cold feet. They were suddenly concerned about the uncertainties related to the ‘O’ rings in SRBs. They collected their data and hastily produced a graph that they believed demonstrated a relationship between ‘O’ ring leaks and ambient temperature, see figure 1. The data only included information on space shuttle flights with confirmed ‘O’ ring leaks. The graph was faxed to NASA and arrived a few minutes before a telecon between NASA and Morten Thiokol. The engineers in Thiokol were not fully in agreement on the potential for a leak, however deferred to the senior engineer and his judgement. The management in Thiokol recommended a delay to the launch and NASA accepted this decision as they could not overrule a launch delay recommendation from their suppliers. NASA however questioned the data presented and criticised the engineers’ judgement related to theirconclusions. In the hard data culture this criticism stung, and the Thiokol management asked for more time to review the data again. Further debate ensued in Thiokol and the management changed their recommendation considering that there was too much uncertainty in the data to contradict the original FRR assessment. When NASA asked whether everyone at Thiokol agreed with this recommendation no one spoke up. The intense discussions did not result in everyone agreeing with the decision. It ended up with the people becoming loyal to the group decision and resigning themselves to the consequences of that decision. ‘Group Think’ is the process and it is a powerful social factor in the dynamics of risk assessments. During the investigation into the disaster astronauts Neil Armstrong and Sally Ride, who were members of the Rogers Commission put together information on the SRB ‘O’ ring leaks for all the previous 24 shuttle flights, see figure 2. Perhaps if the engineers had had more time or had access to Excel the Thiokol decision would have been different.