I recently had the opportunity to attend a talk by Dr Hannah Fry at the Big Data LDN Conference where she explored what data can and can’t tell us.
She used the example of the impact of data analysis and the Challenger disaster of January 1986. The shuttle explosion was caused by a failure of the O-ring seal in the solid rocket booster, producing a breach of burning gas which caused the fuel tank to break apart, engulfing the shuttle in a fireball before it broke apart. Engineers at NASA were already aware of the problem with the O-ring seal and the effect of low temperatures, however, when assessing the data with NASA management, only launches where incidents had occurred were considered. Flights without incidents were excluded from the analysis because they thought that these flights did not contribute any information about the temperature effect. Had this data been included they would have seen that while there had been 4 launches with incidents when the temperature had been in excess of 18 degrees Celsius, there were no incident-free launches below that temperature. On the day of the launch it was unusually cold (-0.6 degrees Celsius). Consequently, the O-ring failed, the shuttle broke apart 73 seconds after launch, and the lives of all 7 crew members were tragically lost.
The key message was that when analysing data to make a decision “doubt is your ally”. We all use data to make decisions in our everyday lives. When managing a project, you are using data to plan, track and analyse your progress regularly. The ability to make rational and informed decisions is key to the success of your project. You need to be able to weigh up the available options, analyse the right data and assess the risks, and decide on the best course of action. We have a wealth of data and analytics available to us, but we should always question the approach and the data in order to reach the best conclusion.
There are numerous models for strategic decision making but there are some common key steps to follow when making a data-informed decision: What is the real question that you are trying to answer? You need to have clarity on your objective in order to determine the right question to be answered to meet that goal. This helps you to focus on collecting the data that you actually need. In the case of Challenger, not asking the right question resulted in important data being omitted from the analysisIdentify the data sources that will answer your question. Your data sources could be statistical databases, such as the schools' exam results league tables when trying to decide which secondary school to apply to for your child, as parents of year 6 students will have recently experienced. Or it may be qualitative data like the school’s prospectus and virtual tours on their websites. Cleanse and organise the information. Remove any incorrect, duplicate, or corrupted data. But, be cautious if removing irrelevant data, as what may appear to be immaterial could actually have a bearing on the analysis and conclusions.
Analyse the data. Determine the facts, what can be inferred from the information and what can be predicted from the patterns and trends. Be aware of biases. For example, this could be seeing what you want to see rather than what is really there (confirmation bias) or base rate bias; ignoring the underlying probability, unconditioned by prior events. Remember that doubt is your ally, so be critical of the data and ask yourself what are you not seeing? Determine what new information you have learned from the analysis, draw your conclusions and make the decision. Once implemented, monitor and compare the new data with the historical data to evaluate the decision and determine whether the results are as expected. While data is a valuable asset when making a decision, remember that it may not always reveal the whole picture. Do not be afraid to doubt and be critical.