Statistics and data science aim to extract understanding from data and guide decision-making. However, before applying any analytical tools, we need absolute clarity about what we want to know or accomplish. Ambiguous objectives inevitably lead to mistaken conclusions and flawed actions. This book investigates the deeper challenges of formulating clear questions and matching analytical methods to those questions - issues that apply as much to elementary statistical tools as to sophisticated techniques. Rather than focusing on standard statistical misuses or data provenance issues, this work examines the critical step of ensuring your analysis actually answers the question you mean to ask.
Drawing from collaborative work across finance, medicine, government, manufacturing, defence, and other fields, the book deliberately emphasises basic and familiar tools so the fundamental issues are accessible to everyone. Following John Tukey's insight about the simplest problems of data analysis, the most detailed discussions centre on averages and comparisons between distributions, though the principles apply with even greater force to advanced methods that fewer people fully understand.
Key Features:
- Focusses on question formulation rather than computational techniques, addressing the step that precedes all successful data analysis
- Emphasises basic statistical tools (averages, comparisons) to make fundamental challenges visible to all practitioners
- Contains 130 text boxes presenting essential ideas in non-technical language, creating a "two-in-one" book accessible to both mathematical and non-mathematical readers
- Provides real-world examples drawn from diverse fields including finance, healthcare, government, manufacturing, and defence
- Offers a deep-dive analysis of a specific comparison method to illustrate the care required for precise statistical reasoning
- Presents a progression from general principles through detailed mathematical exploration to practical applications across various analytical scenarios
This book serves as an essential guide for statisticians, data scientists, researchers, and anyone who uses data to make decisions. Whether you're a practitioner seeking to improve your analytical approach or a student learning to think critically about statistical questions, this work will help you use data analytical tools more effectively and avoid the costly mistakes that arise from asking the wrong questions of your data.
David Hand is Emeritus Professor of Mathematics and Senior Research Investigator at Imperial College, London, where he formerly held the Chair in Statistics. He has consulted widely, in particular including serving as an advisor to the pharmaceutical industry and working extensively with the consumer banking sector, addressing the challenges of credit scoring, fraud detection, and optimal decisions. For eight years he served as Chief Scientific Advisor to Winton Capital Management. He spent eight years on the Board of the UK Statistical Authority, and served twice as President of the Royal Statistical Society. He is an Honorary Fellow of the Institute of Actuaries, a Fellow of the Institute of Mathematics and its Applications, and a Fellow of the British Academy.
He launched the journal Statistics and Computing in 1991, and has published 300 scientific papers and 32 books, including Principles of Data Mining, The Improbability Principle, Dark Data, Measurement Theory and Practice, and The Wellbeing of Nations. He has received various awards, including the IEEE-ICDM Research Contributions Award, the Royal Statistical Society's Guy Medal in Silver, the George Box medal for Business and Industrial Statistics, and the International Federation of Classification Societies Research Medal.
