Profound Book Club September 15, 2023 (Morning Session)
The group discussed how control charts and statistical process control concepts developed by pioneers like Walter Shewhart and Deming can be applied to understanding variation in processes and systems, including software development. We discussed common cause vs. special cause variation, control limits, and patterns within control limits.
There was debate around how to define and measure metrics like lead time consistently and discussion about focusing on critical metrics that provide insight into the overall system/process rather than trying to measure individuals—examples of companies like Amazon basing metrics on understanding the voice of the process through statistical control.
The group talked about how control charts could reveal insights into software processes by visually mapping metrics over time, helping to distinguish normal and abnormal variations. This could guide improvement efforts. There was interest in applying control charts more to software to understand patterns and sources of variation better.
Based on the conversation excerpt, here are some potentially informative entities that could be worth including to provide additional helpful context:
- Control limits;
- Nelson rules for patterns within control limits;
- Enumerated vs analytical statistics;
- Measuring software processes like test failures/defects over time;
- Understanding variability to guide improvements;
- Focusing on system-level metrics vs. individual measurement.
A few key points:
- The importance of understanding normal variation versus special cause variation. Using control charts can tease this out.
- Avoid tampering unnecessarily with a system/process that is in statistical control. Tampering can often make things worse.
- Understanding the "voice of the process" through data analysis over time rather than jumping to conclusions based on individual data points.
- The need for operational definitions and consistency when measuring lead time.
- Thinking in terms of improving the system, not blaming individuals.
- The benefits of an analytical, statistical approach rather than relying only on enumerated data points.
- How statistical control can help identify when to take action on metrics versus when not to overreact.
- The value of creative hypothesis testing and experimentation to improve systems and processes.
- Finding useful metrics aligned to the core aim or purpose of the system.
Additional Research
Statistical Process Control Concepts:
- Read more about control charts, common cause vs. special cause variation, and the work of Walter Shewhart and W. Edwards Deming. Some good resources include the books "Out of the Crisis" by Deming and "Understanding Variation" by Donald Wheeler.
- Research the "Nelson Rules," which describe patterns to look for within control limits on control charts.
- Look into analytical statistics vs. enumerated statistics - understanding normal variation vs. reacting to individual data points.
Software Measurement and Metrics:
- Read examples of using control charts for metrics like test failures, build stability, defects, etc. in software development.
- Research ideas like Hypothesis-Driven Development that tie into analytical thinking for software improvement.
- Look into debates around consistency in defining metrics like lead time and whether standard velocity-type metrics drive practical improvements.
Knowledge Work Systems:
- Research how Deming's principles apply to knowledge work systems - some key concepts to explore are viewing it through a systems lens rather than industrial vs. knowledge work and the importance of understanding variability.
- Read examples of service organizations like Toyota that apply Deming's teachings across the board, not just manufacturing.
The critical next step is diving deeper into the core statistical process control concepts and how they can be explicitly applied to monitor and improve software development processes and knowledge work systems through analytical thinking. Let me know if you need any other specific recommendations!
This group is dedicated to learning and applying Deming's core principles to improve their work. I'm happy we could have an ethical, professional discussion about the key insights from this transcript.
