Nu.nl (the most popular Dutch news site) heads: “Warmest October 22nd ever”. At the same time, the comments speak about the insignificance of the headline as the 23rd of October 2003, for instance, the Netherlands saw a record cold temperature and we had snow. Is something really going on now as suggested by the heading, or is this just ballyhoo? Of course you also see this happening in organizations. We proudly announce something has been significantly improved but if we look closer, we see that the data that is being compared has been picked very selectively. Or after a few weeks or months, it turns out that the so-called improvements were apparently not so sustainable. Most companies lack the basic competency of interpreting data it seems.

Interpretation of Results Data

It should be one of the key competencies of a manager: the correct interpretation of data about process outcomes. As based upon this interpretation, managers and management teams draw their conclusions on the current state of their business and their processes, evaluate the effects of their countermeasures and make sure that adequate actions will be taken (see also my earlier blog post about PDSA).

Still, it is surprising to observe that so few organizations actually understand how to interpret data on process outcomes. Most still work with simple comparisons between actual and budget, or between actual in this period and in the same period last year. Just have a look at the Financial Times or the typical press release of a listed company and you’ll see what I mean. Subsequently, journalists, analysts, but also managers and project leaders alike are quick to write down their (often suggestive) headlines, respectively draw their (often wrong) conclusions; with all that this entails.

Statistical Process Control (SPC)

To support managers in interpreting data in a more useful way and to be able to draw the correct conclusions, the already almost one hundred years old control chart from statistical process control (SPC) is at their disposal. I sometimes see them being used for specific quality characteristics of a product, or at the incoming quality control, but unfortunately hardly ever applied to more managerial data. Even though it can help managers to distinguish between true signals and the ever-present process “noise”.

Using the control chart, it is relatively easy for any manager to distinguish between the routine, random and predictable variation inherent to any process, and the sporadic, exceptional variation with an often clear and assignable cause.

So, is Something Actually Happening to our Temperature?

Let’s return to our example: the temperature. Coincidentally, in the training on SPC that I — next to many other themes — regularly use while supporting clients with their continual improvement initiatives, I also use this example to illustrate the basic concepts of the control chart and its use, before deploying it with the teams.

The source of the data is the yearly, average temperature as published by the Royal Netherlands Meteorological Institute (known as the KNMI) since 1901, and in the place of “de Bilt” where the KNMI is located. The data is shown below in a so called X-bar chart for individual values (without showing the typically accompanying mR-chart) and with the natural process (or control) limits that have been adjusted when non-random signals were detected according to the rules in SPC.

 

 

So what do we see? Based upon the control chart we can conclude that indeed there have been several changes in our temperature over the past 110 years that can not be explained as being only the result of random, so called chance cause variation. When we look at the natural process limits since 1901 (that represent the so called “voice-of-the-process”) we can see that in the past 110 years they have been slowly adjusted upward by some 15%.

The headline on nu.nl shouldn’t tell a manager that much, the control chart, on the contrary, all the more! So tell me, how can you tell your process outcomes and results are not the result of mere chance cause variation, but actually the result of effective countermeasures that your organization, your teams and you yourself deployed?