The title of this post is a line from a Wall Street Journal (WSJ) article “Do our Gadgets Really Threaten Planes? by Daniel Simons and Christopher Chabris, Sept 7 2012. The article poses a question if the ban on phones and such is based on fact or fear. It is a good question because many of us have forgot to turn our stuff off during a flight a time or two.
The article is making a good point, in my opinion, because I do not believe the electronic devices do disrupt planes. Another fact is that the Mythbusters tested it and showed no effect on their test plane even when they increased the phone transmit power 10x.
I write this post because of a subplot in the article. Down near the middle the author states that fear may be driving the ban more than facts. This is where they state that humans are overzealous cause detectors. Two quote the authors “When two events occur close in time, an one plausibly might have caused the other, we tend to assume it did.” This is a trait that most lean six sigma practitioners face in many projects. Humans like to see connectivity and causation when only a correlation may exist. One of my mentors called this “False Knowledge.” People hypothesize a belief after they do something different and the result of a process changes. Now the person must choose between a belief of random correlation or human causation. Most will choose human causation.
As a statistician and Lean Six Sigma MBB, I have gained a deep understanding of variation and see it in everything. I have found that most people do not fully comprehend the nature of random variation. Most people consider variation as “regular” or “common,” but I know that there are random extreme events. Things break, fail, and stop for random extreme events, while normal people assume there is a cause, I more tend to first consider it as a random extreme event. I believe the assumption of a cause for all extreme events leads to much of our false knowledge and many bad decisions.
For lean six sigma projects, I think the greatest successes (for permanent change) are made when you are required to prove the non-random nature of an event prior to implementing changes. This is what the statistical testing provides. The null hypothesis is randomness, the alternate is causation.
An unintended consequence of my belief of greater randomness is that I will recommend that no actions be taken after many seemingly extreme events, because there is not enough evidence to prove causation by events that are found to be correlated during root cause analysis efforts. Plus, a recommendation to take no action goes against human nature. If I have a client that must take a corrective action, because of culture or a contract requirement, I recommend that they take an action that is expected to be non-impactful. These actions are things like re-writing a document to be for readable, change a training program, or brief the workforces. Just do not change the process from a historically good setup because of a single random extreme event.
I read an article by Carl Bialik, “The numbers guy” at the Wall Street Journal titled “Victory by total Medals or Just Gold?” that I wish I would have written.
In this article, Mr. Bailik discusses different method to report the winner of the Olympics. By total medals is the most common method with the next most common being the total number of Gold Medals (both methods show the US winning). How about adjusting for the size of the country? We could use the country GDP divided by the number of medals to consider the economic sizer per medal (shows Russia winning). Another is to scale the count by the population by dividing the population by the number of medals (shows Great Britain winning). This method would identify the country with the highest fraction of winners. Which is right?
The last section of the article asks if we should weight the benefit of the medal count to the chance a person has to win one. A top swimmer or a track star can win gold in up to three events with a single skill (two distances and a medley relay) while a basketball team or a boxer can only win one. If we use a weighting based on this concept, the US is hurt because 58% of all their medals came from swimming and the athletics, while china would do better because they did not have more than 10 medals in any segment of the sports.
I see these same issues in lean six sigma and in our IEE consulting work. Organizations adjust the operational definitions of their metrics so that the organization looks good without any consideration of the honesty. We see it most in the use of index metrics where the performance is reported as a variance to a goal. Negotiate a better goal and you are performing better.
In business, my recommendation is always plot actual numbers in your primary reporting of performance. If you choose to adjust the performance by a factor to account for volume or complexity, do it along with the true data reporting. This will lead to a better and more honest performance report.
A Wall Street Journal article on 12 Apr 2011 by Carrie Lukas, “There is no Male-Female Wage Gap” documents a clear case of how statistics can be used to prove just about anything.
Ms Lukas notes that April 12th is equal pay day, but you are not hearing much about the wage gap and how women are underpaid. It is said that women get paid 77% of what men make, which must be a result of a bias. But is it? In the article, a number of facts are shared that show that the 77% value is a result of mixing non-homogeneous populations. A few examples follow.
The recession hit men more than women (1% higher unemployment for men) because men prefer the jobs that have been more susceptible to losses right now: manual jobs, outside jobs, physical jobs… Women prefer jobs that have been more insulated, such as teaching and healthcare.
The big difference is found in another way to look at the data. The US Department of Labor reports that women average 8.01 hours of work a day compared to men who average 8.75 hours a day. Would you pay more to those who average 9% more work hours? Probably.
When you compare wages under similar conditions (what we would do in the Lean Six Sigma world) such as single, childless, urban workers between 22 and 30, you find that women get 8% more pay than men.
So what is true? Well, all of the statistics are true. But what is causal to the wage differences? Is it gender? Probably not. Is it job preferences? Possibly. Causation in social data is quite difficult to confirm, but it can be done. In this case, the last comparison, when other factors are held constant, shows that there is no bias against women.
Be careful in your projects not to accept any summary statistical comparison without fully understanding the issue of causation and the agenda behind the information provider. As many people have said, data without context are useless, if not deceptive.
An editorial in the December 21 Wall Street Journal discusses the impact of a tax rate change in Oregon. The point of this posting is that predicting outcomes is not easy.
In Oregon’s case, the state voted to increase the marginal tax rate on the wealthy by ~2%. They predicted the financial benefit to the state by estimating the impact of the higher tax rate on the reported income and capital gains from the current year. This is how they sold it to the voters. The following financial year showed that the total revenue from the segment of the population that had the increased tax rate decreased significantly, -27%. How could this happen?
First, this change happened during the recession, across the change. This revenue drop occured while the economy was recovering.
It turns out that there was a steep drop in the number of people paying the higher tax rate (10,000 less or a drop of 26%) So where did they go? It has been assumed that they adjusted their financials and decisions to ensure that they ended up in a lower income bracket. This is done by delaying or eliminating the sale of assets, which was confirmed by the reduction in the capital gains in Oregon by 43% (a drop of $1.5B).
I believe the message is about the predictability of a complex system when you make a change. To model the system without any change in behavior due to the change is ignorant. In an improvement project, we standardize the work processes and end up seeing impacts well beyond the removal of a defect or reduction in time. Complex systems have so many interdependencies, that you must truly pilot test the changes before they are fully understood.