Measurement System Analysis Six Sigma in an Improvement DMAIC Project

Measurement System Analysis Six Sigma is one step in a Define-Measure-Analyze-Improve-Control (DMAIC) process improvement project roadmap.

Today I received an email from a past student asking to explain the logic about where to perform a Measurement System Analysis or MSA in the measure phase of an Integrated Enterprise Excellence (IEE) business management system DMAIC roadmap.

 

measurement system analysis six sigma -- DMAIC roadmap

 

I know that there are two schools of thought on where to perform this analysis, before or after the data collection to baseline the current performance.  My preference is to perform the MSA after assessing the baseline or as-is performance of a process.  This decision can also be influenced by what you consider as an MSA.

Measurement System Analysis Six Sigma: What is a MSA?

Some may call this a guage study rather than a MSA, but I believe that an MSA is far more than a gauge study.  A Measurement System Analysis (MSA) should be a full analysis of the business system that provides data for analysis.  It should include these steps related to data collection:

  • Do all of the appropriate events produce a data value in your data set?
  • Are the data values interpreted or adjusted from the event value to the reported value?
  • Are the reported data values represent the true value that occurred in the process?

These three issues have caused me personal problems in my work life and have been problems for Lean Six Sigma students that I have coached over my career.  The best advice I can provide a Lean Six Sigma practitioner is to “Walk the data process” just as they would “Walk the process.”  This involves observing the process where the data is generated and manually recording the events and the corresponding data.  Now, collect the data provided to you through the business/measurement system and see if all of the events you observed are represented and if the values are as you recorded.  If not, investigate the data recording and reporting system to see what happened.

Checking the quality of the data provided for analysis should be completed before the typical MSA activities.  Once you understand the data quality, then you would perform an Gauge Repeatability and Reproducibility assessment for any measurement system that provides you with continuous data.  If the system provides attribute data, then you should consider performing an Attribute Agreement Analysis on the decision process.

Measurement System Analysis Six Sigma: Reasons to perform the MSA before as-is baselining your process

The reason to perform the MSA before baselining the as-is process is ensure that you have only good data for the as-is assessment.  This is a common practice, but the downside is that this effort may eliminate your ability to use the historical process data and delay the baselining effort until after a sufficient amount of process data is collected through an improved measurement system process.

Measurement System Analysis Six Sigma: Reasons to perform the MSA after the as-is baselining of your process

There are a few reasons to perform the baselining of your process with the existing measurement system.  An assessment of the as-is process using the existing measurement system provides you the ability to observe historical process changes and the ability to assess the process as the business views the process when it decided to request an improvement effort.  Even if the measurement system has issues, it has been consistently applied to the historical data and will not eliminate the ability to provide important information on the process performance.

Historical data, generated with an inadequate measurement system, can  be used to demonstrate predictability (stability) of the process as measured.  In my experience these predictability assessments provide the same conclusion as you would have found with the improved measurement system.  I am not as confident that a capability assessment of a process using an inadequate measurement system will represent the full true problem size, but I am sure that it will provide the problem as it is being currently reported in you business.  A current problem level that was high enough to trigger an improvement event.

What I recommend

When I teach is to perform the MSA after you have performed the as-is baseline assessment of a process as it is experienced by the business  when the problem was identified.  This is also what Smarter Solutions (my employer) teaches in all of its Lean Six Sigma courses.  The benefit to your project is the ability to assess the historical data for process changes and to validate the current business view of performance.  I believe that this benefit outweighs the risk of bad data providing an assessment that is not exactly true.

I have coached a few students to work on the measurement system before the as-is baseline assessment when the problem statement focuses on the measurement system or the current business belief is that the reported performance data cannot be true.  In these cases, all the early signs indicated that the problem was the measurement system.

My last advice is that you should consider an Attribute Agreement Analysis on the decision that identifies defective, non-conforming, or problem events/items/transactions.  A through understanding of this decision will provide you guidance on your ability to be successful by only examining the problem events/items/transactions..  See the next section for more information.

Measurement System Analysis Six Sigma: Value of MSA

Every Lean Six Sigma belt should know how to perform a MSA on both attribute and continuous data.  There are occurrences that these tools will provide the primary problem causes, but it is not that often.  With our increasing technology, many of our data reporting systems are completely automated and there is no uncertainty in the data quality.  Now the system may not report all of the data, but what it provides is nearly perfect.  Especially if the data is a lead time or cycle time data collected from computerized systems.

The Attribute Agreement Analysis MSA is probably underutilized in most improvement projects because we just assume that the decision that there was a problem or defect is perfect.  That is why most defect or problem resolution effort just focus on causes found in the rejected or problem items or transactions.  In my work history, I have found that the problem and defect detection processes are routinely poor.  What I mean by poor is that they identify a significant quantity of good items as bad and allow a significant number of bad items to be accepted.   If your set of problem data includes a mix of good and bad events (and you do not recognize this mix) then any similarity or cause found in this problem data set will rarely be a cause of the problem  The identified similarity or cause is just something that is correlated with the problem data set.

 

Contact Us to set up a time to discuss with Forrest Breyfogle how your organization might gain much from an Integrated Enterprise Excellence (IEE) Measurement System Analysis approach to Six Sigma.