Multivariate Statistical Process Control with Industrial Applications
ASA-SIAM Series on Statistics and Applied Proba...
316 downloads
2397 Views
28MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Multivariate Statistical Process Control with Industrial Applications
ASA-SIAM Series on Statistics and Applied Probability The ASA-SIAM Series on Statistics and Applied Probability is published jointly by the American Statistical Association and the Society for Industrial and Applied Mathematics. The series consists of a broad spectrum of books on topics in statistics and applied probability. The purpose of the series is to provide inexpensive, quality publications of interest to the intersecting membership of the two societies.
Editorial Board Robert N. Rodriguez SAS Institute, Inc., Editor-in-Chief
Gary C. McDonald General Motors R&D Center
Janet P. Buckingham Southwest Research Institute
Jane F. Pendergast University of Iowa
Richard K. Burdick Arizona State University
Alan M. Polansky Northern Illinois University
James A. Calvin Texas A&M University
Paula Roberson University of Arkansas for Medical Sciences
Katherine Bennett Ensor Rice University
Dale L Zimmerman University of Iowa
Robert L. Mason Southwest Research Institute
Mason, R. L. and Young, J. C., Multivariate Statistical Process Control with Industrial Applications Smith, P. L, A Primer for Sampling Solids, Liquids, and Gases: Based on the Seven Sampling Errors of Pierre Gy Meyer, M. A. and Booker, J. M., Eliciting and Analyzing Expert Judgment: A Practical Guide Latouche, G. and Ramaswami, V., Introduction to Matrix Analytic Methods in Stochastic Modeling Peck, R., Haugh, L., and Goodman, A., Statistical Case Studies: A Collaboration Between Academe and Industry, Student Edition Peck, R., Haugh, L., and Goodman, A., Statistical Case Studies: A Collaboration Between Academe and Industry Barlow, R., Engineering Reliability Czitrom, V. and Spagon, P. D., Statistical Case Studies for Industrial Process Improvement
mULTIVARIATE STATISTICAL
Process Control with
Industrial Applications Robert L Mason Southwest Research Institute San Antonio, Texas
John C. Young McNeese State University Lake Charles, Louisiana InControl Technologies, Inc. Houston, Texas
S1HJTL Society for Industrial and Applied Mathematics Philadelphia, Pennsylvania
ASA American Statistical Association Alexandria, Virginia
Copyright © 2002 by the American Statistical Association and the Society for Industrial and Applied Mathematics. 10987654321 All rights reserved. Printed in the United States of America. No part of this book may be reproduced, stored, or transmitted in any manner without the written permission of the publisher. For information, write to the Society for Industrial and Applied Mathematics, 3600 University City Science Center, Philadelphia, PA 19104-2688. Library of Congress Cataloging-in-Publication Data Mason, Robert L, 1946Multivariate statistical process control with industrial applications / Robert L. Mason, John C. Young. p. cm. - (ASA-SIAM series on statistics and applied probability) Includes bibliographical references and index. ISBN 0-89871-496-6 1. Process control-Statistical methods I. Young, John C., 1942- II. Title. III. Series. TS156.8 .M348 2001 658.5'62-dc21 2001034145
is a registered trademark. Windows and Windows NT are registered trademarks of Microsoft Corporation. QualStat is a trademark of InControl Technologies, Inc. The materials on the CD-ROM are for demonstration only and expire after 90 days of use. These materials are subject to the same copyright restrictions as hardcopy publications. No warranties, expressed or implied, are made by the publisher, authors, and their employers that the materials contained on the CD-ROM are free of error. You are responsible for reading, understanding, and adhering to the licensing terms and conditions for each software program contained on the CD-ROM. By using this CD-ROM, you agree not to hold any vendor or SIAM responsible, or liable, for any problems that arise from use of a vendor's software.
^&o ^Saimen an& Qj^am
This page intentionally left blank
Contents Preface
xi
1 Introduction to the T2 Statistic 1.1 Introduction 1.2 Univariate Control Procedures 1.3 Multivariate Control Procedures 1.4 Characteristics of a Multivariate Control Procedure 1.5 Summary
1 3 4 5 9 11
2 Basic Concepts about the T2 Statistic 2.1 Introduction 2.2 Statistical Distance 2.3 T2 and Multivariate Normality 2.4 Student t versus Retelling's T2 2.5 Distributional Properties of the T2 2.6 Alternative Covariance Estimators 2.7 Summary 2.8 Appendix: Matrix Algebra Review 2.8.1 Vector and Matrix Notation 2.8.2 Data Matrix 2.8.3 The Inverse Matrix 2.8.4 Symmetric Matrix 2.8.5 Quadratic Form 2.8.6 Wishart Distribution
13 13 13 17 20 22 26 28 28 29 29 30 30 31 32
3 Checking Assumptions for Using a T2 Statistic 3.1 Introduction 3.2 Assessing the Distribution of the T2 3.3 The T2 and Nonnormal Distributions 3.4 The Sampling Distribution of the T2 Statistic 3.5 Validation of the T2 Distribution 3.6 Transforming Observations to Normality 3.7 Distribution-Free Procedures 3.8 Choice of Sample Size
33 33 33 37 37 41 47 48 49
vii
viii
Contents 3.9 3.10 3.11
Discrete Variables Summary Appendix: Confidence Intervals for UCL
50 50 51
4 Construction of Historical Data Set 4.1 Introduction 4.2 Planning 4.3 Preliminary Data 4.4 Data Collection Procedures 4.5 Missing Data 4.6 Functional Form of Variables 4.7 Detecting Collinearities 4.8 Detecting Autocorrelation 4.9 Example of Autocorrelation Detection Techniques 4.10 Summary 4.11 Appendix 4.11.1 Eigenvalues and Eigenvectors 4.11.2 Principal Component Analysis
53 54 55 57 61 62 64 65 68 72 78 78 78 79
5 Charting the T2 Statistic in Phase I 5.1 Introduction 5.2 The Outlier Problem 5.3 Univariate Outlier Detection 5.4 Multivariate Outlier Detection 5.5 Purging Outliers: Unknown Parameter Case 5.5.1 Temperature Example 5.5.2 Transformer Example 5.6 Purging Outliers: Known Parameter Case 5.7 Unknown T2 Distribution 5.8 Summary
81 81 81 82 85 86 86 89 91 92 96
6 Charting the T2 Statistic in Phase II 6.1 Introduction 6.2 Choice of False Alarm Rate 6.3 T2 Charts with Unknown Parameters 6.4 T2 Charts with Known Parameters 6.5 T2 Charts with Subgroup Means 6.6 Interpretive Features of T2 Charting 6.7 Average Run Length (Optional) 6.8 Plotting in Principal Component Space (Optional) 6.9 Summary
97 98 98 100 105 106 108 Ill 115 118
7 Interpretation of T2 Signals for Two Variables 7.1 Introduction 7.2 Orthogonal Decompositions 7.3 The MYT Decomposition
119 119 120 125
Contents 7.4 7.5 7.6 7.7 7.8 7.9 7.10
ix Interpretation of a Signal on a T2 Component Regression Perspective Distribution of the T2 Components Data Example Conditional Probability Functions (Optional) Summary Appendix: Principal Component Form of T2
127 129 131 135 140 142 143
8 Interpretation of T2 Signals for the General Case 8.1 Introduction 8.2 The MYT Decomposition 8.3 Computing the Decomposition Terms 8.4 Properties of the MYT Decomposition 8.5 Locating Signaling Variables 8.6 Interpretation of a Signal on a T2 Component 8.7 Regression Perspective 8.8 Computational Scheme (Optional) 8.9 Case Study 8.10 Summary
147 147 147 149 152 155 157 162 163 165 169
9 Improving the Sensitivity of the T2 Statistic 9.1 Introduction 9.2 Alternative Forms of Conditional Terms 9.3 Improving Sensitivity to Abrupt Process Changes 9.4 Case Study: Steam Turbine 9.4.1 The Control Procedure 9.4.2 Historical Data Set 9.5 Model Creation Using Expert Knowledge 9.6 Model Creation Using Data Exploration 9.7 Improving Sensitivity to Gradual Process Shifts 9.8 Summary
171 172 172 174 175 175 176 180 183 188 191
10 Autocorrelation in T2 Control Charts 10.1 Introduction 10.2 Autocorrelation Patterns in T2 Charts 10.3 Control Procedure for Uniform Decay 10.4 Example of a Uniform Decay Process 10.4.1 Detection of Autocorrelation 10.4.2 Autoregressive Functions 10.4.3 Estimates 10.4.4 Examination of New Observations 10.5 Control Procedure for Stage Decay Processes 10.6 Summary
193 193 194 199 201 202 202 207 209 211 212
11 The T2 Statistic and Batch Processes 11.1 Introduction
213 213
x
Contents 11.2 11.3 11.4 11.5 11.6 11.7 11.8 11.9 11.10
Types of Batch Processes Estimation in Batch Processes Outlier Removal for Category 1 Batch Processes Example: Category 1 Batch Process Outlier Removal for Category 2 Batch Processes Example: Category 2 Batch Process Phase II Operation with Batch Processes Example of Phase II Operation Summary
213 217 219 221 226 226 230 232 234
Appendix. Distribution Tables
237
Bibliography
253
Index
259
Preface Industry continually faces many challenges. Chief among these is the requirement to improve product quality while lowering production costs. In response to this need, much effort has been given to finding new technological tools. One particularly important development has been the advances made in multivariate statistical process control (SPC). Although univariate control procedures are widely used in industry and are likely to be part of a basic industrial training program, they are inadequate when used to control processes that are inherently multivariate. What is needed is a methodology that allows one to monitor the relationships existing among and between the process variables. The T2 statistic provides such a procedure. Unfortunately, the area of multivariate SPC can be confusing and complicated for the practitioner who is unfamiliar with multivariate statistical techniques. Limited help comes from journal articles on the subject, as they usually include only theoretical developments and a limited number of data examples. Thus, the practitioner is not well prepared to face the problems encountered when applying a multivariate procedure to a real process situation. These problems are further compounded by the lack of adequate computer software to do the required complex computations. The motivation for this book came from facing these problems in our data consulting and finding only a limited array of solutions. We soon decided that there was a strong need for an applied text on the practical development and application of multivariate control techniques. We also felt that limiting discussions to strategies based on Hotelling's T2 statistic would be of most benefit to practitioners. In accomplishing this goal, we decided to minimize the theoretical results associated with the T2 statistic, as well as the distributional properties that describe its behavior. These results can be found in the many excellent texts that exist on the theory of multivariate analysis and in the numerous published papers pertaining to multivariate SPC. Instead, our major intent is to present to the practitioner a modern and comprehensive overview on how to establish and operate an applied multivariate control procedure based on our conceptual view of Hotelling's T2 statistic. The intended audience for this book are professionals or students involved with multivariate quality control. We have assumed the reader is knowledgeable about univariate statistical estimation and control procedures (such as Shewhart charts) and is familiar with certain probability functions, such as the normal, chi-square, t, and F distributions. Some exposure to regression analysis also would be helpful. xi
xii
Preface
Although an understanding of matrix algebra is a prerequisite in studying any area of multivariate analysis, we have purposely downplayed this requirement. Instead, appendices are included in various chapters in order to provide the minimal material on matrix algebra needed for our presentation of the T2 statistic. As might be expected, the T2 control procedure requires the use of advanced statistical software to perform the numerous computations. All the T2 charts presented in this text were generated using the QualStat™ software package, which is a product of InControl Technologies, Inc. On the inside back cover of the book we have included a free demonstration version of this software. You will find that the full-licensed version of the package is easy to apply and provides an extended array of graphical and statistical summaries. It also contains modules for use with most of the procedures discussed in this book. This text contains 11 chapters. These have been designed to progress in the same chronological order as one might expect to follow when actually constructing a multivariate control procedure. Each chapter has numerous data examples and applications to assist the reader in understanding how to apply the methodology. A brief description of each chapter is given below. Chapter 1 provides the incentive and intuitive grasp of statistical distance and presents an overview of the T2 as the ideal control statistic for multivariate processes. Chapter 2 supplements this development by providing the distributional properties of the T2 statistic as they apply to multivariate SPC. Distributional results are stated, and data examples are given that illustrate their use when applied to control strategy. Chapter 3 provides methods for checking the distributional assumptions pertaining to the use of the T2 as a control statistic. When distributional assumptions cannot be satisfied, alternative procedures are introduced for determining the empirical distribution of the T2 statistic. Chapters 4 and 5 discuss the construction of the historical data set and T2 charting procedures for a Phase I operation. This includes obtaining the preliminary data, analyzing data problems such as collinearity and autocorrelation, and purging outliers. Chapter 6 addresses T2 charting procedures and signal detection for a Phase II operation. Various forms of the T2 statistic also are considered. Signal interpretation, based on the MYT (Mason-Young-Tracy) decomposition, is presented in Chapter 7 for the bivariate case. We show how a signal can be isolated to a particular process variable or to a group of variables. In Chapter 8 these procedures are extended to cases involving two or more variables. Procedures for increasing the sensitivity of the T2 statistic to small consistent process changes are covered in Chapter 9. A T2 control procedure for autocorrelated observations is developed in Chapter 10, and the concluding chapter, Chapter 11, addresses methods for monitoring batch processes using the T2 statistic. We would like to express our sincere thanks to PPG Industries, Inc., especially the Chemicals and Glass Divisions, for providing numerous applications of the T2 control procedure for use in this book. From PPG in Lake Charles, LA, we thank Joe Hutchins of the Telphram Development Project; Chuck Stewart and Tommy Hampton from Power Generation; John Carpenter and Walter Oglesby from Vinyl; Brian O'Rourke from Engineering; and Tom Hatfield, Plant Quality Coordinator. The many conversations with Bob Jacobi and Tom Jeffery (retired) were most
Preface
xiii
helpful in the initial stages of development of a T2 control procedure. A special thanks also is due to Cathy Moyer and Dr. Chuck Edge of PPG's Glass Technical Center in Harmerville, PA; Frank Larmon, ABB Industrial Systems, Inc.; Bob Smith, LA Pigment, Inc.; and Stan Martin (retired), Center for Disease Control. Professor Youn-Min Chou of the University of Texas at San Antonio, and Professor Nola Tracey McDaniel of McNeese State University, our academic colleagues, have been most helpful in contributing to the development of the T2 control procedure. We also wish to acknowledge Mike Marcon and Dr. James Callahan of InControl Technologies, Inc., whose contributions to the application of the T2 statistic have been immeasurable. We ask ourselves, where did it all begin? In our case, the inspiration can be traced to the same individual, Professor Anant M. Kshirsagar. It is not possible for us to think about a T2 statistic without incurring the fond memories of his multivariate analysis classes while we were in graduate school together at Southern Methodist University many years ago. Finally, we wish to thank our spouses, Carmen and Pam. A project of this magnitude could not have been completed without their continued love and support. Robert L. Mason John C. Young
This page intentionally left blank
Chapter 1
Introduction to the T2 Statistic
The Saga of Old Blue Imagine that you have recently been promoted to the position of performance engineer. You welcome the change, since you have spent the past few years as one of the process engineers in charge of a reliable processing unit labeled "Old Blue." You know every "nook and cranny" of the processing unit and especially what to do to unclog a feed line. Each valve is like a pet to you, and the furnace is your "baby." You know all the shift operators, having taught many and learned from others. This operational experience formed the basis for your recent promotion, since in order to be a performance engineer, one needs a thorough understanding of the processing unit. You are confident that your experience will serve you well. With this promotion, you soon discover that your job responsibilities have changed. No longer are you in charge of meeting daily production quotas, assigning shifts to the operators, solving personnel problems, and fighting for your share of the maintenance budget. Your new position demands that you adopt the skills of a detective and search for methods to improve unit performance. This is great, since over time you have developed several ideas that should lead to process improvement. One of these is the recently installed electronic data collector that instantly provides observations on all variables associated with the processing unit. Some other areas of responsibility for you include identifying the causes of upset conditions and advising operations. When upsets occur, quick solutions must be found to return the unit to normal operational conditions. With your understanding of the unit, you can quickly and efficiently address all such problems. The newly created position of performance engineer fulfills your every dream. But suddenly your expectations of success are shattered. The boss anxiously confronts you and explains that during the weekend, upset conditions occurred with Old Blue. He gives you a diskette containing process data,
1
2
Chapter 1. Introduction to the T2 Statistic retrieved from the data net for both a good-run time period and the upset time period. He states that the operations staff is demanding that the source of the problem be identified. You immediately empathize with them. Having lived through your share of unit upsets, you know no one associated with the unit will be happy until production is restored and the problem is resolved. There is an entire megabyte of data stored on the diskette, and you must decide how to analyze it to solve this problem. What are your options ? You import the data file to your favorite spreadsheet and observe that there are 10,000 observations on 35 variables. These variables include characteristics of the feedstock, as well as observations on the process, production, and quality variables. The electronic data collector has definitely done its job. You remember a previous upset condition on the unit that was caused by a significant change in the feedstock. Could this be the problem? You scan the 10,000 observations, but there are too many numbers and variables to see any patterns. You cannot decipher anything. The thought strikes you that a picture might be worth 1,000 observations. Thus, you begin constructing graphs of the observations on each variable plotted against time. Is this the answer? Changes in the observations on a variable should be evident in its time-sequence graph. With 35 variables and 10,000 observations, this may involve a considerable time investment, but it should be worthwhile. You readily recall that your college statistics professor used to emphasize that graphical procedures were an excellent technique for gaining data insight. You initially construct graphs of the feedstock characteristics. Success eludes you, however, and nothing is noted in the examination of these plots. All the input components are consistent over the entire data set, including over both the prior good-run period and the upset period. From this analysis, you conclude that the problem must be associated with the 35 process variables. However, the new advanced process control (APC) system was working well when you left the unit. The multivariable system keeps all operational variables within their prescribed operational range. If a variable exceeded the range, an alarm would have signaled this and the operator would have taken corrective action. How could the problem be associated with the process when all the variables are within their operational ranges? Having no other options, you decide to go ahead and examine the process variables. You recall from working with the control engineers in the installation of the APC system that they had been concerned with how the process variables vary together. They had emphasized studying and understanding the correlation structure of these variables, and they had noted that the variables did not move independently of one another, but as a group. You decide to examine scatter plots of the variables as well as time-sequence plots. Again, you recall the emphasis placed on graphical techniques by that old statistics professor. What was his name? You begin the laborious task, soon realizing the enormity of the job. From experience, it is easy to identify the most important control variables and the
1.1. Introduction
3
fundamental relationships existing between and among the variables. Perhaps scatter plots of the most influential variables will suffice in locating the source of the problem. However, you realize that if you do not discover the right combination of variables, you will never find the source of the problem. You are interrupted from your work by the reappearance of your boss, who inquires about your progress. The boss states he needs an immediate success story to justify your newly created position of performance engineer. There is a staff meeting in a few days, and he would like to present the results of this work as the success story. More pressure to succeed. You decide that you cannot disappoint your friends in the processing unit, nor your boss. You feel panic creeping close to the edge of your consciousness. A cup of coffee restores calm. You reevaluate your position. How does one locate the source of the problem? There must be a quicker, easier way than the present approach. We have available a set of data consisting of 10,000 observations on 35 variables. The solution must lie with the use of statistics. You slowly begin to recall the courses you took in college, which included basic courses in statistical procedures and statistical process control (SPC). Would those work here? Yes, we can compare the data from the good-run period to the data from the upset period. How is this done for a group o/35 variables? This was the same comparison made in SPC. The good-run period data served as a baseline and the other operational data were compared to it. Signals occurred when present operational data did not agree with the baseline data. Your excitement increases as you remember more. What was that professor's name, old Dr. or Dr. Old? Your coursework in SPC covered only situations involving 1 variable. You need a procedure that considers all 35 related variables at one time and indicates which variable or group of variables contributes to the signal. A procedure such as this would offer a solution to your problem. You rush to the research center to look for a book that will instruct you on how to solve problems in multivariate SPC.
1.1
Introduction
The problem confronting the young engineer in the above situation is common in industry. Many dollars have been invested in electronic data collectors because of the realization that the answer to most industrial problems is contained in the observations. More money has been spent on multivariable control or APC systems. These units are developed and installed to ensure the containment of process variables within prescribed operational ranges. They do an excellent job in reducing overall system variation, as they restrict the operational range of the variables. However, an APC system does not guarantee that a process will satisfy a set of baseline conditions, and it cannot be used to determine causes of system upsets. As our young engineer will soon realize, a multivariate SPC procedure is needed to work in unison with the electronic data collector and the APC system. Such a
4
Chapter 1. Introduction to the T2 Statistic
Group Number Figure 1.1: Shewhart chart of a process variable.
procedure will signal process upsets and, in many cases, can be used to pinpoint precursors of the upset condition before control is lost. When signals are identified, the procedure allows for the decomposition of the signal in terms of the variables that contributed to it. Such a system is the main subject of this book.
1.2
Univariate Control Procedures
Walter A. Shewhart, in a Bell Telephone Laboratories memorandum dated May 16, 1924, presented the first sketch of a univariate control chart (e.g., see Duncan (1986)). Although his initial chart was for monitoring the percentage defective in a production process, he later extended his idea to control charts for the average and standard deviation of a process. Figure 1.1 shows an example of a Shewhart chart designed to monitor the mean, X, of a group of process observations taken on a process variable at the same time point. Drawn on the chart are the upper control limit (UCL) and the lower control limit (LCL). Shewhart charts are often used in detecting unusual changes in variables that are independent and thus not influenced by the behavior of other variables. These changes occur frequently in industrial settings. For example, consider the main laboratory of a major chemical industry. Many duties are assigned to this facility. These may range from research and development to maintaining the quality of production. Many of the necessary chemical determinations are made using various types of equipment. How do we monitor the accuracy (i.e., closeness to a target value) of the determination made by the equipment? Often a Shewhart chart, constructed from a set of baseline data, is utilized. Suppose a measurement on a sample of known concentration is taken. If the result of the sample falls within the control limits of the Shewhart chart, it is assumed that the equipment is performing normally.
1.3. Multivariate Control Procedures
5
Figure 1.2: Model of a production unit. Otherwise, the equipment is fixed and recalibrated and a new Shewhart chart is established. It may be argued that a specific chemical determination is dependent on other factors such as room temperature and humidity. Although these factors can influence certain types of equipment, compensation is achieved by using temperature and humidity controls. Thus, this influence becomes negligible and determinations are treated as observations on an independent variable.
1.3
Multivariate Control Procedures
There are many industrial settings where process performance is based on the behavior of a set of interrelated variables. Production units, such as the one illustrated in Figure 1.2, are excellent examples. They are designed to change an input to some specific form of output. For example, we may wish to change natural gas, a form of energy, to an alternate state such as steam or electricity. Or, we may wish to convert brine (salt water) to caustic soda and chlorine gas; sand to silica or glass; or hydrochloric acid and ethylene to ethylene dichloride, which in turn is changed to vinyl chloride. Our interest lies in the development of a control procedure that will detect unusual occurrences in such variables. Why not use univariate control procedures for these situations? To answer this question, we first need to describe the differences between univariate and multivariate processes. Although the biggest distinction that is evident to the practitioner is the number of variables, there are more important differences. For example, the characteristics or variables of a multivariate process often are interrelated and form a correlated set. Since the variables do not behave independently of one another, they must be examined together as a group and not separately. Multivariate processes are inherent to many industries, such as the chemical industry, where input is being chemically altered to produce a particular output. A good example is the production of chlorine gas and caustic soda. The input variable is saturated brine (water saturated with salt). Under proper conditions, some of the brine is decomposed by electrolysis to chlorine gas; caustic soda is formed within the brine and is later separated. The variables of interest are the components produced by the electrolysis process. All are related to the performance of the process. In
6
Chapter 1. Introduction to the T2 Statistic
addition, many of the variables follow certain mathematical relationships and form a highly correlated set. The correlation among the variables of a' multivariate system may be due to either association or causation. Correlation due to association in a production unit often occurs because of the effects of some unobservable variable. For example, the blades of a gas or steam turbine will become contaminated (dirty) from use over time. Although the accumulation of dirt is not measurable, megawatt production will show a negative correlation with the length of time from the last cleaning of the turbine. The correlation between megawatt production and length of time since last cleaning is one of association. An example of a correlation due to causation is the relationship between temperature and pressure since an increase in the temperature will produce a pressure change. Such correlation inhibits examining each variable by univariate procedures unless we take into account the influence of the other variable. Multivariate process control is a methodology, based on control charts, that is used to monitor the stability of a multivariate process. Stability is achieved when the means, variances, and covariances of the process variables remain stable over rational subgroups of the observations. The analysis involved in the development of multivariate control procedures requires one to examine the variables relative to the relationships that exist among them. To understand how this is done, consider the following example. Suppose we are analyzing data consisting of four sets of temperature and pressure readings. The coordinates of the points are given as
where the first coordinate value is the temperature and the second value is the pressure. These four data points, as well as the mean point of (175, 75), are plotted in the scatter plot given in Figure 1.3. There also is a line fitted through the points and two circles of varying sizes about the mean point. If the mean point is considered to be typical of the sample data, one form of analysis consists of calculating the distance each point is from the mean point. The distance, say D, between any two points, (ai, a^) and (&i, 62)5 is given by the formula This type of distance measure is known as Euclidean, or straight-line, distance. The distance that each of our four example points is from the mean point (in order of occurrence) is computed as
From these calculations, it is seen that points 1 and 4 are located an equal distance from the mean point on a circle centered at the mean point and having a radius of 3.16. Similarly, points 2 and 3 are located at an equal distance from the mean but on a larger circle with a radius of 7.07.
1.3. Multivariate Control Procedures
7
Figure 1.3: Scatter plot illustrating straight-line distance. There are two major criticisms of this analysis. First, the variation in the two variables has been completely ignored. From Figure 1.3, it appears that the temperature readings contain more variation than the pressure readings, but this could be due to the difference in scale between the two variables. However, in this particular case the temperature readings do contain more variation. The second criticism of this analysis is that the covariation between temperature and pressure has been ignored. It is generally expected that as temperature increases, the pressure will increase. The straight line given in Figure 1.3 depicts this relationship. Observe that as the temperature increases along the horizontal axis, the corresponding value of the pressure increases along the vertical axis. This poses an interesting question. Can a measure of the distance between two points be devised that accounts for the presence of a linear relationship between the corresponding variables and the difference in the variation of the variables? The answer is yes; however, the distance is statistical rather than Euclidean and is not as easy to compute. To calculate statistical distance (SD), a measure of the correlation between the variables of interest must be obtained. This is generally expressed in terms of the covariance between the variables, as covariance provides a measure of how variables vary together. For our example data, the sample covariance between temperature and pressure, denoted as S]2, is computed using the formula
where x\ represents the temperature component of the observation vector and x% represents the pressure component. The number of sample points is given by n. The value of the sample covariance as computed from the temperature-pressure data set is 18.75. Also needed in the computation of the statistical distance is the sample variance of the individual variables. The sample variance of a variable, x,
8
Chapter 1. Introduction to the T2 Statistic
Figure 1.4: Scatter plot illustrating statistical distance. is given by
The sample variances for temperature and pressure as determined from the example data are 22.67 and 17.33, respectively. Using the value of the covariance, and the values of the sample variances and the sample means of the variables, the squared statistical distance, (SD)2, is computed using the formula
where R = S12/S1S2is the sample correlation coefficient The actual SD value is obtained by taking the principal square root of both sides of (1.2). Since (1.2) is the formula for an ellipse, the SD is sometimes referred to as elliptical distance (in contrast to straight-line distance). It also has been labeled Mahalanobis's distance, or Hotelling's T 2 , or simply T2. The concept of statistical distance is explored in more detail in Chapter 2. Calculating the (SD)2 for each of the four points in our temperature-pressure sample produces the following results;
From this analysis it is concluded that our four data points are the same statistical distance from the mean point. This result is illustrated graphically in Figure 1.4. All four points satisfy the equation of the ellipse superimposed on the plot. From a visual perspective, this result appears to be unreasonable. It is obvious that points 1 and 4 are closer to the mean point in Euclidean distance than points 2 and 3. However, when the differences in the variation of the variables and the
1.4. Characteristics of a Multivariate Control Procedure
9
relationships between the variables are considered, the statistical distances are the same. The multivariate control procedures presented in this book are developed using methods based on the above concept of statistical distance.
1.4
Characteristics of a Multivariate Control Procedure
There are at least five desirable characteristics of a multivariate control procedure. These include the following: 1. The monitoring statistic should be easy to chart and helpful in identifying process trends. 2. When out-of-control points occur, it must be easy to determine the cause in terms of the contributing variables. 3. The procedure must be flexible in application. 4. The procedure needs to be sensitive to small but consistent process changes. 5. The procedure should be capable of monitoring the process both on-line as well as off-line. A good charting method not only allows for quick signal detection but also helps in identifying process trends. By examining the plotted values of the charting statistic over time, process behavior is observed and upset conditions are identified in advance of a signal, i.e., an out-of-control point. For a clear understanding of the control procedure, interpretation needs to be global and not isolated to a particular data set. The same must be true for the detection of variable conditions that are precursors to upset or chaotic situations. A control procedure having this ability is a valuable asset. A control procedure should work with both independent and time-dependent process observations and be applicable to both continuous processes and batch processes. It should be flexible enough for use with various forms of the control statistic, such as a sample mean or an individual observation, and work with different estimators of the internal structure of the variables. Most industries are volume-oriented. Small changes in efficiency can be the difference in creating a profit or generating a loss. Sensitivity to small process changes is a necessary component of any multivariate control procedure. Multivariate procedures are computationally intense. This is a necessary component since industrial processes are steadily moving toward total computerization. Recent technological advances in industrial control procedures have greatly improved the quantity and quality of available data. The use of computing hardware, such as electronic data collectors, facilitates the collection of data on a multitude of variables from all phases of production. In many situations, one may be working with a very large number of variables and thousands of observations. These are the data available to a statistical control procedure. Any control procedure must be programmable and able to interface and react with such collected online data.
10
Chapter 1. Introduction to the T2 Statistic
Charting with the T2 Statistic Although many different multivariate control procedures exist, it is our belief that a control procedure built on the T2 statistic possesses all the above characteristics. Like many multivariate charting statistics, the T2 is a univariate statistic. This is true regardless of the number of process variables used in computing it. However, because of its similarity to a univariate Shewhart chart, the T2 control chart is sometimes referred to as a multivariate Shewart chart. This relationship to common univariate charting procedures facilitates the understanding of this charting method. Signal interpretation requires a procedure for isolating the contribution of each variable and/or a particular group of variables. As with univariate control, outof-control situations can be attributed to individual variables being outside their allowable operational range; e.g., the temperature is too high. A second cause of a multivariate signal may be attributed to a fouled relationship between two or more variables; e.g., the pressure is not where it should be for a given temperature reading. The signal interpretation procedure covered in this text is capable of separating a T2 value into independent components. One type of component determines the contribution of the individual variables to a signaling observation, while the other components check the relationships among groups of variables. This procedure is global in nature and not isolated to a particular data set or type of industry. The T2 statistic is one of the more flexible multivariate statistics. It gives excellent performance when used to monitor independent observations from a steadystate continuous process. It also can be based on either a single observation or the mean of a subgroup of n observations. Minor adjustments in the statistic and its distribution allow the movement from one form to the other. Many industrial processes produce observations containing a time dependency. For example, process units with a decaying cycle often produce observations that can be modeled by some type of time-series function. The T2 statistic can be readily adapted to these situations and can be used to produce a time-adjusted statistic. The T2 statistic also is applicable to situations where the time correlation behaves as a step function. We have experienced no problems in applying the T2 statistic to batch or semibatch processes with targets specified or unspecified. In the case of target specification, the T2 statistic measures the statistical distance the observed value is from the specified target. In cases where rework is possible, such as blending, components of the T2 decomposition can be used in determining the blending process. Sensitivity to small process change is achieved with univariate control procedures, such as Shewhart charts, through applications of zonal charts with run rules. Small, consistent process changes in a T2 chart can be detected by using certain components of the decomposition of a T2 statistic. This is achieved by monitoring the residual error inherent to these terms. The detection of small process shifts is so important that a whole chapter of the text is devoted to this procedure. An added benefit of the T2 charting procedure is the potential to do on-line experimentation that can lead to local optimization. Because of the demand of
1.5. Summary
11
production quotas, the creation of dangerous and hazardous conditions, extended upset recovery periods, and numerous other reasons, the use of experimental design is limited in most production units. However, one can tweak the process. Monitoring of the appropriate residual terms allows one to observe the effect of this type of experimentation almost instantaneously. In addition, the monetary value of process changes, due to new equipment or operational procedures, can be quickly determined. This aspect of a T2 control procedure has proved invaluable in many applications. Numerous software computer programs are available for performing a variety of univariate SPC procedures. However, computer packages for doing multivariate SPC are few in number. Some, such as SAS™, can be useful but require individual programming. Others, such as JMP™, a product of SAS, Inc., provide only limited multivariate SPC. The program QualStat™, a product of InControl Technologies, Inc., contains a set of procedures based entirely on the T2 statistic. This program is used extensively in this book to generate the T2 graphs and perform the T2 analyses.
1.5
Summary
Industrial process control generally involves monitoring a set of correlated variables. Such correlation confounds the interpretation of univariate procedures run on individual variables. One method of overcoming this problem is to use a Hotelling's T2 statistic. As demonstrated in our discussion, this statistic is based on the concept of statistical distance. It consolidates the information contained in a multivariate observation to a single value, namely, the statistical distance the observation is from the mean point. Desirable characteristics for a multivariate control chart include ease of application, adequate signal interpretation, flexibility, sensitivity to small process changes, and available software to use it. One multivariate charting procedure that possesses all these characteristics is the method based on the T2 statistic. In the following chapters of this book, we explore the various properties of the T2 charting procedure and demonstrate its value.
This page intentionally left blank
Chapter 2
Basic Concepts about the 72 Statistic
2.1
Introduction
Some fundamental concepts about the T2 statistic must be presented before we can discuss its usage in constructing a multivariate control chart. We begin with a discussion of statistical distance and how it is related to the T2 statistic. How statistical distance differs from straight-line or Euclidean distance is an important part of the coverage. Included also is a discussion of the relationship between the univariate Student t statistic and its multivariate analogue, the T2 statistic. The results lead naturally to the understanding of the probability functions used to describe the T2 statistic under a variety of different circumstances. Having knowledge of these distributions aids in determining the UCL value for a T2 chart, as well as the corresponding false alarm rate.
2.2
Statistical Distance
Hotelling (1947), in a paper on using multivariate procedures to analyze bombsight data, was among the first to examine the problem of analyzing correlated variables from a statistical control perspective. His control procedure was based on a charting statistic that he had introduced in an earlier paper (i.e., Hotelling (1931)) on the generalization of the Student t statistic. The statistic later was named in his honor as Hotelling's T 2 . Slightly prior to 1931, Mahalanobis (1930) proposed the use of a similar statistic, which would later become known as Mahalanobis's distance measure, for use in measuring the squared distance between two populations. Although the two statistics differ only by a constant value, the T2 form is the most popular in multivariate process control and is the main subject of this text. The following discussion provides insight into how the concept of statistical distance, as defined with the T2 statistic, is used in the development of multivariate 13
14
Chapter 2. Basic Concepts about the T2 Statistic
control procedures. The reader unfamiliar with vectors and matrices may find the definitions and details given in this chapter's appendix (section 2.8) to be helpful in understanding these results. Suppose we denote a multivariate observation on p variables in vector form as X' = (x±,x2,..., xp}. Our main concern is in processing the information available on these p variables. One approach is to use graphical techniques, which are usually excellent for this task, but plotting points in a pdimensional space (p > 3) is severely limited. This restriction inhibits overall viewing of the multivariate situation. Another method for examining information provided in a ^-dimensional observation is to reduce the multivariate data vector to a single univariate statistic. If the resulting statistic contains information on all p variables, it can be interpreted and used in making decisions as to the status of a process. There are numerous procedures for achieving this result, and we demonstrate two of them below. Suppose a process generates uncorrelated bivariate observations, (xi,:^), and it is desired to represent them graphically. It is common to construct a twodimensional scatter plot of the points. Also, suppose there is interest in determining the distance a particular point is from the mean point. The distance between two points is always measured as a single number or value. This is true regardless of how many dimensions (variables) are involved in the problem. The usual straight-line (Euclidean) distance measures the distance between two points by the number of units that separate them. The squared straight-line distance, say D, between a point (£1,22) and the population mean point (//i,/^) is defined as Note that we have taken the bivariate observation, (xi,x 2 ), and converted it to a single number D, the distance the observation is from the mean point. If this distance, D, is fixed, all points that are the same distance from the mean point can be represented as a circle with center at the mean point and a radius of D (i.e., see Figure 2.1). Also, any point located inside the circle has a distance to the mean point less than D. Unfortunately, the Euclidean distance measure is unsatisfactory for most statistical work (e.g., see Johnson and Wichern (1998)). Although each coordinate of an observation contributes equally to determining the straight-line distance, no con sideration is given to differences in the variation of the two variables as measured by their variances, a\ and cr|, respectively. To correct this deficiency, consider the standardized values
and all points satisfying the relationship
The value SD, the square root of (SD)2 in (2.1), is known as statistical distance. For a fixed value of SD, all points satisfying (2.1) are the same statistical distance
2.2. Statistical Distance
15
Figure 2.1: Region of same straight-line distance.
from the mean point. The graph of such a group of points forms an ellipse, as is illustrated in the example given in Figure 2.2. Any point inside the ellipse will have a statistical distance less than SD, while any point located outside the ellipse will have a statistical distance greater than SD. In comparing statistical distance to straight-line distance, there are some major differences to be noted. First, since standardized variables are utilized, the statistical distance is dimensionless. This is a useful property in a multivariate process since many of the variables may be measured in different units. Second, any two points on the ellipse in Figure 2.2 have the same SD but could have possibly different Euclidean distances from the mean point. If the two variables have equal variances and are uncorrelated, the statistical and Euclidean distance, apart from a constant multiplier, will be the same; otherwise, they will differ. The major difference between statistical and Euclidean distance in Figure 2.2 is that the two variables used in statistical distance are weighted inversely by their standard deviations, while both variables are equally weighted in the straight-line distance. Thus, a change in a variable with a small standard deviation will contribute more to statistical distance than a change in a variable with a large standard deviation. In other words, statistical distance is a weighted straight-line distance where more importance is placed on the variable with the smaller standard deviation to compensate for its size relative to its mean. It was assumed that the two variables in the above discussion are uncorrelated. Suppose this is not the case and that the two variables are correlated. A scatter plot of two positively correlated variables is presented in Figure 2.3. To construct a statistical distance measure to the mean of these data requires a generalization of (2.1).
16
Chapter 2. Basic Concepts about the T2 Statistic
Figure 2.2: Region of same statistical distance.
Figure 2.3: Scatter plot of correlated variables. From analytical geometry, the general equation of an ellipse is given by
where the a^- are specified constants satisfying the relationship (a^2 — 4ana22) < 0, and c is a fixed value. By properly choosing the a^ in (2.2), we can rotate the
2.3. T2 and Multivariate Normality
17
Figure 2.4: Elliptical region encompassing data points.
ellipse while keeping the scatter of the two variables fixed, until a proper alignment is obtained. For example, the ellipse given in Figure 2.4 is centered at the mean of the two variables vet rotated to reflect the correlation between them.
2.3
72 and Multivariate Normality
Suppose (xi, #2) can be described jointly by a bivariate normal distribution. Under this assumption, the statistical distance between this point and the mean vector (//i, Hz] is the value of the variable part of the exponent of the bivariate normal probability function
where — oo < x^ < oo for i = 1,2, and Oi > 0 represents the standard deviation of Xi. The value of (SD) 2 is given by
where p represents the correlation between the two variables, with — 1 < p < I. The cross-product term between x\ and x^ in (2.4) accounts for the fact that the two variables vary together and are dependent. When x\ and x^ are correlated, the major and minor axes of the resulting ellipse differ from that of the variable space (,TI, x%). If the correlation is positive, the ellipse will tilt upward to the right, and if the correlation is negative, the ellipse will tilt downward to the right. This
Chapter 2. Basic Concepts about the T2 Statistic
18
Figure 2.5: Correlation and the ellipse.
is illustrated in Figure 2.5. If p = 0, so that there is no correlation between x\ and #2, the ellipse will be oriented similar to the one given in Figure 2.2. Equation (2.4) can be expressed in matrix notation (see section 2.8.5) as
where X' = (£1,2:2), // = (//i, ^2), and E
[
2
1
is the inverse of the matrix E. Note
1
a\E ai2 L = po\a-2 where is the covariance between <7i 2:1 and THAT 2 = cr2i x-2- The matrix E is referred to as the covariance matrix between x\ and 2:2- The expression in (2.5) is a form of Hotelling's T2 statistic. Equations for the contours of a bivariate normal density are obtained by fixing the value of SD in (2.4). This can be seen geometrically by examining the bivariate normal probability function presented in Figure 2.6. The locus, or path, of the point X' = (2:1,2:2) traveling around the probability function at a constant height is an ellipse. Ellipses of constant density are referred to as contours and can be determined mathematically to contain a fixed amount of probability. For example, the 75% and 95% contours for the bivariate normal function illustrated in Figure 2.6 are presented in Figure 2.7. The elliptical contours represent all points having the same statistical distance or T2 statistic value.
2.3. T2 and Multivariate Normality
19
Figure 2.6: A bivariate normal probability function (a\ = 1, cr2 = 1, p = 0.8).
Figure 2.7: Bivariate normal contours containing 75% and 95% of the probability.
20
Chapter 2. Basic Concepts about the T2 Statistic
This result can be generalized to the situation where X' — (xi, £2, • • • , %p) is described by the p-variate normal (multivariate normal (MVN)) probability function given by
where — oo < Xi < oo for i = 1,2, . . . , £ > . The mean vector of X' is given by //' = (//I, //2, • • • , (J>P) and the covariance matrix is given by
A diagonal element, an, of the matrix E represents the variance of the ith variable, and an off-diagonal element, a^, represents the covariance between the ith and jth variables. Note that E is a nonsingular, symmetric, and positive definite matrix. In this setting, the equation for an ellipsoidal contour of the MVN distribution in (2.6) is given by
where T2 is a form of Hotelling's T2 statistic. As in the bivariate case, the ellipsoidal regions contain a fixed percentage of the MVN distribution and can be determined exactly.
2.4
Student t versus Hotelling's T"2
The univariate Student t statistic is very familiar to most data analysts. If it is to be compared to a Student t distribution, the statistic is computed from a random sample of n observations taken from a population having a normal distribution with mean /j, and variance a2. Its formula is given by
where
is the sample mean and
is the
corresponding sample standard deviation. The square of the t statistic is given by
and its value is defined as the squared statistical distance between the sample mean and the population mean.
2.4. Student t versus He-telling's T2
21
The numerator of (2.9) is the squared Euclidean distance between x and //. Thus, it is a measure of the closeness of the sample mean to the population mean. As x gets closer to //, the value of t2 approaches zero. Division of the squared Euclidean distance by the estimated variance of x (i.e., by s2/n) produces the squared statistical distance. Hotelling (1931) extended the univariate t statistic to the multivariate case using a form of the T2 statistic based on sample estimates (rather than known values) of the covariance matrix. His derivation is described as follows. Consider a sample of n observations X\, X^, • • • , Xn, where X[ = (xn,Xi2,..., ZIP), i = 1, 2 , . . . , n, is taken from a p-variate normal distribution having a mean vector // and a covariance matrix E. A multivariate generalization of the t2 statistic is given by where X and S are sample estimators of /u and E and are denned by
and
The sample covariance matrix S also can be expressed as
where sa is the sample variance of the ith variable and s^- is the sample covariance between the ith and jth variables. The matrix S has many special properties. Those properties that pertain to our use of the T2 as a control statistic for multivariate processes are discussed in later sections. In terms of probability distributions, the square of the t statistic in (2.9) has the form t 2 = (normal random variable) * (chi-square random variable/dj)~ 1 * (normal random variable), where df represents the n — I degrees of freedom of the chi-square variate, (n — I)s 2 /cr 2 , and the normal random variable is given by ^/n(x — /u)/cr. In this representation, the random variable, x, and the random variable, s 2 . are statistically independent. Similarly, the T2 statistic in (2.10) may be expressed as T2 — (multivariate normal vector) * (Wishart matrix/df) * (multivariate normal vector), where df indicates the n~l degrees of freedom of the Wishart variate, (n—1)5, and
22
Chapter 2. Basic Concepts about the T2 Statistic
the multivariate normal vector is given by ^/n(X — ^). The random vector X and the random matrix S are statistically independent. The Wishart distribution (see section 2.8.6 for details) in (2.12) is the multivariate generalization of the univariate chi-square distribution. Using the two forms presented in (2.10) and (2.12), it is possible to extend Hotelling's T2 statistic to represent the squared statistical distance between many different combinations of p-dimensional points. For example, one can use the T2 statistic to find the statistical distance between an individual observation vector X and either its known population mean // or its population mean estimate X. Hotelling's T2 also can be computed between a sample mean, Xi, of a subgroup and the overall mean, X, of all the subgroups.
2.5
Distributional Properties of the T2
One basic assumption preceding any discussion of the distribution properties of a Hotelling's T2 statistic is that the multivariate observations involved are the result of a random sampling of a p-variate normal population having a mean vector \JL and a covariance matrix E. Thus, the behavior of the independent observations can be described by a probability function with parameters either known or unknown. If the parameters are unknown, it will be assumed that there exists a historical data set (HDS) which was collected under steady-state conditions when the process was in control. This data set is used to produce estimates of the unknown parameters. Our work requires that we transform these p-variate sample observations to a single Hotelling's T2 statistic. Since the original variables are random, these new T2 values also are random and can be described by an appropriate probability function. For example, when the parameters of the underlying multivariate normal distribution are unknown and must be estimated, some form of a univariate F distribution is used in describing the random behavior of the T2 statistic. This is also applicable in the univariate case. If the t statistic in (2.8) can be described by a t distribution with (n — 1) degrees of freedom, the square of the statistic, t2 in (2.9), can be described by an F distribution with 1 and (n — 1) degrees of freedom. The T2 statistic may be computed using a single observation made on p components at a fixed sampling point, or it may be computed using the mean of a sample of size m taken during a fixed time period. Unless otherwise stated in this book, a subgroup of size 1 (i.e., a single observation) will be assumed for the T2 computations. When necessary, stated results will be modified to include the case where the subgroup size exceeds 1. Particular assumptions that govern the distribution of the T2 statistic are separated into two major cases: the parameters IJL and £ of the underlying distribution being either known or unknown. The second case, when the parameters must be estimated, also has two different situations. The first occurs when an observation vector X is independent of the estimates of the parameters. Independence will occur when X is not included in the computation of X and S1, the sample estimates of IJL and S. The second situation occurs when X is included in the computation of the estimates and hence is not independent of them.
2.5. Distributional Properties of the T2
23
Figure 2.8: Chi-square distribution. Several different probability functions can be used in describing the T2 statistic (e.g., see Fuchs and Kenett (1998); or Wierda (1994)). Three key forms are discussed below along with the conditions in which each is applicable. (1) Assume the parameters, [i and S, of the underlying MVN distribution are known. The T2 statistic for an individual observation vector X has the form and distribution given by
where %2 ^ represents a chi-square distribution with p degrees of freedom. The T2 distribution depends only on p, the number of variables in the observation vector X. Graphs of the chi-square distribution function, /(^ 2 ), for various values of p are presented in Figure 2.8, and percentage points of the chi-square distribution are given in Table A.3 in the appendix. For smaller values of p, we observe a skewed distribution with a long tail to the right; a more symmetric form is observed for larger values of p. The chi-square probability function provides the probability distribution of the T2 values along this axis. (2) Assume the parameters of the underlying MVN distribution are unknown and are estimated using the estimators X and S given in (2.11). These values are obtained using an HDS consisting of n observations. The form and distribution of the T2 statistic for an individual observation vector X, independent of X and /S, is
24
Chapter 2. Basic Concepts about the T2 Statistic
Figure 2.9: F distribution. given as
where F(p^n_p^ is an F distribution with p and (n — p) degrees of freedom. The distribution in (2.14) depends on the sample size of the HDS as well as on the number of variables being examined. Graphs of the F distribution, f ( F ] , for various values of the numerator and denominator degrees of freedom are presented in Figure 2.9, and the percentage points of the F distribution are given in Table A.4 in the appendix. Again, we observe a skewed distribution with a long tail to the right. (3) Assume the observation vector X is not independent of the estimators X and S but is included in their computation. In this situation, the form and distribution of the T2 statistic (e.g., see Tracey, Young, and Mason (1992)) is given as
where -B(p/2,(n-P-i)/2) represents a beta distribution with parameters p/2 and (n — p — l)/2. The distribution in (2.15) depends on the number of variables, p, and on the sample size, n, of the HDS. Of the above three probability functions used in describing the random behavior of a T2 statistic, the beta distribution is probably the one that is most unfamiliar to analysts. Unlike the chi-square and F distributions that allow evaluations for any variable values greater than zero, the beta distribution f(B] restricts beta values
2.5. Distributional Properties of the T2
25
Figure 2.10: Beta distribution. to the unit interval (0,1). However, within this interval, the distribution can take on many familiar shapes, such as those associated with the normal, chi-square, and F distributions. Examples of the beta distribution for various parameter values are depicted in Figure 2.10, and percentage points of the beta distribution are given in Table A.5 in the appendix. It was stated earlier that the distribution used in describing a T2 statistic when the parameters of the underlying normal distribution are unknown is some form of an F distribution. However, in (2.15) we have used the beta distribution to describe the T2 statistic. The T 2 , for this case, can be expressed as an F statistic by using a relationship that exists between the F and beta probability functions. The result is given by
where In practice, we generally choose to use the beta distribution rather than the F distribution in (2.16). Although this is done to emphasize that the observation vector X is not independent of the estimates obtained from the HDS, either distribution is acceptable. Since each T2 value obtained from the HDS depends on the same value of X and 5, a weak interdependence among the T2 values is produced. The correlation between any two T2 values computed from an HDS is given as — l/(n — 1) (see Kshirsagar and Young (1971)). It is easily seen that even for modest values of n, this correlation rapidly approaches zero. Although this is not justification for
26
Chapter 2. Basic Concepts about the T2 Statistic
assuming independence, it has been shown (see David (1970) and Hawkins (1981)) that, as n becomes large, the set of T2 values behaves like a set of independent observations. This fact becomes important when subjecting the T2 values of an HDS to other statistical procedures. Process control in certain situations is based on the monitoring of the mean of a sample (i.e., subgroup) of m observations taken at each of k sampling intervals. The distribution that describes the statistical distance between the sample mean of the ith observation vector Xi and the HDS mean X is given by
where F(p,n-p) represents an F distribution with p and (n — p) degrees of freedom. This distribution, which assumes independence between Xi and the parameter estimates, depends on the sample size m, the size of the HDS, and the number of variables in the observation vector, Xi. Although the estimator of S given in (2.11) is used in the statistic of (2.17), it is more common to insert a pooled estimate of 5 given by
where Si represents the sample covariance estimate for the data taken during the ith sampling period, i = 1, 2 , . . . , k. With this estimate, the form and distribution of the T2 statistic becomes
2.6
Alternative Covariance Estimators
The T2 is a versatile statistic as it can be constructed with covariance estimators other than the common estimator S given in (2.11). An example is given in the formula in (2.19), where the pooled estimator Sw is used in the construction of the T2 statistic for monitoring subgroup means. Several other estimators of the covariance matrix and the associated T2 statistic also are available. For example, Holmes and Mergen (1993) as well as Sullivan and Woodall (1996), labeled S&W below, presented an estimator based on the successive difference of consecutive observation vectors (order of occurrence) in computing a covariance estimator when only individual observations are available. The estimator, SD, is given as
The distribution of a T2 statistic using SD is unknown. However, S&W provided the following approximation:
2.6. Alternative Covariance Estimators
27
where is based on a result given in Scholz and Tosch (1994). Note that the formula in (2.21) contains a correction to the expression given in the SfeW article. In selected situations, the statistic in (2.21) serves as an alternative to the common T2 in detecting step and ramp shifts in the mean vector. Other covariance estimators have been constructed in a similar fashion by partitioning the sample in different ways. For example, Wierda (1994) suggested forming a covariance estimator by partitioning the data into independent, nonoverlapping groups of size 2. Consider a sample of size n, where n is even. Suppose group 1 = {Xi,X2}, group2 = {X 3 ,X 4 },..., group(n/2) = {Xn-i,Xn}. The estimated covariance matrix for each group C{, i = 1, 2 , . . . , (n/2), is
and the partition-based covariance estimator is given by
S&W presented a simulation study of five alternative estimators of the covariance matrix and compared the power of the corresponding T2 chart in detecting outliers as well as in detecting step and ramp shifts in the mean vector. Included in the comparisons were the common estimator S, given in (2.11), and the above Sp. These authors showed that, with 5, the T2 chart for individual data is not effective in detecting step shifts near the middle of the data. In such situations, the chart is biased, meaning that the signal probability with out-of-control data is less than with in-control data. Accordingly, S&W recommended the covariance estimator, S1^, given in (2.20), for retrospective analysis of individual observations, and they showed that T2 charts based on SD are more powerful than ones based on Sp. They also showed that the T2 chart based on S was the most effective among those studied in detecting outliers, but was susceptible to masking with a moderate number of outliers. In a follow-up study, Sullivan and Woodall (2000) gave a comprehensive method for detecting step shifts or outliers as well as shifts in the covariance matrix. Chou, Mason, and Young (1999) conducted a separate power analysis of the T2 statistic in detecting outliers using the same five covariance estimators as used by S&W. They too showed that the common estimator, $, is preferred for detecting outliers. Although we have presented a brief description of some alternative covariance estimators, our applications of the T2 statistic in future chapters will rely primarily on usage of the common estimator S. The major exception is in the use of the within-group estimator Sw in the application of the T2 to batch processes (see Chapter 11). We do this for several reasons. In many industrial settings, ramp changes, step changes, and autocorrelation may be inherent qualities of the process. The variation associated with observations taken from such a process, even when the
Chapter 2. Basic Concepts about the T2 Statistic
28
process is in-control, does not represent random fluctuation about a constant mean vector. As we will see later, a multivariate control procedure can accommodate such systematic variations in the mean vector. The common estimator S captures the total variation, including the systematic variation of the mean, whereas many of the above-mentioned alternative estimators estimate only random variation, i.e., stationary variation. Thus, these alternative estimators require additional modeling of the systematic variation to be effective. With autocorrelation, the T2 charts based on S are less likely to signal, either when the data are out-of-control or in-control, unless similar additional modeling is performed (i.e., see Chapter 10). When data are collected from an MVN distribution, the common covariance estimator has many interesting properties. For example, 5 is an unbiased estimator and is the maximum likelihood estimator. The probability function that describes 5 is known, and this is important in deriving the distribution of the T2 statistic. Another important observation on S that is useful for later discussion is that its value is invariant to a permutation of the data. Thus, the value of the estimator is the same regardless of which one of the many possible arrangements of the data, Xi, X-2-, • • • , Xn, is used in its computation.
2.7
Summary
In this chapter, we have demonstrated the relationship between the univariate t statistic and Hotelling's T2 statistic. Both are shown to be a measure of the statistical distance between an observed sample mean and its corresponding population mean. This concept of statistical distance, using the T2 statistic, was expanded to include the distance a single observation is from the population mean (or its sample estimate) and the distance a subgroup mean is from an overall mean. With the assumption of multivariate normality, we presented several probability functions used to describe the T2 statistic. This was done for control procedures based on the monitoring of a single observation or the mean of a subgroup of observations. Since there are many occasions in which the T2 statistic is slightly modified to accommodate a specific purpose, we will continue to introduce appropriate forms of the T2 statistic and its accompanying distribution. The ability to construct a Hotelling's T2 for these different situations adds to its versatility as a useful tool in the development of a multivariate control procedure.
2.8
Appendix: Matrix Algebra Review
In Chapter 1, we introduced the T2 statistic for two variables in an algebraic form given by (1.2). As the number of variables increases, the algebraic form of the T2 becomes intractable and we must resort to a simpler form of the T2 using matrix notation. Many of the properties of the T2 statistic are related to the properties of the matrices that compose this form. As an aid to understanding Chapter 2, we offer a brief matrix review. Our presentation includes only those matrix properties that we deem necessary for a clear understanding of the matrix form of the T2
2.8. Appendix: Matrix Algebra Review
29
statistic. It is assumed that the background of the reader includes knowledge of basic matrix operations. For additional information on this subject, the interested reader is directed to the many textbooks written on matrix algebra (e.g., see Agnew and Knapp (1995)).
2.8.1
Vector and Matrix Notation
An (r x c) matrix is a rectangular array of elements having r rows and c columns. For example, the (2 x 3) matrix A has 2 rows and 3 columns and is given by
where the a^ are constant values. Vectors are matrices with either one row a row vector) or one column (a column vector). Consider a multivariate process involving p process variables. We denote the first process variable as #1, the second as #2, • • • , and the pth process variable as xp. A simple way of denoting an observation (at a given point in time) on all process variables is by using a (p x 1) column vector X, where
The p process variables in X define a p-dimensional variable space, one dimension for each process variable. An observation on the p process variables translates to a point in this p-dimensional space. The transpose of a matrix is obtained by taking each of its columns and making a corresponding row from them. Thus, the first column of a matrix becomes the first row in the transpose, the second column becomes the second row in the transpose, etc. If A denotes the vector, then A' will denote the transpose. For example, the transpose of X is a (1 x p) row vector that is given by
2.8.2
Data Matrix
A sample of n observation vectors on p process variables is designated as X\, X%,..., Xn. The sample mean vector is obtained by
Chapter 2. Basic Concepts about the T2 Statistic
30
The information contained in the sample can be arranged in an (n x p] data matrix given by
Another important form of the data matrix is achieved by subtracting the mean vector from each observation vector. This form is given as
The sample covariance matrix S that is necessary in computing a T2 statistic is computed from the data matrix using
2.8.3 The Inverse Matrix The inverse of a square (p x p} matrix A is defined to be the matrix, A~1 that satisfies where / is the (p x p} identity matrix given by
The inverse matrix exists only if the determinant of A is nonzero. This implies that the matrix A must be nonsingular. If the inverse does not exist, the matrix A is singular. Sophisticated computer algorithms exist to compute accurately the inverses of large matrices and are contained in most computer packages used in the analysis of multivariate data. 2.8.4
Symmetric Matrix
Numerous references are made to symmetric matrices throughout this text. A symmetric matrix A is one that is equal to its transpose, i.e.,
2.8. Appendix: Matrix Algebra Review
31
The definition dictates that the matrix A must be a square matrix, so that the number of rows equals the number of columns. It also implies that the off-diagonal elements of the matrix A, denoted by a^-, i ^ j, are equal, i.e., that
As an example, the sample covariance matrix S is a symmetric matrix, since,
2.8.5
Quadratic Form
Let X' = (#1,2:2) and
and consider the quadratic form expression given by
Performing the matrix and vector multiplication produces the following univariate expression: With the assumption that A is a symmetric matrix, so that 0-12 = 021, the above expression can be written as
In this form, it is easy to see the relationship between the algebraic expression and the matrix notation for a quadratic form. The matrix A of the quadratic form is defined to be positive definite if the quadratic expression is larger than zero for all nonzero values of the vector X. To demonstrate this procedure, consider the quadratic form in three variables given by This expression is written in matrix notation as X'AX, where X' = (2:1, £2,2:3) and
Consider the algebraic expression for a T2 statistic given in (2.4) as
32
Chapter 2. Basic Concepts about the T2 Statistic
With a little manipulation, the above T2 can be written as the Quadratic form
where X' = (xi,X2) and // = (^1,^2)- The inverse of the matrix S, is given by
where a\i = a
2.8.6
Wishart Distribution
When data are collected from an MVN distribution, the (p x p) random matrix, (n — 1)5, where 5 is defined by
is labeled a Wishart matrix. This name comes from Wishart (1928), who generalized the joint distribution of the p(p + l)/2 unique elements of this matrix. This result is predicated on the assumption that the original random sample of n observation vectors is obtained from an NP(/J,, E), so that each Xi ~ NP(^L, E) for i = 1, 2 , . . . , n. It is also assumed that the matrix 5 is a symmetric matrix. The Wishart probability function is given as
The matrix S is positive definite and F(.) is the gamma function (e.g., see Anderson (1984)). Unlike the MVN distribution, the Wishart density function has very little use other than in theoretical derivations (e.g., see Johnson and Wichern (1998)). For the case of p = 1, it can be shown that the Wishart distribution reduces to a constant multiple of a univariate chi-square distribution. It is for this reason that the distribution is thought of as being the multivariate analogue of the chi-square distribution.
Chapter 3
Checking Assumptions for Using a T2 Statistic
3.1
Introduction
As was indicated in Chapter 2, the distributions of various forms of the T2 statistic are well known when the set of p-dimensional variables being sampled follows an MVN distribution. The MVN assumption for the observation vectors guarantees the satisfaction of certain conditions that lead to the known T2 distributions. However, validating this assumption for the observation vectors is not an easy task. An alternative approach for use with nonnormal distributions is to approximate the sampling distribution of the T2. In this chapter, we take this latter approach by seeking to validate only the univariate distribution of the T2 statistic, rather than the MVN distribution of the observation vectors. There are a number of other basic assumptions that must be made and requirements that must be met in order to use the T2 as a control statistic. These conditions include: (1) selecting a sample of independent (random) observations, (2) determining the UCL to use in signal detection, (3) collecting a sufficient sample size, and (4) obtaining a consistent estimator of the covariance matrix for the variables. In this and later chapters, we discuss these assumptions and requirements and show how they relate to the T2 statistic. We also demonstrate techniques for checking their validity and offer alternative procedures when these assumptions cannot be satisfied.
3.2
Assessing the Distribution of the T2
In univariate process control, if the probability function of the control chart statistic is assumed to be a normal distribution, the assumption can be verified by performing a goodness-of-fit test. There are numerous formal tests available for this task, including the Shapiro-Wilk test, the Kolmogorov-Smirnov test, and the 33
34
Chapter 3. Checking Assumptions for Using a T2 Statistic
Anderson-Darling test. These procedures have been documented in numerous statistical textbooks (e.g., see Anderson, Sweeney, and Williams (1994)), and the corresponding test statistics and associated probability functions are readily defined and tabulated. Many different tests have been proposed for assessing multivariate normality for a group of p-variate observations (e.g., for a comprehensive list and discussion, see Seber (1984) and Looney (1995)). These include such familiar ones as those based on the multivariate measures of skewness and kurtosis. Unfortunately, the exact distribution of the test statistic is unknown in these tests. Without this knowledge the critical values used for the various test procedures can only be approximated. Hence, these procedures serve solely as indicators of possible multivariate normality. In view of these results, we take a different approach. Although, for these discussions, we restrict our attention to the case where a baseline data set is being utilized, the findings can easily be extended to the monitoring of future observations. The assumption of multivariate normality guarantees known distribution results for the T2 statistic (e.g., see Tracey, Young, and Mason (1992)). In a baseline data set, when the p-dimensional observation vectors X are distributed as an MVN, the T2 values follow a beta distribution, so that where the T2 statistic is based on the formula given in (2.15). Since the T2 in (3.1) is a univariate statistic with a univariate distribution, we propose performing a goodness-of-fit test on its values to determine if the beta is the appropriate distribution, rather than on the values of X to determine if the MVN is the correct distribution. Although observations taken from an MVN can be transformed to a T2 statistic having a beta distribution, it is unknown whether other multivariate distributions possess this same property. The mathematics for the nonnormal situations quickly becomes intractable. However, we do know that the beta distribution obtained under MVN theory provides a good approximation for some nonnormal situations. We illustrate this phenomenon below using a bivariate example. We begin by generating 1,000 standardized bivariate normal observations having a correlation of 0.80. A scatter plot of these observations is presented in Figure 3.1. For discussion purposes, three bivariate normal contours (i.e., equal altitudes on the surface of the distribution) at fixed T2 values are superimposed on the data. These also are illustrated on the graph. Note the symmetrical dispersion of the points between the concentric T2 ellipses. The concentration of points diminishes from the center outward. Using 31 contours to describe the density of the 1,000 observations, and summing the number of observations between these contours, we obtain the histogram of the T2 values presented in Figure 3.2. The shape of this histogram corresponds to that of a beta distribution. In contrast to the above example, a typical scatter plot of a nonnormal bivariate distribution is presented in Figure 3.3. Note the shape of the plot. The observations in Figure 3.3 are generated from the observations in Figure 3.2 by truncating variable x\ at the value of 1. Distributions such as this occur regularly in industries where run limits are imposed on operational variables. Truncation, which produces
3.2. Assessing the Distribution of the T2
35
Figure 3.1: Bivariate normal scatter plot and contours.
Figure 3.2: Histogram of T2 values for bivariate normal data. long-tailed distributions, also occurs with the use of certain lab data. This can be due to the detection limit imposed by the inability of certain types of equipment to make determinations below (or above) a certain value. In Figure 3.4, three bivariate normal contours at fixed T2 values, computed using the mean vector and covariance matrix of the truncated distribution, are superimposed on the nonnormal data. A major distinction between the scatter plots in Figures 3.1 and 3.4 is the dispersion of points between the elliptical contours. For the bivariate normal data in Figure 3.1, the points are symmetrically dispersed between the contours. This is not the case for the nonnormal data in Figure 3.4.
36
Chapter 3. Checking Assumptions for Using a T2 Statistic
Figure 3.3: Scatter plot of data from a truncated distribution.
Figure 3.4: Bivariate normal contours superimposed on truncated distribution.
For example, note the absence of points in the lower left area between the two outer contours. Nevertheless, possibly due to this particular pattern of points, or due to the size of the sample, the corresponding T2 histogram given in Figure 3.5 for this distribution has a strong resemblance to the histogram given in Figure 3.2. Agreement of this empirical distribution to a beta distribution can be determined by performing a univariate goodness-of-fit test.
3.4. The Sampling Distribution of the T2 Statistic
37
Figure 3.5: Histogram of T2 values for truncated data.
3.3
The T2 and Nonnormal Distributions
We do not want to imply from the above example that the beta distribution can be used indiscriminately to describe the T2 statistic. To better understand the problem, consider the bivariate skewed distribution, f(xi,x<2), illustrated in Figure 3.6. The contours of this density are presented in Figure 3.7. Observe the irregular shape and lack of symmetry of the contours; they definitely lack the elliptical shape of the bivariate normal contours given in Figure 3.1. The T2 statistic, when multiplied by n/(n — I) 2 in a baseline data set, maps the observation vector X into a value on the unit interval (0,1). This is true regardless of the distribution of X. However, multivariate normality guarantees that these transformed values are distributed as a beta variable. This result is illustrated in the histogram of T2 values given in Figure 3.2. For a nonnormal multivariate distribution to produce the same result, the number of observations contained within the bins (i.e., between contours) of the T2 histogram must be close to that produced by the MVN. Some multivariate distributions appear to satisfy this condition, as was the case with the truncated normal example. What characteristics will these nonnormal distributions have in common with the MVN? The answer lies in investigating the sampling distribution of the T2 statistic.
3.4
The Sampling Distribution of the T2 Statistic
Consider a random sample of size n taken from a multivariate population with a finite mean vector JJL and covariance matrix E. When the assumption of multivariate normality is imposed, the sampling distribution of the T2 statistic can be shown to be a beta distribution when a baseline data set is used or, if appropriate, a
38
Chapter 3. Checking Assumptions for Using a T Statistic
Figure 3.6: Bivariate skewed distribution.
Figure 3.7: Contours of skewed distribution. chi-square or F distribution. However, when this MVN assumption is invalid, the sampling distribution of the T2 statistic must be approximated. The kurtosis of the underlying multivariate distribution plays an important role in approximating the sampling distribution of the T2 statistic. To understand this
3.4.
The Sampling Distribution of the T2 Statistic
39
role, consider first the kurtosis, denoted by 0:4, for a imivariate distribution with a known mean fj, and a known standard deviation a. The kurtosis is usually defined as being the expected value of the fourth standard moment, i.e.,
This is a measure of the "peakedness" or "heaviness" of the tail of a distribution. For example, the univariate bell-shaped normal distribution has a kurtosis value of 3. Using this value as a point of reference, the kurtosis values for other distributions can be compared to it. If a distribution has a kurtosis value that exceeds 3, it is labeled "peaked" relative to the normal, and if its kurtosis value is less than 3, it is labeled "flat" relative to the normal. For example, the univariate exponential distribution has a kurtosis value of 9, whereas the univariate uniform and beta distributions have kurtosis values less than 3. We conclude that the uniform and beta distributions are "flatter" than the normal distribution and that the exponential distribution is more "peaked." The gamma, lognormal, and weibull distributions have kurtosis values around 3 and thus are "peaked" similarly to the normal distribution. In a multivariate distribution, the kurtosis measure is closely related to the sampling distribution of the T2 statistic. Consider a ^-dimensional vector X, with a known mean vector /j, and a known covariance matrix E, and assume a T2 statistic is computed using this vector. The first moment of the T2 sampling distribution is given as E[T2} = p. The second moment, E [ ( T 2 ) 2 ] , can be expressed in terms of Mardia's kurtosis statistic, denoted as /?2,p (see Mardia, Kent, and Bibby (1979)), and associated with the multivariate distribution of the original observation vector X. The result is given as
When X follows an MVN distribution, Np(p,, £), the kurtosis value in (3.3) reduces to where T2 is based on the formula given in (2.13). For a sample of size n, taken from a p-dimensional multivariate distribution with an unknown mean vector and covariance matrix, the sample kurtosis, &2,p, is used to estimate /?2,p- The estimate is given by
where T2 is based on the formula given in (2.15). The relationship in (3.5) indicates that large T2 values directly influence the magnitude of the kurtosis measure. We can use the above results to relate the kurtosis value of a multivariate nonnormal distribution to that of an MVN distribution. As an example, consider two uncorrelated variables, (x\,x-2), having a joint uniform distribution represented by a unit cube. Both marginal distributions are uniform and have a kurtosis value of 1.8. Thus, these are "very flat" distributions relative to a univariate normal.
Chapter 3. Checking Assumptions for Using a T2 Statistic
40
Table 3.1: Comparison of kurtosis values. No. of Uniform Variates
No. of Exponential Variates
1 2 3 4 5 6
1 1 1 1 1 1
Kurtosis Value for Multivariate Nonnormal Distribution 12.8 18.6 26.4 36.2 48.0 61.8
Kurtosis Value for Multivariate Normal Distribution 8 15 24 35 48 63
This "flatness" carries over to the joint distribution of the two variables. Using (3.4), the kurtosis of the bivariate normal, with known parameters, is given by p(p + 2) = 2(4) = 8. The kurtosis value for the joint uniform is found by evaluating (3.3), where // = (0.5,0.5) and E is a diagonal matrix with entries of (1/12) on the diagonal. The value is calculated as 5.6. This implies that a bivariate uniform distribution is considerably "flatter" than a bivariate normal distribution. In contrast to the above, there are many combinations of distributions of p independent nonnormal variables that can produce a multivariate distribution that has the same kurtosis value as a p-variate normal. For example, consider a multivariate distribution composed of two independent variables: x\ distributed as a uniform (0,1) and x% distributed as an exponential (i.e., f ( x ) = e~x for x > 0 and zero elsewhere). Using (3.3), the kurtosis of this bivariate nonnormal distribution is 12.8. In comparison, the kurtosis of a bivariate normal distribution is 8. Thus, this distribution is heavier in the tails than a bivariate normal. However, suppose we keep adding another independent uniform variate to the above nonnormal distribution and observe the change in the kurtosis value. The results are provided in Table 3.1. As the number of uniform variables increases in the multivariate nonnormal distribution, the corresponding kurtosis value of the MVN distribution approaches and then exceeds the kurtosis value of the nonnormal distribution. Equivalence of the two kurtosis values occurs at the combination of five uniform variables and one exponential variable. For this combination, the tails of the joint nonnormal distribution are similar in shape to the tails of the corresponding normal. The result also implies that the T2 statistic based on this particular joint nonnormal distribution will have the same variance as the T2 statistic based on the MVN distribution. This example indicates that there do exist combinations of many (independent) univariate nonnormal distributions with the same kurtosis value that is achieved under an MVN assumption. For these cases, the mean and variance of the T2 statistic based on the nonnormal data are the same as for the T2 statistic based on the corresponding normal data. This result does not guarantee a perfect fit of the T2 sampling distribution to a beta (or chi-square or F) distribution, as this would require that all (higher) moments of the sampling distribution of the T2 statistic be identical to those of the corresponding distribution. However, such agreement of
3.5.
Validation of the T2 Distribution
41
the lower moments suggests that, in data analysis using a multivariate nonnormal distribution, it may be beneficial to determine if the sampling distribution of the T2 statistic fits a beta (or chi-square or F) distribution. If such a fit is obtained, the data can then be analyzed as if the MVN assumption were true.
3.5
Validation of the T2 Distribution
A popular graphical procedure that is helpful in assessing if a set of data represents a reference distribution is a quantile-quantile (Q-Q) plot (e.g., see Gnanadesikan (1977) or Sharma (1995)). Thus, this technique can be used in assessing the sampling distribution of the T2 statistic. We emphasize that the Q-Q plot is not a formal test procedure but simply a visual aid for determining if a set of data can be approximated by a known distribution. Alternatively, goodness-of-fit tests can also be used for such determinations. For example, with baseline data, we could construct a Q-Q plot of the ordered sample values, denoted by x^ = [n/(n — 1)2]T?, against the corresponding quantiles, (i), of the reference distribution. If [n/(n — 1)2]T2 can be described by the appropriate beta distribution, /3[ p /2,(n-p-i)/2]> the beta quantiles are computed from the following integral equation:
If a Q-Q plot of the data results in an approximately straight line, it can be concluded that the distribution of the data is not different from the reference distribution. A straight line, with a slope of 1 and an intercept of 0, indicates an excellent fit of the empirical distribution to the hypothesized distribution. A straight-line plot with an intercept different from zero indicates a difference in location for the two distributions, while a distinct curve in the plot suggests a difference in variation. As illustrated in later chapters, the Q-Q plot can provide an additional method of locating atypical observations. To demonstrate the Q-Q plot for use in determining if the T2 statistic follows a beta distribution, we generate a sample of size n = 54 from a bivariate normal distribution. The corresponding T2 values are computed and converted to beta values using (3.1). A Q-Q plot corresponding to a beta distribution, with parameters equal to 1 and 26, is constructed and presented in Figure 3.8. The plot displays an approximate linear trend, along a 45-degree line, though the last five to six points are slightly below the projected trend line. This pattern suggests that the T2 data for this sample follow a beta distribution, a conclusion that is expected given that the underlying data were generated from a bivariate normal distribution. From Table 3.1, we noted that the joint distribution of five independent uniform variables with an additional independent exponential variable has the same kurtosis value as an MVN with six variables. The mean and variance of the corresponding T2
42
Chapter 3. Checking Assumptions for Using a T2 Statistic
Figure 3.8: Q-Q plot of generated bivariate normal data.
Figure 3.9: T chart based on simulated data from nonnormal distribution.
sampling distributions are also equal. Under the MVN assumption, the T2 statistic follows a chi-square distribution with p degrees of freedom. We now illustrate the appropriateness of the same chi-square distribution for the T2 statistic generated from the nonnormal distribution. Two hundred observations for five independent uniform variables and one independent exponential are generated. The T2 chart for the 200 observations are presented in Figure 3.9, where UCL = 16.81 is based on a = 0.01. The corresponding Q-Q plot using chi-square quantiles is presented in Figure 3.10. Our interest lies in the tail of the distribution. For a = 0.01, we would expect two values greater than the chi-square value of 16.81. Although two T2 values in Figure 3.9 are near the UCL, the T2 chart indicates no signaling T2 values. These two large T2 values are located in the extreme right-hand corner of the Q-Q plot in Figure 3.10. The Q-Q plot follows a linear trend and displays little deviation from
3.5. Validation of the T2 Distribution
43
Figure 3.10: Q-Q plot based on simulated data from nonnormal distribution.
Figure 3.11: T2 chart based on simulated data from nonnormal distribution. it. Thus, the T2 values appear to follow a chi-square distribution despite the fact that the underlying multivariate distribution is nonnormal. In contrast, consider a bivariate nonnormal distribution based on one independent uniform and one independent exponential variable. From Table 3.1, it is noted that the kurtosis of this distribution is larger than that of a bivariate normal. This implies a heavier tail (i.e., more large T2 values) than would be expected under normality. We generate 200 observations from this nonnormal distribution and construct the T2 chart presented in Figure 3.11 using UCL = 9.21 and a = 0.01. With this a, we would expect to observe two signals in the chart. However, there are five signaling T2 values, indicating the heavy tailed distribution expected by the high kurtosis value. The corresponding Q-Q plot for this data is presented in Figure 3.12. Note the severe deviation from the trend line in the upper tail of the plot of the T2 values.
44
Chapter 3. Checking Assumptions for Using a T2 Statistic
Figure 3.12: Q-Q plot based on simulated data from nonnormal distribution.
Figure 3.13: Scatter plot of bivariate data.
The conclusion is that a chi-square distribution does not provide a good fit to these data. To illustrate the appropriateness of the use of the beta distribution to describe the sampling distribution of the T2 statistic, we consider 104 bivariate observations taken from an actual industrial process. A scatter plot of the observations on the variables x\ and x 2 is presented in Figure 3.13. Observe the elongated elliptical shape of the data swarm. This is a characteristic of the correlation, r = —0.45, between the two variables and not of the form of their joint distribution. Observe also the presence of one obvious outlier that does not follow the pattern established by the bulk of the data. The presence of outliers poses a problem in assessing the distribution of the T2 statistic. Thus, we must remove the three outliers and recompute the T2 values of the remaining data. These remaining values are plotted in the T2 chart given in Figure 3.14. There remain some large T2 values associated with some of the observations, but none are signals of out-of-control points. Observations of this
3.5. Validation of the T2 Distribution
45
Figure 3.14: T2 chart of bivariate data.
Figure 3.15: Q-Q plot of T2 values of bivariate data.
type (potential outliers) are not removed in this example, although they could possibly affect the fit of the corresponding beta distribution. The corresponding Q-Q plot for this data is presented in Figure 3.15. Since p — 2 and n = 103, the beta distribution fit to the data, using (3.1), is 5^ 50 ). From inspection, the Q-Q plot exhibits a very strong linear trend that closely follows the 45° line imposed on the plot. This indicates an excellent fit between the T2 sampling distribution and the appropriate beta distribution. A question of interest is whether the above beta distribution is describing the process data because the actual observations follow a bivariate normal or because the fit provides a good approximation to the sampling distribution. To address this question, we examine estimates of the marginal distributions of the individual variables x\ and x<±. If the joint distribution is bivariate normal, then each marginal
46
Chapter 3. Checking Assumptions for Using a T2 Statistic
Figure 3.16: Histogram for x\.
Figure 3.17: Histogram for x%.
distribution must follow a univariate normal distribution. Since histograms of the individual variables provide a good indicator of the marginal distributions, these are presented in Figures 3.16 and 3.17. Neither variable appears to have a distribution similar to the normal curves superimposed on the histograms and based on using the means and standard deviations of the individual variables. If the marginal distributions were normal, we would expect agreement between these normal curves and the corresponding histograms. However, the distribution of variable x\ appears to be flat and skewed to the left, while the distribution of x% appears to be somewhat flat and skewed to the right. This is supported by a kurtosis value of 2.68 for the distribution of xi, and a value of 2.73 for the distribution of x% (relative to a value of 3 for a normal distribution). From these observations, we conclude that the joint distribution of the two variables is nonnormal. The good fit of the beta distribution to the data
3.6. Transforming Observations to Normality
47
appears to be due to its excellent approximation to the sampling distribution of the T2 statistic.
3.6
Transforming Observations to Normality
The closer that observations are to fitting a MVN distribution, the better the beta distribution will be in describing the behavior of the T2 statistic. For this reason, we examine, in terms of the MVN, some major reasons why goodness-of-fit test procedures might lead to rejection of the appropriate T2 distribution. The MVN distribution is symmetric about its mean and has long thin tails away from its mean. Thin tails imply that the concentration of probability drops off very rapidly as the statistical distance of the data from the mean increases. Thus, the number of points within the contours will diminish as the distance from the mean point increases. Also, symmetry of the MVN distribution implies a symmetric dispersion of points within the contours. When a beta distribution for the T2 values is rejected, it may be a result of the fact that the data are from a nonnormal distribution having (true) contours that are nonelliptical in shape (e.g., see Figure 3.7). This can occur when the tails of the distribution become thick in one or more directions. It also may be due to the fact that the distribution of the data is skewed in one direction so that the points are not symmetrically dispersed within the contours. Outliers can be a problem in that they can distort the estimate of the mean vector as well as of the covariance matrix. This in turn may lead to contours that are odd-shaped and off-center. Under the assumption of multivariate normality, the probability functions governing the individual variables for an MVN are univariate normal. Also, any subset of variables following an MVN is jointly normally distributed, and any linear combination of the variables is univariate normal. Unfortunately, the converses of these statements are not true. For example, it is possible to construct data sets that have marginal distributions that are normally distributed, yet the overall joint distribution can be non-normal. Nevertheless, although single-variable normality does not guarantee overall multivariate normality, it does improve the overall symmetry of the joint distribution, and this is an important issue when applying the T2 statistic. If the T2 values cannot be shown to follow a beta distribution, it may be possible to transform the individual variables so that the T2 values based on the transformed values follow a beta distribution. One simple means to achieve this might be to transform the individual variables so that each is approximately normally distributed. For example, several methods have been proposed for using the Johnson system to transform a nonnormal variate to univariate normality (e.g., see Chou, Polansky, and Mason (1998)). Tests may then be conducted to determine if the T2 based on the set of transformed variables follows a beta distribution. A similar but more complex technique for making observations more normal is to use a multivariate transformation so that the entire observation vector has an approximate MVN distribution (e.g., see Johnson and Wichern (1998), Velilla (1993)). Also, the transformation based on the above Johnson system has been extended to the multivariate case (e.g., see Johnson (1987)). The difficulty in using
48
Chapter 3. Checking Assumptions for Using a T2 Statistic
these types of transformations is in making inferences from the transformed data back to the original data.
3.7
Distribution-Free Procedures
When the beta distribution for the T2 values is rejected and cannot be used in determining the UCL for a T2 control chart, alternative procedures are needed. One simple but conservative method of determining a UCL is based on an application of Chebyshev's theorem (e.g., see Dudewicz and Mishra (1988)). The theorem states that regardless of the distribution of a;,
where A: is a chosen constant such that k > I and where p, and d1 are the mean and variance, respectively, of x. For example, the probability that a random variable x will take on a value within k = 3.5 standard deviations of its mean is at least 1 — l/k2 = 1 — l/(3.5) 2 = 0.918. Conversely, the probability that x would take on a value outside this interval is no greater than l/k2 = 1 — 0.918 = 0.082. To use the Chebyshev procedure in a T2 control chart, calculate the mean, T, and the standard deviation, s^, of the T2 values obtained from the HDS. Using these as estimates of the parameters HT and a? of the T2 distribution, an approximate UCL is given as The value of k is determined by selecting a, the probability of observing values of x outside the interval bounded by [i±k<j, and solving the equation
Another simple method for estimating the UCL of a T2 statistic is based on finding a distribution-free confidence interval (CI) for the UCL. This approach uses the fact that the UCL represents the (1 —a)th quantile of the T2 distribution, where a is the false alarm rate. A brief summary of the approach is given in this chapter's appendix (section 3.11); also, while Conover (2000) gave a detailed discussion of both one-sided and two-sided CIs for the pth quantile of a distribution. A third method of obtaining an approximate UCL is to fit a distribution to the T2 statistic using the kernel smoothing technique (e.g., see Chou, Mason, and Young (2001)). We then can estimate the UCL using the (1 — a)th quantile of the fitted kernel distribution function of the T2. An example of a fitted set of data and the corresponding smoothed distribution function is illustrated in Figure 3.18. The kernel smoothing approach can provide a good approximation to the UCL of the T2 distribution provided the sample size is reasonably large (i.e., n > 250). However, the estimate will be biased. A detailed example using this method to remove outliers is provided in Chapter 5. For smaller samples, other approaches must be utilized.
3.8. Choice of Sample Size
49
Figure 3.18: Kernel smoothing fit to T histogram.
3.8
Choice of Sample Size
Many different variables need to be examined in a multivariate process, and many different parameters need to be estimated. These requirements result in the need for large sample sizes. For example, in a steam turbine study, the variables of interest might include temperature, pressure, feed flow, and product flow. Due to differences in the standard deviations of these variables, each individual variable might require a separate estimated sample size to achieve a given precision. The problem, then, is to combine these different estimates into a single value of n. One useful solution to this problem is to use the largest estimated sample size for the individual variables as the overall sample size (see Williams (1978)). However, we need to be certain that this choice meets certain conditions. Using the T2 statistic requires a sample in which the number of observations n exceeds the number of variables p. If p < n, neither the inverse covariance matrix S"1 nor its estimate S~l can be computed. However, this is a minimal requirement. In addition, a large number of parameters must be estimated when the mean vector and covariance matrix are unknown. To provide stable and accurate estimates of these parameters, n must be sufficiently large. This occurs because there are p means, p variances, and [p(p — l)/2] covariances, or a total of 2p + [p(p — l)/2] parameters, to be estimated. For large p, this number can be significant. For example, for p = 10 we must estimate 65 parameters, while for p = 20 the number increases to 230. From the above discussion, one can see that the sample size for a multivariate process can be large, particularly when p is large, as there are many parameters to estimate. Other considerations also govern the choice of n. For example, in choosing a preliminary sample for a steam turbine system one might want to include observations under normal operational conditions. To obtain an adequate profile of ambient conditions would require observations from an entire year of operation. For
50
Chapter 3. Checking Assumptions for Using a T2 Statistic
Figure 3.19: Scatter plot with a discrete variable. replication, one would also need multiple observations at various loads (megawatt production) for the different temperatures. An alternative solution to choosing a large sample size is to seek to reduce the dimensionality of the multivariate problem. This can be achieved by reducing the number of parameters that need to be estimated. One useful solution to this problem involves applying principal component analysis (Jackson (1991)).
3.9
Discrete Variables
The MVN distribution is generally used to describe observations on a set of continuous random variables. These include variables that can take on all possible values in a given interval. However, this does not imply that all components of the observation vector must be continuous variables and that discrete variables are prohibited. For example, consider the plot of a hypothetical bivariate sample of data taken on shoe size and body weight that is given in Figure 3.19. The data are concentrated at the different shoe sizes and these occur only at discrete values. With the assumption of a bivariate normal distribution, we would obtain the elliptical region that is superimposed on the plot. In this example, the T2 statistic is usable because there are several different categories for the discrete variable.
3.10
Summary
Fundamental to the use of any statistic as a decision-making tool is the probability function describing its behavior. For the T2 statistic, this is either the chi-square, the beta, or the F distribution. Multivariate normal observation vectors are the basis for these distributions. Since multivariate normality is not easily validated,
3.11. Appendix: Confidence Intervals for UCL
51
an alternative approach is to validate the distribution of the T2 statistic. We offer a number of procedures for accomplishing this task. A Q-Q plot of the ordered T2 values plotted against the quantiles of the beta (or chi-square, or F) distribution is a useful graphical procedure for validating the T2 distribution. Strong linearity of the plotted points suggests that the observed T2 values can be described by the reference distribution. If graphical procedures are too imprecise, we recommend performing a goodness-of-fit test on the observed T2 values. Many of the common goodness-of-fit tests can be used for this purpose. If the above tests indicate that the T2 values do not provide a good fit to the required beta, or chi-square, or F distribution, several alternatives are available for finding the required UCL to use in the T2 control chart. One approach is to transform the individual nonnormal variables to normality. If such transformations can be found, the observed T2 values could then be retested for the appropriate fit. Other procedures involve estimating the UCL using a nonparametric approach based on kernel smoothing, the quantile technique, or Chebyshev's inequality. Sample size selection is always a concern when performing a multivariate analysis. For the T2 control chart, this means having enough observations to estimate the unknown parameters of the mean vector and covariance matrix. As the number of parameters increases, the required sample size can become quite large. This is a restriction that must be considered in any application of the T2methodology.
3.11
Appendix: Confidence Intervals for UCL
Suppose T2^ < T22. < • • • < T,2 N represent n ordered sample T2 values and let a = 0.01. Also let Qp represent the pih quantile of the T2 distribution. A 1007% CI for Qo.gg (the UCL) is given by [T(2r),T(2s)] and satisfies
where r and s are determined by the inequality
and s — r is a minimum. For large n, one may approximate r and s by the two values where 2( 7 /2) is the upper 7/2th quantile of the standard normal distribution. The CIs obtained from (A3.1) and (A3.2) are generally very similar when n is large and 7 ~ 0.95. We choose to use the inequality in (A3.1) to obtain r and s. From (A3.1), one can be at least 1007% sure that the UCL is somewhere between TV2 N) and T\2 l-,. Since there is an infinite number of values between T,\2 ^) and T/\2 )^ , there are infinitely many choices for the UCL. For convenience, we choose the midpoint of the interval as an approximate value for the UCL. It is given by b
r
s
This page intentionally left blank
Chapter 4
Construction of Historical Data Set
Old Blue: The Research Center You enter the research center and immediately start your search for information on SPC. You are amazed at the number of books on univariate process control. Here is one by Montgomery and another by Ryan. However, none of these books appears to address multivariate process control in the detail that you require. You decide to change your approach. Using the key word "multivariate, " you again search for textbooks. Here is one by Anderson, on multivariate analysis, that might contain some helpful information, but it is at the graduate level. Ah, here is one titled, Multivariate Statistical Process Control with Industrial Applications. Could this contain the solution to the problem? A research center associate quickly secures you a copy and you sit down to read it. You are encouraged that the preface states the book is written for the practitioner and from an engineering perspective. You are quickly absorbed in reading and the next few hours slip away. Soon the research associate is asking if you need anything before the lunch break. Your response is, "Yes, I need a personal copy of this book." Back in the office, while you are pondering what you have learned, you hear the boss asking, "Any progress?" With confidence restored, you respond with a very firm "Yes." To solve the problem with Old Blue, you must compare the new data containing upset conditions to past data taken when the process was in control. The in-control data is used to create a baseline or HDS. This was explained in the book as a Phase I operation. This is no different from what you were taught in univariate process control by . . . what was his name? However, the book contains two chapters on this topic. This calls for closer examination. The monitoring of new operational data to ascertain if control is being maintained is referred to as a Phase II operation. You continue thinking about what you have read as you open your lunch bag.
53
54
Chapter 4. Construction of Historical Data Set The statistic used to make the needed comparison is a Hotelling's T2. You twice read the section explaining how this is done. The theory is complex, but from an intuitive point of view, you now understand how multivariate control procedures work. A T2 statistic, the multivariate analogue of a common t-statistic, can assess all 35 variables at the same time. It is written as a quadratic form in matrix notation. You never appreciated that course in matrix algebra until now. It is all very amazing. Suddenly, you realize you still have a most serious problem. How is all of this computing to be done? You can't do it with your favorite spreadsheet without spending days writing macros. How was it done in the text? What software did they use in all of their data examples? A quick search provides the answer, QualStat™, a product of InControl Technologies, Inc. You note that a CD-ROM containing a demonstration version of this program is included with the book. (This chicken salad sandwich is good. You must remember to tell the cafeteria staff that their new recipe is excellent.) Following the instructions on your computer screen, you quickly upload the software. Now, you are ready to work on a Phase I operation and create an HDS.
4.1
Introduction
An in-control set of process data is a necessity in multivariate control procedures. Such a data set, often labeled historical, baseline, or reference, provides the basis for establishing the initial control limits and estimating any unknown parameters. However, the construction of a multivariate HDS is complicated and involves problem areas that do not occur in a univariate situation. It is the purpose of this chapter to explore in detail some of these problem areas and offer possible solutions. The development of the HDS is referred to as a Phase I operation. Using it as a baseline to determine if new observations conform to its structure is termed a Phase II operation. Since there is only one variable to consider, univariate Phase I procedures are easy to apply. Upon deciding which variable to chart, one collects a sample of independent observations (preliminary data) on this variable from the in-control process. The resulting data provide initial estimates of the parameters that characterize the distribution of the variable of interest. The parameter estimates are used to construct a preliminary control procedure whose major purpose is to purge the original data set of any observations that do not conform to the structure of the HDS. These nonconforming or atypical observations are labeled outliers. After the outliers are removed from the preliminary data set, new estimates of the parameters are obtained and the purging process is repeated. This is done as many times as necessary to obtain a homogeneous data set as defined by the control procedure. After all outliers are removed, the remaining data is referred to as the HDS. The role of a multivariate HDS is the same as in the univariate situation. It provides a baseline for the control procedure by characterizing the in-control process. However, construction of a historical data set becomes more complicated when using multivariate systems. For example, we must decide which variables to include and their proper functional forms. This determination may require in-depth process
56
Chapter 4. Construction of Historical Data Set
Planning Establish Goals Study and Map Process Define Good Process Operations Collect Preliminaey Data Set Verify Data Quality Filter Data
Variable Form Thoreticel Relationships Empirical Relationships Transformations
Collection Procedures Human Errors Electronic Errors
Missing Data Estimation Deletion
Detecting Data Problems
Collinearity Effects Detection & Removal
Autocorrelation Effects Detection Outliers Detection Purging Process Alternative Covariance Estimators
Construct Final Historical Data Set
Chart 4.1: Guidelines for constructing an HDS.
4.3. Preliminary Data
57
• The production of caustic soda and chlorine gas is a major industry in the United States. One method of production is through the electrolysis of brine (i.e., salt water). This work is done in an electrolyzer that is composed of one or more cells. A cell is the basic unit where the conversion takes place. The major purpose of a control procedure is to locate cells whose conversion efficiency has dropped so that they can be treated to restore their efficiency • The brine (feed stock) for an electrolyzer must be treated to remove impurities. This operation takes place in a brine treatment facility. The primary purpose of a control procedure on this unit is to maintain the quality of the feed stock for the electrolyzer. "Bad" brine has the potential of destroying the cells and contaminating the caustic soda being produced. • From the electrolyzer, caustic soda is produced in a water solution. The water is removed through evaporation in an evaporating unit. A control procedure on this unit maintains maximum production for a given set of run conditions, maintains the desired caustic strength, and helps locate sources of problems. • One method of transporting the finished caustic product is by railroad tank cars. Overweight tank cars present a number of major problems. Control procedures on the loading of the tank cars can ensure that no car will be loaded above its weight limit. • Control procedures on steam and gas turbines, used in the production of electricity for the electrolysis of the brine, detect deviations in the efficiency of the turbines. Also, they are used to locate sources of problems that occur in operations. Boilers, used in steam production for the evaporation of water, are controlled in a similar fashion. • Control procedures on large pumps and compressors are used for maintenance control to detect any deviation from a set of "ideal" run conditions. It is less expensive to replace worn parts than to replace a blown compressor. • Control procedures on various reactors are used to maintain maximum efficiency for a given set of run conditions and to locate the source of the problem when upsets occur.
4.3
Preliminary Data
The first step in the development of an HDS is to obtain a preliminary data set. This is a sample of observations taken from the process while it is "in control." However, "in control" must be defined. For example, the purpose of the desired control procedure may be to keep the process on target or to minimize some unwanted variation in production. How is a sample containing the desired information obtained? Generally, you are presented with a block of process data collected during a given time period. The data may be unfiltered and may contain information taken when the process was in control as well as out of control. In addition, the resultant sample may include data from different operational levels, different product formulations, different feed stocks, etc. The data may provide a genuine history of the process over the given time period but may not be in the desired form necessary to construct an HDS.
Chapter 4. Construction of Historical Data Set
58
Table 4.1: Chemical process data. Obs. No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Xi 2020 2020 2014 1960 1870 1800 1711 1800 2011 1875 2099 2175 1226 1010 2041 2040 2330 2330 2250 2250 2351 2350 1977 2125 2033 1850 1904 1950 1795 2060
X2 165 255 266 270 185 195 201 250 182 175 252 270 216 180 192 225 131 160 241 195 177 135 181 200 189 150 103 125 290 240
X3
661 675 675 900 850 590 663 875 710 600 566 535 495 520 692 700 483 600 523 480 679 640 705 830 830 800 970 670 629 590
X4 5.90 7.00 7.00 3.00 4.00 3.00 4.00 0.00 2.40 4.30 5.70 4.50 0.00 0.00 6.80 7.50 7.40 5.50 7.90 6.50 1.80 3.30 6.70 6.50 6.50 5.80 0.00 6.50 5.10 6.00
X5 1.60 5.00 3.90 8.00 4.80 4.00 3.60 6.00 7.70 6.00 7.20 10.00 9.60 8.00 9.40 10.00 7.90 8.00 6.20 8.50 7.90 5.50 3.30 4.50 5.90 5.00 4.10 4.00 5.70 4.00
X6
0.28 0.26 0.27 0.27 0.30 0.30 0.29 0.33 0.29 0.29 0.35 0.33 0.29 0.31 0.28 0.32 0.29 0.26 0.31 0.31 0.32 0.28 0.29 0.31 0.32 0.32 0.24 0.29 0.35 0.32
X7 0.56 0.61 0.60 0.65 0.62 0.52 0.61 0.59 0.59 0.57 0.68 0.60 0.56 0.58 0.60 0.64 0.62 0.53 0.58 0.60 0.61 0.58 0.60 0.59 0.60 0.64 0.51 0.52 0.63 0.65
*8
86 84 85 89 90 87 88 88 86 97 86 86 86 92 92 90 88 92 90 90 94 95 87 90 90 92 93 88 90 88
To understand these concepts, consider the data given in Table 4.1. It consists of a sample of 30 observations taken on eight variables, (J^i, X < 2 , . . . , Xg), measured on a chemical process. It is assumed at this stage of data investigation that a decision has been made as to the purposes and types of control procedures required for this process. Suppose it is desired to construct a control procedure using only the observations on the first seven variables presented in Table 4.1. Further, suppose it is most important to maintain variable X^ above a critical value of 5. Any drifting or changes in relationships of the other process variables from values that help maintain X^ above its critical value need to be detected so that corrective action can be taken. Initially, the data must be filtered to obtain a preliminary data set from which the HDS can be constructed. There are 17 observations with X± above its critical value of 5. The obvious action is to sort the data on X± and remove all observations in which X^ has a value below its critical value. This action should produce a set of data with the desired characteristics.
4.3. Preliminary Data
59
Table 4.2: Group means of first seven variables. Group 1 2
% Diff
Xi 1849 2070 0.7
X2
195 201
-3.1
X3
694 657 5.6
X5 X4 2.33 6.55 6.48 6.00 -6.4 9.1
X6 .295 .300 -1.7
X7 .583 .600 -2.8
Figure 4.1: Time-sequence plot of megawatts and fuel usage.
The filtering of data can provide valuable process information. For example, suppose the out-of-specification runs (i.e., X^ < 5) are labeled Group 1 and the inspecification runs (i.e., X^ > 5) are labeled Group 2. The means of the variables of the two groups are presented in Table 4.2. Valuable process information is obtained by closely examining the data. A mean difference on variable 4 is to be expected since it was used to form the groups. However, large percentage differences in the means are observed on variables 1 and 5, and a moderate difference is observed on variable 3. Further investigation is needed in determining how these variables are influencing variable 4. A preliminary data set should be thoroughly examined using both statistical procedures and graphical tools. For example, consider the graph presented in Figure 4.1 of fuel consumption, labeled Fuel, and megawatt-hours production, labeled Megawatts (or MW), of a steam turbine over time (in days of operation). Close examination of the graph produces interesting results. Note the valleys and peaks in the MW trace. These indicate load changes on the unit, whereas the plateaus reflect production at a constant load. When the load is reduced, the MW usage curve follows the fuel graph downward. Similarly, the MW graph follows the fuel graph upwards when the load is increased. This trend indicates there is a lag,
Chapter 4. Construction of Historical Data Set
60
Table 4.3: Data for lag comparison. Obs. No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
X
7.84 9.14 9.20 9.10 9.21 8.62 6.78 9.14 9.15 9.13 9.29 8.95 9.09 9.18 8.33 8.63 4.32 8.74 8.91 8.70 9.20 9.18 8.76 6.25 5.18 5.94 6.13
y
116.93 117.45 118.04 111.40 85.96 117.24 117.42 117.66 118.42 116.84 116.50 116.76 108.05 90.34 79.01 113.24 115.50 111.92 118.54 118.51 119.25 82.01 68.89 77.76 81.05 101.77 111.15
ylagl 4 116.93 117.45 118.04 111.40 85.96 117.24 117.42 117.66 118.42 116.84 116.50 116.76 108.05 90.34 79.01 113.24 115.50 111.92 118.54 118.51 119.25 82.01 68.89 77.76 81.05 101.77 111.15
during a load change, in the response time of the turbine to the amount of fuel being supplied. This is very similar to the operation of a car, since accelerating or decelerating it does not produce an instantaneous response. Lags in only part of the data, as seen in the example in Figure 4.1, often can be easily recognized by graphical inspection. Other methods, however, must be used to detect a lag time extending across an entire processing unit. Observations across a processing unit are made at a single point in time. Before one can use the observations in this form, there must be some guarantee that the output observations match the input observations. Otherwise, the lag time must be determined and the appropriate parts of the observation vector shifted to match the lag. Some processes, such as strippers used to remove unwanted chemical compounds, work instantaneously from input to output. Other processes, such as silica production, have a long retention time from input to output. Consultation with the operators and process engineers can be most helpful in determining the correct lag time of the process. A helpful method for determining if lag relationships exist between two variables is to compute and compare their pairwise correlation with the correlation between one variable and the lag of the other variable. For example, consider hourly
4.4. Data Collection Procedures
61
Table 4.4: Lag correlation with process variable.
y
ylagl ylag2 ylag3
Correlation 0.148 0.447 0.937 0.598
observations taken (at the same sampling period) on two process variables. A sample of size 27 is presented in Table 4.3, where the variable x is a feedstock characteristic and the variable y is an output quality characteristic. We begin by calculating the correlation between x and y. The value is given as 0.148. Next we lag the y values one time period and reconstruct the data set, as presented in Table 4.3. The resulting variable, labeled ylagl, has a correlation of 0.447 when compared to the x variable. We note an increase in the correlation. The observations on the quality variable y could be continuously shuffled downward until its maximum correlation with y is obtained. The correlations for three consecutive lags are presented in Table 4.4. Maximum correlation is obtained by lagging the quality characteristic two time periods. Note the decrease in the correlation for three lags of the quality characteristic. Thus, we estimate the time through the system as being two hours.
4.4
Data Collection Procedures
Careful examination of the recorded data can alleviate problems occurring in data acquisition. For example, data are recorded in many ways within processing units. At one extreme, information may be manually tallied on log sheets, whereas, in more modern units, data may be recorded electronically. Problems have occurred using both methods. Consider data recorded by operators using log sheets. Missing data frequently occur, as the operator is often busy doing something else at the appropriate sampling time and fails to accurately record an observation. Transcribing errors also may be prevalent. If problems such as these occur, they must be resolved before the control procedure is implemented. If observations are costly to obtain, missing data can become a serious problem. Electronic data collectors may eliminate problems like the ones described above, but they can produce others. For example, consider the 20 observations, on seven variables, listed in Table 4.5. The observation vectors are collected electronically at 1-minute intervals. Some of the components are chemical analysis determinations obtained using a separate piece of equipment. Since the recycle times of the different pieces of equipment vary, the same variables are being recorded at different sampling times. For example, the piece of equipment used in recording the observation on variable X-j has a two-minute recycle time. Note that observations 12, 13, and 14 all have the same value of 0.27. Other observations also contain repeats on this variable. To construct a preliminary data set from observations such as these, one
Chapter 4. Construction of Historical Data Set
62
Table 4.5: Electronically gathered data. Time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Xi 138.21 135.72 131.72 128.24 122.96 121.91 118.21 121.04 124.53 131.47 131.60 116.98 122.94 128.78 132.21 134.87 134.64 134.38 131.36 120.79
X2
X3
1.27 1.27 1.27 1.27 1.27 1.28 1.28 1.27 1.27 1.27 1.23 1.27 12.7 12.7 1.27 1.27 1.28 1.27 12.6 1.27
5762.99 5762.99 5763.87 5763.87 5762.11 5762.99 5763.87 5763.87 5763.87 7009.57 7009.57 6232.03 6233.50 6234.38 6233.50 6234.38 6233.50 6234.28 6234.38 5822.46
X4
158.81 154.20 151.13 149.81 152.67 175.08 192.57 204.29 203.34 179.13 160.42 162.14 148.67 141.55 141.13 141.55 166.27 182.23 194.50 200.91
X5
X6
X7
2327.05 2327.05 2327.93 2327.05 2327.05 2327.05 2312.11 2312.11 2312.07 2312.70 2312.70 2076.86 2077.73 2077.73 2075.98 2253.22 2254.10 2253.22 2253.22 2253.22
0.35 0.35 0.34 0.34 0.33 0.33 0.30 0.29 0.28 0.27 0.26 0.24 0.24 0.24 0.28 0.26 0.25 0.23 0.22 0.20
0.16 0.16 0.16 0.16 0.16 0.16 0.16 0.16 0.16 0.16 0.16 0.27 0.27 0.27 0.05 0.05 0.05 0.05 0.05 0.05
needs to examine closely how each observation is determined and the recycle time of each piece of equipment. Another problem area in data collection includes incorrect observations on components. This may be a result of faulty equipment such as transistors and temperature probes. Problems of this type can be identified using various forms of data plots or data verification techniques (usually included in the purging process).
4.5
Missing Data
Missing or incorrect observations on variables can occur for numerous reasons, including operator failure, equipment failure, or incorrect lab determinations. The simplest approach would be to delete the observations or variables with the missing data. However, this method is valid only if the remaining data is still a representative sample from the process and if the cause of the missing data is unrelated to the values themselves. Otherwise, the problem of incomplete data is complex and could have several different solutions. One helpful estimate of a missing value, when using the T2 as the control statistic, is the mean of that variable, conditioned on the observed values of the other vector components. This is simply the predicted value of the variable based on the regression of the variable on the other remaining variables. This value should have little or no influence on the other components of the data vector and minimum effect on the parameters to be estimated. When substituted into the data vector, it should make the observation vector homogeneous with the other observations.
4.5. Missing Data
63
Table 4.6: Chemical process data for six variables. Time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Xi(NaOH) 134.89 129.30 145.50 143.80 146.30 141.50 157.30 141.10 131.30 156.60 135.60 128.39 138.10 140.50 139.30 152.39 139.69 130.30 132.19 134.80 142.30
X2(NaCl) 203.00 203.10 208.60 188.10 189.10 196.19 185.30 209.10 200.80 189.00 192.80 213.10 198.30 186.10 204.00 176.30 186.10 190.50 198.60 196.10 198.80
X3(Ii) 0.05 0.06 0.17 0.11 0.22 0.16 0.09 0.16 0.17 0.19 0.26 0.07 0.15 0.30 0.25 0.19 0.15 0.23 0.09 0.17 0.09
X4(I2) 4.00 1.90 6.10 0.40 0.50 3.50 2.90 0.50 3.80 0.50 0.50 3.60 2.70 0.30 3.80 0.90 1.60 2.60 5.70 4.90 0.30
X5(C12) 98.37 98.37 98.23 98.44 98.44 98.26 98.23 98.69 97.95 97.97 97.65 98.43 98.12 98.15 98.02 98.22 98.30 98.08 98.30 97.98 98.41
X6(02) 1.17 1.17 1.42 1.12 1.11 1.35 1.40 0.86 1.64 1.62 1.94 1.23 1.36 1.37 1.54 1.30 1.25 1.37 1.16 1.50 1.00
Table 4.7: Correlation comparison. Without With
NaOH 0.042 0.050
NaCl 0.246 0.247
Ii
-0.551 -0.563
I2
-0.123 -0.158
02
-0.962 -0.970
To demonstrate this regression procedure for estimating a missing value, consider the 21 observations presented in Table 4.6. Assume this is an HDS for a chemical process and consider the highlighted observation on variable X^ (in the last row) as a missing observation. The regression equation of X§ on X\, X%, ^3, X^, arid ^g as derived from the first 20 observations is given as
and the estimated value of C12 is computed as 98.58. This is in close agreement with the actual value of 98.41 given in Table 4.6. The T2 value, with this estimate of the missing component, is 5.94 as compared to the actual value of 5.92. Thus, substituting the missing value has little influence on the T2 statistic. Similarly, there is negligible change in the mean of X$. The mean of the observations without the missing value is 98.21 versus a mean of 98.23 when the estimated value is included. A comparison of the correlations between ^5 and the other five variables, with and without the predicted value, are given in Table 4.7. There appears to be little difference between these correlations. The fill-in-the-value approach presented above is a simple and quick method for estimating missing values in an HDS. However, among its limitations are the fact
64
Chapter 4. Construction of Historical Data Set
that the estimated value is only as good as the prediction equation that produced it, and the fact that estimation may affect the variance estimates. Many other solutions exist (e.g., see Little and Rubin (1987)), and these can be used when better estimation techniques are preferred.
4.6
Functional Form of Variables
Using the originally measured variables in an HDS may not provide information about relationships between process variables in the best usable form. As stated in Chapter 2, the T2 statistic is constructed using the linear relationships existing between and among the process variables. Many situations occur where a simple transformation on one of the variables can strengthen the linearity, or correlation, among the variables. Examining scatter plots of pairs of variables can be helpful in making these decisions. For example, consider the plot of millivolts versus temperature given in Figure 4.2. The trend in the data appears to follow some form of a logarithmic or exponential relationship. However, the correlation between the variables is very strong with a value of 0.94. This is mainly due to the strong linearity between the two variables when millivolts exceed 5 in value. A downward curvature of the plot occurs toward the lower end of the scale of the millivolts axis. This curvature can be (somewhat) removed with a simple log transformation of millivolts. A plot of log (millivolt) versus temperature is presented in Figure 4.3. Although the plot still exhibits a slight curvature, the correlation between temperature and log (millivolt) has increased slightly to a value of 0.975.
Figure 4.2: Plot of relationship between millivolts and temperature.
4.7. Detecting Collinearities
65
Other methods, such as those based on theoretical knowledge about relationships between the variables, may be more helpful in substituting functional forms for some of the variables. Consulting with the process engineer can be helpful in making these decisions.
4.7
Detecting Collinearities
The formula for a T2 statistic is based on a covariance matrix that is nonsingular and can be inverted. A singular covariance matrix occurs when two or more observed variables are perfectly correlated (i.e., exactly collinear). Computer software packages used in computing the inverse of the matrix produce a warning when this results. However, most computer packages do not provide a warning when the collinearity in a matrix is severe but not exact. Our discussion in this section centers on the latter situation and how to detect and adjust for Collinearities. Several books include discussions on Collinearities and their effects on matrices (e.g., see Belsley, Kuh, and Welsch (1980) or Chatterjee and Price (1999)). Collinearities can occur in a covariance or correlation matrix because of sampling restrictions, because of theoretical relationships existing in the process, and because of outliers in the data. One method of identifying a collinearity is to examine the eigenvalues and eigenvectors of the sample covariance matrix (see subsection 4.11.1 of this chapter's appendix). However, to ease computations, one usually examines the corresponding correlation matrix since the results from this examination are equally applicable to the covariance matrix. A statistical tool that is useful in this process is a principal component analysis (PCA). This procedure can help detect the existence of a near singularity, can be used to determine subgroups of variables that are highly correlated, and can be used to estimate the dimensionality of the system. In some cases, the principal components themselves can give insight into the true nature of these dimensions.
Figure 4.3: Plot of relationship between temperature and log (millivolts).
66
Chapter 4. Construction of Historical Data Set
Figure 4.4: Schematic of diaphragm cell. A brief summary of PCA and its relationship to the eigenvalue problem is contained in subsection 4.11.2 of this chapter's appendix. The reader unfamiliar with these concepts is encouraged to review this appendix before continuing with this section. A more detailed discussion of PCA is provided by Jackson (1991). The effects of a near-singular covariance matrix on the performance of a T2 statistic will be demonstrated in the following example. We begin by expressing the inverse of the sample covariance matrix as
where AI > A2 > • • • > Ap are the eigenvalues of S and t/j, j — 1, 2 , . . . ,p, are the corresponding eigenvectors. If Xp is close to zero, the ratio (1/AP) becomes very large and can have a disproportionate effect on the calculation of the inverse matrix. This distorts any statistic, such as the T2, that uses the inverse matrix in its calculation. To demonstrate how to examine the eigenstructure of a matrix, we examine the correlation matrix of a chlorine (C^)/caustic (NaOH) production unit. A unit schematic is presented in Figure 4.4. This particular process is based on the electrolysis of brine (salt). A current is passed through a concentration of brine solution where the anode and cathode are separated by a porous diaphragm. The chlorine is displaced as a gas and the remaining water/brine solution contains the caustic. The unit performing this work is referred to as a cell, and several of these are housed together (as a unit) to form an electrolyzer. Overall performance of the cell is measured by the percentage of the available power being used in the conversion process. This percentage is a computed variable and is referred to as conversion efficiency (CE). High values of this variable are very desirable. Many variables other than CE are used as indicators of cell performance. Measured variables are the days of life of the cell (DOL), cell gases including chlorine
4.7. Detecting Collinearities
67
Table 4.8: Correlation matrix for chlorine data. NaOH 1.000 -0.013 0.218 0.037 -0.297 0.284 -0.284
NaOH NaCl Ii I2 C12
02 CE
NaCl -0.013 1.000 0.015 0.023 0.006 0.001 -0.001
Ii
0.218 0.015 1.000 0.567 -0.395 0.323 -0.324
12
0.037 0.023 0.567 1.000 -0.402 0.368 -0.369
C12
-0.297 0.006 -0.395 -0.402 1.000 -0.956 0.956
02
0.284 0.001 0.323 0.323 -0.956 1.000 -0.999
CE
-0.284 -0.001 -0.324 -0.324 0.956 -0.999 1.000
Table 4.9: Eigenvalues of correlation matrix. No. 1 2 3 4 5 6 7
Eigenvalue 3.4914 1.1513 0.9959 0.9116 0.3965 0.0534 0.0003
%of Total 49.88 16.45 14.23 13.02 5.66 0.76 0
Cum CE NaOH 02 NaCl % C12 I2 Ii 49.88 0.2073 0.0028 0.2999 0.3039 -0.5105 0.507 -0.507 0.2384 -0.2382 0.22 66.32 -0.1838 -0.6092 -0.6347 -0.1739 0.0654 -0.0655 80.55 -0.0664 0.9761 -0.1487 -0.1027 -0.0401 0.1634 0.1265 -0.1638 0.9151 0.1158 0.2557 -0.1194 93.57 99.24 0.2584 -0.0049 -0.6677 0.0231 0.0792 -0.0232 0.6928 0.3926 -0.3937 0.8278 0.0064 -0.0068 0.001 0.0745 99.99 0.7074 0.7068 0.0007 0.0001 0.0002 100 0.0001 0.0001
and oxygen (Cl? and 02), caustic (NaOH), salt (NaCl), and impurities production (Ii and 12). The levels of impurities are important since their production indicates a waste of electrical power, and they contaminate the caustic. Table 4.8 is the correlation matrix for an HDS (n = 416) based on seven of these variables. Its eigenstructure will be examined in order to determine if a severe collinearity exists among the computed variables. Inspection of the correlation matrix reveals some very large pairwise correlations. For example, the correlation between the two measured gases, C12 and O 2 , has a value of -0.956. Also, the computed CE variable, which contains both Cl2 and O2, has a correlation of 0.956 with C12 and -0.999 with O 2 . Using a PCA, the seven eigenvalues and eigenvectors for the correlation matrix are presented in Table 4.9. Also included is the proportion of the correlation variation explained by the corresponding eigenvectors as well as the cumulative percentage of variation explained. A recommended guideline for identifying a near-singular matrix is based on the size of the square root of the ratio of the maximum eigenvalue to each of the other eigenvalues. These ratios are labeled as condition indices. A condition index greater than 30 implies that a severe collinearity is present. The value of the largest index, labeled the condition number, for the data in Table 4.9 is
which clearly indicates the presence of a severe collinearity among the variables. A severe collinearity in the correlation matrix translates into the presence of a severe collinearity in the associated covariance matrix. Since it is not possible or advisable to use a T2 control statistic when the covariance matrix is singular or near singular, several alternatives are suggested. The first, and simplest to implement, is to remove one of the variables involved in the collinearity. This is especially useful
68
Chapter 4. Construction of Historical Data Set
when one of the collinear variables is computed from several others, since deletion of one of these variables will not remove any process information. To determine which variables are involved in a severe collinearity, one need only examine the linear combination of variables provided by the eigenvector corresponding to the smallest eigenvalue. From Table 4.9, this linear combination corresponding to the smallest eigenvalue of 0.0003 is given by
Ignoring the variables with small coefficients (i.e., small loadings) gives the linear relationship between the two variables that is producing the collinearity problem. This relationship is given as
This relationship confirms the large negative correlation, —0.999, found between CE and O2 in Table 4.8. The information contained in the computed variable CE is redundant with that contained in the measured variable 62- This relationship is producing a near singularity in the correlation matrix. Since CE is a computed variable that can be removed with no loss of additional information, one means of correcting this data deficiency is to compute the T2 statistic using only the remaining six variables. The revised correlation matrix for these six variables is obtained from the correlation matrix presented in Table 4.9 by deleting the row and column corresponding to CE. Another method for removing a collinearity from a covariance matrix is to reconstruct the matrix by excluding the eigenvectors corresponding to the near-zero eigenvalues. The contribution of the smallest eigenvalues would be removed and S~l would be computed using only the larger ones; i.e.,
This approach should be used with caution since, in reducing the number of principal components, one may lose the ability to identify shifts in some directions in terms of the full set of the original variables.
4.8
Detecting Autocorrelation
Most multivariate control procedures require the observation vectors to be uncorrelated over time. Unfortunately, violations of this assumption can weaken the effectiveness of the overall control procedure. Some industrial operations, such as chemical processes, are particularly prone to generating time-correlated, or autocorrelated, data. This situation can occur because of the continuous wear on
4.8. Detecting Autocorrelation
69
Figure 4.5: Continuous decay of heat transfer coefficient over time. equipment, the environmental and chemical contamination of the equipment, and the depletion of certain critical components, such as the availability of catalyst in the process. Autocorrelation, like ordinary correlation, may be the result of either a causeand-effect relationship or only of an association. If it is due to a cause-and-effect relationship, the observation on the time-dependent variable is proportional to the value of the variable at some prior time. If it is a relationship based on association, the present value of the variable is only associated with the past value and not determined by it. Why do certain types of processes have a tendency to generate observations with a time dependency? One possible answer to this very important question is that it is due to an association (correlation) with an unobservable "lurking" variable. Consider two variables that are negatively correlated, so that as one variable increases in value, the other decreases. Suppose one of the variables, the lurking one, cannot be observed and is increasing with time. Since this variable is not measurable, the time association will appear as a time dependency in the second variable. Without knowledge of the first variable, one could conclude that the second variable has a time dependency in its observations. Autocorrelated observations may occur in at least two different forms. The first we label as continuous or uniform decay, as it occurs when the observed value of the variable is dependent on some immediate past value. Certain in-line process filters, used to remove impurities, behave in this fashion. Another example is given by the measure of available heat, in the form of a heat transfer coefficient, which is used to do work in many types of processes. During a life cycle of the unit, the transfer of heat is inhibited due to equipment contamination or other reasons that cannot be observed or measured. A cycle is created when the unit is shut down and cleaned. During the cycle, the process is constantly monitored to insure maximum efficiency. Figure 4.5 is the graph of a heat transfer coefficient over a number of life cycles of a production unit.
70
Chapter 4. Construction of Historical Data Set
Figure 4.6: Stage decay of process variable over time. The second form of autocorrelated data is labeled stage decay (e.g., see Mason, Tracy, and Young (1996)). This occurs when the time change in the variable is inconsistent over shorter time periods, but occurs in a stepwise fashion over extended periods of time. This can occur in certain types of processes where change with time occurs very slowly. The time relationship comes from the performance in one stage being dependent on the process performance in the previous stage(s). The graph of a process variable having two stages of decay is presented in Figure 4.6. If autocorrelation is undetected or ignored, it can create serious problems with control procedures that do not adjust for it. The major problem is similar to the one that occurs when using univariate control procedures on variables of a multivariate process. Univariate procedures ignore relationships between variables. Thus, the effect of one variable is confounded with the effects of other correlated variables. A similar situation occurs with autocorrelated data when the time dependencies are not removed. Adjustment is necessary in order to obtain an undistorted observation on process performance at a given point in time. Control procedures for autocorrelated data in a univariate setting often make these adjustments by modeling the time dependency and plotting the resultant residuals. Under proper assumptions, these residual errors, or adjusted values (i.e., effect of the time dependency removed), can be shown to be independent and normally distributed. Hence, they can be used as the charting statistic for the time-adjusted process. It is also useful to look at forecasts of charting statistics since processes with in-control residuals can drift far from the target values (e.g., see Montgomery (1997)). We offer a somewhat similar solution for autocorrelated data from multivariate processes. However, the problem becomes more complicated. We must be concerned not only with autocorrelated data on some of the variables, but also with how the time variables relate to the other process variables. Autocorrelation
4.8. Detecting Autocorrelation
71
Figure 4.7: Process variable with cycle.
does not eliminate these relationships, but instead confounds them and thus must be removed for clear signal interpretation. How this is done is a major focus of Chapter 10. One simple method of detecting autocorrelation in univariate processes is accomplished by plotting the variable against time. Depending on the nature of the autocorrelation, the points in a graph of the process variable versus time will either move up or down or oscillate back and forth. Subsequent data analysis is used to verify the presence of autocorrelation, determine lag times, and fit appropriate autoregressive models. Observations from a multivariate process are p-dimensional and the components are usually correlated. The simple method of plotting graphs of individual components against time can be inefficient when there are a large number of variables. Also, these time-sequence plots may be influenced by other correlated variables, resulting in incorrect interpretations. For example, considering the cyclic nature over time of the variable depicted in Figure 4.7, one might suspect that some form of autocorrelation is present. However, this effect is due to the temperature of the coolant, which has a seasonal trend. Nevertheless, even with this drawback, we have found that graphing each variable over time is useful. To augment the above graphical method and to reduce the number of individual graphs for study, one could introduce a time-sequence variable in the data set and examine how the individual variables relate to it. If a process variable correlates with the time-sequence variable, it is highly probable that the process variable correlates with itself in time. Using this method, one can locate potential variables that are autocorrelated. Detailed analysis, including the graphing of the variable over time, will either confirm or deny the assertion for individual variables.
72
Chapter 4. Construction of Historical Data Set
When a variable is correlated with time, there will be dependencies between successive observations. This dependency may occur between consecutive observations, every other observation, every third observation, etc. These are denoted as different lag values. One useful technique for displaying this time dependency is to calculate the sample autocorrelation, r^, between observations a lag of k time units apart, where
The rk values (with TQ = 1) provide useful information on the structure of the autocorrelation. For example, if the fcth lag autocorrelation is large in absolute value, one can expect the time dependency at this lag to be substantial. This information can be summarized in a plot of the r^ values against the values of k. This graph of the autocorrelation function is sometimes referred to as a correlogram. More details on the use of these plots can be found in time-series books (e.g., Box and Jenkins (1976)).
4.9
Example of Autocorrelation Detection Techniques
To demonstrate the techniques of detecting autocorrelated data in a multivariate process, we will use a partial data set on a heat transfer coefficient taken from Mason and Young (1997). Many industries, especially the chemical industry, use heat as the energy source in the removal or separation of chemical components. For example, removal of water is necessary to strengthen a caustic solution. Heat available to do such work is measured by a heat transfer coefficient. Most evaporators are efficient only if the coefficient remains above a certain value. When the value drops below a critical level, the unit is cleaned, critical components are replaced, and a new cycle begins. Data consisting of 20 observations on such a unit are presented in Table 4.10. Heat transfer is one of the many variables used in the control of the process. It interests us because, as is indicated in Figure 4.8, its values appear to be linearly decreasing over time. The autocorrelation structure is illustrated with the correlogram in Figure 4.9. Of the various lag autocorrelations presented for this variable, the highest value occurs at ri = 0.9164 and corresponds to a lag of one time unit. Such a linear trend in a time plot is labeled a first-order autoregressive relationship, or simply an AR(1) relationship. Inferential statistical techniques also can be applied to confirm this result. For example, Table 4.11 contains the regression analysis-of-variance table for the fitting of an AR(1) model of the form
4.9. Example of Autocorrelation Detection Techniques
73
Table 4.10: Raw data for heat transfer example. Obs. No.
Heat Transfer
Lag Heat Transfer
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
103 103 106 106 107 105 102 103 99 99 99 98 98 97 94 99 99 96 93 92 90 91 90
106 103 103 106 106 107 105 102 103 99 99 99 98 98 97 94 99 99 96 93 92 90 91
Table 4.11: ANOVA table for AR(1) model. Regression Residual Total
df I 21 22
SS 506.304 96.912 603.217
MS 506.304 4.614
F 109.711
p- value 0.000
where b0 and 61 are the estimated coefficients of the model relating the heat transfer variable yt to its lag value yt-i- The small p-value for the F statistic in the table implies that there is strong evidence that the immediate past heat transfer coefficient is an important predictor of the current heat transfer coefficient. As another example, consider the techniques necessary for detecting autocorrelation in data collected from a reactor used to convert ethylene (C2H4) to ethylene dichloride (EDC). EDC is the basic building block for much of the vinyl product industry. Feed stock for the reactor includes hydrochloric acid gas (HC1), ethylene, and oxygen (02). Conversion of the feed stock to EDC occurs under high temperature in the reactor. The conversion process is labeled oxyhydrochlorination (OHC). There are many different types of OHC reactors available to perform the conversion of ethylene and HC1 to EDC. One type, a fixed life or fixed bed reactor, must have critical components replaced at the end of each run cycle. The components
74
Chapter 4. Construction of Historical Data Set
Figure 4.8: Time-sequence plot of heat transfer coefficient.
Figure 4.9: Correlogram for heat transfer coefficient. are slowly depleted during operation and performance of the reactor follows the depletion of the critical components. The best performance of the reactor is at the beginning of the cycle, as the reactor gradually becomes less efficient during the remainder of the cycle. While other variables have influence on the performance of the reactor, this inherent decay of the reactor produces a time dependency in many of the process and quality variables. We have chosen seven variables to demonstrate how to detect and adjust for autocorrelated data in this type of process. These are presented in Table 4.12. The first variable, RPl, is a measure of feed rate. The next four, Temperature, LI, 1/2, and 1/3, are process variables, and the last two are output variables. Variable PI is an indication of the amount of production for the reactor and variable Cl is an
4.9. Example of Autocorrelation Detection Techniques
75
Table 4.12: Process variables from an OHC reactor. Input Variables RPl
Process Variables Temp LI, L2, L3
Output Variables Cl, PI
Figure 4.10: Time-sequence plot of temperature.
undesirable by-product of the production system. All variables, with the exception of feed rate, show some type of time dependency. Temperature measurements are available from many different locations on a reactor. All of them are important elements in the performance and control of the reactor and increase over the life cycle. To demonstrate the time decay of the measured temperatures, we present in Figure 4.10 a graph of the average temperature over a good production run. The graph indicates the average temperature of the reactor is initially stable, but then it gradually increases over the life cycle of the unit. The time-sequence graphs in Figures 4.11 and 4.12 of the two process variables, L3 and LI, present two contrasting patterns. In Figure 4.12, L3 increases linearly with time and has the appearance of a first-order lag relationship. This is confirmed by the fact that r\ — 0.7533. However, the graph of LI in Figure 4.13 is depicted as a quadratic or exponential across time, but can still be approximated by a first-order lag relationship. In this case, r<2 = 0.9331. The graph of Cl versus time is presented in Figure 4.13. The time trend in this graph differs somewhat from the previous graphs. There appear to be separate stages in the plot: one at the beginning, another in the middle, and the third stage at the end. Of the remaining three variables, none show strong time dependencies. As an example, consider the time-sequence plot for RPl given in Figure 4.14. Across time, the plot of the data is nearly horizontal and shows no trends or patterns.
76
Chapter 4. Construction of Historical Data Set
Figure 4.11: Time-sequence plot of L3.
Figure 4.12: Time-sequence plot of I/I.
Another useful indication of autocorrelation in a process is the presence of a strong pairwise correlation between a variable and the time sequence of its collection. For example, the correlations between the seven variables listed in Table 4.12 and a time-sequence variable (Time) based on the observation number for a run cycle are presented in Table 4.13. Inspection of the pairwise correlations between the seven variables and time indicates there is a moderate time dependency for variables 1/2 and Temp, and a lesser one for the unwanted I/I and Cl. A weak-tomoderate relationship is indicated for RP1, for the process variable L3, and for the production variable PI. Note that the production variable PI decreases over time, as indicated by the negative correlation coefficient.
4.9. Example of Autocorrelation Detection Techniques
Figure 4.13: Time-sequence plot of Cl.
Figure 4.14: Time-sequence plot of RP1.
Table 4.13: Pairwise correlations with time-sequence variable. Process Variables RPl LI LI L3
Temp Cl PI
Time 0.507 0.693 0.811 0.318 0.808 0.691 -0.456
77
78
4.10
Chapter 4. Construction of Historical Data Set
Summary
Control procedures are designed to detect and help in determining the cause of unusual process events. The point of reference for "unusual events" is the historical data set. This is the baseline of any control procedure and must be constructed with great care. The first step in its construction is to acquire an understanding of the process. This knowledge can be obtained from the operators and process engineers. A study of the overall system will reveal problem areas where the applications of a control procedure would be most helpful. This is necessary to determine the type and purpose of the control procedure. With the selection of an appropriate area for application of a control procedure, we can obtain a preliminary data set. However, the data must be carefully filtered of its impurities so that the resulting data set is clean. Graphical tools can be a great aid in this process, as they can be used to identify obvious outliers and, in some cases, determine useful functional relationships among the variables. In addition, data collection and data verification procedures must be examined and any missing data replaced or estimated, or else one must remove the associated observation vector or process variable. After filtering the preliminary data, we strongly recommend checking on the singularity of the covariance matrix. The problems of a singular covariance matrix, or of collinearity among the variables, can be quite critical. Collinearity often occurs when there are many variables to consider or when some of the variables are computed from measured ones. These situations can be detected using the eigenvalues of the covariance or correlation matrices. Principal component analysis can be a useful tool in this determination, as can consultation with the process engineers. Since a severe collinearity can inflate the T2 statistic, appropriate action must be taken to remove this problem. Steady-state control procedures do not work well on autocorrelated processes. Thus, one must investigate for the presence of autocorrelation in the preliminary data set. We offer two procedures for detecting the presence of autocorrelated data in a multivariate system. The first is based on plotting a variable over time and looking for trends or patterns in the plot. The second is based on plotting the sample autocorrelations between observations separated by a specified lag time versus time and examining the observed trends. Large autocorrelations will pinpoint probable cases for further study. In Chapter 10, we discuss procedures for removing the effects of these time dependencies on the T2 statistic.
4.11 4.11.1
Appendix Eigenvalues and Eigenvectors
Consider a square (p x p) matrix A. We seek to find scalar (constants) values A^, i = l,2,...,p, and the corresponding (p x 1) vectors Ui, i = 1, 2 , . . . , p , such that the matrix equation
4.11. Appendix
79
is satisfied. The \i are referred to as the eigenvalues, or characteristic roots, of A, and the corresponding vectors, C/j, are referred to as the eigenvectors, or characteristic vectors, of A. The \i are obtained by solving the pth-degree polynomial in A given by where A—XI \ represents the determinant of the matrix (A—XI). The corresponding eigenvectors Ui are then obtained by solving the homogeneous system of equations given in (A4.1). The eigenvalues (Ai, A 2 , . . . , Xp) are unique to the matrix A] however the corresponding eigenvectors (t/i, C/2, • • • , Up) are not unique. In statistical analysis the eigenvectors are often scaled to unity or normalized, so that Note that the eigenvalues of A~l are the reciprocals of the eigenvalues of A. The corresponding eigenvectors of A~l are the same as those of A. Covariance matrices, such as S, that are associated with a T2 statistic are symmetric, positive definite matrices. For symmetric matrices, the corresponding eigenvalues must be real numbers. For positive definite symmetric matrices, the eigenvalues must be greater than zero. Also, with symmetric matrices the eigenvectors associated with distinct eigenvalues are orthogonal so that UiU'j = 0. Near-singular conditions (i.e., collinearities) exist when one or more eigenvalues are close to zero. Closeness is judged by the size of an eigenvalue relative to the largest eigenvalue. The square root of the ratio of the largest eigenvalue (Ai) to any other eigenvalue (Xi) of a matrix A is known as a condition index and is given by
A recommended guideline for identifying a near-singular matrix is based on the size of this ratio. A ratio greater than 30 implies that a severe collinearity is present. Our main reason for examining eigenvalues and eigenvectors is their use in diagnosing (near) collinearities associated with the T2 statistic. This topic is extensive and cannot be covered in this limited space. For more details, see Belsley (1991) or Myers and Milton (1991). 4.11.2
Principal Component Analysis
The iih principal component, Zi, of a matrix A is obtained by transforming an observation vector X' — (rci, # 2 , . . . , xp) by
where Ui, i = 1, 2 , . . . , p, are the eigenvectors of A. If A is symmetric, its eigenvectors (associated with distinct eigenvalues) are orthogonal to each other. Thus, the resulting principal components (zi, 2 2 , . . . , zp) also would be orthogonal. Also, each principal component in (A4.3) has a variance given by the corresponding eigenvalue, i.e.,
80
Chapter 4. Construction of Historical Data Set
A major use of PCA is to transform the information contained in p correlated process variables into the p independent principal components. The transformation in (A4.3) is made in such a way that the first k of the principal components contain almost all the information related to the variation contained in the original p variables. Used in this manner, PCA is known as a dimension-reduction technique since k < p. The percentage of the total variation explained by the first k principal components is computed by the ratio of the sum of the first k eigenvalues to the total sum of the eigenvalues. It can be shown that when a PCA is performed on a correlation matrix, this reduces to
The principal components of a covariance matrix can be used to identify the variables that are related to a collinearity problem. Suppose the condition index for the smallest eigenvalue, A p , is greater than 100. This implies that Xp is very small relative to the largest eigenvalue of the covariance matrix. It also implies that the variance of the pth principal component is very small, or approximately zero, i.e., that var(zp) ~ 0. For a covariance matrix, this implies that a near-perfect linear relationship is given by the pth principal component, i.e.,
This equation produces the collinear relationship that exists between Xj and the other variables of the system. The theoretical development of PCA is covered in the many texts on multivariate analysis, e.g., Morrison (1990), Seber (1984), and Johnson and Wichern (1998). An especially helpful reference on the applications of PCA is given in Jackson (1991).
Chapter 5
Charting the T2 Statistic in Phase I
5.1
Introduction
In this chapter we discuss methods, based on the T2 statistic, for identifying atypical observations in an HDS. We also include some examples of detection schemes based on distribution-free methods. When attempting to detect such observations, it is assumed that good preliminary data are available and that all other potential data problems have been investigated and resolved. The statistical purging of unusual observations in a Phase I operation is essentially the same as an outlier detection problem. An outlier is an atypical observation located at an extreme distance from the main part of the sample data. Several useful statistical tests have been presented for identifying these observations, and these techniques have been described in numerous articles and books (e.g., see Barnett and Lewis (1994), Hawkins (1980), and Gnanadesikan (1977)). Although the T2 statistic is not necessarily the optimal method for identifying outliers, particularly when used repeatedly as in a control chart, it is a simple procedure to apply and can be very helpful in locating individual outlying observations. Further, as shown in Chapter 7, the T2 statistic has the additional advantage of being capable of determining the process variables causing an observation to signal. For these reasons, we will concentrate only on the T2 statistic.
5.2
The Outlier Problem
The reason we seek to remove outlying observations from the HDS is because their inclusion can result in biased sample estimates of the population mean vector and covariance matrix and lead to inaccurate control procedures. To demonstrate this, consider the scatter plot in Figure 5.1 of a preliminary data set on two variables. Three separate groupings of outliers, denoted as Groups A, B, and C, are presented in the graph. The inclusion of these observations in the HDS, denoted by 81
82
Chapter 5. Charting the T2 Statistic in Phase I
figure 5.1 scater plot with thre eeg roufp f dat a black circle son the graph will bias the estimates of thwe varieanc of the two vart ables and/or the estimates of the correlation between these two variables. For example, the inclusion of Group A data will increase the variation in both variables but will have little effect on their pairwise correlation. In contrast, including the Group C data will distort the correlation between the two variables, though it will increase the variation of mainly the x\ variable. Why do atypical observations, similar to those presented above, occur in an initial sample from an in-control multivariate process? There are many reasons, such as a faulty transistor sending wrong signals, human error in transcribing a log entry, or units operating under abnormal conditions. Most atypical information can be identified using graphs and scatter plots of the variables or by consulting with the process engineer. For example, several of the observations in Groups B and C of Figure 5.1, such as points Bl and Cl, are obvious outliers; however, others may not be as evident. It is for this reason that a good purging procedure is needed. Detecting atypical observations is not as straightforward in multivariate systems as in univariate ones. A nonconforming observation vector in the multivariate sense is one that does not conform to the group. The purging procedure must be able to identify both the components of the observation vectors that are out of tolerance as well as those that have atypical relationships with other components.
5.3
Univariate Outlier Detection
Consider a univariate process where control is to be monitored with a Shewhart chart based on an individual observation x. Assume the mean, //, and the standard deviation, cr, of the underlying normal distribution, 7V(//,<7 2 ), are known for the process. Our goal is to clean the preliminary data set (collected in Phase I) by
5.3. Univariate Outlier Detection
83
Figure 5.2: Normal distribution with control limits.
purging it of outliers. Any observation in the data set that is beyond the control limits of the chart is removed from further consideration. Suppose that the Shewhart upper and lower control limits, denoted UCL and LCL, for this data set are those depicted in Figure 5.2. We assume that any outlier is an observation that does not come from this distribution, but from another normal distribution, N(/j, + d, a 2 ), having the same standard deviation, but with the mean shifted d units to the right. Both distributions are depicted in Figure 5.3. Detecting an outlier in this setting is equivalent to the testing of a statistical null hypothesis. To decide if the given observation is taken from the shifted normal distribution, and thus declared an outlier, we test the null hypothesis
that all observations arise from the normal distribution JV(p,,cj 2 ) against the alternative hypothesis that all observations arise from the shifted normal distribution N({j,-\-d: a 2 ). If the null hypothesis is rejected, we declare the observation to be an outlier and remove it from the preliminary data set. In the above hypothesis test, the distribution under the null hypothesis is referred to as the null distribution and the distribution under the alternative hypothesis is labeled the nonnull distribution. The power of the hypothesis test is denoted in Figure 5.3 by the area of the shaded region under the nonnull distribution, which is the distribution shifted to the right. This is the probability of detecting an observation as an outlier when it indeed comes from the shifted distribution. Comparisons are made among different outlier detection schemes by comparing the power function of the procedures across all values of the mean shift, denoted by d. Many analysts use univariate control chart limits of individual variables to remove outlying observations. In this procedure, all observation vectors that contain
84
Chapter 5. Charting the T2 Statistic in Phase I
Figure 5.3: Original and shifted normal distributions.
Figure 5.4: Univariate Shewhart region and T2 control region.
an observation on a variable outside the 3cr range are excluded. This is equivalent to using the univariate Shewhart limits for individual variables to detect outlying observations. A comparison of this procedure with the T2 procedure is illustrated in Figure 5.4 for the case of two variables.
5.4. Multivariate Outlier Detection
85
The shaded box in the graph is defined by the univariate Shewhart chart for each variable. For moderate-to-strong correlations between the two variables, the T2 control ellipse usually extends beyond the box. This indicates that the operational range of the variables of a multivariate correlated system can be larger than the control chart limits of independent variables. Use of the univariate control chart limits of a set of variables ignores the contribution of their correlations and in most cases restricts the operational range of the individual variables. This restriction produces a conservative control region for the control procedure, which in turn generates an increased number of false signals. This is one of the main reasons for not using univariate control procedures to detect outliers in a multivariate system.
5.4
Multivariate Outlier Detection
As in the univariate case, a preliminary control procedure for a multivariate process must be constructed to purge the data of atypical observations. With the T2 statistic, the corresponding control chart has only a UCL. If the T2 value that is computed for an observation exceeds this limit, the observation is deleted. For (univariate) Shewhart charting, there is very little procedural difference between the preliminary charting procedure and the actual control procedure. However, the multivariate Shewhart chart based on the T2 statistic offers some distinct differences in the computation of the UCL, mainly due to the form of the probability distribution of the T2 statistic. As noted in Chapter 2, depending on the circumstances, the T2 statistic can be described by three different probability functions: the beta, the F, and the chi-square distributions. The beta distribution is used in the purging process of a Phase I operation, whereas the F distribution is used in the development of the control procedure in a Phase II operation. The chi-square distribution has applications in both Phase I and II operations. One begins the purging process by selecting a value for a, the probability of a Type I error. Its choice determines the size, 1 — a, of the control region. A Type I error is made if an observation is declared an outlier when in fact it is not. In making such errors, we exclude good observations from the HDS. These observations have large statistical distances (from the mean vector) and will lie in the tail region of the assumed MVN distribution. For small preliminary data sets, excluding these observations can have a significant effect on estimates of the covariance matrix and mean vector. For larger samples, the effect should be minimal. The situation is reversed when one considers the error of including an outlier in the HDS. For small samples, the effect of one outlier on the estimates of the mean vector and covariance matrix can be substantial. In the face of this dilemma, especially when using small sample sizes, we recommend carefully examining any observation considered for deletion and discussing it with the process engineer.
86
5.5
Chapter 5. Charting the T2 Statistic in Phase I
Purging Outliers: Unknown Parameter Case
Consider a Phase I purging procedure where a single observation vector X' = (xi, X2,. • • , xp] is to be monitored for control of the process using a T2 chart. We assume that the data follow an MVN distribution with an unknown mean vector H and an unknown covariance matrix E. From the preliminary data, we obtain estimates x and S of // and E using the procedures from Chapter 2. We begin the purging process by making an initial pass through the preliminary data. For a given a level, all observation vectors whose T2 values are less than or equal to the UCL will remain in the data set, i.e., retain X if where the control limit is determined by
and where -S[a, p /2,(n-p-i)/2] is the upper crth quantile of the beta distribution, B\p/2,(n-p-i)/2]- If an observation vector has a value greater than the UCL, it is to be purged from the preliminary data. With the remaining observations, we calculate new estimates of the mean vector and covariance matrix. A second pass through the data is now made. Again, we remove all detected outliers and repeat the process until a homogeneous set of observations is obtained. The final set of data is the HDS. When process control is to be based on monitoring the subgroup means of k samples of observations, the actual purging process of the preliminary data set is the same as for individual observations. The data are recorded in samples of size k
m^, i = l , 2 , . . . , f c , yielding a total sample size of n = ^ m^. Since each individual i=l
observation vector comes from the same MVN distribution, we can disregard the k subgroups and treat the observations as one group. With the overall group, we obtain the estimates, and S, and proceed as before. When the process is in control, this approach produces the most efficient estimator of the covariance matrix (e.g., see Wierda (1994) or Chou, Mason, and Young (1999)). 5.5.1
Temperature Example
To illustrate the above procedure when using individual observations, consider the 25 observations presented in Table 5.1. These data are a set of temperature readings from the eight configured burners on a boiler. The burner temperatures are denoted as ti, t2,..., ts, and their correlation matrix is presented in Table 5.2. The control procedure is designed to detect any significant deviation in temperature readings and any change in the correlations among these variables. If this occurs, a "cold spot" develops and inadequate burning results. The T2 values of the 25 observations are computed using (5.1) and are presented in Table 5.3. Using an a — 0.001, the control limit is computed using the formula in (5.2). This yields a value of 17.416. Observation 9 is detected as an outlier since its T2 value of 17.58 exceeds the UCL. Hence, it is removed from the preliminary data set.
5.5. Purging Outliers: Unknown Parameter Case
87
Table 5.1: Boiler temperature data. Obs. No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
ti 507 512 520 520 530 528 522 527 533 530 530 527 529 522 532 531 535 516 514 536 522 520 526 527 529
*2
516 513 512 514 515 516 513 509 514 512 512 513 514 509 515 514 514 515 510 512 514 514 517 514 518
t3 527 533 537 538 542 541 537 537 528 538 541 541 542 539 545 543 542 537 532 540 540 540 546 543 544
*4
*5
516 518 518 516 525 524 518 521 529 524 525 523 525 518 528 525 530 515 512 526 518 518 522 523 525
499 502 503 504 504 505 503 504 508 507 507 506 506 501 507 507 509 501 497 509 497 501 502 502 504
*6
512 510 512 517 512 514 512 508 512 512 511 512 512 510 511 511 511 516 512 512 514 514 516 512 516
ts 477 475 477 479 477 480 477 472 477 477 476 476 477 475 478 477 477 481 476 477 478 478 480 476 481
*7
472 476 480 480 481 482 479 478 482 482 482 481 481 476 481 482 483 476 471 482 475 475 477 475 479
Table 5.2: Correlation matrix for boiler temperature data.
tl *2 *3 *4
*5
t6 t?
ts
1
ts 0.584 0.281
0.281 0.258 0.044 0.659 0.094 0.797
0.444 0.308 0.200 0.396 0.294
tl 1
0.059 0.584 0.901 0.819 -0.147 0.813 0.014
*2
0.059
1
*4
0.901 0.258 0.444 1
0.849 -0.226 0.788 0.018
*5
0.819 0.044 0.308 0.846 1
-0.231 0.924 -0.043
*6
-0.147 0.659 0.200 -0.226 -0.231 1
-0.103 0.893
<7
0.813 0.094 0.396 0.788 0.924 -0.103 1
0.079
ts 0.014 0.797 0.294 0.018 -0.043 0.893 0.079 1
New estimates of the mean vector and covariance matrix are computed from the remaining 24 observations, and the purging process is repeated. The new correlation matrix is presented in Table 5.4. Comparing the correlation matrices of the purged data and the unpurged data, we find a definite change. For example, with the removal of observation 9 the correlation between ti and £3 increases from 0.584 to 0.807. This illustrates the effect a single outlier can have on a correlation coefficient when there is a small sample size (n = 25). Note that such a significant change in the correlation matrix implies a similar change in the covariance matrix. The new T2 values are presented in Table 5.5. Since the new UCL for the reduced set of 24 observations is 17.00, the second pass through the data produces
Chapter 5. Charting the T2 Statistic in Phase I
88
Table 5.3: T2 values for first pass. T2 Value Obs. No. T2 Value 1 14 9.55 13.96 2 9.78 15 7.07 6.52 3 16 5.47 17 14.74 4 4.77 8.74 18 6.58 5 9.84 19 5.31 6 8.64 20 7.89 7 8 12.58 21 9.78 22 2.79 17.58* 9 23 2.79 10 6.09 11 24 7.98 3.29 5.32 12 25 3.63 1.32 13 'Indicates T value is significant at 0.001 level.
Obs. No.
Table 5.4: Correlation matrix with one outlier removed.
*1 *2 *3 *4
t5 *6
t7
ts
tl 1
0.051 0.807 0.899 0.808 -0.141 0.805 0.021
t2
*3
0.051
0.807 0.342 1 0.717 0.506 0.204 0.569 0.320
1 0.342 0.259 0.034 0.662 0.087 0.799
*4
0.899 0.259 0.717 1 0.838 -0.225 0.780 0.027
*5
0.808 0.034 0.506 0.838 1 -0.228 0.922 -0.037
*6
-0.141 0.662 0.204 -0.225 -0.228 1 -0.096 0.893
*7
0.805 0.087 0.569 0.780 0.922 -0.096 1 0.086
«8
0.021 0.799 0.320 0.027 -0.037 0.893 0.086 1
Table 5.5: T2 values for second pass. Obs. No. 1 2 3 4 5 6 7 8 10 11 12 13
T2 value 16.07 9.62 5.41 14.09 6.56 5.43 7.89 9.34 5.26 3.21 3.53 1.22
Obs. No. 14 15 16 17 18 19 20 21 22 23 24 25
T2 value 9.84 7.55 7.49 6.62 8.75 9.66 10.62 12.62 3.26 6.79 7.76 5.44
no additional outliers. The listed observations (excluding 9) form a homogeneous statistical group and can be used as the HDS. An alternative graphical approach to outlier detection based on the T2 statistic is to examine a Q-Q plot (see Chapter 3) of the appropriate T2 values. For the
5.5. Purging Outliers: Unknown Parameter Case
89
Figure 5.5: Q-Q plot of temperature data. above temperature data, with p = 2 and n = 25, the beta distribution for the T2 statistic using the formula given in (2.15) is
A Q-Q plot of the T2 values, converted to beta values by dividing them by 0.922, is presented in Figure 5.5. Several of the plotted points do not fall on the given line through the data. This is especially true for the few points located in the upper right corner of the graph. This is supported by the T2 values given in Table 5.3. where four points. 1, 4, 9, and 21, have T2 values larger than 10. Observation 9, located at the upper end of the line of plotted values, is somewhat removed from the others. Given this result, the point should be investigated as a potential outlier. 5.5.2
Transformer Example
As a second data example, consider 134 observations taken on 23 variables used in monitoring the performance of a large transformer. The T2 values of the preliminary data set are presented in the T2 control chart given in Figure 5.6. An a — 0.00 was used in determining the UCL of 44.798. The chart indicates that an upset condition begins at observation 20 and ends at observation 27. Other than the upset condition, the T2 values for the preliminary data appear to be in steady state and indicate the occurrence of good process operations. A Q-Q plot of the 134 T2 values is presented in Figure 5.7. Although the data generally have a linear trend, the large values associated with the observations contained in the upset condition have a tendency to pull the other observations upward in the upper right corner of the graph. The 7 observations contained in the upset condition of Figure 5.6 were removed from the transformer data set, and the T2 control chart was reconstructed using the remaining 127 observations. The corresponding T2 plot is presented in Figure 5.8.
90
Chapter 5. Charting the T2 Statistic in Phase I
Figure 5.6: T chart for transformer data with upset condition.
Figure 5.7: Q-Q plot of transformer data.
Figure 5.8: T2 chart for transformer data after outlier removal.
5.6.
Purging Outliers: Known Parameter Case
91
Figure 5.9: Q-Q plot for transformer data after outlier removal.
The UCL is recalculated as 44.528. Observe that the system appears to be very consistent and all observations have T2 values below the UCL. The corresponding Q-Q plot of the T2 values is presented in Figure 5.9. Observe the strong linear trend exhibited in the plot and the absence of observations far off the trend line.
5.6
Purging Outliers: Known Parameter Case
Assume the sample data follow an MVN distribution having a known mean vector and known covariance matrix. In this setting, the T2 test statistic for an observation vector Xf = ( x - [ , X 2 , . . . ,xp) becomes
For a given a level, the UCL for the purging process is determined using
where xJa p\ is the upper ath quantile of a chi-square distribution having p degrees of freedom. To illustrate this procedure and contrast it to the case where the parameters are unknown, assume the sample mean vector and covariance matrix of the data in Table 5.1 are the true population values. Using an a — 0.001, the UCL is ^fo 001 2) = 26.125. Comparing the observed T2 values in Table 5.3 to this value we find that no observation is declared an outlier. Thus, observation 9 would not be deleted. A major difference between the T2 statistics in (5.1) and (5.3) is due to how we determine the corresponding UCL. When the mean vector and covariance matrix are estimated, as in (5.1), the beta distribution is applicable, but when these parameters are known, as in (5.4), the chi-square distribution should be used. It can be shown that for large n, the UCL as calculated under the beta distribution (denoted as
Chapter 5. Charting the T2 Statistic in Phase I
92
Table 5.6: Values of UCLB when a = 0.001. n
p
50 2 8.55 6 15.00 30 39.27
100 8.88 15.90 45.39
150 8.99 16.21 47.27
200 9.04 16.36 48.19
250 9.08 16.45 48.74
300 9.10 16.51 49.10
350 9.12 16.55 49.36
400 9.13 16.58 49.55
450 9.14 16.61 49.70
500 00 9.14 9.21 16.63 16.81 49.82 50.89
UCL#) approaches the UCL as specified by the chi-square distribution (denoted as UCL^). This is illustrated in Table 5.6 for various values of n and p at a = 0.001. Notice that, in this example, using the chi-square (instead of the beta) UCL for small n and p, such as p = 2 and n = 50, increases the likelihood of accepting potential outliers in the HDS. This occurs because UCLc- (at n = oo) always exceeds UCL#. For large p, such as p = 30, n should exceed 500 in order to justify usage of the chi-square UCL. A comparison of the UCL for various values of n, p, and a for the two situations can be found in Tracy, Young, and Mason (1992).
5.7
Unknown T2 Distribution
In Chapter 3, we introduced three procedures for determining the UCL in a T2 control chart in cases where the distribution of the T2 statistic was unknown. The first was a method based on the use of Chebyshev's theorem, the second was based on the quantile technique, and the third was based on the fitting of the empirical T2 distribution using a kernel smoothing technique. In this section, we use a data example to show how these methods can be used to identify outliers. To demonstrate the procedure based on Chebyshev's theorem, we consider n = 491 observations on p = 6 variables taken as a preliminary data set under good operational conditions from an industrial process. The corresponding T2 values of the observations are computed and plotted on the control chart given in Figure 5.10. Examination of the chart clearly indicates the presence of a number of large T2 values. These large values are also indicated in the tail of the T2 histogram presented in Figure 5.11. The estimated mean and the standard deviation of the 491 T2 values are computed and the UCL is approximated using the formula in (3.7) from Chapter 3, i.e., with k = 4.472 and a < 0.10. The estimated UCL for this first pass is 19.617, and it is used to remove 13 outliers. The estimation of the UCL and the resultant purging process is repeated until a homogeneous data set of T2 values is obtained. Five passes are required and 28 observations are deleted. The results of each pass of this procedure are presented in Table 5.7.
5.7. Unknown T2 Distribution
93
Figure 5.10: T2 values for industrial data set.
Figure 5.11: Histogram of T2 values for industrial data. Table 5.7: Results of purging process using Chebyshev's procedure. T
ST UCL
# of Outliers Removed
Pass 1 5.988 4.310 19.617 13
Pass 2 5.987 3.630 17.468 6
Pass 3 5.987 3.488 17.019 4
Pass 4 5.987 3.407 16.761 3
Pass 5 5.987 3.348 16.575 2
Pass 6 5.987 3.310 16.455 0
Chapter 5. Charting the T2 Statistic in Phase I
94
Table 5.8: Results of purging process assuming multivariate normality. UCL
# of Outliers Removed
Pass 1 16.627
Pass 2 16.620 7
17
Pass 3 16.617
Pass 4 16.616 1
3
Pass 5 16.616 0
Table 5.9: Results of purging process using nonparametric procedure. UCL
# of Outliers Removed
Pass 1 22.952 5
Pass 2 21.142 8
Pass 3 17.590 6
Pass 4 17.223 3
Pass 5 16.588 5
Pass 6 16.402 (1)
If one assumes that the T2 values are described by a beta distribution and calculates the UCL using (5.2) with an a = 0.01, the same 28 observations are removed. However, the order of removal is not the same and only four passes are required. These results are presented in Table 5.8. In this case, the major difference between the two procedures is that the probability of a Type I error is fixed at a = 0.01 for the beta distribution, whereas the error rate for the Chebyshev approach is only bounded by a < 0.10. To demonstrate the procedure based on the quantile technique, the 491 T2 values are arranged in descending order and an approximate UCL is calculated using a = 0.01 and the formula in (A3.3) from Chapter 3, i.e.,
The estimated UCL for this first pass is 22.952, and it is used to remove five outliers. The estimation of the UCL and the purging process are repeated until a homogeneous data set of T2 values is obtained. In this procedure, there will always be at least one T2 value exceeding the estimated UCL in each pass. Thus, one must stop at the step where only a single outlier is encountered. Since this occurs at step 6 for our data example, only five passes are required and 27 observations are deleted. The results of each pass of this procedure are presented in Table 5.9. The third method for obtaining an appropriate UCL is to fit a distribution to the T2 statistic using the kernel smoothing technique. The UCL can be approximated using the (1 — a)th quantile of the fitted kernel distribution function of the T2. We begin by using the preliminary data of n observations to obtain the estimates X and S of the parameters // and E. Using these estimates, we compute the T2 values. These n values provide the empirical distribution of the T2 statistic for a Phase I operation. As previously noted, we are assuming that the intercorrelation common to the T2 values has little effect on the application of these statistical procedures. We apply the kernel smoothing procedure described by Polansky and Baker (2000) to obtain FK(t), the kernel estimate of the distribution of T 2 , or simply the
5.7. Unknown T Distribution
95
Table 5.10: Results of the purging process using the kernel smoothing technique. UCL
# of Outliers Removed
Pass 1 23.568
Pass 2 22.531
Pass 3 19.418
Pass 4 18.388
Pass 5 17.276
Pass 6 16.626
Pass 7 16.045
5
4
4
4
5
4
3
kernel distribution of the T 2 . It is given as
where $ is the standard normal distribution function, T 2 (j), j = 1 , . . . , n, denote the n T2 values, and /i is the two-stage estimate of the bandwidth. An algorithm for computing h is outlined in Polansky and Baker (2000). The UCL is determined as the (1 — a)th quantile of FK(i) and satisfies the equation (5.7)
Since FK is generally a skewed distribution, the UCL can be large for small a values, such as 0.01 and 0.001. The (1 — a)th sample quantile of the T 2 (j), j = 1 , . . . , n, can be used as the initial value for the UCL in (5.7). Since the kernel distribution tends to fit the data well, for a moderate a value between 0 and 1, approximately no. (rounded to the nearest integer) of the T2 values are beyond the UCL or the upper lOOcrth percentile of the kernel distribution. For n = 491 and a = 0.01, na = 4.91 ~ 5, and one may expect that four to five values of the T2 are above the UCL. After these outliers are removed in each pass, there are always some points above the newly calculated UCL. This seems to be inevitable unless n or a is very small, so that na is around 0. Because the kernel method is based solely on data, one way of determining the UCL for the final stage is to compare the UCLs and the kernel distribution curves for successive passes. If the UCLs for two consecutive passes are very different, this implies that the kernel distribution curves also differ significantly after outliers are removed. However, if the UCLs and the curves for two consecutive passes are nearly the same, this implies that the UCL is the desired UCL for the final stage. For the data in the example, the UCLs for Passes 7 and 8 are 16.045 and 15.829, respectively. The difference between the bandwidths of these kernel estimates is only 0.004. Therefore, the three points in Pass 7 cannot be viewed as outliers and should be kept in the HDS. After six passes, 26 observations are removed and the remaining 465 observations form the HDS. The UCL for the T2 chart should be set at 16.045, as Pass 7 is the final pass. Table 5.10 presents the results of the entire purging process.
Chapter 5. Charting the T2 Statistic in Phase I
96
5.8
Summary
Historical data sets are very important in process control as they provide a baseline for comparison. Any process deviation from this reference data set is considered out of control, even in those situations where the system improves. Because of this criticality, the process needs to be in control and on target when the HDS observations are selected. Inclusion of atypical observations will increase the variation and distort the correlations among the variables. As few as one outlying observation can do this. It is for these reasons that a good outlier purging procedure, such as one based on the T2 statistic, is needed. A common misconception is that the operational range of a variable in a multivariate process is the same as the operational range of a variable in a univariate process. This is only true for independent variables. For correlated variables, the operational range of the variables is increased. Correct outlier purging procedures will determine the appropriate operational ranges on the variables. Several different forms of the T2 can be used in detecting outliers in a Phase I operation. These include situations where the population parameters are both known and unknown and where observations are either individually charted or charted as means. In addition, three alternative procedures are available for use when the assumption of multivariate normality is invalid. These include the Chebyshev approach, the quantile method, and the kernel smoothing approach.
Chapter 6
Charting the T2 Statistic in Phase II
Old Blue: An Invaluable Tool Using the software package, you quickly create a set of scatter plots, histograms, and time-sequence charts. That old statistics professor was right on target when he said graphical procedures provide true insight into the structure and behavior of your data. You visually spot a number of questionable observations. From your process experience, you know these are not typical. You find the procedure for creating the baseline data to be more involved than the methods you used with single-variable data. It takes some time for you to figure out that the heat transfer coefficient is causing the singularity in the covariance matrix. You should have caught that right away since the textbook pointed out that a computed variable sometimes contains redundant information when the variables used in constructing the computed variable are included in the analysis. The extremely high correlation between the heat transfer coefficient and the pressure drop was another clue that you missed. You examine a Q-Q plot of the T2 data from the Phase I operation and decide the beta distribution can be used to describe the T2 statistic for your process. As noted in the initial plots, this plot also indicates the presence of a few outliers. However, it only takes two passes through the data to remove them. These unusual observations are not like the rest of the data set. You wonder why they occurred and decide to set these aside for later investigation. All in all, the Phase I analysis shows "Old Blue" to be very consistent. The T2 values of the baseline data are all small and very close together and show very little variation. Since each T2 is a composite of observations on the available input, process, and output variables, you can easily determine the overall performance of the unit. You reflect on the time you spent in process operations. A tool based on the T2 chart, and used on-line and in real time, would have been invaluable. You recall the constant stress created in determining if all was going well.
Chapter 6. Charting the T2 Statistic in Phase II
98
There was no single monitored variable useful in determining unit performance. You conclude that something must be done about incorporating this powerful measure into the unit operations. You now are prepared to move forward with your project. The next step is the location and isolation of the T2 signals from the incoming process data. You are ready for Phase II.
6.1 Introduction A number of items need to be considered when choosing the appropriate T2 charting procedure for a Phase II operation. These include computing the appropriate charting statistic, selecting a Type I error probability, and determining the UCL. For example, if we monitor a steady-state process that produces independent observations, a T2 charting procedure will suffice. However, if the observations exhibit a time dependency, such as that which is inherent to decay processes, some adjustment for the time dependency must be made to the T2 statistic (i.e., see Chapter 10). The charting of the T2 statistic in a Phase II operation is very similar to the approach used in charting the statistic for a Phase I operation. The major difference is in the probability functions used in determining the control region. Two cases exist. When the mean vector and covariance structure are known, a chi-square distribution is used to describe the behavior of the statistic and determine the upper control limit. When the mean and covariance parameters are unknown and must be estimated from the historical data, an F distribution is used to describe the statistic and locate the upper control limit. In this chapter, we address several different charting procedures for the T2 statistic and examine the advantages and disadvantages of each. We initially discuss monitoring a process using a T2 statistic when only a single observation vector is collected at each time point. This is later extended to the situation where the process is monitored using the mean of a subgroup of observations taken at each time point. Other topics discussed include the choice of the probability of a Type I error, procedures for calculating the average run length to detect a given mean shift, and charts for the probability of detecting a shift in the mean vector. Any nonrandom pattern displayed in a T2 chart can imply process change. For this reason, we include a section on detecting systematic patterns in T2 charts.
6.2
Choice of False Alarm Rate
When constructing a T2 control chart, consideration must be given to the choice of a, the probability of a Type I error. Recall that this topic was briefly discussed in Section 5.4 for a Phase I operation. In a Phase II operation, a Type I error occurs when we conclude that an observation presents a signal when in fact no signal is present. Signaling observations are detected when their T2 values exceed the UCL. In turn, the UCL is primarily determined by the choice of a. This is an important
6.2. Choice of False Alarm Rate
99
decision, since a value of a can be chosen such that the T2 value of an observation exceeds the UCL, even though the observation really contains no signal. Note, also, that the size of the control region is 1 — a. This is the probability of concluding that the process is in control when in fact control is being maintained on all process variables. The size of a cannot be considered without discussion of /3, the probability of a Type II error. This is the error of concluding there is no signal when in fact a signal is present. Type I and Type II errors are interrelated in that an increase in the probability of one will produce a decrease in the probability of the other. Careful consideration must be given to the consequences produced by both types of error. For example, suppose a chemical process is producing a product that becomes hazardous when a particular component increases above a given level. Assume that this component, along with several other correlated components, is observed on a regular basis. A T2 control procedure is used to check the relationships among the components as well as to determine if each is in its desired operational range. If a Type I error is made, needless rework of the product is required since the process is in control. If a Type II error is made, dangerous conditions immediately exist. Since dangerous conditions override the loss of revenue, a very small (3 is desirable. Given this preference for low risk of including an outlier, a large a would be acceptable. The value of a chosen for a T2 chart in a Phase II operation does not have to agree with the value used in constructing the Phase I chart. Instances do exist where making a Type I error in a Phase II operation is not so crucial. For example, suppose change is not initiated in the actual control of a process until more than one signal is observed. This reduces the risk of overcontrolling the process. Situations such as these require a larger a in the Phase II operation. In contrast, a large a for Phase I can produce a conservative estimate of both the mean vector and the covariance matrix, so some balance is necessary. The choice of a for a univariate charting procedure pertains only to the false alarm rate for the specified variable being monitored. For example, the control limits of a Shewhart chart are frequently located at plus or minus three standard deviations from the center line of the charted statistic. This choice fixes the false alarm rate a at a value of 0.0027 and fixes the size of the control region at (1 — a) or 0.9973. This translates to a false alarm rate of about 3 observations per 1,000. The choice of a in monitoring a multivariate process is more complex, as it reflects the simultaneous risk associated with an entire set of variables (e.g., Timm (1996)). Establishing a control procedure for each component of the observation vector would lead to an inappropriate control region for the variables as a group, as individual control does not consider relationships existing among the variables. This is illustrated with the following example. Suppose X' = (£1,0:2) is a bivariate normal observation on a process that is to be monitored by a joint control region defined by using a 3-sigma Shewhart procedure for each individual variable. The shaded box given in Figure 6.1 illustrates the control region. A major problem with this approach is that it ignores the relationship that exists between the process variables and treats them independently. The true joint control procedure, if the two variables were correlated, would be similar to the ellipse, which is superimposed on the box in Figure 6.1.
100
Chapter 6. Charting the T2 Statistic in Phase II
Figure 6.1: Region of joint control. If the p variables being monitored in a multivariate process are independent, the simultaneous false alarm rate, o;s, is computed as
where a is the false alarm rate for each individual variable. Thus the value of as increases as p, the number of variables, increases. For example, if a = 0.0027, the simultaneous false alarm rate for p — 2 is 1 — (0.0027)2 = 0.0053, but for p = 4, the rate increases to 0.0108. This example produces exact probabilities for a process with independent variables. In reality, a process usually consists of a group of correlated variables. Such situations tend to increase the true false alarm rate even beyond (6.1).
6.3
T2 Charts with Unknown Parameters
Consider a continuous steady-state process where the observation vectors are independent and the parameters of the underlying normal distribution are unknown and must be estimated. Assume the process is being monitored by observing a single observation vector, X' = (xi, £2, • • • , xp), on p variables at each time point. The T2 value associated with X is given by
where the common estimates X and S are obtained from the HDS following the procedures described in Chapter 4. In this Phase II setting, the T2 statistic in (6.2) follows the F distribution given in (2.14). For a given a, the UCL is computed as
where n is the size of the HDS and F(a.p^n_p) is the ath quantile of F( p;n _ p ).
6.3. T2 Charts with Unknown Parameters
101
Figure 6.2: Example of a T2 chart. Using these results, we can plot new incoming observations in sequence on a control chart. An example of such a chart is given in Figure 6.2. The chart is the same as that of a Phase I chart used in the purging operation with the exception that the UCL is computed using a different form of the F distribution. Any new observation whose T2 value plots above the UCL is declared a signal, and it is concluded that the observation does not conform to the baseline data set. To illustrate the T2 charting procedure, we consider observations taken from the steam turbine system illustrated in Figure 6.3. The input to this system is fuel in the form of natural gas that is used to produce steam in the boiler. The high-pressure steam is used to turn the turbine, which is connected to the generator that produces electricity (in megawatts-hours). The warm low-pressure vapor from the turbine is moved to the condenser, where it is converted into a liquid state for pumping to the boiler. The boiler uses heat to transform the liquid water into high-pressure steam. The system is considered to be at steady state and works in a continuous cycle. Baseline data for 28 observations on a steam turbine system are presented in Table 6.1. Measurements are made on the following variables: fuel usage (Fuel), the amount of steam (Steam Flow) produced, the steam temperature (Steam Temp), the megawatt-hour production (MW) of the turbine, the coolant temperature (Cool Temp), and the absolute pressure (Pressure) observed from the condenser. Estimates of the mean vector and covariance matrix, obtained from these data, are given as follows:
and
- 5.25£07 2.76£07 -11749.5 S= 3313.42 -408.673 .-169.391
2.76£07 1.91E07 -8112.06 2302.14 -203.795 -115.969
-11749.5 -8112.06 8.6918 -0.93735 0.152381 0.032143
3313.42 -408.673 -169.391 ' 2302.14 -203.795 -115.969 0.93735 0.152381 0.032143 0.285332 -0.02312 -0.0134 0.02312 0.043598 0.003757 0.0134 0.002474. 0.003757
Chapter 6. Charting the T2 Statistic in Phase II
102
Figure 6.3: Steam turbine system. Table 6.1: Phase I steam turbine data. Obs. No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Fuel 232666 237813 240825 240244 239042 239436 234428 232319 233370 237221 238416 235607 241423 233353 231324 243930 252550 251166 252597 243360 238771 239777 219664 228634 231514 235024 239413 228795
Steam Flow 178753 177645 177817 178839 177817 177903 177903 177990 177903 178076 177817 177817 177903 177731 178753 187378 187287 187745 188770 179868 181389 181411 167330 176137 176029 176115 176115 176201
Steam Temp 850 847 848 850 849 850 848 848 848 850 848 848 847 849 846 844 843 842 841 842 843 841 850 846 843 846 845 847
MW 20.53 20.55 20.55 20.57 20.57 20.59 20.57 20.55 20.48 20.49 20.55 20.55 20.55 20.53 20.64 21.67 21.65 21.67 21.78 20.66 20.81 20.88 19.08 20.64 20.24 20.22 20.31 20.24
Cool Temp 54.1 54.2 54.0 53.9 53.9 54.0 53.9 53.7 53.6 53.9 53.9 53.8 53.7 53.6 53.9 53.9 54.2 53.7 53.4 53.7 53.9 54.0 54.1 54.0 53.8 53.6 53.7 54.3
Pressure 29.2 29.2 29.2 29.1 29.2 29.1 29.2 29.1 29.1 29.1 29.1 29.1 29.1 29.1 29.1 29.1 29.1 29.1 29.1 29.1 29.1 29.1 29.2 29.2 29.2 29.2 29.2 29.2
6.3. T2 Charts with Unknown Parameters
103
Table 6.2: Phase II steam turbine data. Steam Fuel Obs. No. Flow 1 234953 181678 247080 189354 2 184419 238323 3 4 248801 189169 246525 5 185511 233215 6 180409 233955 181323 7 238693 181346 8 248048 185307 9 233074 181411 10 11 242833 186216 12 243950 182147 183349 238739 13 14 188012 251963 240058 183372 15 235376 16 182436 "Exceeds UCL = 43.91.
Steam Temp 843 844 845 843 842 845 842 844 844 844 844 844 844 850 846 844
MW 20.84 20.86 21.10 22.18 21.21 20.75 20.82 20.92 21.15 20.93 21.59 21.37 21.01 21.68 21.15 20.99
Cool Temp 54.5 54.4 54.5 54.5 54.6 54.5 54.6 54.8 54.6 54.5 54.4 54.2 54.3 54.4 54.2 54.3
Pressure 29.0 28.9 28.9 28.9 28.9 29.0 29.0 29.0 29.0 29.0 29.0 29.0 29.0 29.0 29.0 29.0
T2 35.00 167.98* 56.82* 69.48* 65.91* 32.56 43.91 49.33* 39.96 34.46 25.51 41.03 23.28 29.33 16.40 24.10
A T2 control procedure is developed, using these estimates, to monitor efficiency by detecting significant changes in any of the six monitored variables. The UCL for the chart is computed from (6.3), with a = 0.001, as
Considering the data in Table 6.1 as the HDS, T2 values for 16 new incoming observations are computed. Table 6.2 contains these values in addition to their corresponding T2 values. The T2 values are computed using (6.2) and the parameter estimates obtained from the historical data in Table 6.1. For example, the T2 value for observation 1 is
where
and the inverse matrix is obtained from S. A T2 chart for the new observations is presented in Figure 6.4. Out-of-control conditions occur at observations 2-5 and again at observation 8. Our conclusion is that these observations do not conform to the HDS presented in Table 6.1.
Reaction to a Signal Any observation that produces a T2 value falling outside its control region is a signal. This implies that conditions have changed from the historical situation.
104
Chapter 6. Charting the T2 Statistic in Phase II
Figure 6.4: T2 chart for new observations.
Although an isolated signal can be due to a chance occurrence of an upset condition, multiple signals often imply a definite shift in the process. Should we react to single signals or do we wait for multiple signals before declaring that there are upset conditions? The answer depends on the process setting, such as whether or not the problem can be immediately corrected or can wait until the next scheduled maintenance period. An engineer may choose not to react to an isolated out-ofcontrol point in order to minimize the likelihood of overcontrolling the process. In such settings, searches are made for trends and patterns in the T2 control chart, and out-of-control conditions are declared only when a number of T2 values plot outside the control limit. Also, consultation with the process engineer and operators helps in determining if upset conditions exist in the process. Consider the performance of a steam turbine that is operating at steady-state conditions. Generally, a fixed amount of fuel is supplied to the turbine and a fixed load (MW) is the output. In this state, one would expect little fluctuation from the baseline data. However, steady-state conditions are known to deteriorate under load changes. For example, MW production varies with fuel usage and increases as the load increases while decreasing when the load decreases. Even though load changes occur over a very short time period, they can produce erratic behavior in the control system. Situations such as these, where there are upsets with known causes, should be identified. When the cause is unknown, upset conditions are declared when several points fall outside the control region. Upsets do not produce a critical state in the operation of a steam turbine. However, upset conditions in certain types of processing units can and do produce critical or dangerous situations. For example, reconsider the chemical process example of section 6.2, where a product is produced that becomes hazardous when a particular component increases beyond a specified level. One would react immediately to any indication of process movement in this system due to the severe consequences of ignoring it. The determination of the occurrence of a signal depends on many factors. We recommend thorough understanding of the process, determination of
6.4. T2 Charts with Known Parameters
105
the critical components, and a risk analysis to determine the consequences of the actions to be taken.
6.4
T2 Charts with Known Parameters
Sometimes the parameters of the underlying MVN distribution for the process data are known. This may occur because extensive past information is available about the operations of the process. For example, known mean vectors can arise in an industrial setting where "old" steady-state processes are studied for extended periods of time. However, our experience indicates that this seldom occurs in practice. Nevertheless, it is useful to consider the case where these parameters are known, as the results provide information on the use and application of the limiting distribution of the T2 statistic. If the parameters of the underlying MVN distribution are known, the T2 value for a single observation vector in a Phase II operation is computed using
The probability function used to describe this T2 statistic is the chi-square distribution with p degrees of freedom as given in (2.13). This is the same distribution used to purge outliers when constructing the HDS. For a given value of a, the UCL is determined as where xfa „) ig the upper ath quantile of % 2 . In this case, the control limit is independent of the size of the HDS. To illustrate the procedure for signal detection in the known parameter case, consider a bivariate industrial process. A sample of 11 new observations and their mean corrected values are presented in Table 6.3. The purpose of the control procedure is to maintain the relationship between the two variables (xi,x-z) and to guarantee that the two variables stay within their operational ranges. The mean vector arid covariance matrix are given as
and
The T2 chart for the data in Table 6.3 is presented in Figure 6.5. Letting a = 0.05, observations 1 and 10 produce signals since their T2 values are above the UCL = 4,05,2) = 5-99. The occurrence of situations where the distribution parameters are known is rare in industry. Processing units, especially in the chemical industry, are in a constant state of flux. A change in the operational range of a single variable of a process can produce a ripple effect throughout the system. Many times these changes
106
Chapter 6. Charting the T2 Statistic in Phase II Table 6.3: New observations from an industrial process. XI Obs. No. 1 150.5 2 147.6 3 143.9 4 147.9 5 144.3 6 147.6 7 149.7 148.2 8 9 140.5 10 137.3 147.1 11 >i = 145.86; /^ =
X2
204.0 201.3 199.3 198.3 198.5 196.3 197.3 195.8 202.1 202.5 194.0 199.04.
xi - M i * 4.64 1.74 -1.97 2.03 -1.56 1.74 3.83 2.33 -5.47 -8.56 1.24
X2 - M 2 *
4.96 2.26 0.26 -0.74 -0.54 -2.74 -1.74 -3.24 3.06 3.46 -5.04
Figure 6.5: T2 chart for industrial process data. are initiated through pilot-plant studies, research center studies under controlled conditions, or from data obtained through other types of experiments. This requires constant updating of the baseline conditions, which in turn demands the use of new estimates of the parameters. Managers, performance engineers, process engineers, and the operators are constantly striving to improve the performance of the unit. There is no status quo.
6.5
T2 Charts with Subgroup Means
When the process observations at a given time point are made on a subgroup consisting of m vectors instead of only on a single vector, the mean of the subgroup is used in computing the charted value. If the individual vectors are described by a
6.5. T2 Charts with Subgroup Means
107
p-variate normal distribution Np(p,: S), we are assured that the mean vector X of a sample of m observations is distributed as a p-variate normal Np(n, E/m) with the same mean vector [i as an individual observation, but with a covariance matrix given by E/ra. If the individual observation vectors are not multivariate normally distributed, we are assured by the central limit theorem that as m increases in size, the distribution of X becomes more like that of an Np(n, E/ra). This produces the following changes in the charting procedure. When the parameters of the underlying MVN distribution are known, the T2 statistic for the iih subgroup mean Xi is computed by
and the UCL for a given a is determined by using (6.5). The control limit is independent of the sample size of either the subgroup or the HDS. When the parameters of the underlying MVN distribution are unknown, the T2 statistic for the ith sample mean Xi is computed as
where X and S are the common estimates of JJL and E obtained from the HDS. The distribution of a T2 statistic based on the mean of a subgroup of m observation vectors is given in (2.17). For a given a, the UCL for use with the statistic given in (6.7) is computed as
where n is the size of the HDS. To illustrate this procedure, we use the set of 21 observations on the electrolyzer data given in Table 4.6 in section 4.5 as an HDS. We calculate X and S from this data. Although overall performance of an electrolyzer is judged by the average efficiency of the 11 cells that compose it, performance is monitored by sampling only 4 of the individual cells. Replacement of the electrolyzer occurs when its monitored efficiency drops below the specified baseline values. Table 6.4 contains data from a sample of m — 4 cells taken from six different electrolyzers. Four variables are observed on each cell along with two composite gas samples from all 11 cells. Observations on C>2 and Cl2 are composite (average) gas samples from all 11 cells, including the four sampled cells. A UCL using a = 0.01 is calculated from (6.8) as
The T2 values for the average of the four cells for each electrolyzer are listed in Table 6.5. When compared to UCL = 10.28, we conclude that electrolyzers 573 and 963 are to be removed from service and refurbished.
Chapter 6. Charting the T2 Statistic in Phase II
108
Table 6.4: Electrolyzer data. Elect No. 573
Cell No. 1 5 6 7
AVG. 372
3 5 7 10
AVG. 834
1 2 8 10
AVG. 1021
2 5 9 11
AVG. 963
3 5 10 11
AVG. 622
3 4 6 9
AVG.
NaOH 126.33 137.83 124.30 131.72 130.04 122.17 129.13 128.83 146.93 131.77 128.17 129.17 138.42 141.73 134.37 141.40 137.29 144.00 139.20 140.47 118.40 144.19 128.10 128.01 129.68 136.81 133.93 140.71 140.40 137.96
NaCl 190.12 188.68 199.23 191.00 192.26 191.72 193.85 188.87 184.57 189.75 192.43 191.15 180.13 176.25 184.99 197.89 201.25 194.03 190.21 195.84 215.24 197.75 205.33 173.79 198.03 201.74 198.98 195.19 192.03 196.98
Ii
0.20 0.22 0.34 0.27 0.26 0.14 0.13 0.15 0.23 0.16 0.12 0.06 0.18 0.20 0.14 0.11 0.18 0.10 0.17 0.14 0.27 0.13 0.34 0.19 0.23 0.19 0.15 0.19 0.26 0.20
12
6.32 14.28 15.56 7.55 10.93 1.37 0.96 0.81 0.60 0.94 1.13 1.19 1.08 1.01 1.10 2.92 9.01 0.30 5.82 4.51 21.60 0.78 10.94 0.77 8.52 3.03 2.38 0.92 0.62 1.74
02
1.47 1.47 1.47 1.47 1.47 1.21 1.21 1.21 1.21 1.21 1.12 1.12 1.12 1.12 1.12 1.64 1.64 1.64 1.64 1.64 1.71 1.71 1.71 1.71 1.71 1.38 1.38 1.38 1.38 1.38
C12
98.10 98.10 98.10 98.10 98.10 98.36 98.36 98.36 98.36 98.36 98.43 98.43 98.43 98.43 98.43 97.91 97.91 97.91 97.91 97.91 97.82 97.82 97.82 97.82 97.82 98.18 98.16 98.16 98.16 98.18
Table 6.5: T2 values for sample means for electrolyzer data. T2 Value for Mean Vector 46.006* 7.391 8.693 1021 3.955 963 19.749* 622 0.803 "Exceeds UCL = 10.28.
Electrolyzer No. 573 372 834
6.6
Interpretive Features of T2 Charting
What do we expect to see in a T2 chart? The first part of the answer to this question is easy. Under ideal conditions, the T2 chart will exhibit the same randomness as one might expect to see in any type of charting procedure. For example, in a univariate Shewhart chart, one expects to see a predominance of the plotted
6.6. Interpretive Features of T2 Charting
109
Figure 6.6: T2 chart for mercury cell data.
points distributed randomly about the center line. This occurs because, for a normal distribution, approximately 68% of the observations are contained within one standard deviation of the mean (centerline). Does something similar occur in a T2 chart? The answer is no, but it is not emphatic. In some types of industries, T2 charts are often unique and can be used to characterize the behavior of the process. Close study of the plotted statistic can produce valuable insight on process performance. Upset conditions, after they occur, become obvious to those involved. However, process conditions leading to upsets are not as obvious. If so. there would be few upsets. If the precursor conditions can be identified by examining the T2 plot, sometimes it is possible to avoid the upset. Figure 6.6 is the plotted T2 statistic for a control procedure on a mercury cell (Hg cell), which is another type of processing unit used to produce chlorine gas and caustic soda. Seven process variables are observed simultaneously in monitoring the performance of the cell. The plotted T2 statistics over the given time period illustrated in Figure 6.6 indicate a very steady-state iri-control process relative to the baseline data set. There is very little change in the pattern of the T2 statistic. The UCL, as determined from the HDS, has a value of 18.393; however, the values of the plotted T2 statistic are consistently located at a substantial distance below this value. Any erratic or consistent movement of the observed T2 values from the established pattern of Figure 6.6 would indicate a process change. Figure 6.7 illustrates such a condition, where the T2 values are increasing towards the control limit. Had process intervention been initiated around observation 1000, it may have been possible to prevent the upset conditions that actually occurred at the end of the chart. Of course, one needs tools to determine what variable or group of variables is the precursor of the upset conditions. These will be discussed in Chapter 7. Figures 6.6 and 6.7 present another interesting T2 pattern. Notice the running U pattern contained in both charts. This is due to the fluctuation in the ambient
110
Chapter 6. Charting the T2 Statistic in Phase II
Figure 6.7: T2 chart with precursor to upset.
Figure 6.8: T2 chart for glass furnace data. conditions from night to day over a 24-hour period, and it represents a source of extraneous variation in the T2 charts. Such variation can distort the true relationships between the variables, and can increase the overall variation of the T2 statistic. For example, the cluster of points between observations 800 and 900 in Figure 6.7 are U-shaped. Do they represent a process change or a change in ambient conditions? Removing the effect of ambient conditions would produce a clear process picture. Another example of a T2 chart is presented in Figure 6.8. Given are the T2 values for data collected on 45 process variables measured in the monitoring of a furnace used in glass production. Observe the steady-state operating conditions of
6.7. Average Run Length (Optional)
111
the process from the beginning of the charting to around observation 500. This pattern reflects a constant trend with minimal variation. After observation 500, the T2 values exhibit a slow trend toward the UCL with upset conditions occurring around observations 500, 550, 600, and 650. Corrections were made to the process and control regained at about observation 650. However, note the increase in variation of the T2 values and some signals between observations 650 and 1350. The T2 plot flattens out beyond this point and the steady-state pattern returns. These examples illustrate another important use of the T2 control chart. After the trends in the T2 chart have been established and studied for a process, any deviation from the established pattern indicates some type of process change. Sometimes the change is for the better, and valuable process knowledge is gained. Other times, the change is for the worse, and upset conditions occur. In either case, we recommend the investigation of any change in the plotted T2 values. Using this approach, expensive upsets that lead to chaotic conditions can be avoided.
6.7
Average Run Length (Optional)
The average run length (ARL) for a control procedure is defined as
where p represents the probability of being outside the control region. For a process that is in control, this probability is equal to a, the probability of a Type I error (see section 6.2). The ARL has a number of uses in both univariate and multivariate control procedures. For example, it can be used to calculate the number of observations that one would expect to observe, on average, before a false alarm occurs. This is given by Another use of the ARL is to compute the number of observations one would expect to observe before detecting a given shift in the process. Consider the two univariate normal distributions presented in Figure 6.9. One is located at the center line (CL) and the other is shifted to the right and located at the UCL. The probability of detecting the shift (i.e., the probability of being in the shaded region in Figure 6.9) equals (1 — /3), where /3 is the probability of a Type II error (see section 6.2). Given the shift, this probability can be determined using standard statistical formulas (e.g.. see Montgomery (2001)). The ARL for detecting the shift is given by From Chapter 5, we recognize that the probability (1 — (3) represents the power of the test of a statistical hypothesis that the mean has shifted. This result produces another major use of the ARL, which consists of comparing one control procedure to another. This is done by comparing the ARLs of the two procedures for a given process shift. Shifts and the probability of detection, (1 — /3), are easy to compute in the univariate case. However, it is more difficult to do these calculations in the multivariate
112
Chapter 6. Charting the T2 Statistic in Phase II
Figure 6.9: Univariate process shift.
Figure 6.10: Multivariate process shift.
case. Consider a bivariate control region and a mean shift of the process as represented in Figure 6.10. We assume that the covariance matrix has not changed and remains constant for the multivariate distributions. Hence, the orientation of the control region for the shifted distribution is the same as that of the control region for the in-control process. The area of the shifted distribution that corresponds to the shaded region in Figure 6.10 equals the probability (1 — /?) of detecting the shift. This probability can be computed analytically for p = 2, but becomes very
6.7. Average Run Length (Optional)
113
Figure 6.11: Control region and chi-square distribution.
difficult for higher dimensions. However, using additional statistical theory and the nonnull distribution of the T2 statistic, the problem can be simplified. Suppose the parameters, \ix and S, of the MVN distribution are known. The T2 control region for an in-control observation vector X is described by a chisquare distribution (see section 6.4) and can be compared to the UCL based on that distribution; i.e.,
In terms of the chi-square distribution, the control region is represented by the shaded region in Figure 6.11. This particular distribution is known as a central chi-square. This is due to the fact that the vector (X — }JLX} is described by an MVN distribution with a mean vector of zero. If one considers an observation vector Y from another MVN distribution with the same covariance matrix but with a different mean vector jjLy. its T2 value is given by but it cannot be described by the central chi-square distribution. This is because the MVN that describes the vector (Y — /i x ) has a mean different from zero. However, we can determine the mean of the normal vector (Y — /j,x) in terms of /j,x and ^y. Consider
where 6 = (/j,y — IJLX} represents the mean shift. With this result, the distribution of T2 is given by
114
Chapter 6. Charting the T2 Statistic in Phase II
Figure 6.12: Shifted chi-squared distribution.
where x'(^ \\ is a noncentral chi-square distribution with p degrees of freedom. A major difference between this distribution and the central chi-square is the additional parameter A, labeled the noncentrality parameter. It can be expressed as
Changing the noncentrality parameter produces changes in the distribution. For example, the mean of a central chi-square is given by the degrees of freedom, p, whereas for the noncentral distribution, the mean equals (p + A). Similarly, the variance changes from 2p for the central chi-square to (2p + 4A) for the noncentral chi-square. As A becomes large, the differences in the two distributions are significant, but as A approaches zero, the noncentral distribution approaches the central chi-square. Our interest lies in computing the probability of detecting a given shift in the process. This is accomplished by computing probabilities from the noncentral chisquare. Another representation of the shift depicted in Figure 6.10 is given in terms of the central and noncentral chi-square rather than in terms of normal distributions. This is represented in Figure 6.12. The central chi-square distribution represents the in-control situation and the noncentral chi-square distribution represents the distribution shifted to the right. It can be shown that the shaded area (1 — /3) under the noncentral chi-square distribution in Figure 6.12 is the same as the shaded region of Figure 6.10. For univariate control procedures, shifts of the process distribution are usually expressed in terms of the standard deviation. This is not feasible with multivariate processes, since shifts involve more than one variable. Shifts of multivariate processes are expressed in terms of values of the noncentrality parameter A, which is a function of both the mean distance between the two distributions and the covariance matrix S.
6.8. Plotting in Principal Component Space (Optional)
115
To determine the probability of detecting a given shift, the noncentral chi-square distribution must be evaluated above the UCL. This integral is given as
where w ~ X 2 p + 2?) , and where UCL = X?a.p,xy A similar discussion can be presented for a Phase II operation where the parameters are unknown. The distributions involved are the central F distribution for the in-coritrol situation and the noncentral F for the shifted distribution. Again, shifts are measured in terms of the same noncentrality parameter A. The probability of detecting various shifts (A) for a given sample size n, significance level a, and number of variables p can be determined by integrating the noncentral F distribution similar to (6.9). The shift studied in this section and depicted in Figure 6.10 is a simple mean shift. Unfortunately, not all shifts are of this type. For example, in a multivariate setting, there can be simultaneous shifts of the mean vector and covariance matrix. Computing the probability of such shifts is much more complicated than the above examples.
6.8
Plotting in Principal Component Space (Optional)
In Chapter 4, we introduced PCA as a tool used in the location of singularities in the covariance matrix. In this section, we expand the use of principal components to include a charting procedure for the T2 statistic. This use requires more mathematical rigor than previously given. The plotting of T2 values in a principal component space can provide additional insight into process performance for certain types of problems. Consider a process generated from a bivariate normal distribution with known parameters /j, and S. For an observation vector X, control of the process is maintained when the T2 statistic is less than the UCL; i.e.,
Recall from Chapter 2 that the curve T2 = UCL establishes an elliptical control region. An example of such a control region, where x\ and x% are positively correlated, is illustrated in Figure 6.13. A number of observations can be made about the control region represented in Figure 6.13. It is referenced by three different coordinate systems. The first is the variable space, represented by (.TI, x ^ ) . This is obtained by expanding (6.10) as a function of x\ and x2 and constructing the graph. If we standardize x\ and £2, using the values
116
Chapter 6. Charting the T2 Statistic in Phase II
Figure 6.13: Control ellipse for a bivariate system. we obtain the translated axes (yi, 7/2) located at the center of the ellipse in Figure 6.13. The T2 statistic in terms of y\ and 7/2 takes the form
r2 = (x - ^'T,-I(X -n} = Y'P~IY, where Y' = (yi, 2/2) and P is the correlation matrix for X. In the (yi, 7/2) space the ellipse remains tilted as it was in the (xi, #2) space since the correlation between yi and 7/2 is the same as the correlation between x\ and x^. The third set of axes in Figure 6.13 is obtained using the transformation
Under this transformation, the T2 has the form
where Z' = (21,22) and the matrix A is a diagonal matrix with the eigenvalues of P along the diagonal. The above rotation of the (xi, x^} space to the (21, 22) space removes the dependency (correlation) between x\ and x^. In the (z\, z^) space, the elliptical control region is not tilted, since z\ and z2 are independent. Further, the z\ and z^ values are expressed as linear combinations of y\ and y^ and, hence, ultimately as linear combinations of x\ and x^. As such, these variables are the principal components of the correlation matrix for x\ and x 2 . If x\ and x% are not standardized, the z\ and z<2 variables are the principal components of the corresponding covariance matrix E and will have a different representation. For purposes of plotting, it is usually best to use the correlation matrix.
6.8. Plotting in Principal Component Space (Optional)
117
Figure 6.14: Principal component control region. The control region for the T2 statistic can be written in terms of z\ and z^ by expanding the matrix multiplication of (6.10) to obtain
where p is the population pairwise correlation between x\ and x^. Note that the eigenvalues of the correlation matrix for x\ and x% are (1 + p) and (1 — p), so the T2 statistic is now expressed in terms of these variables. If the equation in (6.13) is set equal to the UCL, it forms a bivariate elliptical control region that can be used as a charting procedure in the principal component space, i.e., in terms of z\ and z%. For example, given a bivariate observation (xi, £2), we can standardize the observations using (6.11). The principal components z\ and £2 are computed using (6.12) and plotted in the principal component space. Observations plotting outside the elliptical region are out of control, as they do not conform to the HDS. The point A in the principal component control region presented in Figure 6.14 illustrates this. The method of principal component translation can be generalized to the pdimensional case. The control region for the T2 statistic can be expressed in terms of the p principal components of the correlation matrix as
where AI > A2 > computed as
> Ap are the eigenvalues of the correlation matrix. Each Zi is
where Ui: i = 1,2, . . . , p , are the corresponding normalized eigenvectors of the correlation matrix and Y is the standardized observation vector. Again, note that each Zi is a linear combination of the standardized process variables.
Chapter 6. Charting the T2 Statistic in Phase II
118
The graph of (6.14), when set equal to the control limit, is a hyperellipsoid in apdimensional space. However, plotting this region for p > 3 is not currently possible. As an alternative, one can plot any combination of the principal components in a subspace of three or fewer dimensions. This procedure also has a major drawback. Any point (2j, Zj, z^) that plots outside the region defined by (6.14) will produce a signal, but there is no guarantee that a point plotting inside the region does not contain a signal on another principal component.
6.9
Summary
Signal detection is an important part of any control procedure. In this chapter, we have discussed charting the T2 statistic when monitoring a process in a Phase II operation. This includes the charting of the T2 statistic based on a single observation and the T2 statistic based on the mean of a subgroup of observations. Included are procedures to follow when the parameters of the underlying MVN distribution are known and when they are unknown. In both cases it is assumed that the covariance structure is nonsingular. Also contained in this chapter is a discussion on determining the ARL for a T2 chart. To calculate an ARL for a given mean shift in a multivariate distribution involves the introduction of noncentral distributions and the evaluation of some complicated integrals. For the reader who has a deeper interest in this area, there are many excellent texts in multivariate analysis on this subject, e.g., Johnson and Wichern (1998), Fuchs and Kenett (1998), and Wierda (1994). An optional section on the plotting of the T2 statistic in a principal component space was presented. As pointed out in the discussion, the procedure has both advantages and disadvantages. A major advantage is that one can plot and observe signals on particular principal components in a subspace of the principal component space. However, a major disadvantage is that each principal component is a linear combination of all the process variables. This often inhibits a straightforward interpretation procedure in terms of the process variables.
Chapter 7
Interpretation of T2 Signals for Two Variables
7.1 Introduction Univariate process control usually involves monitoring control charts for location and variation. For example, one might choose to monitor mean shifts with an X chart and variation shifts with an R chart, as both procedures are capable of detecting deviations from the historical baseline. In this setting, signal interpretation is simplified, as only one variable needs to be examined. A signal indicates that either the process has shifted and/or the process variation has changed. In multivariate SPC, the situation becomes more complicated. Nonconformity to a given baseline data set can be monitored using the T2 statistic. If the observed T2 value falls outside the control region, a signal is detected. The simplicity of the monitoring scheme, however, stops with signal detection, as a variety of variable relationships can produce a signal. For example, an observation may be identified as being out of control because its value for an individual variable is outside the bounds of process variation established by the HDS. Another cause of a signal is when values on two or more variables do not adhere to the linear correlation structure established by the historical data. The worst case is when the signal is a combination of the above, with some variables being out of control and others being countercorrelated. Several solutions have been posed for the problem of interpreting a multivariate signal. For example, Doganaksoy, Faltin, and Tucker (1991) proposed ranking the components of an observation vector according to their relative contribution to a signal using a univariate t statistic as the criterion. Hawkins (1991, 1993) and Wade and Woodall (1993) separately used regression adjustments for individual variables to improve the diagnostic power of the T2 after signal detection. Runger, Alt, and Montgomery (1996) proposed using a different distance metric, and Timm (1996) used a stepdown procedure for signal location and interpretation. An overview of several of these multivariate process control procedures, including additional 119
120
Chapter 7. Interpretation of T2 Signals for Two Variables
ones by Kourti and MacGregor (1996) and Wierda (1994), can be found in Mason, Champ, Tracy, Wierda, and Young (1997). Also, several comparisons are given in Fuchs and Kenett (1998). In this chapter, we present a method of signal interpretation that is based on the orthogonal decomposition of the T2 statistic. The independent decomposition components, each similar to an individual T2 variate, are used to isolate the source of a signal and simplify its interpretation. The discussion is limited to a two-variable problem, as it is the easiest to geometrically visualize. The more general p-variable case is presented in Chapter 8.
7.2
Orthogonal Decompositions
The typical T2 statistic for an observation vector X' = (xi, X 2 , . . . , xp) is given as
The major purpose of our discussion in this section is to provide a methodology for interpreting a signal with this statistic. A procedure for achieving this goal is to decompose or separate the T2 value into additive orthogonal components that can be related to the p process variables. A close examination of how this is done will provide insight into the workings and understanding of the proposed T2 decomposition. Orthogonal decompositions are standard tools in well-known statistical procedures such as the analysis of variance and regression analysis. For example, in regression analysis, the total sum of squares (i.e., total variation) of the response variable is separated into two independent additive components. One component, the regression sum of squares, measures the contribution of the predictor variables (to the total variation), and the other component, the residual sum of squares, measures the contribution of the model error (to the total variation). A similar decomposition approach can be used with the T2 statistic. Consider a two-variable case where an observation vector is denoted by X' = (xi^x-z) and has a bivariate normal distribution with unknown mean vector and covariance matrix. Assume that the two variables are independent so that, in a sample of n observations, their pairwise correlation is zero. Also assume that the two individual sample variances, s2 and s 2 , are unequal, and denote the two corresponding sample means as x\ and x 2 . AT 2 elliptical control region is created using the following formula:
where c is an appropriately chosen constant that specifies the size of the control region (see section 6.2). A typical control region is illustrated by the interior of the ellipse given in Figure 7.1. Any sample point located on the ellipse would be located the same statistical distance from the sample mean as any other point on the ellipse.
7.2. Orthogonal Decompositions
121
Figure 7.1: Bivariate independent control region with unequal variances.
There are two additive components to the T2 statistic given in (7.1), and these provide a natural decomposition of the corresponding statistic. The components, in fact, are independent due to the independence of the two original x variables. This property is what causes the ellipse not to be tilted. Since the components are unequally weighted due to their unequal variances, we will transform the variables to a form that will provide equal weights. Doing so will produce a circular region and make the statistical distance, represented by the square root of the T2 in (7.1), equivalent to the corresponding Euclidean, or straight-line, distance. This is the view we need in order to interpret the T2 value. Let represent the standardized values of x\ and x2j respectively. Using this transformation, we can re-express the T2 value in (7.1) as
The T2 value is again separated into two independent components as in (7.1), but now the components have equal weight. The first component, y 2 , measures the contribution of x\ to the overall T2 value, and the second component, y2: measures the contribution of x2 to the overall T2 value. Careful examination of the magnitude of these components will isolate the cause of a signal. The control region using the transformation given in (7.3) is illustrated by the interior of the circle depicted in Figure 7.2. A T2 value in this orthogonal transformed space is the same as the squared Euclidean distance that a point (2/1,3/2) is from the origin (0,0), and it can be represented by the squared hypotenuse of the
122
Chapter 7. Interpretation of T2 Signals for Two Variables
Figure 7.2: Bivariate orthogonal control region with equal variances. SD refers to statistical distance.
enclosed right triangle depicted in Figure 7.2. This is equivalent to the statistical distance the point (zi,#2) is from the mean vector (x\, £2)- Thus, all points with the same statistical distance are located on the circle in Figure 7.2, as well as on the ellipse in Figure 7.1. Consider a situation where the variables of X' = (xi,x 2 ) are not independent. Since the pairwise correlation r between x\ and X2 is nonzero, the T2 value would be given as
and the corresponding elliptical control region would be tilted. This is illustrated in Figure 7.3. Again, letting yi and y2 represent the standardized values of xi and a?2, the T2 value in (7.5) can be written as
Unfortunately, as can be seen from examining (7.6), we cannot separate the contributions of yi and 7/2 to the overall T2 value, since transforming to the standardized space does not remove the cross-product term; i.e., the transformation is not orthogonal. Thus, the resultant control region is elliptical and tilted, as is illustrated in Figure 7.4. To express the statistical distance in (7.6) as a Euclidean distance so that it can be visualized, we must first transform the axes of the original (xi, #2) space
7.2. Orthogonal Decompositions
123
Figure 7.3: Bivariate nonindependent control region with unequal variances.
Figure 7.4: Bivariate translated control region.
to the axes of the ellipse. This is not done in either Figure 7.3 or Figure 7.4. For example, the axes of the ellipse in Figure 7.4 do not correspond to the axes of the (yii 2/2) space. The axes can only be aligned through an orthogonal transformation. In this nonindependent case, the transformation in (7.6) is incomplete, as it does not provide the axis rotation that is needed. The T2 statistic in (7.6) can be separated into two additive components using the orthogonal transformation
124
Chapter 7. Interpretation of T2 Signals for Two Variables
Figure 7.5: Bivariate principal component control region with unequal variances.
and
As was shown in Chapter 6, the values z\ and z^ are the first and second principal components of the correlation matrix for x\ and x% (also see the appendix to this chapter, section 7.10). Using this transformation, we can decompose the T2 value in (7.6) as follows:
Unfortunately, a graph of the control region of (7.8) in the principal component space still does not allow easy computation of the statistical distance since the principal components are unequally weighted. This is illustrated in Figure 7.5 by the different sizes of the axes of the ellipse. The first principal component has a weight equal to the reciprocal of AI = (1 + r) and the second principal component is inversely weighted by A2 — (1 — r). The weights are equal only when r = 0, i.e., when the original variables are independent. The above problem can be circumvented by transforming the z values in (7.7) to a new space where the variables have equal weights. One such transformation is
and
7.3. The MYT Decomposition
125
Figure 7.6: Bivariate principal component control region with equal variances.
Using this orthogonal transformation, we can express the T2 value in (7.1) as
The resultant control region, presented in Figure 7.6, is now circular, and the (squared) statistical distance is represented by the hypotenuse of a right triangle. The transformation given in (7.10) provides an orthogonal decomposition of the T2 value. Thus, it will successfully separate a bivariate T2 value into two additive and orthogonal components. However, each w\ and u>2 component in (7.10) is a linear combination of both x\ and x^. Since each component consists of both variables, this hampers clear interpretation as to the source of the signal in terms of the individual process variables. This problem becomes more severe as the number of variables increases. What is needed instead is a methodology that will provide both an orthogonal decomposition and a means of interpreting the individual components. One such procedure is given by the MYT (Mason-Young-Tracy) decomposition, which was first introduced by Mason, Tracy, and Young (1995).
7.3
The MYT Decomposition
Since the T2 statistic is a sum of squares, there are an infinite number of ways to separate it into p independent (orthogonal) components. For example, the decomposition of the T2 statistic into its principal components is one such representation. As discussed in section 7.2, a major problem with decompositions of this type is the interpretation of the components. For example, principal components can be
126
Chapter 7. Interpretation of T2 Signals for Two Variables
difficult to interpret, as they are linear combinations of the p variables of the observation vector. The components of the MYT decomposition of the T2 statistic, in contrast, have global meaning. This is one of the most desirable characteristics of the method. We will demonstrate the MYT procedure for a bivariate observation vector X' = (#1, £2)5 where x\ and x% are correlated. Details on the more general p-variable case can be found in Chapter 8. The MYT decomposition uses an orthogonal transformation to express the T2 values as two orthogonal and equally weighted terms. One such decomposition is given by
where
and In this formulation, x2.i is the estimator of the conditional mean of x% for a given value of xi, and s|.i is the corresponding estimator of the conditional variance of #2 f°r a given value of x\. Details on these estimators are given in the last section of this chapter. The first term of the MYT decomposition in (7.11), Tf, is referred to as an unconditional term, as it depends only on x\. The second term of the orthogonal decomposition, written as T|1? is referred to as a conditional term, as it is conditioned on the value of x\. Using the above notation, we can write (7.11) as
The square root of this T2 value can be plotted and viewed in the T\ and T^.i space. This is illustrated in Figure 7.7. The orthogonal decomposition given in (7.11) is one of two possible MYT decompositions of a T2 value for p = 2. The other decomposition is given as
where
and The representation of T2 given in (7.15) is different from the representation given in (7.11). The first term of the decomposition in (7.15) is an unconditional term for the variable x-z, whereas the first term of (7.11) is an unconditional term for the variable x\. Similarly, the conditional term of (7.15) depends on the conditional density of x\ given #2, while the conditional term of (7.11) depends on the conditional density of #2 given x\. These two conditional terms are not the same except in the case where x\ and x% are uncorrelated.
7.4. Interpretation of a Signal on a T Component
127
Figure 7.7: T2 plot for MYT bivariate decomposition.
7.4
Interpretation of a Signal on a T"2 Component
Equations (7.11) and (7.15) represent the two possible MYT decompositions for an observation vector X' — (a?i, x^}. Considering both decompositions, we have four unique terms: two unconditional terms given by T2 and T| and two conditional terms represented by T 2 2 and Tjl. A signaling T2 value can produce large values on any combination of these terms. Consider a large value for either the unconditional term
or the unconditional term An unconditional term is the square of a univariate one-sample t statistic, and it measures the statistical distance of the observed value Xi from its mean a^. If the observed value of the variable is out of tolerance (i.e., outside its operational range as based on the HDS), a signal is obtained on the unconditional term. Interpretation of a large value on a conditional term is more involved. Consider the conditional term which is a measure of the squared statistical distance of the observed value x% is from the conditional mean £2.1- This distance can be examined graphically. For example, consider the representation of the control region in the variable space
128
Chapter 7. Interpretation of T2 Signals for Two Variables
Figure 7.8: Interpretation of the T221 component.
Figure 7.9: Interpretation of the T 2 2 component. (xi, # 2 ) presented in Figure 7.8. As the value of x\ changes, so does the value of X2.i. Consider a fixed value of xi, say x\ = a. This is represented by the vertical line drawn upwards from the x\ axis. For process control to be maintained at this value of #1, the corresponding value of x% must come from the shaded interval along
7.5. Regression Perspective
129
the x-2 axis. This means the value of x% must be contained in this portion of the conditional density, otherwise, a signal will be obtained on the T|: component. A similar discussion can be used to illustrate a signal on the T^2 term. Suppose the value of x^ is fixed at a point b and we examine the restricted (conditional) interval that must contain the observation on x\. This is depicted in Figure 7.9. If the value of x\ is not contained in the shaded interval, a signal will be obtained on the T^2 component. A large value on a conditional term implies that the observed value of one variable is not where it should be relative to the observed value of the other variable. Observations on the variables (xi, #2) that produce a signal of this type are said to be countercorrelated, as something is astray with the relationship between x\ and X2- Countercorrelations are a frequent cause of a multivariate signal.
7.5
Regression Perspective
As stated earlier, a signal on a conditional term implies something is wrong with the linear relationships among the involved variables. Additional insight for interpreting a signaling conditional term is obtained by examining these terms from a regression perspective. Note that the line labeled x^.i in Figure 7.8 is the regression line of x-2 on xi: and likewise, the line labeled x\^ in Figure 7.9 is the regression line of x\ on x^. In general, Tf- is a standardized observation on the ith variable adjusted by the estimates of the mean and variance from the conditional distribution associated with Xi.j. For the bivariate case, the general form of a conditional term is given as
Consider the estimated mean of x% adjusted for Xj, i.e., Xi.j. This is given as where Xi and Xj are the sample means of Xi and Xj obtained from the historical data, and b is the estimated regression coefficient relating Xi to Xj in this data set. The left-hand side of (7.19) contains Xi.j, which is the predicted value of x^ based on the corresponding value of Xj (i.e., x^ is the dependent variable and x3 is the predictor variable). Thus, the numerator of (7.18) is a regression residual; i.e., Rewriting the conditional variance as where JT^.J is the multiple correlation between Xi and x j ; and substituting r^.j for (xi — Xj.j), we can re-express T? • as or
We use this notation for consistency with the formula used in the p-dimensional case discussed in Chapter 8.
130
Chapter 7. Interpretation of T2 Signals for Two Variables
Figure 7.10: Residual from regression of x% on x\.
The conditional T2 term in (7.20) explains how well a future observation on a particular variable is in agreement with the value predicted by the other variable. When the denominator in (7.20) is very small, as occurs when the pairwise correlation is near 1, we would expect very close agreement between the observed and predicted Xi values. Otherwise, "largeness" of the conditional T2 term will be due to the numerator, which is a function of the agreement between the observed and predicted values of Xj. A significant deviation between these two values will produce a large T2 term. As an example, consider an observation (#1, #2) that is inside the Shewhart control "box" formed using the two variables. Suppose there is significant disagreement in magnitude between an observed x\ value and the corresponding predicted value obtained using the value of £2 and (7.19). This implies that the observation on this particular component is below or above what was predicted by the HDS. To better understand the result in (7.20), consider the two conditional T2 terms whose square roots are given by
and
where r^.i = (#2 — 2:2.1) and ri.2 = (x\ — #1.2) are residuals from the respective regression fits of x% on x\ and x\ on x-2- These residuals are illustrated in Figures 7.10 and 7.11.
7.6. Distribution of the T2 Components
131
Figure 7.11: Residual from regression of x\ on x%-
Notice that the two conditional values in (7.21) and (7.22), apart from the R? j term, are actually standardized residuals having the form of 7Yj/Si. When the residuals (after standardizing) in Figures 7.10 and 7.11 are large, the conditional T2 terms signal. This would occur only when the observed value of x\ differs from the value predicted by x^, or the observed value of x% differs from the value predicted by xi, where prediction is derived from the HDS.
7.6
Distribution of the T"2 Components
Issues pertaining to the largeness of the T2 components of the MYT decomposition can be resolved by determining the probability of observing specific (or larger) values for each individual term. In order to do this, we must know the probability distribution of the individual terms. All terms, both conditional and unconditional, under the assumption of no signal, are described by an F distribution. For example, the unconditional terms that are used to determine whether the individual variables are within tolerance are distributed as
for j = 1,2. Similarly, the conditional terms, T?-, used in checking the linear relationships between the variables are distributed as
where k equals the number of conditioned variables. When k = 0, the distribution in (7.24) reduces to the distribution in (7.23). Derivational details are supplied in
132
Chapter 7. Interpretation of T2 Signals for Two Variables
Mason, Tracy, and Young (1995). Thus, one can use the F distribution to determine when an individual unconditional or conditional term of the decomposition is significantly large and makes a contribution to the signal. The procedure for making this determination is as follows. For a specified a level and HDS sample of size n, obtain ^(a,i,n-fc-i) from the appropriate F table. Compute the UCL for individual terms using
Compare each individual unconditional term of the decomposition to the appropriate UCL. All terms satisfying
imply that the corresponding Xj is contributing to the signal. Likewise, any conditional term greater than its UCL, such as
implies that Xi and Xj are both contributing to the signal. The above procedure for locating contributing terms of a decomposition for a signaling observation vector is not exact. To see this, consider the acceptance region for maintaining control for both conditional and unconditional terms when p = 2. A MYT decomposition for a T2 value can be represented bv
Using this representation, the T? acceptance region is given by
Likewise, the TjA acceptance region is defined by
The relationship between these regions, as denned by the distribution of the individual terms and the elliptical control region, is presented in Figure 7.12. This demonstrates that, when we use the distribution of the individual terms of a T2 decomposition to detect signal contributors, we are approximating the elliptical control region with a parallelogram. This parallelogram acceptance region is equivalent to the acceptance region specified by a cause-selecting (CS) chart (e.g., see Wade and Woodall (1993)). The CS method is based on the regression adjustment of one variable for the value of the other. The same approach is used in constructing the conditional terms of the MYT decomposition. In general, the CS procedure
7.6. Distribution of the T2 Components
133
Figure 7.12: Acceptance regions for T2 and T^.
does not require the assumption of nmltivariate normality. However, without this assumption, or some distributional assumption, it is difficult to detect a signal. We would like the parallelogram and ellipse in Figure 7.12 to be similar in size. The size of the ellipse is determined by the choice of the overall probability, labeled cci, of making a Type I error or of saying the process is out of control, when in fact control is being maintained. The size of the parallelogram is controlled by the choice of the specific probability, labeled a 2 , used for testing the "largeness" of individual terms. Thus, a-2 represents the probability of saying a component is part of the signal when in fact it is not. The two a's are not formally related in this situation. However, ambiguities can be reduced by making the two regions agree in size. We use the F distributions in (7.23) and (7.24) to locate large values among the T2 decomposition terms. This is done because, given that an overall signal exists, the most likely candidates among the unique terms of a total MYT decomposition are the components with large values that occur with small a priori probabilities. Our interest in locating the signaling terms of the MYT decomposition is due to the ease of interpretation for these terms. Consider an example to illustrate the methodology of this section. For p = 2, the control region is illustrated in Figure 7.13. Signals are indicated by points A, B, C. and D. Note that the box encompassing the control region represents the tolerance on variables x\ and x^ for a = 0.05, as specified by (7.26). The tolerance regions are defined by the Shewhart control limits of the individual variables for the appropriate a level.
134
Chapter 7. Interpretation of T2 Signals for Two Variables
Figure 7.13: T2 control region with four signaling points. Table 7.1: Decomposition summary for bivariate example. T2 Value Point A 10.05* B 6.33* 6.63* C D 9.76* * Denotes significance
2 T J 2 1? 10.03* 2.78 3.49 0.11 0.34 3.01 2.54 1.73 at the 0.05 level.
2 T J
1.2
0.02 2.83 6.29* 8.03*
2 T J
2.1
7.27* 6.22* 3.62 7.22*
A summary of the T2 decomposition values for the four signaling points is presented in Table 7.1. Observation A produces a large value on the unconditional term T|, since the observation on x-2 is out of tolerance. The T|x term for this point also is large, since x 2 is not contained in the conditional range of x% given xi. This is evident in the distance A is from the regression line denoted as x 2 .i. Likewise, observation B produces a large value for its T22 x term since the observation is a great distance from the same regression line. However, its two unconditional terms are not significant, since individually the variables are within the box region illustrated in Figure 7.13. Observation C has acceptable values on both unconditional terms, since both variables are within tolerance. However, a large value is produced on T 2 2 , as the observed x\ is not contained in the conditional distribution of x\ given x 2 . Again, note the extreme distance that point C is from the regression line, labeled 0)1.2, of x\ on x 2 . Observations on both variables are within tolerance for point D, but observations on either variable are not where they should be relative to the position of the other variable. This produces large values for the two conditional terms.
7.7. Data Example
135
Figure 7.14: T2 chart for the boiler data.
7.7
Data Example
In this section, we consider a more detailed example to demonstrate the techniques developed for signal interpretation and also to reemphasize previously covered points in the development of a control procedure. As the production unit, we examine a boiler used to produce steam for industrial use. Two of the variables used in the control procedure that monitor boiler efficiency are fuel usage (i.e., fuel) and steam flow (i.e., stmflow). The T2 values for an HDS consisting of 500 data points are presented in Figure 7.14. We now examine the HDS from a statistical perspective. The T2 chart is indicative of the performance of the boiler. All T2 values are below the UCL value of 9; however, there appears to be a certain amount of variation and several step changes in the values. Note the consistency of the T2 values up to observation 250, the higher values between observations 250 and 350, and the lower values thereafter. Later investigation will show that the higher values, from 250 to 350, are caused by a period of consistent low steam production. The unusual spikes are produced when a rapid change in the production rate occurs. This becomes apparent when the two individual variables are plotted in time sequence as shown in Figure 7.15. The vertical axis is labeled "Units." Fuel usage and steam flow are measured in different units. However, the values are close enough together that one axis can be used for both measurement systems. Close examination of Figure 7.15 reveals the strong relationship between the two variables. Fuel usage is always higher than the corresponding steam flow; however, the distance between the two points increases with increases in fuel usage. This is more readily apparent when only the first few points are examined, as shown in Figure 7.16. A comparison of the time-sequence chart in Figure 7.15 with the T2 chart in Figure 7.14 reflects close similarity. Low values on the time-sequence chart for the two variables correspond to high values on the T2 chart, and high values on the time-sequence chart correspond to low T2 values. This is because the HDS is
136
Chapter 7. Interpretation of T2 Signals for Two Variables
Figure 7.15: Time-sequence chart for fuel usage and steam.
Figure 7.16: Time-sequence chart for first few points of boiler data. dominated by moderate-to-high values of the two variables, making the low values farther from the mean vector. Since the T2 statistic is a squared number, values far from the mean are large and positive. A Q-Q plot of the 500 ordered T2 values versus the corresponding beta values for the boiler historical data is presented in Figure 7.17. The upper four points in the graph correspond to the four T2 values of Figure 7.14 that are greater than or equal to a value of 8. Although the linear trend for these four points is not consistent with the linear trend of the other 496 points, the deviation is not severe enough to disqualify the use of the T2 statistic from detecting signals in the Phase II operation.
7.7. Data Example
137
Beta Quantiles Figure 7.17: Q-Q plot of T values for the boiler data.
Table 7.2: Summary statistics for boiler HDS. Mean Minimum Maximum Std Dev
Fuel 374.02 186.46 524.70 98.37
Steam 293.32 138.87 412.53 77.96
Summary statistics for the boiler HDS are presented in Table 7.2. The minimum and maximum values of the variables give a good indication of the operational ranges on the two variables. For example, fuel usage in the HDS ranges from a low of 186.46 units to a high of 524.70 units. With statistical understanding of the boiler system through examination of the HDS, we are ready to move to signal interpretation. The control region is presented in Figure 7.18 in the variable space of fuel usage and steam flow. Also included are three signaling points, designated as points 1, 2, and 3. Examination of the signals in this graphical setting provides insight as to how the terms of the MYT decomposition identify the source of the signal and how the signals are to be interpreted. For example, note the (Euclidean) distance point 1 is from the body of the data (HDS). Also, note the "closeness" of points 2 and 3 to the control region, especially point 2. This leads one to think that the signal for point 1 is more severe than the signals for the other two points. However, this is not the case. Observe the T2 values presented in Table 7.3 for the three signaling points. Point 3 clearly has the largest T2 value, while point 2 is the smallest of the three signaling T2 values. To understand why this occurs, we need to examine the values of the MYT decomposition terms that are presented in Table 7.4. Since there are only two variables, the three signaling points can be plotted in either the (7\, T 2 .i) space or the (T2, 7Y2) space. A representation in the (Ti, T 2 .i) space is presented in Figure 7.19. Geometrically, the circle in Figure 7.19
138
Chapter 7. Interpretation of T2 Signals for Two Variables
Figure 7.18: Control region in variable space for boiler data. Table 7.3: T2 values for three signaling points. Obs. No. 1 2 3
T2 Value 13.25 10.97 405.76
* Significant at a = 0.01, UCL = 9.33.
Table 7.4: MYT decomposition terms of three signaling points Obs. No. 1 2 3
1? 10.70* 0.07 4.18
2 T J 2 11.24* 0.01 0.92
2 T J 1.2
2.01 10.96* 404.84*
2 T J 2.1
2.54 10.90* 401.58*
* Denotes significance at the 0.05 level.
represents a rotation of the elliptical control region given in Figure 7.18. In the transformed space, the T2 statistic is represented by the square of the length of the arrows designated in the plot. The UCL of 9.33 defines the square of the radius of the circular control region. The coordinates of point 1 in Figure 7.19 are (3.27, 1.57). The sum of squares of these values equals the T2 value of point 1, i.e.,
The coordinates of point 2 are (1.3, 5.05), and those of point 3 are (2.04, 20.01). Scalewise, point 3 would be located off the graph of Figure 7.19. However, this
7.7. Data Example
139
Figure 7.19: Control region in (Ti, T2.i) space for boiler data. Table 7.5: Three out-of-control points. Obs. No. 1 2 3
Fuel 52.21 400.00 172.83
Steam 31.95 300.00 218.41
representation is sufficient to demonstrate that point 1 is much closer to the control region than the other two points. Signals on both unconditional terms for point 1 indicate the observed values of fuel usage and steam to be beyond the operational range of the data as specified by the HDS. This is indeed the case, as the observation was taken when the boiler was in an idling mode, so that there was no demand for steam production. In this mode, a minimum amount of fuel is still supplied to the boiler, as it is less expensive to let the unit idle than to start a cold boiler. The observed fuel and steam values for all three points are presented in Table 7.5. Comparison of Table 7.5 with the summary statistics of Table 7.2 shows point 1 to be well beyond the operational range of the variables as specified by the HDS. The T2 signal on point 2 is due to an incorrect relationship between the two variables. Both the T|x and 7\22 terms produced signals. This is because the fuel value (/) is not where it should be relative to the value of steam (s). To see this, consider the regression lines of fuel on steam (i.e., / = 4.43 + 1.26*s), and of steam on fuel (i.e., s = —2.65 + 0.79*/) as derived from the HDS. The predicted value of fuel for the given value of steam for point 2 is
The corresponding observed fuel value of 400.00 is too large for this value of steam. Likewise, the difference between the actual steam value and the predicted steam
140
Chapter 7. Interpretation of T2 Signals for Two Variables
Observation Number Figure 7.20: Time-sequence graph of three signaling points.
value for this point is too large. The predicted steam value is s = -2.65 + 0.79(400.00) = 313.74. When compared to the actual value of 300.00, the residual of 13.74 steam units is too large to be attributed to random fluctuation. Point 3 also has T2 signals on both the conditional terms. This indicates that the linear relationship between the two variables is astray. The reason for this can best be seen by examining the time-sequence graph presented in Figure 7.20 for the three signaling points. These were derived from the observation plot of the HDS in Figure 7.16, where it was established that fuel must be above the corresponding value of steam. For point 3 in Figure 7.20, the relationship is reversed as the value of steam is above the corresponding fuel value. This counterrelationship produces large signaling values on the two conditional T2 terms.
7.8
Conditional Probability Functions (Optional)
A bivariate normal distribution is illustrated in Figure 7.21. This is the joint density function of the observation (x, y), and it is denoted by f ( x , y ) . It explains the behavior of the two variables as they jointly vary. For example, it can be used to determine the probability that x < a and, at the same time, that y < 6; i.e., P(x < o, y < b). The conditional density of x given ?/, denoted by f ( x \ y ) , is used for a different purpose. Its use is to describe the behavior of x when y is fixed at a particular value. For example, it is used to determine the probability that x < a, given that y = b] i.e., P(x < a\y = b). It can be observed geometrically by passing a plane
7.8. Conditional Probability Functions (Optional)
141
Figure 7.21: Example of a bivariate normal distribution.
Figure 7.22: Example of conditional densities of x given y. through the joint density at the fixed value of y, i.e., at y = b. This is illustrated in Figure 7.22 for various values of the constant b. For the MVN distribution with p = 2, the conditional density of x given y is
Close examination of (7.27) reveals the conditional density of x given y to be normal, with a conditional mean [ix\y and conditional variance a\ given by
Chapter 7. Interpretation of T2 Signals for Two Variables
142
Figure 7.23: Regression of x on y. and
Examination of (7.28) reveals that (j,x\y depends on the specified value of y. For example, the conditional mean of the distribution of x for y = b is given as
For various values of the constant b (i.e., for values of y), it can be proven that the line connecting the conditional means (as illustrated in Figure 7.23) is the regression line of re on y. This can also be seen in (7.28) by noting that the regression coefficient (3 (of x on y] is given by Thus, another form of the conditional mean (of the line connecting the means of the conditional densities) is given by
In contrast to the conditional mean, the conditional variance in (7.29) does not depend on the particular value of y. However, it does depend on the strength of the correlation, p, existing between the two variables.
7.9
Summary
In this chapter, we have discussed the essentials for using the MYT decomposition in the interpretation of signals for a bivariate process. We have shown that a signaling T2 value for a bivariate observation vector has two possible MYT decompositions,
7.10. Appendix: Principal Component Form of T2
143
or
Each of the decompositions consists of two independent components. The unconditional terms T2 can be used to check the tolerance of the individual variables, and the conditional terms T2j can be used to check the linear relationship between the two variables. Each component can be described by an appropriate F distribution. Using this methodology, the signal of a T2 statistic can be separated into two orthogonal components. A large value for an unconditional term implies that the designated variable is out of tolerance. A large value on a conditional term implies that a wrong linear relationship exists between the observations on the variables. Thus, this procedure provides a powerful tool for signal determination in terms of the two process variables. In the next chapter, we extend this methodology to the general case involving a p-variable process. In addition, a quick and efficient computing scheme is developed for locating the signaling terms.
7.10
Appendix: Principal Component Form of T2
The T2 statistic can be expressed as a function of the principal components of the estimated covariance matrix (e.g., see Jackson (1991)). The formulas are similar to those presented in section 6.8 for the principal components of the population correlation matrix. For example, an alternate form of the T2 statistic is given by
where \i > \2 > • • • > Xp are the eigenvalues of the estimated covariance matrix S and the z%, i = 1 , . . . ,p, are the corresponding principal components. A principal component is obtained by multiplying the vector quantity (X — X) by the transpose of the normalized eigenvector Ui of S corresponding to A$; i.e.,
Each Zi is a scalar quantity, and the T2 statistic is expressed in terms of these values. The representation in (A7.1) is derived from the fact that the estimated covariance matrix S is a positive definite symmetric matrix. Thus, its singular value decomposition is given as
where U is a p x p orthogonal matrix whose columns are the normalized eigenvectors Ui of 5, and A is a diagonal matrix whose elements are the corresponding eigenvalues
Chapter 7. Interpretation of T2 Signals for Two Variables
144
\i of S. In matrix notation, these matrices are given by
Note that Substituting this quantity into the T2 statistic of (A7.1), we have
where Z = U'(X - X] and Z' = (zi, z2,•.., zp). A Hotelling's T2 statistic for a single observation also can be written as
where R is the estimated correlation matrix and Y is the studentized observation vector of X, i.e.,
where r^ = corr(xi,Xj) and
The matrix R (obtained from S) is a positive definite symmetric matrix and can be represented in terms of its eigenvalues and eigenvectors. Using a transformation similar to (A7.2), the above T2 can be written as
where Wi, W<2,..., Wp are the principal components of the correlation matrix R and the 7^ are the eigenvalues of R. The principal component values are given by Wi = V{ (X — X}, where the Vi are the normalized eigenvectors of R. Equation (A7.4) is not to be confused with (A7.1). The first equation is written in terms of the eigenvalues and eigenvectors of the covariance matrix, and the second is in terms of the eigenvalues and eigenvectors of the estimated correlation matrix.
7.10. Appendix: Principal Component Form of Tz
145
Figure A7.1: Bivariate control region. These are two very different forms of the same Hotellirig's T 2 , as the mathematical transformations are not equivalent. Similarly, (A7.4) should not be confused with (6.13). The equation in (6.13) refers to a situation where the correlation matrix is known, while (A7.4) is for the case where the correlation matrix is estimated. The principal component representation of the T2 plays a number of roles in multivariate SPC. For example, it can be used to show that the control region is elliptical in shape. Consider a control region defined by a UCL. The observations contained in the HDS have T2 values less than the UCL; i.e., for each X1t,
Thus, by (A7.4),
The control region is defined by the equality
which is the equation of a hyperellipsoid in a p-dimensional space, provided the / yl are all positive. The fact that the estimated correlation matrix R is a positive definite matrix guarantees that all the 7^'s are positive. A geometrical representation of a T2 bivariate control region, when ^ and S are unknown, is given in Figure A7.1. The elliptical region is formed using the
146
Chapter 7. Interpretation of T2 Signals for Two Variables
algebraic expression of the T2 statistic and is given by
Substituting y\ and y^ for the standardized values of x\ and #2, we have
and
In the principal component space of the estimated correlation matrix, this reduces to
which gives the equation of the control ellipse. The length of the major axis of the ellipse in (A7.5) is given by 71, and the length of the minor axis is given by 72The axes of this space are the principal components, w\ and w^. The absence of a product term in this representation indicates the independence between wi and W2- This is a characteristic of principal components, since they are transformed to be independent. Assuming that the estimated correlation r is positive, it can be shown that 71 = (1 + r) and 72 = (1 — r). For negative correlations, the 7$ values are reversed. One can also show that the principal components can be expressed as
From these equations, one can obtain the principal components as functions of the original variables.
Chapter 8
Interpretation of T2 Signals for the General Case
8.1
Introduction
In this chapter, we extend the interpretation of signals from a T2 chart to the setting where there are more than two process variables. The MYT decomposition is the primary tool used in this effort, and we examine many interesting properties associated with it. For example, we show that the decomposition terms contain information on the residuals generated by all possible linear regressions of one variable on any subset of the other variables. In addition to being an excellent aid in locating the source of a signal in terms of individual variables or subsets of variables, this property has two other major functions. First, it can be used to increase the sensitivity of the T2 statistic in the area of small process shifts (see Chapter 9). Second, the property is very useful in the development of a control procedure for autocorrelated observations (see Chapter 10).
8.2
The MYT Decomposition
The general MYT decomposition procedure is outlined below. The T2 statistic for a p-dimensional observation vector X' = (x±, x % , . . . , xp) can be represented as
Suppose we partition the vector (X — X^ as
where X^p~1' = (xi, #2, • • • ,£p-i) represents the (p — l)-dimensional variable p ~^ represents the corresponding p x —I p and X^ vector excluding the pth variable 147
148
Chapter 8. Interpretation of T2 Signals for the General Case
elements of the mean vector. Suppose we similarly partition the matrix S so that
where Sxx 'ls the (p— 1) x (p— 1) covariance matrix for the first (p — 1) variables, Sp is the variance of xp, and sxx is a (p — l)-dimensional vector containing the covariances between xp and the remaining (p — 1) variables. The T2 statistic in (8.1) can be partitioned into two independent parts (see Rencher (1993)). These components are given by
The first term in (8.3),
uses the first (p — 1) variables and is itself a T2 statistic. The last term in (8.3) can be shown (see Mason, Tracy, and Young (1995)) to be the square of the pth component of the vector X adjusted by the estimates of the mean and standard deviation of the conditional distribution of xp given (xi, X 2 , . . . , xp~i). It is given as where and
is the (p — l)-dimensional vector estimate of the coefficients from the regression of xp on the (p — 1) variables xi, x % , . . . ,X P _I. It can be shown that the estimate of the conditional variance is given as
Since the first term of (8.3) is a T2 statistic, it too can be separated into two orthogonal parts: The first term, T2_2, is a T2 statistic on the first (p — 2) components of the X vector, and the second term, 7 1 p_ 1-1)2) ... ) p_2> '1S the square of xp^i adjusted by the estimates of the mean and standard deviation of the conditional distribution of £ p _i given (xi, x2, • . . ,z p _ 2 ). Continuing to iterate and partition in this fashion yields one of the many possible MYT decompositions of a T2 statistic. It is given by
8.3. Computing the Decomposition Terms
149
The TI term in (8.6) is the square of the univariate t statistic for the first variable of the vector X and is given as
Note this term is not a conditional term, as its value does not depend on a conditional distribution. In contrast, all other terms of the expansion in (8.6) are conditional terms, since they represent the value of a variable adjusted by the mean and standard deviation from the appropriate conditional distribution. We will represent these terms with the standard dot notation used in multivariate analysis (e.g., see Johnson and Wichern (1999)) to denote conditional distributions. Thus, T2^ k corresponds to the conditional T2 associated with the distribution of Xi adjusted for, or conditioned on, the variables Xj and Xk-
8.3
Computing the Decomposition Terms
There are many different ways of computing the terms of the MYT decomposition. A shortcut approach is discussed in this section. From (8.3), we know that the first (p — I) terms of (8.6) correspond to the T2 value of the subvector X', _^ = (x\, x 2 , . . . ,z p -i); i.e.,
Similarly, the first (p — 2) terms of this expansion correspond to the subvector (P-2) = ( z i , £ 2 , - - . , z P - 2 ) ; i-e.,
X
Continuing in this fashion, we can compute the T2 values for all subvectors of the original vector X. The last subvector, consisting of the first component X^ = (xi), is used to compute the unconditional T2 term given in (8.7); i.e.,
All the T2 values, T?^ X i ,X2 ,. . - , X pv) T?\J<1 ,X2 ,...,JUp— I x) , . . . , T f 2(J,i)' v are computed using^ the general formula A
where X^ represents the appropriate subvector, X^ is the corresponding subvector mean, and Sjj denotes the corresponding covariance submatrix obtained from the overall S matrix given in (8.2) by deleting all unused rows and columns. Thus,
150
Chapter 8. Interpretation of T2 Signals for the General Case
Figure 8.1: Ellipsoidal control region for three process variables, the terms of the MYT decomposition can be computed as follows:
To illustrate this method for computing the conditional and unconditional terms of a MYT decomposition, consider an industrial situation characterized by three process variables. The in-control HDS is represented by 23 observations, and the estimates of the covariance matrix and mean vector are given by
A three-dimensional plot of the data with a 95% (a = 0.05) control ellipsoid is presented in Figure 8.1. For graphing purposes, the data have been centered at the mean value. The T2 value for a new observation vector X' — (533,514,528) is computed using (8.1) and produces a T2 value of 79.994. For an a = 0.05 with 23 observations, the T2 critical value is 11.923. Since the observed T2 is larger than this value,
8.3. Computing the Decomposition Terms
151
Figure 8.2: Ellipsoidal control region with signaling point. the new observation produces a signal. A graphical illustration of the signaling point is presented in Figure 8.2. Recall from Chapter 7 that for p = 2, there were two separate MYT decompositions of the T2 statistic. Likewise, for p = 3, a number of decompositions exists. One possible MYT decomposition for the observation vector is given as
To compute this value, we begin by determining the value of the conditional term Tjl2- From the above discussion, we have
To obtain T? x -, we partition the original estimates of the mean vector and covariance structure to obtain the mean vector and covariance matrix of the subvector X^ = ( x i ^ x z ) . The corresponding partitions are given as
and T|x 2 is calculated as
Chapter 8. Interpretation of T2 Signals for the General Case
152
Similarly, the decomposition for T?
-, is given by
We obtain Tf.i by computing the T2 value of the subvector X^> = ( x i ) . This is the unconditional term T2 and is computed by
Thus, Tn-i is computed as
From this, we have
and the smallness of the first two terms, T2 and T2A, imply that the signal is contained in the third term, T2-^ 2Only one possible MYT decomposition was chosen above to illustrate a computing technique for the decomposition terms. Had we chosen another MYT decomposition, such as other terms of the decomposition would have had large values. With a signaling overall T2 value, we are guaranteed that at least one term of any particular decomposition will be large. We illustrate this important point in later sections.
8.4
Properties of the MYT Decomposition
Many properties are associated with the MYT decomposition. Consider a pdimensional vector defined as X' = (xix?, - . . , xp}. Interchange the first two components to form another vector (x%, #1, • • • , xp] so that the only difference between the two vectors is that the first two components have been permuted. The T2 value of the two vectors is the same; i.e.,
This occurs because T2 values cannot be changed by permuting the components of the observation vector.
8.4. Properties of the MYT Decomposition
153
This invariance property of permuting the T2 components guarantees that each ordering of an observation vector will produce the same overall T2 value. Since there are p\ = (p)(p — l)(p — 2) • • • (2)(1) permutations of the components of the vector ( £ i , £ 2 , . . . , £ p ), this implies we can partition a T2 value in pi different ways. To illustrate this result, suppose p = 3. There are 3! = (3)(2)(1) = 6 decompositions of the T2 value for an individual observation vector. These are listed below:
Each row of (8.10) corresponds to a different permutation of the components of the observation vector. For example, the first row corresponds to the vector written in its original form as (xi, x 2 , £3), whereas the last row represents (x3, £ 2 , xi). Note that all six possible permutations of the original vector components are included. The importance of this result is that it allows one to examine the T2 statistic from many different perspectives. The p terms in any particular decomposition are independent of one another, although the terms across the decompositions are not necessarily independent. With p terms in each partition and p\ partitions, there are p x p\ possible terms to evaluate in a total MYT decomposition of a signaling T2 statistic. Fortunately, all these terms are not unique to a particular partition, as certain terms occur more than once. For example, the T2 term occurs in the first and second decompositions listed in (8.10), and the Tji 3 term occurs in the second and fifth decompositions. In general, there are p x 2^p~1^ distinct terms among the possible decompositions. These unique terms are the ones that need to be examined for possible contribution to a T2 signal. When p is large, computing all these terms can be cumbersome. For example, when p = 10, there are over 5,000 unique terms in the MYT decomposition. To alleviate this problem, several computational shortcuts have been established and are discussed in detail in later sections of this chapter. Consider the MYT decomposition given in (8.6) and suppose T2 dominates the overall value of the T2 statistic. This indicates that the observation on the variable x\ is contributing to the signal. However, to determine if the remaining variables in this observation contribute to the signal, we must examine the T2 value associated with the subvector (x 2 , £3, • . - , £ p ), which excludes the x\ component. Small values of the T2 statistic for this subvector imply that no signal is present. They also indicate that one need not examine any term of the total decomposition involving these (p — 2) variables. The fact that a T2 statistic can be computed for any subvector of the overall observation vector has numerous applications. For example, consider a situation (not uncommon in an industrial setting) where observations on process variables are
154
Chapter 8. Interpretation of T2 Signals for the General Case
Table 8.1: List of possible regressions and conditional T2 terms when p = 3. Regression of x\ on X2 xi on X3 x\ on X2, X3 X2 on xi X2 on xs X2 On Xi, X3
Conditional T2 2 T J 1.2 2 J T 1.3 2 T J 1.2,3 2 T J 2.1
2 T J
2.3
2 r2.1,3 J
£3 on xi
2 T J 3.1
X3 On X2
2 T J 3.2 2 T J 3.1,2
xs on xi, x-2
more frequently available than observations on lab variables. A T2 statistic can be computed for the subvector of the process variables whenever observations become available, and an overall T2 statistic can be determined when both the process and lab variables are available. Another application of this important result is in the area of missing data. For example, we need not shut down a control procedure because of a faulty sensor. Instead, we can drop the variable associated with the sensor from the control procedure and run the T2 on the remaining variables until the sensor is replaced. To demonstrate the use of the T2 statistic in signal location, reconsider the three-variable example discussed in the last section. We computed the T2 value of the signaling observation vector, X' = (533,514,528), to be
Furthermore, we computed the T2 value of the subvector X'^ = (533, 514) to be
The smallness of this latter T2 value suggests that there are no problems with the observations on variables x\ and x 2 . Thus, all T2 terms, both conditional and unconditional, involving only x\ and x% will have small values. Our calculations confirm this result as
From this type of analysis one can conclude that the signal is caused by the observed value on #3. Another important property of the T2 statistic is the fact that the p(2p~1 — 1) unique conditional terms of a MYT decomposition contain the residuals from all possible linear regressions of each variable on all subsets of the other variables. For example, for p = 3, a list of the nine (i.e., 3(23~1 — 1)) linear regressions of each variable on all possible subgroups of the other variables is presented in Table 8.1 along with the corresponding conditional T2 terms. It will be shown in Chapter
8.5. Locating Signaling Variables
155
9 that this property of the T2 statistic provides a procedure for increasing the sensitivity of the T2 statistic to process shifts.
8.5
Locating Signaling Variables
In this section we seek to relate a T2 signal and its interpretation to the components of the MYT decomposition. Consider a signaling observation vector X' = (xi, X27 • • • i xp) such that One method for locating the variables contributing to the signal is to develop a forward-iterative scheme. This is accomplished by finding the subset of variables that do not contribute to the signal. Recall from (8.3) and (8.5) that a T2 statistic can be constructed on any subset of the variables, xi, #2, • • • , xp. Construct the T2 statistic for each individual variable, Xj, j = 1 , 2 , . . . , p, so that
where Xj and s2- are the corresponding mean and variance estimates as determined from the HDS. Compare these individual T2 values to their UCL, where
is computed for an appropriate a level and for a value of p = 1. Exclude from the original set of variables all Xj for which
since observations on this subset of variables are definitely contributing to the signal. From the set of variables not contributing to the signal, compute the T2 statistic for all possible pairs of variables. For example, for all (x^, Xj) with i ^ j, compute
and compare these values to the upper control limit,
Exclude from this group all pairs of variable for which
156
Chapter 8. Interpretation of T2 Signals for the General Case
The excluded pairs of variables, in addition to the excluded single variables, comprise the group of variables contributing to the overall signal. Continue to iterate in this fashion so as to exclude from the remaining group all variables of signaling groups of three variables, four variables, etc. The procedure produces a set of variables that contribute to the signal. To illustrate the above methodology, recall the T2 value of 79.9441 for the threedimensional observation vector X' — (533, 514, 528) from the previous example. The unconditional T2 values for the individual observations are given by
The UCL is computed as
From these computations we observe that T2 and T| are in control. However,
indicates that £3 is part of the signal contained in the observation vector. Next, we separate the original observation into two groups, (x\, x%) and (rr 3 ). We compute and compare it to
We conclude no signal to be present in the (xi, x^] component of the observation vector. Hence, the reason for the signal lies with the observation on the third variable, namely, x$ = 528. One problem with this method is that it provides little information on how the isolated signaling vector component(s) contributes to the signal. For example, in the above set of data, it does not indicate how the observed value of 528 on x% contributes to the overall signal. Nevertheless, it can be used, as indicated, to locate the components of the observation vector that do contribute to the signal. It should be noted also that a backward-elimination scheme, similar to the forward scheme, could be developed to locate the same signaling vector components. Another method of locating the vector components contributing to a signal is to examine the individual terms of the MYT decomposition of a signaling observation vector and to determine which are large in value. This is accomplished by comparing each term to its corresponding critical value. Recall from Chapter 7 that the distribution governing the components of the MYT decomposition for the situation where there are no signals is an F distribution. For the case of p variables, these are given by
8.6. Interpretation of a Signal on a T2 Component
157
Table 8.2: Unique T2 terms of total decomposition. 7\2 T| Tf T£2
= 1.3934 = 0.0641 = 11.6578* = 1.3294
T^g = 28.2305* T| ! = 0.0001 T|3 = 9.5584* Tf^ =38.4949*
Tf 2 = 21.1522* T^g = 58.7278* Tf ^ 3 = 40.0558* T 3.i,2 = 78.5506*
* Denotes significance at the 0.01 level.
for unconditional terms, and by
for conditional terms, where k equals the number of conditioned variables. For k = 0, the distribution in (8.13) reduces to the distribution in (8.12). Using these distributions, critical values (CVs), for a specified a level and an HDS sample of size n for both conditional and unconditional terms are obtained as follows:
We can compare each individual term of the decomposition to its critical value and make the appropriate decision. To illustrate the above discussion, recall the T2 value of 79.9441 for the observation vector X' = (533, 514, 528) taken from the example described in section 8.3. Table 8.2 contains the 12 unique terms and their values for a total decomposition of this T2 value. A large component is determined by comparing the value of each term to the appropriate critical value. The T2 values with asterisks designate those terms that contribute to the overall T2 signal, e.g., T32, T^, T|3, T|_l5 T|2, T?2_3, T^ i 3 , and T^12- All such terms contain the observation on x3. This was the same variable designated by the exact method for detecting signaling variables. Thus, one could conclude that a problem must exist in this variable. However, a strong argument also could be made that the problem is due to the other two variables, since four of the signaling terms contain x\ and four terms contain x^. To address this issue, more understanding of what produces a signal in terms of the decomposition is needed.
8.6
Interpretation of a Signal on a T2 Component
Consider one of the p possible unconditional terms resulting from the decomposition of the T2 statistic associated with a signaling observation. As stated earlier, the term
158
Chapter 8. Interpretation of T2 Signals for the General Case
Figure 8.3: Ellipsoid within a box.
j = 1, 2 , . . . ,p, is the square of a univariate t statistic for the observed value of the j'th variable of an observation vector X. For control to be maintained, this component must be less than its critical value, i.e.,
Since following interval:
we can reexpress this condition as Tj being in th
or as
where £( a /2,n-i) is the appropriate value from a t distribution with n — 1 degrees of freedom. This is equivalent to using a univariate Shewhart control chart for the jfth variable. If the control limits in (8.16) are constructed for each of the p variables and plotted in a p-dimensional space, we would obtain a hyperrectangular "box." This box is the equivalent of a Shewhart control procedure on all individual variables. However, the true control region, based on the T2 statistic, is a hyperellipsoid located within the box. In most situations, the ellipsoid will not fit inside the box. We illustrate this situation for p = 3 in Figure 8.3. The rectangular box represents the control limits on the individual box as computed by (8.16), while the
8.6. Interpretation of a Signal on a T2 Component
159
ellipsoid represents the control region for the overall T2 statistic. Various signaling points are also included for discussion below. If an observation vector plots outside the box, the signaling univariate Tj2 values identify the out-of-control variables, since the observation on the particular variable is varying beyond what is allowed (determined) by the HDS. This is illustrated in Figure 8.3 by point A. Thus, when an unconditional term produces a signal, the implication is that the observation on the particular term is outside its