2151_C000.fm Page i Thursday, October 26, 2006 1:39 PM
Stream of Variation Modeling and Analysis for Multistage Manufa...
121 downloads
1797 Views
12MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
2151_C000.fm Page i Thursday, October 26, 2006 1:39 PM
Stream of Variation Modeling and Analysis for Multistage Manufacturing Processes
2151_C000.fm Page ii Thursday, October 26, 2006 1:39 PM
2151_C000.fm Page iii Thursday, October 26, 2006 1:39 PM
Stream of Variation Modeling and Analysis for Multistage Manufacturing Processes Jianjun Shi University of Michigan, Michigan
Boca Raton London New York
CRC Press is an imprint of the Taylor & Francis Group, an informa business
2151_C000.fm Page iv Thursday, October 26, 2006 1:39 PM
CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2007 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed in the United States of America on acid-free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number-10: 0-8493-2151-4 (Hardcover) International Standard Book Number-13: 978-0-8493-2151-1 (Hardcover) This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www. copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC) 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Shi, Jianjun, 1963Stream of variation modeling and analysis for multistage manufacturing processes / Jianjun Shi. p. cm. Includes bibliographical references and index. ISBN 0-8493-2151-4 1. Manufacturing processes--Statistical methods. I. Title. TS183.S524 2006 658.5--dc22 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com
2006049164
2151_C000.fm Page v Thursday, October 26, 2006 1:39 PM
Preface The aim of this book is to summarize major achievements in stream of variation (SoV) methodology research and implementation in various manufacturing processes. The SoV methodology focuses on developing and implementing a unified, systematic, and generic methodology for variation management and reduction of multistage manufacturing processes (MMPs). An MMP is a complex manufacturing system with common characteristics as: (1) multiple operations/stations are involved to produce a product, (2) the product quality can be quantitatively characterized by a set of features or attributes, and (3) the product quality deviations/variations are contributed by the errors generated at the current station, as well as the accumulated errors transmitted from previous stations. The major challenges of variation reduction in an MMP are to model, analyze, control, and reduce the process-induced variation on final products. In an MMP, the output-workpiece deviation of one station is the input-workpiece deviation of the next station. Each station adds design inherent process variation due to tooling tolerance when no fault occurs, and special assignable process variations due to tooling errors when a fault occurs, to the part variation. The final product variation is an accumulation of variations from all stations. Due to the complexity of an MMP, the variation propagation depends on both product design and process design. The variation stack-up is normally not simply additive, and could be nonlinear. This book addresses those fundamental issues by developing the unified methodology, which includes: (1) model variation propagation in an MMP using a state space model, (2) expanding the concepts of system and control theory for variation management through design synthesis and optimization to achieve an affordable minimum variability at the design stage, (3) developing statistical methods driven by engineering models for root cause identification and quick failure recovery in the manufacturing stage; and (4) integrating process tool degradation, reliability model, and product quality information for prognostics and defect prevention to maintain the minimum variability throughout the manufacturing lifetime. The book can be used as a major reference or text book for researchers, engineers, and students who are interested in manufacturing design and analysis, manufacturing process control, advanced quality control, industrial statistics, or general areas of manufacturing process modeling, monitoring, diagnostics, and control.
OUTLINE OF THE BOOK The book is organized into 5 parts and 18 chapters. A brief summary of those 5 sections is summarized below:
2151_C000.fm Page vi Thursday, October 26, 2006 1:39 PM
•
•
•
•
•
Part I (Chapter 2 to Chapter 5) provides a review of some bases of matrix theory and multivariate statistics, which will be used in later sections in the SoV study. Topics include basics of matrix theory (Chapter 2), multivariate statistics (Chapter 3), statistical inference (Chapter 4), and principal component analysis and factor analysis (Chapter 5). Part II (Chapter 6 to Chapter 8) discusses variation propagation modeling issues, including SoV modeling based on the first principles (engineering knowledge) for assembly processes (Chapter 6), machining processes (Chapter 7), and SoV modeling based on observational data (Chapter 8). Part III addresses issues on variation source diagnosis. The section starts with diagnosability study (Chapter 9), then follows with two diagnostic methods as pattern matching methods (Chapter 10) and estimation based methods (Chapter 11). Part IV investigates issues related to design for variation reduction, which includes optimal sensor placement and distribution (Chapter 12), design evaluation and process capability analysis (Chapter 13), optimal fixture layout (Chapter 14), and process-oriented tolerance synthesis (Chapter 15). The last part introduces additional SoV related topics, including QR chain modeling and analysis (Chapter 16), quality-ensured maintenance (Chapter 17), and a review of other SoV research achievements (Chapter 18).
In addition to serving as a major reference book for researchers and engineers, this book can also be used as a text book for senior undergraduate students or graduate students in advanced quality control courses or manufacturing design courses. The book can be organized into three different tracks for a one-semester course according to students’ backgrounds and interests: 1. Data-driven track (fits industrial engineering–style quality and manufacturing for senior undergraduate and junior graduate students): include Chapter 2 to Chapter 5 and Chapter 8 to Chapter 11. 2. Physics-model-based track (fits mechanical engineering–style manufacturing process control for senior undergraduate and junior graduate students): includes Chapter 2, Chapter 3, Chapter 5, Chapter 6 or Chapter 7, Chapter 10, Chapter 14, and Chapter 15. 3. Advanced track (senior graduate and Ph.D. students): includes Chapter 6 (or Chapter 7) and any chapter from Chapter 8 to Chapter 17.
2151_C000.fm Page vii Thursday, October 26, 2006 1:39 PM
Acknowledgments I have been fortunate to have had three advisors in my graduate study: Professor Jun Zhang (M.S. degree advisor in the Automatic Control Department, Beijing Institute of Technology), who introduced me to the control and system theories; Prof. Zhifang Zhang (Ph.D. advisor in electrical engineering at Beijing Institute of Technology), who trained me with system identification and adaptive control, and Prof. S. M. Wu (Ph.D. advisor in mechanical engineering at the University of Michigan), who provided me with the opportunities to study at the University of Michigan and led me into the area of manufacturing science and engineering. Many of the research ideas and achievements described in this book originated from their guidance and training. I am indebted to many of my former and current students with whom I have collaborated, or am still collaborating, in regard to the technical ideas described in this book. The individuals include, but are not limited to, D. Ceglarek, D. Khorzad, B. W. Shiu, D. W. Apley, J. Jin, Y. Ding, S. Zhou, Y. Chen, Q. Huang, F. Tsung, Q. Rong, P. Chaipradubkiat, J. Li, J. Liu, L. E. Izquierdo, J. Zhong, R. Jin, and H. Zheng. I am also indebted to other friends and collaborators, including S. J. Hu, J. Ni, Y. Koren, G. Ulsoy, C. F. J. Wu, V. Nair, D. Djurdjanovic, J. Camelio, C. W. Wampler, and Z. Kong. The experiences of working with them are invaluable, and I have learned a great deal from them, which benefitted me both personally and professionally. My appreciation is also due to the many research associates and students as well as managers and engineers in companies who have directly contributed to the technology transfer and the implementation of the SoV methodologies in industrial practices. Special acknowledgement goes to three individuals who directly helped in contributing or writing some of the chapters: Y. Ding for Chapter 6 and Chapter 11 to Chapter 14; S. Zhou for Chapter 7, Chapter 9, and Chapter 10; and Y. Chen for Chapter 15 to Chapter 17. They also reviewed the manuscript draft of the book and provided many invaluable comments for improvement. Without their strong support and contribution, this manuscript would not have been completed. I am also grateful for the many students, collaborators, more senior colleagues, and others who provided valuable critiques and feedbacks on the manuscript. I have been fortunate to have research support from many sources, including the National Science Foundation (NSF), NSF Engineering Research Center for Reconfigurable Manufacturing Systems, NIST-Advanced Technology Program, General Motors, Ford, DaimlerChrysler, Auto Body Consortium, and many other industrial companies. Their generous funding support made many of these SoV research initiatives feasible.
2151_C000.fm Page viii Thursday, October 26, 2006 1:39 PM
Finally, a very special note of appreciation is extended to my wife, Liping Luo, and my two daughters, Helen and Katherine, who have provided continuous support in the past years.
2151_C000.fm Page ix Thursday, October 26, 2006 1:39 PM
Author Jianjun Shi is a professor of the Department of Industrial and Operations Engineering and a professor of Department of Mechanical Engineering at the University of Michigan. He received his bachelor of science and master of science in electrical engineering at the Beijing Institute of Technology in 1984 and 1987, respectively, and his Ph.D. in mechanical engineering at the University of Michigan in 1992. His research interests focus on the fusion of advanced statistical and domain knowledge to develop methodologies for modeling, monitoring, diagnosis, and control for complex systems in a data-rich environment. He is one of the early pioneers in the field. He has guided 19 Ph.D. graduates and published more than 100 papers. He has also been closely working with industrial companies. He has led various research projects funded by National Science Foundation, NIST Advanced Technology Program, General Motors, DaimlerChrysler, Ford, Lockheed-Martin, Honeywell, Pfizer, and various other industrial companies and funding agencies. The technologies developed in his research group have been implemented in various production systems with significant economic impacts. Professor Shi is the founding chairperson of the Quality, Statistics and Reliability (QSR) Subdivision, Institute for Operations Research and Management Science (INFORMS). He also serves as the director of the laboratory for In-Process Quality Improvement Research (IPQI) at the University of Michigan. He is currently serving as the departmental editor, IIE Transactions on Quality and Reliability, and associate editor for the International Journal of Flexible Manufacturing Systems. He is a member of ASME, ASQ, IIE, SME, and ASA. More information about Dr. Shi can be found on his web site: http://www-personal.engin.umich.edu/~shihang/index.html
2151_C000.fm Page x Thursday, October 26, 2006 1:39 PM
2151_C000.fm Page xi Thursday, October 26, 2006 1:39 PM
Table of Contents Chapter 1
What Is Stream of Variation for Multistage Manufacturing Processes?.............................................................................................1
1.1
History and Current Status of SoV .................................................................1 1.1.1 SoV Initiation and Impact in Industry.................................................1 1.1.2 The History of SoV Methodology Development ................................3 1.2 Overview of SoV Methodology.......................................................................5 1.3 Relationship of SoV Methodologies with Other Existing Methods ...............9 References................................................................................................................11
PART I Basis of Matrix Theory and Multivariate Statistics Chapter 2
Basics of Matrix Theory ....................................................................15
2.1 2.2
Introduction ....................................................................................................15 Definitions of Vector, Matrix, and Operations ..............................................15 2.2.1 Definitions ..........................................................................................15 2.2.2 Partitioned Matrices ...........................................................................18 2.3 Quadratic Forms.............................................................................................19 2.4 Vector Space and Geometrical Interpretations ..............................................20 2.5 Eigenvalues and Eigenvectors of a Matrix ....................................................22 2.6 Vector and Matrix Differentiation, Maximization, and Operators................24 2.6.1 Differentiation with Vectors...............................................................24 2.6.2 Matrix Inequalities and Maximization ..............................................26 2.6.3 Vec Operator, Kronecker Product, and Hadamard Product ..............27 2.6.3.1 Vec Operator and Kronecker Product ................................27 2.6.3.2 Hadamard Product ..............................................................30 2.7 Exercises.........................................................................................................30 References................................................................................................................32 Chapter 3 3.1 3.2
Basics of Multivariate Statistical Analysis ........................................33
Introduction ....................................................................................................33 Multivariate Distribution and Properties .......................................................33 3.2.1 Random Vectors, Cumulative Distribution Functions (CDF), and Probability Density Functions (PDF) .........................................34 3.2.2 Marginal and Conditional Distributions ............................................35 3.2.3 Population Moments ..........................................................................37 3.2.4 Correlation Coefficients .....................................................................40
2151_C000.fm Page xii Thursday, October 26, 2006 1:39 PM
3.3
Multivariate Normal Distribution and Quadratic Forms ...............................40 3.3.1 Multivariate Normal Distribution and Its Properties.........................43 3.3.2 Quadratic Forms.................................................................................44 3.3.3 Noncentral χ2 and F Distributions.....................................................45 3.4 Sampling Theory............................................................................................45 3.4.1 Sample Geometry...............................................................................49 3.4.2 Random Samples and the Expected Values of the Sample Mean and Covariance Matrix.......................................................................50 3.4.3 Some Important Results of Sampling from Multivariate Normal Distributions .......................................................................................50 3.5 The Wishart Distribution and Some Properties.............................................51 3.6 Exercises.........................................................................................................53 References................................................................................................................55 Chapter 4
Statistical Inferences on Mean Vectors and Linear Models..............57
4.1
Statistical Inferences on Mean Vectors..........................................................57 4.1.1 Hotelling’s T 2 Test .............................................................................57 4.1.2 Confidence Regions and Simultaneous Confidence Intervals...........59 4.1.2.1 Confidence Regions ............................................................59 4.1.2.2 Simultaneous Confidence Intervals ....................................61 4.1.3 Inference on a Population Mean Vector with Large Sample Size .....63 4.1.4 Multivariate Quality Control Charts ..................................................64 4.1.4.1 Control Charts for One Multivariate Sample.....................64 4.1.4.2 Control Charts Based on Subgroup Means........................66 4.2 Multiple Linear Regression ...........................................................................68 4.2.1 Model Description..............................................................................69 4.2.2 Least-Squares Estimates ....................................................................70 4.2.3 Inferences about the Regression Model ............................................73 4.2.4 Model Checking: Normality Checking and Outlier Detection .........75 4.2.5 Inferences from the Estimated Regression Model ............................76 4.2.6 Multivariate Linear Regression..........................................................78 4.2.6.1 Multivariate Linear Regression Model...............................78 4.2.6.2 Least-Squares Estimation (LSE) and Maximum Likelihood Estimation (MLE) of Parameters.....................79 4.2.6.3 Inferences for Multivariate Regression Model under Normality Assumption........................................................82 4.2.6.4 Predictions from Multivariate Regression..........................83 4.2.6.5 Selection Method of Independent Variables ......................85 4.3 Exercises.........................................................................................................86 References................................................................................................................89 Chapter 5 5.1
Principal Component Analysis and Factory Analysis .......................91
Principal Component Analysis.......................................................................91 5.1.1 Mathematical Model of PCA.............................................................91 5.1.2 Geometrical Interpretation .................................................................95
2151_C000.fm Page xiii Thursday, October 26, 2006 1:39 PM
5.1.3 Inferences on PCs ..............................................................................96 5.1.4 Applying PCs to Process Control......................................................97 5.2 Factor Analysis...............................................................................................99 5.2.1 Introduction ........................................................................................99 5.2.2 Comparison of Factor Analysis and PCA .........................................99 5.2.3 Orthogonal Factor Model.................................................................100 5.2.4 Statistical Interpretations of Factor Loadings and Communality ...102 5.2.4.1 Interpretation of Factor Loadings.....................................102 5.2.4.2 Interpretation of Communality .........................................102 5.2.5 Estimation of Loading Matrix .........................................................103 5.2.6 Factor Rotation.................................................................................105 5.2.7 Factor Score .....................................................................................109 5.2.8 General Procedures for Factor Analysis..........................................111 5.3 Exercises.......................................................................................................111 References..............................................................................................................113
PART II Variation Propagation Modeling in MMP Chapter 6
State Space Modeling for Assembly Processes...............................117
6.1 6.2
Introduction of Multistage Assembly Processes .........................................117 Variation Factors and Assumptions .............................................................119 6.2.1 Station-Level Variation Factors........................................................119 6.2.2 Across-Station Variation Factors .....................................................120 6.2.3 Summary and Assumptions for Modeling.......................................121 6.3 State Space Modeling ..................................................................................122 6.3.1 Representation of Part Position and Its Deviation State.................122 6.3.2 Some Preliminary Results................................................................123 6.3.3 State Space Representation..............................................................125 6.4 Model Validation ..........................................................................................131 6.5 Exercises.......................................................................................................133 Appendix 6.1: Determination of the Deviations of a Part (Lemma 6.1) .............137 Appendix 6.2: Effect of Fixture Deviation on Part Deviation (Lemma 6.2) .......138 References..............................................................................................................140 Chapter 7 7.1
7.2
State Space Modeling for Machining Processes .............................143
Introduction ..................................................................................................143 7.1.1 Introduction to Machining Processes and Dimensional Variation Sources .............................................................................................143 7.1.2 Modeling of Variation Propagation in Multistage Machining Processes ..........................................................................................145 7.1.3 Model Formulation ..........................................................................147 Derivation of Variation Propagation Model ................................................148 7.2.1 Basics of Kinematic Analysis of Machining Operations ................148 7.2.2 Representation of Workpiece Geometric Deviation ........................152
2151_C000.fm Page xiv Thursday, October 26, 2006 1:39 PM
7.2.3
Single-Stage Modeling of Dimensional Variation...........................153 7.2.3.1 Analysis of Datum-Induced Error ....................................154 7.2.3.2 Analysis of Fixture Errors ................................................158 7.2.3.3 Identify the Overall Dimensional Error by Combining Error Sources Together.....................................................159 7.2.3.4 Modeling Variation Propagation in Multistage Machining Processes ........................................................161 7.3 Model Validation ..........................................................................................162 7.3.1 Introduction to the Experimental Machining Process.....................162 7.3.2 Comparison between the Real Measurement and the Model Prediction .........................................................................................164 7.4 Summary ......................................................................................................166 7.5 Exercises.......................................................................................................166 7.6 Appendix: The System Matrices Used in Equation 7.27............................169 References..............................................................................................................173 Chapter 8
A Factor Analysis Method for Variability Modeling ......................175
8.1 8.2
Introduction ..................................................................................................175 A Factor Analysis Model for Process Variability........................................176 8.2.1 Model Structure................................................................................176 8.2.2 Interpretation of Variation Patterns..................................................177 8.3 Limitations of PCA and Factor Rotation ....................................................180 8.4 Estimating the Number of Faults.................................................................181 8.4.1 Likelihood Ratio Test.......................................................................181 8.4.2 AIC and MDL Information Criteria ................................................182 8.4.3 An Illustrative Example ...................................................................183 8.5 Unique Identification of Multiple Faults .....................................................187 8.5.1 Estimating the Fault Geometry Vectors...........................................187 8.5.2 Identifying Subgroups......................................................................189 8.5.3 Fault Interpretation and Illustrative Example..................................190 8.6 Statistical Properties.....................................................................................193 8.7 Summary ......................................................................................................195 8.8 Exercises.......................................................................................................196 8.9 Appendix 8.1: Discussion of the Distributions of Test Statistics ...............196 References..............................................................................................................197
PART III Variation Source Diagnosis Chapter 9 9.1 9.2
Diagnosability Analysis for Variation Source Identification...........201
Motivation and Formulation of Diagnosability Study ................................201 Definitions of Diagnosability.......................................................................203
2151_C000.fm Page xv Thursday, October 26, 2006 1:39 PM
9.3 9.4 9.5
Criterion of Fault Diagnosability.................................................................205 Minimal Diagnosable Class .........................................................................207 Gauging System Evaluation Based on Minimal Diagnosable Class ..........210 9.5.1 Information Quantity........................................................................210 9.5.2 Information Quality..........................................................................211 9.5.3 System Flexibility ............................................................................211 9.6 Case Study....................................................................................................212 9.6.1 Case Study of A Multistage Assembly Process ..............................212 9.6.2 Case Study of A Multistage Machining Process.............................219 9.7 Summary ......................................................................................................221 9.8 Exercises.......................................................................................................224 Reference ...............................................................................................................226 Chapter 10 Diagnosis through Variation Pattern Matching ...............................229 10.1 Introduction to Variation Patterns ................................................................229 10.2 Links between the Fault-Quality Model and Variation Patterns.................231 10.3 Procedure of Pattern Matching for Variation Source Identification ...........233 10.3.1 Disturbance due to Unstructured Noises .........................................235 10.3.2 Disturbance due to Sampling Uncertainty ......................................236 10.3.3 A Robust Pattern Matching Procedure ............................................237 10.4 Case Study....................................................................................................240 10.4.1 A Machining Process and Its Variation Propagation Model...........240 10.4.2 Pattern Matching for Root Cause Identification in the Machining Process ...........................................................................241 10.5 Summary ......................................................................................................244 10.6 Exercises.......................................................................................................244 References..............................................................................................................248 Chapter 11 Estimation-Based Diagnosis ............................................................249 11.1 LS Estimators for Variance Components ....................................................249 11.1.1 Deviation LS Estimator....................................................................250 11.1.2 Variation LS Estimator.....................................................................251 11.1.3 Other Variation LS Estimators.........................................................252 11.2 Relationship among Variance Estimators ....................................................254 11.3 Comparison of Variance Estimators ............................................................262 11.3.1 Unbiasedness of Variance Estimators ..............................................262 11.3.2 Dispersion of Variance Estimators...................................................263 11.3.3 Comparison of Variance Estimators ................................................264 11.4 Chapter Summary ........................................................................................270 11.5 Exercises.......................................................................................................270 References..............................................................................................................272
2151_C000.fm Page xvi Thursday, October 26, 2006 1:39 PM
PART IV Design for Variation Reduction Chapter 12 Optimal Sensor Placement and Distribution ...................................275 12.1 Introduction ..................................................................................................275 12.2 Design Criteria for Sensor Placement .........................................................278 12.2.1 Diagnosability Index as Design Criterion .......................................278 12.2.2 Sensitivity Index As Design Criterion.............................................279 12.3 Single-Station Sensor Placement .................................................................282 12.3.1 Optimization Formulation................................................................282 12.3.2 Exchange Algorithms from Optimal Experimental Design ............283 12.3.3 Fast Exchange Algorithm with Sort-and-Cut ..................................284 12.3.4 Comparison among Alternative Algorithms ....................................286 12.3.5 Results of Optimal Sensor Layout on a Single Station ..................288 12.4 Multiple-Station Sensor Distribution...........................................................290 12.4.1 Optimization Formulation................................................................290 12.4.2 Variation Transmissibility Ratio ......................................................291 12.4.3 Detection Power on An Individual Station......................................293 12.4.4 Optimal Strategy of Sensor Distribution .........................................296 12.4.4.1 Strategy of Sensor Distribution ........................................297 12.4.5 Example............................................................................................298 12.5 Summary ......................................................................................................301 12.6 Exercises.......................................................................................................301 References..............................................................................................................302 Chapter 13 Design Evaluation and Process Capability Analysis.......................305 13.1 13.2 13.3 13.4
Introduction ..................................................................................................305 Sensitivity-Based Design Evaluation...........................................................307 Multivariate Process Capability Analysis ....................................................313 Examples ......................................................................................................315 13.4.1 Sensitivity-Based Design Evaluation...............................................315 13.4.2 Multivariate Process Capability Analysis ........................................319 13.5 Exercises.......................................................................................................321 13.6 Appendix 13.1: System Matrices of Configuration C1...............................322 13.7 Appendix 13.2: System Matrices of Configuration C2...............................326 13.8 Appendix 13.3: System Matrices of Configuration C3...............................327 13.9 Appendix 13.4: System Matrices of Configuration C4...............................330 References..............................................................................................................331 Chapter 14 Optimal Fixture Layout Design.......................................................333 14.1 Introduction ..................................................................................................333 14.2 Design Criteria for Variation Reduction......................................................335 14.3 Data-Mining-Aided Design Algorithm........................................................338
2151_C000.fm Page xvii Thursday, October 26, 2006 1:39 PM
14.3.1 Overview of the Data-Mining-Aided Design ..................................339 14.3.2 Candidate Design Space ..................................................................341 14.3.3 Uniform Coverage Selection............................................................342 14.3.4 Feature and Feature Function ..........................................................344 14.3.5 Clustering Method............................................................................346 14.3.6 Classification Method ......................................................................347 14.3.7 Selection of K and J ........................................................................349 14.3.8 An Overall Description of the Data-Mining-Aided Design............352 14.4 Example and Performance Comparison ......................................................353 14.5 Summary ......................................................................................................356 14.6 Exercises.......................................................................................................356 References..............................................................................................................358 Chapter 15 Process-Oriented Tolerance Synthesis .............................................361 15.1 Concept of Process-Oriented Tolerancing ...................................................361 15.2 Framework of Process-Oriented Tolerancing ..............................................362 15.2.1 Overview ..........................................................................................362 15.2.2 Variation Propagation Model ...........................................................363 15.2.3 Relationship between Tolerance and Variation................................364 15.2.4 Process Degradation Model .............................................................366 15.2.5 Cost Function ...................................................................................368 15.2.6 Optimization Formulation and Optimality ......................................368 15.3 Case Study for Process-Oriented Tolerancing.............................................368 15.3.1 Tolerance Allocation when Tooling Degradation Is Not Considered........................................................................................368 15.3.2 Tolerance Allocation Considering Tooling Degradation .................370 15.4 Integration of Process-Oriented Tolerance Synthesis and Maintenance Planning........................................................................................................372 15.4.1 Decision Variables of Tolerance and Maintenance Design.............372 15.4.2 Cost Components .............................................................................373 15.4.2.1 Tolerance Cost ..................................................................373 15.4.2.2 Maintenance Cost .............................................................373 15.4.2.3 Quality Loss Function ......................................................374 15.4.3 Formulation of Optimization Problems...........................................374 15.4.3.1 Optimization Formulation Using a Quality Loss Function (F1) ....................................................................374 15.4.3.2 Optimization Formulation with a Quality Constraint ......375 15.5 Integrated Tolerance and Maintenance Design in BIW Assembly Processes ......................................................................................................375 15.6 Optimizations and Optimality for Integrated Tolerance and Maintenance Design...........................................................................................................377 15.6.1 Optimality Analysis of Optimization Formulation F1 ....................377 15.6.2 Optimality Analysis of Optimization Formulation F2 ....................379 15.7 Case Study for Integrated Tolerance and Maintenance Design..................380
2151_C000.fm Page xviii Thursday, October 26, 2006 1:39 PM
15.7.1 Optimal Tolerance and Maintenance Design for Optimization Formulation F1.................................................................................380 15.7.2 Optimal Tolerance and Maintenance Design for Optimization Formulation F2.................................................................................381 15.7.3 Cost Comparison under Different Design Schemes........................382 15.8 Exercises.......................................................................................................384 References..............................................................................................................385
PART V Quality and Reliability Integration and Advanced Topics Chapter 16 Quality and Reliability Chain Modeling and Analysis ...................389 16.1 Introduction ..................................................................................................389 16.1.1 Example 1: Machining Processes ....................................................391 16.1.2 Example 2: Transfer or Progressive Die-Stamping Processes ........392 16.2 QR-Chain Modeling.....................................................................................393 16.2.1 Relationship between Component Performance and Product Quality ..............................................................................................394 16.2.2 System Component Degradation .....................................................394 16.2.3 Product Quality Assessment and System Failure due to Nonconforming Products .................................................................395 16.2.4 Component Catastrophic Failure and Its Induced System Catastrophic Failure .........................................................................396 16.3 System Reliability Evaluation......................................................................396 16.3.1 Challenges in System Reliability Evaluation ..................................397 16.3.2 System Reliability Evaluation of MMPs.........................................397 16.3.3 Self-Improvement of Product Quality and the Upper Bound of System Reliability ............................................................................400 16.4 Implementation of QR-Chain Modeling and Analysis in Body-in-White Assembly Processes .....................................................................................401 16.4.1 QR-Chain in Multistation BIW Assembly Processes .....................402 16.4.2 QR-Chain Model of a BIW Assembly Process...............................402 16.4.2.1 Locating-Pin Degradation Model .....................................403 16.4.2.2 Relationship between Process Variables and Deviations of Quality Characteristics .................................................404 16.4.2.3 Product Quality Assessment and Pin Catastrophic Failure ...............................................................................407 16.4.2.4 System Reliability Evaluation ..........................................408 16.4.3 Case Study........................................................................................408 16.5 Exercises.......................................................................................................410 16.6 Appendix 16.1: Derivation of Equation 16.4 ..............................................410 16.7 Appendix 16.2: Proof of Result 16.1 ..........................................................412
2151_C000.fm Page xix Thursday, October 26, 2006 1:39 PM
16.8 Appendix 16.3: Derivation of Equation 16.10...........................................413 16.9 Appendix 16.4: Proof of Lemma 16.1 .......................................................414 16.10 Appendix 16.5: Proof of Result 16.4 .........................................................415 16.11 Appendix 16.6: Distribution of Random Variable ξ(tk) ............................415 References..............................................................................................................416 Chapter 17 Quality-Oriented Maintenance for Multiple Interactive System Components......................................................................................419 17.1 Quality-Oriented Maintenance Model .........................................................419 17.2 Multicomponent Maintenance Policies .......................................................422 17.2.1 Simple Block Resetting (Replacement)...........................................422 17.2.2 Modified Block Resetting ................................................................425 17.2.3 Age Resetting...................................................................................427 17.3 Further Discussion on Solutions of Optimization Problems ......................429 17.3.1 Optimal Solutions of SBR Policy....................................................430 17.3.2 Optimal Solutions of Modified Block Resetting Policy .................432 17.4 Case Study....................................................................................................432 17.5 Exercises.......................................................................................................436 17.6 Appendix 17.1: Proof of Lemma 17.7 ........................................................437 References..............................................................................................................438 Chapter 18 Additional Topics on Stream of Variation .......................................439 18.1 SoV Modeling for Multistation Assembly Process of Compliant Parts.....439 18.1.1 Introduction ......................................................................................439 18.1.2 SoV Modeling of Compliant Parts in an MMP ..............................440 18.1.2.1 Single Station Assembly Modeling ..................................441 18.1.2.2 Multistation Assembly Modeling .....................................442 18.1.3 Additional Comments ......................................................................444 18.2 SoV Modeling for Serial-Parallel Multistage Manufacturing Systems ......445 18.2.1 Introduction ......................................................................................445 18.2.2 SoV Modeling for Serial-Parallel MMPs........................................446 18.2.2.1 State Space Modeling for Multiple SoVs in an SP-MMP ...........................................................................446 18.2.2.2 Model Dimension Reduction............................................448 18.2.3 Additional Comments ......................................................................449 18.3 SoV-Based Quality-Ensured Setup Planning...............................................449 18.3.1 Introduction ......................................................................................449 18.3.2 Quality-Ensured Setup Planning Methodologies ............................450 18.3.2.1 Variation Propagation Modeling for Setup Planning.......450 18.3.2.2 Optimization Formulation ................................................451 18.3.3 Additional Comments ......................................................................452 18.4 Active Control for Variation Reduction of Multistage Manufacturing Processes ......................................................................................................453
2151_C000.fm Page xx Thursday, October 26, 2006 1:39 PM
18.4.1 Introduction ......................................................................................453 18.4.2 Active Control for Variation Reduction of MMP ...........................454 18.4.3 Additional Comments ......................................................................456 References..............................................................................................................457 Index ......................................................................................................................459 Related Titles .........................................................................................................469
2151_book.fm Page 1 Friday, October 20, 2006 5:04 PM
1
What Is Stream of Variation for Multistage Manufacturing Processes?
This chapter will present some background information about the Stream of Variation (SoV) methodology and its applications. We will particularly discuss the history of SoV initiation and development, the topics addressed in SoV, and the relationship of SoV with other existing methods.
1.1 HISTORY AND CURRENT STATUS OF SOV 1.1.1 SOV INITIATION
AND IMPACT IN INDUSTRY
SoV research was initiated by the need for variation reduction in automotive body assembly processes in the late 1980s. In those days, U.S. automobile producers were losing market share to Asian and European competitors. Quality, mainly reflected as the dimensional variability, of a body-in-white (BIW) was identified as one of the most important factors that impacted the competitiveness of U.S. automobile industries [1]. Variation reduction in automotive body assembly is a very challenging task [2]. A typical BIW has about 100 to 150 sheet metal parts, which are assembled at 80 to 120 assembly stations. A BIW assembly line normally has about 1500 to 2000 fixture locators and approximately 4000 welding spots. Figure 1.1 shows several layers of a BIW assembly process. If any deviations (or failures) occur at a locator, at a welding spot, or from an incoming part, a dimensional variation will be generated on the assembled part at this station, which will propagate to downstream assembly stations, and finally accumulate at the BIW. After assembly is finished, the door, hood, and deck-lid panels are installed into the openings of a BIW; and after the painting process, the windshield and backlight are installed into the appropriate openings. The dimensional variations accumulated at the BIW openings, compounded by variations associated with panels or other subassemblies, will significantly increase the production complexity, leading to more tooling failures and unexpected downtime and reducing both product quality and production throughput. In addition, the dimensional variation in auto body assembly also directly impacts gaps and flushes of panel fitting, which leads to the increase of wind noise and the risk of water leakage. Variation reduction demands effective process control techniques through monitoring and diagnosis based on the dimensional measurement data.
1
2151_book.fm Page 2 Friday, October 20, 2006 5:04 PM
2
Stream of Variation Modeling and Analysis for MMPs
FIGURE 1.1 A body-in-white (BIW) and major closure panels.
In the late 1980s, the inline optical coordination measurement machine (OCMM) was introduced to the automotive body assembly plant. The OCMM was installed at the end of an assembly line and used optical laser sensors to measure critical features of a BIW. An OCMM had about 100 laser sensors. Each sensor targeted a critical feature of a BIW, and provided dimension readings on the body coordinates. As a result, a large amount of measurement data were obtained from each auto body produced in an assembly line. The tremendous amount of inline quality data provided significant opportunities for more effective process control. However, it also challenged existing process control technologies. With hundreds of quality attributes being measured, some out-of-control conditions were invariably detected by using SPC control charts. In general, reacting to those out-of-control conditions was not easy using SPC techniques alone: they could not be quickly resolved because of the complexity of the process and time-consuming efforts in root cause identification. Therefore, the massive dimensional data generated from the OCMM was not fully utilized for variation reduction in body assembly. More effective models and data analysis methods were needed to address this challenging problem. Professor S.M. Wu at the University of Michigan took up the challenge and initiated the “Two Millimeter Program.” The main goal of this program was to increase the competitiveness of U.S. automakers by reducing the BIW dimensional variation. In the early 1990s, the dimensional variation (by Six Sigma) was about 2.0 mm for Japanese automakers, 2.5 mm for European automakers, and more than 3.0 mm (even close to 8.0 mm) for U.S. automakers. The 2 mm program aimed to reduce car body variation to the lowest level possible: the Six Sigma of all key dimensional quality attributes on a car body should be reduced to below 2 mm — a benchmark of the best quality of Japanese cars at that time. Under the leadership of Professor Wu and with strong support from GM, Chrysler, Ford, Auto Body Consortium, and NIST-Advanced Technology Programs (NIST-ATP), a group of researchers and engineers devoted much effort to studying various critical issues on variation reduction for automotive body assembly and enabled concurrent technology transfer and implementation in the automotive industry.
2151_book.fm Page 3 Friday, October 20, 2006 5:04 PM
What Is Stream of Variation for Multistage Manufacturing Processes?
3
The 2 mm program made a significant impact on automotive body assembly plants. In December 1992, one U.S. assembly plant located in Detroit, MI, achieved the “2 mm” variation level and marked the first success of the 2 mm program. In 1995, a “process navigator” software package was developed by Auto Body Consortium and the University of Michigan that provided an integrated environment for BIW dimensional data monitoring, analysis, and root cause diagnosis for the entire body assembly process. In 1998, distributed sensing strategy was deployed in a body shop for more effective data acquisition for process control. By 2000, the developed variation reduction methodologies had been successfully implemented in design improvement and the product launch of 32 new vehicle programs [1]. The implementation of those methodologies significantly improved the quality of assembly and also reduced the ramp-up time during the new production launch in automotive body assembly processes. The SoV methodology was expanded to the multistage machining processes in 1998 with the support of the NSF Engineering Research Center on Reconfigurable Manufacturing Systems (NSF ERC on RMS) and its industrial member companies. Unlike assembly, which is a part addition process, the machining is a materialremoving process. A set of new technologies was developed for the SoV methodology in machining. A test bed with four CNC machines, one CMM, and one reconfigurable inspection machine (RIM) was built in the NSF-ERC for RMS for the SoV research and validation in 1999. The developed SoV-based technologies for machining processes were validated and implemented in several powertrain plants from 2002 to 2006. The SoV-based sensitivity study was conducted to evaluate a new engine head machining line with 22 stations to identify critical machining stations that contributed more variability to the process in 2004. Gauging strategies were studied for quality feature selection and measurement of engine and transmission processes in 2005. Root cause diagnosis techniques were successfully implemented in process control in a diesel engine plant and adopted as the standard practice for the system in 2005. In 2003 the DCS Company won a grant from the NIST-Advanced Technology Program to develop commercial software based on the SoV methodology to support product and process modeling, design analysis, and diagnostic control of multistage manufacturing processes.
1.1.2 THE HISTORY
OF
SOV METHODOLOGY DEVELOPMENT
SoV methodology development attempts to develop a math-based technique for variation reduction in both design and manufacturing of a multistage manufacturing process. The methodology development can be divided into two phases: initial learning and technologies development (1990–1997) and a unified SoV methodology development based on state space models (1997–2006). Phase I research (1990–1997) targeted the development of advanced methods to analyze massive OCMM data to find root causes of variability. Some representative achievements in the early phase of technology development included (1) proposing the concept of using principal component analysis (PCA) to analyze the multivariate OCMM data [3], (2) developing a hierarchical representation or model and associated ad hoc diagnostic rules for failures in multistation assembly systems [4], (3) developing
2151_book.fm Page 4 Friday, October 20, 2006 5:04 PM
4
Stream of Variation Modeling and Analysis for MMPs
a theoretical basis by proving the inherent relationships between the fixture failure pattern obtained from design and the first principal component calculated from the OCMM data [5], (4) developing multiple fault diagnosis techniques based on geometric modeling and least-squares estimation [6], and (5) variation modeling and analysis of complaint parts by integrating the part dimensional variability and material properties [7,8]. In 1997, Hu discussed the variation and its propagation problem in multistage manufacturing processes using the term stream of variation (SoV), as suggested by Yoram Koren [8]. Although various significant technological advancements were achieved in Phase I, the methodologies in this phase were specifically devised to address particular variation reduction objectives. In addition, most of them focused on a singlestage manufacturing system (e.g., single fixture, single tooling). The hierarchical representation of multistation systems [4] was still an engineering knowledge “representation,” rather than a mathematical model, for body assembly system. Thus, there was no systematic, unified methodology for variation reduction for multistage manufacturing systems at that time. The concept of using a state space model to describe the variation and its propagation ushered in a new era of SoV methodology development [9,10]. The state space model was typically used in system or automatic control theories, but not in discrete-part manufacturing. The introduction of a state space modeling structure to describe variation and its propagation successfully resolved two fundamental issues in variation reduction of multistage manufacturing processes: (1) the lack of analytical models for variation propagation modeling and (2) the lack of systematic methodologies for variation propagation analysis and optimization. The development of state space models to describe variation propagation provided a solid scientific foundation for manufacturing information (product design, process design, and quality measurements) integration for variation reduction in multistage manufacturing processes. With the state space model as the basis, various SoV-related research efforts have been conducted in both product and process design and the manufacturing process control. The major technological milestones in SoV development are summarized in Figure 1.2, which consists of three major paths: SoV modeling, SoVbased design, and SoV-based process control. More than 40 papers on those topics were published in various journals, including ASME Transactions, IIE Transactions, IEEE Transactions, Technometrics, etc., which represent a truly multidisciplinary research effort contributing mechanical engineering, industrial engineering, system and control engineering, and applied statistics to the knowledge base. The work was also broadly recognized by different research disciplines, which were evidenced by the Best Paper Awards received from American Society of Mechanical Engineering (ASME), North American Manufacturing Engineering Institute (NAMRI), Institute of Industrial Engineering (IIE), and Institute of Operations Research and Management Science (INFORMS). The SoV methodology, as a unified approach, has made fundamental contributions and has had a significant impact on variation reduction of multistage manufacturing processes.
2151_book.fm Page 5 Friday, October 20, 2006 5:04 PM
What Is Stream of Variation for Multistage Manufacturing Processes?
5
FIGURE 1.2 Important milestones in SoV methodology development.
1.2 OVERVIEW OF SOV METHODOLOGY Although SoV methodology originated from dimensional variation control of auto body assembly processes, it has been expanded to address variation management and reduction issues for generic complex multistage manufacturing processes (MMPs). An MMP is referred to as a process involving multiple stations to produce a product. Examples of such processes include: (1) an automotive body assembly that has multiple parts assembled at multiple stations, (2) an engine head production that involves multiple machining operations of a single part at multiple stations, (3) a transfer or progressive stamping process that involves multiple die stations to form a part, and (4) semiconductor manufacturing processes in which a silicon wafer develops in several stages with several layers to form a chip. In an MMP, workpiece dimensional errors are contributed by the errors generated at the current stage, as well as accumulated errors transmitted from previous stages. Thus, the nature of variation and its propagation introduce significant challenges to variation management and reduction for an MMP. This book will discuss how to conduct variation reduction by presenting SoV methodologies. SoV attempts to describe the complex production stream and data stream involved in modeling and analysis of variation and its propagation in an MMP. The production stream is referred to as the physical layout of the multistage manufacturing processes. As shown in Figure 1.3, multiple stations form a subassembly line, and multiple subassembly lines merge into the main assembly line in an automotive body assembly process. Each station generates a dimensional variation on the subassembly, and the subassembly is transferred to the next station to be assembled with more parts. As the parts or subassemblies go through all production
2151_book.fm Page 6 Friday, October 20, 2006 5:04 PM
6
Stream of Variation Modeling and Analysis for MMPs
FIGURE 1.3 Layout of a multistage automotive body assembly process.
lines, a stream of part flow, as well as an associated flow of part variation, will be generated. One interpretation of SoV reflects these multiple stations and multiple line configurations, which leads to the variation and its propagation through the MMP. An interpretation of SoV reflects the complex data relationships in an MMP. As shown in Figure 1.4, the x-axis represents the manufacturing stages; the y axis, time; the z-axis, the quality attributes; and Mi, the quality features. In an MMP, there are three types of correlations among those data streams: (1) the quality attributes are autocorrelated in terms of the stages along the production line and are shown as M2 along the x-axis; (2) the quality attributes are crosscorrelated among them within the same stage and represented as [M1, M2, …, Mm] at stage N along the z-axis; and (3) each quality attribute is also autocorrelated in terms of time owing to the
FIGURE 1.4 Complex data relationships in an MMP.
2151_book.fm Page 7 Friday, October 20, 2006 5:04 PM
What Is Stream of Variation for Multistage Manufacturing Processes?
7
FIGURE 1.5 Variation propagation and notations in SoV modeling.
degradation or wear of production tooling over time and represented as Mi (i = 1, 2, … m) along the y axis. Those three types of correlations, observed as a stream of data, introduce significant challenges in variation modeling, analysis, and control. To investigate the variations of those data streams, this book presents the development of the SoV methodology and its applications. The foundation of the SoV methodology is a mathematical model that links the key product quality characteristics (KPC) with key control characteristics (KCC) (e.g., fixture error, machine error, etc.). This model has a state space representation that describes the deviation and its propagation in an N-station process (as shown in Figure 1.5), i.e., x k = A k −1x k −1 + B k u k + w k , k = 1, 2, …, N,
(1.1)
y k = C k x k + v k , {k} ⊂ {1, 2, …, N},
(1.2)
where k = 1, 2, …, N and is the stage index. xk is the state vector representing the key quality characteristics of the product (or intermediate workpiece) after stage k. uk is the control vector representing the tooling errors (e.g., tolerance when no faults occur, or deviation when failures occur on the tooling) at stage k. yk is the measurement vector representing product quality measurements at stage k. wk and vk are the modeling error and sensing error, respectively. The coefficient matrices Ak, Bk, and Ck are determined by product and process design information: Ak represents the impact of deviation transition from stage k – 1 to stage k, Bk represents the impact of the local tooling deviation on product quality at stage k, and Ck is the measurement matrix, which can be obtained from the defined key product quality features at stage k. If we repeat the modeling efforts for each stage from k = 1 to N, we will get the deviation and its propagation throughout the MMPs. By taking variations on both sides of Equation 1.1 and Equation 1.2, and making certain assumptions, we will obtain the “variation” and its propagation model for the MMPs. With the mathematical model, variation reduction can be achieved in both the design and manufacturing stages through rigorous mathematical decision making. However, significant challenges must be overcome in model development and utilization to realize the benefits of the analytical capability of this model. These challenges are addressed in this book. SoV methodology will address the following important questions for variation reduction in an MMP in detail:
2151_book.fm Page 8 Friday, October 20, 2006 5:04 PM
8
Stream of Variation Modeling and Analysis for MMPs
•
•
•
•
How to model and represent the product and process design information to achieve variation reduction? In this book, we introduce two basic methods: the physical modeling method and data-driven modeling methods. In the former method, the kinematics relationship between KCC and KPC is identified through a detailed physical analysis of the manufacturing process; whereas in the latter method, the model is achieved through a statistical fitting based on the historical process measurement data. Details of SoV modeling will be discussed in Chapter 6 to Chapter 8. How to systematically find the root causes of variability in terms of which manufacturing station, and what in the station introduce the variability? During continuous production, a larger product variation may occur at any stage of an MMP because of worn-out tooling, tooling breakage, and incoming part variation. This book will present a systematic approach to root cause identification. In this approach, a new concept of “statistical methods driven by engineering models” is proposed to integrate the product and process design knowledge with online statistics. The variation model, developed from the design information, is used to link the product variation (quality attributes) with the tooling variation (or potential failure). During production, the product features are measured and the data are used to conduct statistical analysis — based on Equation 1.1 and Equation 1.2 — to identify root causes. Advanced statistics and estimation theory will be used in these efforts. This book presents two types of diagnosis techniques for root cause identification in MMPs: variation pattern matching (Chapter 10) and estimation-based diagnosis (Chapter 11). How to distribute measurement gauges for effective process control in an MMP by determining when, where, and what to measure on the final and intermediate workpieces? One of the major tasks in variation reduction is to design the gauging strategies to measure the product features in a manufacturing process. Most existing industrial practices focus on product coherence inspection (i.e., product-oriented measurements), which is effective in detecting product imperfection but may not be as effective in identifying root causes of product variation. This book proposes a “process-oriented” measurement concept with distributed sensing strategy. In this strategy, selected key control and product characteristics are measured at specific stages to both detect product defects and identify their root causes. The diagnosability issue, i.e., the capability of identifying potential root causes of the process variation for a given measurement strategy, is discussed in Chapter 9. The related issues of optimal sensor placement and distribution are discussed in Chapter 12. How to conduct design evaluation and tolerance synthesis for an MMP? Variation analysis and design evaluation are conducted on the product and during the process design stage to identify the critical components, features, and manufacturing operations. With the SoV model defined in Equation 1.1, the following three tasks can be performed: (1) tolerance
2151_book.fm Page 9 Friday, October 20, 2006 5:04 PM
What Is Stream of Variation for Multistage Manufacturing Processes?
•
9
analysis by allocating the part tolerance (x0) and tooling tolerance (uk) and then predicting the final product tolerance (xN) by solving the difference equations, (2) tolerance synthesis by fixing the final product tolerance (xN) and then assigning the tolerance for individual parts (x0) and tooling components (uk) with minimized cost objective functions, and (3) sensitivity study by identifying the critical parts (xk) or tooling components (uk) that have significant impacts on the final part variation through the evaluation of defined sensitivity indices. The details of those topics will be discussed in Chapter 13 to Chapter 15 of this book. How to integrate product quality and production tooling reliability for effective system design and maintenance decision making? There is a complex and intriguing relationship between product quality and tooling reliability. A degraded (or failed) production tool will lead to greater product variability or number of defects; in addition, the variability of product quality features will impact the degradation rate or failure rate of production tooling. For an MMP, those interactions are more complex, as variations propagate from one stage to the next stage. Thus, a “chain effect” between the product quality and tooling reliability can be observed, and this is known as the “QR Chain” effect. The modeling of the QR Chain is an integrated effort of the SoV model and the semi-Markov process model. Chapter 16 will discuss the modeling of the QR Chain for MMPs. Chapter 17 investigates the applications of the QR Chain effect in reliability and maintenance decisions.
1.3 RELATIONSHIP OF SOV METHODOLOGIES WITH OTHER EXISTING METHODS A typical product realization involves product design, process design, production ramp-up, production and process control and, finally, system maintenance. Various methodologies have been developed and implemented in each stage of the product realization. However, very few techniques exist to systematically address issues related to variation reduction and management. There is a lack of a unified methodology to conduct variation management and reduction over all the different steps from design and manufacturing to maintenance in MMPs. Currently, computer-aided design (CAD) and computer-aided manufacturing/engineering (CAM or CAE) have been widely used in product and process design, and have significantly impacted product realization. CAD mainly focuses on product design in terms of functionality, part geometry, component interactions, and their tolerances. CAM addresses issues of process planning, sequencing, and tooling design for a given product. There is usually an efficient interface for data communication and data transfer among CAD and CAM systems. However, few of them address issues related to “variation” modeling and analysis. The closest efforts in variation modeling and analysis are Variation Simulation Analysis (VSA), or similar industrial packages such as DVA, 3DCS, or Valysis. These technologies use homogeneous transformation matrices to model the variation and its propagation based on geometrical data from the product and tooling design. Then, numerical
2151_book.fm Page 10 Friday, October 20, 2006 5:04 PM
10
Stream of Variation Modeling and Analysis for MMPs
simulations, mainly based on the Monte-Carlo method, are conducted to predict the final product variation with the given process sequence, tooling layout, and tooling tolerance. In general, these efforts can be time consuming and only lead to the “best available” solution from different simulation trials. The VSA-based technology mainly serves as a tolerance analysis and design evaluation tool rather than a tolerance synthesis and design optimization tool for MMPs. Unlike VSA-based technologies, SoV models the variation and its propagation with a station-indexed state space model based on the product and process design. The SoV model in Equation 1.1 contains all essential design information that impacts the product variability. That information includes production sequence information, fixture locator layout and datum changes along the process, tooling and part interactions, part and tooling tolerance, and key product quality features. All those design parameters are integrated into the SoV model in an analytical format. Thus, it facilitates effective computation to obtain analytical solutions for design evaluations (unlike Monte-Carlo simulations). It also provides capabilities for tolerance synthesis (or allocation) and design optimizations for process sequencing, tooling layout, and variation management. In production and process control, statistical process control (SPC) plays an essential role in monitoring process data and detecting mean or variation changes. Although SPC is effective in terms of process change detection, it does not systematically integrate the product and process design data into the process control efforts. Thus, human experience plays a major role in identifying root causes of variation. These efforts could be very challenging and time consuming, especially for a new production line with experience which is limited, or for MMPs in which the process and product parameters have complex and intricate relationships. The SoV models the product measurement data with an observation equation (Equation 1.2), which can be used jointly with the state space equation (Equation 1.1) representing the variation and its propagations based on the product design. As a result, SoV-based process control can systematically integrate production measurement data with the process and product design information. This leads to effective process monitoring, change detection, and root cause identification in terms of which stage introduces product variability, and what process in that stage introduces the variability. In addition, the SoV-based process control also leads to systematic distribution of sensors (e.g., gauging strategies) to ensure identification of the root causes of variability. This effort leads to the transformation of the gauging strategy from the end-of-line “product-inspection”-oriented sensing placement to distributed “process-diagnosis”-oriented sensing strategy. Therefore, more effective process control capabilities can be achieved. Variation reduction and management also affects tooling reliability and maintenance decisions during production. Tooling reliability modeling and analysis are typically separated from the product quality and process control in current industrial practice. However, recent research has indicated that product quality and tooling reliability are inherently related. The QR Chain is an important aspect of an MMP. The SoV model provides a basis for integrated modeling and analysis for product quality and tooling reliability integration, which further leads to integrated design
2151_book.fm Page 11 Friday, October 20, 2006 5:04 PM
What Is Stream of Variation for Multistage Manufacturing Processes?
11
of tolerance optimization and maintenance decision making with consideration of the life cycle of production. In summary, SoV methodology expands the current technological base by developing a unified tool for variation modeling, design analysis, and process control with a focus on variation reduction and management in an MMP.
References 1. Long, B., March 1999, A Systems Solution to a Quality Problem in Auto Body Manufacturing, NIST Special Pub 950-1. 2. Ceglarek, D. and Shi, J., 1995, Dimensional variation reduction for automotive body assembly, Journal of Manufacturing Review, 8, 139–154. 3. Hu, S.J. and Wu, S.M., May 1992, Identifying sources of variation in automobile body assembly using principal component analysis, Transactions of NAMRI, XX, 311–316. 4. Ceglarek, D., Shi, J., and Wu, S.M., 1994, A knowledge-based diagnostic approach for the launch of the auto-body assembly process, ASME Transactions, Journal of Engineering for Industry, 116, 491–499. (Also published in 1993, PED-Vol. 64, Manufacturing Science and Engineering, 401–412.) 5. Ceglarek, D. and Shi, J., 1996, Fixture failure diagnosis for auto body assembly using pattern recognition, ASME Transactions, Journal of Engineering for Industry, 118, 55–65. 6. Apley, D. and Shi, J., 1998, Diagnosis of multiple fixture faults in panel assembly, ASME Transactions, Journal of Manufacturing Science and Engineering, 120, 793–801. (Also Proceedings of the 95’ ASME Winter Annual Meeting, ASME MEDVol. 4, November 1996, 575–581.) 7. Liu, S.C. and Hu, S.J., 1997, Variation simulation for deformable sheet metal assembly using finite element methods, ASME Journal of Manufacturing Science and Engineering, 119, 368–374. 8. Hu, S.J., 1997, Stream of variation theory for automotive body assembly, Annals of the CIRP, 46(1), 1–6. 9. Shi, J. and Jin, J., 1997, Modeling and diagnosis for automotive body assembly process using state space models, Proceedings of International Intelligent Manufacturing System’97, Seoul, Korea, pp.189–196. 10. Jin, J. and Shi, J., November 1999, State space modeling of sheet metal assembly for dimensional control, ASME Transactions, Journal of Manufacturing Science and Engineering, 121, 756–762. 11. Huang, Q., Zhou, N., and Shi, J., 2000, Stream of variation modeling and diagnosis of multi-station machining processes, Proceedings of the 2000 ASME International Mechanical Engineering Congress and Exposition, MED-Vol. 11, November 5–10, Orlando, FL, pp. 81–88 12. Zhou, S., Huang, Q., and Shi, J., April 2003, State space modeling for dimensional monitoring of multistage machining process using differential motion vector, IEEE Transactions on Robotics and Automation, 19(2), 296–309. 13. Camelio, J., Hu, S.J., and Ceglarek, D., 2003, Modeling variation propagation of multi-station assembly systems with compliant parts, ASME Transactions, Journal of Mechanical Design, 125, 673–681.
2151_book.fm Page 12 Friday, October 20, 2006 5:04 PM
12
Stream of Variation Modeling and Analysis for MMPs 14. Chen, Y., Jin, J., and Shi, J., 2004, Integration of dimensional quality and locator reliability in design and evaluation of multi-station body-in-white assembly processes, IIE Transaction, 36(9), 827–839. 15. Ding. Y., Jin. J., Ceglarek, D., and Shi, J., 2000, Process-oriented tolerance synthesis for multi-station manufacturing systems, Proceedings of the 2000 International Mechanical Engineering Congress and Expositions, November 5–10, Orland, FL, pp. 15–22. 16. Ding, Y., Jin, J., Ceglarek, D., Shi, J., 2005, Process-oriented tolerancing for multistation assembly systems, IIE Transactions, 37(6), 493–508. 17. Ding, Y., Ceglarek, D., and Shi, J., 2002, Design evaluation of multi-station assembly processes by using state space approach, ASME Transactions, Journal of Mechanical Design, 124(2), 408–418. 18. Kim, P. and Ding Y., 2004, Optimal design of fixture layout in multi-station assembly processes, IEEE Transactions on Automation Science and Engineering, 1(2), 133–145. 19. Ding, Y., Ceglarek, D., Shi, J., 2002, Fault diagnosis of multistage manufacturing processes by using state space approach, ASME Transactions, Journal of Manufacturing Science and Engineering, 124(2), 313–322. 20. Ding, Y., Shi, J., and Ceglarek, D., 2002, Diagnosability analysis of multi-station manufacturing processes, ASME Transactions, Journal of Dynamics Systems, Measurement, and Control, 124, 1–13. 21. Ding, Y., Kim, P., Ceglarek, D., and Jin, J., 2003, Optimal sensor distribution for variation diagnosis for multi-station assembly processes, IEEE Transactions on Robotics and Automation, 19(4), 543–556. 22. Zhou, S., Chen, Y., and Shi, J., 2004, Root cause identification for quality improvement of multistage machining processes, IEEE transactions on Robotics and Automation, 1(1), 73–83.
2151_book.fm Page 13 Friday, October 20, 2006 5:04 PM
Part I Basis of Matrix Theory and Multivariate Statistics
2151_book.fm Page 14 Friday, October 20, 2006 5:04 PM
2151_book.fm Page 15 Friday, October 20, 2006 5:04 PM
2
Basics of Matrix Theory
2.1 INTRODUCTION Matrix concepts, techniques, and theorems are crucial tools for multivariate analysis and its applications, e.g., process monitoring and quality improvement for multiple significant characteristics. By employing matrix notation, results can be expressed in a clear and compact manner. It can also facilitate geometrical interpretation and algebraic explanation of calculations. In this chapter, we will summarize the concepts, operations, and theorems of matrix algebra that will be required in later sections of this book. More readings on the topic can be found in References 1–5.
2.2 DEFINITIONS OF VECTOR, MATRIX, AND OPERATIONS 2.2.1 DEFINITIONS Definition 2.1 (Matrix): A matrix of dimension m × n is defined as a rectangular array of elements arranged in m rows and n columns, denoted by an uppercase boldface letter as follows:
( )
A = aij
a11 a = 21 am 1
a12 a22 am 2
a1n a2 n ∈ R m×n , amn m × n
(2.1)
where aij denotes the element located in the i th row and the j th column of the matrix A. Definition 2.2 (Vector): A vector is a matrix with only a single row or column, named row vector or column vector, respectively. Remark 2.1: In this book, without special indication, we will refer to a vector as a column vector, which is customarily denoted by a lowercase boldface letter as follows: T
a = a1 a2 am ∈ R m×1.
(2.2)
Remark 2.2: From Definition 2.1 and Definition 2.2, a matrix can also be viewed as a collection of row or column vectors, i.e., A = [a 1T a T2 a Tm ]T = [a1 a 2 a n ],
(2.3)
15
2151_book.fm Page 16 Friday, October 20, 2006 5:04 PM
16
Stream of Variation Modeling and Analysis for MMPs
where a~ i = [ai1 ai2 ain] is the i th row vector with dimension n, and aj = [a1j a2j amj]T is the j th column vector with dimension m. Definition 2.3 (Determinant of a square matrix): Let A be an m × m square matrix. The determinant of A, denoted by A or det(A), is defined as: A =
∑ ( −1)
α
a1 j1 a2 j2 amjm .
(2.4)
m
All products consist of one element from each row and column and multiplied by (–1) if the number of inversions of the particular permutation j1, j2 ,..., jm from the standard order 1, 2, …, m. The summation extends through all m! permutations of the column subscripts. The number of inversions α in a particular permutation is defined as the total number of times an element is followed by numbers that would ordinarily precede it in the standard order 1, 2, …, m. Matrix A is referred to as a singular matrix if A = 0. Example 1: A = a11 , A = a11 a Example 2: A = 11 a21
a12 , A = a11a22 − a12 a21 a22
a11 Example 3: A = a21 a31
a12 a22 a32
a13 a23 , a33
A = a11a22 a33 + a12 a23 a31 + a13 a21a32 − a13 a22 a31 − a11a23 a32 − a12 a21a33 Definition 2.4 (Trace): Let A be an m × m square matrix. The sum of diagonal elements is called the trace of A, denoted by m
tr ( A) =
∑a
ii
(2.5)
i =1
Definition 2.5 (Inverse): The inverse of the m × m square matrix A = ( aij ) , denoted by A–1, is the unique matrix such that AA–1 = A–1A = I
(2.6)
2151_book.fm Page 17 Friday, October 20, 2006 5:04 PM
Basics of Matrix Theory
17
If such an A −1 exists, then A is nonsingular; otherwise, A is singular. Some useful results related to inverses are summarized below. Here, we assume that all the inverses exist. 1. ( A −1 )T = ( A T )−1 2. ( A −1 )−1 = A 3. ( AB)−1 = B−1A −1 1 4. ( kA)−1 = A −1, k is a nonzero scalar k −1 a11 1 / a11 0 0 0 0 a22 0 0 0 5. = 0 0 0 0 0 0 amm 0 0
0 1 / a22 0 0
0 0 0
0 0 0 1 / amm
6. If A is orthogonal, then A −1 = A T Definition 2.6 (Reduced row echelon form of a matrix): A matrix is said to be in reduced row echelon form if the following conditions are satisfied: • • • •
The first nonzero number in a row is 1 (we call it the leading 1). All rows of zeros (if any) are together at the bottom of the matrix. Each column that contains a leading 1 has only zeros below it. Each column that contains a leading 1 has zeros everywhere else.
If all these properties except the last are satisfied, then the matrix is in row echelon form. Remark 2.3: The reduced row echelon form of a matrix is unique. Remark 2.4: The reduced row echelon form of a matrix can be obtained through row operations, i.e., trading two rows, multiplying a row by a nonzero scalar, or adding a scalar multiple of one row to another row. For example, Matrix 1 A = 2 1
1 3 3
1 7 −2
3 0 17
can be transformed into reduced row echelon form as 1 0 0
0 1 0
0 0 1
1 4 . −2
2151_book.fm Page 18 Friday, October 20, 2006 5:04 PM
18
Stream of Variation Modeling and Analysis for MMPs
2.2.2 PARTITIONED MATRICES In certain cases, some blocks in a matrix possess common characteristics. Thus, it is sometimes convenient to partition the whole matrix into several blocks, for example; A11 A A = ( A ij ) = 21 A m 1
A12 A 22 Am 2
A1n A2 n A mn
(2.7)
where each submatrix A ij has pi rows and q j columns. Remark 2.5: In Equation 2.7, submatrices in the same row of A have the same number of rows; so do the submatrices in the same column of A. Remark 2.6: The operations of partitioned matrices are similar to those of unpartitioned ones, except that the elements in partitioned matrices are submatrices. For example, if A = ( Aij) and B = (Bij) are partitioned into submatrices with similar dimensions, then A + B = ( Aij + Bij) , n A1k B k 1 k =1 AB= n A mk B k 1 = k 1
∑
∑
A1k B kp k =1 n A mk B kp k =1 n
∑
∑
in which the dimensions of each submatrix product must conform. By employing submatrices, we can simplify computations of the inverse and determinant. Consider a particular important partition of the form: A A = 11 A 21
A12 A 22
(2.8)
If both A11 and A 22 are square, then the inverse and determinant of A can be readily calculated in terms of submatrices: 1. If A 22 has an inverse, then A −1 = ( A11 − A12 A −221A 21 )−1 −1 −1 −1 − A 22 A 21 ( A11 − A12 A 22 A 21 )
(2.9) A + A A 21 ( A11 − A12 A A 21 ) A12 A
−( A11 − A12 A −221A 21 )−1 A12 A −221 −1 22
−1 22
−1 22
−1
−1 22
2151_book.fm Page 19 Friday, October 20, 2006 5:04 PM
Basics of Matrix Theory
2.
19
−1 A = A11 ⋅ A 22 − A 21A11 A12 if A11 has an inverse
−1 A 21 if A 22 has an inverse = A 22 ⋅ A11 − A12 A 22
(2.10)
3. If both A11 and A 22 have inverses, then −1 −1 −1 −1 −1 ( A11 − A12 A 22 A 21 )−1 = A11 + A11 A12 ( A 22 − A 21A11 A12 )−1 A 21A11 .
(2.11)
As a special case, if A is an m × m nonsingular matrix, a is an m × 1 vector, and k is a scalar, then we have the identity: ( A + kaa T )−1 = A −1 −
k A −1aa T A −1 T −1 1 + ka A a
(2.12)
2.3 QUADRATIC FORMS Definition 2.7 (Quadratic Form): A quadratic form of the variables xi (i = 1, 2, …, n) is of the type: f ( x1 , x 2 ,..., x n ) = a11x12 + a22 x 22 + + ann x n2 + 2a12 x1x 2 + + 2a1n x1x n + + 2a( n−1) n x n−1x n
(2.13)
Let x T = x1 x 2 x n and A = (aij) be a symmetric matrix of order n × n . Then Equation 2.13 can readily be expressed in an equivalent matrix notation: f (x) = x T A x
(2.13P′)
Remark 2.7: If x is a random vector, the quadratic form plays an important role in univariate and multivariate analysis. Definition 2.8 (Positive definite and positive semidefinite matrix): A symmetric matrix A = (aij) and its associate quadratic form in Equation 2.13 are called positive definite if xTAx > 0 for ∀x ≠ 0
(2.14)
xTAx ≥ 0 for ∀x ≠ 0
(2.15)
and positive semidefinite if
2151_book.fm Page 20 Friday, October 20, 2006 5:04 PM
20
Stream of Variation Modeling and Analysis for MMPs
2.4 VECTOR SPACE AND GEOMETRICAL INTERPRETATIONS As a column vector a = [a1 a2 am]T can be viewed as a special matrix of m × 1 order, as defined in Definition 2.2, addition and multiplication by a scalar also hold for vectors. Hence, we can introduce the definition of vector space as follows: Definition 2.9 (Vector space): Vector space is the space of all the real vectors a = [a1 a2 am]T with the operations of addition and multiplication by a scalar. Definition 2.10 (Linear combination and linear span): The linear combination of vectors ai (i = 1, 2, …, n) is defined as a vector b of the form: n
b=
∑k a
(2.16)
i i
i =1
where all ki ‘s are scalars. The linear span of ai (i = 1, 2, …, n) is formed by the set of all linear combinations of ai (i = 1, 2, …, n). Remark 2.8: An m-dimensional vector a = [a1 a2 am]T can be geometrically viewed as a directed line in m-dimensional space with component ai along the i-axis, i = 1, 2, …, m. The 2-D case is illustrated in Figure 2.1. Thus, we can define the length and angle between two vectors. Definition 2.11 (Length of a vector): Let a = [a1 a2 am]T be an m-dimensional vector. Its length, la , is defined by: la = a12 + a22 + + am2
(2.17)
Remark 2.9: For the scalar multiplication ka, the direction of a does not change if k > 0, but reverses if k < 0.
a2 a
1
0 FIGURE 2.1 A 2-D vector.
a1
a1 a2
2151_book.fm Page 21 Friday, October 20, 2006 5:04 PM
Basics of Matrix Theory
21
Remark 2.10: For the scalar multiplication ka, its length is changed in the form: lka = ( ka1 )2 + ( ka2 )2 + + ( kam )2 = k
a12 + a22 + + am2 = k la (2.18)
Thus, length increases if k >1, and decreases if 0 < k <1. Before introducing the angle between two vectors, we will first define their inner product. Definition 2.12 (Inner product): The inner product of two m-dimensional vectors a and b is defined by: m
T
T
(a, b) = a b = b a =
∑a b
i i
(2.19)
i =1
Remark 2.11: la = (a, a ) = a T a Definition 2.13 (Angle): The angle θ between two m-dimensional vectors a and b is defined by: cos(θ) =
(a, b) (a, a ) ⋅ (b, b)
=
aT b bT a = la ⋅ lb la ⋅ lb
(2.20)
From Equation 2.20, cos(θ) = 0 only if aT b = 0 and we call a and b perpendicular, denoted by a ⊥ b . Given two m-dimensional vectors a and b, we can project one on the other. For example, when m = 2, Figure 2.2 illustrates the projection of a on b.
FIGURE 2.2 Angle between two vectors and projection of a on b.
2151_book.fm Page 22 Friday, October 20, 2006 5:04 PM
22
Stream of Variation Modeling and Analysis for MMPs
Definition 2.14 (Projection of vectors): The projection of a on b is defined by (a, b) aT b b ⋅ b= (b, b) lb lb where
(2.21)
b is the unit vector along the direction of b. The length of the projection is lb aT b aT b = la ⋅ = la | cos(θ) | lb la ⋅ lb
(2.22)
2.5 EIGENVALUES AND EIGENVECTORS OF A MATRIX Let A be an m × m matrix. It is said to have an eigenvalue λ and corresponding eigenvector x ≠ 0 if Ax = λx
(2.23)
Remark 2.12: Equation 2.23 has a clear geometrical interpretation. The left side A x indicates a linear transformation of vector x. The right side λ x means the new length of x. So Equation 2.23 indicates that when a linear transformation A is applied to an eigenvector, its length is multiplied by λ; however, its direction remains unchanged if λ > 0 and is reversed if λ < 0. The set of eigenvalues, denoted by λ(A), can be obtained by solving the socalled mth order characteristics polynomial or equation: A − λI = 0
(2.24)
A − λI = (− λ)m + t1 (− λ)m −1 + ... + tm −1 (− λ)1 + A
(2.25)
By using Laplace expansion,
where ti is the sum of all i × i principal minor determinants. So: m
t1 =
∑a
ii
= tr ( A)
(2.26)
i =1
Based on the theory of polynomial equations, it follows that: m
∏λ = A i
i =1
(2.27)
2151_book.fm Page 23 Friday, October 20, 2006 5:04 PM
Basics of Matrix Theory
23
m
∑ λ = tr (A)
(2.28)
i
i =1
Some properties of eigenvalues are summarized as follows: 1. The eigenvalues of a real symmetric matrix are all real. 2. The eigenvalues of a positive definite matrix are all positive. 3. For an m × m positive semidefinite symmetric matrix with rank r, i.e., | A | ≥ 0 and rank (A) = r, A has exactly r positive eigenvalues and m – r zero eigenvalues. 4. λ( AB) = λ(BA). 5. λ diag ( a11, a22 ,..., amm ) = a11, a22 ,..., amm .
(
) {
} After obtaining the set of eigenvalues λ( A) = {λ , λ , ..., λ } , the corresponding 1
2
m
eigenvectors can be calculated by solving the equation: A − λ i I ⋅ x i = 0
(2.29)
Remark 2.13: From Equation 2.29, it is apparent that the eigenvector x i corresponding to the eigenvalue λ i is not unique. For a symmetric matrix A, the eigenvalues and eigenvectors have the following properties: 1. If λ i ≠ λ j are two distinct eigenvalues of symmetric matrix A, then the associated eigenvectors x i and x j are perpendicular, i.e., x i ⊥ x j . 2. For any real symmetric matrix A , there exists an orthogonal matrix P such that
(
PTAP = diag λ1, λ 2 , …, λ m
)
(2.30)
Remark 2.14: Using the preceding properties, the quadratic form of symmetric matrices can be simplified by orthogonal transformation: x = Py
(2.31)
It follows immediately that m
x Ax = y P APy = T
T
T
∑λ y
2 i i
(2.32)
i =1
Remark 2.15: If A is not full rank, say rank(A) = r, then only r terms are left in the summation. Generally, the so-called spectral decomposition of an m × m symmetric matrix A is given by:
2151_book.fm Page 24 Friday, October 20, 2006 5:04 PM
24
Stream of Variation Modeling and Analysis for MMPs
A = λ1e1e1T + λ 2 e2 eT2 + ... + λ m em emT
(2.33)
where λi’s, i = 1, 2, …, m are eigenvalues of A and ei’s, i = 1, 2, …, m are associated normalized eigenvectors. So 1, i = j , i, j = 1, 2, ..., m eTi e j = 0, i ≠ j
(2.34)
Remark 2.16: Comparing Equation 2.32 and Equation 2.33, we can see that yi = x T ei , i = 1, 2, ..., m
(2.35)
2.6 VECTOR AND MATRIX DIFFERENTIATION, MAXIMIZATION, AND OPERATORS 2.6.1 DIFFERENTIATION
WITH
VECTORS
In later chapters, we will need to determine the partial derivatives of likelihoods and other scalar functions of vectors and matrices. Let f ( x ) = f ( x1, x2 ,..., xm ) be a continuous function whose first and second partial derivatives ∂f ( x ) ∂2 f ( x ) and ∂xi ∂xi ∂x j
(2.36)
exist for all points x in the span of m-dimensional Euclidean space. Definition 2.15 (Partial differentiation with vectors): The partial differentiation of function f ( x ) = f ( x1, x2 ,..., xm ) with respect to (w.r.t.) vector x is defined by: ∂f ( x ) ∂x 1 ∂f ( x ) ∂f ( x ) = ∂x2 . ∂x ∂f ( x ) ∂xm
(2.37)
The partial derivatives of some special forms of f ( x ) are summarized as follows.
2151_book.fm Page 25 Friday, October 20, 2006 5:04 PM
Basics of Matrix Theory
25
1. f (x) = constant for all x ∂f ( x ) =0 ∂x
(2.38)
∂f ( x ) =a ∂x
(2.39)
2. f (x) = a T x = x T a
3. Quadratic form: f ( x ) = x T Ax
(
)
∂f ( x ) = A + AT x ∂x
(2.40)
4. General quadratic form: f ( x ) = (a − Bx )T A(a − Bx ) where a is an m × 1 vector, B = [B1 B2 Bn] is an m × m matrix, and A is an m × m symmetric matrix. By using the chain rule: ∂f ( x ) = −2 BT A(a − Bx ) ∂x
(2.41)
Definition 2.16 (Hessian matrix): A Hessian matrix is defined by the matrix of second-order partial derivatives of a function f ( x ) = f ( x1, x2 ,..., xm ) w.r.t. vector x, which is ∂2 f ( x ) 2 ∂x1 ∂2 f ( x ) = H= ∂x T ∂x 2 ∂ f (x) ∂x ∂x 1 m
∂2 f ( x ) ∂x1∂xm ∂2 f ( x ) ∂xm2
(2.42)
As an example, the Hessian matrix of the quadratic form f ( x ) = x T Ax is 2A, if A is symmetric. After introducing definitions of first- and second-order partial derivatives, we can state the necessary condition for a maximum or minimum of f ( x ) at x = x*: ∂f (x) =0 ∂x x= x*
(2.43)
2151_book.fm Page 26 Friday, October 20, 2006 5:04 PM
26
Stream of Variation Modeling and Analysis for MMPs
Remark 2.17: This kind of maximum or minimum is called a stationary maximum or minimum because a global extreme may be located in the boundary of the feasible region of x, or as a cusp or another form at which the first-order derivative does not exist. A sufficient condition for a maximum (or minimum) at the point x* satisfying Equation 2.43 is that the Hessian matrix at x* H=
∂ 2 f (x) ∂x T ∂x x= x*
(2.44)
be negative (or positive) definite. Remark 2.18: If the Hessian matrix is indefinite at x*, then the point x* is not an extreme point.
2.6.2 MATRIX INEQUALITIES
AND
MAXIMIZATION
The matrix inequalities help obtain the maximization results. First, some important matrix inequalities are introduced. Cauchy–Schwarz Inequality: Let a and b be any two m × 1 vectors. Then (a, b)2 ≤ (a, a ) ⋅ (b, b) or (a T b)2 ≤ (a T a ) ⋅ (b T b)
(2.45)
The equality holds if and only if a = kb or b = ka, where k is a constant. An important extension of Cauchy–Schwarz Inequality is given as follows: Extended Cauchy–Schwarz Inequality: Let a and b be any two m × 1 vectors, and D be an m × m positive definite matrix. Then (a, b)2 ≤ (a, Da ) ⋅ (b, D −1b) or (a T b)2 ≤ (a T Da ) ⋅ (b T D −1b)
(2.46)
The equality holds if and only if a = kD–1b or b = kDa, where k is a constant. The proofs of Equation 2.45 and Equation 2.46 can be found in Reference 1. Based on the extended Cauchy–Schwarz Inequality, we have the following maximization theorem: Theorem 2.1 (Maximization): Suppose A is a positive definite matrix of order m × m and a is a given m × 1 vector. Then for ∀ x ≠ 0 of dimension m × 1 , max x ≠0
( x, a )2 = a T A −1a ( x, Ax )
(2.47)
The maximum is obtained when x = kA −1a for any constant k ≠ 0 . The following theorem gives the maximization result of quadratic forms for points on the unit sphere:
2151_book.fm Page 27 Friday, October 20, 2006 5:04 PM
Basics of Matrix Theory
27
Theorem 2.2 (Maximization of quadratic forms for points on the unit sphere): Suppose A is a positive definite matrix of dimension m × m with eigenvalues λ1 ≥ λ 2 ≥ ≥ λ m > 0
(2.48)
and corresponding eigenvectors ei , i = 1, 2, ..., m . Then ( x, Ax ) = λ1 obtained when x = e1 ( x, x ) ( x, Ax ) 2. min = λ m obtained when x = em x ≠ 0 ( x, x ) ( x, Ax ) 3. max = λ i+1 obtained when x = ei+1 i = 1, 2, ..., m − 1 x ⊥ e1,...,ei ( x, x ) 1. max x ≠0
(2.49) (2.50) (2.51)
The proofs can be found in Reference 1.
2.6.3 VEC OPERATOR, KRONECKER PRODUCT, AND HADAMARD PRODUCT 2.6.3.1 Vec Operator and Kronecker Product The concepts and results of the Kronecker product, which is also often called the direct product, are mentioned in many papers on variation modeling and analysis. Before introducing the definitions and some of the important properties of this product, we will first cover the concept of a vector operator, vec(⋅) . Definition 2.17 (Vector operator or Vec operation): Let X = [X1 X2 Xq] be a p × q matrix with columns X i ∈ R p ×1, i = 1, 2, …, q. Then the vec operator, vec(X) , is defined by stacking all the q columns of X in one column, i.e., X1 X2 vec (X) = ∈ R pq ×1 X q
(2.52)
Definition 2.18 (Kronecker product, or direct product): Given matrices A = ( aij ) ∈ R p ×q and B = (bij ) ∈ R r ×t , the Kronecker product of A and B, denoted by A ⊗ B, is the pr × qt matrix, a11B a21B A⊗B = a p1B
a12 B
a22 B
a p2B
a1q B a2 q B a pq B
(2.53) pr × qt
2151_book.fm Page 28 Friday, October 20, 2006 5:04 PM
28
Stream of Variation Modeling and Analysis for MMPs
Remark 2.19: Based on this definition, we can derive a special result about blockdiagonal matrices as follows: B 0 0
0 B 0
0 0 = Ip ⊗ B B pr × pt
(2.54)
where B ∈ R r ×t occurs p times on the diagonal. From the definition, it is easy to prove some important properties about the Kronecker product, which are summarized as follows: Property 1: ( aA) ⊗ (bB) = ab ( A ⊗ B) holds for any scalars a and b. Property 2: If A ∈ R p ×q , B ∈ R p ×q , and C ∈ R s ×t , then ( A + B) ⊗ C = A ⊗ C + B ⊗ C Property 3: ( A ⊗ B) ⊗ C = A ⊗ (B ⊗ C) Property 4: If A ∈ R m × n , B ∈ R p ×q , C ∈ R n ×s , and D ∈ R q ×t , then
( )
( A ⊗ B)(C ⊗ D) = AC ⊗ (BD) Property 5: ( A ⊗ B)T = A T ⊗ BT Property 6: If A and B are both nonsingular matrices, then ( A ⊗ B)−1 = A −1 ⊗ B−1 Property 7: If A ∈ R p × p and B ∈ R p × p , then tr ( A ⊗ B) = tr ( A) ⋅ tr (B) Property 8: If A and B are both orthogonal matrices, then A ⊗ B is also an orthogonal matrix. Property 9: If A ∈ R p × p , B ∈ R q ×q , then det( A ⊗ B) = [det( A)]q ⋅ [det(B)] p Property 10: If A > 0 and B > 0, then A ⊗ B > 0
2151_book.fm Page 29 Friday, October 20, 2006 5:04 PM
Basics of Matrix Theory
29
Property 11: If A ∈ R p × p and B ∈ R q ×q have eigenvalues αi, i = 1, 2, …, p and βj, i = 1, 2, …, q, respectively, then A ⊗ B has eigenvalues αiβj, i = 1, 2, …, p, j = 1, 2, …, q. The relationships between the vec operator and the Kronecker products are given in the following lemmas. Lemma 2.1: Let A ∈ R p ×m , X ∈ R m × n , and B ∈ R n ×q . Then vec ( AXB) = (BT ⊗ A) ⋅ vec (X)
(2.55)
The following remarks provide an application of this lemma. Remark 2.20: If X = [X1 X 2 X n ] ∈ R m × n is a random matrix whose columns are independent m × 1 random vectors each with the same covariance matrix Σ, i.e., cov(X i ) = Σ for i = 1, 2, …, n, then X1 X vec (X) = 2 ∈ R mn ×1 X n ∑ 0 cov[ vec (X)] = 0
0 ∑ 0
0 0 = In ⊗ ∑ ∈ R mn× nm ∑ mn× nm
(2.56)
(2.57)
Equation 2.57 holds because all the Xi’s are all independent with the same covariance matrix. Remark 2.21: Let A ∈ R p ×m and B ∈ R n ×q be constant matrices, and X ∈ R m × n be a random matrix whose columns are independent. Then applying this lemma and properties of the Kronecker product, the transformed random matrix Y = AXB has the following relationships: 1. E[vec( Y)] = (BT ⊗ A) ⋅ E[vec(X)] 2. cov[vec( Y)] = cov[(BT ⊗ A) ⋅ vec(X)] = (BT ⊗ A) ⋅ cov[vec(X)] ⋅ (BT ⊗ A)T = (BT ⊗ A) ⋅ (In ⊗ Σ) ⋅ (B ⊗ AT ) = (BT B) ⊗ ( AΣAT )
2151_book.fm Page 30 Friday, October 20, 2006 5:04 PM
30
Stream of Variation Modeling and Analysis for MMPs
Lemma 2.2: For appropriate sizes of matrices, the following statements hold: 1. vec( AB) = (I ⊗ A) ⋅ vec(B) = (BT ⊗ I) ⋅ vec( A) = (BT ⊗ A) ⋅ vec(I) 2. tr( ABC) = [ vec ( A T )]T (I ⊗ B) vec (C) 3. tr ( AX T BXC) = [ vec (X)]T ( A T C T ⊗ B) vec (X) = [ vec (X)]T (CA ⊗ BT ) vec (X) 2.6.3.2 Hadamard Product Definition 2.19 (Hadamard product): Suppose A and B are two m × m matrices. Their Hadamard product is the entrywise product of A and B, i.e., the m × n matrix A B whose (i,j)th entry is aijbij. For the following properties, suppose A, B, and C are m × m matrices and λ is a scalar. Property 1: A B = B A. Property 2: A (B + C) = A B + A C. Property 3: A (λB) = λ( A B). Property 4: If A and B are diagonal matrices, then A B = AB. Property 5 (Oppenheim inequality): If A and B are positive definite matrices and aii’s are the diagonal entries of A, then AB ≥
m
∏ a = B ii
i =1
with equality if and only if A is a diagonal matrix.
2.7 EXERCISES 1. Suppose aT = [3, 1, 4] and bT = [2, –3, 1]. a.Determine the length of a and b. b.What is the angle between a and b? c.What is the projection of a on b? d.Find the length of the projection in (c). 2. Let 3 A = 5 4 5
4 5 3 − 5
2151_book.fm Page 31 Friday, October 20, 2006 5:04 PM
Basics of Matrix Theory
31
a. Show if AA T = A T A = I , i.e., whether A is an orthogonal matrix. b. If A is an orthogonal matrix, verify that A −1 = A T . 3. Let 3 A= −2
−2 6
a. Check whether A is symmetric. b. Verify that A is positive definite. 4. Suppose 2 A= 2
2 . 5
Determine the following: a. The eigenvalues and eigenvectors of A. b. Calculate A–1. c. Calculate the eigenvalues and eigenvectors of A–1. 5. Let 1 A= 4
3 5
a. What is the determinant of A? b. Determine the trace of A. c. Determine whether the product of the eigenvalues of A is the same as the determinant of A. d. Determine whether the sum of the eigenvalues of A is the same as the trace of A. 6. Consider the symmetric matrix 5 A = 2 3
2 9 −2
3 −2 6
Verify whether A = λ1e1e1T + λ 2 e2 eT2 + λ 3e3eT3 . This is the so-called spectral decomposition of a symmetric matrix A. 7. Suppose f ( x ) = 2 x12 + 3 x1 x23 + 7 x32 . Find a.
∂f ( x ) ∂x
b. The Hessian matrix or
∂ 2 f (x) . ∂x T ∂x
2151_book.fm Page 32 Friday, October 20, 2006 5:04 PM
32
Stream of Variation Modeling and Analysis for MMPs
8. Suppose f (x) is in a quadratic form ( f (x) = xT Ax), i.e., 2 f ( x ) = x1 x2 x3 4 −1 Determine whether
(
3 −1 0
1 x1 5 x2 2 x3
)
∂f ( x ) = A + AT x . ∂x
References 1. Johnson, R.A. and Wichern, D.W., Applied Multivariate Statistical Analysis, 4th ed., Prentice Hall, Upper Saddle River, NJ, 1998. 2. Anderson, T.W., An Introduction to Multivariate Statistical Analysis, John Wiley & Sons, New York, 1958. 3. Graybill, F.A., Introduction to Matrices with Applications in Statistics, Wadsworth Publishing Company, Inc., Belmont, CA, 1969. 4. Harville, D.A., Matrix Algebra from a Statistician’s Perspective, Springer-Verlag, New York, 1997. 5. Noble, B. and Daniel, J.W., Applied Linear Algebra, 3rd ed., Prentice Hall, Englewood Cliffs, NJ, 1987.
2151_book.fm Page 33 Friday, October 20, 2006 5:04 PM
3
Basics of Multivariate Statistical Analysis
3.1 INTRODUCTION In variation modeling and analysis, it is common to consider multiple random variables. Thus, multivariate statistical analysis is essential to understand and master variation reduction methodologies. This chapter provides some basics of multivariate statistical analysis, including underlying theoretical population settings for random phenomena involving multivariate distributions, marginal distributions, conditional distributions, population means, variances, covariances, correlations, and other characteristics. The most important multivariate distribution, the multivariate normal distribution, is discussed in detail and some important results are summarized. An introduction of sampling theory is also included in this chapter. It should be noted that all the proofs are not provided here. Readers who want to learn more about the proofs may please refer to related mathematical statistics materials [1–10].
3.2 MULTIVARIATE DISTRIBUTION AND PROPERTIES 3.2.1 RANDOM VECTORS, CUMULATIVE DISTRIBUTION FUNCTIONS (CDF), AND PROBABILITY DENSITY FUNCTIONS (PDF) A random vector is a vector comprising elements that are random variables. Uppercase letters denote random variables and random vectors, and corresponding lowercase letters denote realizations of the random variables or random vectors. Definition 3.1 (Cumulative distribution function): Let X = [ X1, X2 , , X p ]T be a random vector with X1, X2 , , X p as the p random variables. The associated joint CDF is the function F defined by: F ( x ) = Pr(X ≤ x ) = Pr( X1 ≤ x1 , X2 ≤ x2 , , X p ≤ x p ) ,
(3.1)
where x = [ x1, x2 , , x p ]T is a realization of random vector X. Two important cases are absolutely continuous and discrete distributions. Definition 3.2 (Absolutely continuous distributions): A random vector X is absolutely continuous if there exists a probability density function (PDF), f ( x ) , s.t., F (x) =
∫
x
−∞
∫ ∫ xp
f (u) du =
−∞
x1
−∞
f (u1, u2 , , u p ) du1 du p
33
2151_book.fm Page 34 Friday, October 20, 2006 5:04 PM
34
Stream of Variation Modeling and Analysis for MMPs
Here, f ( x ) is called a joint PDF. Note that for any measurable set D ⊆ R p , Pr(X ∈ D ) =
∫ f (u)du
(3.2)
D
and
∫
+∞
−∞
f (u) du = 1
(3.3)
Definition 3.3 (Discrete distributions): For a discrete random vector X, the total probability is concentrated on a countable (or finite) set of points {χi ; i = 1, 2, }. Its probability mass function (PMF) is defined as: f (x i ) = Pr(X = x i ) for i = 1, 2, ; and f (x) = 0, otherwise.
(3.4)
Similarly, Pr(X ∈ D) =
∑ f (x ) i
i:xi ∈D
Then, for both cases, the support S of X is defined as the set:
{
S = x ∈ R p : f (x) > 0
3.2.2 MARGINAL
AND
}
CONDITIONAL DISTRIBUTIONS
Consider the partitioned vector X X = 1, X2 where X1 ∈ R k ×1 and X 2 ∈ R ( p − k )×1 , respectively. Definition 3.4 (Marginal CDF and PDF): Under the pervious partition, the function:
(
)
(
Pr X1 ≤ x10 = Pr X11 ≤ x10 , X 21 ≤ x 20 ,, X k1 ≤ x k0 , X k1+1 ≤ ∞,, X 1p ≤ ∞
)
(3.5)
is called the marginal CDF of X1. Let f ( x ) be the joint PDF of X. Then the marginal PDF of X1 is given by: f1 ( x 1 ) =
∫
+∞
−∞
f ( x 1 , x 2 ) dx 2
The marginal PDF of X2 can be defined in a similar manner.
(3.6)
2151_book.fm Page 35 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
35
Definition 3.5 (Conditional PDF): Under the previous partition, the conditional PDF of X2, given X1 = x10 , is defined as:
(
f x 2 | X1 = x
0 1
) = (f ( x ) ) f x10 , x 2
(3.7)
0 1
1
(Assume f1 ( x10 ) ≠ 0). Similarly, we can define the conditional PDF of X1 given X2 = x 02 . Based on the conditional distribution, we can introduce the important concept statistical independence as follows: Definition 3.6 (Statistical independence): When the conditional PDF f (x2|X1 = x 10 ) is the same for all possible values of x10 , then X1 and X2 are said to be statistically independent of each other, denoted as: X1 ⊥⊥ X2. Theorem 3.1: If X1 ⊥⊥ X2, then f ( x ) = f1 ( x 1 ) f 2 ( x 2 )
(3.8)
3.2.3 POPULATION MOMENTS Definition 3.7 (Population mean vector): Suppose X = [ X1, X2 , , X p ]T is a random vector with PDF f ( x ) . Then the expectation or mean of random vector X is defined as: E ( X1 ) E(X2 ) µX = E X = E ( X p )
( )
∫
(3.9)
+∞
xi fi ( xi ) dxi , i = 1, 2, , p . µ X is also called where µX = [µ1, µ2, …, µp]T and µ i = −∞ the population mean vector. It is easy to show that the expectation operator has the following properties:
(
)
( ) X Partition: Let X = , then E ( X ) = ∫ x X If X ⊥⊥ X , then E ( X X ) = E ( X ) E ( X ) .
1. Linearity: E AX + b = AE X + b for any A ∈ R q × p and b ∈R q . 2.
+∞
1
1
2
3.
1
2
1
2
1
−∞
f ( x 1 ) dx 1 .
1 1
2
Definition 3.8 (Population covariance matrix): The covariance matrix (or variancecovariance matrix, or dispersion matrix) of random vector X = [ X1, X2 , , X p ]T is defined as:
(
∑ X = var(X) = E (X − µ X )(X − µ X )T
)
(3.10)
2151_book.fm Page 36 Friday, October 20, 2006 5:04 PM
36
Stream of Variation Modeling and Analysis for MMPs
For simplicity, if random vector X = [ X1, X2 , , X p ]T has mean vector µ X and covariance matrix ∑ X , we denote it as X ~ ( µ X , ∑ X ) or ( µ , ∑ ). It can be proved that the covariance has the following properties: 1. ∑ X is semidefinite, i.e., for all a ∈R p , a T ∑ X a ≥ 0. 2. ∑ X is symmetric, i.e., ∑TX = ∑ X with diagonal elements σ ii = var( Xi ) and the (i, j)th element σ ij = cov( Xi , X j ) for i ≠ j. Thus,
( )
∑ X = σ ij σ11 σ 21 σ p1
p× p
=
σ12
σ 22
σ p2
σ1 p var( X1 ) σ 2 p cov( X1 , X 2 ) = σ pp cov( X1 , X p )
cov( X1 , X 2 )
var( X 2 )
cov( X 2 , X p )
cov( X1 , X p ) cov( X 2 , X p ) . var( X p )
∑ X = E (XXT ) − µ Xµ TX . For all constant vectors, a ∈R p , var(a T X) = a T ∑ X a. var( AX + b) = A ∑ X AT , where A is a matrix of an appropriate size. cov(X, X) = ∑ X. cov(X1 + X 2 , Y) = cov(X1, Y) + cov(X 2 , Y) . T Let X, Y ∈R p, then cov(X, Y) = cov( Y, X) and var(X + Y) = ∑ X + cov(X, Y) + cov( Y, X) + ∑ Y . 9. cov( AX, BY) = A cov(X, Y)BT, where A and B are matrices of appropriate sizes. 10. If X ⊥⊥ Y, then cov( X , Y ) = 0, but the converse may not be true. 3. 4. 5. 6. 7. 8.
Based on the covariance matrix, we can define the statistical distance between two random vectors. Clearly, it is a special case of Mahalanobis distance. Definition 3.9 (Statistical distance or Mahalanobis distance): If X, Y ∈R p , then the statistical distance between X and Y with matrix ∑ is defined as: d ∑ (X, Y) = (X − Y)T ∑ −1 (X − Y)
(3.11)
For convenience, the matrix ∑ is usually chosen to be the covariance matrix. For example: µ, ∑). The Mahalanobis distance between X and µ, d ∑ (X, µ ), is 1. Let X ~ (µ a random variable. 2. Let X ~ ( µ X , ∑) and Y ~ ( µ Y , ∑). Then d ∑ (µ X , µ Y ) is a Mahalanobis distance between the parameters. Also, for any nonsingular matrix A ∈ R p × p , d ∑ (µ X , µ Y ) is invariant under the following transformations:
2151_book.fm Page 37 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
37
= AX + b X :→ X = AY + b Y :→ Y = A ∑ AT ∑ :→ ∑ i.e., d ∑ (µ X , µ Y ) = d ∑ (µ X , µ Y ).
3.2.4 CORRELATION COEFFICIENTS There are three types of correlation coefficients: simple correlation coefficients, partial correlation coefficients, and multiple correlation coefficients. It should be noted that partial correlation coefficients and multiple correlation coefficients are special cases of simple correlation coefficients. µx, ∑x). As before, suppose X = [ X1, X2 , , X p ]T is a random vector with X ~ (µ Definition 3.10 (Simple correlation coefficients): The correlation coefficient between any two random variables Xi and Xj is defined as: ρij =
cov( Xi , X j ) var( Xi ) ⋅ var( X j )
=
σ ij σ ii ⋅ σ jj
(3.12)
Clearly, ρij is the normalized covariance of Xi and Xj , for i, j = 1, 2, …, p. Properties of ρij : 1. ρij is dimensionless. 2. –1 ≤ ρij ≤ +1. 3. If ρij = –1 or +1, there is a line in the Xi and X j sample space of the form X j = a + bXi such that all the nonzero probability values for the bivariate distribution of Xi and X j lie on the line. Furthermore, if ρij > 0, the slope is greater than zero; if ρij < 0, the slope is less than zero. 4. If –1 < ρij < +1, the value of ρij is a relatively subjective measure of the tendency of Xi’s and Xj’s probabilities to concentrate about the same line. It can be seen that the simple correlation coefficient between any two random variables Xi and X j describes their joint behavior, whereas the partial correlation coefficient reflects the behavior of two random variables when other random variables are held fixed. For example, considering random variables Y , X1, X2 , , Xl , the partial correlation between Y and X1 reflects the behavior between them when the other random variables X2 , , Xl are held fixed, which is commonly denoted by ρX1Y•X2X3Xl or ρ X1Y |X2 X3Xl . Without loss of generality, let X1, X2 , , X k and X k +1, X k +2 , , X k +r be any two subsets of a collection of random variables X1, X2 , , X k , X k +1, , X p (2 ≤ k < p).
2151_book.fm Page 38 Friday, October 20, 2006 5:04 PM
38
Stream of Variation Modeling and Analysis for MMPs
Definition 3.11 (Partial correlation coefficients or conditional correlation coefficients): Suppose Xi and Xj are any two of the random variables X1, X2 , , X k , and σ ij•k +1,k +2,,k +r is the conditional covariance of Xi and Xj in the joint conditional distribution, the partial correlation coefficient of Xi and Xj given X k +1, X k +2 , , X k +r is defined as: ρ X i X j • X k +1 X k + 2 X k + r =
σ ij•k +1,k +2,,k +r σ ii•k +1,k +2,,k +r ⋅ σ jj•k +1,k +2,,k +r
(3.13)
The partial correlation coefficient can be calculated from simple correlation coefficients. Let Z be the collection of random variables X k +1, X k +2 , , X k +r ; then Equation 3.13 can be expressed in a more explicit but equivalent form: ρ Xi X j •Z =
σ Xi X j •Z σ Xi Xi •Z ⋅ σ X j X j •Z
ρ Xi X j − ρ XiZ ⋅ ρ X j Z
=
(1 − ρ2XiZ )(1 − ρ2X jZ )
(3.13)
Equation (3.13) also gives the calculation approach. The interpretation of a partial correlation coefficient is the same as that of a simple correlation coefficient except that the partial correlation coefficient generally depends on the conditional values of the conditional random variables. Graphically, the simple correlation coefficient describes the data in an ordinary scatter plot, whereas the partial correlation coefficient describes the data in the partial regression residual plot. For example, considering the collection of random variables Y, X1, X2, , Xl, suppose Y and X1 are of interest and the remaining variables are fixed values. The residuals of Y and X1 are calculated by regressing on X2 , , Xl , respectively. These residuals are parts of Y and X1 that cannot be predicted from X2 , , Xl . The partial correlation coefficient between Y and X1 after adjusting X2 , , Xl is the correlation between these two sets of residuals. Furthermore, the regression coefficient when Y’s residuals are regressed on X1’s residuals is equal to the regression coefficient of X1 in the multiple regression equation when Y is regressed on X1, X2 , , Xl . For more details, please refer to the regression section in Chapter 4. The simple correlation coefficient is a measure of the linear relationship between two random variables. The partial correlation coefficient is a measure of the linear relationship between two random variables when other random variables are held fixed, and the multiple correlation coefficients are a measure of the linear relationship between a random variable and a set of other random variables. Definition 3.12 (Multiple correlation coefficients): Without loss of generality, suppose X1 is a random variable and X2, X3, , Xp are a set of other random variables. Let p
Y=
∑a X i
i=2
i
2151_book.fm Page 39 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
39
be an arbitrary linear combination of X2, X3, , Xp. Then the multiple correlation coefficient of X1 and X2, X3, , Xp is the maximum simple correlation coefficient of X1 and Y for variation over ai, i = 2, 3, , p. For example, let ρ X1Y ( a2 , a3 , , a p ) =
cov( X1, Y ) var( X1 ) ⋅ var(Y )
(3.14)
Then the multiple correlation coefficient of X1 and X2 , X3 , , X p , denoted by ρ1•2 3p, is defined as the maximum of ρ X1Y ( a2 , a3 , , a p ) w.r.t. a2 , a3 , , a p , i.e.,
{
}
ρ1•23p = max ρX1Y (a2 , a3 , , a p ) ai ,2≤i ≤ p
It can be calculated according to the following procedure: 1. Let X = [ X1, X2 , , X p ]T ; partition ∑ in the following manner ∑ ∑ = var(X) = 11 T ∑12
∑12 ∑ 22
where Σ11 = var( X1 ) = σ11 ∈ R 1×1 , ∑12 = [cov( X1 , X 2 ) cov( X1, X p )] = [σ12 σ1 p ] and ∑ 22 = cov(X −1, X −1 ) , where X −1 = [ X2 , X3 , , X p ]T . 2. Based on the definition of ρ X1Y ( a2 , a3 , , a p ) , compute
ρ2 X1Y ( a2 , a3 , , a p ) =
(Σ
(
a
12 −1
)
2
σ11 a T−1Σ 22a −1
)
where a −1 = [a2 a p ]. 3. Calculate ρ21•23p by
{
}
ρ21•23p = max ρ2 X1Y ( a2 , a3 , , a p ) . ai ,2 ≤i≤ p
It can be shown that ρ21•23p =
(
)
1 T . ∑12 ∑ −221 ∑12 σ11
(3.15)
2151_book.fm Page 40 Friday, October 20, 2006 5:04 PM
40
Stream of Variation Modeling and Analysis for MMPs
4. Determine ρ1•23p by taking the square root of ρ21•23p , i.e., ρ1•23p = ± ρ21•23p . Remark 3.1: Let S S = 11 T S12
S12 S22
be the sample covariance matrix. Then the sample multiple correlation coefficient is: r 21•23 p =
(
)
1 T S12S−221S12 . S11
Remark 3.2: The multiple correlation coefficient can be used to test whether X1 is independent of X2 , X3 , , X p by the F-test with appropriate degrees of freedom. Remark 3.3: The multiple correlation coefficient describes the correlation between one dependent variable X1 and its best estimate from a regression on several independent variables X2 , X3 , , X p. The squared multiple correlation coefficient is the proportion of variance accounted for the multiple regression.
3.3 MULTIVARIATE NORMAL DISTRIBUTION AND QUADRATIC FORMS The basic and central distribution in multivariate analysis is the multivariate normal distribution. This is due to two main factors. First, multivariate observations are often normally distributed, or approximately normally distributed, due to the central limit theorem. This fact lays the foundation for formal inferences. Second, the multivariate normal distribution and the sampling distributions are tractable. This is not generally the case with other multivariate distributions, even for those that are close to normality.
3.3.1 MULTIVARIATE NORMAL DISTRIBUTION
AND ITS
PROPERTIES
Definition 3.13 (Multivariate normal distribution): Let X = [X1, X2, , Xp]T ∈R p µx, ∑ X ), where all the p random variables Xi are be a random vector with X ~ (µ continuous s.t. –∞ < Xi < ∞, and ∑ X is a positive definite matrix. Thus S = {( x1 , x 2 ,, x p )T | − ∞ < xi < ∞, i = 1, 2, ..., p} forms the sample space for X. Then random vector X is said to have a p-variate normal distribution if its multivariate probability distribution is given by: f (X) =
1 (2π )
p/ 2
∑X
1/ 2
1 exp − X − µ X 2
(
)
T
∑ X−1 X − µ X
(
Often, it is denoted as X ~ N p (µ X , ∑ X ), or simply N p (µ , ∑).
)
(3.16)
2151_book.fm Page 41 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
41
Properties of multivariate normal distribution: 1. If X follows a p-variate normal distribution or X ~ N p (µ X , ∑ X ), then both µX ≡ E(X) and ∑ X ≡ cov(X) exist and the distribution of X is uniquely determined by µ X and ∑ X. 2. If X ~ N p (µ , ∑), and A ∈ R q × p , b ∈R q , then Y = AX + b has a q-variate µ + b, A ∑ AT). normal distribution with Y ~ N q (Aµ 3. If X ~ N p (µ , ∑), then the marginal distribution of any subset of k(k < p) components of X is k-variate normal. 4. If X ~ N p (µ , ∑), for the partition: X µ ∑ X = 1 , µ = 1 , ∑ = 11 X2 µ 2 ∑ 21
∑12 , ∑ 22
where X1 ∈R k, µ1 ∈R k , and ∑11 ∈ R k × k, the subvectors X1 and X2 are independent iff ∑12 = 0, i.e., X1 ⊥⊥ X 2 ⇔ ∑12 = 0
(3.17)
Remark 3.4: Property 4 can be easily extended to the case that X is partitioned into a number of blocks with more than two subvectors. Remark 3.5: Property 4 shows that, in order to determine whether two subvectors of a normally distributed vector are independent, it suffices to check that covariance matrices between these two subvectors are zero. 5. Contours of N p (µ , ∑) are defined as:
(X − µ )
T
(
)
Σ −1 X − µ = C 2
(3.18)
where C is a position constant. So Equation 3.18 defines an ellipsoid with its center at µ and axes ±C λ i ei , where λ i and ei are the eigenvalues and associated eigenvectors of ∑ . So µ determines its position and ∑ defines its shape and orientation. 6. (Regression) Given X ~ N p (µ , ∑), and the partition: X µ Σ X = 1 , µ = 1 , and Σ = 11 X2 µ 2 Σ 21
Σ12 Σ 22
where X1 ∈R k , µ1 ∈R k , and Σ11 ∈ R k × k . Let ∑ −22 be the generalized inverse (1) − − of ∑ 22 , i.e., ∑ 22 = ∑ 22 ∑ 22 ∑ 22 , and define ∑11 = ∑11 − ∑12 ∑ 22 ∑ 21 . Then:
2151_book.fm Page 42 Friday, October 20, 2006 5:04 PM
42
Stream of Variation Modeling and Analysis for MMPs − − (1) a. X1 − ∑12 ∑ 22 X 2 ~ N k (µ1 − ∑12 ∑ 22 µ 2 , ∑11 ); furthermore, − X1 − ∑12 ∑ 22 X2 ⊥⊥ X 2
(3.19)
b. Conditional distribution of X1 given X 2 = x 2 is: − (1) X1 X 2 = x 2 ~ N k (µ1 + ∑12 ∑ 22 (x 2 − µ 2 ), ∑11 ) − Remark 3.6: E( X1 X 2 ) = µ1 + ∑12 ∑ 22 (X 2 − µ 2 )
(3.20) (3.21)
(1) − var( X1 X 2 ) = ∑11 = ∑11 − ∑12 ∑ 22 ∑ 21
(3.22)
Rewrite E( X1 X 2 ) as: E(X1 X 2 ) = (µ1 − ∑12 ∑ −22 µ 2 ) + ∑12 ∑ −22 X 2 = b + AX 2
(3.23)
− − µ 2 ), A = ∑12 ∑ 22 . So E( X1 X 2 ) is usually called the regreswhere b = (µ1 − ∑12 ∑ 22 sion function of X1 on X2 with the intercept vector b and the regression matrix A. It is a linear regression function because it depends linearly on the variable X2 being held constant, whereas var( X1 X 2 ) is constant without depending on X2.
7. For random vectors X and Y ∈R p , if X ⊥⊥ Y and X + Y is p-variate normally distributed, then both X and Y are p-variate normal. 8. Linear combination of random vectors: If X1, X2, …, XN ∈R p are all independent random vectors and Xi ~ N p (µ i , ∑i ) for i = 1, 2, …, N, then: a. For any fixed constant a T = [a1 aN ], N
∑ i =1
ai X i ~ N p
N
N
∑
aiµ i ,
i =1
∑ i =1
ai2 ∑ i
(3.24)
b. As a special case of Equation 3.24, suppose Xi iid ~ N p (µ , ∑) for i = 1, 2, , N and a1 = = aN = 1/N, then: 1 X ~ N p µ, ∑ , N where X=
1 N
N
∑X i =1
is called the sample mean vector.
i
(3.25)
2151_book.fm Page 43 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
43
9. Let X ~ N p (µ , ∑), where ∑ is nonsingular, so that there exists a nonsingular matrix D such that ∑ = DDT . Then the transformation Y = D–1 (X – µ) ~ N p (0, I p ).
3.3.2 QUADRATIC FORMS Quadratic forms in multivariate normal random variables arise very often and play an important role in multivariate analysis. Some important results of certain quadratic forms under various parametric conditions are given in this subsection. Theorem 3.2: Suppose X ~ N p (µ , ∑), where ∑ is nonsingular, then: 1. (X − µ )T ∑ −1 (X − µ ) ~ χ2p and Prob. {(X – µ)T ∑ −1 (X − µ ) ≤ χp2(α)} = 1 – α, (3.26) where χ2p (α ) is the upper α percentile of the χ2 distribution with p degree of freedom. 2. XT ∑ −1 X ~ χ2p ( ∆) (3.27) T −1 where ∆ = µ ∑ µ . Remark 3.7: This result is used in testing the hypothesis about the mean of a multivariate normal distribution when the covariance matrix is known, e.g., as a χ2 control chart in a quality control problem. Remark 3.8: From this theorem and Property 8, it is easy to show that: N (X − µ )T ∑ −1 (X − µ ) ~ χ2p
(3.28)
Remark 3.9: If X ~ N p (µ , ∑), then (X − µ )T ∑ −1 (X − µ ) is the squared statistical distance (squared Mahalanobis distance) from X to the population mean vector µ . Remark 3.10: If σ ii > σ jj , then Xi contributes less to the statistical distance than Xj; if σij > σst, then Xi and X j contribute less to the statistical distance than Xs and Xt . Theorem 3.3: Suppose X ~ N p (µ , ∑), where ∑ is nonsingular. Then for the partition: X µ ∑ X = 1 , µ = 1 , and ∑ = 11 X2 µ 2 ∑ 21
∑12 , ∑ 22
where X1 ∈R k , , µ1 ∈R k , and ∑11 ∈ R k × k , the following result holds:
(
)
−1 (X − µ )T ∑ −1 (X − µ ) − (X1 − µ1 )T ∑11 X1 − µ1 ~ χ2p − k
(3.29)
Theorem 3.4 Suppose X ~ N p (0, ∑), where rank (∑) = r (r ≤ p), and ∑ − is the generalized inverse of Σ, i.e., ΣΣ − Σ = Σ , then XT ∑ − X ~ χ2r
(3.30)
2151_book.fm Page 44 Friday, October 20, 2006 5:04 PM
44
Stream of Variation Modeling and Analysis for MMPs
3.3.3 NONCENTRAL χ2
AND
F DISTRIBUTIONS
The noncentral χ2 distribution can be expressed as a mixture of central χ2 distributions where the weights are Poisson probabilities. Definition 3.14 (Noncentral χ2 distribution): If X ~ N p (µ , I p ), then the random variable Z = X T X has the CDF:
∞
F ( z; p, δ) =
∑ i=0
1 i δ δ 2 e − 2 Pr(χ2 ≤ z) p +2 i i!
(3.31)
where δ = µ T µ. Z is said to have the noncentral χ2 distribution with degree of freedom (d.o.f.) p and noncentrality parameter δ denoted by χ2p (δ) . Corollary: If Z ~ χ2p (δ) , then its PDF can be expressed as: ∞
∑ P (K = k )g
( z) , z > 0
(3.32)
1 e − z / 2 z r / 2 −1 Γ ( 21 r )
(3.33)
p +2 k
k =0
where K ~ Poisson (mean = δ/2), and gr ( z) =
2
r /2
is the density function of the χ2r distribution. Property 1: If Z ~ χ2p (δ) , then E ( Z ) = p + δ , var( Z ) = 2 p + 4δ . Property 2: If Z1 ~ χ2p1 (δ1 ) and Z 2 ~ χ2p2 (δ 2 ) , then Z1 + Z 2 ~ χ2p1+ p2 (δ1 + δ 2 ) . Recall that the central F distribution is obtained by taking the ratio of two independent χ2 variables divided by their degrees of freedom. The noncentral F distribution is obtained by allowing the numerator variable to be noncentral χ2. Definition 3.15 (Noncentral F distribution): If Z1 ~ χ2p1 (δ) , Z 2 ~ χ2p2 , and Z1 is independent of Z2, then F=
Z1 /p1 Z 2 /p2
(3.34)
is said to have the noncentral F distribution with d.o.f. p1 and p2, and noncentrality parameter δ, denoted by Fp1 , p2 (δ).
2151_book.fm Page 45 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
45
Property: If F ~ Fp1, p2 (δ) , then E (F ) =
p2 ( p1 + δ) , p2 > 2 p1 ( p2 − 2 )
(3.35)
2
p ( p + δ)2 + ( p1 + 2 δ)( p2 − 2 ) var( F ) = 2 2 1 , p2 > 4 p1 ( p2 − 2 )2 ( p2 − 4)
(3.36)
Remark 3.11: The noncentral χ2 distribution can be calculated by the Matlab® function NCX2CDF(x,df,delta) or SAS function PROBCHI(x,df,delta). Both return the noncentral chi-square probability with df degrees of freedom and noncentrality parameter, delta, at the values in x. Remark 3.12: The noncentral F distribution can be calculated by the Matlab function NCFCDF(x,ndf,ddf,delta) or SAS function PROBF(x,ndf,ddf,delta). Both return the noncentral F CDF with numerator degree of freedom, ndf; denominator degree of freedom, ddf; and noncentrality parameter, delta, at the values in x. For more details about these functions please refer to the related Matlab [11] and SAS manual [12].
3.4 SAMPLING THEORY 3.4.1 SAMPLE GEOMETRY Assume X1, X2, …, Xn ∈R p are n random vectors and x1, x 2 ,..., x n are associated realizations. Let x be an n × p matrix: x1T T x x= 2= T x n
x11 x21 x n1
x12
x22
xn 2
x1 p x2 p x np
(3.37) n× p
Definition 3.16 (Observation matrix, sample matrix): An observation matrix x with n multivariate observations (or sample size n) is defined as shown in Equation 3.37; each row is called a single multivariate observation and is the collection of measurements of p different variables taken at the same trial. Remark 3.13: Matrix x is also referred to as a sample of size n from a p-variate population. From Definition 3.16, each row of an observation matrix is a realization of p-variate random vectors.
2151_book.fm Page 46 Friday, October 20, 2006 5:04 PM
46
Stream of Variation Modeling and Analysis for MMPs
Remark 3.14: In observation matrix x, all the rows are independent of each other. For simplicity, we introduce the 1-vector: T
1n = 1 1 1 ∈ R n
(3.38)
We define the sample mean vector x, sample covariance matrix s n , and sample correlation matrix as follows: Sample mean vector: 1 x= n
n
∑ x = n1 x 1 T
(3.39)
i
i =1
Sample covariance matrix: 1 sn = n −1
n
∑ (x − x)(x − x) i
T
i
i =1
(3.40)
(
1 x − 1x T = n −1
) ( x − 1x ) T
T
1 T 1 = x I − 11T x n −1 n
Sample correlation matrix: s ij R n = rij = sii s jj
( )
(3.41)
where sij is the (i, j) entry of sn. To simplify the notation, let n
A=
∑ (x − x)(x − x) i
i
T
(3.42)
i =1
So Equation 3.40 becomes
sn =
1 A n −1
(3.43)
1 Remark 3.15: sn is a symmetric matrix with p variances and p ( p − 1) potential 2 different covariances.
2151_book.fm Page 47 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
47
Remark 3.16: sn can be denoted by
( )
s n = sij , i, j = 1, 2, …, p
(3.44)
where sij =
1 n −1
n
∑ (x
ki
− xi )( x kj − x j ) .
k =1
The observation matrix defined in Equation 3.37 can be geometrically interpreted in the following ways: 1. Express x1T T x x = 2 = [x1 x 2 x n ]T . T x n According to this expression, the total n data points, xi = [xi1 xi2 xip]T (i = 1, 2, …, n) can be plotted in R p space. This scatter plot contains rich information about the locations and variability of these points. 2. Express x as x = [x 1 x 2 x p ], where x1 j x2 j x j = , j = 1, 2, …, p x nj Here, there are a total of p vectors in R n space. Each x j can be decomposed into two mutually perpendicular parts, x1 j − x j x2 j − x j d j = x j − x j 1 n = and x j 1 n , x nj − x j which is illustrated in Figure 3.1.
2151_book.fm Page 48 Friday, October 20, 2006 5:04 PM
48
Stream of Variation Modeling and Analysis for MMPs
~ xj
~ xj
x j 1n
x j 1n FIGURE 3.1 Illustration of the decomposition of each column vector x~ j of data matrix x = [x~ 1 x~ 2 x~ p], where x–j is the mean value of the jth column vector x~ j, i.e., xj =
1 n
∑x n
ij
=
i =1
1 n
T x j ⋅ 1n.
In the decomposition x j = x j 1 n + d j , j = 1, 2, …, p
(3.45)
each x j 1 n is called a mean vector and d j is called the associated residual vector. The residual matrix is defined by:
d1
x1T T x d p = 2 T x n
d2
− xT − xT = x − 1n x T − x T
The length of each residual vector d j is given by: n
ld j = d j = d Tj d j =
∑ (x
kj
− x j )2 = ( n − 1)s jj
(3.46)
k =1
For any two residual vectors d i and d j , the inner product (d i , d j ) = d Ti d j n
=
∑
( x ki − xi )( x kj − x j ) = (n − 1)sij
(3.47)
k =1
= l di l d j cos(Q ij )
(3.48)
2151_book.fm Page 49 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
49
where θij is the angle between two residual vectors d i and d j . If we substitute Equation 3.48 with Equation 3.46, then: n
(d i , d j ) =
∑ (x
ki
− xi )( x kj − x j )
k =1
n
=
∑
n
( x ki − xi )2
∑ (x
kj
− x j )2 cos(θij )
k =1
k =1
This implies that: cos(θij ) =
sij sii s jj
= rij
(3.49)
i.e., cos(θij) is the sample correlation coefficient. Note the following: 1. If di is nearly parallel to dj with the same direction, i.e., θij ≈ 0°, then ρij ≈ 1; if di is nearly parallel to dj but with opposite direction, i.e., θij ≈ 180°, then ρij ≈ –1. 2. If di and dj are mutually perpendicular, then ρij = 0.
3.4.2 RANDOM SAMPLES AND THE EXPECTED VALUES OF THE SAMPLE MEAN AND COVARIANCE MATRIX The definition data matrix x is clearly constructed by the observations or realizations of a p-variate random vector. But before taking the actual measurements, we cannot predict these values exactly. So each set of measurements X i on p variables is a random vector and the matrix X =[X1 X2 Xn]T is actually a random matrix. Definition 3.17 (Random sample): A p-variate Xi, for i = 1, 2, …, n, is said to be a random sample taken from a common joint distribution with density function f ( x ) = f ( x1, x2 , , x p ) if the row vectors in X =[X1 X2 Xn]T, i.e., XTi, i = 1, 2, …, n, are independent observations from f (x), i.e., the joint PDF of X1 X2 Xn is given by n
∏ f (x ) , i
i =1
where f ( x i ) = f ( xi1, xi 2 , , xip ) is the PDF for the ith row vector.
2151_book.fm Page 50 Friday, October 20, 2006 5:04 PM
50
Stream of Variation Modeling and Analysis for MMPs
As the sample mean vector X is also a random vector, Theorem 3.5 gives its expected value and covariance matrix. Before introducing the theorem, we need the following lemma. Lemma 3.1: For any random vector Y with mean vector E( Y) = µ and covariance matrix cov( Y) = ∑ . The following relationship holds: E ( YY T ) = ∑ +µµ T
(3.50)
Theorem 3.5: Let X1 X2 Xn ∈R p be a random sample taken from a joint µ, ∑). Then the sample mean vector X is an unbiased estimator distribution with (µ of µ and its covariance matrix is n1 ∑, i.e, E(X) = µ
cov(X) =
(3.51)
1 ∑ n
(3.52)
The sample variance-covariance matrix s n defined in Equation 3.40 is an unbiased estimator for ∑, i.e., E (s n ) = ∑
3.4.3 SOME IMPORTANT RESULTS OF SAMPLING MULTIVARIATE NORMAL DISTRIBUTIONS
(3.53) FROM
For multivariate normal distributed random vectors, there exist the following fundamental results: Theorem 3.6 (Distributions of sample means and covariance matrices): Suppose X1, X2, , Xn are independently identically distributed (i.i.d.) random samples drawn from N p (µ , ∑), then: 1. Sample mean vector X =
1 n
n
∑ X ~ N (µ, n1 ∑). So i
p
i =1
n (X − µ ) ~ N p (0, ∑).
(3.54)
2. (n – 1)sn is distributed as a Wishart random matrix with d.o.f. = n – 1. – 3. X and sn are independent where
sn =
1 n −1
n
∑ (X − X)(X − X) . T
i
i =1
i
2151_book.fm Page 51 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
51
Theorem 3.7 (Multivariate central limit theorem, Cramer): Let X1, X2, , Xn be i.i.d random vectors with mean µ and positive definite ∑, and let 1 X= n
n
∑X . i
i =1
Then, (1) n (X − µ ) → N p (0, ∑), as n → ∞. (2) n(X − µ )T s −n1 (X − µ ) → χ 2p , for n – p is large.
(3.55)
3.5 THE WISHART DISTRIBUTION AND SOME PROPERTIES In many cases, observations are collected on k independent groups of sampling units. The responses are described by a multinormal random variable with mean vector µh being the hth group and a covariance matrix common to all groups. So the observation matrix for the hth group is: x1Th x11h T x x21h x h = 2 h = T x nh h x nh 1h
x12 h
x22 h
x nh 2 h
x1 ph x2 ph x nh ph
(3.56)
The maximum likelihood estimation (MLE) of µh is the sample mean vector x h of the hth group, and the unbiased estimate of ∑ is given by: 1 s= n−k
k
nh
∑∑ h =1 i =1
1 ( x ih − x h )( x ih − x h ) = n−k T
k
∑A
h
h =1
where k is the total number of groups, and nh
Ah =
∑ (x
ih
− x h )( x ih − x h )T
i =1
is the sums of squares and cross products within the hth group and k
n=
∑n . h
h =1
(3.57)
2151_book.fm Page 52 Friday, October 20, 2006 5:04 PM
52
Stream of Variation Modeling and Analysis for MMPs
Remark 3.17: From the definition of A and Ah, they can be expressed as the sums of products of n – 1 and n – k independent p-variate random vectors with the common distribution N p (0,∑ ∑). In general, any symmetric positive definite matrix A ∈ R p × p of quadratic and bilinear forms which can be transformed to the sum: n
∑YY , T i
i
(3.58)
i=1
with p-variate vectors Yi independently distributed according to the distribution N p (0, ∑) is said to have the Wishart distribution [13]. The density function of a positive definite A with d.o.f. = n is given by: ( n − p−1)/ 2
A
Wn ( A | ∑) = 2
/2 np
π
(
exp − 12 tr ( A ∑ −1 )
p ( p−1)/ 4
∑
n / 2
)
p
∑ Γ(n + 1 − i)
(3.59)
i =1
where Γ(⋅) is the gamma function. If A is not positive definite, then Wn ( A | ∑) = 0. In Equation 3.59, n = n − 1 for a single-sample matrix, n = n − k for the k sample within-groups matrix. The d.o.f. n and ∑ specifies the form of the density function. Remark 3.18: If p = 1 and ∑ =1, then the Wishart density reduces to χ2n . Wishart-distributed matrices have many similar properties of χ2 variates. Some properties are summarized as follows: Property 1: If A1, A 2 , , A k are independent Wishart-distributed matrices with the common parameter matrix ∑, i.e., A i ~ Wni ( Ai | ∑) for i =1, 2, ..., k, then k
∑ i =1
A i ~ Wn
k
∑ i =1
Ai | ∑ , where n =
k
∑n . i
i =1
Property 2: If A ~ Wn ( A | ∑), then CAC T ~ Wn (CACT C ∑ CT ), where C is a constant matrix with proper orders. Property 3: The sample mean vector x and the matrix of sums of squares and cross products n
A=
∑ (x − x)(x − x) i
T
i
i =1
computed from the same sample are independently distributed.
2151_book.fm Page 53 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
53
3.6 EXERCISES 1. A transportation company gives the list of ticket prices in dollars (x1) for different travel distances in miles (x2) as follows: x1
2
6
6
8
8
10
12
15
17
20
x2
4
7
10
15
20
25
35
45
55
70
a. Construct a scatter plot of the data. b. Calculate the mean of the ticket price x1 and travel distances x2 . c. By looking at the scatter plot, what will the sign of the covariance s12 be? d. Calculate the variance s11 and s22. e. Calculate the covariance s12 and the correlation coefficient r12. Explain the physical meaning of these values. 2. Suppose two variables x1 and x2 are measured and shown to be the following: x1 x2
2 4
5 6
3 5
9 13
6 7
1 3
8 11
4 5
3 4
7 9
a. Construct a scatter plot. x 2T , where b. Express x as xT = x 1T x1 j x x j = 2 j , j = 1, 2. x10 j c. Determine d j = x j − x j 1 n for j = 1, 2, where n = 10 in this case. dj is called the associated residual vector. d. Find the lengths of each residual vector dj for j = 1, 2. e. Find the angle between two residual vectors d1 and d1. f. Verify that cos(θ12) = r12. 3. The measurements of two quality characteristics are as follows: Time
1
2
3
4
5
6
7
8
9
10
x1 x2
4.5 3
2 1.5
2.5 2
4 3
3.5 2.5
4.5 4
3 2
4 3.5
5 4
3 2.5
a. Find the correlation coefficient r between x1 and x2. b. Find the covariance matrix S for x1 and x2, determine the eigenvalues λ1 and λ2, and the eigenvectors {u1, u2} of S.
2151_book.fm Page 54 Friday, October 20, 2006 5:04 PM
54
Stream of Variation Modeling and Analysis for MMPs
c. Use the preceding eigenvectors to form the transformation matrix Q. Then the variables x1 and x2 can be transformed into independent variables z1, and z2, by using z1 T x1 = Q . z 2 x2 Calculate the corresponding values of z1 for all time steps. 4. Suppose 3 X = 9 3
5 7 6
4 2 , aT = [3 –1 2], bT = [2 4 –2], and cT = [–3 –1 1]. 3
There are p = 3 variables X1, X2, and X3. Each variable has n = 3 observations. a. Find the sample means of aT X, bT X, and cT X. b. Find the variances of aT X, bT X, and cT X. c. Calculate the covariances of aT X and bT X, bT X and cT X, and aT X and cT X. 5. Suppose the random vector X T = X1, X2 , X3 , X 4 has µ Tx = [5,1,4,6] and covariance matrix 1 2 ∑X = 3 −4
2 3 0 2
−4 2 0 8
3 0 7 0
X is partitioned as X1 X1 X = X2 = X3 X 2 X4 Suppose 2 A = and B = 1 1
2
−2
2151_book.fm Page 55 Friday, October 20, 2006 5:04 PM
Basics of Multivariate Statistical Analysis
55
Determine the following: a. E(X1) and E(X2). b. E(AX1) and E(BX2). c. cov(X1) and cov(X2). d. cov(AX1) and cov(BX2). e. cov(X2, X1). f. cov(BX2, AX1). 6. Suppose x1 and x2 are two random measurements from a process and each follows a Gaussian distribution x1 ~ N(2,1) and x2 ~ N(5,4). Suppose also that x1 and x2 are not independent and have covariance cov(x1,x2) = 1.5. Calculate: a. The mean vector and covariance matrix of the 2-D random vector x ≡ [x1 x2]T. b. The joint probability density function of x and marginal probability density functions of x1 and x2, respectively. c. Draw the equidensity contour of x (i.e., the curves over which pdf(x) assumes a constant value) and pdfs of x1 and x2 on the same figure. d. If cov(x1,x2) = 0, repeat (a) to (c).
References 1. Kshirsagar, A.M., Multivariate Analysis, Marcel Dekker, New York, 1972. 2. Mardia, K.V. et al., Multivariate Analysis, Academic Press, London, 1979. 3. Kleinbaum, D.G. et al., Applied Regression Analysis and Other Multivariable Methods, 2nd ed., PWS-Kent Publishing Company, Boston, MA, 1987. 4. Hair, J.F. et al., Multivariate Data Analysis, 3rd ed., Macmillan Publishing Company, New York, 1992. 5. Muirhead, R.J., Aspects of Multivariate Statistical Theory, John Wiley & Sons, New York, 1982. 6. Karson, M.J., Multivariate Statistical Methods, The Iowa State University Press, Ames, IA, 1982. 7. Morrison, D.F., Multivariate Statistical Methods, 2nd ed., McGraw-Hill, New York, 1976. 8. Takeuchi, K. et al., The Foundations of Multivariate Analysis, Wiley Eastern Limited, New Delhi, India, 1982. 9. Srivastava, M.S., Methods of Multivariate Statistics, John Wiley & Sons, New York, 2002. 10. Manly, B.F.J., Multivariate Statistical Methods: A Primer, Chapman and Hall, London, 1986. 11. The MathWorks, Getting Started with Matlab Version 7, The MathWorks, Inc., Natick, MA, 2006. 12. SAS, SAS 9.1 Documentation, http://support.sas.com/documentation/index.html, SAS Institute, Inc., Cary, NC, 2006. 13. Wishart, J., Biometrika, A., The Generalized Product Moment Distribution in Samples from a Normal Multivariate Population, 20(1928), pp. 32–52.
2151_book.fm Page 56 Friday, October 20, 2006 5:04 PM
2151_book.fm Page 57 Friday, October 20, 2006 5:04 PM
4
Statistical Inferences on Mean Vectors and Linear Models
4.1 STATISTICAL INFERENCES ON MEAN VECTORS In this section, we will discuss systematic approaches to making inferences about mean vectors based on the sampling information from a multivariate distribution. The necessary assumption is that the data should follow a multivariate normal distribution.
4.1.1 HOTELLING’S T 2 TEST Let Xi, i = 1, 2, …, n, be a sequence of independently identically distributed (i.i.d) p-dimensional random vectors with N p (µ , ∑), and xi, i = 1, 2, …, n, be their associated realizations. For a multivariate hypothesis testing problem, we are required to test whether a vector µ0 ∈R p is a true mean vector for the multivariate normal distribution. The squared distance in a multivariate case is defined as: −1
1 T 2 = (X − µ 0 )T S (X − µ 0 ) = n(X − µ 0 )T S−1 (X − µ 0 ), n
(4.1)
where µ0 = [µ10 µ20 … µp0]T ∈Rp and X and S are the sample mean vector and sample covariance matrix, respectively, which are defined as: X=
1 n
n
∑
Xi ∈ R p , S =
i =1
1 n −1
n
∑ (X − X)(X − X) i
i
T
∈ R p× p
i =1
1
Remark 4.1: In Equation 4.1, n S is the estimate of cov( X). Remark 4.2: The statistic T 2 defined in Equation 4.1 is called Hotelling’s T 2 in honor of Harold Hotelling, and is distributed as ( n − 1) p Fp , n − p , n− p when µ = µ 0 holds. Its sampling distribution is as follows: 57
2151_book.fm Page 58 Friday, October 20, 2006 5:04 PM
58
Stream of Variation Modeling and Analysis for MMPs
For a random sample xi, i = 1,2, …, n, which is drawn from N p (µ , ∑) population, no matter what the true µ and ∑ are, (n − 1) p Pr T 2 > Fp,n− p (α) = n − p (n − 1) p Pr n( x − µ )T S−1 ( x − µ ) > Fp,n− p (α) = α n− p
(4.2)
where Fp,n − p (α) is the upper (100α)th percentile of the F-distribution with degrees of freedom (d.o.f.) p and n – p. Remark 4.3 (Property of the T 2 statistic): It is easy to verify that the T 2 statistic is invariant for different scales of observations, i.e., for the nonsingular transformation Y = CX + d T 2 is identical for both X and Y, where C and d are a constant matrix and a vector, respectively, with proper dimensions. For the multivariate hypothesis testing problem, we have the following result. Theorem 4.1: For the multivariate hypothesis testing problem with estimated covariance matrix S: H 0 : µ = µ 0 , H1: µ ≠ µ 0
(4.3)
H0 is rejected if T 2 = n( x − µ 0 )T S−1 ( x − µ 0 ) >
(n − 1) p Fp,n− p (α) n− p
(4.4)
Remark 4.4: The preceding Hotelling’s T 2 test is based on the assumption that the covariance matrix ∑ is unknown. But if ∑ is known, the statistic becomes: T 2 = n(X − µ 0 )T ∑ −1 (X − µ 0 ),
(4.5)
where T 2 follows chi-square distribution with d.o.f. p under H0. This is also true for S ≈ ∑, if there are a large number of samples. Remark 4.5: As p univariate tests do not account for the dependency between these variables in multivariate measurement, a multivariate hypothesis should not be tested using p separate univariate tests, because it can lead to wrong conclusions.
2151_book.fm Page 59 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
59
Remark 4.6: It can be proved [1] that: n
T2 =
ˆ (n − 1) ∑ 0 ˆ ∑
∑ (X − µ )(X − µ ) 0
i
− (n − 1) = (n − 1)
i
i =1
n
∑ (X − x)(X − x) i
T
0
− (n − 1).
(4.6)
T
i
i =1
Equation 4.6 facilitates the calculation of T 2 by avoiding the calculation of S–1.
4.1.2 CONFIDENCE REGIONS AND SIMULTANEOUS CONFIDENCE INTERVALS 4.1.2.1 Confidence Regions Definition 4.1 (Confidence region): Let θ ∈Θ, where Θ is the parameter space. Given the data matrix x = [x1 x2 xn]T and the significance level α, before the sample is selected, the 100 (1 − α )% confidence region CR(x) is defined by 1− α = Pr(θ ∈CR(x)) = Pr(CR(x) will contain the trrue θ)
(4.7)
which is calculated under the unknown true parameter θ. For example, before the sample is selected, from Equation 4.2 and Equation 4.7, we have
{
}
1 − α = Pr µ ∈CR(x)
(n − 1) p = Pr n( x − µ )T S−1 ( x − µ ) ≤ Fp,n− p (α) − n p
(4.8)
regardless of the values of unknown µ and ∑. Therefore, for the p-dimensional normal cases, CR(x) can be determined by the following theorem. Theorem 4.2 (Confidence region): Given sample data xi, i = 1, 2, …, n, a 100(1 – α)% confidence region for the mean of a p-dimensional normal distribution is the space within the ellipsoid centered at x , i.e., CR(x): n( x − µ )T S−1 ( x − µ ) ≤
(n − 1) p Fp,n− p (α) = Tα2 n− p
where 1 x= n
n
∑ i =1
1 xi ∈ R , S = n −1 p
n
∑ (x − x)(x − x) i
i =1
i
T
∈ R p× p .
(4.9)
2151_book.fm Page 60 Friday, October 20, 2006 5:04 PM
60
Stream of Variation Modeling and Analysis for MMPs
~ Remark 4.7: To determine whether µ ∈CR(x), we need to compare the values of T −1 n( x − µ ) S ( x − µ ) and
Tα2 =
( n − 1) p Fp,n − p (α) . n− p
If n( x − µ )T S−1 ( x − µ ) ≤ Tα2 , then µ ∈CR(x); otherwise, µ ∉CR(x). Remark 4.8: The application of the confidence region CR(x) defined in Equation 4.9 is equivalent to testing H 0 : µ = µ 0 , H1 : µ ≠ µ 0 because CR(x) contains all possible µ 0 vectors for which the T 2-test fails to reject H0 at significance level α. Remark 4.9: The joint confidence region for µ cannot be graphed when the dimension p ≥ 4. It can be determined by the directions and lengths of the axes of the ellipsoid defined in Equation 4.9 under the scaling factor Tα n
λ i , i = 1, 2, …, p,
along the eigenvectors ei , where λ i and ei are eigenvalues and associated eigenvectors of S. The procedure for determining the ellipsoid involves the following steps: 1. Determine x , the center of ellipsoid; 2. Determine the axes of ellipsoid:
± λi
p(n − 1) Fp,n− p (α ) ei , i = 1, 2, …, p n(n − p)
(4.10)
where λi’s and ei’s are calculated by solving S ei = λ i ei From Equation 4.10, the half-length of axes are
λi
p ( n − 1) Fp,n − p (α ) for i = 1, 2, …, p. n(n − p)
(4.11)
2151_book.fm Page 61 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
61
So the elongation of the confidence ellipsoid is given by the ratio of the lengths of the axes, i.e.,
2 λi 2 λj
p ( n − 1) Fp,n − p (α ) n(n − p) p ( n − 1) Fp,n − p (α) n(n − p)
=
λi λj
, i≠ j
(4.12)
4.1.2.2 Simultaneous Confidence Intervals Based on the matrix theory presented in Chapter 2, it can be shown that
max a≠0
[a T ( x − µ )]2 = ( x − µ )T S−1 ( x − µ ) a T Sa
(4.13)
Thus, Equation 4.8 is equivalent to:
{
{
}
1 − α = Pr µ ∈CR(x) = Pr n( x − µ )T S−1( x − µ ) ≤ Tα2
}
[a T (x − µ )]2 = Pr n ≤ Ta2 , ∀a ∈ R p , a ≠ 0 T a Sa
(4.14)
where Tα2 =
( n − 1) p Fp,n − p (α) . n− p
Hence, Equation 4.14 becomes a T x − Tα
a T Sa n
≤ a T µ ≤ a T x + Tα
a T Sa n
, ∀a ∈ R p and a ≠ 0
(4.15)
This region consists of all plausible values of µ at the significance level α. In summary, we have the following theorem: Theorem 4.3 (Simultaneous intervals, or T 2-intervals): Suppose xi, i = 1, 2, …, n, is a random sample drawn from N p (µ , ∑) population with ∑ positive definite. Then simultaneously, for all a, the interval: a T x − Tα
a T Sa n
, a T x + Tα
a T Sa n
(4.16)
2151_book.fm Page 62 Friday, October 20, 2006 5:04 PM
62
Stream of Variation Modeling and Analysis for MMPs
will contain aT µ with probability 1 – α, where Tα2 =
( n − 1) p Fp,n − p (α). n− p
The confidence interval in Equation 4.16 is called Roy’s confidence interval. This theorem states that the confidence coefficient 1–α remains unchanged for any a ∈R p. Some special cases are discussed in the following remarks: Remark 4.10: Construct 100(1–α)% simultaneous confidence intervals for µi , i = 1, 2, …, p. If we select aT in the following forms: 1
0
0 , 0
1
0 , …, 0
0
1 .
Then the T 2-intervals become: xi − Tα
Sii n
, xi + Tα
Sii , for i = 1, 2, ..., p n
(4.17)
All intervals hold simultaneously with confidence coefficient 1–α. Remark 4.11: Construct 100(1–α)% confidence intervals for differences µ i − µ j , i ≠ j, i, j = 1, 2, …, p. If we select aT = [0 0 ai 0 aj 0 0], where ai = 1 and aj = –1, then T a Sa = sii − 2 sij + s jj . So the T 2-intervals become: ( xi − x j ) − Tα
sii − 2sij + s jj n
, ( xi − x j ) + Tα
sii − 2sij + s jj . n
(4.18)
It holds with confidence 1–α. Remark 4.12: Construct 100(1–α)% confidence ellipses for pairs of means. (µ i , µ j ) is within the sample mean-centered ellipses
n xi − µ i
sii x j − µ j sij
sij s jj
−1
xi − µ i (n − 1) p Fp,n− p (α) = Tα2 (4.19) ≤ n− p x j − µ j
It holds with confidence 1–α. Remark 4.13: If we want to obtain 100(1–α)% simultaneous confidence intervals for aiTµ, i = 1, 2, …, k, and if k is small, then Roy’s confidence intervals defined by
2151_book.fm Page 63 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
63
Equation 4.16 will be too wide. But the Bonferroni method will give narrower and more precise intervals if k (the number of linear combinations of interest) is small. More discussions on the Bonferroni confidence interval can be found in Reference 1.
4.1.3 INFERENCE ON A POPULATION MEAN VECTOR WITH LARGE SAMPLE SIZE So far, the hypothesis testing and confidence regions for µ have been constructed based on the assumption of a normal population. But when the sample size n is large, this normality assumption can be ignored even though there is a serious departure from normality. All large-sample inferences on µ are based on a chi-square distribution. Because n(X − µ )T S−1 (X − µ ) is approximately χ2p with d.o.f. p, when the sample size n is large enough,
{
}
Pr n( x − µ)Τ S−1 ( x − µ ) ≤ χ 2p (α ) ≈ 1 − α ,
(4.20)
where χ2p (α ) is the upper (100α)th percentile of the χ2p distribution. From Equation 4.20, we can obtain the hypothesis test and simultaneous confidence regions in the case of the large sample size. Theorem 4.4: Suppose xi, i = 1, 2, …, n, is a random sample drawn from a population with mean vector µ and positive definite covariance matrix ∑, and assume n − p is large. Then we have the following: 1. Hypothesis testing: Hypothesis H 0 : µ = µ 0 H1 : µ ≠ µ 0 will reject H 0 at a level of significance approximately α, if n( x − µ 0 )T S−1 ( x − µ 0 ) > χ 2p (α).
(4.21)
where χ2p (α ) is the upper (100α)th percentile of the χ2p distribution. 2. Simultaneous confidence regions: For any a ∈R p, the confidence intervals a T Sa T a T Sa a T x − χ 2p (α )· , a x + χ 2p (α )· n n will contain aT µ with probability approximately 1–α.
(4.22)
2151_book.fm Page 64 Friday, October 20, 2006 5:04 PM
64
Stream of Variation Modeling and Analysis for MMPs
Remark 4.14: As with the remarks in Theorem 4.3, the 100(1–α)% simultaneous confidence intervals for µi are: s s xi − χ2p (α ) ⋅ ii , xi + χ2p (α ) ⋅ ii , for i = 1, 2, ..., p n n
(4.23)
Remark 4.15: 100(1–α)% confidence ellipses for pairs of means (µ i , µ j ) can be constructed by: −1 sii sij xi − µ i 2 ≤ χ p (α ) . (µ i , µ j ) : n xi − µ i x j − µ j s s x j − µ j ij jj
(4.24)
4.1.4 MULTIVARIATE QUALITY CONTROL CHARTS Quality is of great importance in any industry. To improve the quality and performance of products and services, the process should be stable and no special causes leading to variation. A quality control chart can be effectively employed to continuously monitor the process and help identify special causes of variation. A typical control chart is composed of data (e.g., individual observations, or sample means) plotted in time order and control limits that indicate the quantity of variation due to common causes. A univariate control chart is used to monitor the individual characteristics in a process. But when more than one important feature occurs, it cannot monitor such processes due to the correlation between these features. Hence, it is necessary to develop multivariate control charts to monitor process stability. The multivariate control chart has two main tasks. One is to monitor the stability of a process with multivariate observations; the other is to set a control region for future observations. T 2 control charts will be introduced in this section. 4.1.4.1 Control Charts for One Multivariate Sample Two useful control charts are described in the following text: 1. T 2 control chart to monitor stability Suppose xi, i = 1, 2, …, n, is a random sample drawn from a normal µ,∑). After simple calculation, population Np (µ
xi − x = xi −
1 n
n
∑ i =1
1 1 x i = 1 − xi − n n
n
∑x. j
j =1, j ≠ i
2151_book.fm Page 65 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
65
It is easy to verify that 2
2
1 (n − 1) 1 E (x i − x ) = 0 and cov(x i − x ) = 1 − ∑ + ⋅ (n − 1) ∑ = ∑. n n n Thus, x i − x is also a multivariate normal distribution with zero mean vector, but not independent of the sample covariance matrix S. A chisquare distribution can be used to set the control limits. A T 2 control chart can be obtained through the following three steps: Step 1: Construct control limits Set up the 100 (1 − α )% control limit: UCL = χ 2p (α ) , LCL = 0
(4.25)
where p is the total number of monitored features. Step 2: Determine the points to be plotted Calculate and plot the T 2 statistics for the ith point: Ti 2 = (x i − x )T S−1 (x i − x ).
(4.26)
Step 3: Identify special causes If some points are out of control limits, we need to identify which variables are responsible for these points. This can be done by the Bonferroni intervals. For example, the jth variable of the ith sample is out of control if
(
xij ∉ x j − tn−1 (α / 2 p) s jj , x j + tn−1 (α / 2 p) s jj
)
(4.27)
2. T 2 control chart for future individual measurements If a T 2 control chart indicates that the process is stable, we can employ the collected data to construct another control chart, that can be used to predict future measurements. For the T 2 control chart, the control limits are set: (n − 1) p UCL = n − p Fp,n− p (α) , LCL = 0
(4.28)
2151_book.fm Page 66 Friday, October 20, 2006 5:04 PM
66
Stream of Variation Modeling and Analysis for MMPs
and the following T 2 calculated from the new measurement x is then plotted: T2 =
n ( x − x )T S −1 ( x − x ) n +1
(4.29)
where x=
n
∑ x ∈R
1 n
p
i
i =1
and S=
1 n −1
n
∑ (x − x)(x − x) i
i
T
∈ R p× p
i =1
are the sample mean vector and covariance matrix, respectively. If the T 2 from the future measurement x exceeds the upper control limit, we can conclude that it is an out-of-control signal and further action may be taken to identify the special causes. 4.1.4.2 Control Charts Based on Subgroup Means Two useful control charts are described in the following text: 1. Control ellipse and T 2 chart for monitoring stability A multivariate control chart can also be constructed based on subgroup means. The basic assumption is that each multivariate sample is drawn from N p (µ , ∑). The subgroup size is m > 1. Suppose the subgroup mean vectors and covariance matrices are xi , Si , i = 1, 2, …, n, where n is the sample size and xi =
1 m
m
∑x
ij
.
j =1
Because the population is a multivariate normal distribution, xi and S i are independent of each other. The grand mean vector and pooled sample covariance is calculated by x=
1 n
n
∑x
i
i =1
and Sp =
1 n
n
∑S . i
i =1
2151_book.fm Page 67 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
67
Following procedures similar to those in Subsection 4.1.4.1, it is easy to verify that E ( xi − x ) = 0 and 2
n −1 n −1 1 ∑, cov( xi − x ) = 1 − cov( xi ) + 2 cov( xi ) = mn n n where m > 1 is the subgroup size and n is the sample size. Remark 4.16: The pooled S p is an estimate of ∑ . Remark 4.17: n (m − 1)S p is independent of each xi , and thus the grand mean x. Remark 4.18: n (m − 1)S p is distributed as a Wishart random matrix with d.o.f. n(m – 1). As ∑ can be estimated by each subgroup, when all the estimators are pooled together, a single estimator is obtained and thus has a large number of degrees of freedom. Consequently, T2 =
mn n(m − 1) p ( xi − x )T S−p1 ( xi − x ) ~ Fp,mn− n− p+1 , n −1 mn − n − p + 1
(4.30)
where Fp,mn − n − p +1 is an F-distribution with d.o.f p and mn − n − p + 1. For different dimensions of p, we can construct a T 2 control chart with control limits as: (n − 1)(m − 1) p UCL = mn − n − p + 1 Fp,mn− n− p+1 (α) . LCL = 0
(4.31)
For a large sample size n, (n − 1)(m − 1) p Fp,mn− n− p+1 (α) ≈ χ 2p (α ). mn − n − p + 1 The plotted points are: Ti 2 = m( xi − x )T S−p1 ( xi − x ), i = 1, 2, ..., n
(4.32)
Any Ti 2 exceeding the UCL may indicate a potential out-of-control signal, and appropriate action should be taken to identify the responsible special causes.
2151_book.fm Page 68 Friday, October 20, 2006 5:04 PM
68
Stream of Variation Modeling and Analysis for MMPs
2. T 2 Control chart for future subgroup measurements Once we have determined that the process is stable based on the T 2 control chart, we can use the measurements drawn from this stable process to set up the control limits for future measurement means. n +1 Suppose X is the future subgroup mean. Because X − X ~ N p (0, ∑), we mn have: mn n (m − 1) p (X − X)T S −p1 (X − X) ~ Fp,mn − n − p +1 n +1 mn − n − p + 1
(4.33)
Based on Equation 4.33, we can obtain the following T 2 control chart for future data monitoring: For 100 (1 − α )% prediction T 2 control chart, the control limits are: (n + 1)(m − 1) p UCL = mn − n − p + 1 Fp,mn− n− p+1 (α) . LCL = 0
(4.34)
For a large sample size m, (n + 1)(m − 1) p Fp,mn− n− p+1 (α) ≈ χ 2p (α ) . mn − n − p + 1 The plotted points in time order are the T 2 statistic defined by T 2 = m( x − x )T S−1 p ( x − x ). Any point exceeding the UCL may indicate a potential out-of-control signal, and appropriate corrective action should be taken to eliminate the special causes. Remark 4.19: So far, we have discussed the hypothesis testing of a multivariate one-sample mean vector from a single multivariate population. In some situations, we are required to compare two or more populations. Interested readers can find approaches to making inferences on multivariate two-sample testing problems in Reference 1.
4.2 MULTIPLE LINEAR REGRESSION Regression analysis is a collection of techniques to help assess how much variation of certain variables can be explained as being due to the effects of another set of variables. Similar to a simple linear regression analysis, a multiple regression analysis is a statistical technique that can be used to analyze the relationship between dependent (or response) variables and independent (or controlled) variables. The
2151_book.fm Page 69 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
69
objective of the multiple regression analysis is to use the independent variables whose values are known to predict the dependent values. The result is a linear combination of the independent variables that “best” predicts the dependent variables. The variables are weighted by the weights denoting their relative contribution to the overall prediction, so that the regression model has the maximum prediction power.
4.2.1 MODEL DESCRIPTION The accuracy of prediction of y based on a single variable x is determined by the correlation coefficient ρxy . If ρ xy is low, it is necessary to employ multiple variables as explanatory variables to increase the accuracy of prediction. So multiple regression analysis is a method to predict the value of a dependent variable y based on a set of independent variables, x1, x2, …, xp. If y is centered, i.e., E ( y ) = 0 , then a multiple linear regression model can be described by: y = a1 x1 + a2 x2 + a p x p + ξ
(4.35)
where y is called dependent variable (or response variable, or criterion); p variables xi , i = 1, 2, …, p, are called independent variables (or controlled, or explanatory variables), having values that are considered known or fixed, not random; ai, i = 1, 2, …, p, are called regression parameters whose values are unknown. Some basic assumptions are made regarding the error term ξ: (A1) E(ξ) = 0; (A2) var(ξ) = σ 2 So from Equation 4.35 and the assumptions (A1) and (A2), we obtain: E ( y ) = a1 x1 + a2 x2 + a p x p + E (ξ) = a1 x1 + a2 x2 + a p x p ; var( y ) = var(ξ) = σ 2 Suppose n observations are made, yi = a1 xi1 + a2 xi 2 + a p xip + ξ i , i = 1, 2, …, n. We assume that these observations are uncorrelated, i.e., (A3) cov( yi , y j ) = cov(ξ i , ξ j ) = 0 for i ≠ j .
(4.36)
2151_book.fm Page 70 Friday, October 20, 2006 5:04 PM
70
Stream of Variation Modeling and Analysis for MMPs
To simplify the notation, we rewrite Equation 4.36 in the equivalent matrix form: Y = Xa + ξ,
(4.36)
where y1 x11 y2 x 21 Y = , X = X1 X 2 X p = x n1 y n
x12
x 22
xn 2
x1 p x2 p and Xi ∈ R n x np
a1 ξ1 a2 ξ a = , ξ = 2 . ξ n a p Here, matrix X is usually referred to as a design matrix with full column rank. Thus, assumptions (A1)~(A3) can be rewritten in a matrix form known as Gauss–Markov conditions (GM conditions): (A1′) E(ξξ) = 0; (A2′) cov(ξξ) = σ2I. So Equation 4.36′ together with (A1′) and (A2′) is called a multivariate linear regression model. The unknown parameters are regression parameters vector a and variance σ2. Efforts are made to estimate and evaluate these parameters.
4.2.2 LEAST-SQUARES ESTIMATES We need to find the linear composite vector of p
ˆ = Xa = Y
∑a X i
i
i =1
ˆ is an approximation of vector Y, i.e., find as close to vector Y as possible where Y p a ∈R to minimize: ˆ = Y − Xa , ξ = Y−Y 2
2
2
(4.37)
2151_book.fm Page 71 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
71
or minimize ˆ )T ( Y − Y ˆ ) = ( Y − Xa )T ( Y − Xa ). ξT ξ = (Y − Y
(4.38)
Definition 4.2 (Least-squares estimate [LSE], residual vector, and residual sum of squares [RSS]): The coefficient vector a selected by minimizing Equation 4.38 is called the least-squares estimate of the regression parameters a, denoted by aˆ , i.e., aˆ = arg min( Y − Xa )T ( Y − Xa ) = arg min Y − Xa a ∈R
a ∈R
p
p
2 2
(4.39)
The difference between Y and its predictors Xaˆ is called residual vector, denoted by ξˆ . So, ˆ = Y − Xaˆ , ξˆ = Y − Y and ξˆ T ξˆ = ( Y − Xaˆ )T ( Y − Xaˆ )
(4.40)
are called the residual sum of squares, or RSS. The LSE of a can be obtained by the following theorem: Theorem 4.5 (LSE): If X has full rank, i.e., rank(X) = p ≤ n, then the LSE of a in Equation 4.39 and the residual vector ξˆ are given by: aˆ = (XT X)−1 XT Y ˆ = I − X(XT X)−1 XT Y ξˆ = Y − Y n
(4.41)
ˆ = a and cov(a) ˆ = σ2 (XT X)–1. Further, E(a) Remark 4.20 (BLUE: the best linear unbiased estimator): From Equation 4.41, aˆ is a linear function of Y. It is also an unbiased estimator under GM conditions. Moreover, the Gauss–Markov theorem (2, Theorem 2.6, p.41) ensures that aˆ is the best estimate because it has the smallest covariance among all linear unbiased estimators of a. Remark 4.21 (Collinearity): If X is not full column rank, the inverse in Equation 4.41 is replaced by a generalized inverse. In this case, some combinations, e.g., Xa~ , must be 0, and the columns are said to be collinear. In most regression analyses, Xa~ is not likely to be exactly 0. But a very small magnitude of |Xa~ | will lead to numerically unstable (XT X)–1. Some diagonal elements of (XT X)–1 will be large and, consequently, the estimated variance for the corresponding âi’s are large. This will make it difficult to detect the “significant” regression coefficient ai .
2151_book.fm Page 72 Friday, October 20, 2006 5:04 PM
72
Stream of Variation Modeling and Analysis for MMPs
The following two methods can be used to solve the problems caused by collinearity: 1. Deleting one predictor variable from a strongly correlated pair 2. Representing the original predictor variables using their principal components and then regressing Y on these new predictor variables Remark 4.22 (Hat matrix): Define matrix: H = X(XT X)−1 XT ,
(4.42)
ˆ = HY and ξˆ = Y – Y ˆ = (I – where H is usually called the “hat” matrix, then: Y n H)Y = (In – H)ξξ. Regarding the linear function of a, suppose p
cT a =
∑c a , i i
i =1
the following Gauss theorem [1, p.389] gives its “best” estimator. Theorem 4.6 (Gauss least-squares theorem): For the multiple regression model: Y = Xa + ξ , where ξ ~ (0, σ 2I) and X has full column rank p. Then for any vector c ∈R p , the estimator with the smallest possible variance among all linear estimators given by c T aˆ is unbiased for c T a. ˆ = Theorem 4.7: The residual vector ξˆ defined in Equation 4.41, i.e., ξˆ = Y − Y [In − H]Y, has the following property:
( )
E ξˆ T ξˆ = (n − p)σ 2 .
(4.43)
Remark 4.23: From Theorem 4.7, an unbiased estimator of σ2 is given by: s2 =
1 Y T ( I − H) Y n− p
=
1 ˆ )T ( Y − Y ˆ) (Y − Y n− p
=
1 1 ( Y − Xaˆ )T ( Y − Xaˆ ) = ( Y T Y − Y T Xaˆ ). n− p n− p
(4.44)
2151_book.fm Page 73 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
73
Remark 4.24 (Decomposition of sum of squares): The total sum of squares (TSS) can be decomposed into the regression sum of squares (RegSS) and the RSS, i.e., 2
2 2 ˆ + ξˆ . Y = Y 2
2
(4.45)
2
ˆ and variance of ξˆ as sY2, s 2 , If we denote the total variance of Y, variance of Y, Yx 2 and sY⊥ , respectively, then, x
sY2 = sY2 x + sY2 ⊥ . x
4.2.3 INFERENCES
ABOUT THE
(4.46)
REGRESSION MODEL
In this subsection, we give methods for testing the general linear hypothesis and obtaining the confidence intervals under the conditions in Theorem 4.8. It can be proved that the MLE for regression parameters is the same as the LSE under normality condition [1, pp.390–392], which is established by the following theorems: Theorem 4.8 (MLE for a and σ2): Given regression model Y = Xa + ξ, where a1 y1 ξ1 a2 y2 ξ2 Y = , a = , ξ == , a y ξ n n p design matrix X = [X1 X 2 X p ], Xi ∈ R n and X has full column rank, and the error vector follows multinormal distribution, ξ ~ N n (0, σ 2 I) . Then: 1. The MLE of a is the same as the LSE aˆ . Moreover, aˆ = (X T X)−1 X T Y ~ N p (a, σ 2 (X T X)−1 )
(4.47)
2. aˆ is distributed independently of the residuals ξˆ = Y − Xaˆ . Furthermore, ξˆ T ξˆ = n σˆ 2 ~ σ 2χ 2n− p where σˆ 2 =
1 ( Y − Xaˆ )T ( Y − Xaˆ ) is the MLE of σ2. n
(4.48)
2151_book.fm Page 74 Friday, October 20, 2006 5:04 PM
74
Stream of Variation Modeling and Analysis for MMPs
To draw inferences about regression parameters, the confidence ellipsoid for a is constructed based on the following theorem in terms of the estimated covariance matrix s 2 (X T X)−1 . Theorem 4.9 (Confidence region and confidence interval for parameters a): According to the assumption in Theorem 4.8, the 100 (1 − α )% confidence region for a is given by: (a − aˆ )T (X T X)(a − aˆ ) ≤ ps 2 Fp,n − p (α)
(4.49)
where s2 =
1 ˆT ˆ ξ ξ. n− p
Simultaneous 100 (1 − α )% confidence intervals for ai , i = 1, 2, ..., p are given by: aˆi ± var (aˆi ) pFp,n− p (α).
(4.50)
Remark 4.25: If the regression model contains a constant term, e.g., Y = a0 + a1X1 + + ar Xr + ξ
(4.51)
then the 100 (1 − α )% confidence region for a and simultaneous 100 (1 − α )% confidence intervals for ai , i = 0, 1, 2, ..., r are defined as: (a − aˆ )T (X T X)(a − aˆ ) ≤ (r + 1)s 2 Fr +1,n −r −1 (α)
(4.52)
aˆi ± var (aˆi ) (r + 1) Fr +1,n−r −1 (α),
(4.53)
where s2 =
1 ξˆ T ξˆ . n − r −1
Remark 4.26: The confidence ellipsoid is centered at the MLE â, and its orientation and size are determined by the eigenvalues and eigenvectors pair (λ, e) of XT X, i.e., λe = (X T X)e . If λ ≈ 0, then the confidence ellipsoid will be very long in the direction of the corresponding eigenvector. Remark 4.27: If ignoring the “simultaneous” confidence properties defined in Equation 4.50 or Equation 4.53, the pFp,n − p (α) in Equation 4.50 or (r + 1)Fr+1,n–r–1(α) in
2151_book.fm Page 75 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
75
Equation 4.53 can be replaced by a one-at-a-time t value, tn–p(α/2) or tn–r–1(α/2). This can be used to search for important independent variables. Remark 4.28: If the confidence interval for ai contains 0, then the associated independent variable xi may not have an influence on the dependent variable. In other words, it might contribute nothing to the response. Hence, it can be dropped from the regression model.
4.2.4 MODEL CHECKING: NORMALITY CHECKING AND OUTLIER DETECTION Regression modeling requires the assumption of normality and no outliers in the ˆ = (I – H)Y = (I – H)ξ. If the model is valid, then ξˆ is a data. We have ξˆ = Y – Y normally distributed random vector with E(ξˆ ) = 0 and var(ξˆ ) = σ2 (I – H). Although the covariance matrix is not diagonal, the off-diagonal elements are usually small and the diagonal elements are nearly equal. Clearly if the diagonal elements of H vary greatly, then the variance of each ξˆ i will also accordingly differ to a large extent from one another. Using residual mean square s2 =
1 Y − Xaˆ n− p
2 2
as the estimate of σ2, we obtain the estimation of var(ξˆ i ) : var (ξˆ i ) = s 2 (1 − hii ), i = 1, 2, …, n,
(4.54)
and the studentized residuals: ξˆ − E (ξˆ i ) ξˆ *i = i = var (ξˆ i )
ξˆ i s 2 (1 − hii )
, i = 1, 2, …, n.
(4.55)
The studentized residuals are expected to be approximately independently distributed as N (0, 1) . By ranking the ξˆ *i , i = 1, 2, …, n, we get the order statistics ξˆ *(i ), i = 1, 2, …, n. Then normality can be checked visually or analytically. The commonly used graphical approaches include: 1. Plot ( yˆi , ξˆ i ): If there is a trend on the plot (e.g., incorrect calculations, or if a constant term is ignored in the regression model) or the variance is not constant (e.g., transformations or weighted least squares are required), then the model assumptions do not hold. 2. Plot (xji, ξˆ i) or (xji xki , ξˆ i ): If there is a systematic pattern occurring in the plot, then it suggests that more terms are needed in the regression model. 3. Plot (i , ξˆ i ): This plot can be used to check independence over time. A systematic pattern suggests the data are time dependent.
2151_book.fm Page 76 Friday, October 20, 2006 5:04 PM
76
Stream of Variation Modeling and Analysis for MMPs
4. Q-Q plot: The Q-Q plot can be used to examine severe departures from normality or the presence of abnormal observations. If the sample size n is large, the extreme points may depart from normality but will not affect inferences about the regression parameter a. Statistical approaches include the Shapiro–Wilk test or Kolmogorov’s test. Interested readers may refer to Reference 3 in Chapter 3. An outlier can be detected using the methods described in Reference 3 (p. 274): 1. Construct the test statistic: P = max ξˆ *i , 1≤ i ≤ n
(4.56)
where ξˆ *i is defined in Equation 4.55. 2. Make inferences: There is an outlier in the data if P > F1,n–1(α/n); furthermore, if i0 = arg max ξˆ *i , 1≤ i ≤ n
(4.57)
then the i0th observation is an outlier.
4.2.5 INFERENCES
FROM THE
ESTIMATED REGRESSION MODEL
After having checked the fitted regression model and evaluated it as proper, one can use it to make some predictions, given a particular predictor vector x0 = [x10 x20 xp0 ]T. Based on x0 and MLE â, we can estimate the regression function and forecast a new observation at x0: 1. Estimating the regression function x0Ta at x0: Suppose y0 is the value of the response when the predictor vector x0 is given. Then:
(
)
E y0 x 0 = x T0 a
(4.58)
Its LSE is x0T â. Theorem 4.10 (Estimating): For the multiple regression model given in Equation 4.36 or Equation 4.36′: Y = Xa + ξ , E (ξ ) = 0, cov(ξ ) = σ 2I
2151_book.fm Page 77 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
77
x0T â is the best linear unbiased estimator (BLUE) of E(y0 |x0) in terms of minimum variance. Its variance is given by: var (x T0 aˆ ) = σ 2x T0 (XT X)−1 x 0
(4.59)
Further, if ξ is normally distributed, then a 100(1–α)% confidence interval for E(y0 |x0) is given by:
(
α x T0 aˆ ± tn− p s 2 x T0 (XT X)−1 x 0 2
)
(4.60)
where tn-p(α/2) is the upper 100(α/2)th percentile of a t-distribution with d.o.f. n – p. 2. Predicting a new observation y0 at x0: Predicting a new observation is more uncertain than estimating the expected value of y0. For the regression model: y0 = x T0 a + ξ 0 , ξ 0 ~ N (0, σ 2 ) where y0 is the new response; x0Ta, the expected value of y0 at x0; and ξ0, the new error. Theorem 4.11 (Predicting): For the multiple regression model given in Equation 4.36 or Equation 4.36′: Y = Xa + ξ , E (ξ ) = 0, cov(ξ ) = σ 2I a new observation of y0 has the unbiased predictor x T0 aˆ . The variance of the forecast error y0 − x T0 aˆ is given by: var ( y0 − x T0 aˆ ) = σ 2 (1 + x T0 (XT X)−1 x 0 )
(4.61)
Further, if error ξ is normally distributed, then a 100(1–α)% prediction interval for y0 is given by:
(
α x T0 aˆ ± tn− p s 2 1 + x T0 (XT X)−1 x 0 2
)
(4.62)
where tn–p(α/2) is the upper 100(α/2)th percentile of a t-distribution with d.o.f. of n – p. The proofs for Theorem 4.10 and Theorem 4.11 can be found in [1, pp. 400~402].
2151_book.fm Page 78 Friday, October 20, 2006 5:04 PM
78
Stream of Variation Modeling and Analysis for MMPs
Remark 4.29: By comparing Equation 4.60 with Equation 4.62, we can find:
(
)
(
α α tn− p s 2 1 + x T0 (XT X)−1 x 0 > tn− p s 2 x T0 (XT X)−1 x 0 2 2
)
(4.63)
i.e., the half width of the prediction interval for y0 is longer than the half width of the confidence interval for estimating the value of the regression function E(y0 |x0) = x0Ta. This additional uncertainty in forecasting y0 comes from the presence of the new unknown error term ξ0.
4.2.6 MULTIVARIATE LINEAR REGRESSION 4.2.6.1 Multivariate Linear Regression Model Multiple linear regression is used to model the linear relationship between one response, Y, and a set of independent variables X1, X2, , Xp. But in some cases, there are many response variables, Y(1), Y(2), …, Y(q). We need to model the relationship between the q variables and independent variables. Each response follows its own multiple regression model: Y( j ) = Xa ( j ) + ξ ( j ) , j = 1, 2, …, q. If each response has n observations, we introduce the notation: x11 x 21 X= x n1 y11 y21 Y = yn1 a11 a21 A= a p1
y12 y22
yn 2
a12
a22
a p2
x12
x 22
xn 2
x1 p x2 p x np
, n× p
y11 y2 q = Y(1) Y( 2) Y( q ) , ynq n× q a1q a2 q a pq
= a (1) a ( 2) a ( q ) , p× q
(4.64)
2151_book.fm Page 79 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
ξ11 ξ 21 ξ= ξ n1
ξ12
ξ 22
ξn2
ξ1q ξ 2q ξ nq
n× q
79
ξ1T ξT 2 = ξ (1) ξ ( 2) ξ ( q ) = , T ξ n
Thus, the multivariate linear regression model is defined as: Y(i ) = Xa (i ) + ξ (i ) , i = 1, 2, …, q, or Y = XA + ξ , E (ξ (i ) ) = 0, cov(ξ (i ) , ξ ( j ) ) = σ ij I for i, j = 1, 2, …, q,
(4.65)
where Y ∈ R n ×q , X ∈ R n × p , A ∈ R p ×q , ξ ∈ R n ×q . Remark 4.30: The n observations on the kth sample have the covariance matrix ∑, but observations for different samples are uncorrelated. Remark 4.31: In the multivariate linear regression model, A and σij, i, j = 1, 2, …, q are unknown parameters. Remark 4.32: The error term ξ corresponding to different responses may be correlated. 4.2.6.2 Least-Squares Estimation (LSE) and Maximum Likelihood Estimation (MLE) of Parameters Based on the results of multiple regression analysis, the LSE of a(i) is given by minimizing Y(i) – Xa(i)22 as: aˆ (i ) = (XT X)−1 XT Y(i ) , i = 1, 2, …, q
(4.66)
Thus, the criterion to be minimized for multivariate regression analysis is given by:
(
ˆ ) = tr Y − XA ˆ f (A
) ( Y − XAˆ ) T
(4.67)
ˆ by differentiating f(A) ˆ w.r.t. each element of A: We can obtain the LSE A ˆ = (XT X)−1 XT Y A
(4.68)
2151_book.fm Page 80 Friday, October 20, 2006 5:04 PM
80
Stream of Variation Modeling and Analysis for MMPs
ˆ we can obtain the matrices of predicted values and residuals: So, using the LSE A, ˆ = X (XT X)−1 XT Y ˆ = XA Y
(4.69)
ˆ = [I − X (XT X)−1 XT ] Y. ξˆ = Y − Y
(4.70)
We know that multiple regression analysis possesses orthogonality properties among residuals, predicted values, and columns of X. In multivariate regression analysis, these properties also hold. Property 1: Columns of X and residual ξˆ are orthogonal, i.e., XT ξˆ = 0
(4.71)
It can be easily verified by XT ξˆ = XT [I − X(XT X)−1 XT ] Y = 0. ˆ and residual ξˆ are orthogonal, i.e., Property 2: Predicted value Y ˆ T ξˆ = 0 Y
(4.72)
ˆ T ξˆ = (X (XT X)–1 XT Y)T [I – X (XT X)–1 XT] Y = 0. This property holds because Y Similar to multiple regression analysis, the total sum of squares can also be decomposed into two components because: ˆ + (Y − X A ˆ) ˆ + ξˆ = X A Y=Y Using Property 2, we have: ˆ TY ˆ + ξˆ T ξˆ + 0 T + 0 = Y ˆ TY ˆ + ξˆ T ξˆ ˆ + ξˆ ) = Y ˆ + ξˆ )T ( Y YT Y = (Y
(4.73)
= Y T X(XT X)−1 XT Y + ( Y T Y − Y T X(XT X)−1 XT Y)
(4.74)
Thus, the TSS and cross products can be decomposed into two components: Predicted sum of squares (RegSS) and cross products, and residual sum of squares (ReSS) and cross product, i.e., ˆ TY ˆ + ξˆ T ξˆ YT Y = Y
(4.75)
ˆ T X XT A ˆ ˆ TY ˆ = YT Y − A ξˆ T ξˆ = Y T Y − Y
(4.76)
So,
2151_book.fm Page 81 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
81
The following results give the LSE ξˆ its properties, as in the multiple regression model. Theorem 4.12 (Mean vectors and covariance matrices): Let the multivariate multiple regression model in Equation 4.65 have full rank, i.e., rank(X) = p < n. Then: ˆ = [aˆ aˆ aˆ ] is an unbiased estimator of 1. The LSE A (1) ( 2) (q) A = [a (1) a ( 2) a ( q ) ], i.e., ˆ = A. E(A) 2. cov( aˆ (i ) , aˆ ( j ) ) = σij (XT X)–1, i, j = 1, 2, …, q. ˆ satisfies 3. The residual ξˆ = ξˆ (1) ξˆ ( 2) ξˆ ( q ) = Y – XA
( )
(
)
E ξˆ (i ) = 0, E ξˆ T(i ) ξˆ ( j ) = (n − p)σ ij
(4.77) (4.78)
(4.79)
Thus,
()
1 ˆT ˆ E ξˆ = 0, E ξ (i ) ξ ( j ) = ∑ . n− p ˆ and the LSE A ˆ are 4. The residual ξˆ = ξˆ (1) ξˆ ( 2) ξˆ ( q ) = Y – XA uncorrelated. The detailed proof can be found in Reference 1 (p. 417). Similar to multiple regression analysis, when the error ξ follows a normal distribution, we can obtain the MLE and its distribution. This result is stated in the following theorems: Theorem 4.13 (MLE): Assume that the multivariate regression model in Equation 4.65 has full rank, i.e., rank(X) = p, p + q ≤ n, and the error ξ is normally distributed. Then: 1. The MLE of A is: ˆ = (XT X)–1 XT Y A
(4.80)
ˆ is normally distributed with 2. A ˆ ) = A, and cov( aˆ (i ), aˆ ( j )) = σij (XT X)–1, i, j = 1, 2, …, q E(A
(4.81)
2151_book.fm Page 82 Friday, October 20, 2006 5:04 PM
82
Stream of Variation Modeling and Analysis for MMPs
ˆ of positive definite ∑, where ˆ is independent of the MLE ∑ 3. A ˆ = 1 ξˆ T ξˆ = 1 ( Y − X A ˆ )T ( Y − X A ˆ) ∑ n n ˆ ~ W (∑ ) 4. n ∑ n− p
(4.82) (4.83)
4.2.6.3 Inferences for Multivariate Regression Model under Normality Assumption Suppose we want to test the hypothesis that the responses do not depend on some independent variables. Without loss of generality, we can split the matrices in the following manner: A(1) A = , A(1) ∈ Rl × q , A( 2) ∈ R( p−l )× q A( 2) p× q X = X1 X 2 , X1 ∈ R n×l , X 2 ∈ R n×( p−l ) n× p So the general model can be rewritten as: A(1) E(Y) = XA = X1 X 2 = X1A(1) + X 2 A( 2) A( 2)
(4.84)
Under the null hypothesis H 0 : A (2 ) = 0, the model becomes Y= X1A(1) + ξ
(4.85)
Then the extra sum of squares and cross products is given by: ˆ ˆ )T ( Y − X A ˆ ˆ T ˆ ( Y − X1 A (1) 1 (1) ) − ( Y − X A) ( Y − X A) = n(∑1 − ∑ ) where ˆ = 1 (Y − X A ˆ T ˆ ˆ = (XT X )−1 XT Y and ∑ A (1) 1 1 1 1 1 (1) ) ( Y − X1 A(1) ). n
(4.86)
2151_book.fm Page 83 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
83
Thus, the likelihood ratio, Λ, can be expressed based on generalized variances: ˆ ) ˆ ,∑ L(A 1 (1) Λ= = = ˆ) ˆ ,∑ max L ( A, ∑) L ( A A ,∑ max L ( A(1) , ∑)
A(1) ,∑
ˆ ∑ ˆ ∑ 1
n/2
(4.87)
or equivalently Wilks’ lambda statistics Λ 2/n =
ˆ ∑ ˆ ∑ 1
(4.88)
Theorem 4.14 (Likelihood ratio test): Suppose the multivariate regression model in Equation 4.65 holds with X full rank, i.e., rank(X) = p, p + q ≤ n. Assume that the ˆ ~ W (∑) and is indeerror ξ follows normal distribution. Under H0: A(2) = 0, n∑ n–p ˆ pendent of n(∑1 − ∑), which follows Wp−l (∑). The likelihood ratio test of H 0 is equivalent to rejecting H 0 for large values of −2 ln( Λ) = − n ln
ˆ ˆ n∑ ∑ = − n ln ˆ ˆ + n(∑ ˆ −∑ ˆ) n∑ ∑ 1 1
(4.89)
When the sample size n is large, the distribution of modified statistics: ˆ ∑ 1 − n − p − (q − p + l + 2 ) ln ˆ 2 ∑ 1
(4.90)
is close to a χ2 distribution with d.o.f. q(p – l – 1). The theorem and its proof can be found on page 422 in Reference 1, where the possibility that X is not full rank is also discussed. Other than the likelihood ratio test, there are also other multivariate test statistics for testing H0: A(2) = 0, e.g., Wilks’ lambda, Pillai’s trace, Hotelling–Lawley trace, and Roy’s greatest root. These four multivariate test statistics are commonly used in computer-based packages. Interested readers may refer to Reference 1. 4.2.6.4 Predictions from Multivariate Regression As in multiple regression analysis, the multivariate regression model can also be used for predictions if the model has been fit and checked for any inadequacies. Suppose the multivariate regression model has the form Y = XA + ξ
2151_book.fm Page 84 Friday, October 20, 2006 5:04 PM
84
Stream of Variation Modeling and Analysis for MMPs
where ξ is normally distributed. If the model is adequate, two predictive problems can be solved using the fitted model. 1. Predicting the mean responses corresponding to fixed values x0: Based on the theorem (MLE) and the remarks therein, we have
(
ˆ T x ~ N AT x , x T (XT X)−1 x ∑ A 0 0 0 0 q
)
(4.91)
ˆ ~ W (∑ ) n∑ n− p
(4.92)
ˆ T x . So Clearly, the unknown value of the regression function at x0 is A 0 2 the T statistic can be defined as: A ˆ T x − AT x 0 0 T2 = x T0 (XT X)−1 x 0
T
−1 n ˆ T x − AT x A 0 0 ˆ ∑ n − p x T0 (XT X)−1 x 0
(4.93)
The 100(1-α)th confidence ellipsoid for AT x0 is given by the inequality:
(
ˆ Tx AT x 0 − A 0
)
T
n ˆ n − p ∑
−1
(A x T
0
)
ˆ Tx ≤ −A 0
q(n − p) x ( X X) x 0 Fq ,n− p+1− q (α) 1 n − p + − q T 0
T
(4.94)
−1
where Fq,n − p +1−q (α) is the upper (100α)% percentile of an F-distribution with d.o.f q, n − p + 1 − q. The 100(1-α)% simultaneous confidence intervals for E(Yi) = x T0 a (i ) are: q(n − p) n x T0 aˆ (i ) ± Fq ,n− p+1− q (α) x T0 (XT X)−1 x 0 σˆ ii , n − p + 1− q n− p
(4.95)
i = 1, 2, …, q ˆ. ˆ and σˆ is the ith diagonal element of ∑ where aˆ (i ) is the ith column of A ii 2. Forecasting new responses Y0 at x0: If the model is adequate, it can be used to forecast new responses Y0 = AT x0 + ξ0 at x0, where ξ0 is independent of ξ.
2151_book.fm Page 85 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
85
Because ˆ T x = AT x + ξ – A ˆ T x = (AT – A ˆ T) x + ξ Y0 – A 0 0 0 0 0 0 its distribution is: ˆ T x ~ N (0, ∑ + x T (XT X)–1 x ∑) Y0 – A 0 q 0 0
(4.96)
ˆ. which is independent of n ∑ Thus, the 100(1–α)% prediction region is within the ellipsoid centered at Y0: −1
ˆ ˆ T x )T n ∑ ˆT ( Y0 − A 0 n − p ( Y0 − A x 0 ) ≤ q(n − p) Fq ,n− p+1− q (α) (1 + x (X X) x 0 ) n − p + 1 − q T 0
T
(4.97)
−1
The 100(1–α)% simultaneous prediction intervals for the individual responses y0i are: q(n − p) n x T0 aˆ (i ) ± Fq ,n− p+1− q (α) (1 + x T0 (XT X)−1 x 0 ) σˆ ii , n − p + 1− q n − p 4.98) i = 1, 2, …, q ˆ. ˆ and σˆ is the ith diagonal element of ∑ where aˆ (i ) is the ith column of A ii Remark 4.33: By comparing the previous two 100(1–α)% simultaneous intervals, ˆ 0i, the prediction we can see that, because of the presence of the random error σ intervals for the actual values of the response variables are wider than the corresponding intervals for the expected values. 4.2.6.5 Selection Method of Independent Variables In the previous chapters, we have discussed that the multiple correlation coefficient increases with an increase in the number of independent variables involved in the regression model. If the number of independent variables exceeds n – 1, where n is the number of samples, the values of multiple correlation coefficient become irrelevant to the magnitude of the correlation between X1, X2, …, Xp. To deal with this problem, some methods are useful in determining a set of extra variables to be included in the regression function, and the multiple correlation
2151_book.fm Page 86 Friday, October 20, 2006 5:04 PM
86
Stream of Variation Modeling and Analysis for MMPs
coefficient is adjusted for d.o.f. Among these methods, the AIC criterion, which was first proposed by Akaike [4], and the PSS criterion ([5]) are commonly used. Detailed descriptions are available in Reference 6.
4.3 EXERCISES 1. Let X be a sequence of i.i.d 2-D random vectors with a multivariate normal distribution. Suppose that the observations x are: 3 5 7 x= 2 4 6
6 4 5 7 10 9
Given that µ 0 = [5 7]T : a. Conduct the hypothesis testing for H0: µ = µ0. Determine T 2 using the given data. b. From (a), what is the distribution of T 2 ? Also, determine all parameters of this distribution. c. Given α = .05 level and based on the given data, what will be the conclusion for the test in (a)? 2. The following data are the off-target measurements (in mm) of an automobile body in the panel assembly process. X1, X2, X3, and X4 are the associated deviations from their nominal positions in mm (based on sensors). Twenty-five parts are measured. Part
X1
X2
X3
X4
1 2 3 4 5 6 7 8 9 10 11 12
0.0388 –0.0322 –0.0195 0.0254 0.0054 0.0425 –0.0059 –0.0352 –0.0457 –0.0620 –0.0076 –0.0181
0.0050 0.0004 –0.0146 –0.0747 –0.0476 –0.0504 0.0047 0.0404 –0.0094 0.0531 –0.0063 0.0219
0.0073 0.0161 0.0581 0.0644 –0.0065 –0.0158 0.0156 –0.1069 –0.0309 0.0007 –0.0101 0.0274
0.4319 0.4479 0.3606 0.4115 0.3864 0.3508 0.4185 0.3803 0.4165 0.3587 0.3819 0.4491
2151_book.fm Page 87 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
Part
X1
X2
X3
X4
13 14 15 16 17 18 19 20 21 22 23 24 25
0.0142 0.0166 0.0151 0.0358 –0.0166 0.0298 –0.0040 –0.0175 0.0403 0.0274 –0.0154 0.0026 0.0128
0.0290 0.0805 –0.0351 0.0091 0.0375 0.0137 0.0458 0.0117 0.0355 –0.0487 –0.0581 –0.0097 –0.0092
0.0157 –0.0255 –0.0156 –0.0223 0.0181 0.0736 0.0412 0.0271 0.0196 0.0115 0.0120 0.0253 0.0272
0.3799 0.3346 0.4253 0.3699 0.3903 0.3818 0.3279 0.5460 0.3710 0.3898 0.5007 0.4102 0.3958
Given that the true mean vector is µ0 = [0 0]T, Use the data on X1 and X2 to do the following: a. Conduct the hypothesis testing for H0: µ = µ0. Determine T 2 using the given data. b. Calculate T 2 using Equation 4.6. Is it the same as the result from (a)? c. Find the distribution of T 2. d. Given α = .05 level and based on the given data, what will be the conclusion of the test in (a)? 3. Using the data on X1 and X2 in Problem 2, assume that α = 0.05 and µ0 = [0 0]T. The hypothesis testing is H 0 : µ = µ 0 . H1: µ ≠ µ 0
4. 5. 6. 7.
Construct a confidence ellipse. Is the result from the confidence ellipse the same as the conclusion in Problem 2d? Using the data in Problem 2 and given α = 0.05, construct a T 2 chart and a control ellipse for X3 and X4. From the data on X1 and X2 in Problem 2, construct a control ellipse for a future observation xT = (x1, x2). Comment on the result. Repeat Problem 5, but use the data on X3 and X4 in Problem 2. Suppose 1 X = 3 1 T
1 5 3
1 8 4
1 7 4
1 9 5
1 4 and Y T = 15 9 20 18 25 16 . 2
87
2151_book.fm Page 88 Friday, October 20, 2006 5:04 PM
88
Stream of Variation Modeling and Analysis for MMPs
The multiple linear regression model is Y = Xa + ξ, where ξ1 a0 ξ a = a1 and ξ = 2 . a2 ξ 6 a. Determine the least-squares estimates â. ˆ b. Determine the fitted values Y. ˆ c. Calculate the residuals ξ. d. Calculate the residual sum of squares, ξˆ T ξˆ . 8. The relationship between measurement deviation (y) and the pins deviations (u) in a single-station assembly process can be represented in a linear relationship as the following: y = Γ ⋅u + ε where Γ is the geometric relationship between y and u, and ε is the model error. Suppose y = δx1 u = δP1x
δz1
δx3
δP1z
δz3
δx5
δx9
T
δP2 z ,
,
σ 2ε = 1.7361 × 10 −5 , and 0.9972 −0.0093 1 Γ= 0 −0.0150 1.0072 −0.0246
−0.1157 0.6196 0 1 0.3846 0.2961 −0.0049
T
δz9 ,
0.1157 0.3804 0 0 . 0.6154 −0.2961 1.0049
Let N = 25, i.e., there are 25 parts observed.
2151_book.fm Page 89 Friday, October 20, 2006 5:04 PM
Statistical Inferences on Mean Vectors and Linear Models
89
The observation data for y is provided in the table below: Part
δx 1
δz 1
δx 3
δz 3
δx 5
δx 9
δz 9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
0.4231 0.4337 0.3563 0.4174 0.3748 0.3472 0.4237 0.3879 0.4197 0.3532 0.3686 0.4454 0.3769 0.3320 0.4306 0.3921 0.3849 0.3738 0.3237 0.5418 0.3687 0.3925 0.4846 0.3888 0.3828
0.0073 0.0161 0.0581 0.0644 –0.0065 –0.0158 0.0156 –0.1069 –0.0309 0.0007 –0.0101 0.0274 0.0157 –0.0255 –0.0156 –0.0223 0.0181 0.0736 0.0412 0.0271 0.0196 0.0115 0.0120 0.0253 0.0272
0.4319 0.4479 0.3606 0.4115 0.3864 0.3508 0.4185 0.3803 0.4165 0.3587 0.3819 0.4491 0.3799 0.3346 0.4253 0.3699 0.3903 0.3818 0.3279 0.5460 0.3710 0.3898 0.5007 0.4102 0.3958
0.0331 0.0351 0.0834 0.0404 0.0496 –0.0077 –0.0119 –0.1330 –0.0251 –0.0085 0.0113 0.0329 0.0078 –0.0219 –0.0107 –0.0895 0.0078 0.0890 0.0652 0.0439 0.0279 0.0119 0.0294 0.0713 0.0618
0.4226 0.4396 0.3513 0.4137 0.3769 0.3411 0.4228 0.3965 0.4195 0.3510 0.3736 0.4412 0.3786 0.3292 0.4301 0.3885 0.3949 0.3748 0.3167 0.5421 0.3687 0.3846 0.4868 0.3875 0.3805
0.4496 0.4669 0.3873 0.4032 0.4227 0.3683 0.4078 0.3644 0.4231 0.3496 0.3968 0.4569 0.3707 0.3217 0.4360 0.3265 0.3669 0.3861 0.3464 0.5565 0.3781 0.3875 0.5132 0.4248 0.4221
–0.0348 –0.0380 0.0088 0.0856 –0.0881 –0.0430 0.0399 –0.0595 –0.0486 0.0010 –0.0478 0.0202 0.0407 –0.0262 –0.0249 0.0688 0.0463 0.0584 0.0032 –0.0107 0.0088 –0.0038 –0.0238 –0.0426 –0.0199
a. b. c. d.
Use least-squares estimation to find uˆ . From Theorem 4.7, find an unbiased estimator of σ 2ε ? Given that α = 0.05, find the confidence region for u. Determine the simultaneous 100 (1 − α )% confidence intervals for the ui, i = 1, 2, 3. Use α = 0.05.
REFERENCES 1. Johnson, R.A. and Wichern, D.W., Applied Multivariate Statistical Analysis, 4th ed., Prentice Hall, Upper Saddle River, NJ, 1998. 2. Sen, A. and Srivastava, M., Regression Analysis: Theory, Methods and Applications, Springer-Verlag, New York, 1990. 3. Srivastatva, M.S., Methods of Multivariate Statistics (Wiley Series in Probability and Statistics), John Wiley & Sons, New York, 2002. 4. Akaike, H., Autoregressive model fitting for control, Annals of the Institute of Statistical Mathematics, 23, 163–180, 1971.
2151_book.fm Page 90 Friday, October 20, 2006 5:04 PM
90
Stream of Variation Modeling and Analysis for MMPs 5. Allen, D.M., Mean squared error of prediction as a criterion for selecting variables, Technometrics, 13, 469–475, 1971. 6. Takeuchi, K., Yanai, H., and Mukherjee, B.N., The Foundations of Multivariate Analysis: A Unified Approach by Means of Projection onto Linar Subspaces, John Wiley & Sons, New York, 1982. 7. Muirhead, R.J., Aspects of Multivariate Statistical Theory (Wiley Series in Probability and Statistics), John Wiley & Sons, New York, 1982. 8. Morrison, D.F., Multivariate Statistical Methods, 2nd ed., McGraw-Hill, New York, 1976. 9. Kleinbaum, D.G. et al., Applied Regression Analysis and Other Multivariate Methods, 2nd ed., PWS-KENT Publishing Company, Boston, MA, 1988. 10. Manly, B.F.J., Multivariate Statistical Methods: A Primer, Chapman and Hall, London, 1986. 11. Kshirsagar, A.M., Multivariate Analysis, Marcel Dekker, New York, 1972. 12. Karson, M.J., Multivariate Statistical Methods, The Iowa State University Press, Ames, IO, 1982. 13. Hair, J.F. et al., Multivariate Data Analysis, 3rd ed., Macmillan Publishing Company, New York, 1992.
ADDITIONAL READING Hair, J.F. et al., Multivariate Data Analysis, 3rd ed., Macmillan Publishing Company, New York, 1992. Karson, M.J., Multivariate Statistical Methods, The Iowa State University Press, Ames, IO, 1982. Kleinbaum, D.G. et al., Applied Regression Analysis and Other Multivariate Methods, 2nd ed., PWS-KENT Publishing Company, Boston, MA, 1988. Kshirsagar, A.M., Multivariate Analysis, Marcel Dekker, New York, 1972. Manly, B.F.J., Multivariate Statistical Methods: A Primer, Chapman and Hall, London, 1986. Morrison, D.F., Multivariate Statistical Methods, 2nd ed., McGraw-Hill, New York, 1976. Muirhead, R.J., Aspects of Multivariate Statistical Theory (Wiley Series in Probability and Statistics), John Wiley & Sons, New York, 1982.
2151_book.fm Page 91 Friday, October 20, 2006 5:04 PM
5
Principal Component Analysis and Factory Analysis
5.1 PRINCIPAL COMPONENT ANALYSIS A complex system may involve large volumes of data with more than hundreds of different variables. In such situations, it is desirable to find ways to effectively reduce the dimensions of the data with minimum loss of information, and interpret the data with physical meanings. Principal component analysis (PCA) provides the capability to identify patterns in high-dimensional data and reconstruct it to highlight similarities and differences. The concept of PCA was first introduced by Karl Pearson in 1901 and formally treated by Hotelling [1] and Rao [2]. PCA has been widely and successfully applied in various fields by analyzing the variance-covariance structure through linear combinations of multivariate data. In this section, the basic mathematical model of PCA, its geometrical interpretation, as well as related statistical tests will be discussed. A potential application in process control is also presented.
5.1.1 MATHEMATICAL MODEL
OF
PCA
Suppose X T = [X1 X2 Xp] is a random vector with variance-covariance
(
)
∑ x = var(X) = E (X − µ X )(X − µ X )T ,
(5.1)
Clearly, ∑x is at least positive semidefinite with rank r ≤ p; thus, all the eigenvalues are real and nonnegative. Without loss of generality, let the eigenvalues be λ1 ≥ λ2 ≥ ≥ λp ≥ 0 and the standardized eigenvectors be e1, e2, …, ep. Denote
E = e1
e2
e11 e21 e p = e p1
e12
e22
ep2
e1 p e2 p e pp
T
(5.2)
then EET = ETE = I and ET∑xE = Λ, or ∑x = EΛET where Λ = diag(λ1, λ2, , λp). Consider the linear combination: 91
2151_book.fm Page 92 Friday, October 20, 2006 5:04 PM
92
Stream of Variation Modeling and Analysis for MMPs
Yi = ai1 X1 + ai 2 X2 + + aip X p = a Ti X , (i = 1, 2, …, p)
(5.3)
where a Ti = ai1 ai 2 aip , a Ti a i = 1 and X T = [X1 X2 Xp]. The variance-covariance structure of Yi is determined by: var(Yi ) = var(a Ti X) = a Ti ∑ X a i cov(Yi , Yj ) = cov(a iT X, a Tj X) = a Ti ∑ X a j , for i, j = 1, 2, …, p.
(5.4)
As ∑ X = EΛET and λ1 ≥ λ2 ≥ ≥ λp ≥ 0 p
var(Yi ) = a Ti ∑ x a i = a Ti EΛET a i =
∑ λ (a e ) i
T i
2
j
j =1
(5.5)
p
≤ λ1
∑ (a e ) T i
j
2
= λ1a Ti EET a i = λ1a Ti a i = λ1
j =1
when a i = e1, “=” holds. So a i = e1 maximizes var(Yi ) and, λ1 = var(e1T X) = e1T ∑ X e1T = max a T ∑ X a a:a T a =1
(5.6)
It can be shown that for any i = 1, 2, …, p λ i = var(eTi X) , and cov(eTi X, eTj X) = eTi ∑ X e j = e Ti EΛET e j = 0 for i ≠ j
(5.7)
So the principal components (PCs) of Xi, i = 1, 2, …, p, are the uncorrelated linear combinations of Xi, i = 1, 2, …, p with the components of eigenvectors as combination coefficients. Property 1: Let ∑X be the covariance matrix of random vector X T = [X1 X2 Xp] and its eigenvalue-eigenvector pairs be (λi, ei) i = 1, 2, …, p with λ1 ≥ λ2 ≥ ≥ λp ≥ 0. Then the ith PC is defined by the linear combination: Yi = ei1 X1 + ei 2 X 2 + ⋅ ⋅ ⋅ + eip X p = eTi X, i = 1, 2, …, p
(5.8)
and var(Yi ) = λ i . Remark 5.1: var(Y1 ) ≥ var(Y2 ) ≥ ≥ var(Y p ) ≥ 0 because λ1 ≥ λ2 ≥ ≥ λp ≥ 0.
2151_book.fm Page 93 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
93
Remark 5.2: If some eigenvalues are identical, the choice of ei , and hence Yi , is not unique. P
P
Property 2:
∑
var(Yi ) =
i =1
∑ var( X ) . i
i =1
Property 3: The correlation between PCs and original variables is:
ρ(Yi , X j ) =
cov(Yi , X j ) var(Yi ) var( X j )
=
eij λ i σ ii
, i, j = 1, 2, …, p.
From Property 2, we can see that the total variance will not be significantly affected by ignoring components with smaller variances. This is why the PCA approach can be used to reduce variables without losing too much information. The contribution rate of the ith principal component can be defined as: var(Y j )
=
p
λi
(5.9)
p
∑ var(Y ) ∑ λ j
j
j =1
j =1
The larger this rate, the larger the capability of this PC to summarize the information in Xi , i = 1, 2, ..., p . The accumulated contribution rate of the first k PCs is given by: k
k
∑ var(Y ) ∑ λ j
j =1 p
=
∑ var(Y ) ∑ λ j
j =1
j
j =1 p
(5.10) j
j =1
If this rate becomes 85%, it indicates that the first k PCs extract 85% original variation information in Xi , i = 1, 2, ..., p . Thus, many variables are reduced effectively without losing too much information. We define the Loss of Information (LoI) using the first k principal components instead of all the variables Xi , i = 1, 2, ..., p as: p
LoI =
∑ i=1
p
k
var( Xi ) –
∑
var(Yi ) =
i =1
where k is selected by minimizing LoI.
∑ i =1
p
k
λi –
∑ i =1
λi =
∑λ
i = k +1
i
(5.11)
2151_book.fm Page 94 Friday, October 20, 2006 5:04 PM
94
Stream of Variation Modeling and Analysis for MMPs
p
p
k
k
∑ var( X ) – ∑ var(a X) ≥ ∑ var( X ) – ∑ var (e X) T i
i
i =1
i=1
T i
i
(5.12)
i =1
i=1
Equality holds if and only if ai = ei. PCs can also be calculated using “standardized” variables. Remark 5.3: In general, the eigenvalue–eigenvector pairs obtained from a covariance matrix are different from those obtained from the correlation matrix. This leads to different PCs from the covariance matrix ∑ and correlation matrix R. The covariance matrix ∑ is recommended when the variables are measured in the same, or comparable, units. Sometimes, the population variance-covariance ∑ X is not known. In this case, the sample covariance (or correlation) matrix is used in PCA. Corresponding to the results from the population, there are similar results from the random sample. Property 4: Let x be the sampling data matrix and S be the sample covariance matrix with eigenvalue–eigenvector pairs ( λˆ i , eˆ i ), i = 1, 2, …, p, with λˆ 1 ≥ λˆ 2 ≥ ≥ λˆ p ≥ 0 . Then the ith PC is defined by: yˆi = eˆi1 x1 + eˆi 2 x2 + … + eˆip x p = eˆ Ti x , i = 1, 2, …, p .
(5.13)
The variance-covariance matrix of yˆ T = [ yˆ1 , yˆ2 ,..., yˆ p ] is: var( yˆ ) = diag (λˆ 1, λˆ 2 ,..., λˆ p )
(5.14)
The total sample variance is: p
∑
p
var( yˆi ) =
i=1
∑
p
var( xi ) =
i =1
∑
p
sii =
i=1
∑ λˆ
i
(5.15)
i =1
and the sample correlation matrix R between PCs and x with (i, j) entry is:
ρ( yˆi , x j ) =
cov( yˆi , x j ) var( yˆi ) var( x j )
=
eˆij λˆ i sii
, i, j = 1, 2, ..., p
(5.16)
In summary, PCA procedures using sampling measurements are: Step 1: Obtain sample matrix x and standardize data if the variables are not of the same unit. Step 2: Calculate the sample covariance matrix S or correlation matrix R.
2151_book.fm Page 95 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
95
X2
Y2
Y1
O
X1
FIGURE 5.1 Relationship between PCs Y1, Y2 and original variables X1, X2.
Step 3: Compute the sample eigenvalue–eigenvector pairs (λˆ i , eˆ i ), i = 1, 2, …, p, with λˆ 1 ≥ λˆ 2 ≥ ≥ λˆ p ≥ 0 . Step 4: Obtain the sample PCs: yˆ i = eˆi1 x1 + eˆi 2 x2 + + eˆip x p = eˆ Ti x, i = 1, 2, …, p. Remark 5.4: From the derivation of PCs, it is evident that they only depend on the covariance matrix ∑ X and multinormal distribution is not required. However, normality is required for statistical tests.
5.1.2 GEOMETRICAL INTERPRETATION From the previous subsection, the p PCs can be algebraically viewed as special linear combinations of original variables Xi , i = 1, 2, …, p, i.e. Yi = ei1 X 1 + ei 2 X 2 + + eip X p = eTi X , i = 1, 2, …, p
(5.17)
From a geometrical point of view, these uncorrelated linear combinations Yi , i = 1, 2, …, p, form a new coordinate system by rotating the old coordinate system spanned by Xi , i = 1, 2, …, p. The PCs have the maximum variance. For simplicity, we will illustrate this transformation using p = 2. The 2-D scatter plot of data is shown in Figure 5.1. Suppose we have n samples. Let the original variables be X1, X2 and the two PCs be Y1,Y2. Further, let us assume XT = (X1, X2) ~ N(µ , ∑). Without loss of generality, we can assume that the data are centered at the origin, i.e., µ = 0. For a bivariate normal distributed random variable X, the n points are scattered in an ellipse with the direction of Y1 being its long-axis direction and the direction of Y2 being its short-axis direction. This is obtained by rotating the original coordinate system to a new coordinate system with angle θ. Let cos(θ) E= − sin(θ)
sin(θ) cos(θ)
(5.18)
2151_book.fm Page 96 Friday, October 20, 2006 5:04 PM
96
Stream of Variation Modeling and Analysis for MMPs
Clearly, E is an orthogonal matrix. Denote the preceding relationship in the matrix form: Y1 cos(θ) = Y2 − sin(θ)
sin(θ) X1 ⋅ = E⋅X cos(θ) X2
(5.19)
From Figure 5.1, the first PC, Y1, exhibits the maximum variance and the second PC, Y2 exhibits the second largest variance. So by choosing the linear combination with the maximum dispersion, most variation in X1, X2 can be explained. If the ellipse is very flat, then the total sample variation can be approximated by the dispersion in the Y1 direction, and thus the effect of Y2 on total variation can be ignored. In this sense, the one-dimensional PC is obtained by reducing the original 2-D data. In general, p variables Xi, i = 1, 2, …, p span a p-dimensional space, and n samples can be viewed as n points in this p-dimensional space. Then, selecting p PCs is equivalent to identifying the p pivotal axes of the p-dimensional ellipse.
5.1.3 INFERENCES
ON
PCS
In this subsection, we will consider two types of inferences in which normal distribution assumption about X ~ N p (µ , ∑) is required: 1. Statistical testing for independence of the original variables, i.e., H0: R = I, where R is the correlation matrix of multinormal variables X. For a large sample size n, a likelihood ratio test for H0 is given by p
|R| =
∏ λˆ
i
(5.20)
i=1
where λˆ i , i = 1, 2, …, p, are eigenvalues of correlation matrix R. H0 is rejected if:
( )
− m ln R > χ 2p( p−1)/ 2 (α)
5.21)
where m = n − 1−
2p +5 . 6
It should be noted that PCA can only be performed if H0 is rejected. 2. Statistical testing for isotropy for eigenvalues, i.e., H0: λq+1 = λq+2 = … = λp = λ, which can be used to determine the number of PCs.
2151_book.fm Page 97 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
97
Equation 5.10 or Equation 5.11 can be used to determine the number of PCs, i.e., to choose the first few PCs to replace the original p (usually very large) variables without much loss of information. Here, an alternative method is given based on the testing of the isotropy: H0: λ q +1 = λ q +2 = ... = λ p = λ
(5.22)
H a : λ i ≠ λ for at least one i = q + 1, q + 2, ..., p where λ is an unknown constant. If H0 is not rejected, then in addition to the first q PCs, all the remaining PCs must be added if any one of the PCs is added, because they are isotropic. The likelihood ratio test for H0 is based on the statistic: 1/( p − q )
p λˆ i i=q +1 Q= p 1 λˆ i p − q i=q +1
∏
∑
(5.23)
where λˆ i is an eigenvalue of the sample covariance matrix S, i =1, 2, …, p. Lawley [3] showed that 2 1 Q * = − n − 1 − q − 2( p − q) + 1 + ⋅ ln(Q ) 6 p − q
(5.24)
is asymptotically distributed as χ 2f (α ) where f =
1 ( p − q )( p − q + 1) − 1 . 2
So H0 is rejected if Q* > χ 2f (α ). When H0 is rejected, more than q PCs are needed. If H0 is not rejected, then at the significance level α, the first q PCs are enough to replace the p original variables without much loss of information.
5.1.4 APPLYING PCS
TO
PROCESS CONTROL
It is known from previous chapters that quality ellipses and T 2 charts can be used as multivariate control charts. With a large number of variables, PCs can be used to construct a control chart.
2151_book.fm Page 98 Friday, October 20, 2006 5:04 PM
98
Stream of Variation Modeling and Analysis for MMPs
Consider data matrix x11 x 21 x= x n1
x12
x 22
xn 2
x1 p x2 p x np
with (µ , ∑). The first two sample PCs yˆ1 and yˆ2 are considered because they account for the largest cumulative proportion of the total sample variance and can be visualized easily. Richard Johnson [4] suggested two types of control charts: 1. Ellipse format chart based on the first two PCs for large n: The first two PCs yˆ1 and yˆ2 are independent, so the control ellipse is constructed by: yˆ12 yˆ22 + ≤ χ 22 (α ) ˆλ λˆ 2 1
(5.25)
This ellipse can be used to check the stability of measurements. If there are out-of-control points, special causes may exist in the process. 2. T 2 chart based on the remaining PCs: Assume X − µ ∼ N p (0, ∑). Let Yi, i = 1, 2, …, p, be centered PCs with mean 0. A T 2 chart is constructed from the residual information not contained in the first two PCs. Based on the last p – 2 PCs, the statistic p
∑λ
Y j2
j =3
~ χ2p−2 .
j
As Yˆj , j = 3, 4, ..., p, λˆ j
are independent and normally distributed, the T 2 chart can be created based on the statistic p
Ti 2 =
∑ λˆ j =3
yˆij2 j
2151_book.fm Page 99 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
99
and the upper control limit (UCL) can be defined as UCL = χ2p−2 (α ) . This chart can be used to identify special causes of variation.
5.2 FACTOR ANALYSIS 5.2.1 INTRODUCTION In many areas such as psychology, sociology, and health care, it is not always possible to directly measure concepts of major interest such as intelligence, satisfaction, or quality of life. In the manufacturing and service industries, the quality of a product or service is the critical issue and is usually affected by many direct or indirect factors. The effective identification and interpretation of these common factors become the key to the success. The early stages of most statistical investigations in the categories listed earlier are characterized by little prior quantitative information being available about the area of study. Consequently, early research efforts are hit-and-miss by nature — many variables are studied in the hope of identifying some kind of order that will deepen understanding of the subject. Although correlation analysis can be used to identify the interrelationship among variables, practical problems arise when we have to deal with a large number of variables. The problem is that the number of coefficients keeps accelerating as the number of variables increases. So, a higherorder data reduction technique is required that can systematically summarize large covariance or correlation matrices. Factor analysis is one of the data reduction techniques. It is a family of procedures for removing redundancy from a set of correlated variables and representing the variables with a smaller set of “derived” variables, or factors. Alternatively, the factor analysis procedure can be thought of as removing duplicated information from a set of variables. We may also think of it loosely as a group of similar variables, although these variables are often unobservable.
5.2.2 COMPARISON
OF
FACTOR ANALYSIS
AND
PCA
Factor analysis and PCA are functionally very similar and can be used for data dimension reduction. However, they are quite different in terms of underlying assumptions. PCA is concerned with explaining the variance-covariance structure of a set of variables (XT = [X1 X2 Xp]) through a few linear combinations of these variables (YT = [Y1 Y2 Yp]). The PCs are those uncorrelated linear combinations that can be defined as Y = UX. Algebraically, PCs (Yi’s) are particular (uncorrelated) linear combinations of the random variables (Xi’s). Geometrically, these linear combinations represent the selection of a new coordinate system obtained by rotating the original system with U. Therefore, PCA is a method for analyzing the structure of the data covariance, but not for modeling the relationship between data and some latent variables. Factor analysis assumes that the variance of a single variable can be decomposed into a common variance that is shared by other variables included in the model, and
2151_book.fm Page 100 Friday, October 20, 2006 5:04 PM
100
Stream of Variation Modeling and Analysis for MMPs
a specific variance that is unique to a particular variable, including the error component. Factor analysis analyzes only the common variance of the observed variables, whereas PCA considers the total variance and makes no distinction between common and unique variances [5]. Stevens [6] described two criteria that could be used to choose one technique over the other: •
•
The objective of the analysis: Factor analysis and PCA are similar as the purpose of both methods is to reduce the original variables into fewer composite variables, called factors or PCs. However, they are distinct because the obtained composite variables serve different purposes. In factor analysis, a small number of factors are extracted to account for the intercorrelations among the observed variables, i.e., to identify the latent variables that explain why they are correlated with one another. The objective of PCA is to account for the maximum portion of the variance present in the original set of variables with a minimum number of composite variables, called PCs. The assumption about the variance in the original variables: If the observed variable measurements are relatively error free (for example, age, years of education, or number of family members), or if it is assumed that the error and specific variance represent a small portion of the total variance in the original set of variables, then PCA is appropriate. But if the observed variables are only indicators of the latent structures to be measured (such as test scores or responses to attitude scales), or if the error variance represents a significant portion of the total variance, then factor analysis will be the more appropriate technique.
5.2.3 ORTHOGONAL FACTOR MODEL Factor analysis is a multivariate analysis technique aiming at reducing the number of variables while preserving as much original information as possible. In other words, it is used to uncover the latent structure (dimensions) of a set of variables. The essential purpose of factor analysis is to describe the covariance relationship among many variables in terms of a few underlying, but unobservable, and random quantities. The factor model is motivated by the argument that variables can be grouped according to their correlations. That is, suppose all variables within a particular group are highly correlated among themselves, but have relatively small correlations with variables in a different group. Then it is conceivable that each group of variables represents a single underlying factor that is responsible for the observed correlations. We suppose X1, X2, …, Xp are p observable random variables. Remember that if these p variables do not have the same unit, standardization should be done to avoid unreasonable results. The factor model postulates that these variables are linearly dependent on a few unobservable random variables F1, F2, …, Fm, and p additional sources of variation ε1, ε2, …, εp, which cannot be explained by Fj , j = 1, 2, …, m. In particular, the factor analysis model in a matrix notation is:
2151_book.fm Page 101 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
X1 a11 X2 = a21 : X p a p1
a12 a22
... ...
a p2
...
a1m F1 ε1 a2 m F2 ε 2 + ... a pm Fm ε p
101
(5.26)
or X p×1 = A p× m Fm×1 + ε p×1
(5.26′)
where F1, F2, …, Fm are called common factors and ε1, ε2, …, εp are called residual errors or specific factors. The coefficient aij is called the loading of the ith variable on the jth factor. So the matrix A is the loading matrix of factor analysis. Because the p variables, X1, X2, …, Xp, are expressed in terms of p + m unobservable random variables, F1, F2, …, Fm and ε1, ε2, …, εp, it is impossible to give a direct verification of the factor model from the observations on X1, X2, …, Xp with so many unobservable quantities. Thus, additional assumptions on F1, F2, …, Fm and ε1, ε2, …, εp must be made to simplify the covariance relationships: 1. m ≤ p. 2. E(εε) = 0, ∑ ε = Cov(ε ) = diag σ12 , σ 22 ,…, σ 2p , i.e., ε1, ε2, …, εp are not correlated and the variances are different. 3. Cov(F, ε ) = 0, i.e., F and ε are uncorrelated. 4. E(F) = 0, ∑ F = Cov(F) = diag 1, 1, …, 1 = Im , i.e., the F1, …, Fm are uncorrelated and their variances are 1.
(
)
(
)
The preceding assumptions make the model Equation 5.26 an orthogonal factor model. Remark 5.5: The models of Equation 5.26 or Equation 5.26′ give the objective of factor analysis, i.e., to use F as a surrogate of X such that the dimension of the variable space is reduced from p to m. Remark 5.6: In PCA, the coefficient matrix A is determined and unique. But in factor analysis, the loading matrix A is not unique, because for any m × m orthogonal matrix Γ , Equation 5.26′ can be rewritten as: X p×1 = ( A p× m Γ m× m )(Γ Tm× m Fm×1 ) + ε p×1
(5.26″)
It is easy to verify that Equation 5.26″ still satisfies the assumptions, i.e., ∑ Γ T F = Cov(Γ T F) = Im , Cov(Γ T F, ε ) = 0 So Γ Tm× m Fm×1 is also a common factor, and A p× m Γ m× m is the loading matrix.
2151_book.fm Page 102 Friday, October 20, 2006 5:04 PM
102
Stream of Variation Modeling and Analysis for MMPs
Remark 5.7: This nonuniqueness provides a nice property. When the structure of A is complex, it can be simplified by an appropriate transformation such that the new factors have meanings that can be better interpreted. Unlike the coefficients in a regression model, the common factor F is unobservable in the factor analysis model.
5.2.4 STATISTICAL INTERPRETATIONS AND COMMUNALITY
OF
FACTOR LOADINGS
5.2.4.1 Interpretation of Factor Loadings Under assumptions listed in the previous section, we know that E(F) = 0, σ i2 = var(εi) = 1, µ = E(X) = 0, and var( Xi ) = σ 2ii , i= 1, 2, …, p. The orthogonal factor model given in Equation 5.26 implies a covariance structure for X: cov(X, F) = E (X − µ )(F − 0)T = E ( AF + ε )FT = A ∑ F + E (εFT ) = A ∑ F = A or cov( Xi , F j ) = aij
(5.27)
So the factor loadings are the covariance coefficients between the observable variables and factors. Similarly, if the data X is standardized, i.e., var(Xi) = σii2 = 1, factor loadings are the correlation coefficients between the observable variables and factors, i.e., ρ( Xi , F j ) = aij . Define p
Sj =
∑ a , j = 1, 2, ..., m 2 ij
i =1
where the squared factor loading aij2 is the percentage of variance in variable Xi accounted for by the factor Fj. Sj is the contribution of Fj to X, i.e., the sum of variance is contributed by Fj to all X. To obtain the percentage of variance in all the variables explained by each factor, the sum of the squared factor loadings for that factor is calculated and then divided by the number of variables. 5.2.4.2 Interpretation of Communality As mentioned earlier, we have: ∑ X = Cov(X) = E (X − µ )(X − µ )T = AAT + ∑ ε or 2 σ 2ii = var( Xi ) = ai21 + ... + aim + σ i2 , Cov ( Xi , X j ) = ai1a j 1 + ... + aim a jm
(5.28)
2151_book.fm Page 103 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
103
The portion of the variance of the ith variable var(Xi) contributed by the m 2 common factors, i.e., ai21 + ... + aim , is called the ith communality. The portion of var(Xi) due to the specific factor, i.e., σ 2i , is called the uniqueness or specific variance. Thus, the variance of the ith variable can be decomposed into two parts as follows: var(Xi) = communality + specific variance, i = 1, 2, …, p
(5.29)
2 Denoting the ith communality by hi2 = ai21 + ... + aim , the ith communality is the sum of squares of the loadings of the ith variable on the m common factors. Hence, from Equation 5.28, the “standardization” yields:
σ 2ii = hi2 + σ i2 = 1
(5.29)
If hi2 ≈ σ ii2 , then almost all the original information contained in the ith variable Xi is extracted by common factors, and thus hi2 is critical component of var(Xi). However, if hi2 ≈ 0, then the common factors have a small impact on Xi, and the specific factor ε i dominates the Xi. So the communality measures the percentage of variance in a given variable accounted for by all the factors.
5.2.5 ESTIMATION
OF
LOADING MATRIX
Suppose we are given n observations x1, x2, …, xn on p generally correlated variables. To construct the factor analysis model, we need to estimate the factor loading matrix A based on these observations. There are some popular methods of parameter estimation, such as, the principal component method and the maximum likelihood method. Here, we will only concentrate on the first method. The MLE method can be seen in Reference 4. Let ∑ X = Cov(X) have eigenvalue λ1 ≥ λ 2 ≥ ... ≥ λ p ≥ 0 and associated normalized eigenvectors ei , i = 1, 2, …, p then:
p
∑X =
∑λ e e
T i i i
i =1
= λ1 e1
λ eT 1 1 λp ep T λ p e p
(5.30)
If m = p, then from Equation 5.28 and Equation 5.30, ∑ X = AAT + 0 = AAT
(5.31)
A comparison of Equation 5.30 and Equation 5.31 yields A = λ1 e1
λpep
(5.32)
2151_book.fm Page 104 Friday, October 20, 2006 5:04 PM
104
Stream of Variation Modeling and Analysis for MMPs
If m < p and the last p – m eigenvalues are small, we neglect the contribution of p
∑λee
T i i i
to ∑ X ;
i = m +1
then:
∑ X ≈ λ1 e1
λ1 e1T λ m em T λ m e m
(5.33)
λ1 e1T λ m e m + ∑ ε = AAT + ∑ ε T λ m e m
(5.34)
If we consider the specific factors, then:
∑ X ≈ λ1 e1
To use this method in a data set x1, x2, …, xn, if the variables do not have the same unit, it is first customary to standardize the data by x ji − xi
, j = 1, 2, …, n, i = 1, 2, …, p
(5.35)
sii If ∑ X is unknown, we can use the sample variance-covariance matrix S obtained from the standardized data. So the estimated factor loading matrix is given by ˆ = λˆ eˆ A 1 1
λˆ m eˆ m
(5.36)
where λˆ 1 ≥ λˆ 2 ≥ ≥ λˆ p are eigenvalues of sample covariance matrix S (or correlation matrix R), and eˆ i , i = 1, 2, …, p, are the associated normalized orthogonal eigenvectors of S or R. The estimated specific variances are provided by the diagonal ˆ = diag σˆ 2 , σˆ 2 , …, σˆwith ˆ ˆ T and, thus, ∑ elements of the matrix S − AA ε m +1 m+2
(
m
σˆ 2i = sii −
∑ aˆ
ij
2
.
j =1
2 The communalities are estimated by hˆi2 = aˆi21 + ... + aˆim .
2151_book.fm Page 105 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
105
Fk
~ Fk
kj
Fj ~ Fj
FIGURE 5.2 Orthogonal rotation of common factors.
5.2.6 FACTOR ROTATION The objective of factor modeling is not only to find the common factors Fj, j = 1, 2, …, m, and group the original variables Xi, i = 1, 2, …, p, but also to make each common factor more interpretable and meaningful. Thus, factor rotation is usually necessary to facilitate the interpretation of factors. The nonuniqueness of a loading matrix makes the rotation feasible. Through factor rotation, the structure of the loading matrix is simplified in the sense of making each variable have larger loading only on one common factor, and smaller or moderate loading on other common factors. There are two categories of rotational methods to choose from: orthogonal rotation or oblique rotation. Oblique rotations allow the factors to be correlated, and so a factor correlation matrix is generated when oblique is requested. Normally, however, with orthogonal rotation, the axes are kept perpendicular as they are rotated and no factor correlation matrix is produced, as the correlation of any factor with another is zero. In this subsection, the commonly used maximum variance orthogonal rotation is introduced. The oblique rotation methods is illustrated in Figure 5.2 and can be found in Reference 4. We start with m = 2. Let the factor loading matrix a11 a21 A= a p1
a12 a22 . a p 2
To avoid imbalance caused by communality hi2 = ai21 + ai22 , i = 1, 2, …, p, the ith row of this loading matrix is preprocessed by dividing hi2 for i = 1, 2, …, p. For simplification, we still use A. Let cos ϕ T= sin ϕ
− sin ϕ cos ϕ
2151_book.fm Page 106 Friday, October 20, 2006 5:04 PM
106
Stream of Variation Modeling and Analysis for MMPs
be an orthogonal matrix and define the new matrix a11 cos ϕ + a12 sin ϕ a21 cos ϕ + a22 sin ϕ B = AT = a p1 cos ϕ + a p 2 sin ϕ
− a11 sin ϕ + a12 cos ϕ − a21 sin ϕ + a22 cos ϕ − a p1 sin ϕ + a p2 cos ϕ
b11 b 21 b p1
b12 b22 (5.37) b p 2
The rotation angle ϕ is selected by making the structure of loading matrix A as simple as possible. In other words, we hope to divide the variables Xi , i = 1, 2, …, p into two categories, one related to F1 and the other related to F1. This is achieved by making the two variances V1 and V2 as large as possible, where V1 and V2 are obtained from the two column vectors of B, respectively. Thus, we select rotation angle ϕ such that ϕ = arg max(V ) = arg max(V1 + V2 ) ϕ
(5.38)
ϕ
where
1 Vj = p
p
∑ i =1
1 (b ) − p 2 2 ij
Using the principle of maximization
p
∑ i =1
2
b , j = 1, 2. 2 ij
dV = 0, we have: dϕ D−
tan( 4ϕ) =
1 (2 AB) p
(5.39)
1 C − ( A2 − B2 ) p
where p
A=
∑
p
αi , B =
i =1
∑
p
βi , C =
i =1
2
∑ i =1
2
p
(α 2i − β2i ) , D = 2
∑ α β , and i
i
i =1
a a a a α i = i1 − i 2 , βi = 2 i1 i 2 for i = 1, 2, …, p hi hi hi hi
2151_book.fm Page 107 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
107
TABLE 5.1 The Range of Rotation Angle ϕ Sign of Nominator
Sign of Denominator
+
+
+
–
Range of 4 ϕ
π
+
–
8 π
~π
–π ~ − π
π
0~
2
–
–
π
0~
2 –
Range of ϕ
~
8 π
−
2
−
~0
2
π 4
π 4 π
~−
π 8
~0
8
The range of ϕ is determined according to the signs of denominators and nominators of tan(4ϕ), which are summarized in Table 5.1. If there are m > 2 common factors, we need to conduct the preceding orthogonal rotation on each pair of common factors, i.e., orthogonally rotate the factor space span {Fk, Fj} with ϕ kj for k = 1, 2, …, (m – 1) and j = k + 1, …, m. Each ϕ kj is determined by rotating a11 a21 A= a p1
a12 a22
... ...
a p2
...
a1m a2 m ... a pm
to b11 b21 B = ATkj = b p1
b12 b22 bp2
... ... ...
b1m b2 m ... b pm
by maximizing m
m
V=
1 p
p
∑ V =∑ ∑ j
j =1
j =1
i =1
1 (b ) − p 2 2 ij
p
∑ i =1
b 2 ij
2
.
2151_book.fm Page 108 Friday, October 20, 2006 5:04 PM
108
Stream of Variation Modeling and Analysis for MMPs
where Tkj is the fundamental rotation matrix, and bik = aik cos ϕ + aij sin ϕ bij = − aik sin ϕ + aij sin ϕ
(5.40)
bil = ail where i = 1, 2, …, p and j, k, l = 1, 2, …, m. The rotation angle ϕ is determined as before. m For m factors, the total number of rotations is ; thus, the rotated loading 2 matrix is given by: B(1) = AT12 T1m T(m −1)m = AC1
(5.41)
where C1 = T12 T1m T(m −1)m . Let V(1) be the variance of B(1). Continuing the earlier procedure, we get the sequences: B(1) , B( 2) , B(3) … and
(5.42)
V(1) ≤ V(2 ) ≤ V(3) ≤
(5.43)
The sequence has an upper bound because the absolute value of factor loading cannot exceed 1. Thus lim V( k ) exists; let k →∞
V * = lim V( k ) . k →∞
For a given precision δ, when k is sufficiently large, | V( k ) − V * |< δ
(5.44)
In practice, if | V( k ) − V( k +1) | < ε (ε > 0 is a small number), then we stop the rotation and take B( k ) = AC1C 2 C 3 C k = AC
(5.45)
as the factor loading matrix. Remark 5.8: The sum of eigenvalues is not affected by rotation, but the rotation will alter the eigenvalues of particular factors and will change the factor loadings.
2151_book.fm Page 109 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
109
Because multiple rotations may explain the same variance (have the same total eigenvalue) but have different factor loadings, and because factor loadings are used to intuit the meaning of factors, this implies that different meanings may be ascribed to the factors depending on the rotation — a problem some cite as a drawback of factor analysis.
5.2.7 FACTOR SCORE The factor model is given in Equation 5.26, Xi = ai1F1 + ai 2 F2 + aim Fm, i = 1, 2, …, p The common factors Fj , j = 1, 2, …, m, reflect the relationship between original variables Xi , i = 1, 2, …, p. Usually, it is convenient to study the characteristics of the problem using the common factors in place of original variables. Suppose common factors can be linearly represented by the original variables, i.e., F j = β j 1 X1 + β j 2 X2 + + β jp X p , j = 1, 2, …, m
(5.46)
Equation 5.46 is called the factor score function and can be used to compute the common factor score for each sample. For example, when m = 2, we can plug in the p variables of each sample and compute F1 and F2. The scatter plot of F2 vs. F1 can be used to classify the samples or conduct further analysis. When m < p, the factor scores cannot be precisely computed. There are many methods, such as, weighted least-squares regression, that can be used to estimate the factor scores. Here, Thomson’s [7] regression method is introduced. Suppose the original variables and common factors have been standardized. If common factors Fj , j = 1, 2, …, m, can be regressed on Xi , i = 1, 2, …, p, then the regression equation can be expressed as: Fˆ j = b j 1 X1 + b j 2 X2 + + b jp X p , j = 1, 2, …, m
(5.47)
where the common factors are unknown and need to be estimated. From Equation 5.27 and Equation 5.47, for i = 1, 2, …, p and j = 1, 2, …, m, we can obtain: aij = cov( X i , Fj ) = E ( X i Fj ) = E[ X i (b j1X1 + b j 2 X 2 + + b jp X p )] = b j1E ( X i X1 ) + b j 2 E ( X i X 2 ) + + b jp E ( X i X p ) = b j1 cov( X i , X1 ) + b j 2 cov( X i , X 2 ) + + b jp cov( X i , X p ) = b j1ri1 + b j 2ri 2 + + b jprip
2151_book.fm Page 110 Friday, October 20, 2006 5:04 PM
110
Stream of Variation Modeling and Analysis for MMPs
i.e., b j 1r11 + b j 2 r12 + + b jp r1 p = a1 j b j 1r21 + b j 2 r22 + + b jp r2 p = a2 j b j 1rp1 + b j 2 rp 2 + + b jp rpp = a pj
(5.48)
In matrix notation, Equation 5.48 can be expressed as: Rb j = a j
(5.49)
where b j = (b j 1, b j 2 ,..., b jp )T , a j = ( a1 j , a2 j ,..., a pj )T , and R is the correlation matrix. By solving Equation 5.49, we get: b j = R −1a j
(5.50)
Let b1T b11 T b b21 B= 2 = T b m bm 1
b12
b22
bm 2
b1 p b2 p bmp
Then: (R −1a1 )T −1 T (R a 2 ) B= = A T R −1 −1 T (R a m )
(5.51)
Thus, from Equation 5.47 and Equation 5.51, the common factors can be estimated by: Fˆ1 b T X 1T b X Fˆ Fˆ = 2 = 2 = BX = A T R −1X Fˆm b Tm X
(5.52)
2151_book.fm Page 111 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
5.2.8 GENERAL PROCEDURES
FOR
111
FACTOR ANALYSIS
The general procedures for factor analysis are summarized as follows: Step 1: Standardize original data if the variables do not have the same unit. Step 2: Derive the covariance matrix S or correlation matrix R. Step 3: Derive the eigenvalue and eigenvector pairs of S or R to get the factor loadings according to the cumulative contribution of the eigenvalues. Step 4: Rotate the factor loading. Step 5: Name and interpret the factors.
5.3 EXERCISES 1. It is given that the covariance matrix of the three (p = 3) standardized random variables Z1, Z2, and Z3 is 1.0 ρ = .07 .63
.07 1.0 .09
.63 .09 1.0
a. In order to generate Z1, Z2, and Z3 by an m = 1 factor model, list the necessary assumptions regarding ε ( ε = [ε1 ε2 ε3]T) and F1. b. Show that, based on the assumptions listed in (1), Z1, Z2, and Z3 can be generated by Z1 = .7F1 + ε1
.51 Z 2 = .1F1 + ε 2 and Ψ = cov(ε ) = 0 0 Z 3 = .9F1 + ε 3
0 .99 0
0 0 . .19
That is, write ρ in the form ρ = LLT + Ψ. 2. The eigenvalues and eigenvectors of the correlation matrix ρ in Problem 1 are λ1 = 1.64 , e1 = [.6953
.1716
.6980 ]T
λ 2 = .98 , e2 = [.1375
−.9849
.1052 ]T
λ 3 = .37 , e3 = [.7055
.0228
−.7084]T
2151_book.fm Page 112 Friday, October 20, 2006 5:04 PM
112
Stream of Variation Modeling and Analysis for MMPs
a. Assuming an m = 2 factor model, calculate the loading matrix L, and specific variance Ψ using the principal component solution method. b. What proportion of the total variance is explained by the m = 2 common factors? 3. Using the basic factor analysis model and assumptions, proof the following statements: a. If T is an m × m orthogonal matrix, then cov(TTF) = I and cov(TTF,εε) = 0. b. The factor model is X− µ = L
( p×1)
( p×1)
F + ε (m < p),
( p× m ) ( m ×1)
( p×1)
let L* = LT and F* = T T F, where T is an m × m orthogonal matrix, then cov(X) = L*L*T + Ψ. c. If the factor score is given by Fˆ = LT ∑ −1 (X − µ ), where L is estimated from covariance matrix ∑ by the principal component decomposition method (i.e., L = [ λ 1 e1 λ 2 e2 λ m em]), then E(Fˆ ) = 0 and Cov(Fˆ ) = I . Hint: i. cov(A) = E[(A – E(A)A – E(A))(A – E(A))T], where E(A) is the expectation of matrix A. ii. L and ∑ are deterministic, i.e., E(L) = L and E(∑) = ∑. iii. {λj, ej}, j = 1, 2, …, m are the eigenvalue and eigenvector pairs of ∑, which has the property that λj ej = ∑ej for j = 1, 2, …, m. 4. Consider an assembly process. A side panel of a car body is located on the fixture through a 4-way pin P1 and a 2-way pin P2. To monitor the side panel’s quality, ten points are measured and their nominal positions are shown in Figure 5.3. In this case, only three potential sources of variation are considered: a. The x direction variation of P1 b. The z direction variation of P1 c. The z direction variation of P2
FIGURE 5.3 Product and process description.
2151_book.fm Page 113 Friday, October 20, 2006 5:04 PM
Principal Component Analysis and Factory Analysis
113
The process is simulated by deliberately introducing process variation sources. The measurement data of points M1, M2, M5, and M6 are collected and given in the attached data file (Problem 4 dataset.xls). Use the factor analysis method to identify the introduced process variation sources. Hint: You may follow the following procedures to solve this diagnosis problem: i. Derive the data covariance S or correlation R matrix. Indicate the reason why you have chosen one (either S or R) rather than the other. ii. Derive the eigenvalue/eigenvector pairs from the matrix you selected in (i). iii. Determine the number of common factors. Indicate the criteria for your decision. iv. Derive the factor loadings for the common factors and try to interpret and link them with potential process variation sources considered. v. Are the factor loadings in (iv) interpretable? If not, try to improve the interpretability of the factor loading by deriving the new loading vectors. Indicate the method and criteria for achieving your new factor loadings. Interpret the new factor loadings, and figure out the variation sources introduced.
REFERENCES 1. Hotelling, H., Analysis of a complex of statistical variables into principal components, Journal of Educational Psychology, 24, 417–441,498–520, 1933. 2. Rao, C.R., The use and interpretation of principal component analysis in applied research, Sankhya A, 26, 329–358, 1964. 3. Lawley, D.N., Tests of significance for the latent roots of covariance and correlation matrices, Biometrika, 43, 128, 1956. 4. Johnson, R.A. and Wichern, D.W., Applied Multivariate Statistical Analysis, 4th ed., Prentice Hall, Upper Saddle River, NJ, 1998. 5. Morrison, D.F., Multivariate Statistical Methods, McGraw-Hill, New York, 1990. 6. Stevens, J., Applied Multivariate Statistics for the Social Sciences, 4th ed., Lawrence Erlbaum Associates, Mahwah, NJ, 2001. 7. Thomson, G.H., The Factor Analysis of Human Ability, Houghton Mifflin Company, New York, 1939. 8. Darlington, R.B., Weinberg, S., and Walberg, H., Canonical variate analysis and related techniques, Review of Educational Research, 453–454, 1973. 9. Rubenstein, D.I., Ecological Aspects of Social Evolution: Birds and Mammals, Princeton University Press, Princeton, NJ, 1986.
ADDITIONAL READING Darlington, R.B., Weinberg, S., and Walberg, H., Canonical variate analysis and related techniques, Review of Educational Research, 453–454, 1973. Rubenstein, D.I., Ecological Aspects of Social Evolution: Birds and Mammals, Princeton University Press, Princeton, NJ, 1986.
2151_book.fm Page 114 Friday, October 20, 2006 5:04 PM
2151_book.fm Page 115 Friday, October 20, 2006 5:04 PM
Part II Variation Propagation Modeling in MMP
2151_book.fm Page 116 Friday, October 20, 2006 5:04 PM
2151_book.fm Page 117 Friday, October 20, 2006 5:04 PM
6
State Space Modeling for Assembly Processes
This chapter develops a state space model characterizing variation propagation in a multistage rigid-part assembly process. First, the background of a multistage assembly process is briefly described using automotive body assembly processes. Major variation sources are then identified and modeling assumptions are summarized, on which a station-indexed state space model is subsequently developed. The state space model for depicting stream of dimensional variation is actually a result of recursively applying standard kinematic analysis to individual operation stages. However, the state space model is not a simple summation of multiple single-stage models, because the across-stage transitions will affect how variation propagates and complicate the dynamics of the entire system. The model is validated through a comparison with commercially available variation simulation software.
6.1 INTRODUCTION OF MULTISTAGE ASSEMBLY PROCESSES A multistage assembly process involves multiple stations or operations to assemble individual components into a sophisticated product. Typical multistage assembly processes include automotive assembly, printed circuit board (PCB) assembly, and aircraft fuselage assembly, to name a few. We will use the car body assembly process to highlight the features of this method. The major part of a car body is the body-in-white (BIW), namely, the structural skeleton of a car. Figure 6.1 shows an example of the BIW. The BIW provides a physical frame on which closure panels, including doors, hood, fenders, and lift gate or trunk, will be mounted. The finished BIW will be measured in a coordinate system as shown in Figure 6.1. This coordinate system is known as the body coordinate system in the automotive industry. A BIW is usually made of 100 to 150 sheet metal panels, which are assembled by a process involving 55 to 75 assembly stations [1]. Figure 6.2 shows the corresponding process layout for producing the BIW in Figure 6.1. The BIW assembly process starts with the component subassembly process, in which a sequence of assembly stations builds up a set of large subassemblies, including the left-hand side aperture (LH-APT), the right-hand side aperture (RH-APT), and the underbody. These subassemblies will then be fed into a framing station, where industrial robot welders will joint them together to form the BIW. Figure 6.2 also marks the optical coordinate measuring machine (OCMM) at a few different locations. OCMM stations are where the key dimensional features of a finished or semifinished car are measured in the production line. In a modern assembly system, OCMM stations are 117
2151_book.fm Page 118 Friday, October 20, 2006 5:04 PM
118
Stream of Variation Modeling and Analysis for MMPs
FIGURE 6.1 Example of an automotive body (body-in-white) and inline sensing using OCMM.
OCMM
Station #2
Left Hand Aperture Assembly
Station #1 Framing
OCMM Right Hand Aperture Assembly Station #5
Station #4
Station #3
Station #6 Underbody Assembly
OCMM
OCMM: Optical Coordinate Measuring Machine
Body-In-White
OCMM
FIGURE 6.2 BIW assembly layout (arrows represent a part input/output at each assembly station).
commonly distributed along the assembly line, instead of being implemented only at the end of a production line. The taxonomy of this treelike representation is the exact graphical equivalent of the physical process. More details can be found in [1]. On each assembly station, part assembly is initiated in two or more sets of fixture locators that aim to constrain the degrees of freedom of a part during a joining operation to ensure a repeatable build. Figure 6.3 illustrates a typical 3-2-1 fixture set used in car body assembly processes. It consists of two locating pins, LP4-way and LP2-way , and three net contact blocks, NCi, i = 1, 2, 3. The two locating pins constrain three degrees of freedom in the x-z plane, where the 4-way pin controls part motion in both the x and z directions and the 2-way pin controls part motion in the z direction. Three NC blocks constrain other degrees of freedom of the workpiece. When a workpiece is nonrigid, more than three NC blocks may be needed to reduce part deformation. An n-2-1 fixture layout, denoted by {LP4-way , LP2-way , NCi , i = 1, 2, .., n}, is a more generic setting in multistage assembly processes. For notation simplicity, unless otherwise indicated, we will also use LP1 to denote a 4-way locator and LP2 for a 2-way locator on a part. Hence, for a 2-D part, we can use the simplified notation {LP1, LP2} to denote the fixture locator pair used to position the part.
2151_book.fm Page 119 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
119
NC 3
LP 2way NC 2
LP 4way NC1
workpiece
y x z FIGURE 6.3 Illustration of a 3-2-1 fixture.
6.2 VARIATION FACTORS AND ASSUMPTIONS Dimensional variation is a major problem affecting product quality in multistage assembly processes. Previous studies have shown that dimensional problems contribute to roughly two thirds of all quality-related problems during new product launches [2,3]. In a multistage system, the major variation factors can be generally classified into those on individual stations and those induced by across-station transitions.
6.2.1 STATION-LEVEL VARIATION FACTORS One of the major factors for dimensional quality of the finished product at the station level is the repeatability of fixture locators used to hold the parts, generally referred to as fixture variation. Fixture locators are used extensively in processes such as car body assembly to provide physical support and dimensional reference within the body coordinate system, thereby determining the dimensional accuracy of the final assembly [3,4]. Fixture locators may fail to provide the desired positioning repeatability (relative to tolerances) during production owing to gradual deterioration of locators and catastrophic events, such as broken or deformed locators. Figure 6.4 shows an example of fixture fault manifestation, where δLP2(z) is a z-directional fixturing deviation caused by the malfunctioning pinhole at LP2. This deviation will affect the positions of the measurement points SPi, i = 1, 2, 3. Another variation factor is the manufacturing imperfection of individual parts, such as deviations in part dimensions or geometrical features from their design,
sensor andSP i : sensor position fixture locators
SP 2
SP 1
z
LP 2 x
workpiece: a panel
LP 1
LP 2(z)
LP 1(z) LP 1(x)
4-way locator, positioning variability in two directions
LP 2(z)
SP 3
FIGURE 6.4 Fixture-locator-induced dimensional variation.
2-way locator, positioning variability in one direction
2151_book.fm Page 120 Friday, October 20, 2006 5:04 PM
120
Stream of Variation Modeling and Analysis for MMPs mating feature Part 1
(a) Lap joint
Part 2
Part 1
Part 2
(b) Butt joint
Part 1
Part 2
(c) Mixed joint
Defined direction
FIGURE 6.5 Cross-sectional views of joint geometries.
generally referred to as part variation. Part variation is actually a result of the processes fabricating the parts: as the manufacturing process is inherently imperfect, part variation is unavoidable. Nevertheless, whether part variation affects the downstream process or not depends on what type of part-to-part joint is involved. In car body assembly processes, Ceglarek and Shi [5] reported that there are two basic important types of joints: one is the lap joint (Figure 6.5a) and the other is the butt joint (Figure 6.5b). The lap joint can absorb variation in the direction of the slip plane and the butt joint can be used as a datum to locate another part. Therefore, variation will propagate in the defined direction through the mating feature only when using butt joints. Figure 6.5c shows a generic joint geometry that can partially allow motion in the defined direction. The third variation factor is related to the flexibility of a part. If a part is flexible, such as the metal panels of a car body, it can deform during the assembly operation owing to incorrect locator positions, uneven forces, or part-to-part interferences. This deformation will cause additional dimensional variability in the finished product. In this chapter, we will assume that rigid body parts are used in assembly operations.
6.2.2 ACROSS-STATION VARIATION FACTORS Shiu et al. [6] discovered a phenomenon called reorientation in a multioperation process, when a change in fixture layouts occurs as the subassembly proceeds to a new station. Figure 6.6 illustrates the reorientation phenomenon, in which the dashed line represents the nominal position of parts and the solid line indicates the actual position. Assume that positioning errors are associated with fixture locators on station k, but not with those on station k + 1. Then, when part 1 and part 2 are assembled on station k, there is some relative position deviation between these two components; i.e., if part 1 is positioned according to its nominal position, part 2 will deviate from its nominal position, and vice versa. When the subassembly of part 1 and part 2 is transferred to station k + 1, it will be repositioned on station k + 1 in order to be assembled with a new part 3. Suppose that the pin-holes on part 2 are used to hold subassembly “1+2.” Even if fixture locators that hold part 2 on station k + 1 are free of error, as shown in Figure 6.6, part 1 may deviate from its nominal position in subassembly “1+2+3.” This deviation of part 1 is not a result of locator deviation at station k + 1. Rather, it is a combined result of deviation of locators on the previous
2151_book.fm Page 121 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
Part 2
Part 1
Fixture deviation
121
Station k
Deviation of part 1 comes from re-orientation of subassembly 1+2 from station k to station k+1 Part 1
Part 2
Station k+1 Part 3
FIGURE 6.6 Across-station variation factor: reorientation.
station and the subassembly repositioning at the current station. The significance of this reorientation-induced deviation is that it actually couples the variations originating from individual stations, as we will see in Subsection 6.3.2. This reorientation-induced error would be almost unavoidable for a multistation assembly process when a different set of fixture locators or a different datum scheme is used to reposition a subassembly on a downstream station. When the parts are considered flexible, the reorientation effect is influenced by flexibility-related effects (such as spring back), and they will further complicate the way variation propagates across stations. The modeling of variation and its propagation taking part flexibility into account can be found in Camelio et al. [7].
6.2.3 SUMMARY
AND
ASSUMPTIONS
FOR
MODELING
The key variation factors can be summarized as: • • • •
Dimensional errors associated with fixture locators at each assembly station Dimensional errors due to part fabrication Across-station reorientation-induced error Flexibility effects of individual components
In this chapter, we will focus on a simplified rigid-part assembly process. That is, we will limit ourselves to considering only 2-D panel-like parts positioned by a 3-2-1 fixture layout. Moreover, we will concentrate on the modeling and analysis of fixture-locator-related errors and assume that only the lap joint is present between parts. Owing to the variation-absorbing nature of a lap joint, part fabrication errors will therefore not affect the stream of variation in the whole production line. The assumptions are summarized as follows: • • •
2-D rigid body part 3-2-1 Fixture layout for rigid part Lap joint between parts
2151_book.fm Page 122 Friday, October 20, 2006 5:04 PM
122
Stream of Variation Modeling and Analysis for MMPs
The resulting model will also be applicable to an n-2-1 fixture layout if the fixture variations being considered cause panel motion largely in the plane of rigidity. Shiu et al. [6] indicated that this simplified process is applicable to 60% of situations in a typical automotive body assembly process. The part fabrication errors will be explicitly considered and modeled using a differential motion vector in Chapter 7 for a machining process.
6.3 STATE SPACE MODELING 6.3.1 REPRESENTATION
OF
PART POSITION
AND ITS
DEVIATION STATE
In dimensional quality control, each part is characterized by its deviation from the nominal position. Let x, y, and z be the translation coordinate variables and, α, β, and φ be the corresponding rotation coordinate variables (please refer to Figure 6.7 for the six coordinate variables [or degree of freedoms]). As we will only explicitly model a 2-D assembly process in this chapter, the position of each single part or a multipart subassembly is represented by three coordinate variables, x, z, and β. In Figure 6.7, the subscripts (i, k) of the coordinate variables indicate that they are for part i on station k, where i = 1, …, np, k = 1, …, N, np is the total number of parts, and N is the total number of stations. To define the position and orientation of a part, we need to specify reference points. In this chapter, we will assign two reference points to each part, and make them coincide with the locating points denoted by LP1,i,k and LP2,i,k. On station k, the position of a part will be represented by the instantaneous position of LP1,i,k, (LP1,i,k (x), LP1,i,k (z)), in the body coordinate system, and its orientation represented by the angle between the line connecting LP1,i,k and LP2,i,k and the x-axis of the body coordinate system. For a subassembly that consists of more than one part, its position and orientation is similarly defined. Each 2-D rigid body subassembly will be positioned by a 4-way locator and a 2-way locator, which are chosen from the pool of 4-way and 2-way locators on the subassembly. The 4-way/2-way locator pair used to position the subassembly is treated as the reference points for the subassembly, and they are
parti on station k
LP 2,i,k LP 1,i,k
z
(LP1,i,k(x), LP 1,i,k(z))
y x
FIGURE 6.7 Representation of part deviation.
i,k
2151_book.fm Page 123 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
123
denoted by LS1,s,k and LS2,s,k, respectively, where the subscripts have a similar meaning but s is the index for subassembly here. The part position indicated in a design blueprint is known as the nominal position, which is the ideal situation when neither fixture variation nor part variation exists. As locating tools and part fabrications are not perfect in reality, the part will deviate from its nominal position. Given that one knows part nominal positions from a design blueprint, people often use deviations from such positions to represent its actual position. Denote the random deviations associated with each of the six degrees of freedom (three translations and three rotations) of part i at station k by xi,k ≡ [δxi,k δyi,k δzi,k δαi,k δβi,k δφi,k]T, where δ in front of a coordinate variable represents a small deviation. For a 2-D assembly, the deviation state of each xi,k can be simplified as [δxi,k δzi,k δβi,k]T. For all the np parts in the final assembly, the deviation state of the whole assembly on station k can be expressed as x k ≡ [x1T,k x Tn p ,k ]T. In a multistation process, part i may not yet have appeared on station k. If not, the corresponding xi,k = 0. We denote the index of the first station where part i appears by ξi. For the deviation state of a multipart subassembly s at station k, we use the notation qs,k to distinguish it from an individual part, but the notation’s physical meaning is the same. We will use uk to denote the random deviation associated with locator pairs on station k. For example, for the locator pair that supports subassembly s on station k, its deviation will be us,k = [δLS1,s,k (x) δLS1,s,k (z) δLS2,s,k (x) δLS2,s,k (z)]T. For all the nk locator pairs used on station k, the fixture deviation vector is xk ≡ [u1T,k u Tnk ,k ]T.
6.3.2 SOME PRELIMINARY RESULTS Two lemmas are presented here before we derive the state space model, which will be used intensively during later derivations. They are based on rigid part kinematics. The detailed proofs of both lemmas can be found in the appendix 6.1 and 6.2, and also in Reference 8 or Reference 9. Lemma 6.1: When a rigid body undergoes a translation and a rotation away from its nominal position, if the rotation angle (denoted as δβ) is small, then deviations of any two points on the same part (denoted as [ δxa, δza ]T and [δxb, δzb ]T) have the relationship: δxb δx a a ,b δ z = R 1 δza b δβ δβ
(6.1)
1 = 0 0
(6.2)
where
R
a ,b 1
0 1 0
−( zb − za ) xb − x a 1
and (xa, za), (xb, zb) are the nominal coordinates of those two points.
2151_book.fm Page 124 Friday, October 20, 2006 5:04 PM
124
Stream of Variation Modeling and Analysis for MMPs
Lemma 6.2: The part deviation state xi,k can be related to the deviations of its reference points using a linear approximation, provided that the deviations at these points are much smaller than the distance between them; that is:
x i,k = R 2i,k
δLP1,i,k ( x ) δLP1,i,k ( z) δLP2,i,k ( x ) δLP2,i,k ( z)
(6.3)
where
R
i, k 2
1 = 0 sin βi,k LDi,k
0 1 cos βi,k − LDi,k
0 0 sin βi,k − LDi,k
0 0 cos βi,k LDi,k 3× 4
(6.4)
and LDi,k is the Euclidean distance between LP1,i,k and LP2,i,k . Lemma 6.2 can be easily extended to a multipart subassembly, as stated in the following two corollaries. Corollary 6.1: If subassembly s at station k is located by points LS1,s,k and LS2,s,k, then its deviation state due to small deviations at the locating points is:
q s ,k = R s3,k
δLS1,s ,k ( x ) δLS1,s ,k ( z) = R s ,k ⋅ u 3 s ,k δLS2,s ,k ( x ) δLS2,s ,k ( z)
(6.5)
where
R
s ,k 3
1 = 0 sin βs ,k SDs ,k
0 1 cos βs ,k − SDs ,k
0 0 0 0 sin βs ,k cos βs ,k − SDs ,k SDs ,k
(6.6)
and SDs,k is the Euclidian distance between LS1,s,k and LS2,s,k. The next corollary characterizes the reorientation when a subassembly is transferred to the next station. As explained in Subsection 6.2.2, reorientation will induce deviation in a part even if the current locating points are free of error.
2151_book.fm Page 125 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
125
Corollary 6.2: Suppose that subassembly s is supported by LS1,s,k and LS2,s,k at station k and also assume that these two locating points are free of error at the current station k. The deviation state of subassembly s on station k when moving from station k – 1 to station k can be expressed as a linear combination of deviations accumulated in its locating points LS1 and LS2 at the previous station k – 1.
q s ,k = R k4−1,k
δLS1,s ,k −1 ( x ) δLS1,s ,k −1 ( z) δLS2,s ,k −1 ( x ) δLS2,s ,k −1 ( z)
(6.7)
where
R
k −1, k 4
−1 = 0 sin βs ,k − SDs ,k
0 −1 cos βs ,k SDs ,k
0 0 0 0 sin βs ,k cos βs ,k − SDs ,k SDs ,k
(6.8)
6.3.3 STATE SPACE REPRESENTATION Based on the previous discussions, as well as the analysis done by Jin and Shi [10], the deviation state of part i on station k can be expressed as a summation: x i,k = x i,k −1 + d if,k + d ir,k
(6.9)
where d if,k and d ir,k are the fixture-error-induced deviation and reorientation-induced deviation, respectively, and the superscripts have the same meaning as the subscript of x. Equation 6.9 is illustrated in Figure 6.8, in which x i,k −1 is the intermediate state of part i during the operations on station k. The two deviation terms d if,k and d ir,k in Equation 6.9 are given by the following theorems.
Deviated part from the previous station
On station k dr
xi,k-1
Re-orientation
~ x i,k 1
df Fixture error
FIGURE 6.8 Deviation accumulation of part i on station k.
x i,k
Input to the next station
2151_book.fm Page 126 Friday, October 20, 2006 5:04 PM
126
Stream of Variation Modeling and Analysis for MMPs
Part i
Subassembly s
LP 2,i,k.
LP 1,i,k
z
LS1,s,k
LS2,s,k
x
FIGURE 6.9 Fixture error-induced deviation.
Theorem 6.1: Suppose part i is in subassembly s on station k. As illustrated in Figure 6.9, its fixture error-induced deviation d if,k can be expressed as d if,k = B1i,k u k
(6.10)
where R 1,s ,k B1i ,k = 1 0 3× 4nk LS
W1 (s) = ∆1s I4× 4
,LP1,i ,k
R 3s ,k W1 (s) if k ≥ ξ i if k < ξ i
∆ 2 s I 4× 4
∆ hs I4× 4
(6.11)
4 × 4 nk
(6.12)
The Kronecker Delta is 1 if h = s ∆ hs = 0 if h ≠ s
(6.13)
Proof: According to Corollary 6.1, the deviation of subassembly s due to fixture errors at LS1,s,k and LS2,s,k is the qs,k expressed in Equation 6.5. Because part i is on subassembly s, Lemma 6.1 can be employed to obtain the deviation state of part i, which is d if,k = R 1
LS1,s , k , LP1,i , k
q s ,k
(6.14)
Substituting Equation 6.5 into Equation 6.14 yields d if,k = R 1
LS1,s , k , LP1,i , k
R s3,k ⋅ u s ,k
(6.15)
2151_book.fm Page 127 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
127
The W1’s defined in Equation 6.12 is a selecting matrix to determine which set of locator pairs is the one affecting the deviation of part i, i.e., u s ,k = W1 (s ) ⋅ u k
(6.16)
Combining Equation 6.15 and Equation 6.16, we end up with the expression of the upper part in Equation 6.11. The lower part in Equation 6.11 implies that fixture error will not be reflected in the deviation of part i before this part appears in the assembly stream. Theorem 6.2: If part i is on subassembly s on station k, its reorientation-induced deviation d ir,k can be expressed as d ir,k = B2i,k F1x J ,k −1 + B2i,k F2 x G ,k −1
(6.17)
where LS , LP R 1,s ,k 1,i,k R k4−1,k Bi2,k = 1 0
1 0 F1 = 0 0 0 0 F2 = 1 0
0 0 0 1
iff k > ξ i if k ≤ ξ i
(6.18)
0 0 0 0
(6.19)
0 0 LP1,G ,k ,LP2 ,G ,k R1 0 0
(6.20)
0 1 0 0
and J and G are the indices of parts on which the points LS1,s,k and LS2,s,k are located, respectively. Proof: The d ir,k is actually caused by the locating deviation accumulated up to station k – 1; therefore, it can be expressed as:
d ir,k = R 1
LS1,s , k , LP1,i , k
R k4−1,k
δLS1,s ,k −1 ( x ) δLS1,s ,k −1 ( z) δLS2,s ,k −1 ( x ) δLS2,s ,k −1 ( z)
(6.21)
2151_book.fm Page 128 Friday, October 20, 2006 5:04 PM
128
Stream of Variation Modeling and Analysis for MMPs
where δLS h,s ,k −1 ( x ) and δLS h,s ,k −1 ( z) (h = 1, 2) have the same meaning as those in Corollary 6.2. Here, R k4−1,k converts fixture deviations at the previous station into the deviation state of the associated subassembly during part transferring across stations (Corollary 6.2), whereas R 1LS1,LP1 further connects the deviation of subassembly s to that of part i using Lemma 6.1. Provided that LS1,s,k is on part J and LS2,s,k on part G, respectively, the deviation information of these two points at the previous station k–1 is actually preserved in xJ,k-1 and xG,k-1, that is, δLS1,s ,k −1 ( x ) 1 = δLS1,s ,k −1 ( z) 0
0 x J ,k −1 0
(6.22)
0 LP1,G ,k ,LP2 ,G ,k x G ,k −1 R1 0
(6.23)
0 1
and δLS2,s ,k −1 ( x ) 1 = δLS2,s ,k −1 ( z) 0
0 1
Putting them together, Equation 6.22 and Equation 6.23 yield δLS1,s ,k −1 ( x ) 1 δLS1,s ,k −1 ( z ) 0 = δLS2,s ,k −1 ( x ) 0 0 δLS2,s ,k −1 ( z ) 0 0 + 1 0
0 1 1 0 0 0 0 0 1 0 0 1
0 x J ,k −1 0
0 1
(6.24) 0 1
0 LP1,G ,k ,LP2 ,G ,k xG ,k −1 R1 0
Substituting Equation 6.24 into Equation 6.21, and also using definitions in Equation 6.18 to Equation 6.20, Equation 6.17 can be obtained. Again, the lower part of Equation 6.18 is 0 because the reorientation-induced deviation only begins to contribute from station k + 1 if part i first appears on station k. Utilizing Theorem 6.1 and Theorem 6.2, Equation 6.9 can be written as, x i,k = x i,k −1 + B1i,k u k + Bi2,k F1x J ,k −1 + Bi2,k F2 x G ,k−1
(6.25)
This equation implies that the deviation state of part i is coupled with that of other parts. It is then impossible to write a decoupled deviation propagation equation
2151_book.fm Page 129 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
129
for each individual part. Alternatively, we can write a state space equation using the deviation state vector of all np parts, which is x k = [x1T,k x Tn p ,k ]T. First, we need to replace x J ,k −1 and x G ,k −1 in Equation 6.25 with the aggregated state vector xk. Define W2, another selecting matrix, as W11 W2 (s, k ) = W21
W12
W22
W1n p W2 n p 6 ×3 n p
(6.26)
where I W jg = 3×3 03×3
if (j, g ) = (1, J ) or (2, G ) otherwise
(6.27)
W2 is used to determine which parts on subassembly s contribute to the reorientation-induced error at station k. Then, define Hi,k as Hi,k = [Bi2,k F1
Bi2,k F2 ]
(6.28)
Using Equation 6.26 and Equation 6.28, Equation 6.25 turns out to be x i,k = x i,k −1 + B1i,k u k + Hi,k W2 (s, k ) x k
(6.29)
Furthermore, define Bk and Hk–1 as B k = [(B11,k )T H k −1 = [(H1,k W2 (s, k ))T
(B12,k )T
...
(B11,nk )T ]T
(H2,k W2 (s, k ))T
...
(H n p ,k W2 (s, k ))T ]T for k ≥ 2,
(6.30)
and H0 = 03 n p ×3 n p. Then, Equation 6.29 can be written as x k = (I3 n p ×3 n p + H k −1 ) x k −1 + B k u k , or one step further, x k = A k −1x k −1 + B k u k where A k −1 = I3 n p ×3 n p + H k −1 .
(6.31)
2151_book.fm Page 130 Friday, October 20, 2006 5:04 PM
130
Stream of Variation Modeling and Analysis for MMPs
Equation 6.31 is the state equation in our station-indexed state space model, governing the deviation propagation in an assembly process under assumptions listed in Subsection 6.2.3. If coordinate sensors are placed on assembly stations along the assembly line, an observation equation should be employed to describe the relationship between the dimensional measurements and the deviation state. Denote mi,k as the number of measurement points on part i at station k and mk as the total number of measurement points at station k. Then the deviation of each point on part i is: y ij,k = R 0
LP1,i , k , SPj ,i
x i, k
j = 1, 2, , mi,k
(6.32)
0 LP1,i,k ,SPj ,i R1 0
(6.33)
where 1 = 0
LP1,i , k , SPj ,i
R0
0 1
and SPj,i is the location of the jth sensor on part i. Thus, the measurement vector of part i on station k is y i,k = [( y1i,k )T
( y i2,k )T
m
( y i,ki,k )T ]T
(6.34)
and it can be written as y i, k = C i, k x i, k
(6.35)
where
C i, k
R 0LP1,i,k ,SP1,i LP ,SP R 1,i,k 2 ,i = 0 LP1,i,k ,SPmi,k ,i R 0 2 mi , k × 3
(6.36)
Then, the deviation vector of all the measurement points on station k is y k ≡ [ y1T where
y Tmk ]T = C k x k
(6.37)
2151_book.fm Page 131 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
C1,k 0 C k = 0
0 C 2,k 0
131
0 0 C n p ,k
(6.38)
If process background disturbances, higher-order terms due to linearization, and measurement noises are considered together with Equation 6.31 and Equation 6.32, the state space model governing dimensional deviation propagation in an N-station process will be: x k = A k −1x k −1 + B k u k + w k , k = 1, 2, …, N
(6.39)
y k = C k x k + v k {k} ⊂ {1, 2, …, N}
(6.40)
where wk and vk are the random vectors representing process disturbances or higherorder terms and measurement noises, respectively.
6.4 MODEL VALIDATION To validate the state space model developed in Section 6.3, consider the assembly process of the side frame of an SUV. The final product, the inner-panel-complete, comprises four components: A-pillar, B-pillar, rail roof side panel, and rear quarter panel, which are shown in Figure 6.10. The four parts are assembled on three stations (Stations I, II, and III). Then, the final assembly is inspected at Station IV, where SP1 to SP 10 marked in Figure 6.10d are the key dimensional features. In such a multistation process, the aforementioned 3-2-1 fixture is used on every station to ensure product dimensional accuracy. In Figure 6.10, LP1 to LP8 are locating points, often called principal locating points (PLPs) in the automotive industry. A fixture layout in a multistation process can be represented using these PLPs as follows: {{LP1, LP2}, {LP3, LP4}}I → {{LP1, LP4}, { LP5, LP6}}II → {{ LP1, LP6}, {{LP7, LP8}}III → {{ LP1, LP8}}IV , where the assembly process starts from Station I (indicated by the subscript) and the arrow represents a transition from one station to the next. As an example, {{LP1, LP4},{LP5, LP6}}II means that at Station II, the first workpiece, the subassembly “A-pillar+B-pillar,” is located by LP1 and LP4 and the second workpiece, the rail roof side panel, is located by LP5 and LP6. The design nominal x and z coordinates for fixture locators and measurement points are given in Table 6.1 and Table 6.2, respectively.
2151_book.fm Page 132 Friday, October 20, 2006 5:04 PM
132
Stream of Variation Modeling and Analysis for MMPs
Rail roof side panel
LP 2 A-Pillar
A-pillar
LP 5 LP 6
LP 3 LP 1 B-pillar
LP 4
z
LP 1 y
(a) Station I: pillar and pillar assembly
(b) Station II: addition of rail roof side panel
SP 7
SP 6
SP 2
LP 4
B-pillar
x
LP 8
A-pillar
SP 1
LP 6
SP 10 SP 3 SP 4
LP 7
Rail roof side panel
SP 9
SP 5
LP 1
SP 8
Rear quarter panel B-pillar
(d) Station IV: key product features
(c) Station III: final assembly of inner-panel
FIGURE 6.10 Assembly process of an SUV side frame.
TABLE 6.1 Coordinates of Fixture Locators (in Figure 6.10a, Figure 6.10b, and Figure 6.10c; units: mm) (x,z)
LP1 (367.8,906.05)
LP2 (667.47,1295.35)
LP3 (1301,1368.89)
LP4 (1272.73,537.37)
(x,z)
LP5 (1470.71,1640.40)
LP6 (1770.50,1702.62)
LP7 (2941.42,1691.31)
LP8 (2120.32,1402.83)
TABLE 6.2 Coordinates of Measurement Points (in Figure 6.10d; units: mm) (x,z)
SP1 (271.50,905)
SP2 (565.7, 1634.7)
SP3 (1289.7,1227.5)
SP4 (1306.5,633.5)
SP5 (1244.5,85)
(x,z)
SP6 (1604.5,1781.8)
SP7 (2884.8, 1951.5)
SP8 (2743.5, 475.2)
SP9 (1838.4,226.3)
SP10 (1979.8,1459.4)
A state space model is established, following the general modeling procedure, as outlined in Section 6.2, to represent the variation propagation in such a fourstation assembly process. Suppose that there are tolerances of ±1.0 mm associated with all the locating points on each of the three assembly stations and we assume
2151_book.fm Page 133 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
133
TABLE 6.3 Comparison between the State Space Model and VSA (unit: mm)
Mean
σ) Std(σ
Mean
σ) Std(σ
Discrepancy between Std’s (%)
271.5002 905.0007 565.7050 1634.6972 1289.6876 1227.4772 1306.5058 633.4785 1244.4226 84.9773 1640.5004 1781.7971 2884.8178 1951.5027 2743.5147 475.2045 1838.4151 226.3067 1979.8176 1459.4052
0.0020 0.1862 1.4103 0.3833 0.8361 0.9885 1.0123 0.9999 1.7122 0.9608 0.9545 0.8902 1.1852 1.1136 1.6114 0.9210 1.9107 0.4698 0.9165 0.3229
271.5000 905.0007 565.7053 1634.6986 1289.6876 1227.4778 1306.5059 633.4783 1244.5228 84.9764 1640.5007 1781.7985 2884.8065 1951.5037 2743.5094 475.2034 1838.4099 226.3016 1979.8075 1459.4019
0.0020 0.1861 1.4102 0.3833 0.8361 0.9885 1.0123 0.9999 1.7122 0.9608 0.9545 0.8900 1.1853 1.1136 1.6115 0.9211 1.9107 0.4698 0.9165 0.3229
0.0000 0.0537 0.0071 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0225 0.0084 0.0000 0.0062 0.0109 0.0000 0.0000 0.0000 0.0000
VSA
SP1(x) SP1(z) SP2(x) SP2(z) SP3(x) SP3(z) SP4(x) SP4(z) SP5(x) SP5(z) SP6(x) SP6(z) SP7(x) SP7(z) SP8(x) SP8(z) SP9(x) SP9(z) SP10(x) SP10(z)
State Space Model
that fixture locators at the measurement station (Station IV) are free of error. Also, it is assumed that deviations at the locating points are normally distributed and the tolerance limits (±1.0) correspond to the 6σ value of a corresponding normal distribution. Variations observed at the sensor locations are calculated using the state space model and a simulation software package, Variation System Analysis (VSA), based on 5000 simulations. The mean value and the standard deviation (std) of all measurement points from both models are compared in Table 6.3. The discrepancy between the std values from the two models is less than 0.054% for this four-station example. We feel that the state space model can provide an adequate linear variation propagation model that can predict dimensional variation as accurately as VSA does under some circumstances.
6.5 EXERCISES This problem aims to provide an exercise to help get familiar with the state space model and perform assembly process simulations. The process analyzed corresponds to the assembly of an automobile side frame, as presented in Figure 6.11. The side frame has three parts: the fender, the side ring, and the rear quarter.
2151_book.fm Page 134 Friday, October 20, 2006 5:04 PM
134
Stream of Variation Modeling and Analysis for MMPs
FIGURE 6.11 Description of the components of a sedan side frame.
FIGURE 6.12 Assembly sequence of the sedan side frame.
The assembly sequence is presented in Figure 6.12. In station 1, the fender and the side ring are assembled together. The subassembly is then moved to Station 2, where the rear quarter is added. Finally, the whole assembly is transferred to the measurement station to measure the KPCs given as the measurement points (M1 to M8). The dots on each part represent the locating points used to locate the parts in each station. The odd numbers correspond to the 4-way locators and the even numbers to the 2-way locators. It is important to note that in station 2, the subassembly coming from station 1 is held by locators P1 and P4. In the measurement station, the locators used are P1 and P6. The position of the locators and measurement points are presented in Table 6.4. Problem 6.1 (Single-part model): In this problem, consider the complete side frame as one part, as shown in Figure 6.13. 1. Derive the relationships that relate the deviations at the measurement points M4 and M6 with respect to the deviations of the locating points P1 and P6. (Hint: Obtain matrix T (T = R1 R2) in this case.) 2. From the result in (1), determine the amount of deviations in the locators (P1 and P6) that causes the deviations of the measurement points for different parts, as presented in Table 6.5. Provide a table with the faults that you identified and their associated deviations.
2151_book.fm Page 135 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
135
TABLE 6.4 Location of the Locators and the Measurement Points X (mm)
P1 P2 P3 P4 P5 P6
(4-way) (2-way) (4-way) (2-way) (4-way) (2-way)
M1 M2 M3 M4 M5 M6 M7 M8
Locators 250 700 1078 2992 3437 3905
Z(mm)
360 365 770 812 495 503
Measurement Points 456 401.7 770 555.3 1078 770 1740 1146.4 1807 35 2456 1260 4050 800 4250 402
FIGURE 6.13 Simplified model of a sedan side frame (single-part model).
Note that: • •
It is recommended to use MatlabTM (Matlab is a registered trademark of The MathWorks, Inc.) to calculate matrices R1 and R2. To determine the deviation of the locators in (b), you should use the command pinv (pseudoinverse) in Matlab instead of inv to calculate the inverse of the matrix.
Problem 6.2 (Multiple-parts model): In this problem, consider the complete assembly process of the three parts that form the side frame presented in Figure 6.12, and the position of the locators and the measurement points presented in Table 6.4.
2151_book.fm Page 136 Friday, October 20, 2006 5:04 PM
136
Stream of Variation Modeling and Analysis for MMPs
TABLE 6.5 Observed Measurements (unit: mm) Part #
M4 (x)
M4 (z)
M6 (x)
M6 (z)
1 2 3 4 5 6 7
0.992 0.215 0 0.43 1.206 0.215 0.562
0.016 0.407 1 0.186 0.609 0.593 0.17
0.990 0.246 0 0.492 1.236 0.246 0.499
0.024 0.603 1 0.205 0.421 0.397 0.229
1. Obtain the state space model matrices (A, B for each station and matrix C for the measurement station) for the process presented in Figure 6.12. The locators used at station 1 are P1 and P2 for the fender, and P3 and P4 for the side ring. At station 2, the locators used are P1 and P4 for the subassembly and P5 and P6 for the rear quarter. At the measurement station, the locators P1 and P6 are used to hold the complete side frame. Note that in order to verify the results, consider that A0(1,1) = 1; A1(3,3) = 1; B1(3,2) = 0.0022; B2(3,2) = 0.0004; C3(16,9) = 813. Remember that the matrices A are index-shifted to 1 because no index can be 0 in Matlab. 2. Compare the following two mounting strategies presented in terms of their impact on quality (the simulation conditions are stated at the end of the problem): • First strategy: Locators P1 and P6 are used to hold the frame at the measurement station. • Second strategy: Locators P1 and P4 are used (i.e., P4 instead of P6) to hold the frame at the measurement station. Do each of the following: a. Compare the standard deviation of both strategies for each individual measurement point M1 to M8 in x and z directions (use the Matlab command std to calculate the standard deviation). b. Compare the 2-norm (sum of squared elements of a vector) of the vectors containing the standard deviations of the eight measurement points for both strategies (use the command norm in Matlab to calculate the vector norms). c. Which mounting strategy do you recommend? When solving this problem, please consider that the only source of variation in the whole process is the fixture variations at each station, which are independent and normally distributed with zero mean and standard deviation 0.0416 mm.
2151_book.fm Page 137 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
137
Deviated position B’ δZ B
Nominal position δX B B
A’
δZ A
A Z
δβ
δX A
β X
FIGURE 6.14 Part deviating from its nominal position.
APPENDIX 6.1: DETERMINATION OF THE DEVIATIONS OF A PART (LEMMA 6.1) Figure 6.14 presents a view of a part deviating from its nominal position. As the deviation is in the 2-D plane (x-z) and the part is assumed to be rigid, three parameters are required to completely characterize the deviations of the part. Those are the deviations in the x and z directions and the rotation along the y- axis (going into the plane, and henceforth considered positive in the y direction). The position of any arbitrary point B on the part can be determined based on the known position of any other arbitrary point A and the part rotation. In Figure 6.14, point A coincides with the location of the part hole. However, the position of point A can be any point on the part. Then the position of point B can be determined as: xb = x a + x ab zb = za + zab
(6.41)
where xa, za, xb, and zb are the positions of points A and B in the x- and z-axis directions (absolute position), respectively. The terms xab and zab stand for the distance between points A and B in the x and z directions and are calculated as xab = xb – xa and zab = zb – za . If the part deviates in the plane and the deviations of the reference point A are known (including part rotation), then it is possible to determine the final position of point B xb′ and zb′ as x b′ = x a′ + x ab ⋅ Cos(δβ) − zab ⋅ Sin(δβ) zb′ = za′ + x ab ⋅ Sin(δβ) + zab ⋅ Cos(δβ)
(6.42)
2151_book.fm Page 138 Friday, October 20, 2006 5:04 PM
138
Stream of Variation Modeling and Analysis for MMPs
where xa′ and za′ are the positions of point A after the deviation of the part in the x and z directions, and δβ is the part rotation. Now considering that the rotations of the part are small (less than 10˚), Equation 6.42 can be simplified (based on the linearization of the trigonometric functions) as x b′ = x a′ + x ab − zab ⋅ δβ zb′ = za′ + x ab ⋅ δβ + zab
(6.43)
By subtracting Equation 6.41 from Equation 6.43, it is possible to determine the deviations of point B with respect to its nominal position as δx b = x b′ − x b = δx a − zab ⋅ δβ δzb = zb′ − zb = δza + x ab ⋅ δβ
(6.44)
Then Equation 6.44 can be rewritten as a matrix multiplication: δxb 1 δzb = 0 δβ 0
0 1 0
δx a −( zb − za ) δx a a ,b xb − x a ⋅ δza = R 1 ⋅ δza δβ 1 δβ
where matrix R1a,b is defined as
R
a ,b 1
1 = 0 0
0 1 0
−( zb − za ) xb − x a 1
APPENDIX 6.2: EFFECT OF FIXTURE DEVIATION ON PART DEVIATION (LEMMA 6.2) Lemma 6.2 relates the part deviations or state (xi,k) as a function of the deviations of the hole and slot, which is caused by the deviations of the pins that fit into those orifices. In the current analysis, the reference point in the part where the deviations are calculated, coincides with the location of the hole. The relations between fixture or pin deviations and the part, deviations are obtained through the decomposition of effects that single deviations have on the deviation state. The case presented in Figure 6.15a corresponds to the general case where the locators of part i in station k (LP1,i,k and LP2,i,k) are not necessarily aligned along the x or z directions. Figure 6.15b presents the effects when only a deviation in the z direction of the pin that fits into the hole (LP1,i,k), has on the deviations of the part.
2151_book.fm Page 139 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
139
LP 2
LP1 ( z)
Z X
LP 1 (a)
(b)
FIGURE 6.15 Effects of the deviation of LP1 on the part (Note: In the figure, the subscripts i,k have been omitted for simplicity). (a) Case where locators are not aligned with X or Z axis. (b) Impact of LP2(z) on part orientation.
Those effects are a translation of the reference point LP1,i,k in the z direction and the rotation of the part. Figure 6.16 presents the detail of the effects that the displacement of LP1,i,k (δLP1,i,k (z)) has on the part. The part rotation δβi,k can be determined as a function of the displacement using geometric relations, as presented in Equation 6.45. The rotation is negative according to the axis defined in Figure 6.15. δβi,k ≈ −Cos (βi,k ) ⋅
δLP1,i,k ( z) LDi,k
(6.45)
The same type of analysis helps to determine the effects that deviations δLP1,i,k (x), δLP2,i,k (x), and δLP2,i,k (z) have on the rotation of the part, as presented in Equation 6.46. In this case, a deviation in the x direction of LP1,i,k and in the z direction of LP2,i,k will generate a positive rotation; deviation in the x direction of LP2,i,k will generate a negative one as presented in Equation 6.46. δβi ,k ≈ sin(β) ⋅
(δLP
1,i ,k
( x ) − δLP2,i ,k ( x ) LDi ,k
) + cos(β) ⋅ (δLP
2 ,i ,k
( z ) − δLP1,i ,k ( z ) LDi ,k
)
(6.46)
LD
LP1 ( z )
LP1 ( z)
LP1 ( z ) Cos ( )
FIGURE 6.16 Details of the effect that the deviation of LP1,i,k (z) has on the deviations of the rotation (Note: In the figure, the subscripts i,k have been omitted for simplicity).
2151_book.fm Page 140 Friday, October 20, 2006 5:04 PM
140
Stream of Variation Modeling and Analysis for MMPs
Finally, the total effect of the deviations of the reference or locating points on the deviations of the part xi,k (state) can be represented as:
x i, k
δxi,k = δzi,k = R i2,k δβi,k
δLP1,i,k ( x ) δLP1,i,k ( z) δLP2,i,k ( x ) δLP2,i,k ( z)
where matrix R2i,k is defined as:
R
i, k 2
1 = 0 sin βi,k LDi,k
0 1 cos βi,k − LDi,k
0 0 sin βi,k − LDi,k
0 0 cos βi,k LDi,k 3× 4
The first two rows of R2i,k account for the translations, and the last row accounts for the rotation. The same analysis is used to derive Corollary 6.1, which explains the effect of locating point deviations of subassemblies on the deviations of the subassembly. Corollary 6.2 explains the effect of reorientation due to errors in the previous station. This is why the terms in the matrix in Equation 6.8 are the negation of the elements in the matrix in Equation 6.6; because in this case it is a locator error (inherited from station k – 1) and not a pin error in station k.
References 1. Ceglarek, D., Shi, J., and Wu, S.M., A Knowledge-based diagnostic approach for the launch of the auto-body assembly process, ASME Transactions, Journal of Engineering for Industry, 116, pp. 491–499, 1994. 2. Shalon, D., Gossard, D., Ulrich, K., and Fitzpatrick, D., Representing geometric variations in complex structural assemblies on CAD systems, Proceedings of the 19th Annual ASME Advances in Design Automation Conference, 44, 121, 1992. 3. Ceglarek, D. and Shi, J., Dimensional variation reduction for automotive body assembly, Manufacturing Review, 8, 139, 1995. 4. Cunningham, T.W. et al., Definition, analysis, and planning of a flexible assembly process, Proceedings of the 1996 Japan/USA Symposium on Flexible Automation, 2, 767, 1996. 5. Ceglarek, D. and Shi, J., Design evaluation of sheet metal joints for dimensional integrity, ASME Transactions, Journal of Manufacturing Science and Engineering, 120, 452, 1998. 6. Shiu, B.W., Ceglarek, D., and Shi, J., Multi-Station sheet metal assembly modeling and diagnostics, NAMRI/SME Transactions, XXIV, 199, 1996.
2151_book.fm Page 141 Friday, October 20, 2006 5:04 PM
State Space Modeling for Assembly Processes
141
7. Camelio, A.J., Hu, S.J., and Ceglarek, D.J., Modeling variation propagation of multistation assembly systems with compliant parts, Transactions of the ASME, Journal of Mechanical Design, 125, 673, 2003. 8. Ding, Y., Ceglarek, D., and Shi, J., Modeling and diagnosis of multistage manufacturing processes, Proceedings of the 2000 Japan/USA Symposium on Flexible Automation, July 23–26, Ann Arbor, MI, 2000JUSFA-13146. 9. Ding, Y., Modeling and Analysis of Stream-of-Variation in Multistage Manufacturing Processes, Ph.D. dissertation, The University of Michigan, Ann Arbor, MI, 2001. 10. Jin, J. and Shi, J., State space modeling of sheet metal assembly for dimensional control, ASME Transactions, Journal of Manufacturing Science and Engineering, 121, 756, 1999.
2151_book.fm Page 142 Friday, October 20, 2006 5:04 PM
2151_book.fm Page 143 Friday, October 20, 2006 5:04 PM
7
State Space Modeling for Machining Processes*
7.1 INTRODUCTION Chapter 6 presented the variation propagation modeling for multistage assembly processes. In this chapter, stream of variation (SoV) modeling for multistage machining processes will be investigated.
7.1.1 INTRODUCTION TO MACHINING PROCESSES AND DIMENSIONAL VARIATION SOURCES Machining is a very important manufacturing process that removes materials from a workpiece to obtain design dimensional accuracy and better surface finishing, or to obtain complicated surface forms that cannot be obtained by other processes. The basic steps of machining are illustrated in Figure 7.1. The first step in a machining process is to set up the workpiece in a fixture system as illustrated in Figure 7.1b. A fixture is a device that holds a workpiece position in a particular setup and provides a means to reference and align the cutting tool to the workpiece. Proper location of the workpiece is essential to ensure accuracy and repeatability of the machining process. Figure 7.2 depicts the most popular 3-2-1 fixture system. Figure 7.2a presents the 6 degrees of freedom (d.o.f.’s) of a workpiece in the space (3 translational and 3 rotational d.o.f.’s). By requiring the workpiece to touch the 6 locators as illustrated by Figure 7.2b, the workpiece is fully constrained and the required positional and orientational relationship between the workpiece and the locators and, hence the machine tool, can be attained. The feature of the workpiece is defined as a physically identifiable portion of a workpiece, such as a plane, hole, slot, and chamfer. The features that touch the fixture locators are called datum or datum feature in machining literature. For example, the bottom plane of the cube in Figure 7.2b is the primary datum. The second step in a machining process is to clamp the workpiece on the fixture system and then mount the fixture system on the machine working table (for some dedicated machining systems, the fixture is an integrated part of the machine). One point worth mentioning is that the purpose of clamping is to fasten the workpiece to the fixture, and it does not change the location of the workpiece in space. In the last step, the cutting tool cuts the workpiece that is located on the fixture. It is worth noting that a cutting-tool path is programmed according to the working table.
* Part of chapter material is based on Reference 22 (pp. 296–309).
143
2151_book.fm Page 144 Friday, October 20, 2006 5:04 PM
144
Stream of Variation Modeling and Analysis for MMPs
(c) Machining Operation
(b) Fixturing System
(a) Workpiece
FIGURE 7.1 Basic steps in machining.
3
z axis
z axis Locator {6}
6
Locators {4,5} 5
y axis y axis 2
4 x axis 1
x axis
(a) Degree of Freedom of a Workpiece
Locators {1 2 3 }
(b) 3-2-1 Fixture System
FIGURE 7.2 Illustration of a fixture system.
A machining operation often produces multiple features on a workpiece. The dimensional accuracy of the workpiece is determined by the geometrical shape of the features and the relative positions and orientations of different features. Because of the complexity of the machining operation, the dimensional accuracy of the final product of a machining process is affected by many interrelated factors. To understand the influential factors of the dimensional accuracy, the relationships among the key elements of a machining process are illustrated in Figure 7.3. From Figure 7.3, it is clear that if the cutting-tool path is at the nominal location of the geometric feature with respect to the workpiece, then the dimensions of the resulting feature will be perfect and there will be no error. However, because it is unrealistic to measure precisely the location of each workpiece after it is mounted in the fixture, it is very difficult, if not impossible, to program the cutting-tool path with respect to the workpiece location. Therefore, in most cases it is programmed with respect to the working table of the machine. Clearly, the errors of the fixture location with respect to the working table (called fixture error) and the errors of the workpiece location with respect to the fixture (called datum error) will result in dimensional error of the features in this case. These two types of errors are often referred to as setup errors in machining processes. Besides the setup error, machine error is another important error source: the true cutting path is different from the programmed path because of various errors in the machine itself, such as geometric and dimensional errors of the kinematic links of the machine, thermal errors, wear of the cuttingtool, etc. The deviation of cutting tool path will certainly cause dimensional errors
2151_book.fm Page 145 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
145
Cutting tool Workpiece Fixture Machine Table
FIGURE 7.3 The key elements and their relationships in machining.
in the workpiece. These errors are defined as machine errors in this book. Some error sources will induce both machine error and setup error. For example, the cutting force will cause deformation in both machine and fixture. Therefore, cutting-forceinduced error is a combined error source. These aforementioned errors involved in machining operation often cause large variations in product dimensional quality and, thus, they are also called variation sources in this book. We will use “process fault,” “process error,” and “variation source” interchangeably in this book. Instead of classifying the error sources in machining processes into setup error, machine error, and combined error, these errors are also classified as quasi-static errors and dynamic errors [1]. Quasi-static errors are the static or slow-varying errors between the cutting tool and the workpiece. They include geometric and kinematic errors, thermal errors, cutting-force-induced errors, tool-wear-induced error, fixturing error, etc. Quasi-static errors account for about 70% of overall errors. Dynamic errors are fast-changing errors, caused by sources such as spindle error, machine structure vibration, controller error, etc. They are more dependent on the particular operating conditions of the machine.
7.1.2 MODELING OF VARIATION PROPAGATION MACHINING PROCESSES
IN
MULTISTAGE
The analysis and compensation of a specific type of error on a single-stage operation have been intensively studied. A huge body of literature can be found on the modeling and compensation of machine geometric errors [2–6], thermal errors [7–10], fixtureinduced errors [11–14], and force-induced errors [15–17]. However, the research on building a comprehensive model that includes multiple error sources and multiple operation stages is limited. For a multistage machining process, the product variation at a certain stage consists of two components: the variation brought by the volumetric error of current machine stage and the variation brought by the datum feature error introduced in previous stages. The second component exists because we have to use features produced by previous stages as the reference in the current operation. The variation
2151_book.fm Page 146 Friday, October 20, 2006 5:04 PM
146
Stream of Variation Modeling and Analysis for MMPs
Datum Error D
D
D
C
C
C
(a)
(b)
(c)
FIGURE 7.4 A two-step machining example.
from previous stages will be accumulated onto current operation. The variation propagation can be illustrated in a simple two-step machining example, as shown in Figure 7.4. The workpiece is a cube of metal (the front view is shown). Surface C of the workpiece is milled in the first step (Figure 7.4a). Because of an error in the milling operation, surface C is not perpendicular to surface D. A hole is drilled on surface D using surface C as one of the datum features (Figure 7.4b) in the second step. The location and orientation of the workpiece are determined by both the fixture and the datum features. Clearly, the resulting hole is not perpendicular to surface D (Figure 7.4c). The geometric error of the hole is not caused by the drilling operation; rather, it is caused by the milling operation in the first step. From this example, we have the following key observations: • •
The dimensional variation propagation and the interactions among multiple machining stages exist in machining processes in general. The key reason of variation propagation in machining processes is that the previously machined features are used as the datum in the subsequent machining operations.
From the preceding observations, we will develop a quantitative state space model to describe the variation propagation and the relationship between process error sources and product dimensional quality in multistage machining processes. It is unrealistic to include all the error sources in the model, because of the complexity of a machining process. Instead, we will focus on modeling the following error sources, which mainly account for the variation propagation: 1. Datum error refers to the dimensional and geometric error of datum features. It causes deviation of workpiece in the fixture. The effect of datum error can be obtained through a kinematics analysis based on product/process design information, such as datum feature selection, the nominal locations of datum features and new features, and fixture layout information. Datum error captures the variation propagated from previous stages, because datum features are introduced at previous stages. 2. Fixture error is caused by the errors in a fixture system, such as setup errors, wear of locating pin, clamping errors, etc. Similar to datuminduced error, the contribution of this error can be obtained by a kinematics analysis based on the product/process design information.
2151_book.fm Page 147 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
147
3. Machine error is the most complicated contributor. It involves multiple error sources, and the impact of some of those sources, such as geometric error and thermal error, can only be identified empirically. Fortunately, many of these error sources have been well studied, and effective analysis and compensation schemes have been developed [1,18]. Because the focus of the modeling is on variation propagation, the detailed modeling of certain machine errors is not included. Instead, the machine error in the model is represented as a deviation of the tool path from its nominal path. The mapping of certain machining error components to the tool path deviation can be done but is not included in this chapter for the sake of simplicity. In this chapter, a quantitative state space model will be developed to capture the combined impact of the process error sources and the dimensional variation propagation among different stages. The formulation of this model is further introduced in the following subsection.
7.1.3 MODEL FORMULATION The most challenging issue in variation analysis and reduction for complicated multistage manufacturing processes is the modeling of variation accumulation and propagation along different stages. To attack this problem, a quantitative state space model is proposed (Figure 7.5). For an N-stage process, the state space model is x k = A k −1x k −1 + B k u k + w k and y k = C k x k + v k
(7.1)
where k = 1, 2, …, N, is the stage index. The key quality characteristics of the product (e.g., dimensional deviations of key features) after each stage are represented by the state vector xk. The process faults (e.g., fixturing errors and tooling errors) are denoted by uk. Natural variation and unmodeled errors in the process are represented by a noise input to the system, wk. The product quality measurements are collected in yk. Output yk is not necessarily available at every stage. The measurement noise is denoted by vector vk. The coefficient matrices Ak, Bk, and Ck are determined by the system layout and specific characteristics of the process. Particularly, Ak represents the impact of datum error, i.e., propagated variation source, on the product quality, Bk represents the impact of the local variation sources such as fixture error on the product quality, and Ck is the measurement matrix that maps xk to yk . yk
yN
vk x0
Stage 1 x1 u1
w1
…
x k-2
Stage k-1 xk-1 uk-1
wk-1
Stage k xk uk
wk
vN … xN-1
Stage N uN
FIGURE 7.5 Diagram of a complicated multistage manufacturing process.
wN
xN
2151_book.fm Page 148 Friday, October 20, 2006 5:04 PM
148
Stream of Variation Modeling and Analysis for MMPs
Complicated variation propagation is handled automatically in this model through the state transition. To construct this model, we only need to study the local relationship among xk–1, xk , and uk at each individual stage k. Hence, the large body of knowledge about single-stage operation can be readily reused in the model construction. Furthermore, because of its chainlike structure, the state space model is very flexible; we can easily pick up and study any critical segments of the process. Another advantage of this model is its linear structure. Although the relationships between the variation sources and the key product quality characteristics are nonlinear, in general those relationships can often be linearized around a nominal working condition using Taylor series approximation. A linear state space model Equation 7.1 can significantly reduce the complexity of the subsequent analysis and synthesis. The derivation of this model through the kinematic analysis of the machinecutting operation is presented in Section 7.2. An experimental validation of the developed model is included in Section 7.3. The last section concludes this chapter and provides some coverage of applications of this model.
7.2 DERIVATION OF VARIATION PROPAGATION MODEL 7.2.1 BASICS OF KINEMATIC ANALYSIS OF MACHINING OPERATIONS Homogeneous transformation (HT) is commonly used in kinematic analysis. In this subsection, we will briefly introduce the basic properties of HT, which are useful for the analysis of machining operations. In space, the location of a point is represented by a 3 × 1 vector consisting of three coordinates of this point in a coordinate system. As shown in Figure 7.6, the location of point p can be represented by its coordinates in coordinate system (CS) 1 as vp,1 = [xp,1 yp,1 zp,1]T, where the subscript 1 indicates that the coordinates are w.r.t. CS1. The point p can also be represented by the coordinates of the same point in CS 2 as vp,2. The transformation between vp,1 and vp,2 is given by v p,2 v p,1 2 = H1 ⋅ 1 1
(7.2)
where R 2 H12 = 1 0
t12 1
is a 4 × 4 homogeneous transformation matrix from CS 2 to CS1. The translational vector t12 is the coordinates of the origin of CS 1 in CS 2. The rotation matrix R 12 is a 3 × 3 matrix that captures the orientational difference between CS 2 and CS 1. The rotation matrix is an orthogonal matrix and, thus, R 12 only has three independent parameters. People often use three Euler angles (φ, θ, and ψ) to represent these three
2151_book.fm Page 149 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
149
z2
y2
p
x2
z1
t12
Coordinate System 2
y1 x1 Coordinate System 1
FIGURE 7.6 Coordinate transformation through homogeneous transformation.
independent parameters. The orientation of CS 1 can be viewed as rotating CS 2 around z2 by angle of φ, then rotating the new coordinate system around the new y2 axis by θ, and finally rotating the new coordinate system around the new x2 axis by ψ. Given the three Euler angles, the rotation matrix is cos φ cos θ cos ψ − sin φ sin ψ R = sin φ cos θ cos ψ + cos φ sin ψ − sin θ cos ψ 2 1
− cos φ cos θ sin ψ − sin φ cos ψ − sin φ cos θ sin ψ + cos φ cos ψ sin θ sin ψ
cos φ sin θ sin φ sin θ cos θ (7.3)
Although the expression of the rotation matrix looks complicated, R 12 is an orthogonal matrix; hence (R 12 )−1 = (R 12 )T and the determinant of R 12 is 1. Because of the property of R 12, we have −(R 12 )T t12 1
(R 2 )T (H12 )−1 = 1 0
(7.4)
It is also noteworthy that (H12 )−1 is actually H12 , where v p,1 v p,2 1 = H2 ⋅ 1 1
(7.5)
If the rotational deviation between CS 1 and CS 2 is small, the expression of R 12 can be simplified as in Reference 19 1 R ≈φ −θ 2 1
−φ 1 ψ
θ −ψ 1
(7.6)
2151_book.fm Page 150 Friday, October 20, 2006 5:04 PM
150
Stream of Variation Modeling and Analysis for MMPs
The concept of differential motion vector [19] is used to describe the small deviation between CS 2 and CS 1: t2 x12 = 12 ω1
(7.7)
where t12 = [t12x t12y t12z ]T and ω 12 = [φ θ ψ ]T. In other words, the differential motion vector from CS 2 to CS 1 is a stack of vectors consisting of independent parameters H12 when the deviations between CS 2 and CS 1 are small. We have the identity 1 φ H( x12 ) = −θ 0
−φ 1
θ −ψ
ψ 0
1 0
t12x t12y t12z 1
(7.8)
Another important concept in describing small deviations between coordinate systems is the differential transformation matrix (DTM), which is often denoted as ∆12 . The matrix of ∆12 comes from the identity of H12 ≈ I4× 4 + ∆12 , and thus the DTM can be written as 0 φ ∆12 = −θ 0
−φ
θ −ψ
0 ψ
0 0
0
t12x t12y t12z 0
(7.9)
If a skew-symmetric matrix associated with a vector p is defined as 0 pˆ = p3 − p2
− p3 0 p1
p2 − p1 , 0
then DTM ∆ (x12 ) can be written as ωˆ 2 ∆ (x12 ) = 1 0
t12 . 0
One very useful fact about DTM is ∆12 = − ∆ 12
(7.10)
2151_book.fm Page 151 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
CS 0
CS 1'
CS 1
H10
H 02
151
CS 2'
H12 CS 2" CS 2
FIGURE 7.7 Transition of differential motion vector.
where ∆12 and ∆12 are the differential transformation matrices associated with H12 and H12 , respectively. Because the differential motion vector is heavily used in the derivation of the variation propagation model, we now prove some of its properties. Corollary 7.1: Consider the coordinate systems CS 0, CS 1, CS 1′, CS 2, CS 2′, and CS 2″ and given H10 , H 02 , H12′′′ , the differential motion vector x11′ that describes the deviation between CS 1 and CS 1' and x 22′′′ that describes the deviation between CS 2″ and CS 2′, as shown in Figure 7.7, we have the following: if H10 · H12′′′ = H 02 , then (R 1′ )T x 22′ = 2′′ 0
−(R 12′′′ )T ⋅ (tˆ21′′′ ) (R 12′′′ )T
I3×3 0
0 x11′ I3×3 x 22′′′
(7.11)
Proof: Note that H02′ = H10 ⋅ H( x11′ ) ⋅ H12′′′ ⋅ H( x 22′′′ ) and H02′ = H02 ⋅ H( x 22′ ) ; we have H10 ⋅ H( x11′ ) ⋅ H12′′′ ⋅ H( x 22′′′ ) = H02 ⋅ H( x 22′ ) . Substituting H10 ⋅ H12′′′ = H02 into this equation, we have H( x11′ ) ⋅ H12′′′ ⋅ H( x 22′′′ ) = H12′′′ ⋅ H( x 22′ )
(7.12)
In Equation 7.12, only x 22′ is unknown. If the second-order small values are ignored, Equation 7.11 can be obtain through a straightforward derivation from Equation 7.12. Corollary 7.2: Consider the coordinate systems CS 0, CS 1, CS 1′, CS 2, CS 2′, and CS 2″ and given H10 , H02 , and H12′′′ , the differential motion vector x11′ that describes the deviation between CS 1 and CS 1′ and x 22′′′ that describes the deviation between CS 2″ and CS 2′, as shown in Figure 7.7, we have the following: if H10 ⋅ H12′′′ = H02 , −(R 12′′′ )T x 22′′′ = 0
(R 12′′′ )T ⋅ (tˆ21′′′ ) −(R 12′′′ )T
I3×3 0
0 x11′ I3×3 x 22′
(7.13)
2151_book.fm Page 152 Friday, October 20, 2006 5:04 PM
152
Stream of Variation Modeling and Analysis for MMPs
Proof: The proof is very similar to the proof of Corollary 7.1. Again, Equation 7.12 is used. In this case, only x 22′′′ is unknown. From Equation 7.12, Equation 7.13 can be obtained through a straightforward derivation. These two corollaries are very useful when we switch the reference coordinates in the derivation of the variation propagation model. Equipped with the basic knowledge, we can move on to the detailed model derivation.
7.2.2 REPRESENTATION
OF
WORKPIECE GEOMETRIC DEVIATION
To regulate the dimensional deviations of features of a workpiece, standards were developed for geometric dimensioning and tolerancing (ISO 1101 [1983] or ANSI Y14.5[1982]). However, these conventional geometric tolerances originated from gauging practice. They are not suitable for the operational principle of the coordinate measurement machine (CMM) that now serves as the standard measurement equipment for machining process. In addition, the representation of workpiece features in conventional geometric tolerances do not conform to the part representations used in CAD/CAM systems. Recently, a new vectorial dimensioning tolerancing (VD&T) strategy was proposed and has drawn significant attention [20]. The principle of VD&T is based on the concept of substitute elements or substitute features. A substitute feature is an imaginary, geometrical, ideal feature (e.g., plane, circle, line) whose location, orientation, and size (if applicable) are calculated from the measurement data points of a workpiece surface. Roughly speaking, a substitute feature is an ideal feature fitted from the measurement data of a corresponding true feature. Substitute features are represented by the location vector, orientation vector, and sizes. The location vector indicates the location of a specified point of the substitute feature. The substitute orientation vector is a unit vector that is normal to the substitute plane or parallel to the substitute axis (cylinder, cone, etc.). The size is available for some features. For example, the diameter is the size of a circular hole. The VD&T workpiece feature representation follows the operational principle of CMM and CAD/CAM systems. The measurement data from CMM can be analyzed and compared with the design model directly. The difference between the true feature and the design requirement can be fed back to the manufacturing process directly. In this chapter, we adopt a feature representation that is consistent with VD&T. A location vector and a vector that consists of three rotating Euler angles are used to represent a workpiece feature. Because the size of a feature is usually formed at one machining stage, it is not considered in the modeling derivation. The part representation is illustrated in Figure 7.8. CS 0 is the reference coordinate system. CS 1 is attached on the nominal of the cylinder feature. The position and orientation of the nominal cylinder can be represented by H10 . After the machining operation, the true location and the orientation of the cylinder feature are represented by CS 1′, and they deviate from its nominal values. The deviation is captured by the differential motion vector x11′. To describe the accuracy of a machined workpiece, we need to study the relationships among different features. These relationships can be described by the relationships of corresponding coordinate systems attached to the local features. If
2151_book.fm Page 153 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
153
D1 D0
Nominal Feature Featur Z
True Feature Featur
z
z CS 1
y
Y
x
CS 1'
x
H( x11 )
CS 0
O
y
X
FIGURE 7.8 Illustration of feature representation.
CS i is attached to the nominal feature i and CS i′ is attached to the true feature i, the deviation of the feature is described by the differential motion vector ti x ii′ = ii′ . ω i′ In summary, the relative position and orientation of a feature is described as a homogenous transformation matrix. The deviation of a feature from its nominal value is represented by a differential motion vector. The rationale of this representation is that it conforms to the working principles of CMM and CAD/CAM models. Moreover, various mathematical tools are available for analyzing this representation.
7.2.3 SINGLE-STAGE MODELING
OF
DIMENSIONAL VARIATION
Based on the differential motion vector representation, we can set up a state space model to describe the variation propagation in a multistage machining process. As discussed in the previous section, various coordinate systems are attached to the workpiece, fixture, and cutting tool to represent the position and orientation of workpiece, fixture, and the newly generated features. The definitions of the coordinate systems involved in the derivation are illustrated in Figure 7.9 and listed as follows: • •
•
CS 0 is the reference part coordinate system in this analysis. The primary datum feature is often selected as the reference for convenience. CS 1 and CS 1′ are the nominal and true fixture coordinate systems, respectively. Because of the datum error, the true fixture coordinate system deviates from the nominal fixture coordinate system w.r.t. CS 0. The nominal fixture coordinate system w.r.t. CS 0 is represented by CS 1″ in Figure 7.9. CS 2 and CS 2′ are the nominal and true newly generated feature coordinate systems, respectively. Similarly, the nominal feature coordinate system w.r.t. CS 0 is represented by CS 2″.
2151_book.fm Page 154 Friday, October 20, 2006 5:04 PM
154
Stream of Variation Modeling and Analysis for MMPs
CS 2' CS 2 CS 2" CS 1
CS 0
CS 1' CS 1"
FIGURE 7.9 Coordinate systems involved in the model derivation.
TABLE 7.1 Transformations Among Coordinate Systems From → To CS 0 → CS 1″ CS 1″ → CS 1′ CS 1′ → CS 1 CS 1 → CS 2 CS 2 → CS 2′ CS 1″ → CS 2″
CS 2″ → CS 2′
Representation and Explanation H10′′ — the designed (nominal) value of the fixture w.r.t. CS 0, and is known from product/process design x11′′′ — caused by the datum errors x11′ — caused by the fixture locator errors H12 — the designed (nominal) value representing the cutting-tool path (also the newly generated feature) w.r.t. CS 1 and is known from product/process design x 22′ — caused by machine errors ′′ H12′′ — the designed (nominal) value representing the cutting-tool path (also, the newly generated feature) w.r.t. CS 1″ and is known from product/process design; it is worth ′′ noticing that H12′′ equals H12 2 ′′ x 2′ — the final overall deviation of the newly generated feature from its nominal location and orientation w.r.t. the part coordinate system CS 0
The transformations and the root causes of the deviations among these coordinate systems are listed in the following table: From Table 7.1, it is clear that the deviations x11′′′ caused by datum errors, x1′1 caused by fixture errors, and x 22′ caused by machine errors need to be identified. The summation of those three errors will be the final deviation of the newly generated feature at a single stage. 7.2.3.1 Analysis of Datum-Induced Error The most commonly used fixture scheme in practice is the 3-2-1 fixturing scheme. A general 3-2-1 layout is shown in Figure 7.10. Surface ABCD defines the primary datum plane, which constrains two rotational and one translational motions. ADHE is the secondary datum plane, which constrains
2151_book.fm Page 155 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
H
E
G
XF
OF D F1
OR L3
XR
F2 L2
C
F3 L1
ZF YF
F
P3
P1
P2
155
YR
ZR
A
B
FIGURE 7.10 The 3-2-1 fixture setup.
one rotational and one translational motion. CDHG is the tertiary datum plane, which constrains the last translational motion. The points F1, F2, and F3 are the perpendicular projection points of the locators P1, P2, and P3 on the primary datum. The fixture coordinate system (CS 1′) is shown in Figure 7.10 as OF XFYF ZF . F1F2 is the y-axis of CS 1′, the line passing F3 and perpendicular to F1F2 is the x-axis of CS 1′, and the z-axis is perpendicular to the primary datum plane. The coordinate system attached on the primary datum plane (surface ABCD) is taken as CS 0. The deviation of the secondary datum plane and the tertiary datum plane are represented as x dd 22′ and x dd 33′, respectively. The symbols “di” and “di” represent the coordinate systems attached to the ith nominal and true datum features, respectively. Given x dd 22′ and x dd 33′, the deviation of CS 1′ w.r.t. CS 1″, x11′′′ , can be obtained by Equation 7.14 x11′′′ = T1x dd 22′ + T2x dd 33′
(7.14)
where T1 and T2 are determined by the nominal positions of the secondary and tertiary datum features and the positions of fixture locators. The value of T1 and T2 can be obtained by the following general procedure. Let p1, p2 be the datum points touching the secondary datum and p3 be the datum point touching the tertiary datum. The nominal coordinates of these three points in CS 1′ are denoted as p1,1′ , p 2,1′ , p 3,1′ . Denoting p˜ = [pT 1]T, we have H0d 2′ H10′ p 1,1′ = p 1,d 2′ d 2′ 0 H0 H1′ p 2,1′ = p 2,dd 2′ H d 3′ H0 p = p 3, d 3′ 0 1′ 3,1′
(7.15)
2151_book.fm Page 156 Friday, October 20, 2006 5:04 PM
156
Stream of Variation Modeling and Analysis for MMPs
To guarantee that these points touch the corresponding part surface, the z coordinates of p 1,d 2′ , p 2,d 2′, and p 3,d 3′ should be zeros (the z direction is defined as the normal direction of the surface). Consider the first entry in Equation 7.15. Note that in H d0 2′ = (H d0 2′ )−1 = (H d0 2 ⋅ H dd 22′ )−1 = (H dd 22′ )−1 ⋅ H d0 2 = ( ∆ dd 22′ + I) ⋅ H d0 2 a n d H10′ = H10′′ ⋅ ( ∆11′′′ + I), t h e left-hand side changes to ( ∆ dd 22′ + I) ⋅ H d0 2 ⋅ H10′′ ⋅ ( ∆ 11′′′ + I) p 1,1′ ≈ ( ∆ dd 22′ ⋅ H1d′′2 + H1d′′2 ⋅ ∆11′′′ + H1d′′2 ) p 1,1′
(7.16)
The third element of p 1,d 2′ is zero in order to guarantee touching; therefore, [( ∆ dd 22′ ⋅ H1d′′2 + H1d′′2 ⋅ ∆ 11′′′ + H1d′′2 ) p 1,1′ ](3) = 0
(7.17)
where the subscript “(3)” means the 3rd element of [⋅]. Under nominal conditions, p1, p2, and p3 should touch the datum plane. Hence, because p 1,1′ is the nominal coordinate of the locators in CS 1′, [H1d′′2 ⋅ p 1,1′ ](3) = [H1d′′2 ⋅ p 1,1′′ ](3) = 0 . Equation 7.17 changes to [ ∆ dd 22′ ⋅ H1d′′2 ⋅ p 1,1′ ](3) = [H1d′′2 ⋅ ∆ 11′′′ ⋅ p 1,1′ ](3)
(7.18)
Similarly, we can get the other two equations for p2 and p3. In Equation 7.18, H1d 2 and p 1,1′ are the nominal values that are known from the design specifications, and ∆ dd 22′ contains the datum errors x dd 22′ . The DTM ∆11′′′ contains the differential motion vector x11′′′ . Although there are six parameters in x11′′′ , only three of them are unknown. By solving the equation system consisting of Equation 7.18 and the corresponding equations for p2 and p3, the relationship between x11′′′ , x dd 22′ and x dd 33′ can be obtained and put in the form of Equation 7.14 by rearranging the terms. The expression of T1 and T2 for a general 3-2-1 setup is very complicated. However, if datum features are orthogonal to each other, which is very common in practice, T1 and T2 can be significantly simplified. For example, given the nominal positions for the second and tertiary datum and locating pins in Figure 7.10 as
H1d′′2
0 0 = −1 0
0 1
1 0
0 0
0 0
1 t1d′′2x d2 t1′′ y 0 d3 H = , 1′′ 0 0 0 1
0 0
0 1
−1 0
0 0
t1d′′3x t1d′′3y , 0 1
p1,1′ = [0 p1y p1z ]T , p2,1′ = [0 p2 y p2 z ]T, and p3,1′ = [ p3 x 0 p3z ]T, the relationships can be obtained as Equation 7.14, where
2151_book.fm Page 157 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
0 0 T1 = 0 0 0 0
157
p2 z + t1d"3x +
0 0 0 0 0 0
p2 y ( p2 z − p1z ) p1y − p2 y
0
−1
−t1d"2x
0
0
− p3 x
p3 x ( p1z − p2 z ) p1y − p2 y
0 0
0 0
0 0
0 0
0
0
0
0
0
1
0 p − p2 z − 1z p1y − p2 y
0 0 0 T2 = 0 0 0
0 0
0 −1
0 − p3 z − t1d′′3y
0 p3 x + t1d′′3x
0
0
0
0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0 0 0 0
(7.19)
A very common variation of the general 3-2-1 fixture setup is shown in Figure 7.11.
D P1 P3
C
L3 XF
OR P2
L2
ZF
XR
L1
YF A
YR
ZR
FIGURE 7.11 Another setup of 3-2-1 fixture setup.
B
2151_book.fm Page 158 Friday, October 20, 2006 5:04 PM
158
Stream of Variation Modeling and Analysis for MMPs
The six degrees of freedom of the workpiece are constrained by the plane ABCD (two rotational and one translational motions), a circular short hole P1 and P3 (two translational motions), and a slot P2 (one rotational motion). Given the nominal positions for the secondary the tertiary datum feature and locating pins in Figure 7.11 as
H1d′′2
0 0 = −1 0
0 1 0 0
1 0 0 0
1 0 0 d 3 0 , H1′′ = 0 0 1 0
0 0 −1 0
0 0 , 0 1
0 1 0 0
p1,1′ = [0 0 0]T , p2,1′ = [0 p2 y 0]T , and p3,1′ = [0 0 0]T , the T1 and T2 matrices in Equation 7.14 can be obtained as 0 0 0 T1 = 0 0 0
0 0 0 0 0 0
−1 0 0 0 0 0
0 0 0 0 0 1
0 0 0 0 0 0
0 0 0 0 0 0 , T2 = 0 0 0 0 0 0
0 0 0 0 0 0
0 −1 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 (7.20) 0 0 0
The procedure presented in this subsection can be used to study the datuminduced error for a general 3-2-1 fixture setup. Following this derivation method, similar results can be obtained for other fixture setups. 7.2.3.2 Analysis of Fixture Errors In a general 3-2-1 fixture scheme as shown in Figure 7.10, the workpiece position is located by six locators (P1, P2, P3, L1, L2, and L3). Assume the nominal coordinates of these six points in the nominal fixture coordinate system CS 1 are (0, p1y , p1z), (0, p2y , p2z), (p3x, 0, p3z), (L1x, L1y , 0), (L2x, L2y , 0), and (L3x, L3y , 0), respectively. (Note that we assume the z coordinate of P1 and P2 are the same to simplify the problem). If there are small deviations on these 6 locators, (∆pix, ∆piy , ∆piz) and (∆Lix, ∆Liy , ∆Liz), where i = 1, 2, 3, the true fixture coordinate system CS 1′ will deviate from its nominal CS 1. Cai et al. [21] gave an analytical infinitesimal error analysis for a rigid body locating scheme with general six locating points. The fixture error can be derived based on their results. In Figure 7.10, the surface norm vector for L1, L2, and L3 is (0, 0, 1); for P1 and P2, (–1, 0, 0); and for P3, (0, –1, 0). With the locators’ position vectors, Equation 3.1.6 in Cai et al. [21] can be applied to obtain x11′ = T3 ⋅ [ ∆L1z
∆L2 z
∆L3 z
∆p1x
∆p2 x
∆p3 y ]T
(7.21)
2151_book.fm Page 159 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
159
where T3 = ( L2 y − L3 y ) p1z /C ( L3 x − L2 x ) p3 z /C ( L L − L L ) /C 3y 2 x 2 y 3x ( L2 x − L3 x ) /C ( L2 y − L3 y ) /C 0
(L (L
− L ) p /C ( L − L ) p /C − p / ( p − p ) p / ( p − p ) 0 1 y 1z 1y 2 y 1z 2y 1y 2y 1y 1y 2y ) p /C ( L − L ) p /C p / ( p − p ) − p / ( p − p ) 1 3x 3z 2x 1x 3z 3x 1y 2y 3x 1y 2y 0 0 0 L − L L ) /C ( L L − L L ) /C
3y
1x
(L
−L
3 y 1x
3 x 1y
(L (L
2 y 1x
−L
) /C
(L
−L
2x
3y
− L ) /C
(L
−L
2y
1x
1x
1y
1y
0
0 0
1y 2 x
3x
) /C
0
) /C
0
0 1/ ( p
0
1y
0
0
− p
2y
−1/ ( p
)
1y
− p
2y
)
and C = L3 x L1y − L1y L2 x + L3 y L2 x + L2 y L1x − L2 y L3 x − L3 y L1x . We need to know x1′1 as well. From the properties of DTM, x11′ = − x11′ = −T3 ⋅[ ∆L1z
∆L2 z
∆L3 z
∆p1x
∆p2 x
∆p3 y ]T
(7.22)
For the fixture setup shown in Figure 7.11, the nominal coordinates of P1, P2, and P3 in CS 1 are (0, 0, 0), (0, p2y , 0), and (0, 0, 0), respectively. Substituting these values in the expression of T3, we can obtain another T3 matrix for the particular setup shown in Figure 7.11 as T3 = ( L3 y L 2 x ( L2 x ( L2 y
0
0
0
1
0
0
0
0
0
0
( L3 x L1 y − L3 y L1 x ) /C
( L2 y L1 x − L1 y L2 x ) /C
0
0
− L3 x ) /C
( L3 x − L1 x ) /C
( L1 x − L2 x ) /C
0
0
− L3 y ) /C
( L3 y − L1 y ) /C
( L1 y − L2 y ) /C
0
0
0
0
−1/p2 y
1/p2 y
− L2 y L3 x ) /C
0
0
0 0 0 0 1
The fixture error analysis procedure for a general 3-2-1 fixture scheme is presented in this subsection. For other fixture systems, the analysis can follow the same procedure. 7.2.3.3 Identify the Overall Dimensional Error by Combining Error Sources Together In previous sections, the expressions of x11′′′ because of datum errors and x1′1 because of fixture errors are obtained. To obtain the final overall feature deviation x 22′′′ , these two errors need to be combined with x 22′, which is the machining error of the newly
2151_book.fm Page 160 Friday, October 20, 2006 5:04 PM
160
Stream of Variation Modeling and Analysis for MMPs
CS 2' CS 2' CS 2
CS 2" CS 2"'
CS 2"'
CS 1
CS 1'
CS 1'
CS 1"
(a) Calculation of x 22
(b) Calculation of x 22
FIGURE 7.12 Derivation of x2′2 and x2′2″.
generated feature. This can be achieved by applying Corollary 7.1 twice as shown in Figure 7.12a and Figure 7.12b. Corollary 7.1 can be directly applied to Figure 7.12a to obtain x 22′′′′ by knowing 1′ x1 , H12 , x 22′, and H12′′′′ , where CS 2′′′ is the nominal location of the cutting-tool path w.r.t. CS 1′ and H12′′′′ is taken to be the same as H12 . In this case, we can take H12 as H12′′′ , x1′1 as x11′, and x 22′ as x 22′′′ in Corollary 7.1 to identify x 22′′′′ as (R1 )T x 22′′′′ = 2 0
−(R12 )T ⋅ (tˆ12 ) 1′ 2 ⋅ x1 + x 2′ (R12 )T
(7.23)
Corollary 7.1 can be further directly applied to Figure 7.12b to obtain x 22′′′ by knowing x11′′′ , H12′′′′ that equals H12 , x 22′′′′ , and H12′′′′ that equals H12 . In this case, we can take H12 as H12′′′ , x11′′′ as x11′ , and x 22′′′′ as x 22′′′ in Corollary 7.1 to identify x 22′′′ as (R1 )T x 22′′′ = 2 0
−(R12 )T ⋅ (tˆ12 ) 1′′ 2′′′ ⋅ x1′ + x 2′ (R12 )T
(7.24)
The final expression for the overall feature deviation x 22′′′ can be obtained by substituting Equation 7.23 into Equation 7.24 to get (R1 )T x 22′′′ = 2 0
−(R12 )T ⋅ (tˆ12 ) 1′′ (R12 )T ⋅ x1′ + (R12 )T 0
−(R12 )T ⋅ (tˆ12 ) 1′ 2 ⋅ x1 + x 2′ 1 T (R 2 )
(7.25)
After the overall deviations of the newly generated features are obtained, the modeling of variation propagation can be achieved by linking multiple stages together. The steps are presented in the following subsection.
2151_book.fm Page 161 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
x k-1
S 5. assemble the newly generated feature with other features
S 1. extract information on datum and relocate the part
S 2. gather information on fixture error
161
S3. calculate the datum 1 1 error x1 and fixture error x1
xk
S 4. calculate the overall deviation of the newly generated feature
FIGURE 7.13 Procedures for variation propagation model development.
7.2.3.4 Modeling Variation Propagation in Multistage Machining Processes Figure 7.13 shows the steps of the modeling of variation propagation in multiple stages. The vector of xk is the collection of dimension deviations of all key features on the product. The deviation of each feature is represented by a differential motion vector. If a feature has not been generated after the kth stage, the corresponding components of that feature in xk are set to be zero, and after it has been generated, the zero components are replaced by nonzero deviations. The detailed procedure is presented as follows. In this step, S1, we reorient the part on the current stage and collect information of datum deviation. This is an important step because in single-stage analysis, we always take the primary datum feature as the part coordinate system (CS 0), and the feature deviations are obtained w.r.t. the primary datum feature. Clearly, if the same primary datum as that in the previous stage is used in current stage no extra transformations need be made to the feature deviations obtained from previous stages. However, if the primary datum changes to a different feature in the current stage, a transformation is needed to change the feature deviations into the new CS 0. This can be done using Corollary 7.2. The transformation is illustrated in Figure 7.7. In Figure 7.7, if we treat CS 0 as the primary datum at the previous stage, CS 1′ as the primary datum at the current stage, and CS 2′ as a feature, then deviation of CS 2′ w.r.t. CS 1′ ( x 22′′′ ) can be obtained using Equation 7.13 by knowing H12′′′ and the deviations of CS 1′ and CS 2′ w.r.t CS 0 (i.e., x11′ and x 22′ ). Corollary 7.2 can be applied to all features from the previous stage to transform the deviations into the new part coordinate system. After the transformation is done, the deviations of the secondary and the tertiary datum can be collected. In other words, x dd 22′ and x dd 33′ can be obtained. This is an important step in building the state transition model; it transforms the coordinates of xk–1 from the previous stage to the current stage. In the step S2, the fixture imperfections can be aggregated as a vector uf,k. In the step S3, the datum-induced error x11′′′ can then be obtained by using Equation 7.14 and the fixture error x1′1 using Equation 7.22. The matrices T1 and T2 in Equation 7.14 contribute to the expression of Ak–1, and T3 in Equation 7.22 contributes to the expression of Bk. Following the derivation in Subsection 7.2.3.3, in step S4, the overall deviation of the newly generated feature x 22′′′ can be obtained by combining x11′′′ , x1′1 , and x 22′ .
2151_book.fm Page 162 Friday, October 20, 2006 5:04 PM
162
Stream of Variation Modeling and Analysis for MMPs
This procedure can be applied to all newly generated features to get their predicted deviations, may be noted that for different newly generated features, x11′′′ and x1′1 are kept unchanged whereas x 22′ may change from feature to feature. By substituting Equation 7.14 and Equation 7.22 into Equation 7.25, we can obtain the parts of Ak–1 and Bk corresponding to the newly generated feature. The last step S5, is to assemble the deviations of the newly generated features together with the unchanging features to form xk as the output of the state vector from the current stage. Further, the calculations in steps S1 through S4 can also be assembled together to put the state transition in the form of Equation 7.1. The equation formation is quite straightforward; however, the final expression is fairly tedious. The detailed expression is not considered here. The interested reader can refer to Zhou et al. [22]. After the state space equation is obtained, the observation equation in the state space model is easy to obtain. We can simply use selection matrix as Ck to select the measured features as the output of yk. If the reference coordinate system changes in the measurement station, a transformation similar to S1 transformation is needed in the observation equation. The preceding model describes the deviation propagation among a multistage machining process. An experimental validation of this model is presented in Section 7.3.
7.3 MODEL VALIDATION The deviation propagation model has been validated on an experimental multistage machining process.
7.3.1 INTRODUCTION
TO THE
EXPERIMENTAL MACHINING PROCESS
The product is a V-6 automotive engine head. Its key datum features are shown in Figure 7.14. Joint Face A
H101 B
Longitude Slot S
H104 C
H103
H102
Cover Face M
X3 X2
X1
H105
H106
H107
H108
Z Y1
Y2
(a) Joint face of the engine head FIGURE 7.14 The workpiece: a V-6 engine head.
(b) Cover face of the engine head
2151_book.fm Page 163 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
163
TABLE 7.2 Description of the Operations Operation #
Datum (Primary + Secondary + Tertiary)
Operation Descriptions
1 2 3
(X1, X2, X3) + (Y1, Y2) + Z M + (Y1, Y2) + Z A+B+C
Mill cover face M Mill joint face A, drill hole B and C Mill slot S
TABLE 7.3 Nominal Positions and Orientations of Key Features w.r.t. the Reference Coordinate System Feature Number
Feature Name
Euler Angles
Translational Vector
1 2 3 4 5 6
Surface D Joint face A Cover face M Hole B Hole C Slot
[0, 0, 0] [0, 0, 0] [0, π, 0] [0, 0, 0] [0, 0, 0] [0, π, 0]
[133.7, 134.2, 0] [0, 0, 2.5] [0, 0, –117.03] [0, 0, 2.5] [0, 306, 2.5] [0, 0, –100.9]
For the sake of simplicity, we only consider three operations in this process. Each operation and its corresponding datum setup are described in Table 7.2. In the modeling of variation propagation, we focus on six features. Their nominal positions and orientations are listed in Table 7.3. These numbers are w.r.t. a reference coordinate system. The origin of the reference coordinate is the intersection point between the center line of hole B and the plane determined by rough datum X1, X2, and X3. The y-axis is the line connecting the centers of hole B and hole C, the z-axis coincides with the center line of hole B and points from the cover face to the joint face, and the x-axis is determined by the right-hand principle. It is straightforward to obtain the nominal relationship between any two features from the design parameters listed in Table 7.3. To develop the state space model, the nominal locations of the fixture locators are also needed. Those values are listed in Table 7.4. Note that the nominal fixture locations are given in the coordinate systems attached to the corresponding touching feature (or datum feature). With this known information, a state space model with three stages can be established to link the process root causes such as datum, fixture, and machine errors with the product quality measurements as follows: x k = A k −1x k −1 + B k u k + w k , k = 1, 2, 3
(7.26)
2151_book.fm Page 164 Friday, October 20, 2006 5:04 PM
164
Stream of Variation Modeling and Analysis for MMPs
TABLE 7.4 Nominal Locator Positions w.r.t. the Touching Feature Operation
Datum Primary
#1
Secondary Tertiary Primary
#2 Secondary Tertiary Primary #3
Secondary Tertiary
Locator Positions
Touching Feature Number
[–4.6, –92.2, 0] [–4.6, 128.8, 0] [–169, 38.3, 0] [0, –92.2, –2] [0, 128.8, –2] [0, 0, 0] [42.43, 74, 0] [42.43, 274.5, 0] [–148.07, 157.5, 0] [0, –92.2, –2] [0, 128.8, –2] [0, 0, 0] [10, 95.25, 0] [–95.25, 0, 0] [–95.25, 317.5, 0] [0, 0, 0] [0, 0, 0] [0, 0, 0]
D
M
D
A C B B
yk = Ck xk + vk
(7.27)
The details of the system matrices are listed in the appendix.
7.3.2 COMPARISON BETWEEN THE REAL MEASUREMENT AND THE MODEL PREDICTION To validate the model, a workpiece is machined in the test bed with multistage machining operations. In the machining, a fixture error is intentionally added to the process at each stage. The inputs to the model that correspond to the fixture error are uf,1 = [0 0 –0.2 0 0 0]T, uf,2 = [–0.39 0 0 0 0 0]T, and uf,3 = [–0.39 0 0 0 0 0]T. The nonzero values in uf,k correspond to the mean shift magnitudes of the fixture errors. All the inputs corresponding to the machining error um,k are set as zeros. The overall inputs to the model are an aggregation of uf,k and um,k, i.e., uk = u Tf ,k u Tm ,k . T
The workpiece is measured on a CMM after it is machined. The measurement results from CMM are listed in Table 7.5. For a plane feature, the CMM measurement gives the coordinates of a point on the surface and the norm direction vector of the surface. For the cylinder (hole B and hole C) feature, the CMM measurement gives the
2151_book.fm Page 165 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
165
TABLE 7.5 CMM Measurement Results
Rough datum Joint face Hole B Hole C Cover face Slot
x
y
Z
I
j
k
–361.661 –364.264 –357.147 –358.721 –244.887 –261.027
75.708 103.454 154.084 153.795 103.429 99.433
27.553 35.086 –132.65 173.319 45.558 31.59
–1 1 1 1 1 1
–0.005 0.005 0.005 0.003 0.006 0.008
–0.003 0.005 0.005 0.005 0.003 0.005
Note: Unit — mm for x, y, and z.
coordinate of a point on the cylinder axis, the norm direction vector of the axis, and the diameter. The diameters of the holes are not listed in the table, because they are not used in this study. All the measurements are w.r.t. the global CMM coordinate system. These data can be easily transformed to the same coordinate system used in the state space model. In the state space model, the coefficient matrices A, B, and C and the input to the model u are known. These coefficient matrices are listed in the appendix of this chapter. If we neglect the model noise and measurement noise and set the initial state vector x0 as zero, we can iteratively calculate the state vectors at the following stations using the state space model. In this way, the measurement value can be predicted. The comparison between the real measurement and the model prediction is listed in Table 7.6. For a plane feature, the relevant parameter is its orientation and the distance from the datum plane. The deviation of the orientation of a plane is denoted by [φ, θ, ψ], and the deviation of the distance from the datum plane is denoted by dz. The value of dz is measured at the origin of LCS of the plane w.r.t. the datum plane. For the cylinder feature (hole B and hole C), only the deviation in orientation is considered.
TABLE 7.6 Comparison between the Measurement and Model Prediction Measurement
Cover face w.r.t. rough datum Joint face w.r.t. cover face Hole B w.r.t. cover face Hole C w.r.t. cover face Slot w.r.t. joint face
φ
q
y
0.0000 0.0020 0.0020 0.0020 0.0000
0.0010 –0.0010 –0.0010 –0.0030 0.0030
0.0000 0.0000 0.0000 0.0000 0.0000
Note: Unit — rad for φ, θ, and ψ; mm for dz.
Model Output dz
f
–0.113 0.0000 –0.508 0.0019 — 0.0019 — 0.0019 –0.370 0
q
y
dz
0.0012 –0.0012 –0.0012 –0.0012 0.0037
0 0 0 0 0
–0.157 –0.483 — — –0.353
2151_book.fm Page 166 Friday, October 20, 2006 5:04 PM
166
Stream of Variation Modeling and Analysis for MMPs
From the data presented in Table 7.6, the discrepancies between the model prediction and real measurement of joint face, hole B, and the slot are reasonably small. Because only the fixture errors are taken as the inputs to the model in this case study, these discrepancies are because of other machining errors, such as machine geometric error, thermal error, and force-induced error, that are neglected in the model input. The small discrepancies between the model prediction and real measurement of these three features show that the fixture error is dominant in this machining process. Relatively large discrepancies happen at the cover face and hole C. The large difference at the cover face is understandable because the reference datum feature is the rough datum that contains large natural variation itself. The large difference at hole C is an indication of large geometric error with the machine. Because the joint face, hole B, and hole C are machined at the same time under the same setup, they should have the same orientation that the model predicts if only fixture errors were present. A difference in the orientation between hole C, hole B, and the joint face indicates that the cutting-tool orientation is different at different machine configurations, which is considered a geometric error. Another point that needs to be pointed out is that the deviation patterns (i.e., the sign of each deviation and their relative magnitude) are correctly predicted by the model. This is important because we can conduct root cause identification by considering the error patterns in the measurement. The application of this model to root cause identification will be discussed in Chapter 9 to Chapter 11.
7.4 SUMMARY The complexity of a multistage machining process makes it very difficult to understand and model the variation and its propagation. In this chapter, an analytical linear model is developed to describe the propagation of workpiece geometric deviation among multiple machining stages. Using the state transitions among multiple machining stages, this model describes the geometric error accumulation and transformation when the workpiece passes through the whole process. A systematic procedure is presented to model the workpiece setup and machining process. This model has great potential to be applied in the various fields in variation reduction such as variation analysis of multistage machining process, variation source identification, sensor placement optimization, and fixture design and optimization. All these potential applications are presented in the following chapters of this book.
7.5 EXERCISES In Problem 1 to Problem 3, the following conditions are common. In Figure 7.15, the nominal dimensions of the 2-D workpiece are given in the drawing and expressed in millimeters. The CS0 is defined in Figure 7.15 as (Oxy), and the nominal PCS is defined as being the CS0.
2151_book.fm Page 167 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
167
y f4
A
10
f5 C
f3
18
f2 10
O
f1
15
B
x
FIGURE 7.15 The nominal dimension of the 2-D workpiece (for Problem 1 to Problem 3).
1. Assume the raw material is a square with dimensions 20 mm × 20 mm. The workpiece is mounted in a 2-1 locating system (note you may consider this system as a special 3-2-1 locating system with perfect L1, L2, and L3). The nominal position of the locators given in the CS0 is given in the following table. Also assume that feature f1 is used as the primary datum and feature f5 is used as secondary datum. The goal in this machining stage is to mill the top surface (feature f4). Locating Point
(x,y)
P1 P2 P3
(0,5) (0,13) (8,0)
a. What is the workpiece’s degree of freedom? If locator P1 deviates from its nominal position in the y direction by 0.1 mm b. What is the corresponding deviation of the CS1? c. What is the corresponding deviation of point A on feature f4 in the (x,y)-axis after the machining operation is performed? d. What is the corresponding orientation deviation of feature f4 after the machining operation is performed? 2. Rework Problem 1 questions (b), (c), and (d) if locator P2 is deviated form its nominal direction in the direction of (y) by 0.1 mm. 3. We want to evaluate a different locating scheme. The new positions of the locating points are given in the table as follows: Locating Point
(x,y)
P1 P2 P3
(0,5) (0,7) (8,0)
2151_book.fm Page 168 Friday, October 20, 2006 5:04 PM
168
Stream of Variation Modeling and Analysis for MMPs
a. Rework (1) for the new locating scheme b. Compare with your results obtained in (1); which fixture design would you recommend? 4. Assume the raw material is a rectangle with dimensions 20 mm × 18 mm. at the current stage, milling operation will be performed on the raw material to obtain feature f3 and f2. Suppose that we want to evaluate two fixture designs, and at this stage feature f4 has the deviation found in Problem 1c and 1d, please answer the following questions: Locating Point
(x,y)
P1 P2 P3 fixture design
Locating Point
(0,15) (0,12) (5,18) 1
P1 P2 P3 fixture design
(x,y) (0,15) (0,12) (13,0) 2
a. Given that both sets of locators are at their nominal locations, determine the deviation in the (x,y) axis of point C in the middle of feature f3 and point B on feature f2. What can you conclude? b. Given that locator L3 has some deviation in the y direction of 0.07 mm, determine the deviation in the (x,y) axis of point C in the middle of feature f3 and point B on feature f2. What can you conclude? c. By comparing the results, what fixture design would you recommend? 5. The nominal dimensions of the 2-D workpiece are given in Figure 7.16 and expressed in millimeters. The CS0 is defined in the figure as (Oxy), and the nominal CS1 is defined as being the CS0. The raw material is shown in solid lines and the feature to be created is shown in dashed line in Figure 7.16. Suppose that we want to evaluate two fixture designs. The locator’s nominal position is given in the following table. Suppose that feature f1 is used as primary datum and that feature f2 is used as secondary datum y
f4
A
18
10
15
f2
10
f1
x
O
f3
FIGURE 7.16 The nominal dimensions of the 2-D workpiece (for Problem 5).
2151_book.fm Page 169 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
169
in the first design and all f1,f3, and f2 are used as datums in the second design (note you may revise the T3 in equation 7.21 and equation 7.22 to deal with the locating scheme used in this problem). Locating Point
(x,y)
P1 P2 P3 fixture design
Locating Point
(3,0) (8,0) (0,10) 1
(x,y)
P1 (3,0) P2 (12.5,4) P3 (0,10) fixture design 2
a. What is the workpiece’s degree of freedom for each fixture design? b. What is the sensitivity of the orientation of feature f4 (point A) to a deviation of locator P1 for both fixture designs? c. What is the sensitivity of the orientation of feature f4 (point A) to a deviation of locator P2 for both fixture designs? d. By comparing the results obtained in (b) and (c), which design would you recommend?
7.6 APPENDIX: THE SYSTEM MATRICES USED IN EQUATION 7.27 The system matrices for the state space model used in model validation are as follows: A0 =
0 6× 6 0 30×6
0 6×30 , I30×30
012×6
B1 =
0.2865
0.4132
0.5295
−0.5295
0.8685
−0.6538
−0.0045
0.0045
0.0025
0.0036
0
0
−0.6997
012×6 1.1900
−0.1900
0
0
0.6050
−0.6050
−1
0.7853
0
0
0
0
0
0
0
0
0
0
−00.0061 0
0.0045 018×6
−0.0045
I 6× 6
0 018×6
,
2151_book.fm Page 170 Friday, October 20, 2006 5:04 PM
170
Stream of Variation Modeling and Analysis for MMPs
1
0
0
0
−117.03
−134.2 2
0
−1
0
−117.03
0
133.7
0
0
1
134.2
133.70
0
0
0
0
1
0
0
0
0
0
0
−1
0
0 6× 6
0
0
0
0
0
1
I 6× 6
1
0
0
0
−115.03
0
0 6× 6
0
−1
0
−117.03
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
I 6×6 1
0
0
0
−2
134.2
0
1
0
0
0
−133.7
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0 6× 6
A1 =
018×18
0 6× 6 1
0
0
0
−2
134.2
1
0
0
0
−115.03
0
0
1
0
0
0
−133.7
0
−1
0
−117.03
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
1
1
0
0
0
−2
−171.88
1
0
0
0
−115.03
0
1
0
0
0
−133.7
0
−1
0
−117.03
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
1
0 6× 6
018×6
0 6× 6
−306
,
I18×18
2151_book.fm Page 171 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
171
0 6× 6 −0.0236
0.1378
0.0098
0.0125
−0.0125
0
1.239
−0.4618
0.2227
−0.005
0.005
0.0031
0.0022
0
0
0
1.19
−0.1900
0
−0.6050
0.6050
−1
0
0
0
−0.0052 0
0
0
0
0 0
0 6× 6 I 6× 6
012×12
0
0.00452
−0.00452
1.1900
−0.1900
0
0.6050
−1
0
0 6× 6
B2 =
−0.0236
0.0138
0.0098
0.0125
−0.0125
0
1.2391
−0.4618
0.2227
0
0
0
−0.0050
0.0050
0
0
0
0
0.0031
0.0022
0
0
0
0
0
0.0123
0.0087
0.0100
−0.0100
−0.2871
1.0643
−0.0050
0.0050
0.0031
0.0022
0
0
−0.6050
−0.0052 0
0.0045
−0.0210 0
−00.0045
0
−0.1946
1.1946
0
−0.6050
0.6050
-1
0.2 2227
0
0
0
0
0
0
0
0
0
0
−0.0052 0
0.0045
0 6× 6
−0.0045
018×6
0 6×12 I12×12
0
0 6× 6
0 6×12
,
2151_book.fm Page 172 Friday, October 20, 2006 5:04 PM
172
Stream of Variation Modeling and Analysis for MMPs
I 6×6 0 6×6
−1
0
0
0
2.5
134.2
0
−1
0
−2.5
0
−133.7
0
0
−1
−134.2
133.7
0
0
0
0
−1
0
0
0
0
0
0
−1
0
0
0
0
0
0
−1
0 12 × 24
0 6×6
A2 =
0 6×6
1
0
0
0
−119.525
0
0
−1
0
−119.525
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
−1
0
0
0
0
0
0
1
I 6×6
,
0 6 ×18
− I 6×6
0 18 × 6
−1
0
0
0
0.5
306
0
−1
0
−0.5
0
0
0
0
−1
−306
0
0
0
0
0
−1
0
0
0
0
0
0
−1
0
0
0
0
0
0
−1
1
0
0
0
0
0
−1
0
−1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.0016
1
−0.0033
0
0
0
0
0 18 × 6
I 6×6
0 6×6
0 6×6
I 6×6
0 12 × 6
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.0033
0
0
0
0
0
I 6×6
2151_book.fm Page 173 Friday, October 20, 2006 5:04 PM
State Space Modeling for Machining Processes
−0.9824
B3 =
0.6877
0 30 × 6 0.2947
173
0 30 × 6 1
0
0
0
0.3257
−0.3257
0
−1
0
0.9050
0.3665
−0.2715
0
0
0
−0.0031
0.0031
0
0
0
0.0067
0.0029
0
0
0
0
0
0.0033
0
0 −0.0095 0
I 6×6
.
−0.0033
References 1. Ramesh, R., Mannan, M.A., and Poo, A.N., Error compensation in machine tools — a review, part I: geometric, cutting-force induced and fixture dependent errors, International Journal of Machine Tools and Manufacture, 40, 1235, 2000. 2. Kreng, V.B., Liu, C.R., and Chu, C.N., A kinematic model for machine tool accuracy characterization, International Journal of Advanced Manufacturing Technology, 9, 79, 1994. 3. Lee, E.S., Suh, S.H., and Shon, J.W., A comprehensive method for calibration of volumetric positioning accuracy of CNC-Machines, International Journal of Advanced Manufacturing Technology, 14, 43, 1998. 4. Chen, G., Yuan, J., and Ni, J., A displacement measurement approach for machine geometric error assessment, International Journal of Machine Tools and Manufacture, 41, 149, 2001. 5. Ni, J. and Wu, S.M., An on-line measurement technique for machine volumetric error compensation, ASME Transactions, Journal of Engineering for Industry, 115, 85, 1993. 6. Okafor, A.C. and Ertekin, Y.M., Derivation of machine tool error models and error compensation procedure for three axes vertical machining center using rigid body kinematics, International Journal of Machine Tools and Manufacture, 40, 1199, 2000. 7. Lee, J.H. and Yang, S.H., Statistical optimization and assessment of a thermal error model for CNC machine tools, International Journal of Machine Tools and Manufacture, 42, 147, 2002. 8. Mize, C.D. and Ziegert, J.C., Neural network thermal error compensation of a machining center, Precision Engineering, 24(4): 338–346, 2000. 9. Lo, C., Yuan, J., and Ni, J., Optimal temperature variable selection by grouping approach for thermal error modeling and compensation, International Journal of Machine Tools and Manufacture, 39, 1383, 1999. 10. Yang, J., Yuan, J., and Ni, J., Thermal error mode analysis and robust modeling for error compensation on a CNC turning center, International Journal of Machine Tools and Manufacture, 39, 1367, 1999. 11. Fuh, J., Chang, C., and Melkanoff, M., An integrated fixture planning and analysis system for machining processes, Robotics and Computer-Integrated Manufacturing, 10, 339, 1993. 12. De Meter, E.C., Min-Max load model for optimizing machining fixture performance, ASME Transaction, Journal of Engineering for Industry, 117, 186, 1995.
2151_book.fm Page 174 Friday, October 20, 2006 5:04 PM
174
Stream of Variation Modeling and Analysis for MMPs
13. Hockenberger, M.J. and De Meter, E.C., The application of meta functions to the Quasi-Static analysis of workpiece displacement within a machining fixture, ASME Transactions, Journal of Manufacturing Science and Engineering, 118, 325, 1996. 14. Rong, Y. and Bai, Y., Machining accuracy analysis for computer-aided fixture design verification, ASME Transactions, Journal of Manufacturing Science and Engineering, 118, 289, 1996. 15. Chen, S.G., Ulsoy, A.G., and Koren, Y., Error source diagnostics using a turning process simulator, ASME Transactions, Journal of Manufacturing Science and Engineering, 120, 409, 1998. 16. Mayer, J.R., Phan, A.V., and Cloutier, G., Prediction of diameter errors in bar turning: a computationally effective model, Applied Mathematical Modeling, 24, 943, 2000. 17. Yang, J., Yuan, J., and Ni, J., Real-Time cutting force induced error compensation on a turning center, International Journal of Machine Tools and Manufacture, 37, 1597, 1997. 18. Ramesh, R., Mannan, M.A., and Poo, A.N., Error compensation in machine tools — a review, part II: thermal errors, International Journal of Machine Tools and Manufacturing, 40, 1235, 2000. 19. Paul, R.P., Robot Manipulators: Mathematics, Programming, and Control, The MIT Press, Cambridge, MA, 1981. 20. Henzold, G., Handbook of Geometrical Tolerancing: Design, Manufacturing and Inspection, John Wiley & Sons, New York, 1995. 21. Cai, W., Hu, J., and Yuan, J., 1997, A variational method of robust fixture configuration design for 3-D workpieces, Journal of Manufacturing Science and Engineering, 119, 593, 1997. 22. Zhou, S., Huang, Q., and Shi, J., State space modeling of dimensional variation propagation in multistage machining process using differential motion vectors, IEEE Transactions on Robotics and Automation, 19, 296, 2003.
2151_book.fm Page 175 Friday, October 20, 2006 5:04 PM
8
A Factor Analysis Method for Variability Modeling*
8.1 INTRODUCTION In the last two chapters, a state space model was used to describe variation and its propagation in a multistage manufacturing process. Product and process design information were assumed to be available, and the first principles were used to get the state space model coefficient matrices. The prerequisite for using the methodology is that it must be possible for the variation to be obtained from the design of the product and its process. However, if the variation model cannot be obtained offline from the product and process design, the methodology of the stream of variation (SoV) modeling will not be directly applicable. Apley and Shi [1] have developed a methodology that models and analyzes the variation of a complex system with factor analysis based on product measurement data. This chapter presents a method to facilitate the identification and elimination of root causes of variability in manufacturing processes, in which the coefficient of the latent model cannot be obtained from the design data directly. This method extracts diagnostic information from what would typically be large quantities of multivariate process measurement data. As the technology for automated in-process measurement is becoming more affordable, accurate, reliable, and widely used by manufacturing industries, there are many applications in which the proposed method could be used to reduce variability. The effectiveness of the method will be demonstrated with an example from auto body assembly. The problem formulation and objectives in the methodology presented in this chapter are closely related to the factor analysis problem. The variability patterns (the fault geometry vectors) must be estimated from process data with no a priori knowledge of the faults, except for an assumed model structure. In contrast to traditional factor analysis methods, much greater emphasis has been placed on the underlying physical model that describes the effects of the faults on process variability. The criterion for “rotating” the factors is that the results should correspond closely to the actual faults, as opposed to artificial criteria that seek to aggregate the maximum amount of variability into the minimum number of variables. This allows more physically meaningful interpretations that better facilitate the overall goal of identifying and eliminating the root causes of variability. The remainder of the chapter is organized as follows. Section 8.2 describes the model used to represent the effects of process faults on variability. Section 8.3 discusses the capabilities and limitations of principal component analysis (PCA) and * Part of chapter material is based on Reference 5 (pp. 84–95).
175
2151_book.fm Page 176 Friday, October 20, 2006 5:04 PM
176
Stream of Variation Modeling and Analysis for MMPs
factor rotation for process diagnosis. Section 8.4 discusses methods for estimating the number of faults that are simultaneously present. Section 8.5 presents the main results of the chapter, a technique for estimating the effect of each individual fault when multiple faults are present, and an illustrative example. In Section 8.6, the statistical properties of the estimates are discussed.
8.2 A FACTOR ANALYSIS MODEL FOR PROCESS VARIABILITY 8.2.1 MODEL STRUCTURE One of the keys to diagnosing variability faults is the incorporation of a suitable fault model. This subsection presents the model structure that is assumed throughout this chapter and illustrates with an example from auto body assembly. Let y = [y1, y2, …, yn]T be an n × 1 random vector that represents a set of n measured features from the product or process. Let yi, i = 1, 2, …, N, be a random sample of N observations of y. It is assumed that y obeys the model y = Cu + w
(8.1)
where C = [c1, c2, …, cp] is an n × p constant matrix, u = [u1, u2, …, up]T is a p × 1 zero-mean random vector with covariance matrix ∑u = I, and w is an n × 1 zeromean random vector that is independent of u and has covariance matrix ∑w = σ2I. I denotes the identity matrix of appropriate dimension. It can be shown that the model of Equation 8.1 can be obtained from the state space model of Equation 6.39 and Equation 6.40 if such a model can be obtained from design information, as discussed in earlier chapters. Thus, the model of Equation 8.1 represents the same underlying physics as that of the system described in Chapter 6 and Chapter 7. The interpretation of the model is that there are p separate uncorrelated faults that affect the measurement vector y. Each fault has a linear effect on y that is dictated by the corresponding column of C. Together, ciui describes the effect of the ith fault on y. The direction of ci indicates the nature of the variation pattern caused by the ith fault. Specifically, it indicates how the fault causes the different measured features to vary with respect to each other. ci will be referred to as the “fault geometry” vector. Because the elements of u are scaled to have unit variance, ci also indicates the magnitude or severity of the ith fault. w represents the aggregated effects of measurement noise and any inherent unmodeled variation in the manufacturing process. It is assumed that p ≤ n and that C has full rank p. The focus of this chapter is on faults that act as sources of variation, as opposed to mean shifts. All random variables are assumed to be zero mean. If not, the mean of y should be estimated and subtracted out from the data. The objective is to estimate the number of faults p that are contributing to process variability, as well as each of the p fault geometry vectors in C, using the set of multivariate observations of y. The presumption is that if C can be accurately
2151_book.fm Page 177 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
177
estimated, it may be used to diagnose each of the p faults and aid in the identification and elimination of their root causes. The model and the objective of estimating C are similar to the standard linear orthogonal factor analysis problem [2], in which the elements of u are referred to as the factors, C is referred to as the factor loading matrix, and ∑w is typically only assumed to be diagonal. For the method presented in this chapter to produce estimates of the fault geometry vectors that have the desired interpretability, it is necessary to assume ∑w = σ2I. This can be assumed without loss of generality if w is known up to a constant scalar multiple. Whereas this may not be a reasonable assumption for typical applications of factor analysis, for statistical process control (SPC) of manufacturing processes it is often reasonable. Essentially, it is equivalent to assuming that a sufficient quantity of data has been collected when the process was known to be in control (i.e., when there were no faults present, y = w). In this case, the sample covariance matrix of the in-control data would provide an estimate of ∑w. Before applying the method of this chapter, the data would first be transformed via ∑ −w1/ 2 y, where ∑ −w1/ 2 is the inverse of the square root of ∑w. The noise covariance matrix for the transformed data would then be the identity matrix. After estimating C for the transformed data, the estimates should be transformed back by premultiplying by ∑1w/ 2 before interpreting the results.
8.2.2 INTERPRETATION
OF
VARIATION PATTERNS
Although factor analysis is widely used in the social sciences and other fields, it is seldom used as a tool for diagnosing variability in manufacturing processes. The applicability of the model is illustrated with the following example from auto body assembly shown in Figure 8.1. Figure 8.2 shows the layout of 26 measurement points (labeled 1–26 in the figure) taken at the BIW stage of the assembly process. All three coordinates (x, y, and z in the figure) are measured for each point, except for points 10 and 23, for Laser sensors
z (up) y (right) x (fore)
FIGURE 8.1 Laser sensing at the BIW stage in auto body assembly.
2151_book.fm Page 178 Friday, October 20, 2006 5:04 PM
178
Stream of Variation Modeling and Analysis for MMPs
Rear window opening
Rear door opening
Front door opening 20 22
18
Left bodyside 16 21 17 15
19 26 25 24 Lift-gate support
13
7
Roof header 14
Plenum
12
9
5
11 3 8 4 2
23
6
10 Cross-member
Right bodyside 1
z (up) x (fore) y (right)
FIGURE 8.2 Measurement layout at the BIW stage.
which only the x and z coordinates are measured. Thus, the measurement vector y has a total of n = 76 elements. The cross-member, roof header, and plenum (in addition to the roof, underbody, cowl, and additional roof bows, which have all been omitted from the figure for clarity) join the left and the right body sides to form the BIW. Before the body sides and connecting members are welded together, they must be accurately located with respect to one another with fixtures and then clamped into place. The body sides are positioned in the x-z plane with pins rigidly attached to the fixtures. Each pin mates with either a hole or a slot in the auto body panel, so that the panel position is constrained but not overly constrained. Through repeated use (a thousand panels per day may be placed into each fixture), the pins frequently become worn or loose. In this event, the panel will no longer be constrained to lie in its proper location when it is placed into the fixture. For example, suppose the pin that constrains the right body side in the x direction becomes loose. When a body side is placed into the fixture, it may be positioned too far forward (in the positive x direction) by, say, 1 mm. When it is subsequently clamped into place and welded to the rest of the BIW, it will retain the incorrect position. For the next auto body, the right body side may be positioned too far backward (in the negative x direction). From auto body to auto body, the loose pin will cause a distinct variation pattern in the BIW dimensions. The elements of y that represent measurements in the x direction on the right body side will reflect this pattern when the BIW is
2151_book.fm Page 179 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
179
measured. If we refer to this loose pin as fault 1, u1 is the random variable representing by how much the right body side translates in the x direction for each auto body (scaled to have unit variance). All elements of c1 that do not correspond to x direction measurements on the right body side would be zero. All elements that do correspond to x direction measurements on the right body side would be the same constant value, equal to the standard deviation of the body side translations. Likewise, suppose a second pin becomes loose or worn and causes the right body side to rotate in the x-z plane before being welded to the BIW. This, referred to as fault 2, will also cause a distinct variation pattern in the BIW measurements that is well represented by the model of Equation 8.1. The c2 vector will be determined by the measurement layout and the geometry of the panel and fixture. Apley and Shi [3] provide details on how to analytically model the effects of tooling faults and give a justification for the linear structure of model of Equation 8.1. Technically, the rotation of a panel is nonlinear in the parameters, but is closely approximated as linear when small angles of rotation are involved. The linear model structure is quite versatile and provides a good representation of a variety of commonly encountered faults, including those that are not rigid body translations and rotations. This includes variability introduced by stamping, welding, and material-handling faults, in which case the model of Equation 8.1 can be viewed as a linearization of a more accurate nonlinear model [3]. Remark 8.1: We point out that it is not necessary to analytically model the faults to use the method presented in this chapter. This is the main distinction between this work and that of Apley and Shi [3], or SoV modeling techniques presented in Chapter 6 and Chapter 7. In Apley and Shi [3] it is assumed that an exhaustive set of potential faults can be analytically modeled offline to obtain the C matrix. The problem then reduces to one of fault classification, where, based on the online measurements, one seeks to identify which of the modeled faults is present. The method of this chapter attempts to estimate C directly from the data, with no a priori knowledge of the faults, and use the results to gain insight into the root causes of the variability. To uniquely estimate C with no a priori knowledge of the faults, an additional assumption must be made regarding the structure of C. The reason this assumption is necessary is discussed in Section 8.3. It is assumed that C has the ragged lower triangular form c1,1 c 2,1 C = c 3,1 c p,1
c 2,2 c 3,2
c 3,3
c p,2
c p,3
c p, p
(8.2)
where ci,j is an ni × 1 vector with ni ≥ 2 and ∑ip=1 ni = n. The interpretation of this structure is that there exists a subgroup of n1 measurements {y1, y2, …, yn1} that are
2151_book.fm Page 180 Friday, October 20, 2006 5:04 PM
180
Stream of Variation Modeling and Analysis for MMPs
affected by only one fault and not by the remaining p – 1 faults. Furthermore, there must exist a second subgroup of n2 measurements {yn1+1, yn1+2, …, yn1+n2} that are affected by only one of the remaining p – 1 faults. Note that these measurements may also be affected by the first fault. There must also exist a third subgroup of measurements affected by only one of the remaining p – 2 faults, and so on. In the previous example, {y1, y2, …, yn1} could be taken to be any set of z direction measurements on the right body side. Such a group would be affected by fault 2, which causes the right body side to rotate, but not by fault 1, which causes only an x-direction translation of the body side and does not affect the z direction measurements. Upon appropriate reordering of the measurements and faults, C has the structure in Equation 8.2. If the faults are such that C does not have the assumed structure of Equation 8.2, the method of this chapter cannot be applied. When n is large relative to p, this structure is often satisfied in auto body assembly. An additional example in which this is the case will be given in Subsection 8.5.3.
8.3 LIMITATIONS OF PCA AND FACTOR ROTATION From the model structure and assumptions, the covariance matrix of y is ∑ y = E[(Cu + w)(Cu + w)T ] = CCT + σ 2I
(8.3)
Let zi, i = 1, 2, …, n, denote an orthogonal set of unit norm eigenvectors of ∑ y , Let λi, i = 1, 2, …, n, denote the corresponding eigenvalues, arranged in descending order. It follows from Equation 8.3 that λ1 ≥ λ2 ≥ … ≥ λp > σ2 = λp+1 = λp+2 = … = λn, and span{Zi}ip=1 = span{ci}ip=1 . PCA can also be used to decompose ∑ y in terms of its eigenvectors and eigenvalues as p
n
∑y =
∑ i =1
λ i zi zTi =
∑( i =1
)
λ i − σ 2 zi zTi + σ 2
n
∑z z
T i i
i =1
= Z p Λ p − σ 2I ZTp + σ 2I (8.4)
where Zp = [z1, z2, …, z p], and Λp = diag{λ1, λ2, …, λ p}. Comparing Equation 8.3 and Equation 8.4, it is obvious that one possible estimate of C is Z p [Λ Λ p − σ 2I]1/ 2 . Another possible estimate that would preserve the covariance structure is Z p [Λ p − σ 2I]1/ 2 Q, where Q is any p × p orthogonal matrix. When there is a single fault present, PCA is an effective tool for diagnosing process variability. ∑y has a single dominant eigenvalue, and a unique estimate of C (= c1) is Z1[λ1 – σ2]1/2. σ2 can be taken to be any of the smallest n – 1 eigenvalues. In practice, one must work with the sample covariance matrix. In this case, it may not be clear how many eigenvalues are “dominant” and, thus, how many faults are present. Methods of estimating the number of faults from the sample covariance matrix are discussed in Section 8.4. When multiple faults are present (p > 1), the fault geometry vectors will not correspond one-to-one with the eigenvectors unless the fault geometry vectors happen
2151_book.fm Page 181 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
181
to be orthogonal. PCA does provide an estimate of the number of faults that are present (refer to Section 8.4), but estimation of C is less straightforward. As discussed earlier, the estimate is not unique. Treating the eigenvectors as (scaled) estimates of the fault geometry vectors may yield noninterpretable results and provide little useful diagnostic information. To improve interpretability, a commonly used procedure is factor rotation [2,4]. The standard factor rotation problem is to find the p × p orthogonal matrix Q so that the resulting estimate of C (typically of the form Z p Λ1p/ 2Q ) provides the clearest interpretability. What is meant by “interpretability” is subjective, but the most widely used criterion is the varimax rotation [2,4]. The varimax criterion for best interpretability is that each column of Z p Λ1p/ 2Q should consist of elements that are either very large in magnitude or very small in magnitude with as few moderate-sized elements as possible. For diagnosis of manufacturing variability, there is little justification for the varimax criterion. The Q matrix that results in the clearest interpretability would be such that Z p [Λ p − σ 2I]1/ 2 Q = C, whose columns are the physical fault geometry vectors themselves. That is, knowing the fault geometry vectors for the actual faults will surely provide the most effective root cause diagnosis. As shown in Section 8.5, C may be uniquely identified if it has the structure assumed in Equation 8.2.
8.4 ESTIMATING THE NUMBER OF FAULTS The method for estimating C that will be presented in section 8.5 requires an estimate of the number of faults present. If p faults are present, λ1 ≥ λ2 ≥ …≥ λp > σ2 = λp+1 = λp+2 = …= λn. Thus, a natural means of estimating p is to look at the eigenvalues ˆ . In this chapter, the symbol “^” will be ˆ i} in=1 of the sample covariance matrix ∑ {λ y used to denote an estimate of a parameter. A number of methods for estimating p have been suggested, the most popular of which can be classified as either maximumlikelihood based or information based. Strictly speaking, these methods require that the data be multivariate Gaussian.
8.4.1 LIKELIHOOD RATIO TEST Anderson [5] developed results for the asymptotic distribution of eigenvectors and eigenvalues of a sample covariance matrix for Gaussian data. This yields an asymptotically (for large enough N) valid likelihood ratio test of the null hypothesis that λm+1 = λm+2 = … = λn, for some fixed m, in contrast to the alternative hypothesis that not all of the n–m smallest eigenvalues are equal. Note that when m ≥ p, the null hypothesis holds. For m = 0, 1, …, n – 1, one calculates the test statistics a Λ(m )= N ( n − m ) log m gm where a m and g m are the arithmetic and geometric means, respectively, of the n – m ˆ . Under the null hypothesis, Λ(m) is asymptotically chismallest eigenvalues of ∑ y squared distributed with (n – m)(n – m + 1)/2 – 1 degrees-of-freedom. A set of
2151_book.fm Page 182 Friday, October 20, 2006 5:04 PM
182
Stream of Variation Modeling and Analysis for MMPs
thresholds η(m), m = 0, 1, …, n – 1, are specified, typically based on the null distribution. The suggested procedure for estimating p is, while increasing m sequentially from 0, to choose pˆ to be the first m for which Λ(m) < η(m). To improve the chi-squared approximation to the null distribution for finite N, Lawley [6] introduced the modified statistic
(
) ( ) ( )
2 m 2 n−m + n−m +2 1 Λ mod (m ) = 1 − − + N N 6N n − m
m
∑ i =1
2 am Λ(m ) λ − a i m
which has the same asymptotic chi-squared null distribution with (n – m)(n – m + 1)/ 2 – 1 degrees of freedom. The procedure for estimating p is the same.
8.4.2 AIC
AND
MDL INFORMATION CRITERIA
Alternative procedures for estimating p are based on the Akaike (AIC) and minimum description length (MDL) information criteria introduced by Akaike [7], Schwartz [8], and Rissanen [9]. For the problem of estimating the number of significant factors in PCA, the AIC criterion was applied in Akaike [10] and Wax and Kailath [11], and the MDL criterion was applied in Wax and Kailath [11]. The AIC and MDL tests require calculation of, for m = 0, 1, …, n – 1, a AIC (m )= N ( n − m ) log m + m (2 n − m ) , and gm a MDL( m )= N (n − m) log m + m(2n − m) log( N ) /22 gm Using the AIC or MDL criterion, pˆ is chosen to be the m that minimizes AIC(m) or MDL(m), respectively. One major advantage of using either the AIC test or MDL test is simplicity. For the likelihood ratio tests one must select, somewhat arbitrarily, the set of thresholds. The information-based tests do not require this. Some properties of the various methods of estimating p are addressed in the following remarks: Remark 8.2: The likelihood ratio methods often lead to overestimating p and choosing a higher number of factors than can be interpreted [12]. The MDL and AIC criteria are less dependent on the validity of the asymptotic chi-squared approximation. Remark 8.3: The AIC estimate is not consistent (in the sense of yielding the true number of faults with probability one as N approaches infinity), whereas the other three methods do produce consistent estimates [11]. The AIC method slightly overestimates p asymptotically.
2151_book.fm Page 183 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
183
8.4.3 AN ILLUSTRATIVE EXAMPLE For some problems of typical scale (i.e., typical n, N, and p) in auto body assembly, we present a Monte Carlo comparison of the different methods. 10,000 Monte Carlo trials were used for all simulations, and the generated data were Gaussian. Note that, as shown in the appendix, the distribution of each of the four test statistics depends only on n, N, and {λ i /σ 2}ip=1 , and not on the individual eigenvectors. Table 8.1 through Table 8.3 show the probability mass functions of pˆ for various cases when at least
TABLE 8.1 Probability Mass Function of pˆ from AIC for Various n, N, p λi /σ σ2}i=1 and {λ pj ≡ Probability {pˆ = j} n
N
40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 200 200 200 200 200 200 200 200 200 200 200
50 50 50 50 50 50 100 100 100 100 100 100 500 500 500 500 500 500 250 250 250 250 250 250 800 800 800 800 800
p λi /σ σ2}i=1 {λ
{11,11,11} {5,5,5} {3,3,3} {11} {5} {3} {11,11,11} {5,5,5} {2,2,2} {11} {5} {2} {11,11,11} {3,3,3} {2,2,2} {11} {3} {2} {11,11,11} {5,5,5} {3,3,3} {11} {5} {3} {5,5,5} {3,3,3} {2,2,2} {3} {2}
p0
p1
p2
p3
p4
p5
0 0 0.084 0 0.006 0.365 0 0 0.294 0 0 0.608 0 0 0 0 0 0 0 0 0.003 0 0 0.181 0 0 0 0 0.047
0 0.002 0.334 0.975 0.973 0.621 0 0 0.446 0.962 0.961 0.379 0 0 0 0.914 0.917 0.918 0 0 0.122 1 1 0.819 0 0 0.009 1 0.953
0 0.067 0.394 0.024 0.02 0.014 0 0 0.22 0.038 0.039 0.013 0 0 0 0.085 0.082 0.081 0 0 0.567 0 0 0 0 0 0.319 0 0
0.951 0.892 0.18 0.001 0.001 0 0.941 0.947 0.039 0 0 0 0.906 0.914 0.927 0.001 0.001 0.001 1 1 0.308 0 0 0 1 1 0.672 0 0
0.046 0.037 0.008 0 0 0 0.058 0.052 0.001 0 0 0 0.091 0.083 0.071 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.003 0.002 0 0 0 0 0.001 0.001 0 0 0 0 0.003 0.003 0.002 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Note: The box corresponding to the actual value of p is shaded.
p6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
2151_book.fm Page 184 Friday, October 20, 2006 5:04 PM
184
Stream of Variation Modeling and Analysis for MMPs
TABLE 8.2 Probability Mass Function of pˆ from MDL for Various n, N, p λi /σ σ2}i=1 and {λ pj ≡ Probability {pˆ = j} n
N
40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 200 200 200 200 200 200 200 200 200 200 200
50 50 50 50 50 50 100 100 100 100 100 100 500 500 500 500 500 500 250 250 250 250 250 250 800 800 800 800 800
p σ2}i=1 λi /σ {λ
{11,11,11} {5,5,5} {3,3,3} {11} {5} {3} {11,11,11} {5,5,5} {2,2,2} {11} {5} {2} {11,11,11} {3,3,3} {2,2,2} {11} {3} {2} {11,11,11} {5,5,5} {3,3,3} {11} {5} {3} {5,5,5} {3,3,3} {2,2,2} {3} {2}
p0
p1
p2
p3
0 0.39 0.995 0 0.577 0.997 0 0.005 1 0 0.047 1 0 0 0.931 0 0 0.96 0 0.995 1 0 1 1 0 1 1 1 1
0 0.405 0.005 1 0.423 0.003 0 0.029 0 1 0.953 0 0 0 0.067 1 1 0.04 0 0.005 0 1 0 0 0 0 0 0 0
0.008 0.168 0 0 0 0 0 0.233 0 0 0 0 0 0 0.002 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.992 0.037 0 0 0 0 1 0.733 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0
p4
p5
p6
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Note: The box corresponding to the actual value of p is shaded.
one fault is present, using AIC, MDL, and Λmod, respectively. The values are rounded to three decimal places. The results using the unmodified likelihood ratio statistic Λ are not shown. Except for the case where n = 40 and N = 500, this method greatly overestimated the number of faults and will not be further considered. For the Λmod method, all thresholds were chosen to be the upper 0.001 percentile of the appropriate chi-squared distribution. The values of {λ i /σ 2}ip=1 were chosen to span the range of what can be considered small (in terms of the tests having difficulty in detecting the
2151_book.fm Page 185 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
185
TABLE 8.3 Probability Mass Function of pˆ from Λmod for Various n, N, p λi /σ σ2}i=1 and {λ pj ≡ Probability {pˆ = j} n
N
40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 200 200 200 200 200 200 200 200 200 200 200
50 50 50 50 50 50 100 100 100 100 100 100 500 500 500 500 500 500 250 250 250 250 250 250 800 800 800 800 800
p λi /σ σ2}i=1 {λ
{11,11,11} {5,5,5} {3,3,3} {11} {5} {3} {11,11,11} {5,5,5} {2,2,2} {11} {5} {2} {11,11,11} {3,3,3} {2,2,2} {11} {3} {2} {11,11,11} {5,5,5} {3,3,3} {11} {5} {3} {5,5,5} {3,3,3} {2,2,2} {3} {2}
p0
p1
p2
p3
p4
p5
p6
0 0.002 0.289 0.005 0.315 0.678 0 0 0.858 0 0.108 0.985 0 0 0 0 0 0.419 0 0 0 0 0 0 0 0 0.289 0.301 0.939
0 0.111 0.31 0.878 0.573 0.212 0 0 0.12 0.999 0.891 0.013 0 0 0.012 0.998 0.998 0.578 0 0 0 0 0 0 0 0.003 0.475 0.693 0.055
0.033 0.462 0.202 0.047 0.047 0.045 0 0.358 0.017 0 0 0 0 0 0.669 0.001 0.001 0.002 0 0 0 0 0 0 0 0.461 0.208 0.005 0.005
0.867 0.319 0.099 0.027 0.021 0.02 0.998 0.639 0.004 0.001 0.001 0.002 0.998 0.999 0.318 0 0 0 0 0 0 0 0 0 0.995 0.523 0.025 0.001 0.001
0.034 0.04 0.038 0.014 0.016 0.016 0.001 0.002 0.001 0 0 0 0.001 0.001 0.001 0 0.001 0.001 0 0 0 0 0 0 0.005 0.004 0.003 0 0
0.021 0.022 0.022 0.011 0.007 0.008 0.001 0.001 0 0 0 0 0.001 0 0 0.001 0 0 0 0 0 0 0 0 0 0.003 0 0 0
0.021 0.016 0.011 0.005 0.01 0.011 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.002 0 0 0
Note: The box corresponding to the actual value of p is shaded.
faults) to large (in terms of the tests detecting the faults with high probability). Note that for the case p = 1, c1T c1 /σ 2 = λ1 /σ 2 − 1, c1T c1 /σ 2 can be viewed as the total variance (summed over all n measurements) due to the fault, divided by the average variance (averaged over all n measurements) due to the noise. Table 8.4 shows the probability of correctly concluding p = 0 when there are truly no faults present. This situation is of particular interest if an eigenvalue test is used to detect the presence of a fault. In this event, the probabilities shown in
2151_book.fm Page 186 Friday, October 20, 2006 5:04 PM
186
Stream of Variation Modeling and Analysis for MMPs
TABLE 8.4 Probability That pˆ = 0 when No Fault Is Present, for Various n and N Probability {pˆ = 0} n
N
AIC
MDL
mod
20 20 20 20 50 50 50 50 100 100 100 200 200
50 100 200 500 75 150 250 500 120 200 500 250 500
0.926 0.911 0.879 0.858 0.991 0.978 0.968 0.947 1.000 1.000 0.992 1.000 1.000
1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000
0.998 0.999 0.999 0.999 0.952 0.998 0.999 0.999 0.016 0.979 1.000 0.000 0.961
Table 8.4 are equal to one minus the alpha error. The values in Table 8.4 are also rounded to three decimal places. The results in Table 8.1 through Table 8.3 agree with the fact that the MDL and Λmod methods provide consistent estimates of p, whereas the AIC method asymptotically overestimates p (consider the n = 40, N = 500 cases). On the other hand, the AIC method was generally able to detect much smaller magnitude faults than either the MDL or Λmod methods. For example, consider the case when n = 40, N = 100, and three faults are present with {λ i /σ 2}ip=1 = {5,5,5}. The AIC method correctly estimates p with 0.947 probability, whereas MDL and Λmod correctly estimate p with only 0.733 and 0.639 probability, respectively. Consider also the cases where n = 200, N = 250, and {λ i /σ 2}ip=1 = {5,5,5}, {3,3,3}, {5}, or {3}. In these situations, the AIC method provides much more accurate estimates of p. Furthermore, for n (large) and N (not sufficiently large), Λmod greatly overestimates p. Although not shown in Table 8.3, for n = 200 and N = 250, Λmod often estimates p to be greater than 20. These observations also extend to the case when there are no faults present, as shown in Table 8.4. The alpha error is generally very low for the MDL method and slightly higher for the AIC method. Note that for all cases the alpha error for the MDL is zero to three decimal places. For large N/n (e.g., n = 20 or 50, N = 500), Λmod (which is consistent) has a lower alpha error than AIC (which asymptotically overestimates p). For small N/n (e.g., n = 100, N = 120), AIC still has a very small alpha error, whereas the alpha error for Λmod is close to one. In light of this, we suggest using either the AIC or MDL methods. If smallmagnitude faults are expected and it is desired that they be detected, the AIC method is more suitable. Otherwise, the MDL method is preferable.
2151_book.fm Page 187 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
187
8.5 UNIQUE IDENTIFICATION OF MULTIPLE FAULTS 8.5.1 ESTIMATING
THE
FAULT GEOMETRY VECTORS
Assume y follows the model of Equation 8.1, where C has the structure in Equation 8.2, and further assume that ∑ y is known. Assume also that one has identified a subgroup of measurements that are affected by a single fault, for which a procedure is given in Subsection 8.5.2. Define the “latent” covariance matrix ∑C to be the component of ∑ y that is due to the underlying latent variables that represent the faults, i.e., ∑C = Z p Λ p − σ 2I ZTp = CCT = c1,1 c 2,1 = c1T,1 c p,1
c
T 2 ,1
∑
p i =1
c i c iT
0 c + 0 T p ,1
0 ∑C( 2)
(8.5)
The last equality follows from the assumed structure of C. From Equation 8.5, the upper-left n1 × n1 block of ∑C is the rank-1 matrix c1,1c1T,1 , which has a single nonzero eigenvalue λ1,1 = c1T,1c1,1 with eigenvector z1,1 = c1,1λ1−,11/ 2 . Thus, the first fault geometry vector can be obtained from ∑C via z1,1λ1−,11/ 2 ∑C = 0 c1,1 0 c 2,1 T T T c1,1 c 2,1 c p,1 + 0 c p,1
0 c1,1 c1T,1c1,1 ∑C( 2) 0
(
)
−1
c1,1 c = 2,1 = c 1 c p,1
After identifying c1, deflate the latent covariance matrix via ∑ C − c1c1T =
∑
0 c i c Ti = i=2 0 p
0 ∑ C( 2)
where
∑ C( 2)
c 2,2 c 3,2 T c = 2,2 c p,2
c T3,2
0 c Tp,2 + 0
0 ∑ C(3)
2151_book.fm Page 188 Friday, October 20, 2006 5:04 PM
188
Stream of Variation Modeling and Analysis for MMPs
The upper-left n2 × n2 block of ∑ C( 2) is the rank-1 matrix c2,2 cT2,2 , which has a single nonzero eigenvalue λ 2,2 = cT2,2 c2,2 with eigenvector z 2,2 = c2,2 λ 2−,12/ 2 . Thus, c2 can be obtained via 0 ∑ C − c1c1T z 2,2λ −2,12/ 2 = 0 0 0 c 2,2 T T 0 c 2,2 c p,2 + 0 0 c p,2
0 0 T 0 c 2,2 c 2,2c 2,2 ∑ C(3) 0
0 0 0
(
)
−1
= c2
The entire process can be repeated, each time deflating the latent covariance matrix with the most recently identified fault geometry vector, until cp is identified. At the final stage, C is completely determined. ˆ must In practice, ∑ y will not be known, and the sample covariance matrix ∑ y be used instead. We suggest using the preceding procedure with all quantities ˆ pˆ and Λˆ pˆ from replaced by their estimates. Let pˆ denote an estimate of p. Form Z ˆ 2 ˆ the p largest eigenvalues/eigenvectors of ∑ y , and estimate σ via n
σˆ 2 = ( n − pˆ )−1
∑ λˆ
i = p +1
i
(8.6)
Anderson [5] has shown that Equation 8.6 is asymptotically the maximum ˆ =Z ˆ ˆ Λˆ ˆ − σˆ 2I Z ˆT likelihood estimate of σ2. An estimate of ∑ C would then be ∑ y p p pˆ . At each stage, the estimate of ci is ˆ − cˆ i = ∑ C
i −1
∑ j=0
0 cˆ cˆ zˆ i ,i λˆ i−,1i / 2 0 T j j
(8.7)
λˆ i,i and zˆ i,i denote the dominant eigenvalue/eigenvector pair of the ni × ni block of the deflated latent covariance matrix, corresponding to the subgroup of measurements affected by only one of the remaining faults. The subgroups can be identified by the procedure given in Subsection 8.5.2. Although the upper-left n1 × n1 submatrix of ∑ C is exactly rank 1, the correˆ will not necessarily be so. Its rank will be close to –1, sponding submatrix of ∑ C ˆ however, if ∑ C is sufficiently close to ∑ C . One eigenvalue of the upper-left n1 × n1 ˆ will then be much larger than the others, and it should be clear how to block of ∑ C
2151_book.fm Page 189 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
189
choose λˆ 1,1. Likewise for λˆ i,i , i = 2, 3, …, pˆ . This issue is related to whether the subgroups affected by only one fault have been appropriately selected, and will be further discussed in subsection 8.5.2.
8.5.2
IDENTIFYING SUBGROUPS
The accuracy in estimating the fault geometry vectors depends on, among other factors, whether the subgroups of measurements affected by only one of the remaining faults can be identified. This subsection discusses a simple procedure to accomplish this. If {x1, x2, …, xn1} is a subgroup affected by only one fault, the submatrix of ∑ C corresponding to this subgroup is the rank-1 matrix c1,1c1T,1 . Thus, to identify the first subgroup it is necessary to find a set of measurements whose latent covariance matrix has rank 1. Define R C to be the latent correlation matrix associated with ∑ C , obtained in the usual way by scaling each element of ∑ C by the square roots of the corresponding diagonal elements. A submatrix of ∑ C will have unit rank 1 if and only if all elements of the corresponding submatrix of R C have unit magnitude. This follows from the fact that the covariance matrix of a set of random variables has unit rank 1 if and only if the variables are linearly related, which occurs if and only if their correlation coefficients are either 1 or –1. Consequently, finding a subgroup of measurements whose latent covariance matrix has unit rank 1 reduces to finding a subgroup whose latent correlation matrix has all elements either 1 or 1. In the event that ∑ C is known, this can be easily accomplished by inspection of the elements of R C . If ∑ C is estimated, no submatrix of Rˆ C will have all unit magnitude elements with positive probability. Therefore, Rˆ C must be searched for submatrices whose elements are all close to one in magnitude. If n is large, we recommend using a clustering technique with the magnitude of the latent correlation coefficients as the similarity measure. We have found agglomerative hierarchical methods with complete linkage to work well in practice. Refer to Everitt [13] for details. The result of the clustering will be a set of candidate subgroups, one of which must be selected as the first subgroup {x1, x2, …, xn1}. To accomplish this, we ˆ that correspond to recommend calculating the eigenvalues of the submatrices of ∑ C each candidate subgroup. For notational convenience, denote the eigenvalues for a given subgroup (of size n1) as {λ i }in1= 1, arranged in descending order. We suggest choosing the subgroup that maximizes the criterion λi
(8.8)
n1
( n1 − 1)
−1
∑λ
i
i =1
This criterion is the ratio of the largest eigenvalue to the average of the remaining eigenvalues, and helps to ensure that the rank of the corresponding submatrix ˆ is close to 1. After identifying the first subgroup, the first fault geometry vector of ∑ C ˆ is deflated as described in Subsection 8.5.1. The preceding is estimated and ∑ C
2151_book.fm Page 190 Friday, October 20, 2006 5:04 PM
190
Stream of Variation Modeling and Analysis for MMPs
procedure can then be repeated on the deflated latent covariance matrix to identify the second subgroup, and so on.
8.5.3 FAULT INTERPRETATION
AND ILLUSTRATIVE
EXAMPLE
The estimated fault geometry vectors can provide powerful diagnostic tools for identifying root causes of process variability, as illustrated in the following example. Consider again the measurement layout on the BIW described in Subsection 8.2 and shown in Figure 8.2. The sample consists of N = 200 auto bodies, produced and measured over a 4-hr period. The measurements are in units of millimeters. All three ˆ was formed from the four dominant methods discussed in Section 8.4 gave pˆ = 4. ∑ C 2 eigenvalue/eigenvector pairs, with σˆ = 0.040 estimated from the remaining 72 eigenvalues. Using the clustering procedure outlined in Subsection 8.5.2, a number of candidate subgroups were found. The subgroup {11X, 13X, 15X, 16X, 17X, 18X, 24X, 25X} maximized Equation 8.8 and was selected to be the first subgroup. The ˆ were {0.2873, 0.0008, 0.0004, eigenvalues of the corresponding submatrix of ∑ C 0.0004, 0, 0, 0, 0}, and the corresponding submatrix of Rˆ C was 1 .994 .997 .991 .995 .985 .993 .998
.994 1 .994 .985 .997 .982 .998 .991
.997 .994 1 .996 .998 .994 .997 .999
.991 .985 .996 1 .996 .989 .993 .995
.995 .9997 .998 .996 1 .987 .999 .995
.985 .982 .994 .989 .987 1 .989 .994
.993 .998 .997 .993 .999 .989 1 .994
.998 .991 .999 .995 .995 .994 .994 1
Thus, it appears that subgroup one is affected by only a single fault. cˆ 1 , estimated via Equation 8.7, and is shown graphically in Figure 8.3. The length of each arrow is the Six Sigma value, due to fault 1, for that coordinate, i.e., the arrow length is 6 times the magnitude of the corresponding element of cˆ 1. To make the plot less cluttered, an arrow was omitted if the Six Sigma level was less than 0.25 mm (deemed insignificant from a practical viewpoint). Fault 1 appears to affect only fore/aft direction coordinates, except for a relatively minor effect on point 1Y. Moreover, it appears to cause all points on the vehicle to move by approximately the same amount in the fore/aft direction. In other words, the entire vehicle is translating in the fore/aft direction. Because the fault does not appear to cause the auto body panels to vary relative to each other, it was suspected that the cause was measurement error. Investigation quickly revealed the following. In the measurement station, the vehicle's position is fixed via a pin/hole/slot combination in the underbody, as described in Section 8.2. The same underbody hole is used to locate the vehicle throughout the assembly process, prior to measurement. Through repeated use, the hole in each underbody was significantly enlarged in the fore/aft direction. This allowed the vehicles to translate in the fore/aft direction when placed into the measurement station fixture, resulting in the measurement error
2151_book.fm Page 191 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
191
1.0 mm 1.0 mm
FIGURE 8.3 Graphical illustration of cˆ 1 for the example.
represented by fault 1. The problem was solved by changing the design of the pin/hole. ˆ was deflated and the procedure was repeated. After estimating the first fault, ∑ C The clustering procedure produced as a second subgroup {1X, 2X, 3X, 4X, 5X, 6X, 7X, 8X, 9X}. The eigenvalues of the corresponding submatrix of the deflated latent covariance matrix were {0.5205, 0.0018, 0.0008, 0, 0, 0, 0, 0, 0}, and the corresponding correlation matrix contained elements mostly greater than 0.99 in magnitude. cˆ 2, estimated via Equation 8.7, is shown graphically in Figure 8.4. It appears the second fault also affects only fore/aft direction points. In contrast to fault 1, fault 2 only affects points on the right body side, causing each point to translate by approximately the same amount. Thus, it appears fault 2 results in the right body side being incorrectly positioned with respect to the rest of the vehicle, identical to the variation pattern described in Section 8.2. From this it was suspected that, in the station in which the body sides are joined to the roof and underbody, the pin that locates the right body side in the fore/aft direction was loose or worn. Investigation revealed that it was loose. The latent covariance matrix was deflated again, and the procedure was repeated twice more to estimate the two remaining fault geometry vectors. Although not shown, faults three and four affected the y-direction measurements predominantly and also had clear explanations. One of the faults was because of a misaligned robotic weld gun. Together, the four fault geometry vectors resulted in a C matrix with the structure of Equation 8.2. These results can be compared with standard PCA, in which the four dominant ˆ are directly plotted. Two of the eigenvectors, zˆ and zˆ , are eigenvectors of ∑ y 1 3 shown in Figure 8.5 and Figure 8.6.
2151_book.fm Page 192 Friday, October 20, 2006 5:04 PM
192
Stream of Variation Modeling and Analysis for MMPs
1.0 mm 1.0 mm
FIGURE 8.4 Graphical illustration of cˆ 2 for the example.
1.0 mm 1.0 mm
FIGURE 8.5 Graphical illustration of zˆ 1 in the example.
Interpretation of zˆ 1 could possibly have led to the same conclusion as cˆ 1 , although zˆ 1 makes it appear that the fault causes the right body side to translate by larger amounts than the rest of the vehicle. cˆ 1 more clearly represents the actual fault. zˆ 3 would be difficult to interpret, appearing that the fault causes the right and left sides of the vehicle to move in opposite directions. ˆ ˆ [ Λˆ ˆ − σˆ 2I]1/ 2 Q, ˆ in this example can be written in the form Z We point out that C p p where Q is a 4×4 orthogonal matrix. This, in fact, must be the case, because the
2151_book.fm Page 193 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
193
1.0 mm 1.0 mm
FIGURE 8.6 Graphical illustration of zˆ 3 in the example.
ˆ that preserves the latent covariance structure. method of this chapter provides a C ˆ . Thus, the ˆ ˆ ˆ T is always equal to ∑ Specifically, because of the way C is formed, CC C method of this chapter represents a form of factor rotation, where the criterion for ˆ are the actual fault geometry vectors. “best” interpretability is that the columns of C As in PCA, a breakdown of the percentage of total sample variation that is due to each fault can be easily obtained. The estimated variation due to the ith fault is ˆ , or, equivalently, the sum of its eigenvalues. cˆ Ti cˆ i . The total variation is the trace of ∑ y For the preceding example, the total variation is 5.18 mm2. The percentages due to fault 1 through fault 4 are 15.5%, 10.3%, 10.4%, and 5.6%, respectively. Together, the four faults account for 41.8% of the total sample variability.
8.6 STATISTICAL PROPERTIES The statistical properties of the estimated fault geometry vectors are complicated, in general, and depend on a number of factors that include n, N, p, σ2, and {ci }ip=1. The fact that the estimates depend on which measurements are selected as the subgroups, which involves user subjectivity, further complicates the analysis. Some results can be obtained, however, in special situations. It is assumed that the data are normally distributed. A case of particular interest is where there is only one fault present. Suppose that this is the case and that the number of faults has been correctly estimated. Then ˆ has rank 1, and ∑ C cˆ 1 = λˆ 1 − σˆ 2 zˆ 1.
2151_book.fm Page 194 Friday, October 20, 2006 5:04 PM
194
Stream of Variation Modeling and Analysis for MMPs
Although the distribution of eigenvalues and eigenvectors of a sample covariance matrix is, in general, complicated, Anderson [5] provides some asymptotic (for large N) results when the data are normally distributed. Using these results, the following asymptotic expression for the accuracy of cˆ 1 is derived in Appendix B of Reference 1. cˆ − c 1 1 κ≡E c 2 1
2
(
1+ γ ≅ 1 n − 1 γ 1 + γ + N 2
(
) (
)
)
2
+
γ2 2 n −1
(
)
(8.9)
where γ ≡ σ 2 /c1T c1. γ can be viewed as the inverse of a “signal-to-noise ratio,” because c1T c1 is the total variance due to the fault. For large n, the third term can be neglected, and Equation 8.9 reduces to
κ≅
1+ γ 1− γ nγ + N 2
(8.10)
Clearly, increasing sample size N reduces κ and improves accuracy. It can be verified that the partial derivative of Equation 8.10 with respect to γ is always positive. Consequently, holding n and N fixed and increasing the signal-to-noise ratio (decreasing γ) always reduces κ, which is intuitively obvious. The effects of n are less obvious. Equation 8.10 seems to imply that increasing n increases κ and decreases accuracy. γ, however, may also change with n, because c1 will increase in dimension and c1T c1 may change. As n increases, if the additional measurements are not affected by the fault, then γ remains constant, and κ does increase. Thus, adding irrelevant measurements will decrease accuracy. On the other hand, suppose the additional measurements are affected by the fault. In this case, it is convenient to define γ ≡ σ 2 / n −1c1T c1 = nγ , which can be viewed as the average variance due to the noise, divided by the average variance due to the fault. As n increases, suppose the additional measurements are affected by the fault in such a way that γ remains constant. If we substitute γ = γ /n into Equation 8.10, it can also be verified that the partial derivative of κ with respect to n (holding γ and N fixed) is always negative for n > 1. Consequently, adding measurements that are affected by the fault (to the point that γ does not decrease) will reduce κ and improve accuracy. The general case where p > 1 is more difficult to analyze. Suppose the MDL method, which is known to yield a consistent estimate of p, is used, so that asymptotically
(
)
∑ ( λˆ − σ )zˆ zˆ p
ˆ =Z ˆ Λˆ − σ 2I Z ˆT ∑ C p p p=
2
i
T i i
i =1
For the case that λ1 > λ2 > …> λp, the results of Anderson [5] apply directly, yielding for i = 1, 2, …, p,
2151_book.fm Page 195 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
195
2λ 2 λˆ i ~ N λ i , i N 2σ 4 σˆ 2 ~ N σ 2 , (n − p) N and λ zˆ i ~ N zi , i N
n
∑ k =1 k ≠i
λk T zk zk (λ k − λ i ) 2
ˆ . Because {λˆ i , zˆ i }ip=1 and σˆ 2 are clearly consistent estimates. Therefore, so is ∑ C p ˆ {cˆ i }i=1 are calculated entirely from ∑ C , they also are consistent estimates, providing the measurement subgroups have been correctly identified. For the case where λj = λj+1 for some 1 ≤ j < p, Anderson's results do not apply. In fact, zj and zj+1 are not even uniquely defined when their eigenvalue has multiplicity 2, although the projection operator onto their eigenspace is. Consider the extreme case where λ1 = λ2 = . . . = λp ≡ λ. Note that Z p Z Tp , the projection operator onto the eigenspace of λ, is well defined. Tyler [14] showed that
{
1 Z p Z Tp B I − Z p Z Tp + I − Z p Z Tp BZ p Z Tp N Zˆ p Zˆ Tp − Z p Z Tp → λ − σ2
}
where the convergence is in distribution as N → ∞. Here, B is an n × n multivariate normal matrix with zero mean and covariance given by cov([B]i,j,[B]k,l) = [∑y]i,k , [∑y]j,l + [∑y]i,l , [∑y]j,k. Thus, Zˆ p Zˆ Tp is a consistent estimate of Z p Z Tp . Because Λˆ p ˆ is also consisand σˆ 2 are consistent estimates of Λ p = λI and σ 2 , respectively, ∑ C tent. Similar arguments can be applied when some, but not all, of {λ i }ip=1 are equal.
8.7 SUMMARY The applicability of the method depends predominantly on whether the linear (or linearized) model of Equation 8.1, with the assumed structure of Equation 8.2 for C, adequately represents the effects of process faults. Ultimately, this depends on the underlying physics of the process and the faults, although we believe the model provides a reasonable representation of manufacturing processes. Additionally, the method requires some degree of subjective judgment by the user in identifying the measurement subgroups. The development of more “black-box” methods that remove the subjectivity and relax the model assumptions would be a valuable contribution.
2151_book.fm Page 196 Friday, October 20, 2006 5:04 PM
196
Stream of Variation Modeling and Analysis for MMPs
8.8 EXERCISES 1. Assume eight quality variables are measured from an assembly process. The sample covariance matrix obtained from data of 150 observations is as follows. Determine the number of process faults with scree plot. 0.1553 0.0068 0.1535 0.0024 S= 0.1549 0.0106 0.15377 0.0184
0.0068 0.1535 0.0024 0.1549 0.0106 0.1537 7 0.0184 0.0183 0.0009 0.0011 0.0056 0.02988 0.0024 0.0534 0.0009 0.1541 0.0020 0.15388 0.0008 0.1536 0.0009 0.0011 0.0020 0.00033 0.0023 0.0018 0.0021 0.0033 0.0056 0.1538 8 0.0023 0.1551 0.0087 0.1538 0.0149 0.02988 0.0008 0.0018 0.0087 0.0494 0.0034 0.0880 0.0024 0.1536 0.0021 0.1538 0.0034 0.1536 0.00544 0.0534 0.0009 0.0033 0.0149 0.0880 0.0054 4 0.1576
2. For the same sample covariance matrix given in (1) determine the number of process faults with the likelihood ratio test method. 3. For the same sample covariance matrix given in (1) determine the number of process faults with the AIC and MDL criteria. 4. Derive Equation 8.7 based on the properties of eigen-decomposition. 5. For the same diagnosis problem described in Chapter 5, Problem 4: a. Derive the data covariance S or correlation R matrix. b. Derive the eigenvalue/eigenvector pairs from the matrix you selected in (a). c. Determine the number of common factors (or number of process faults). d. Estimate the fault geometry vectors for each of the process faults with the methodology introduced in this chapter. e. Compare the results from (d) with that from Problem 4 of Chapter 5.
8.9 APPENDIX 8.1: DISCUSSION OF THE DISTRIBUTIONS OF TEST STATISTICS In this appendix, it is proved that the distributions of Λ(m), Λmod(m), AIC(m) and MDL(m) depend only on n, N, and {λ i , σ 2}ip=1 , and not on the individual eigenvectors. Assume y is Gaussian. Consider the eigenvector decomposition of ∑ y in Equation 8.4. Expanding y in terms of its principal components, it follows that y is distributed as Ze, where e follows a multivariate normal distribution with covariance matrix , and Z = [z1, z2, …, zn]. Then, ˆ ≡ ∑ y
1 N −1
N
∑ i =1
1 ( yi − y )( yi − y )T = Z N − 1
N
∑ (e − e)(e − e)) Z T
i
i =1
i
T
2151_book.fm Page 197 Friday, October 20, 2006 5:04 PM
A Factor Analysis Method for Variability Modeling
197
where y and e are the sample averages for y and e, respectively. Because Z is an ˆ are the eigenvalues of the matrix in brackorthogonal matrix, the eigenvalues of ∑ y ets, the distribution of which depends only on n, N, and {λ i }in=1, and not on Z. Now suppose that y ′ ≡ a1/ 2 y for some positive constant a, so that the covariance ˆ are distributed as the eigenvalues of matrix of y ′ is ZaΛa T . The eigenvalues of ∑ y ˆ ∑ y multiplied by the constant a. By inspection of each of the four test statistics, it is clear that the constant drops out of the distribution. Consequently, if all eigenvalues of ∑ y are divided by a constant, say σ2, the distribution of each of the test statistics ˆ depends remains unchanged. Therefore, the distribution of the eigenvalues of ∑ y 2 n 2 only on n, N, and {λ i , σ }i =1. Noting that λ i , σ = 1, for i = p + 1, p + 2, …, n, completes the proof.
References 1. Apley, D. and Shi, J., A factor-analysis method for diagnosing variability in multivariate manufacturing processes, Technometrics, 43, 84, 2001. 2. Johnson, R.A. and Wichern, D.W., Applied Multivariate Statistical Analysis, 4th ed., Prentice Hall, Upper Saddle River, NJ, 1998. 3. Apley, D.W., and Shi, J., Diagnosis of multiple fixture faults in panel assembly, ASME Journal of Manufacturing Science and Engineering, 120, 793, 1998. 4. Jackson, J.E., Principal components and factor analysis: part II — additional topics related to principal components, Journal of Quality Technology, 13, 46, 1981. 5. Anderson, T.W., Asymptotic theory for principal components analysis, The Annals of Mathematical Statistics, 34, 122, 1963. 6. Lawley, D.N., Tests of significance for the latent roots of covariance and correlation matrices, Biometrika, 43, 128, 1956. 7. Akaike, H., Autoregressive model fitting for control, Annals of the Institute of Statistical Mathematics, 23, 163, 1971. 8. Schwartz, G., Estimating the dimensions of a model, The Annals of Statistics, 6, 461, 1978. 9. Rissanen, J., Modeling by shortest description length, Automatica, 14, 465, 1978. 10. Akaike, H., Factor analysis and AIC, Psychometrika, 52, 317, 1987. 11. Wax, M. and Kailath, T., Detection of signals by information theoretic criteria, IEEE Transactions on Acoustics, Speech, and Signal Processing, 33, 387, 1985. 12. Basilevsky, A., Statistical Factor Analysis and Related Methods, John Wiley & Sons, New York, 1994. 13. Everitt, B.S., Cluster Analysis, 3rd ed., Edward Arnold, London, 1993. 14. Tyler, D.E., Asymptotic inference for eigenvectors, The Annals of Statistics, 9, 725, 1981.
2151_book.fm Page 198 Friday, October 20, 2006 5:04 PM
2151_book.fm Page 199 Friday, October 20, 2006 5:04 PM
Part III Variation Source Diagnosis
2151_book.fm Page 200 Friday, October 20, 2006 5:04 PM
2151_book.fm Page 201 Friday, October 20, 2006 5:04 PM
9
Diagnosability Analysis for Variation Source Identification*
9.1 MOTIVATION AND FORMULATION OF DIAGNOSABILITY STUDY The state space model developed in Chapter 6 and Chapter 7 links process variation sources with product quality. Based on this quantitative model, variation sources in the process can be identified. However, before discussing variation source diagnosis, the diagnosability aspect needs to be thoroughly examined. The issue of diagnosability refers to the problem of whether product measurements contain sufficient information for the diagnosis of critical process faults, i.e., if root causes of process faults can be diagnosed. This problem has significant engineering relevance. Let us begin by revisiting the simple machining process shown in Figure 7.4, in which the perpendicularity problem of the hole is caused by the milling operation. The same process is shown again in Figure 9.1; here the milling operation is fine but the fixture in (b) has an error. Clearly, the fixture error will cause the same perpendicularity problem in the hole as shown in Figure 7.4. This implies that we cannot distinguish the datum error in Figure 7.4 and the fixture error in Figure 9.1 if only the hole orientation is measured. The diagnosability issue is particularly relevant for a multistage manufacturing system. First, it is challenging to evaluate diagnosability in a multistage system. As in Figure 9.1, the quality characteristic of the hole at the second station can be affected by either the locators on the second station or the machining errors on the first station. It is not obvious what kind of information can be obtained when the dimensional quality of the workpiece is measured. Overall, are all process faults diagnosable, given current measurements of product features? If not, what is the “aliasing” structure among the coupled process faults? Second, even if it is technically feasible, it is not cost-effective to install sensors or probes at every intermediate manufacturing stage. Therefore, the quantitative performance evaluation of a gauging strategy is very important. There is a great need for a systematic methodology that captures this phenomenon. The diagnosability analysis is based on a fault-quality diagnostic model that links process faults and product quality measurements. As presented in Chapter 6 to Chapter 8, this fault-quality model can be obtained through engineering analysis of the process- or data-driven techniques. In general, the fault-quality model for multistage processes can be expressed in a state space model form as * Part of chapter material is based on Reference 9 (pp. 312–325).
201
2151_book.fm Page 202 Friday, October 20, 2006 5:04 PM
202
Stream of Variation Modeling and Analysis for MMPs
Fixture Error D D
D
C C
C (a)
(b)
(c)
FIGURE 9.1 A machining process with fixture error.
x k = A k −1x k −1 + B k u k + w k
and
yk = Ck xk + vk
(9.1)
where k = 1, 2, …, N; A k −1x k −1 represents the transformation of quality information from stage k – 1 to stage k; B k u k represents the product quality as it is affected by the process faults at stage k; and Ck is the observation matrix that maps process states to measurements. System matrices Ak, Bk, and Ck are constant matrices. They are determined by the process or product design information. The state space model can be transformed into a general mixed linear model as follows. First, it can be written in an input-output format as: y k = Σ ik=1C k Φk ,i Bi u i + C k Φk ,0 x 0 + Σ ik=1C k Φk ,i w i + v k
(9.2)
where Φk ,i = A k −1A k − 2 Ai for k > i and Φk ,k = Inx , where nx is the dimension of xk. The quality characteristics x 0 correspond to the initial condition of the product before it goes into the manufacturing line. If the measurement of x0 is available, C k Φk ,0 x 0 can be moved to the left side of Equation 9.2 and the difference y k − C k Φk ,0 x 0 can then be treated as a new measurement. If the measurement of x0 is not available, we can treat it as an additional process fault input. Without loss of generality, we set x0 to 0. Define µ k as the expectation of uk and u k = u k − µ k . Combining all available measurements from station 1 to station N, we have y1 µ1 µ 1 w1 v1 y2 = G µ 2 + Γ µ 2 + Ψ w2 + v2 w N v N y N µ N µ N where C1B1 C 2Φ2,1B1 Γ= C Φ N N ,1B1
0 C2B2 C N ΦN ,2B2
0 0 , C N B N
(9.3)
2151_book.fm Page 203 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
C1 C 2Φ2,1 Ψ= C N ΦN ,1
0 C2 C N ΦN ,2
203
, C N 0 0
µ k is an unknown constant vector, and u k , w k , v k are zero mean random vectors. As yk is not necessarily available at every stage, if no measurement is available at station k, the corresponding rows should be eliminated. For the sake of simplicity, we can define y as [y1T y2T … yNT ]T and µ, u ,v, and w are defined in a similar way. Then Equation 9.3 can be rewritten as: y = Γ ⋅ µ + Γ ⋅ u + Ψ ⋅ w + v
(9.4)
Let P denote the total number of potential faults (i.e., the dimension of µ) and Q denote the number of system noises (i.e., the dimension of w) considered at all the stages. We use the lowercase ui, i = 1, …, P, to represent the ith element of the vector of u and ui’s variance is denoted as σ 2ui . Similarly, we use the lowercase wi to represent the ith element of the vector of w and wi’s variance is denoted as σ 2wi . With this notation, the variance components of process faults and system noises, and the variance of measurement noises are represented by {σ 2ui }i=1...P , {σ 2wi }i=1...Q , and σ 2v , respectively. During production, multiple samples of the product are available at each stage. Assume we have n samples that can be stacked up as y a = (1n ⊗ Γ ) ⋅ µ + (In ⊗ Γ ) ⋅ u a + (In ⊗ Ψ ) ⋅ w a + v a
(9.5)
where y a is a stack of the n samples of y, i.e., ya = [y(1)T y(n)T]T, and y(i) is the ith sample of y. The vectors u a, wa, and va are defined in a similar manner as ya, ⊗ is the Kronecker matrix product [1], and 1 n is the summing vector whose n elements equal unity. The process faults manifest themselves as the mean deviation and variance of process variation sources. The diagnosability problem can then be restated: From n samples, can we identify the value of {µ i }i=1...P and {σ 2ui }i=1…P ?
9.2 DEFINITIONS OF DIAGNOSABILITY There is currently limited reported research on diagnosability of variation root causes in multistage manufacturing processes. Ding et at. [2] conducted a preliminary study. They proposed the concepts of within-station diagnosability and between-station diagnosability to distinguish the local process faults and propagated process faults. The diagnosability condition listed in their work is a special case of the diagnosability analysis presented here.
2151_book.fm Page 204 Friday, October 20, 2006 5:04 PM
204
Stream of Variation Modeling and Analysis for MMPs
Fault-Quality Model
General Mixed Linear Model
Measurement Error rr
y
~ u w
µ
Residual Error Fixed Effects
Mean of Faults
v
y=X +Zu+e
System Matrices
Design Matrices Fault Variations and Model Noise
Random Effects
FIGURE 9.2 Model comparison.
In this chapter, the diagnosability problem is investigated under the framework of variance components analysis (VCA). The model in Equation 9.5 fits a general mixed linear model given by Rao and Kleffe (1988) as: β + Zu + e y = Xβ
(9.6)
where y is the observation vector; X is a known constant matrix; β is a vector of unknown constants; Z is a known constant matrix; u is a vector of independent variables with zero mean and unknown variances; and e is a vector of independent variables with zero mean and unknown variance σ 2e . The unknown variances of u and e are called “variance components.” A mixed model is used to describe both fixed and random effects. This model is often applied to biological and agricultural data. In designed experiments, the matrices X and Z are determined by designers. They often contain only 0 or 1, depending upon whether the relevant effect contributes to the measurement. Given a mixed model, researchers are primarily interested in estimating the fixed effects and variance components. A large body of literature about VCA is available. Excellent overviews can be found in Rao and Kleffe [3] and Searle et al. [4]. We can establish a one-to-one corresponding relationship between terms in our fault-quality model Equation 9.4 and those in the mixed model Equation 9.6, as shown in Figure 9.2. In our fault diagnosis problem, however, the matrices X and Z are computed from system matrices Ak , Bk, and Ck, k = 1, …, N, which are determined by process design information and measurement deployment informaµ) of process faults, and the random tion. The fixed effects are the mean values (µ w, and v. Fault diagnosis is effects are the process faults and the process noises, u, thus equivalent to the problem of variance components estimation. The definition of diagnosability in this chapter follows the same concept of identifiability in VCA [3]. The term “diagnosability” is used because it is more relevant in the context of our engineering applications. Based on Equation 9.5, we have T
E (y a ) = Γ T ... Γ T ... Γ T • µ
(9.7)
2151_book.fm Page 205 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
Cov ( y a ) = F1σ u21 + ... + FP σ 2uP + FP +1σ w2 1 + ... + FP +Q σ 2wQ + FP +Q +1σ 2v
205
(9.8)
where E(⋅) represents the expectation, I ⊗ (Γ :i Γ :Ti ) Fi = n T In ⊗ (Ψ:(i − P )Ψ:(i − P ) )
when 1 ≤ i ≤ P when P < i ≤ P + Q
where the subscript “:i ” represents the ith column of the matrix, and FP+Q+1 is an identity matrix with the appropriate dimension. Define [σ u21 ... σ u2P σ 2w1 ... σ 2wQ σ 2v ]T in Equation 9.8 as θ, EU as the space containing all possible values of U, and ES as the space containing all possible values of θ (in the most general case, EU is ℜ P ×1 and ES is a (P + Q + 1) × 1 space spanned by nonnegative real numbers). Definition 9.1: In the model of Equation 9.5, a linear parametric function pTα, p ∈ℜ P ×1, α ∈EU is said to be diagnosable if, ∀ α1 , α 2 ∈EU , pT α 1 ≠ pT α 2 ⇒ E ( Y)
U = α1
≠ E ( Y)
U =α 2
(9.9)
A linear parametric function f Tθ, f ∈ℜ( P +Q +1)×1 , θ ∈E S is said to be diagnosable if, ∀ θ1 , θ 2 ∈E S , f T θ1 ≠ f T θ 2 ⇒ Cov( Y)
θ = θ1
≠ Cov( Y)
θ =θ 2
(9.10)
In model of Equation 9.5, we are only concerned with the mean and variance of process faults. Therefore, only the first- and second-order moments are considered in the definition. The intuition behind this definition is that if for any change in a linear combination of the mean or the variance of the process faults, we can see a corresponding change in the observation ya, then this fault combination is diagnosable. Clearly, this definition is independent of the estimation algorithms. This definition is also very flexible. By selecting different p and f, the diagnosability of different fault combinations can be evaluated. For example, by selecting p or f = [1 0 … 0]T, we can check if the mean or variance of the first fault is diagnosable. If it is diagnosable, we say the mean or variance of this fault can be uniquely identified or diagnosed.
9.3 CRITERION OF FAULT DIAGNOSABILITY From the preceding definition, the diagnosability of the system can be checked according to the system fault-quality model. Theorem 9.1 provides the fundamental results that can be used for checking.
2151_book.fm Page 206 Friday, October 20, 2006 5:04 PM
206
Stream of Variation Modeling and Analysis for MMPs
Λ Ψ]. In the Theorem 9.1: Define the range space of a matrix as R(⋅) and D = [Λ model of Equation 9.5, 1. pTα is diagnosable if and only if p ∈R(Γ T ). 2. fTθ is diagnosable if and only if f ∈R(H) , where H is symmetric and given as (D:T1 D:1 )2 (D T D )2 :i :1 H= T 2 (D:( P +Q ) D:1 ) T D:1 D:1
...
(D:T1 D:i )2
...
(D D:i )2
...
(D D:( P +Q ) )2
... ... ...
T :i
(D
T :( P + Q ) T :i
D:i )
D D:i
2
(D:T1 D:( P +Q ) )2 T :i
...
(D
T :( P + Q )
D:( P +Q ) )2
D:(TP +Q ) D:( P +Q )
...
D:Ti D:i T D:( P +Q ) D:( P +Q ) L (9.11) D:T1 D:1
T
where L is the dimension of y1T y T2 y TN in Equation 9.3, i.e., L = Σ Nk =1qk . Proof: To prove this theorem, we need the following results in Rao and Kleffe [3]. β + ε, where β represents the fixed effects and Consider a linear mixed model Y = Xβ ε is zero mean and Cov(εε) = θ1V1 + … + θrVr. Denote θ = [θ1 … θr]T as variance components. Rao and Kleffe [3] prove that pTβ is identifiable if and only if p ∈R(XT), fTθ is identifiable if and only if f ∈R(H) , H′ = (tr(ViVj)), 1 ≤ i ≤ r, 1 ≤ j ≤ r. Γ T … Γ T ]) in model Based on this result, pTα is diagnosable if and only if p ∈R([Γ T T T Γ … Γ ]) = R(Γ Γ ). Therefore, (i) holds. For of Equation 9.5. It is clear that R([Γ (ii), fTθ is diagnosable if and only if f ∈ R(H ′) , where H′ = (tr(FiFj)), 1 ≤ i ≤ P + Q + 1, 1 ≤ j ≤ P + Q + 1, and Fi and Fj are defined in Equation 9.8. It can be further shown that H ′ = nH . Because a constant coefficient does not affect the range space of a matrix, the result of (part 2 in Theorem 9.1) follows. Q.E.D. Theorem 9.1 gives us a powerful tool to test if some combinations of faults are diagnosable. From Theorem 9.1, it is clear that the means of all the faults are uniquely diagnosable if and only if Γ T is of full rank. The variances of all the faults are uniquely diagnosable if and only if H is of full rank. For the preceding criterion, the diagnosability of the variance of a process fault includes the effects of the modeling error w and the observation noise v. This means that even if a fault can be distinguished from other faults, it could still be nonuniquely diagnosable if it is mixed up with the modeling error or the observation noise. In some cases, if the modeling error and the observation noise can be assumed small or their variance can be estimated from the normal working condition of a manufacturing process, we can ignore their effects when exploring the diagnosability of process faults. The testing matrix is revised accordingly by reducing θ to include only [σ u21 σ u2P ] and reducing the H matrix in Theorem 9.1 to Hr , where Hr is a subblock of H, i.e.,
2151_book.fm Page 207 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
(Γ :T1 Γ :1 )2 M Hr = (Γ :Ti Γ :1 )2 M (Γ T Γ ) 2 :P :1
... M ... M ...
(Γ :T1 Γ :i )2 M T (Γ :i Γ :i )2 M T (Γ :P Γ :i )2
... M ... M ...
(Γ :T1 Γ :P )2 M (Γ :Ti Γ :P )2 . M (Γ :TP Γ :P )2
207
(9.12)
Under the condition that noises w and v are assumed negligible, the diagnosability matrix was defined in Ding, Shi, and Ceglarek [2] as π(Γ ), where π(⋅) is a matrix transformation defined in their paper. The variances of process faults are considered fully diagnosable if and only if π(Γ )T π(Γ ) is of full rank. In fact, this Γ)T condition is the same as what we derived here. It can be shown that R(Hr) = R(π(Γ Γ)). Therefore, their work can be considered as a special case of the general π(Γ framework presented in this chapter. If noise terms are not included, Ding et al. [2] showed that the mean being diagnosable is a sufficient condition for variance being diagnosable. However, the converse is not true. This is illustrated in the case study of a machining process in Section 9.6.
9.4 MINIMAL DIAGNOSABLE CLASS Theorem 9.1 alone is not very effective in analyzing a partially diagnosable system in which not all faults are diagnosable. It is not obvious from Theorem 9.1 which faults are mixed or coupled together. To analyze a partially diagnosable system, we propose the concept of a minimal diagnosable class. We will first introduce the concept of the diagnosable class and then present the definition of the minimal diagnosable class. Definition 9.2: A nonempty set of n faults {ui1 … uin} forms a mean or variance diagnosable class if a nontrivial linear combination of their means {ui1 … uin} or variances {σ i21 ... σ i2n } is diagnosable. “Nontrivial” means at least one coefficient of the linear combination is nonzero. Definition 9.3: A nonempty set of n faults {ui1 … uin} forms a minimal mean or variance diagnosable class if no strict subset of {ui1 … uin} is mean or variance diagnosable. The diagnosability of the mean and variance can be dealt with separately, and the testing procedures are very similar (the only difference is the testing matrix; it is ΓT for mean and H for variance). Hence, no distinction between mean or variance diagnosability will be made hereafter unless otherwise indicated. The minimal diagnosable classes expose the interrelationship between different faults. Intuitively, a minimal diagnosable class is a group of faults that cannot be further distinguished. Minimal diagnosable classes expose the “aliasing” structure of the nondiagnosable faults in a partially diagnosable system. A minimal diagnosable class represents a set of faults that are closely coupled together. We can only identify a linear combination of them, but we cannot identify any strict subset. With
2151_book.fm Page 208 Friday, October 20, 2006 5:04 PM
208
Stream of Variation Modeling and Analysis for MMPs
this information, we can show the coupling relationship among faults and learn what additional information is needed to identify certain faults. We found that the minimal diagnosable class can be generated from the reduced row echelon form (RREF) (see Chapter 2) of the transpose of testing matrices. This result is stated in the following theorem: Theorem 9.2: Given a testing matrix G ∈ℜ n ×m (G is ΓT or H) and n faults θ = [u1 … un]T corresponding to G, the fault set θ[v] is a minimal diagnosable class if v is a nonzero row of the RREF of GT, where θ[v] is a subset of θ such that θ(i) θ[v] if v(i) (the ith element of v) ≠ 0. (the ith element of θ) ∈θ Proof: Denote the row and column space of a matrix as Row(⋅) and Col(⋅), respectively, the RREF of GT as GTr , and the nonzero row vectors of GTr as {vi}i=1…ρ, where ρ is the rank of GTr . Noting that GTr is unique and Row( GTr ) = Row(GT), we have v i ∈Col(G). Hence, θ[vi] is a diagnosable class. We need to further prove that θ[vi] is a minimal diagnosable class. From the algorithm to obtain RREF, the leftmost element of vi is always a “leading 1.” The position of such a “leading 1” in vi is called the pivot position. Denote the set of all pivot positions contributed by the rows of GTr as Ξ. It is known that: (i) given an i∈{1, …, ρ}, there is only one nonzero element in {v i ( j)} j∈ΞΞ; (ii) if {ci}i=1…n are columns of GTr , then there is only one nonzero element in ci if i ∈Ξ Ξ. From (i), θ[vi] Ξ, i1…ik ∈Ξ Ξ. Assuming that θ[vi] is not must be of the form {upi, ui1, …, uik}, pi ∈Ξ a minimal diagnosable class, we can then find a vector v i′ such that θ[vi′] ⊂ θ[vi], v i′ ∈ Row(GTr ) , and v i′ can thus be written as ρ
v i′ =
∑a v . j
j
j =1
However, from (ii), if there is a j such that aj is nonzero, u p j must be in θ[v′i ], where pj is the pivot position of vj. Because θ[vi] only contains one fault at the pivot position pi, ai is the only possible nonzero coefficient. Then θ[v′i ] = θ[v i ]. This contradicts the assumption that θ[v′i ] ⊂ θ[v i ], implying that θ[vi] is a minimal diagnosable class. Q.E.D. When the RREF of GT is calculated, we can obtain some of the minimal diagnosable classes. The following corollary shows that by rearranging the columns in GT, we can obtain all the possible minimal diagnosable classes. The rearranging process is known as matrix permutation. The permuted matrix is defined as: if {ci}i=1…n, denote the column vectors of GT and correspond to the faults θ = [u1 … un]T, the columnwise permuted matrix G′ T = [c i1 … c in ] is called the permuted matrix corresponding to the fault permutation θ′ = [ui1 … uin ]T . Corollary 9.1: Given a testing matrix G ∈ ℜ n× m and if Θ′ = {ui1 , …, uis is a minimal diagnosable class, then θ[v] = Θ, where v is the last nonzero row of G′r T . G′r T is the RREF of the permuted matrix of GT corresponding to the fault permutation uis +1 ... uin ui1 ... uis . Proof: Denote {vi}i=1…ρ as the nonzero row vectors of G′r T . We want to prove that the pivot position of the last row vρ must be n – s + 1 (this position corresponds to ui1).
2151_book.fm Page 209 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
209
First, suppose that the pivot position of vρ is larger than n – s + 1. If so, θ[v ρ ] ⊂ Θ. According to Theorem 9.2, however, θ[vρ] is a diagnosable class. This contradicts the fact that Θ is minimal. Second, assume that the pivot position of vρ is smaller than n – s + 1. If so, a fault among {uis+1, …, uin} must belong to θ[vρ]. Because the pivot position of vρ is the largest among all the pivot positions of {vi}i=1…ρ, given any vector v f = Σ ρj =1a j v j (defined as an arbitrary nontrivial linear combination of {vi}i=1…ρ), θ[vf ] contains at least one element among {uis+1, …, uin}. According to Theorem 9.1, any diagnosable class should contain at least one element among {uis+1, …, uin} because vf is an arbitrary vector in Row( G′r T ). This contradicts the assertion that Θ = {ui1, …, uis} is a minimal diagnosable class. Therefore, the pivot position of vρ is at n – s + 1, i.e., θ[vρ] ⊆ Θ. Because θ[vρ] and Θ are both minimal, θ[vρ] = Θ. Q.E.D. Corollary 9.1 indicates that a complete list of minimal diagnosable classes can be obtained by thoroughly permuting GT. However, the number of permutations will rapidly become very large if the number of faults is large. To handle this problem, we need the concept of the “connected fault class.” Given the RREF of GT, assume we can divide its nonzero rows into two sets of rows (C1 and C2) such that for any v i ∈C1 and v j ∈C2, v i * v j = 0 , where * is the Hadamard product [6]. In other words, v i does not share any common nonzero column positions with v j . Define symbol θ[C] as the fault set of ∪(θ[v k ]) for all k v k ∈C, where C is a set of rows. We can show that for an arbitrary minimal diagnosable class θ[v], either θ[v] ⊆ θ[C1 ] or θ[v] ⊆ θ[C2 ]. From Theorem 9.1, v is in the space spanned by the rows of GT. Thus, v = a1v1 + a2 v 2 , where v1 and v2 are in the space spanned by the rows in C1 and C2, respectively. However, if a1 and a2 are both nonzero, the fact that v i * v j = 0 and θ[v1], θ[v2] are both diagnosable will lead to the contradiction that θ[v] is not minimal. The implication is that the complete list of minimal diagnosable classes can be obtained by only permuting the faults within θ[C1] and θ[C2], respectively. Following the same rule, C1 and C2 can be further divided into smaller groups iteratively until they are no longer dividable. If Ci is an undividable set of rows, θ[C1] is called a connected fault class. Following a similar argument, we know that the complete list of minimal fault classes can be obtained through permutations only within each connected fault class. If there are many small connected fault classes in the system, the computational load required to find all minimal diagnosable classes can be significantly reduced. The worst case is that all faults are connected in a big fault class. However, that is usually not a common situation. For instance, one principle in manufacturing process design is to reduce the accumulation and propagation chain of process faults [5]. For many actual engineering systems, the entire fault set can often be partitioned into much smaller connected fault subsets, as we will see in the case studies of Section 9.6. In summary, the algorithm obtaining all the minimal diagnosable classes is as follows: 1. Calculate the RREF of GT. 2. Remove all the uniquely identifiable faults because each of them will form a minimal diagnosable class; remove the faults corresponding to zero
2151_book.fm Page 210 Friday, October 20, 2006 5:04 PM
210
Stream of Variation Modeling and Analysis for MMPs
1
2
4 3
5
FIGURE 9.3 Illustration of minimal diagnosable class.
columns because they are invisible to the measurement system and hence not diagnosable, and will not appear in any minimal diagnosable classes. 3. Find the connected fault classes based on the RREF of GT. 4. Permute the columns within the connected fault classes and obtain the minimal diagnosable classes based on the permuted matrices until all possible permutations are visited. The minimal diagnosable classes expose the “aliasing” structure among the faults in the system, revealing critical fault diagnosability information. For example, if a single fault forms a minimal diagnosable class, it is uniquely diagnosable. If a fault is not uniquely diagnosable and forms a minimal diagnosable class with several other faults, it can be identified when all other faults are known. Thus, by looking at the minimal diagnosable classes, we can identify which fault can be identified from the measurements, and if not, what other faults need to be known to identify it. This is illustrated in Figure 9.3, where there are five faults in the system. Fault 3 is uniquely diagnosable, whereas faults 1 and 2, and faults 4 and 5 are coupled together.
9.5 GAUGING SYSTEM EVALUATION BASED ON MINIMAL DIAGNOSABLE CLASS To evaluate a gauging system, we may need several easy-to-interpret indices to characterize the information obtained through the system. We propose three criteria for evaluation of gauging systems: information quantity, information quality, and system flexibility.
9.5.1 INFORMATION QUANTITY Information quantity refers to the level of knowledge regarding process faults we have obtained from the measurement data. When two gauging systems are used for the same manufacturing system, the number of potential faults is the same. However, for two different partially diagnosable systems, the number of faults that we need to know to ensure full diagnosability will often be different. This number can be used to quantify the amount of information obtained by different gauging systems. The following corollary indicates that the rank of the diagnosability testing matrix should be used to quantify the amount of measurement information. Corollary 9.2: Given a testing matrix G ∈ℜ n ×m and n faults θ = [u1 … un]T corresponding to G, if the rank of G is ρ, then n – ρ faults need to be known to uniquely identify all n faults.
2151_book.fm Page 211 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
211
The proof is omitted here. It uses the property of the RREF of a matrix. An intuitive understanding of this corollary is given as follows. The solvability condition of a linear system Y = AX can be determined by analyzing the RREF of A. In such a linear system, n – ρ free variables need to be known before uniquely solving X, where n is the dimension of X and ρ = rank (A). If we consider the testing matrix G to be in a similar situation as matrix A, the result of Corollary 9.2 is not surprising.
9.5.2 INFORMATION QUALITY The second criterion is information quality. Even if two gauging systems provide the same amount of information as per the criterion developed earlier, the detailed information content could be very different. In practice, it is always desirable to have unique identification of a fault so that corrective action can immediately be undertaken to eliminate the fault and restore the system to its normal condition. The decision of corrective action cannot be made for a fault coupled with others without further investigation or measurement. Thus, we use the number of uniquely identifiable faults to benchmark the quality of measurement information. Uniquely identifiable faults can be easily found by counting the number of minimal diagnosable classes that contain only one single fault.
9.5.3 SYSTEM FLEXIBILITY The third criterion is the flexibility provided by the current gauging system toward achieving full diagnosability. Some gauging systems could be rigid because certain faults or fault combinations, which may be difficult to measure in practice, have to be known to achieve a fully diagnosable system. Other gauging systems may provide information in a flexible manner, i.e., many fault combinations can be selected to make the system fully diagnosable. This comparison needs the concept of minimal complementary classes. A minimal complementary class is a minimal set of faults such that if they are known, all the faults of the system can be uniquely identified. Consider a system with four faults and three minimal diagnosable classes as {u1, u2}, {u1, u3, u4}, and {u2, u3, u4}. One can verify that the minimal complementary classes for this system are {u1, u3}, {u1, u4}, {u2, u3}, {u2, u4}, and {u3, u4}. The number of minimal complementary classes is five. A system with more minimal complementary classes is considered to be more flexible. In general, it is difficult to find the complete sets of minimal complementary classes by simply trying out different fault combinations, especially for a complex system with a large number of faults and intricate fault combinations. Corollary 3 facilitates the determination of minimal complimentary classes. Corollary 9.3: A set of faults forms a minimal complementary class if and only if the set contains n – ρ faults but does not contain any minimal diagnosable class, where n is the total number of faults and ρ is the rank of the diagnosability testing matrix. Proof: From Corollary 9.2, it is clear that a minimal complementary class should contain exactly n – ρ faults. Assume that a minimal complementary class contains a minimal diagnosable class that includes n1 faults. Because a minimal diagnosable class is diagnosable, we only need to know n1 – 1 faults in the minimal diagnosable
2151_book.fm Page 212 Friday, October 20, 2006 5:04 PM
212
Stream of Variation Modeling and Analysis for MMPs
class to identify all the n1 faults. Then, the number of faults in the minimal complementary class can be reduced by 1. Thus, a fault class is a minimal complementary class only if it does not contain any minimal diagnosable class. Now we need to prove that if a fault class with n – ρ elements does not include any minimal diagnosable class, it is a minimal complementary class. Assume that a fault class {ui1 … uin–ρ} does not contain any minimal diagnosable class. Consider the RREF of the permuted matrix G ′ T corresponding to the fault permutation in–ρ+1 … in i1 … in–ρ. As ui1 … uin–ρ does not include any minimal diagnosable class, the last n – ρ columns of the RREF should not include any pivot positions, according to Corollary 9.1. However, because there are a total of ρ pivot positions, every column among the first ρ columns of the RREF should contain only a “leading 1.” Hence, it is clear that all the faults can be uniquely identified if the n – ρ faults that correspond to the last n – ρ columns are known. Q.E.D. With Corollary 9.3, all minimal complementary classes can be found through a search among all fault sets with n – ρ faults. If the entire fault set can be partitioned into many smaller distinct, connected fault classes, the computational load of searching the complete minimal complementary classes can be further substantially reduced. Corollary 9.3 can be applied to a connected fault class but n should be the total number of faults in the connected fault class and ρ should be the rank of the space spanned by the associated row vectors in the RREF of the transpose of the testing matrix. Individual searches can be conducted within each connected fault class. The complete set of minimal complementary classes can then be obtained by joining the minimal complementary classes from each connected fault class and adding the nondiagnosable faults. An example will be given in Section 9.6 to illustrate this procedure. The order of using the three criteria generally depends on the requirements of individual applications. In some cases, when the ultimate goal is to design a gauging system providing the full diagnosability, we can skip the second criterion and compare the number of minimal complementary classes directly. In other cases, the second criterion can be used before the first criterion if the uniquely identified fault is highly desired. Based on our experience, using the three criteria in the sequence in which they were presented here is an effective way to gauge system evaluation in many industrial applications.
9.6 CASE STUDY 9.6.1 CASE STUDY
OF
A MULTISTAGE ASSEMBLY PROCESS
Let us consider the assembly processes shown in Figure 9.4. In this process, three stations are involved in assembling four parts (labeled 1, 2, 3, and 4, respectively, in Figure 9.4) and inspecting the assembly: part 1 and part 2 are assembled at station I; subassembly “1 + 2” is assembled with part 3 and part 4 at station II; and the final assembly with four parts is inspected at station III for surface finish, joint quality, and dimensional defects. Each part is restrained by a set of fixtures consisting of a 4-way locator, which controls motion in both x- and z-directions, and a 2-way locator, which controls motion only in the z-direction. A subassembly with several parts also needs a 4-way locator and a 2-way locator to completely control its degrees of freedom. The active locating points are marked as Pi , i = 1, ..., 8, in Figure 9.4. In
2151_book.fm Page 213 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
1
P2 3
5
4
P4
500
P1 8 7
1
z
6
2
M55
P8
4
P4
P7 4
M4
P8 15
500
3
1
10
12 P 5 P6 3
9 13 14
x
(c) Station III
17 P11
P3
11
2
M33
950
700
16
(b) Station II
450
P 12 1
2
1100
600
M1
400
M22
(a) Station I
213
M5
18
P i –locating points Mi –measurement points -active 4-way locator -active 2-way locator -inactive 4-way locator -inactive2-way locator -measurement location
4-way locator, 2 potentntial faults 2-way locator, 1 potential fault
FIGURE 9.4 A multistage assembly process.
this process, five coordinate sensors are installed on all three stations. Each coordinate sensor measures the position of a part feature, such as a corner, in two orthogonal directions (x and z). The measurement points are marked as {Mi, i = 1, …, 5} in Figure 9.4. There is another measurement point on station III, which is marked as M 5′ . This point represents another gauging strategy: the point M 5′ is measured on station III instead of the point M 5 . Counting potential locator errors on all stations, we have a total of n = 18 potential faults, which are assigned serial numbers from 1 to 18, as shown in Figure 9.4. It is interesting to compare the diagnosability of these 18 faults under these two gauging strategies. The product state variable xk is denoted by random deviations associated with the degrees of freedom (d.o.f.’s) of each part. Each 2-D part in this example has three d.o.f.’s (two translational and one rotational) and the size of xk is 12 × 1, given that there are four parts. The state vector xk is expressed as x k = [δx1,k
δz1,k
δα1,k
δx 4,k
δz4,k
δα 4,k ]T
(9.13)
where δ is the deviation operator, δxi,k , δzi,k , and δα i ,k are the two translational and one rotational deviations of part i on station k, respectively. If part i has not yet appeared on station k, the corresponding δxi,k , δzi,k , and δα i ,k are zero. The input vector uk represents the random deviations associated with fixture locators on station k. There are a total of 18 components of fixture deviations on three stations, as indicated by the numbers 1 to 18 (i.e., the 18 faults) in Figure 9.4. Thus, we have u1 = [δp1 δp6]T, u2 = [δp7 δp15]T, u3 = [δp16 δp17 δp18]T, where δpi is the deviation associated with fault i. The measurement y contains positional deviations detected at Mi, i = 1,…, 5. In this 2-D case, each Mi can deviate in x- or z-directions. Hence: y1 = [δM1(x) δM1(z) δM2(x) δM2(z)]T, y2 = [δM3(x) δM3(z) δM4(x) δM4(z)]T, and y3 = [δM5(x) δM5(z)]T. The state space representation of this process is shown as follows: x1 = B1u1 + w1 and x k = A k −1x k −1 + B k u k + w k , k = 2, 3
(9.14)
y k = C k x k + v k , k = 1, 2, 3
(9.15)
2151_book.fm Page 214 Friday, October 20, 2006 5:04 PM
214
Stream of Variation Modeling and Analysis for MMPs
Matrices Ak , Bk , and Ck are determined by process design and sensor deployment. Ak characterizes the change in product state when a product is transferred from station k to station k + 1. Thus, Ak depends on the coordinates of fixture locators on two adjacent stations, k and k + 1. Bk determines how fixture deviations affect product deviations on station k and is thus determined by the coordinates of fixture locators on station k. Ck is determined by the coordinates of measurement points such as M1 to M5 in this example. Following the model development presented in Chapter 6, we give the numerical expressions of A’s, B’s, and C’s of the assembly processes shown in Figure 9.4, respectively. The A’s, B’s, C1, and C2 are the same for these two processes because their fixture layouts are the same for all stations and the sensor deployments are the same for station I and station II. In this particular example, since each locator pair used in each station have the same z coordinate, the deviation of a 2-way locator in x direction will not affect the part location because the deviation will be absorbed by the slot. Comparing to the models in Chapter 6, the third element in each us,k does not exist. This also reflects in i,k equaling 0, thus the third column in R3s ,k and R4k −1,k will degenerate to a zero column, in Equations 6.11 and 6.18, respectively. In this special condition, simply omitting the two-way locator deviation in x direction in models developed in Chapter 6 will give the B matrices in 9.17. 0 0 0 A1 = −1 0 0 0 0 0 −1 0 0 A2 = −1 0 0 −1 0 0
0 0 0.0007 0 −0.3497 0.0007
0 0 0.0005 0 −0.5550 0.0005 −0.2153 −0.2392 0.0005 0 −0.2392 0.0005
0 0 1 0 0 0
0 0 1 0 0 0 0 0 0 0 0 0
0 0 0 1 0 0 0 6× 6
0
3× 6
I 6× 6
0 3×6
0 0 0 0 −0.0007 −0.3497 0 0 0.3497 −325.17 −0.0007 0.6503
0 6× 6
I 6× 6 0 0 0 0 0 0 0 0 0 1 0 0
12×12
0 0 0 0 −0.0005 −0.2392 0 0 −0.4450 −222.49 −0.00005 −0.2392 0.2153 107.655 −0.7608 −380.38 −0.0005 −0.2392 0 −0.0005 0.2392 −380.38 −0.0005 0.7608 12×12
(9.16)
2151_book.fm Page 215 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
1 0 0 B1 = 0 0 0
0 1 −0.0014 0 0 0
0 0 0 0 0.0014 0 0 1 0 0 0 0 6× 6 0
1 0 0 1 0 0 B2 =
0 1 −0.0007 0 0.3497 −0.0007
0 0 0.0007 0 0.6503 0.0007
1 0 0 1 0 0 B3 = 1 0 0 1 0 0
0 6 ×3
0 0 0 0 1 −0.002
0 0 0 0 0 0.002 12×6
0 6×6
1 0 0 0 0 0
0 0 1 0 −0.0010 0.0010 0 0 0.5550 0.4450 −0.00005 0.0005 0.2153 −0.2153 0.2392 0.7608 −0.0005 0.0005 0 0 0.2392 0.7608 −0.0005 0.0005 12×3
0 1 −0.002 0 0 0
0 0 0.002 0 0 0
0 0 0 1 0 0
215
(9.17)
0 0 0 0 0 0 0 0 1 0 −0.002 0.002 12×9
2151_book.fm Page 216 Friday, October 20, 2006 5:04 PM
216
Stream of Variation Modeling and Analysis for MMPs
1 0 C1 =
−550 −100
0 1 0
1 0 C2 =
1 0
2× 3
−550 −100
0 1
−550 −630
0 1
0 2× 3 1 0
−300 −740
0 1
(9.18)
0 2× 3 0 2× 3 4×12
0 2× 3
0 2× 3
0 2× 3
0 2× 6 0 2× 6 4×12
0 2× 3
We use C13 and C 23 to denote C3 of these two gauging systems, respectively. Their expressions are: 1 C13 = 0
C32 = 0 2×99 0 2× 9 2×12
−550 −100
0 1
1 0
−200 (9.19) 620 2×12
0 1
For simplicity, we will only discuss the variance diagnosability of fixture faults in this study. Thus, we use Hr in Equation 9.12 as the testing matrix. To use Hr , we need to obtain Γ first. Substituting A’s, B’s, and C’s in Equation 9.16–Equation 9.19 into Equation 9.3 yields: 1 0 0 0 0 Γ1 = 0 0 0 0 0 and
0.786 −0.786 0
0
0
0
0
0
0
0
0
0 0
0
0
0
1.143 −0.143 0
0
0
0
0
0
0
0
0
0 0
0
0
0
0
0
1
−1.1 0
0
0
0
0
0
0 0
0
0
0
0
0
0 2.26 −1.26 0
0
0
0
0
0
0 0
0
0
0
1.1
0.401 −0.786 0
0
0.385 1 0.385 −0.385 0
0
0
0 0
0
0
0
0.073 −0.143 0
0
0.070 0 1.070 −0.070 0
0
0
0 0
0
0
0
0.6
−0.6
0
0
0
0
0
0
0
0
1
0 0
0
0
0
0
0
0
0
0
0
0
0
0 2.48 −1.48 0 0
0
0
0
0.401 −0.786 0
0
0.385 0 0.122 −0.385 0
0
0
0 0 0.263 1 0.263
0.073 −0.143 0
0
0.070 0 0.022 −0.070 0
0
0
0 0 0.048 0 1.048
0 0 0 0 0 0 −0.263 −0.048 0
0
(9.20)
2151_book.fm Page 217 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
1 0 0 0 0 Γ2 = 0 0 0 0 0
217
0.786 −0.786 0
0
0
0
0
0
0
0
0
0
0
0
0
0
1.143 −0.143 0
0
0
0
0
0
0
0
0
0
0
0
0
0
−1.1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1 1.1
0
0
0 2.26 −1.26 0
0
0
0
0
0
0
0
0
0.401 −0.786 0
0
0.385 1
0.385 −0.385 0
0
0
0
0
0
0
0
0.073 −0.143 0
0
0.070 0
1.070 −0.070 0
0
0
0
0
0
0
0
−0.6 0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0 2.48 −1.48 0
0
−1 −0.096
0
0
0
0
1
0
0
0
0
0
0 −0.24 0.183 0 −0.057
0.057
1 0.6
0 0.4
0 0 0 0 0 0 −0.096 1.057 (9.21) 0
−0.304 1 0.096
where the superscript 1 or 2 indicates which gauging strategy the Γ is associated with. Further, Hr can be obtained following its definition in Equation 9.12. Their expressions are: 1 0.617 0.617 0 0 0 0 0 0 H1r = 0 0 0 0 0 0 0 0 0
and
0.617 0.617 0
0
5.089 2.050 0
0 0
2.050 3.661 0
0
0
0
0
0
0
0
0 0
0.102 0.161 0.08 0.102
0
0
0
0 0
0.390 0.617 0.307 0.390
0
0
0
0 0
1.21 1.21
0
0
0
0
0
0
0 0
1.21 39.91 16.46
0
0
0
0
0
0
0 0
0.102 0.390 1.21 16.46 9.63 0.148 0.073 0.093
0
0
0
0 0
0.161 0.617 0
0
0.148 1.0 0.148 0.148
0
0
0
0 0
0.08 0.307 0
0
0.073 0.148 1.711 0.073
0
0
0
0 0
0.102 0.390 0
0
0.093 0.148 0.073 0.093
0
0
0
0 0
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0.36 0.36 0 0
0
0
0
0
0
0
0
0
0.36 42.39 16.24 0 0
0
0
0
0
0
0
0
0
0.36 16.24 6.505 0 0
0
0
0
0
0
0
0
0
0
0
0
0 0
0
0
0
0
0
0
0
0
0
0
0
0 0
0.012 0.046 0
0
0.011
0
0.001 0.011
0
0
0
0 0
0.161 0.617 0
0
0.148
0
0.015 0.148
0
0
0
0 0
0.033 0.127 0
0
0.030
0
0.003 0.0330
0
0
0
0 0
0.012 0.046 0
0
0.0011
0
0.001 0.011
0
0
0
0 0
0.012 0.161 0.033 0.012 0.046 0.617 0.127 0.046 0 0 0 0 0 0 0 0 0.011 0.148 0.030 0.011 0 0 0 0 0.001 0.015 0.003 0.001 0.011 0.148 0.03 0.0011 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.005 0.069 0.014 0.005 0.069 1 0.069 0.069 0.014 0.069 1.362 0.014 0.005 0.069 0.014 0.005 (9.22) 0
0
0
0
2151_book.fm Page 218 Friday, October 20, 2006 5:04 PM
218
Stream of Variation Modeling and Analysis for MMPs
1 0.617 0.617 0 0 0 0.617 4.367 1.224 0 0 0.025 0.617 1.224 1.627 0 0 0.098 0 0 0 1 1.21 1.21 0 0 0 1.21 39.91 16.46 0 0.025 0.098 1.21 16.46 8.705 0 0.161 0.617 0 0 0.148 0 0.054 0.207 0 0 0.050 0 0.025 0.098 0 0 0.023 Hr2 = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0
0
0
0
0
0
0
0
0
0
0
0.161 0.054 0.025
0
0
0
0
0
0
0
0
0.617 0.207 0.098
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.231 0.14 48
0
0
0
1
1
0.009
0.148 0.050 0.023 4.0
0.16 0.093
0.231 1.703 0.050
0
0
0
0.148
0.05
0.023
0
0
0
0.009 0.003 0.002 0.009 0.00002 0
0
0
0
0
0
0
0
1
0.36
0.36
0
0
0
0
0
0
0
0
0.36 42.39 16.24
0
0
0
0
0
0.36 16.24 6.505
0
0
0
0
0
1
0.009
0
0
0
1
0.009
0
0
0
0
0.16
0.003
0
0
0
0
0.16 0.047 0.027 0.16
0.093 0.002
0
0
0
0
0.093 0.027 0.016 0.093 0.002
1
1
1
0.16 0.093
0.16 0.093
1
0.003
0.009
0
0
0
0
0.009 0.0002
0
0
0
0
0.009 0.003 0.002 0.009 0.0002
0..009
0.009 0.005
0
0
0
0
0.009 0.085 0.049 0.009 0.005
0 0 0 0 0.009 0.005 0 . 0 0 0 0.009 0.085 0.049 0.009 0.005 1.271 (9.23) 0
0
The RREF of Hr’s and the corresponding fault structures are compared in Table 9.1. For gauging strategy 1, 14 rows have only one nonzero element, corresponding to 14 uniquely identified faults and, hence, minimal diagnosable classes, {1}, …, {12}, {16}, {17}. Two faults {13, 14} correspond to zero columns and are therefore not diagnosable. The 13th row has two nonzero elements, i.e., [01×12 | 0 0 1 0 0 1], indicating that {15, 18} is a minimal diagnosable class. The class {15, 18} is also a connected fault class and because it is already minimal, no further permutation is needed. Similarly, for gauging strategy 2, there are 13 uniquely diagnosable classes, {1}, …, {12}, {18}. Two minimal diagnosable classes, {13, 16} and {14, 15, 17}, correspond to the 13th and 14th row, respectively. No permutation of Hr is needed for gauging strategy 2, either. For gauging strategy 1, to achieve a fully diagnosable system, at least n – ρ = 3 faults need to be known. We first search fault set {15,18} with n = 2 and ρ = 1. It is clear that {15} and {18} are two minimal complementary fault classes for the connected fault class {15, 18}. Adding the nondiagnosable faults {13, 14}, we obtain the minimal complementary classes as {13, 14, 15} and {13, 14, 18}. The number of minimal complementary classes is two. For gauging strategy 2, to find the minimal complementary class, we search for the faults among {13, 16} with n = 2 and ρ = 1, and among {14, 15, 17} with n = 3 and ρ = 1. The search yields {13} and {16} for {13, 16} and {14, 15}, {14, 17}, and {15, 17} for {14, 15, 17}. Joining these two fault groups together gives us
2151_book.fm Page 219 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
219
TABLE 9.1 Comparison of Gauging Systems 1 and 2 Gauging Strategy 1 RREF(Hr)
I 0 6×12
12 ×12
Number of potential faults Rank of testing matrix Minimal diagnosable classes Number of uniquely identified faults Minimal complementary classes Number of minimal complementary classes
0
12 × 6
0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 18 ×18
Gauging Strategy 2
I 0 6×12
12 ×12
0 1 0
0
12 × 6
1
0
0 1 0.58 0 0.06 0 0
0
0
0
0 0
0
0
0
0 0
0
0
0
0 0
0
0
0
0 0 1 0 0 0 18 ×18
18 15 {1},…,{12}, {16}, {17}, {15, 18} 14
18 15 {1}, …, {12}, {18}, {13,16}, {14, 15, 17} 13
{13, 14, 15}, {13, 14, 18}
{13, 14, 15}, {13, 14, 17}, {13, 15, 17}, {16, 14, 15}, {16, 14, 17}, {16, 15, 17} 6
2
C21 ⋅ C31 = 6 minimal complementary classes, which are listed in Table 9.1. This analysis verifies that although engineering systems have many potential faults (18 faults in this case), they can often be partitioned into smaller connected fault classes. Neither gauging system provides full diagnosability because both their Hr’s are not of full rank. Ranks of Hr’s are the same (ρ = 15), suggesting that the amount of information obtained by both systems is the same. But gauging strategy 1 can uniquely identify 14 faults, which are faults 1–12, 16, and 17, whereas gauging strategy 2 can only uniquely identify 13 faults, which are faults 1–12 and 18. The information quality provided by gauging strategy 1 is considered better than that of gauging strategy 2. In this sense, gauging strategy 1 provides more valuable information. However, one may also notice that gauging strategy 2 can have six possible ways of measuring additional faults in achieving a fully diagnosable system, but gauging strategy 1 only has two possibilities. This difference indicates that gauging strategy 2 is more flexible. If the third criterion is of higher priority, gauging strategy 2 is more favorable.
9.6.2 CASE STUDY
OF
A MULTISTAGE MACHINING PROCESS
The proposed evaluation criteria can also be applied to multistage machining processes. To machine a workpiece, we need to first fix the location of the workpiece in the space. Figure 9.5 shows a widely used 3-2-1 fixturing setup. If we require the
2151_book.fm Page 220 Friday, October 20, 2006 5:04 PM
220
Stream of Variation Modeling and Analysis for MMPs
CuttingTool G
H
F
P1 P2
Z
Workpiece
P3
Fixture System E
A L2
Y
D
C L1
Machine Table
L3 B
X
FIGURE 9.5 A typical 3-2-1 fixturing configuration.
workpiece to touch all the locating pads (L1~L3) and locating pins (P1~P3), the location of the workpiece in the machine coordinate system XYZ is fixed. The surface of the workpiece that touches the locating pads (L1~L3) (surface ABCD in Figure 9.5) is called the “primary datum.” Similarly, surface ADHE is called the “secondary datum” and DCGH is called the “tertiary datum” in Figure 9.5. Because the primary datum (surface ABCD) touches L1–L3, the translational motion in the z direction and the rotational motion in x and y directions are restrained. Similarly, the secondary datum constrains the translational motion in the x direction and the rotational motion in the z direction; the tertiary datum constrains the translational motion in the y direction. Therefore, all six d.o.f.’s associated with the workpiece are constrained by these three datum surfaces and the corresponding locating pins and pads. The cutting toolpath is calibrated w.r.t. the machine coordinate system XYZ. Clearly, an error in the position of locating pads and pins will cause a geometric error in the machined feature. Suppose that we mill a slot on surface EFGH in Figure 9.5. If L1 is higher than its nominal position, the workpiece will be tilted w.r.t. XYZ. However, the cutting tool path is still determined w.r.t. XYZ. Hence, the bottom surface of the finished slot will not be parallel to the primary datum (ABCD). Besides the fixture error, the geometric errors in the datum feature will also affect the workpiece quality. For example, if the primary datum (ABCD) is not perpendicular to the secondary datum (ADHE), the milled slot will not be perpendicular to the secondary datum, either. A three-stage machining process using this 3-2-1 fixture setup is shown in Figure 9.6. The product is an automotive engine head. The features are the cover face (M), joint face, and the slot (S). The cover face, joint face, and the slot are milled at the first (Figure 9.6a), second (Figure 9.6b), and third (Figure 9.6c) stages, respectively. We treat the positional errors of product features after stage k as state vector xk , the errors of fixture and the cutting tool path at stage k as input uk, and the measurements of positions and orientations of the machined product features as yk, which can be obtained by a coordinate measuring machine (CMM). The state space model in Equation 9.1 can be obtained through a similar (to the preceding panel assembly) but more complicated 3-D kinematics analysis, where Ak1xk1 is the error contributed
2151_book.fm Page 221 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
Cover Face (M)
(a)
Joint Face
(b)
221
Slot (S)
(c)
FIGURE 9.6 Process layout at three stages.
by the errors of datum features (these features are produced in previous stages) and Bkuk is the error contributed by the fixture or cutting tool at stage k. Details of this process and the corresponding state space model can be found in Chapter 7. After the model in Equation 9.1 is obtained, the diagnosability study for the multistage machining process can be conducted following the theories from Sections 9.2 to 9.5. We will focus on the fixture error in this case study. For a 3-2-1 fixture setup, there are six potential fixture errors at each stage (each locating pad and pin could have one error). Hence, there are 18 potential faults in the whole system, where faults 1–6 represent locator errors at the first stage, faults 7–12 at the second stage, and faults 13–18 at the third stage, respectively. Three gauging systems are used to measure slot S, cover face M, and the rough datum, respectively, where the rough datum is the primary datum at the first stage and can be seen from the joint face. The results of a fault diagnosability for the three systems are listed in Table 9.2. The RREFs of the testing matrices of gauging strategies 1 and 2 have a very simple structure. For gauging strategy 3 (the fourth column in Table 9.2), the first Γ) share common nonzero column positions. The corresponding three rows of RREF(Γ faults, {1, 2, 3, 7, 8, 9}, form a connected fault class regarding its mean diagnosΓ), we can generate 15 ability. By permuting the corresponding columns of RREF(Γ minimal diagnosable classes (each has four faults) within this connected fault class, as shown in Table 9.2. The minimal complementary class of this connected fault 3 class can be found by searching the class with n = 6, ρ = 3. We obtain C6 = 20 minimal complementary classes for the connected class: {1, 7, 8}, {2, 7, 8}, {3, 7, 8}, {1, 7, 9}, {2, 7, 9}, {3, 7, 9}, {1, 8, 9}, {2, 8, 9}, {3, 8, 9}, {1, 2, 7}, {1, 2, 8}, {1, 2, 9}, {1, 3, 7}, {1, 3, 8}, {1, 3, 9}, {2, 3, 7}, {2, 3, 8}, {2, 3, 9}, {1, 2, 3}, {7, 8, 9}. Adding the nondiagnosable faults {4, 5, 6, 13~18}, we can obtain 20 minimal complementary classes for the system regarding the mean diagnosability. It is also interesting to see that although the faults {1, 2, 3, 7, 8, 9} form a connected fault class regarding their mean diagnosability, they are uniquely diagnosable regarding their variance diagnosability. This verifies our previous remark that mean diagnosability requires a stronger condition than variance diagnosability.
9.7 SUMMARY This chapter studied the diagnosability of process faults given the product quality measurements in a complicated multistage manufacturing process. The study reveals
Gauging System Mean diagnosability: Γ) RREF(Γ
06 ×12 I 6 × 6 30 ×12 30 × 6 0 0 36 × 18
System 1 (Slot S)
6×3 6×6 I 0 030 × 3 030 × 6 0
30 × 3
0 0 0
0 0 0
0 0 0
0 0 −1
0 1 0
1 0 0
6×6 0 30 × 6 0 36 × 18
System 2 (Cover Face M)
−0.63 I 3× 3 03× 3 0.47 −0.49 0 3× 3 0 3× 3 30 × 3 30 × 3 0 0
0
3× 3
30 × 3
0
−0..48 −0.04
0
3× 3
3× 3
30 × 3
I
−0.57 −0.90 0
0.53 −0.90
System 3 (Rough Datum)
0 3× 6 0 30 × 6 0 3× 6
222
TABLE 9.2 Comparison of Gauging Systems
2151_book.fm Page 222 Friday, October 20, 2006 5:04 PM
Stream of Variation Modeling and Analysis for MMPs
Number of uniquely identified faults Number of minimal complementary classes
Minimal diagnosable classes
Number of potential faults Rank of testing matrix
Variance diagnosability: RREF(Hr)
mean variance mean variance
variance
18 Γ Hr mean
{13}, {14}, {15}, {16}, {17}, {18} 6 6 1 1
6 6 {13}, {14}, {15}, {16}, {17}, {18}
06 ×12 I 6 × 6 12 ×12 12 × 6 0 0 18 × 18
0
12 × 3
0 0 0
0 0 0
0 0 0
0 0 1
0 1 0
1 0 0
3 3 8 8
{7}, {8}, {9}, {4, 10}, {5, 11}, {6, 12}
18 6 6 {7}, {8}, {9}, {4, 10}, {5, 11}, {6, 12}
6×3 6×6 0 I 012 × 3 012 × 6
6×6 0 12 × 6 0 18 × 18
3 9 20 1
18 6 9 {10}, {11}, {12}, {1, 7, 8, 9}, {2, 7, 8, 9}, {3, 7, 8, 9}, {1, 3, 8, 9}, {2, 3, 8, 9}, {1, 7, 3, 9}, {2, 7, 3, 9}, {1, 7, 8, 3}, {2, 7, 8, 3}, {1, 2, 8, 9}, {1, 7, 2, 9}, {1, 7, 8, 2}, {1, 2, 3, 9}, {1, 2, 8, 3}, {1, 7, 2, 3} {1}, {2}, {3}, {7}, {8}, {9}, {10}, {11}, {12}
I 3× 3 0 3× 3 0 3× 6 0 3× 6 6×3 6×3 6×6 6×6 0 0 I 0 09 × 3 09 × 3 09 × 6 09 × 6
2151_book.fm Page 223 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification 223
2151_book.fm Page 224 Friday, October 20, 2006 5:04 PM
224
Stream of Variation Modeling and Analysis for MMPs
that the diagnosis capability that a gauging strategy can provide strongly depends on sensor deployment in a multistage manufacturing system. A poorly designed gauging strategy is likely to result in loss of diagnosability. On the contrary, a welldesigned gauging system that achieves the desired level of diagnosability can not only monitor the process change but also quickly identify the process root causes of quality-related problems. Quick root cause identification will lead to product quality improvement, production downtime reduction, and a remarkable cost reduction in manufacturing systems. This study was a model-based approach; a linear fault-quality model was used. The results can be used when a linear diagnostic model is available. Because the errors of tooling elements considered in quality control problems are often much smaller than the nominal parameters, most manufacturing systems can be linearized and then represented by a linear model under the small error assumption. Many of the linear state space models were validated through comparison with either a commercial software simulation [6] or with experimental data [7,8]. Thus, the small error assumption is not restrictive, and the methodology presented in this article is generic and applicable to various manufacturing systems. Another note on the applicability of the reported methodology is that for some poorly designed manufacturing system, a large number of process faults could possibly be coupled together and form a single, huge connected fault class. As a result, it would be impractical to exhaust matrix column-permutation in finding the complete list of minimal diagnosable classes, and the diagnosability study itself then becomes intractable.
9.8 EXERCISES 1. In an assembly process, a liftgate (Figure 9.7b) will be mounted on the back of a car (Figure 9.7a). Six points are measured on the liftgate, as illustrated in Figure 9.7c. Among them, 1, 2, 3, and 4. Measure the horizontal deviation only, 5 and 6 measure the up-down deviation only. In practice, five variation patterns of the liftgate are identified, as given in Figure 9.8. The dotted line shows the ideal liftgate, and the solid line shows the deformation pattern of the liftgate.
liftgate opening
roof cross-member 5 left bodyside
(a) A body in white
6
2
4
1
3
right bodyside
(b) A liftgate
FIGURE 9.7 Liftgate assemble and measurement process.
(c) Measurement points on a liftgate
2151_book.fm Page 225 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
225
FIGURE 9.8 Variation patterns of the liftgate in assembly.
We use a vector u to represent the amount of deviation along each of the patterns shown in Figure 9.8. For example, the first element of u represents the amount by which the liftgate opening enlarges on the auto body, the second element of u represents the amount the liftgate opening translates on the auto body, etc. Further, we use a vector y to represent the six measurements. Answer the following questions: a. If a linear model is used to link the error patterns and the measurements as y = Γu + ε , then what is Γ? (Hint: Γ is a 6 × 5 matrix that only contains 1, –1, and 0.) b. If we ignore the influence of noise, then is u mean diagnosable? Is u variance diagnosable? c. If measurement points 1 and 4 are removed, then is u still mean diagnosable? Is u still variance diagnosable? If no, what faults are coupled together? d. If measurement points 1 and 4 are removed and measurement point 5 is moved to the middle of the roof crossmember, then is u mean diagnosable? Is u variance diagnosable? e. Based on the results of (d), what is the difference between mean diagnosis and variance diagnosis? 2. Figure 9.9a shows an assembly station in which two parts are assembled together. Suppose we are not concerned with the displacement of the parts in the x direction and are only considering the four variance components that represent the z-direction deviations of the four pin/hole or pin/slot combinations. If the z direction deviation of the four pins are denoted as u and the four measurements are denoted as y, and the relationship between u and y is modeled linearly as y = Γ · u + ε, answer the following questions: a. For the setup in Figure 9.9a, what is Γ? b. For the setup in Figure 9.9a, if the measurement noise is ignored, then is u mean diagnosable? Is u variance diagnosable? c. For the setup in Figure 9.9a, if the measurement noise is included as a separate variation source and its covariance is of the structure of σ2I, then is u mean diagnosable? Is u variance diagnosable? d. For the setup in Figure 9.9b (the distance between y1 and y2 and y3 and y4 is 0.5), repeat (a) to (c). e. Which setup is better in terms of diagnosability?
2151_book.fm Page 226 Friday, October 20, 2006 5:04 PM
226
Stream of Variation Modeling and Analysis for MMPs
(a) Set up 1.
(b) Set up 2.
FIGURE 9.9 An assembly process.
3. For a linear system y = Γu + ε, answer the following questions (noise can be ignored in the analysis): a. If Γ is a 4 × 4 matrix as 0.9501 0.2311 0.6068 0.4860
0.8913 0.7621 0.4565 0.0185
0.8214 0.4447 0.6154 0.7919
0.9218 0.7382 0.1763 0.4057,
what is the mean and variance diagnosability of u? b. If Γ is a 4 × 4 matrix as 0.9501 0.2311 0.6068 0.4860
0.8913 0.7621 0.4565 0.0185
0.8214 0.4447 0.6154 0.7919
0.8712 0.2119 0.5564 0.4456,
what is the mean and variance diagnosability of u? c. If Γ is a 4 × 4 matrix as 0.9501 0.2311 0.6068 0.4860
0.8913 0.7621 0.4565 0.0185
0.3695 0.2823 0.1962 0.0347
1.1863 0.7759 0.6569 0.2159,
what is the mean and variance diagnosability of u?
Reference 1. Schott, J.R., Matrix Analysis for Statistics, John Wiley & Sons, New York, 1997. 2. Ding, Y., Shi, J., and Ceglarek, D., Diagnosability analysis of multistage manufacturing processes, ASME Journal of Dynamic Systems, Measurement and Control, 124, 1, 2002. 3. Rao, C.R. and Kleffe, J., Estimation of Variance Components and Applications, NorthHolland, Amsterdam, 1988. 4. Searle, S.R., Casella, G., and McCulloch, C.E., Variance Components, John Wiley & Sons, New York, 1992. 5. Halevi, G. and Weill, R.D., Principles of Process Planning: A Logical Approach, Chapman and Hall, New York, 1995.
2151_book.fm Page 227 Friday, October 20, 2006 5:04 PM
Diagnosability Analysis for Variation Source Identification
227
6. Ding, Y., Ceglarek, D., and Shi, J., Modeling and diagnosis of multistage manufacturing processes: part I — state space model, Proceedings of the 2000 Japan/USA Symposium on Flexible Automation, July 23–26, Ann Arbor, MI, 2000JUSFA-13146, 2000. 7. Djurdjanovic, D. and Ni, J., Linear state space modeling of dimensional machining errors, Transactions of NAMRI/SME, XXIX, 541, 2001. 8. Zhou, S., Huang, Q., and Shi, J., State space modeling for dimensional monitoring of multistage machining process using differential motion vector, IEEE Transactions on Robotics and Automation, 19, 296, 2003. 9. Zhou, S., Ding, Y., Chen, Y., and Shi, J., Diagnosability study of multistage manufacturing processes based on linear mixed-effects models, Technometrics, 45, 312, 2003.
2151_book.fm Page 228 Friday, October 20, 2006 5:04 PM
2151_book.fm Page 229 Friday, October 20, 2006 5:04 PM
10
Diagnosis through Variation Pattern Matching*
10.1 INTRODUCTION TO VARIATION PATTERNS The relationships between product quality and process variation sources such as fixture errors and machine errors in a multistage manufacturing process are modeled in a linear state space form. They have the same linear model structure because the process faults are assumed to be of smaller magnitude as compared to nominal product dimensions. The chain like state space model can be further transformed into an input-output model and can be generally expressed as: y = Γ ⋅u + ε
(10.1)
where y is an n × 1 vector of product quality measurements, Γ is an n × p constant system matrix determined by product/process designs, u is a p × 1 random vector representing process faults, ε is an n × 1 random vector representing measurement noises, unmodeled faults, and high-order nonlinear terms. Equation 10.1 is equivalent to Equation 9.4. The term u in Equation 10.1 is actually µ + u in Equation 9.4, and ε is actually Ψ ⋅ w + v in Equation 9.4. Without loss of generality, we can further assume that the columns of Γ matrix are of unit length. This can be achieved through simple scaling. Because process variation errors such as variability increase are more difficult to diagnose and compensate than the mean change of process faults, most of the root cause identification methods focus on identifying root causes relevant to the variance changes of process faults. Therefore, the process fault vector u is often modeled as a random vector to describe variation errors, as opposed to a simple mean-shift type of error in the process. Because the mean of quality measurements can always be subtracted, it is often assumed that u and ε are zero mean. In Equation 10.1, the column vectors of Γ determine how a specific process fault affects product quality characteristics and causes specific variation patterns in product quality measurements. This concept can be illustrated in the following examples from assembly and machining processes. Variation pattern in assembly: Figure 10.1 shows the dimensional variation in a 2-D panel assembly process. The rectangle in Figure 10.1a represents a sheet metal * Part of chapter material is based on Reference 4.
229
2151_book.fm Page 230 Friday, October 20, 2006 5:04 PM
230
Stream of Variation Modeling and Analysis for MMPs Cutting Tool
G
H
Workpiece
y2
P3 E P2
z y
P1 y1
x
P2
P1
F D L2
u2 Z
y3
Y
A
L1
C L3 B
X
(a) Variation pattern in an assembly operation
(b) Variation pattern in a machining operation
FIGURE 10.1 Illustration of dimensional variation pattern.
part that needs to be assembled with another piece of sheet metal. The position of the sheet metal is determined by a 4-way pin locator (P1), which controls the translations in x and z directions, and a 2-way pin locator (P2), which controls rotation around the y-axis. The dash-line rectangle is the nominal position of the sheet metal. If there is a variation in the position of P2, the position of the sheet metal will vary as indicated by the two solid-line rectangles. A key observation in this case is that the variation of the position of the sheet metal is not purely random: the sheet metal actually rotates around P1. In other words, the variations in the multiple dimensional measurements satisfy a certain relationship. For example, the ratios among the magnitudes of the variations in y1, y2, and y3 will be fixed. This fixed relation among the variations of quality measurements are called variation patterns. Variation pattern in machining: A similar phenomenon can be found in a machining process as shown in Figure 10.1b. In this operation, the top surface (EFGH) will be milled. The position of the workpiece is determined by a 3-2-1 fixture layout. If there is a variation in the locator L2, then the machined surface will have an upward shift or a downward shift with a slope that is determined by the magnitude of the L2 deviation. Clearly, the location and orientation of the top surface shows a pattern in this case. From the preceding two examples, it can be seen that the variation sources in the process will manifest themselves as patterns in the product quality measurements (i.e., a fixed relationship among the variables of multivariate quality measurements). Based on this observation, the variation source identification can be realized through pattern matching. The basic idea is illustrated in Figure 10.2. First, based on the fault-quality model, we can obtain the unique patterns of each potential fault. Meanwhile, the symptom of an occurring fault can be extracted from measurement data. Finally, the occurring fault can be identified if there is a matched pattern between the fault symptom and the fault pattern obtained from the model.
2151_book.fm Page 231 Friday, October 20, 2006 5:04 PM
Diagnosis through Variation Pattern Matching
231 in-line
off-line
In-process sensing data
Engineering Knowledge Linear Fault-Quality Model
Data collection and variance analysis
Variation signatures of potential faults
Estimation of an occurring fault symptom
Pattern matching between signature and symptom Root cause of the fault
FIGURE 10.2 Outline of the pattern matching method.
10.2 LINKS BETWEEN THE FAULT-QUALITY MODEL AND VARIATION PATTERNS Section 10.1 only provide an intuitive illustration of variation pattern. To develop a rigorous pattern matching technique, a formal mathematical analysis is needed to link the quality variation and the parameters of the fault-quality model, which is the focus of this section. Assuming that the process fault u is independent of system noise ε, we can obtain the following from Equation 10.1: Σy = Γ ⋅ Σu ⋅ ΓT + Σε
(10.2)
where Σu and Σε are the covariance matrices of u and ε, respectively. Furthermore, it is usually assumed that process faults (i.e., the elements in u) are independent of each other and, hence, Σu is a diagonal matrix. Without loss of generality, let us assume that the ith fault occurs and only a single diagonal element of Σu, σ 2i , is nonzero. Then, we have Σ y = σ i2 ⋅ ri ⋅ riT + Σ ε
(10.3)
where ri is the ith column vector of Γ. If we multiply ri on both sides of Equation 10.3, we have Σ y ⋅ ri = σ i2 ⋅ ri ⋅ riT ⋅ ri + Σ ε ⋅ ri . If the components in ε are independent of each other and have the same variances, then Σε will have the simplest form σ 2ε ⋅ I, where σ 2ε is the variance of the noise and I is an identity matrix with appropriate dimension. Under these conditions, we have Σ y ⋅ ri = σ i2 (ri ⋅ riT ⋅ ri ) + σ 2ε ⋅ ri , and further
2151_book.fm Page 232 Friday, October 20, 2006 5:04 PM
232
Stream of Variation Modeling and Analysis for MMPs
Σ y ⋅ ri = (σ i2 (riT ⋅ ri ) + σ 2ε ) ⋅ ri
(10.4)
Because σ i2 (riT ⋅ ri ) + σ ε2 is a scalar, ri is apparently an eigenvector of Σy . If we multiply both sides of Equation 10.3 by ri⊥ , where ri⊥ is any vector perpendicular to ri and has unit length, we have Σ y ⋅ ri⊥ = σ i2 (ri ⋅ riT ⋅ ri⊥ ) + σ ε2 ⋅ ri⊥ . Because riT ⋅ ri⊥ is zero, we get Σ y ⋅ ri⊥ = σ 2ε ⋅ ri⊥
(10.5)
which suggests that ri⊥ is also an eigenvector of Σy . The difference between ri and ri⊥ is the eigenvalues they are associated with: ri is associated with the largest eigenvalue of Σy and ri⊥ is associated with a small eigenvalue of Σy . This analysis indicates that when the ith process fault occurs, the eigenvector associated with the largest eigenvalue of Σy , known as its principal eigenvector, should match the ith column vector of Γ. Thus, the principal eigenvector of Σy can be viewed as the fault symptom and the corresponding column vector of Γ can be viewed as variation pattern or signature determined by the fault-quality model. An example is given to demonstrate the variation patterns. We consider a simple machining operation in which the top surface of a cubic metal is milled, as seen in Figure 10.3. The primary datum is located at the bottom surface indicated by u1, u2, and u3. Their x and y coordinates are (10,10), (110,10), and (10,90), respectively. Because we are only concerned with the depth of cutting, which will be solely determined by the primary datum, the other datums are not shown in this figure. After milling, we measure the z coordinates of four points on the top surface to evaluate the depth of cutting. These four points are labeled as M1, M2, M3, and M4, and their x and y coordinates are (10,10), (200, 10), (200,200), and (10,200), respectively. Clearly, the z coordinates of the locators will certainly influence the z coordinates of the measurement points. A kinematical analysis shows that the following linearized relationship holds:
z
Cutting Tool
M4 M1
M3 M2
y u3 u1
u2
FIGURE 10.3 An example from machining processes.
x
2151_book.fm Page 233 Friday, October 20, 2006 5:04 PM
Diagnosis through Variation Pattern Matching
z M1 −1 z M2 = 0.9 z M3 3.275 z M4 1.375
0 −1.9 −1.9 0
233
0 zu1 0 z + ε −2.375 u2 z −2.375 u3
(10.6)
where zMi and zui are the deviation of the measurement points and the locators in z direction, respectively, and ε represents the linearization and measurement noises. Each column of the coefficient matrix actually is the variation pattern associated with each locator variation error. For example, the first column [–1 0.9 3.275 1.375]T indicates that if the first locator u1 has a 1 unit upward shift and the other two locators are kept at nominal locations, then the z coordinates of the four measurements will have deviations of –1, 0.9, 3.275, and 1.375 units, respectively, after milling. Note that the lengths of the columns of the coefficient matrix are not one in Equation 10.6. However, a simple scaling yields z M1 −0.263 z M2 = 0.237 z M3 0.862 z M4 0.362
0 −0.707 −0.707 0
0 3.798 ⋅ zu1 0 2.687 ⋅ z + ε u2 −0.707 3.359 ⋅ zu3 −0.707
and then we can treat [3.798 ⋅ zu1 2.687 ⋅ zu2 3.359 ⋅ zu3 ] as the standardized variation sources to transform the lengths of the columns into one. If u1 is loose and thus zu1 has a large variation, then the first eigenvector of the covariance matrix of [ zM1 zM2 zM3 zM 4 ]T will be close to [–0.263 0.237 0.862 0.362]T from previous derivations. This result can lead us to obtain the following variation source identification procedure. If only one variation source exists in the process, we need to check if the symptom vector falls within a confidence boundary of a specific signature vector. This boundary is due to the sample uncertainty and measurement noises. If a match is found, then the fault represented by the specific signature vector occurs.
10.3 PROCEDURE OF PATTERN MATCHING FOR VARIATION SOURCE IDENTIFICATION If a single fault happens in the system, we need to match the symptom vector and the corresponding signature vector. However, in practice the value of the population covariance matrix Σy is unavailable. Instead, only the sample covariance is available. Sampling uncertainty in Sy makes the sample principal eigenvector a random vector. Instead of checking if the sample principal eigenvector equals the theoretical fault signature vector in an exact and deterministic sense, we should establish a confidence boundary to see if they are equal in a probabilistic sense.
2151_book.fm Page 234 Friday, October 20, 2006 5:04 PM
234
Stream of Variation Modeling and Analysis for MMPs
We assumed the covariance of the noise ε is in the form of Σε = σε2 · I in Section 10.2. This assumption can probably only represent well for the case where identical measurement devices are used to measure quality features. When different measurement devices are used, the variances of different components in ε will be different. Moreover, the noise term ε could comprise unmodeled faults and high-order nonlinear residuals, which are more likely to be correlated and makes even a diagonal structure in Σε less likely. For this reason, it is more practical to study Σy in a general structure. The unstructured Σε will disturb the principal eigenvector of Σy and make its direction deviate from the corresponding signature vector. Thus, in pattern matching, we need to consider not only sample uncertainty, but also the disturbance of unstructured noise. Several researchers have contributed to pattern-matching based variation source identification. Ceglarek and Shi [1] developed a pattern matching method for fixture fault diagnosis in automotive body assembly processes by assuming a structured noise and large sample size (namely, Σy is known). Their method is extended by Rong et al. [2] by considering the sample properties of the principal eigenvector but retaining the structured noise assumption. Ding et al. [3] studied the impact of unstructured noise under the large-sample assumption. In this chapter, a pattern matching technique considering both sample uncertainty and unstructured noise will be presented. This work is based on Li et al. [4]. Figure 10.4 illustrates the basic idea of this pattern matching technique. Vector ri is a column vector of Γ, which is the same as the principal eigenvector of Γ Σ u ΓT when the ith fault happens. With the presence of unstructured noise, the principal eigenvector of Σy , denoted as v (i 1) (the subscript i means the ith fault happens, and the superscript (1) represents that it is the eigenvector, also called principal eigenvector, associated with the largest eigenvalue), is not the same as ri anymore. Instead, it will fall in a cone, as shown in Figure 10.4. The boundary of the cone can be represented by the angle γ c . The sample principal eigenvector of Sy, denoted as us, will be different from v (i 1) because of sample uncertainty. The final boundary for us can be illustrated by the dashed line, as shown in Figure 10.4. If the sample principal eigenvector falls in the confidence boundary of a fault signature vector ri, we can claim that the corresponding fault occurs. Note that Figure 10.4 exaggerates the size of the region for the sake of illustration. When the sample size is large and the noise variance is small, the confidence range of us is actually small.
Boundary of us Boundary of v (i1)
ri c
FIGURE 10.4 The boundary for pattern matching of the columns of Γ.
2151_book.fm Page 235 Friday, October 20, 2006 5:04 PM
Diagnosis through Variation Pattern Matching
235
The subsequent sections will present the confidence boundary of us. We list our assumptions as follows: 1. The fault-quality relation can be adequately described by Equation 10.1. The process faults and the system noises are assumed to be independent of each other. 2. The covariance of ε is in a general form as Σε . Σε may be unknown but the range of its eigenvalues is assumed known. 3. One fault occurs at a time, meaning that multiple faults do not occur in the system simultaneously. The rationale of the second assumption is that ε is usually dominated by measurement noises, whose range of eigenvalues represents the range of sensor accuracy that can be obtained through the sensor vendor’s specification. Even if ε comprises unmodeled faults and high-order nonlinear residuals, offline calibration could provide information regarding the range of eigenvalues of Σε .
10.3.1 DISTURBANCE
DUE TO
UNSTRUCTURED NOISES
In this subsection we focus on the perturbation of unstructured noise on the eigenvectors of Σy . When the ith fault happens, the principal eigenvalue and eigenvector pair of Γ Σ u ΓT, Σ y and Sy, are { λ (u,1)i , h (i 1) }, { λ (y,1)i , v (i 1)}, and { ly,(1i) , us}, respectively. As previously discussed, if only the ith fault occurs, the principal eigenvector ( h (i 1) ) of Γ Σ u ΓT is the same as the ith column vector (ri) of Γ. In the presence of unstructured noises the principal eigenvector ( v (i 1)) of Σy differs from ri, and the boundary of this difference is related to the additive noise covariance matrix Σε . A useful result regarding the perturbation boundary caused by unstructured noise is given by Ding et al. [3]: When only the ith fault occurs in the system (i.e., only λ (u,1)i , the principal eigenvalue of Γ Σ u ΓT is nonzero) and || Σε ||2 ≤ λ (u,1)i / 4, then 4 dist(span{ri},span{v i(1)}) ≤ (1) λ u ,i
2 ( Σ ε ) − λ 2min ( Σ ε ) λ max
(10.7)
where λ 2max ( Σ ε ) and λ 2min ( Σ ε ) are the largest and the smallest eigenvalues of Σε , and dist (span{ri }, span{v i(1)}) is the distance [5] between the space spanned by ri and v (1) i , respectively. The condition Σε ≤ 2
λ (u,1)i 4
implies that the largest variance of the noise term is four times smaller than the variance of the process fault, which is not restrictive in practice. Because ri and v (i 1) are two unit vectors (in the sense of its 2-norm) in Rn, Rn, the distance between two subspaces of span{ri } and span{v (i 1)}, equals the sine of the angle (∆θ) between ri and v (i 1). That is, sin( ∆θ) = dist(span{ri }, span{v (i 1)}). The detailed proof of this result can be found in Li, Zhou, and Ding [4]. Based on this result and Equation 10.7, we have
2151_book.fm Page 236 Friday, October 20, 2006 5:04 PM
236
Stream of Variation Modeling and Analysis for MMPs
4 ∆θ ≤ sin −1 (1) λ u ,i
2 2 λ max ( Σ ε ) − λ min (Σ ε ) = γ c
(10.8)
Obviously, the angle between ri and v (i 1) can be calculated by using cos(∆θ) = v ⋅ ri . Its boundary will be denoted by (1) T i
4 γ c = sin −1 (1) λ u ,i
2 2 λ max ( Σ ε ) − λ min (Σ ε )
hereafter. The preceding result tells us that the difference between ri and v (i 1), represented by the angle ∆θ between them (as shown in Figure 10.4), is determined by the eigenvalue of Γ Σ u ΓT and the extreme eigenvalues of Σε . Also, note that the boundary specified by Equation 10.8 is a worst-case boundary. The geometric meaning of this boundary is shown in Figure 10.4; that is, v (i 1) will construct a hyperdimensional cone with its central axis as ri. Equation 10.8 suggests that the perturbation boundary between ri and v (i 1) depends on λ (u,1)i /λ max ( Σ ε ) and λ max ( Σ ε ) /λ min ( Σ ε ). The value of λ (u,1)i /λ max ( Σ ε ) can be regarded as the signal-to-noise ratio, whereas λ max ( Σ ε ) / λ min ( Σ ε ) indicates the imbalance in accuracy associated with different measurement devices. With the increase in signal-to-noise ratio, the perturbation boundary will get smaller. For instance, if λ max ( Σ ε ) =0.030, λ (u,1)i /λ max ( Σ ε ) is 10, and λ max ( Σ ε ) /λ min ( Σ ε ) is around 10, the perturbation boundary (i.e., the angle) will be around 20°. However, if λ (u,1)i /λ max ( Σ ε ) is around 50, meaning the fault magnitude is five times larger than before, the perturbation angle will reduce to about 5˚. On the other hand, with a higher imbalance among eigenvalues of Σε , the perturbation boundary will become larger. If all eigenvalues of Σε are the same, there will be no perturbation on v (i 1) even if the measurement device is not very accurate, corresponding to the situation that the absolute values of eigenvalues of Σε are high but equal.
10.3.2 DISTURBANCE
DUE TO
SAMPLING UNCERTAINTY
This subsection presents the result of a testing procedure for the principal eigenvector of a sample covariance matrix. We use the same notation as before; namely, us and v (i 1) are the principal eigenvector of Sy and Σy , respectively, and the subscript i indicates that the ith fault happens. According to Murihead [6], if λ y,i is a distinct eigenvalue (which is true in our 1case when a single fault occurs) and y follows a /2 normal distribution, then N − 1 u s − v (i 1) asymptotically follows an n-variate normal distribution with zero mean and covariance matrix of Π, where
(
) (
)
n
Π = λ (y1,)i
∑ (λ j=2
λ (yj,i) (1) y ,i
− λ (yj,i) )2
v (i j )v i( j )T
(10.9)
and N is the sample size, and n is the dimension of measurement in Equation 10.1. Apparently, because of the sampling uncertainty, different realizations of us will generate a hyper dimensional cone centering around the corresponding vector v (i 1) .
2151_book.fm Page 237 Friday, October 20, 2006 5:04 PM
Diagnosis through Variation Pattern Matching
237
Based on Equation 10.9, given a symptom vector us, an asymptotic test can be developed to determine if the principal eigenvector of Σy is v (i 1) . The test statistic can be formulated as 1 Wi ≡ ( N − 1)(v (i1) − u s )T ly(1,i)S−y1 − 2I + (1) Sy (v (i1) − u s ) ly ,i
(10.10)
It is known that Wi asymptotically follows the distribution of χ2n−1 if the hypothesis is true. If Wi > χ2n −1 (α ) , where χ2n−1 (α ) is the upper 100α% point of the χ2n−1 distribution, we reject the hypothesis, meaning that v (i 1) is not the principal eigenvector of Σy under the sampling uncertainty. Otherwise, we accept the hypothesis, meaning that v (i 1) is the principal eigenvector of Σy.
10.3.3 A ROBUST PATTERN MATCHING PROCEDURE A robust pattern matching procedure considering both effects of noise perturbation and sampling uncertainty is developed in this subsection. Consider two faults, represented by ri and rj , in Figure 10.5. Vectors ri and rj are fault signature vectors from matrix Γ. Because of the perturbation of unstructured noises, there are two cones centered around ri and rj , respectively, within which the eigenvectors of Σy , v (i 1) and v (j1), will fall. Once measurement data are collected, the fault symptom vector us, the principal eigenvector, can be calculated from Sy. We want to identify exactly which fault happens. There are two cases: (1) when us falls into either one of the two cones and (2) when us falls outside both cones. For case 1, because us, under extra sampling uncertainty, still falls within the confidence boundary of population covariance matrices, the meaning is that the corresponding fault occurs. The decision rule is then simple and the same as when there is no sampling uncertainty, namely, we can immediately claim that a fault represented by either ri or rj, whichever is applicable, occurs. For case 2, we need to further calculate a test statistic similar to that in Equation 10.10 to decide which fault has occurred.
v (i1)
fuc s
ra i Boundary due to noise
Boundary due to noise v (j1)
ar j
FIGURE 10.5 The method to identify the boundary of vi(1).
2151_book.fm Page 238 Friday, October 20, 2006 5:04 PM
238
Stream of Variation Modeling and Analysis for MMPs
However, owing to the perturbation from unstructured noise ε, we do not know the precise position of v (i 1) . Instead, we only know the worst case boundary, within which v (i 1) falls. One straightforward approach would be to use the worst-case boundary as the position of v (i 1) . However, a simple numerical study shows that the test statistic in Equation 10.10 using a “worst-case” v (i 1) (i.e., using the value on the worst-case boundary) often yields a very large Wi value so that the hypothesis that v (i 1) is the principal eigenvector of Σy is always rejected and a high miss detection rate results. To reduce the miss detection rate, we adopt a conservative approach to locate v (i 1) for testing Equation10.10. Instead of using a boundary vector, we try to find a vector located within the hyperdimensional cone (including the boundary) that can minimize the Wi value. Then this vector is substituted in Equation 10.10 as v (i 1) to calculate Wi*. A typical nonlinear programming problem to find an appropriate v (i 1) can be formulated as follows:
Objective:
1 Wi* = min( N − 1)(v (i1) − u s )T ly(1,i)S−y1 − 2I + (1) Sy (v i(1) − u s ) (1) vi ly ,i
Constraints: v i(1)T v (i 1) = 1 ∆θi = cos −1
(v (i1)T ri ) ≤ γc 11ri 11
In the preceding formulation, the objective function is to find a v (i 1) that minimizes Wi while satisfying a set of constraints. The constraints mean that v (i 1) should remain as a unit vector and the angle between v (i 1) and ri should be smaller than the worst-case boundary range γc. Based on the Kuhn–Tucker conditions [7], we can convert the inequality constraint to an equation and get the solution of v (i 1) . In our numerical study, we use the function fmincon in Matlab software to solve this nonlinear programming problem. After getting the vectors of v (i 1) and us, the final step is to calculate the value of Wi* and compare this value with the upper 100α% point of the χ2n−1 distribution. Our decision rule using the aforementioned optimized v (i 1) and Wi* follows what was previously stated: if Wi* is not greater than the critical value, we claim that the corresponding fault occurs; otherwise we claim this fault does not occur. This conclusion is conservative in terms of fault isolation. Because the vector that minimizes the Wi* statistic might not be the true v (i 1) , we are more inclined to conclude that a fault indeed occur. Subsequent investigations are still needed to verify the conclusion. In practice, this strategy fits fault-critical applications. In this case, we would prefer a small miss detection rate and be very cautious about any potential signs of significant process faults because of the very high warranty cost associated with quality problems. After identifying the ith process fault, we can further estimate the variance magnitude of the fault. We have [5],
2151_book.fm Page 239 Friday, October 20, 2006 5:04 PM
Diagnosis through Variation Pattern Matching
239
λ (u1,)i + λ min ( Σ ε ) ≤ λ (y1,)i ≤ λ (u1,)i + λ max ( Σ ε )
(10.11)
Based on Equation 10.11, we can get the range of eigenvalue of Γ Σ u ΓT as follows, λ (y1,)i − λ max ( Σ ε ) ≤ λ (u1,)i ≤ λ (y1,)i − λ min ( Σ ε )
(10.12)
It is known that σ 2i (riT ⋅ ri ) is the eigenvalue of Γ Σ u ΓT corresponding to the process fault. Substituting λ u,i = σ 2i (riT ⋅ ri ) in Equation 10.12 and using the average of the upper and lower bounds in Equation 10.12 as an approximation for σ 2i , we then estimate the variation magnitude of a fault as, σ i2 ≈
1
{
ly(1,i) − λ min ( Σ ε ) + ly(1,i) − λ max ( Σ ε ) 2(r ⋅ ri ) T i
}
(10.13)
where λ (y,1)i is substituted by its sample version ly,(1i) . Apparently, if the variance of the noise is relatively small compared to that of a process fault, which is usually the case, the approximation error will be small. The reason is that when the largest and smallest eigenvalues of ∑y become smaller, so does the range stated in Equation 10.12. To summarize, the proposed pattern matching procedure is listed as follows: S1: A fault-quality model y = Γ ⋅ u + ε is developed. Based on the model, fault signature vectors rk , k = 1,…, p, are obtained from the columns of Γ. S2: The multivariate measurements y of product quality features are obtained during production. The sample size is N and the dimension of measurements is n. S3: Based on y, calculate the sample covariance matrix Sy and the principal eigenvector us using PCA. Σε) and λmin(Σ Σε) from an analysis of accuracy specification S4: Estimate λmax(Σ of the measurement system. Based on Equation 10.8, we can calculate γ c . Here, we often need to substitute λ (u,1)i with the tolerance specification of the corresponding process variable if the precise value of λ (u,1)i is not yet known. Then, we calculate the angles ∆θ k between rk and us, k = 1, …, p. If all of the angles are larger than γ c , go to the next step, otherwise go to step S6. S5: Select a confidence level 1 – α. For the nonlinear programming problem, we find the optimal value of v (k1), k = 1, …, p, and calculate the value of Wk*, k = 1, …, p. If all the values are larger than χ2n−1 (α ) , we can claim that none of the known faults happen and the system is working normally; otherwise go to the next step. S6: If ∆θ k ≤ γ c or Wk* ≤ χ2n −1 (α ) , we can claim the kth fault, represented by the kth column of matrix Γ, happens. After that, the variation magnitude of the fault can be calculated by Equation 10.13. If all Wk*s, k = 1, …, p, are larger than χ2n−1 (α ) , then we claim that none of the known faults happen and the system is working normally.
2151_book.fm Page 240 Friday, October 20, 2006 5:04 PM
240
Stream of Variation Modeling and Analysis for MMPs
10.4 CASE STUDY 10.4.1 A MACHINING PROCESS PROPAGATION MODEL
AND ITS
VARIATION
The manufacturing process considered here is the same as that used in Chapter 7. To simplify the problem, we consider only the first two operations (milling the cover face and the joint face) in this case study. A total of 15 and 16 points on the joint face and the cover face are measured to determine the quality of the machining operation. These measurement points are evenly distributed on these two surfaces. Therefore, y will be a 31 × 1 vector (i.e., n = 31), consisting of the deviations at 15 points on the cover face and 16 points on the joint face. Fault u is then a 6 × 1 vector (i.e., p = 6), where the first three elements correspond to three pins at the first operation and the last three elements correspond to three pins at the second operation. A fault-quality model can be established from a sophisticated kinematic analysis (refer to Chapter 7 for details). Matrix Γ of this specific process is given as: Γ Γ = 1 Γ 2
(10.14)
Γ11 0], Γ2 = [0 Γ22], and where Γ1 = [Γ Γ11 = –0.5402 –0.6455 –0.2664 0.0683 0.5158 0.8555 0.9902 1.1863 1.3106 1.0177 0.6394 0.3403 0.0530 –0.0820 –0.2776 0.9690 0.5991 0.2203 –0.1140 –0.5612 –0.9005 –0.7058 –0.4225 –0.2428 0.1052 0.4597 0.7585 1.0455 0.8192 0.5367 0.5711 1.0464 1.0461 1.0458 1.0453 1.0450 0.7156 0.2362 –0.0678 –0.1229 –0.0991 –0.0988 –0.0985 0.2628 0.7409
T
Γ22 = -0.6349 -0.4590 -0.0332 0.2510 0.6955 1.0215 1.1372 1.1155 0.9048 0.6472 0.5864 0.3875 0.0777 -0.1490 -0.4063 0.0776 0.9602 0.6817 0.2556 -0.0288 -0.4736 -0.5857 -0.5031 -0.3477 -0.1368 0.1210 -0.0142 0.3998 0.7098 0.9368 0.9339 0.4496 0.6747 0.7773 0.7776 0.7778 0.7781 0.5641 0.3659 0.2322 0.2320 0.2318 0.4279 0.2127 0.2124 0.2123 0.4725 0.4728
T
The six column vectors of Γ are the signature vectors of six potential faults. In this example, noise ε is dominated by errors associated with the measurement device. Because the orientations of a CMM probe are different when it is used to measure the joint face and the cover face, the variances of measurement errors are therefore different for measurement points on these two different surfaces. Based on engineering analysis, the standard deviations of components in εi are between 0.008 and 0.015 mm. Furthermore, from the design specification of our fixture system, the locating tolerance is around 0.050 mm. If the standard deviation of components in u is larger than this tolerance, we deem that the corresponding fault occurs. Next we perform root cause identification using the pattern matching technique based on this faultquality model and the CMM measurements.
2151_book.fm Page 241 Friday, October 20, 2006 5:04 PM
Diagnosis through Variation Pattern Matching
241
10.4.2 PATTERN MATCHING FOR ROOT CAUSE IDENTIFICATION IN THE MACHINING PROCESS Given that the standard deviations of components in εi are between 0.008 and 0.015 mm, we randomly generate a positive definite matrix with diagonal elements falling in the range of 64- to 225 × 10–6 mm2. Then, we use this matrix as the noise Σε)and λmin(Σ Σε), can covariance matrix, whose largest and smallest eigenvalues, λmax(Σ be easily computed. We consider the situation when the third fault occurs. The variations of u are specified accordingly as: c mm 2 , k = 3 var u( k ) = 1 2 c 2 mm , k ≠ 3 where c 1 is larger than 2.5 × 10–3, the square of the locating tolerance, and c 2 is smaller than this value. This means that the number 3 variation source occurs in the system. We set the significant level of hypothesis testing as α = 0.01. The theoretical chi-square critical value in Equation 10.12 is χ230 (0.01) = 50.892 . First, we would like to compare the miss detection rates of the proposed method with two prior methods developed in Ding et al. [3] and Rong et al. [2], respectively. The method in Ding et al. [3] considers perturbation of unstructured noises but not sampling uncertainty, whereas the one in Rong et al. [2] considers sampling uncertainty in the absence of unstructured noise. We conducted 1000 replicates of fault detection using the aforementioned three procedures for a set of combinations of sample size N and perturbation angle γ c . Table 10.1 lists the resulting miss detection rates of different procedures, which are
TABLE 10.1 Miss Detection Rates of Three Pattern Matching Methods γc (degrees) N 50
100
400
1000
e e1 e2 e3 e1 e2 e3 e1 e2 e3 e1 e2 e3
0 100% 83.8% 83.6% 100% 30.1% 30.0% 100% 4.0% 4.0% 100% 2.3% 2.3%
0.54 100% 85.7% 0 100% 30.2% 0 100% 3.5% 0 99.9% 3.1% 0
0.91 100% 85.9% 0 100% 30.0% 0 100% 4.1% 0 95.7% 3.0% 0
2.42 100% 86.4% 0 99.8% 30.0% 0 69.7% 4.1% 0 19.4% 3.4% 0
4.06 96.7% 87.6% 0 74.2% 30.4% 0 7.4% 6.1% 0 0 7.4% 0
9.94 67.5% 88.3% 0 30.4% 33.8% 0 0 10.7% 0 0 21.7% 0
15.62 42.6% 88.7% 0 15.6% 37.7% 0 0 14.4% 0 0 31.5% 0
20.29 36.5% 88.3% 0 10.1% 37.4% 0 0 16.3% 0 0 43.2% 0
29.08 26.7% 90.1% 0 6.7% 47.4% 0 0 21.4% 0 0 51.2% 0
2151_book.fm Page 242 Friday, October 20, 2006 5:04 PM
242
Stream of Variation Modeling and Analysis for MMPs
the rates of the number of miss detection cases over the total number of tests (i.e., 1000 replications). We denote by e1, e2, and e3 the miss detection rate of the procedure without considering sample uncertainty, the procedure without considering unstructured disturbances, and the robust method described in this chapter, respectively. From the table we can observe the following: 1. The value of e1 could be very high when the sample size is small. For instance, e1 is nearly 100% for a sample size as high as N = 1000 when γ c < 2.42°. The miss detection ratio e1 depends on the value of γ c ; given the same sample size, the larger γ c , the smaller e1. This is understandable because γ c provides the worst-case boundary, and a larger γ c can help in offs