Stochastic Processes: with Applications to Reliability Theory

Springer Series in Reliability Engineering

For further volumes: http://www.springer.com/series/6917

Toshio Nakagawa

Stochastic Processes with Applications to Reliability Theory

123

Dr. Toshio Nakagawa Department of Business Administration Aichi Institute of Technology 1247 Yachigusa, Yakusa-cho 470-0392 Toyota Japan e-mail: [email protected]

ISSN 1614-7839 ISBN 978-0-85729-273-5

e-ISBN 978-0-85729-274-2

DOI 10.1007/978-0-85729-274-2 Springer London Dordrecht Heidelberg New York British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Ó Springer-Verlag London Limited 2011 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms of licenses issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. The use of registered names, trademarks, etc., in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant laws and regulations and therefore free for general use. The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. Cover design: eStudio Calamar, Berlin/Figueres Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface

I have learned first reliability theory from the book Mathematical Theory of Reliability [1] written by Barlow in 1965 that is the most famous and excellent one theoretically up to now. I was a marvelously lucky reader, unfortunately, I could not understand throughly this book at that time because my poor mathematical tools and abysmal ignorance about stochastic processes. I have now already published three monographs Maintenance Theory of Reliability [2], Shock and Damage Models in Reliability Theory [3], and Advanced Reliability Models and Maintenance Policies [4] in which I have surveyed mainly maintenance policies in reliability theory on the research results of the author and my colleagues. Most of the three books have been written based on basic theory of stochastic processes and their mathematical tools. A number of graduate students, researchers, and engineers demand of us some book written in an easy style on stochastic processes to be able to understand readily reliability theory. Recently, plants, satellites, computer and information systems have become more large-scale and complex, and most products are distributed all over the world. If they or some of them would fail and have trouble, it might incur many serious and catastrophic situations and heavy damage. More and better maintenance is required constantly from the economical and environmental points as public infrastructures and operating plants become old in advanced nations. Reliability and maintenance theory is more useful for protecting such severe matters and environmental considerations, and moreover, for making good and safe living. Stochastic processes can be described by probabilistic phenomena in some space at each point of time t. The knowledge of stochastic processes and mathematical tools is indispensable for engineers, managers and researchers in reliability and maintenance. This book introduces basic stochastic processes from the viewpoint of elementary mathematics, mainly based on the books Stochastic Processes [5], Applied Stochastic System Modeling [6], and Introduction to Stochastic Processes [7]. Many reliability examples are cited in this book to explain concretely stochastic processes. They are quoted mainly from the books [1–4]. Furthermore, several interesting reliability examples appeared in two famous and v

vi

Preface

classical books [8, 9]. However, they have become old and their important contributions are going to be forgotten. This book is aimed at quoting possible examples from the two books. This book is composed of eight chapters: Chapter 1 is devoted to the introduction to stochastic processes and reliability theory. Chapters 2 –5 are devoted to standard and fundamental stochastic processes such as Poisson processes, renewal processes, Markov chains, Markov processes, and Markov renewal processes. These four chapters are written mainly based on the books [5–7] and are introduced from the viewpoint of elementary mathematics without using hard proof. Chapter 6 is devoted to cumulative processes that are related greatly to shock and damage models in one field of reliability. Chapter 7 introduces simply Brownian motion and Lévy processes, that might be omitted at the first reading. To understand and explain stochastic processes easily and actually, a lot of examples are quoted from reliability models through each chapter. As final examples of reliability models, Chapter 8 takes up redundant systems and shows systematically how to use well the tools of stochastic processes to analyze them. The reader could learn both stochastic process and reliability theory from this book at the same time. I wish to thank most kindly Professor E. Çinlar, Professor S. M. Ross, and Professor S. Osaki for referring to the good books written by them and the other books in references. I wish to express my special thanks to Professor Fumio Ohi and Professor Mingchih Chen for their careful reviews of this book, and to Dr. Kodo Ito, Dr. Satoshi Mizutani, Dr. Sayori Maeji, and Mr. Xufeng Zhao for their support in writing and typing this book. Finally, I would like to express my sincere appreciation to Professor Hoang Pham, Rutgers University, and Editor Anthony Doyle, Springer-Verlag, London, for providing the opportunity for me to write this book. Toyota, April 2010

Toshio Nakagawa

References 1. 2. 3. 4. 5. 6. 7. 8. 9.

Barlow RE, Proschan F (1965) Mathematical theory of reliability. Wiley, NewYork Nakagawa T (2005) Maintenance theory of reliability. Springer, London Nakagawa T (2007) Shock and damage models in reliability theory. Springer, London Nakagawa T (2008) Advanced reliability models and maintenance policies. Springer, London Ross SM (1983) Stochastic processes. Wiley, New York Osaki S (1992) Applied stochastic system modeling. Springer, Berlin Çinlar E (1975) Introduction to stochastic processes. Prentice-Hall, Englewood Cliffs, NJ Takács L (1960) Stochastic processes. Wiley, New York Cox DR (1962) Renewal theory. Methuen, London

Contents

1

Introduction . . . . . . . . . 1.1 Reliability Models . 1.1.1 Redundancy . 1.1.2 Maintenance . 1.2 Stochastic Processes. 1.3 Further Studies . . . . 1.4 Problems 1 . . . . . . . References . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

1 2 2 2 2 5 5 6

2

Poisson Processes . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Exponential Distribution. . . . . . . . . . . . . . . . 2.1.1 Properties of Exponential Distribution 2.1.2 Poisson and Gamma Distributions . . . 2.2 Poisson Process . . . . . . . . . . . . . . . . . . . . . 2.3 Nonhomogeneous Poisson Process . . . . . . . . 2.4 Applications to Reliability Models . . . . . . . . 2.4.1 Replacement at the Nth Failure. . . . . . 2.4.2 Software Reliability Model . . . . . . . . 2.5 Compound Poisson Process . . . . . . . . . . . . . 2.6 Problems 2 . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

7 8 9 14 18 27 34 34 36 36 42 45

3

Renewal Processes . . . . . . . . . . . . . . . . . . . . . 3.1 Definition of Renewal Process . . . . . . . . . 3.2 Renewal Function . . . . . . . . . . . . . . . . . . 3.2.1 Age and Residual Life Distributions 3.2.2 Expected Number of Failures . . . . 3.2.3 Computation of Renewal Function . 3.3 Age Replacement Policies . . . . . . . . . . . . 3.3.1 Renewal Equation . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

47 48 50 58 61 64 66 66

. . . . . . . .

. . . . . . . .

vii

viii

Contents

3.3.2 Optimum Replacement Policies . . . . . Alternating Renewal Process . . . . . . . . . . . . 3.4.1 Ordinary Alternating Renewal Process 3.4.2 Interval Reliability . . . . . . . . . . . . . . 3.4.3 Off Distribution . . . . . . . . . . . . . . . . 3.4.4 Terminating Renewal Process . . . . . . 3.5 Modified Renewal Processes. . . . . . . . . . . . . 3.5.1 Geometric Renewal Process . . . . . . . 3.5.2 Discrete Renewal Process . . . . . . . . . 3.6 Problems 3 . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

69 73 73 76 79 82 84 84 86 91 93

4

Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Discrete-Time Markov Chain . . . . . . . . . . . . . . . . . . . 4.1.1 Transition Probabilities . . . . . . . . . . . . . . . . . . 4.1.2 Classification of States . . . . . . . . . . . . . . . . . . 4.1.3 Limiting Probabilities . . . . . . . . . . . . . . . . . . . 4.1.4 Absorbing Markov Chain . . . . . . . . . . . . . . . . 4.2 Continuous-Time Markov Chain . . . . . . . . . . . . . . . . 4.2.1 Transition Probabilities . . . . . . . . . . . . . . . . . . 4.2.2 Pure Birth Process and Birth and Death Process . 4.2.3 Limiting Probabilities . . . . . . . . . . . . . . . . . . . 4.3 Problems 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

95 97 97 101 103 108 111 111 112 119 120 121

5

Semi-Markov and Markov Renewal Processes . . . . . . . . 5.1 Markov Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Embedded Markov Chain . . . . . . . . . . . . . . . . . . . . 5.2.1 Transition Probabilities . . . . . . . . . . . . . . . . . 5.2.2 First-Passage Distributions . . . . . . . . . . . . . . . 5.2.3 Expected Numbers of Visits to States . . . . . . . 5.2.4 Optimization Problems . . . . . . . . . . . . . . . . . 5.3 Markov Renewal Process with Nonregeneration Points 5.3.1 Type 1 Markov Renewal Process . . . . . . . . . . 5.3.2 Type 2 Markov Renewal Process . . . . . . . . . . 5.4 Problems 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

123 124 132 133 136 138 140 140 141 143 147 148

6

Cumulative Processes . . . . . . . . . 6.1 Standard Cumulative Process . 6.2 Independent Process . . . . . . . 6.3 Modified Damage Models . . . 6.3.1 Imperfect Shock . . . . . 6.3.2 Random Failure Level .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

149 150 155 162 162 164

3.4

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . . . . . . .

. . . . . .

. . . . . . . . . . .

. . . . . .

. . . . . . . . . . .

. . . . . .

. . . . . . . . . . .

. . . . . .

. . . . . . . . . . .

. . . . . .

. . . . . . . . . . .

. . . . . .

Contents

6.3.3 Damage with Annealing. . . . . 6.3.4 Increasing Damage with Time 6.4 Replacement Models . . . . . . . . . . . . 6.4.1 Three Replacement Policies . 6.4.2 Optimum Policies . . . . . . . . 6.5 Problems 6 . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . .

ix

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

166 169 170 171 173 179 180

7

Brownian Motion and Lévy Processes. . . . . . . . . . . . . . 7.1 Brownian Motion and Wiener Processes. . . . . . . . . . 7.2 Three Replacements of Cumulative Damage Models 7.2.1 Three Damage Models . . . . . . . . . . . . . . . . 7.2.2 Numerical Examples . . . . . . . . . . . . . . . . . 7.3 Lévy Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Problems 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

183 183 187 187 189 191 196 196

8

Redundant Systems . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 One-Unit System . . . . . . . . . . . . . . . . . . . . . . . 8.1.1 Poisson Process . . . . . . . . . . . . . . . . . . . 8.1.2 Nonhomogeneous Poisson Process . . . . . . 8.1.3 Renewal Process . . . . . . . . . . . . . . . . . . . 8.1.4 Alternating Renewal Process . . . . . . . . . . 8.2 Two-Unit Standby System . . . . . . . . . . . . . . . . . 8.2.1 Exponential Failure and Repair Times . . . 8.2.2 General Failure and Repair Times . . . . . . 8.3 Standby System with Spare Units . . . . . . . . . . . . 8.3.1 First-Passage Time to Systems Failure. . . . 8.3.2 Expected Number of Failed Units . . . . . . . 8.3.3 Expected Cost and Optimization Problems . 8.4 Problems 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

199 200 200 200 201 202 203 204 205 211 211 213 215 216 217

Appendix A: Laplace Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . .

219

Appendix B: Answers to Selected Problems . . . . . . . . . . . . . . . . . . . .

225

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

249

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

Chapter 1

Introduction

All things from near oneself to space form stochastic processes whether large or small because their real lives are limited. Of course, each lifetime of human being forms a stochastic process [1], however, such process could not be analyzed exactly because given conditions and constrains would be too many and much complex at the present, as shown in a toy of life games. Most reliability models are much simpler and more clear than human being, and much more, they have to be analyzed practically and be of great use in actual fields. The importance of reliability will be greatly enhanced by environmental considerations, and moreover, for the protection of natural resources and the earth. Reliability techniques have to be developed and expanded as objective models become more complex and large-scale. They also will be applied not only to daily life, but also to a variety of other fields because consumers, workers, and managers must make, buy, sell, and use, and handle articles and products with a sense of safety and security. A wide knowledge of probabilities, statistics, and stochastic processes are needed for learning reliability theory mathematically. Stochastic processes have been applied in many fields such as engineering, physics, biology, electronics, management science, economics, and psychology, and in operations research, they have spread widely over queueing, finance, and insurance. Especially, in reliability theory, stochastic processes are the most powerful mathematical tools for analyzing reliability models. We aim to write this book about stochastic processes from viewpoints of reliability theory and applications: This book introduces basic stochastic processes such as Poisson processes, renewal processes, Markov chains and processes, and furthermore, introduces cumulative processes, Brownian motion, and Lévy processes. A lot of examples cited from reliability models demonstrate how to apply stochastic processes correctly.

T. Nakagawa, Stochastic Processes, Springer Series in Reliability Engineering, DOI: 10.1007/978-0-85729-274-2_1, Ó Springer-Verlag London Limited 2011

1

2

1 Introduction

1.1 Reliability Models 1.1.1 Redundancy High system reliability can be achieved by redundancy and maintenance. A classical problem is to determine how reliability can be improved by using redundant units. Various expressions of reliability quantities of many redundant systems were summarized [2, 3]. A two-unit system is the most fundamental redundant model, and its reliability measures and optimum maintenance policies were surveyed deeply [4]. For systems with redundant or spare units, we have to determine how many number of units should be provided initially. It is well-known as fault tolerance [5] in which systems continue to function correctly in the presence of hardware failures and software errors. Redundancy techniques of systems for improving reliability and achieving fault tolerance are classified commonly in the following forms [5, 6]: hardware redundancy, software redundancy, information redundancy, and time redundancy.

1.1.2 Maintenance Maintenance of units after failure may be costly, and sometimes requires a long time. Thus, the most important problem is to determine when and how to maintain preventively units before failure. However, it is not wise to maintain units with unnecessary frequency. From such viewpoints, the commonly considered maintenance policies are preventive replacement for units without repair, and preventive maintenance for units with repair on a specific schedule. Classifying into three large groups, planned maintenance of units are done at a certain age, a periodic time, or a specified number of occurrences. Consequently, the object of optimization problems is to determine the frequency and timing of several kinds of maintenance and replacement policies in accordance with maintenance costs and effects, and system structures and circumstances.

1.2 Stochastic Processes All reliability models consisting of random time factors form stochastic processes. We briefly explain stochastic processes as examples of reliability systems with maintenance: Consider the simplest one-unit system with repair or replacement whose time is negligible, i.e., the unit is operating and is repaired or replaced at failure, where the time required for repair and replacement is negligible. When the repair or replacement is completed, the unit becomes as good as new and begins to operate.

1.2 Stochastic Processes

3

If failures occur exponentially, i.e., the unit fails constantly during the time interval ðt; t þ dt irrespective of time t, the system forms a Poisson process. Roughly speaking, failures occur randomly in ½t; t þ dt with probability kdt for constant k [ 0; and interarrival times between failures have an exponential distribution ð1  ekt Þ: Then, it is said that failures occur in a Poisson process with rate k: A Poisson process is the simplest and the most fundamental stochastic process. In Chap. 2, we define a Poisson process theoretically and explain various properties of an exponential distribution with memoryless property. As examples of a Poisson process, we consider a parallel system, a spare part problem, an inspection policy, a backward time problem, and so on. Furthermore, we introduce a nonhomogeneous Poisson process in which failures occur in ½t; t þ dt with probability kðtÞdt; and a compound Poisson process in which an amount of some quantities such as customers, damage, costs and profits occurs in a Poisson process. To apply two processes to reliability models, we take up periodic replacements, and shock and damage models. If failures occur generously as an extended process of a Poisson one, the system forms a renewal process, i.e., a renewal theory arises from the study of selfrenewing aggregates. The process plays a major role in the analysis of probability models with sums of independent nonnegative random variables, and is more useful tool in reliability theory because many reliability models have a renewal property by replacement and maintenance. In Chap. 3, we define a renewal process theoretically, and investigate and give a renewal function in theorem forms. The most important matters in a renewal process are how to compute renewal functions and to form renewal equations. Optimum age and periodic replacement policies are shown as practical examples. Furthermore, we introduce an alternating renewal process where the repair or replacement time needs a nonnegligible time for a one-unit system, and a discrete renewal process with only discrete times. As examples of two processes, we give a one-unit system with repair and a discrete age replacement, and obtain their reliability quantities using a renewal theory. We have considered a Poisson process with only one state in which the unit is operating. When we consider a one-unit system whose repair time is nonnegligible, the process forms an alternating renewal process with two states that repeats up and down alternately. In addition, when we consider two types of repair maintenance according to minor or major failures of the unit, the process forms a stochastic process with three states. In such ways, if the process transits among many states and has a Markov property in which the future behavior depends only on the present state and is independent of its past history, it forms a Markov process. If the duration times of states are multiples of time unit such as day, week, month, year, and a certain number of occurrences, then the process is called a discrete-time Markov chain, and if the duration time are distributed exponentially, it is called a continuous-time Markov chain. In Chap. 4, we show how to compute transition probabilities of a Markov chain and to obtain its several useful quantities from them. As examples of reliability models, we take up a one-unit system with repair

4

1 Introduction

and a two-unit parallel system, and derive their reliability measures by using the techniques of a Markov chain. We consider a one-unit system with two types of repair maintenance. If neither up and down times are exponential, the process does not have the Markov property at any time except only transition times which are called regeneration points. The process forms a Markov renewal process or a semi-Markov process that is a marriage of renewal processes and Markov chains. If we are interested mainly in the number of visits to states, the process is called a Markov renewal process, and if we are interested in what state the process is at time t, it is called a semi-Markov process. However, there is not so much difference between two given names. In Chap. 5, we derive first-passage distributions, transition probabilities, and renewal functions by forming renewal equations. As an example of a standby system with repair, we obtain a mean time to failure, transition probabilities, and expected numbers of visits to states. Markov renewal processes give powerful plays for analyzing reliability systems because they include renewal and Markov processes. Learning the techniques of Markov renewal processes is positively necessary for analyzing reliability models. Naturally, such processes are not all-round. For example, we consider a two-unit parallel system with one repair person when both failure and repair times are not exponential that seems to be a simpler system. However, we cannot analyze the system because there is no generation point, i.e., the process has no Markov property at any time. Next, consider a two-unit standby system with one repair person. Then, the process has two regeneration points only at times when one unit fails and another is in standby, and the repair of one unit is completed and another waits for repair [4]. From the above viewpoints, we introduce two types of Markov renewal processes with nonregeneration points and analyze an n-unit parallel and a two-unit standby systems with repair. Suppose that shocks occur in a stochastic process and each amount of damage due to shocks is additive. This process forms a cumulative process that is also called a jump process [7] or doubly stochastic process [8]. The analysis of such processes are more difficult than the already defined stochastic processes because the process is made up of two dependent stochastic processes. In Chap. 6, we define a cumulative process and investigate its properties. There are many examples of shock models, cumulative damage models, and independent damage models, and furthermore, their optimum maintenance policies. State space of the above stochastic processes may be finite because they are usually defined as the number of operating or failed units and the degree of wear for most reliability models. As far as reliability applications are concerned, we consider the processes with only a finite number of states. A continuous-time Markov process with continuous state space forms a Brownian motion, a Wiener process, and a Lévy process. This is now one of the most useful stochastic processes in applied sciences such as physics, economics, communication theory, biology, management science, and mathematical statistics. However, such processes cannot be much applicable to reliability models because of continuous state space. In Chap. 7, we make only the definitions of a Brownian motion and a Lévy

1.2 Stochastic Processes

5

process. As examples, we give continuous wear models in which wear increases continuously with time and consider their replacement policies. Finally, in Chap. 8, as examples of some redundant systems, we show how to analyze them by ready and handy tools of the stochastic processes. This would be greatly helpful for understanding the stochastic processes and learning reliability theory.

1.3 Further Studies This book is written mainly based on the books [9–11] for learning basic stochastic processes. In similar contents of such books, we recommend to read the books [12, 13] for a deeper understanding. There were contained many examples of stochastic processes in various fields of applications [14, 15] and in reliability systems [16, 17]. The names of stochastic processes such as stationary process, Gaussian process, martingales, branching process, and so on, appeared [18]. For more applications of stochastic processes, there were [19–21] for queuing theory, [22] for insurance and finance, and [20] for computer applications. It would be impossible to form stochastic models and to analyze them for more complex systems with too many constrains. Then, simulation would be a useful tool for analyzing such systems. We do not touch on the simulation at all in this book and recommend to master its techniques by [21, 23, 24]. We introduce simply the Brownian motion and Lévy process because these processes are not very applicable to reliability models and are more difficult to understand over our mathematical level. For further studies of such processes, refer to [25–27]. This book would be suited well to learn elementary stochastic processes for students with a major of reliability, and researchers and workers engaged in reliability engineering and management, with reference to books [6, 28–30]. As other text books, refer to recent published books [31–33]. For more further and recent studies of reliability, edited books and hand books [34–36] would be advisable to look through. In Appendix A and B, we present Laplace and LaplaceStieltjes transforms and make answers to selected problems.

1.4 Problems 1 1.1 Search for examples of stochastic processes on your daily life, or read the books [14–17] and give some examples. 1.2 Examine why there is no generation point for a two-unit parallel system with one repair person where the failure and repair times are not exponential [4]. What about the same system with two repair persons?

6

1 Introduction

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24.

25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36.

Klein JP, Moeschberger ML (2003) Survival analysis. Springer, New York Ushakov IA (1994) Handbook of reliability engineering. Wiley, New York Birolini A (1999) Reliability engineering theory and practice. Springer, New York Nakagawa T (2002) Two-unit redundant models. In: Osaki S (eds) Stochastic models in reliability and maintenance. Springer, Berlin, pp 165–185 Lala PK (1985) Fault tolerant and fault testable hardware design. Prentice-Hall, London Nakagawa T (2008) Advanced reliability models and maintenance policies. Springer, London Abdel-Hameed M (1984) Life distribution properties of devices subject to a pure jump damage process. J Appl Probab 21:816–825 Grandell J (1976) Doubly stochastic poisson process. Lecture notes in mathematics 529. Springer, New York Ross SM (1983) Stochastic processes. Wiley, New York Osaki S (1992) Applied stochastic system modeling. Springer, Berlin Çinlar E (1975) Introduction to stochastic processes. Prentice-Hall, Englewood Cliffs Karlin S, Taylor HM (1975) A first course in stochastic processes. Academic Press, New York Tijms HC (2003) A first course in stochastic models. Wiley, Chichester Ross SM (2000) Introduction to probability models. Academic Press, San Diego Bhat UN, Miller GK (2002) Elements of applied stochastic processes. Wiley, Hoboken, NJ Beichelt FE, Fatti LP (2002) Stochastic processes and their applications. CRC Press, Boca Raton Beichelt FE (2006) Stochastic processes in science, engineering and finance. Chapman & Hall, Boca Raton Prabhu NU (2007) Stochastic processes. World Scientific, Singapore Bhat UN, Basawa IV (1992) Queeing and related models. Oxford University Press, Oxford Trivedi KS (2002) Probabilty and statistics with reliability, queueing and computer science applications. Wiley, New York Stewart WJ (2009) Probability, Markov chains, queues, and simulation. Princeton University Press, Princeton Rolski T, Schmidli H, Schmidt V, Teugels J (1999) Stochastic processes for insurance and finance. Wiley, Chichester Ross SM (1997) Simulation. Academic Press, New York Papadopoulos C, Limnious N (2002) Quick Monte Carlo methods in stochastic systems and reliability. In: Osaki S (eds) Stochastic models in reliability and maintenance. Springer, Berlin, pp 307–333 Karatzas I, Shreve S (1991) Brownian motion and stochastic calculus. Springer, New York Bertoin J (1996) Lévy processes. Cambridge University Press, Cambridge Barndorff-Nielsen OE, Mikosch T, Resnick SI (eds) (2001) Lévy processes: theory and applications. Birkhävser, Boston Barlow RE, Proschan F (1965) Mathematical theory of reliability. Wiley, New York Nakagawa T (2005) Maintenance theory of reliability. Springer, London Nakagawa T (2007) Shock and damage models in reliability theory. Springer, London Dhillon BS (2002) Applied reliability and quality. Springer, London O’Connor PDT (2002) Practical reliability engineering. Wiley, Chichester Rausand M, H/yland A (2004) System reliability theory. Wiley, Hoboken Osaki S (eds) (2002) Stochastic models in reliability and maintenance. Springer, Berlin Pham H (eds) (2003) Handbook of reliability engineering. Springer, London Misra KB (2008) Handbook of performability engineering. Springer, London

Chapter 2

Poisson Processes

It is well-known that most units operating in a useful life period, and complex systems that consist of many kinds of components, fail normally due to random causes independently over the time interval. Then, it is said in technical terms of stochastic processes, that failures occur in a Poisson process that counts the number of failures through time. This is a natural modeling tool in reliability problems. Some reliability measures such as MTTF (Mean Time To Failure), availability, and failure rate are estimated statistically from life data and are in practical use under such modelings without much theoretical arguments. Furthermore, because a Poisson process has stationary and independent properties, it is much convenient for formulating stochastic models in mathematical reliability theory. A Poisson process is the simplest stochastic process that arises in many applications for arrival processes. It is of great importance to know the properties of a Poisson process and to learn how to apply the process to reliability models in the beginning: In Sect. 2.1, we investigate properties of an exponential distribution, because most units are assumed practically to have its distribution as their failure times. Then, it is shown that the number of failures has a Poisson distribution when the failure time has an exponential distribution. As applied examples, we discuss about the number of spare units for constructing parallel and standby systems that should be previously prepared. In Sect. 2.2, a Poisson process is defined theoretically, based on such above facts, and its interarrival times are shown to have an exponential distribution. Furthermore, when the failure was detected at time t; its failure occurs uniformly over ½0; t: As such applied examples, we take up an inspection and two backup models, and derive their optimum policies. It has been assumed often in maintenance models that a unit undergoes minimal repair at failures. This means that the failure rate is not changed by any maintenance, and forms a nonhomogeneous Poisson process. Underlying the process, failures occur successively, depending on the operating time t: In Sect. 2.3, we summarize the properties of the process. In Sect. 2.4, we apply these results to the replacement model where the unit is replaced at the Nth failure and the software

T. Nakagawa, Stochastic Processes, Springer Series in Reliability Engineering, DOI: 10.1007/978-0-85729-274-2_2, Ó Springer-Verlag London Limited 2011

7

8

2 Poisson Processes

reliability model, and investigate its properties. Finally, in Sect. 2.5, we consider a cumulative damage model where shocks occur in a Poisson process and the total damage due to shocks are additive. This is called a compound Poisson process. We survey probabilistic properties of the process and apply damage models with a failure level K where the unit fails when the total damage exceeds K: For a more deep study of Poisson processes, refer to the book [1].

2.1 Exponential Distribution Suppose that a unit begins to operate at time 0 and is replaced with a new identical one immediately at each failure, where the time required for replacement is negligible. The successive operating units have failure P times Xk ðk ¼ 1; 2; . . .Þ from the beginning of their operation to failure and Sn  nk¼1 Xk in Fig. 2.1, where the horizontal axis represents the time-axis t from t ¼ 0 to the right direction in all figures of this book. It is assumed that Xk are independent and identically distributed random variables, and have an identical exponential distribution FðtÞ  PrfXk  tg ¼ 1  ekt ;

ð2:1Þ

and its density function f ðtÞ 

dFðtÞ ¼ F 0 ðtÞ ¼ kekt dt

for k [ 0 and any k; that decreases strictly from k to 0. The mean and variance are EfXk g ¼

Z1

1 tkekt dt ¼ ; k

0

VfXk g ¼

Z1

ð2:2Þ 1 t2 kekt dt  ðEfXk gÞ2 ¼ 2 : k

0

Then, it is said that the unit has a failure time X; omitting subscript k; and fails according to an exponential distribution ð1  ekt Þ with parameter k or MTTF 1=k: Furthermore, from Appendix A, the LS (Laplace-Stieltjes) transform of an exponential distribution is

Fig. 2.1 Process of failures

2.1 Exponential Distribution



9

sX

F ðsÞ  Efe

g¼

Z1 e

st

dFðtÞ ¼

0

Z1

est kekt dt ¼

k sþk

ð2:3Þ

0

for any ReðsÞ [ 0; where note that we omit ReðsÞ [ 0 throughout this book. From the assumptions that random variables are independent and identical, n EfX1 þ X2 þ    þ Xn g ¼ nEfXg ¼ ; k n VfX1 þ X2 þ    þ Xn g ¼ nVfXg ¼ 2 ; k  n   k sðX1 þX2 þþXn Þ sX n g ¼ Efe g ¼ : Efe sþk

ð2:4Þ

If two random variables X1 and X2 are independent and have the respective exponential distributions with mean 1=k1 and 1=k2 ; then PrfX1  X2 g ¼

Z1

ek2 t k1 ek1 t dt ¼

k1 : k1 þ k2

0

In general, if k random variables Xk are independent and have an exponential distribution with mean 1=kk ðk ¼ 1; 2; . . .Þ; and Xm ¼ minfX1 ; X2 ; . . .; Xn g; then PrfX1 ¼ Xm g ¼

Z1

eðk2 þk3 þþkn Þt k1 ek1 t dt ¼

k1 : k1 þ k2 þ    þ kn

0

2.1.1 Properties of Exponential Distribution We investigate and summarize various properties of an exponential distribution ð1  ekt Þ appearing frequently in reliability theory: (1) From the MTTF or the mean life time of the unit 1=k; Fð1=kÞ ¼ 1  e1  0:632; i.e., 63.2% of units have failed at the time of MTTF (Example 2.13). (2) The reliability function RðtÞ  FðtÞ ¼ 1  FðtÞ that is the probability that the unit is operating at time t without failure is RðtÞ ¼ ekt : Thus, the failure rate or hazard rate is [2, p. 5, 3, p. 22]

10

2 Poisson Processes

hðtÞ 

f ðtÞ kekt ¼ kt ¼ k; e RðtÞ

irrespective of time t: Thus, if the failure rate would be estimated from actual life data, then the MTTF would be obtained by taking the reciprocal of k: For example, when k ¼ 104 (1/h), the MTTF is 104 h and vice versa. Therefore, because hðtÞDt  Prft\X  t þ DtjX [ tg means the probability that the unit with age t will fail in ðt; t þ Dt for small Dt; the probability that a failure occurs in a small interval ðt; t þ Dt is constant for any time t: That is, the failure rate of an exponential distribution has the property of random failures occurring constantly in a small interval time t; irrespective of ages or elapsed operating times. In general, the failure rate curve, i.e., the bathtub curve [4, p. 188] (Fig. 2.5) is divided into three periods; (1) the initial failure period in which hðtÞ is DFR (Decreasing Failure Rate), (2) the random failure rate period in which hðtÞ is CFR (Constant Failure Rate), and (3) the wearout failure period in which hðtÞ is IFR (Increasing Failure Rate). In this case, an exponential distribution has the properties of DFR and IFR, and shows the failure phenomenon observed in a random failure period. Thus, when units have an exponential failure distribution, there is no necessity for making any maintenance of them theoretically. For such reasons, the exponential distribution might be of limited applications. However, the exponential distribution plays a fitting role for complex systems that consist of many kinds of components, each of which has a variety pattern of malfunction and replacement and makes the failure pattern of the system as a whole [2, p. 18]. It is well-known that the exponential distribution is fitted well to life data of electric and electronic components. (3) Using the Taylor expansion of an exponential function, ðkhÞ2 ðkhÞ3 þ     ¼ kh þ oðhÞ; 2! 3! where oðhÞ is an infinite decimal higher than h; i.e., oðhÞ=h ! 0 as h ! 0: Thus, we have the inequalities ðkhÞ2 \1  ekh \kh kh  2 1  ekh ¼ kh 

for a small kh [ 0: These inequalities are very useful for computing equations numerically, involving exponential functions because kt is very small in actual reliability models, as shown in Table 2.1. Table 2.1 Comparative table of 1  ekh and approximation kh  ðkhÞ2 =2 for kh kh 1  ekh kh  ðkhÞ2 =2 1 101 102 103 104

0.6321205 0:951625 101 0:99501 102 0:99949 103 0:9999 104

0.5 0:95 101 0:995 102 0:9995 103 0:9999 104

2.1 Exponential Distribution

11

Example 2.1 (MTTF of exponential distribution) When the failure distribution is an exponential distribution ð1  ekt Þ; we ask the following questions: (a) How much is the failure rate k and the MTTF 1=k when RðtÞ ¼ 0:99 at t ¼ 103 h? (b) What fraction time t of the MTTF are to satisfy that FðtÞ ¼ 101 ; 102 ; 103 ; and 104 ? The solution of (a) is easily given as 3

ek 10 ¼ 0:99; i.e., k ¼ 103 log 0:99  1:005 105 (1/h); 1 MTTF ¼ ¼ 99;499 (h): k We can compute approximately, k 103 ¼ 1  0:99 ¼ 0:01; i.e., e k ¼ 105 (1/h);

f ¼ 105 (h): M TTF

In this case, the difference between the exact and approximate values are very small. The solution of (b) is given by computing kt ¼

t t ¼ : 1=k MTTF

For example, 1  ekt ¼ 0:1; kt ¼  log 0:9 ¼ 0:10536: Approximately, from (3), e ¼ 0:1 ¼ 1=10: The time t that satisfies FðtÞ ¼ 0:1 is also called a B10 life. kt e for FðtÞ ¼ 101 ; 102 ; 103 ; Table 2.2 presents kt and their approximate values kt 4 and 10 : This table indicates that when FðtÞ ¼ 10n ðn ¼ 1; 2; . . .Þ; the fraction time t of the MTTF is approximately 10n ¼ 1=10n ; and its value becomes very good as n becomes large. From such results, we can easily estimate the fraction time of the MTTF for a small FðtÞ by setting t=MTTF ¼ FðtÞ: Example 2.2 (Preventive maintenance) [3, p. 8] Consider operating units such as scales, computers, and production systems where its preventive maintenance is

e for FðtÞ Table 2.2 Comparative table of kt and approximation kt FðtÞ kt

e kt

101 102 103 104

101 102 103 104

1:0536 101 1:0050 102 1:0005 103 1:0000 104

12

2 Poisson Processes

done only at time T ð0\T  1Þ and its failure is detected. It is supposed that the unit has a failure distribution FðtÞ. We get some earning per unit of time and does not have any earning during the interval if it fails before time T: The average time during ½0; T in which we get some earning is lðTÞ ¼ 0 FðTÞ þ TFðTÞ ¼ TFðTÞ;

ð2:5Þ

and lð0Þ ¼ lð1Þ ¼ 0: Differentiating lðTÞ with respect to T and setting it equal to zero, FðTÞ  Tf ðTÞ ¼ 0;

i.e.;

hðTÞ ¼

f ðTÞ 1 ¼ : FðTÞ T

ð2:6Þ

Thus, an optimum time T  that maximizes lðTÞ in (2.5) is given by a unique solution of (2.6), when FðtÞ is IFR. In particular, when FðtÞ ¼ 1  ekt ; T ¼ 1=k; i.e., we should do the preventive maintenance at the MTTF. Example 2.3 (Discount cost) [5, p. 82] When we consider the total expected cost of an operating unit for an infinite time span, we should evaluate the present values of all replacement costs by using an appropriate discount rate [3, p. 78]. Suppose that a continuous discount with rate a ð0\a\1Þ is used for the cost incurred at replacement time. That is, the present value of replacement cost c at at at time 0. Then, because the total replacement cost is time P1 t is ce c n¼1 exp½aðX1 þ X2 þ    þ Xn Þ; the expected cost is Cc

1 X

Efexp½aðX1 þ X2 þ    þ Xn Þg ¼ c

n¼1

1  X

n EfeaX g :

n¼1

Thus, when X is distributed exponentially, from (2.4), n n 1 Z 1 1  X X k ck at kt e ke dt ¼ c ¼ : C¼c a þ k a 0 n¼1 n¼1 For example, if the mean failure time is 1,000 h, the replacement cost is 20 dollars, and the interest rate is 5 % per year, i.e., k ¼ 1=1; 000; c ¼ 20; and a ¼ 0:05=ð365 24Þ; then C ¼ 20 365 24=ð1000 0:05Þ ¼ 3; 504 dollars. In general, for any negative function gðtÞ for t 0 (Problem 2.1), ( ) Z1 1 X gðSn Þ ¼ k gðtÞdt: E n¼1

0

In this example, gðtÞ ¼ ceat : (4) From the assumption that X is a random variable with an exponential distribution ð1  ekt Þ; the conditional survival probability that the unit does not fail in a finite interval time t; given that it is operating at time u ðu 0Þ is [4, p. 74]

2.1 Exponential Distribution

13

PrfX [ u þ tjX [ ug ¼

PrfX [ u þ tg ekðuþtÞ ¼ ku e PrfX [ ug

¼ ekt ¼ PrfX [ tg; that is independent of any time u; i.e., for all t; u 0; PrfX [ u þ tg ¼ PrfX [ ug PrfX [ tg:

ð2:7Þ

The condition that the unit is operating at time u; i.e., its age is u; has no effect on the next operating time t; i.e., its residual or remaining time. This is called the memoryless property or lack of memory property. Conversely, the only continuous and positive function that satisfies (2.7) is an exponential function PrfX [ tg ¼ ekt (Problem 2.2). That is, only the exponential distribution has the memoryless property. For such reasons, as one of failure distributions, an exponential distribution comes to be frequently used for forming simpler and analyzing easier stochastic models. Example 2.4 (Parallel system) Consider a parallel system that consists of n identical units and fails when all units have failed, i.e., at least one of n units is operating, the system is also operating [6, p. 8]. Each unit has an independent and identical failure distribution ð1  ekt Þ: Because the system with n units has a failure distribution ð1  ekt Þn ; its MTTF is (Problems 2.3–2.8) Z1 Z1 n   1 1  xn 1X 1 kt n 1  ð1  e Þ dt ¼ dx ¼ : ð2:8Þ 1x k k k¼1 k 0

0

On the other hand, let XðkÞ ðk ¼ 1; 2; . . .; nÞ be the failure time from the ðk  1Þth to kth unit of n units. The failure time of the system is given by Xð1Þ þ   Xð2Þ þ    XðnÞ : Then, because XðkÞ has an exponential distribution 1  ekðnkþ1Þt with mean 1=½ðn  k þ 1Þk from the memoryless property, the MTTF is n n 1X 1 1X 1 ¼ ; EfXð1Þ þ Xð2Þ þ    þ XðnÞ g ¼ k k¼1 n  k þ 1 k k¼1 k that agrees with (2.8), and it is called a harmonic series (Problem 2.8). We have the inequalities [7] n 1 X 1 log n þ \ \ log n þ 1 ðn ¼ 2; 3; . . .Þ; n k¼1 k and

n X 1 k¼1

k

ð2:9Þ

 C þ log n;

where C is Euler’s constant and C  0:577215. . .: From such results, the MTTF of a parallel system increases with n very slowly and logarithmically to infinity, and is given approximately by (2.9) for large n (Problem 2.5).

14

2 Poisson Processes

Next, because the reliability function is  n Rn ðtÞ ¼ 1  1  ekt ;

ð2:10Þ

its failure density function is  n1 fn ðtÞ ¼ R0n ðtÞ ¼ nkekt 1  ekt : Hence,  n n1  k Rn ðtÞ 1 1  1  ekt 1 X ¼ ¼ 1  ekt ; n1 fn ðtÞ nk ekt ð1  ekt Þ nk k¼0 that decreases with t: Thus, the failure rate hn ðtÞ ¼

fn ðtÞ nk ¼ Pn1 kt Þk Rn ðtÞ k¼0 ð1  e

ð2:11Þ

increases strictly with t from 0 to k ðn 2Þ that is the failure rate of each unit. Furthermore, because 1 hnþ1 ðtÞ



n1 h X n  k i 1 1 1  ekt  1  ekt ¼ [ 0; hn ðtÞ nðn þ 1Þk k¼0

the failure rate hn ðtÞ decreases strictly with n ðn 1Þ from k to 0 for a given t [ 0:

2.1.2 Poisson and Gamma Distributions We are often interested in the number of failures and replacements of units that have occurred in the interval ½0; t: For this purpose, we introduce the total sum of random variables Sn  X1 þ X2 þ    þ Xn ðn ¼ 1; 2; . . .Þ that represents the time of the nth failure (Fig. 2.1) when FðtÞ  PrfXk  tg ¼ 1  ekt for all k 1: Then, we derive the distribution PrfSn  tg ¼

Zt 0

1 X kðkuÞn1 ku ðktÞk kt e du ¼ e ðn  1Þ! k! k¼n

¼1

n1 X ðktÞk k¼0

k!

ekt

ðn ¼ 1; 2; . . .Þ;

ð2:12Þ

that is called an Erlang distribution or a gamma distribution with order n: We prove (2.12) by the mathematical induction as follows: Using the total   sum of independent random variables with an exponential distribution 1  ekt ;

2.1 Exponential Distribution

15

PrfS2  tg ¼ PrfX1 þ X2  tg ¼

Zt h

i 1  ekðtuÞ keku du

0

¼ 1  ð1 þ ktÞe

kt

:

Assuming that when n ¼ k; PrfSk  tg ¼

Zt

kðkuÞk1 ku e du; ðk  1Þ!

0

we easily have Zt h ikðkuÞk1 1  ekðtuÞ PrfSkþ1  tg ¼ PrfSk þ Xkþ1  tg ¼ eku du ðk  1Þ! 0

¼

1 X ðktÞj

j!

j¼k

1 X ðktÞ kt ðktÞj kt e ¼ e ; k! j! j¼kþ1 k

ekt 

that concludes that (2.12) holds for all n 1: Equation (2.12) is alternatively proved by the technique of the LS transform in Appendix A: Recalling that the LS  transform of a random variable Xk with an exponential distribution 1  ekt is given in (2.3), the LS transform of Sn is  n k : EfesSn g ¼ EfesðX1 þX2 þþXn Þ g ¼ sþk On the other hand, the LS transform of (2.12) is Z1

est

kðktÞn1 kt e dt ¼ ðn  1Þ!



k sþk

n :

0

Therefore, because an original function is in one-to-one correspondence with its LS transform, it is shown that Sn has a gamma distribution in (2.12), and its mean and variance are also given in (2.4). A gamma distribution with order n represents the distribution of the nth failure of a unit with an exponential distribution. If a unit fails every at the nth minor fault, each of which occurs exponentially, its failure time has a gamma distribution. In other words, a standby redundant system (Example 2.6) with n units fail according to a gamma distribution with order n: A density function of Sn given in (2.12) is generalized as gðtÞ 

kðktÞa1 kt e CðaÞ

k; a [ 0;

ð2:13Þ

16

2 Poisson Processes

R1 where CðaÞ  0 xa1 ex dx for a [ 0; and is called a gamma function with order a: The mean, variance, and LS transform of a random variable X with a gamma density function in (2.13) are, respectively,  a a a k sX ; ð2:14Þ EfXg ¼ ; VfXg ¼ 2 ; Efe g ¼ k sþk k and the failure rate is hðtÞ ¼ R 1 t

ðktÞa1 ekt ðkuÞa1 eku du

:

ð2:15Þ

Thus, 1 ¼ hðtÞ

a1 Z1  Z1  a1 u x kðutÞ 1þ e du ¼ ekx dx; t t t

0

that implies that hðtÞ decreases from 1 to k for 0\a\1; is constant k for a ¼ 1; and increases from 0 to k for a [ 1: We define NðtÞ  maxn fSn  tg that represents the number of failures in ½0; t; where fNðtÞg is called a counting process, that is one of stochastic processes. If S1 ¼ X1 [ t; then NðtÞ ¼ 0; i.e., no failure occurs in ½0; t: Then, we have the relation fSn  tg , fNðtÞ ng;

ð2:16Þ

because if the nth failure occurs up to time t; then the number of failures in time t is greater than n; and conversely, if the number of failures in time t is greater than n; then the nth failure occurs up to time t (Fig. 2.1). Thus, from (2.12) and (2.16), PrfNðtÞ ng ¼ PrfSn  tg ¼

Zt 0

1 X kðkuÞn1 ku ðktÞk kt e du ¼ e : ðn  1Þ! k! k¼n

Hence, PrfNðtÞ ¼ ng ¼ PrfSn  tg  PrfSnþ1  tg ¼

ðktÞn kt e n!

ðn ¼ 0; 1; 2; . . .Þ;

ð2:17Þ

and its mean and variance are 1 X ðktÞn kt n e ¼ kt; n! n¼1 1 X ðktÞn kt VfNðtÞg ¼ e  ðktÞ2 ¼ kt: n2 n! n¼1

EfNðtÞg ¼

ð2:18Þ

2.1 Exponential Distribution

17

Equation (2.17) is called a Poisson distribution with mean kt that was originally derived as a limiting distribution of a binomial distribution by S.O. Poisson (1873) (Problem 2.9). The number of defects due to random causes normally in some time, area, volume, and so on, has a Poisson distribution. Example 2.5 (Poisson distribution) A unit fails exponentially at a failure rate k ¼ 103 (1/h). Then, calculate the probabilities that k ðk ¼ 0; 1; 2Þ failures occur in 500 h. In this case, because kt ¼ 103 500 ¼ 0:5; PrfNð500Þ ¼ 0g ¼ e0:5  0:607; PrfNð500Þ ¼ 1g ¼ 0:5e0:5  0:303; PrfNð500Þ ¼ 2g ¼

ð0:5Þ2 0:5  0:076: e 2!

Furthermore, the probability that the number of failures is equal to or more than 3 in 500 h is PrfNð500Þ 3g ¼ 1  PrfNð500Þ  2g   0:25 0:5 ¼ 1  1 þ 0:5 þ  0:0144: e 2 This probability is very small because the mean failure time 1=k ¼ 103 h is two times of t ¼ 500: Example 2.6 (Scheduling problem) Consider a standby system where an operating unit is replaced with one of identical spares at each failure and each unit has an   exponential failure distribution 1  ekt : We are interested in how many number of spare units should be provided initially in order to assure with probability a ð0\a\1Þ that the system will remain operating in time t [2, p. 49]. An optimum number n for fixed a and t is given by a unique minimum that satisfies PrfNðtÞ  ng ¼

n X ðktÞk k¼0

k!

ekt a:

Next, consider the scheduling problem where the system has to work for a job with working time S and its distribution WðtÞ  PrfS  tg [6, p. 84] and WðtÞ  1  WðtÞ: It is assumed that if the work of a job is accomplished when at least one unit is operating, it needs cost c2 ; if the work is not accomplished when all units have failed, it needs cost c1 with c1 [ c2 ; and the cost of a standby system with n units is c0 n: Then, the expected cost is CðnÞ ¼ c2 þ ðc1  c2 Þ

Z1 0

WðtÞdF ðnÞ ðtÞ þ c0 n ðn ¼ 0; 1; 2; . . .Þ:

ð2:19Þ

18

2 Poisson Processes

Table 2.3 Optimum numbers n of units for a standby system k=w c0 =ðc1  c2 Þ 0.5 0.3 0.1 0.05 0.01

1.0

2.0

5.0

0 1 3 4 6

0 1 3 5 9

0 0 3 7 16

In particular, when WðtÞ ¼ 1  ewt ; CðnÞ ¼ c2 þ ðc1  c2 Þ½F  ðwÞn þc0 n:

ð2:20Þ

To find an optimum number n that minimizes CðnÞ; from the inequality Cðn þ 1Þ  CðnÞ 0; ½F  ðwÞn ½1  F  ðwÞ 

c0 c 1  c2

ðn ¼ 0; 1; 2; . . .Þ;

ð2:21Þ

whose left-hand decreases strictly from 1  F  ðwÞ to 0. Thus, if F  ðwÞ\ðc1  c2  c0 Þ=ðc1  c2 Þ; then there exists a finite and unique minimum n ð1  n \1Þ that satisfies (2.21). If F  ðwÞ ðc1  c2  c0 Þ=ðc1  c2 Þ; then n ¼ 0; i.e., we should not provide any units for a job. In addition, when FðtÞ ¼ 1  ekt ; (2.21) is wkn ðw þ kÞ

nþ1



c0 c1  c2

ðn ¼ 0; 1; 2; . . .Þ:

ð2:22Þ

If w=ðw þ kÞ [ c0 =ðc1  c2 Þ; then a positive n exists. Table 2.3. presents the optimum n that decreases with c0 =ðc1  c2 Þ (Problem 2.10).

2.2 Poisson Process We consider a stochastic process fXðtÞ; t 2 Tg in which XðtÞ changes randomly with time t 2 T in a space by the probabilistic law, i.e., XðtÞ is a random variable for each t in time set and fXðtÞ; t 2 Tg is its collection. We restrict ourselves to a nonnegative time parameter t for the time set T: If the time set is countable, i.e., T ¼ f0; 1; 2; . . .g; the process fXðkÞ; k ¼ 0; 1; 2; . . .g is called a discrete-time stochastic process, and if the time set is continuous, i.e., T ¼ Set of fa nonnegative real numberg; the process fXðtÞ; t 0g is called a continuous-time stochastic process. If a random variable XðtÞ presents the working condition of a unit at every time and at every day or week, the time set is continuous and discrete, respectively. We omit fk ¼ 0; 1; 2; . . .g and ft 0g for simplicity unless otherwise stated. A set of all possible values of XðtÞ for the process fXðtÞg is called a state space. We restrict ourselves to a discrete state space, where XðtÞ ¼ 0; 1; 2; . . .; except for the cumulative processes in Chap. 6, and Brownian motion and Lévy process in

2.2 Poisson Process

19

Chap. 7. When the state space in such stochastic processes is continuous, we need more mathematical tools for their analysis. If the process fXðtÞg is observed over time t; such realization of XðtÞ is called a sample path or sample function. Furthermore, we introduce the following two properties of a stochastic process: It is said that the difference Xðtk Þ  Xðtk1 Þ is called the increment, and XðtÞ has independent increments if Xðt1 Þ  Xðt0 Þ; Xðt2 Þ  Xðt1 Þ; . . .; Xðtn Þ  Xðtn1 Þ are independent for all 0  t0 \t1 \    \tn1 \tn ; where it is called that tk is an arrival time of the kth event and tk  tk1 is an interarrival time. In addition, the process fXðtÞg is said to have stationary or homogeneous increments if Xðt þ uÞ  XðtÞ has the same distribution for all t 0: If XðtÞ has independent and stationary increments, then XðtÞ and Xðt þ uÞ  XðtÞ are independent, and Xðt þ uÞ  XðtÞ has the same distribution as XðuÞ: Thus, we have the relation EfXðt þ uÞg ¼ EfXðt þ uÞ  XðtÞ þ XðtÞg ¼ EfXðuÞg þ EfXðtÞg: The properties of stationary and independent increments in reliability mean intuitively that any failures occur independently and stationarity in ½t; t þ u; being dependent only on time interval u irrespective of the beginning time t and other intervals. For example, when we consider two intervals ½t1 ; t1 þ u and ½t2 ; t2 þ u for t1 þ u\t2 ; failures occur independently in two intervals and the probability of failures in two intervals is equal to that in ½0; u: In general, the property of independent increment is more important than that of stationary increment for most reliability models, because they might be easy to fail with their age. We consider the stochastic process fNðtÞg; where NðtÞ ¼ 0; 1; 2; . . .; that is called a counting process fNðtÞg; because NðtÞ counts the total number of events occurred in time t: Definition 2.1 If (i) (ii) (iii) (iv)

Nð0Þ ¼ 0; the process fNðtÞg has stationary and independent increments, PrfNðhÞ ¼ 1g ¼ kh þ oðhÞ; PrfNðhÞ 2g ¼ oðhÞ;

then a counting process fNðtÞg is said to be a Poisson process with rate k [ 0: Using this definition, we derive the probability Pk ðtÞ  PrfNðtÞ ¼ kjNð0Þ ¼ 0g ¼ PrfNðtÞ ¼ kg ðk ¼ 0; 1; 2; . . .Þ; ð2:23Þ P1 that represents the probability that k events occur in ½0; t; and k¼0 Pk ðtÞ ¼ 1: From (ii), (iii), and (iv), 8 < 1  kh þ oðhÞ for k ¼ j; PrfNðt þ hÞ ¼ k j NðtÞ ¼ jg ¼ kh þ oðhÞ for k ¼ j þ 1; ð2:24Þ : oðhÞ for k [ j þ 1:

20

2 Poisson Processes

Thus, from (ii) and (2.24), P0 ðt þ hÞ ¼ PrfNðt þ hÞ ¼ 0g ¼ PrfNðtÞ ¼ 0; Nðt þ hÞ  NðtÞ ¼ 0g ¼ PrfNðtÞ ¼ 0g PrfNðt þ hÞ  NðtÞ ¼ 0g ¼ PrfNðtÞ ¼ 0g PrfNðhÞ ¼ 0g ¼ P0 ðtÞ½1  kh þ oðhÞ: Hence, P0 ðt þ hÞ  P0 ðtÞ oðhÞ ¼ kP0 ðtÞ þ : h h Letting h ! 0; we have the differential equation dP0 ðtÞ ¼ kP0 ðtÞ; dt

i.e.;

dP0 ðtÞ=dt ¼ k: P0 ðtÞ

By integration and setting P0 ð0Þ ¼ 1 from (i), P0 ðtÞ ¼ ekt :

ð2:25Þ

Similarly, from (2.24), Pk ðt þ hÞ ¼ PrfNðt þ hÞ ¼ kg ¼ PrfNðtÞ ¼ k  1; Nðt þ hÞ  NðtÞ ¼ 1g þ PrfNðtÞ ¼ k; Nðt þ hÞ  NðtÞ ¼ 0g þ

k X

PrfNðtÞ ¼ k  j; Nðt þ hÞ  NðtÞ ¼ jg

j¼2

¼ Pk1 ðtÞ½kh þ oðhÞ þ Pk ðtÞ½1  kh þ oðhÞ þ oðhÞ: Hence, Pk ðt þ hÞ  Pk ðtÞ oðhÞ ¼ kPk1 ðtÞ  kPk ðtÞ þ : h h Letting h ! 0; dPk ðtÞ ¼ kPk1 ðtÞ  kPk ðtÞ; dt i.e., e

kt



P0k ðtÞ

  d ekt Pk ðtÞ þ kPk ðtÞ ¼ ¼ kekt Pk1 ðtÞ: dt 

Recalling that P0 ðtÞ ¼ ekt in (2.25), when k ¼ 1; d½ekt P1 ðtÞ ¼ k: dt

ð2:26Þ

2.2 Poisson Process

21

By integration under the initial condition of P1 ð0Þ ¼ 0; P1 ðtÞ ¼ ktekt : h i Using the mathematical induction by setting Pk1 ðtÞ ¼ ðktÞk1 =ðk  1Þ! ekt in (2.26),   d ekt Pk ðtÞ kðktÞk1 : ¼ ðk  1Þ! dt Similarly, by integration under Pk ð0Þ ¼ 0; Pk ðtÞ ¼

ðktÞk kt e k!

ðk ¼ 0; 1; 2; . . .Þ;

ð2:27Þ

that corresponds to a Poisson distribution in (2.17). Conversely, when a counting process NðtÞ has a Poisson distribution in (2.27), and identically distributed the interarrival times Xk ðk ¼ 1; 2; . . .Þ are independent  kt [4, p. 69]: It is easily shown according to an exponential distribution 1  e that from (2.16), PrfX1  tg ¼ PrfS1  tg ¼ PrfNðtÞ 1g ¼ 1  PrfNðtÞ ¼ 0g ¼ 1  ekt ;   i.e., X1 has an exponential distribution 1  ekt : Similarly, PrfX2  tjX1 ¼ ug ¼ PrfNðt þ uÞ  NðuÞ 1jX1 ¼ ug ¼ 1  PrfNðt þ uÞ  NðuÞ ¼ 0g ¼ 1  PrfNðtÞ ¼ 0g ¼ 1  ekt : Generally, PrfXnþ1  tjSn ¼ ug ¼ PrfNðt þ uÞ  NðuÞ 1jSn ¼ ug ¼ 1  PrfNðt þ uÞ  NðuÞ ¼ 0g ¼ 1  ekt : Therefore, each interarrival time Xk is independent and identically distributed exponentially in (2.1). Thus, a Poisson process has memoryless property. In addition, because an exponential density kekt decreases strictly, a Poisson process has shorter interarrival times than longer ones. From (2.16), the arrival time Sn of the nth event has a gamma distribution 1 X ðktÞk kt e : PrfSn  tg ¼ k! k¼n Applying the stationary property of (ii) to (2.27), PrfNðt þ uÞ  NðuÞ ¼ kjNð0Þ ¼ 0g ¼ PrfNðtÞ ¼ kg ¼ Pk ðtÞ for any t; u 0: Using this result, we can make an alternative definition of a Poisson process [4, p. 67, 8, p. 31]:

22

2 Poisson Processes

Definition 2.2 If (i) Nð0Þ ¼ 0; (ii) the process fNðtÞg has independent increments, (iii) the probability that k events occur in any interval t is PrfNðt þ uÞ  NðuÞ ¼ kg ¼

ðktÞk kt e k!

ðk ¼ 0; 1; 2; . . .Þ

for any t; u 0; then a counting process fNðtÞg is a Poisson process. It can be easily seen that two definitions are equivalent: From the above results, Definition 2.1 implies Definition 2.2. It can be clearly verified that the process has stationary increments from (iii) of Definition 2.2. Furthermore, from (iii) and (3) of Sect. 2.1.1, PrfNðhÞ ¼ 1g ¼ khekh ¼ kh þ oðhÞ; 1 X ðkhÞk kh PrfNðhÞ 2g ¼ e ¼ oðhÞ; k! k¼2 that follows that Definition 2.2 implies Definition 2.1. A Poisson process involves all properties shown in Definitions 2.1 and 2.2. Example 2.7 (Poisson distribution) It has been estimated from the past life data that failures of some unit occur in a Poisson process and its expected number is twice the average per year, i.e., 365 24 ¼ 8; 760 h. Then, from kt ¼ 2; the MTTF is 1 t 8; 760 ¼ ¼ ¼ 4; 380 h: k 2 2   Thus, the unit has a failure exponential distribution 1  et=4380 and failures n o occur in a Poisson distribution ½t=4380k =k! et=4380 ðk ¼ 0; 1; 2; . . .Þ: The number of units that should be provided to assure with probability 0:90 to operate continuously during one year without replenishment is given by the minimum number that satisfies, from Example 2.6, n X 2k k¼0

k!

e2 0:9;

i.e., its number is 5, including the first operating unit. Example 2.8 (Superposition of Poisson process) [5, p. 87] Let N1 ðtÞ be the number of failed units from one factory arriving at a repair shop in ½0; t and N2 ðtÞ be the number of failed units from the other. If the arrival times from two factories are independent and have the respective Poisson processes with rates k1 and k2 ; the total number NðtÞ of failed units arriving at a repair shop has the probability

2.2 Poisson Process

23

PrfNðtÞ ¼ ng ¼ PrfN1 ðtÞ þ N2 ðtÞ ¼ ng ¼

n X

PrfN1 ðtÞ ¼ k; N2 ðtÞ ¼ n  kg

k¼0

¼

n X ðk1 tÞk k¼0

k!

ek1 t

ðk2 tÞnk k2 t e ðn  kÞ!

¼

 k  nk n ½ðk1 þ k2 Þtn ðk1 þk2 Þt X n! k1 k2 e n! k1 þ k2 k!ðn  kÞ! k1 þ k2 k¼0

¼

½ðk1 þ k2 Þtn ðk1 þk2 Þt e : n!

Thus, the process fNðtÞg is also a Poisson process with rate k1 þ k2 (Problem 2.11). Example 2.9 (Decomposition of Poisson process) [5, p. 88] Let NðtÞ be the number of failed units occurring at a factory in ½0; t and be a Poisson process with rate k: Classifying into two large groups of failed units, the number N1 ðtÞ of minor ones occurs with probability p and the number N2 ðtÞ of major ones occurs with probability q  1  p; independent of N1 ðtÞ: Then, because the number N1 ðtÞ has a binomial distribution with p; given that n failures have occurred, the joint probability is PrfN1 ðtÞ ¼ k; N2 ðtÞ ¼ n  kg ¼ PrfN1 ðtÞ ¼ k; N2 ðtÞ ¼ n  kjNðtÞ ¼ ng PrfNðtÞ ¼ ng   n k nk ðktÞn kt ðpktÞk pkt ðqktÞnk qkt e ¼ e e pq ¼ : n! k! ðn  kÞ! k Thus, the two Poisson processes fN1 ðtÞg and fN2 ðtÞg are independent and have the Poisson processes with rates pk and qk; respectively (Problem 2.12). In general, when the total number NðtÞ of failed units with a Poisson process with rate k are classified into k groups of Nj ðtÞ ðj ¼ 1; 2; . . .; kÞ with probability pj P P where kj¼1 Nj ðtÞ ¼ NðtÞ and kj¼1 pj ¼ 1; the joint probability is PrfN1 ðtÞ ¼ n1 ; N2 ðtÞ ¼ n2 ; . . .; Nk ðtÞ ¼ nk g ðp1 ktÞn1 p1 kt ðp2 ktÞn2 p2 kt ðpk ktÞnk pk kt ¼ ... ; e e e n1 ! n2 ! nk !

ð2:28Þ

that is called a multi-Poisson process. Next, when events occur in a Poisson process, we obtain the distribution of the interarrival time X; given that there was an event in ½0; t [4, p.71, 8, p.36]. This probability is given by, for u  t;

24

2 Poisson Processes

PrfX  ujNðtÞ ¼ 1g ¼ ¼

PrfNðuÞ ¼ 1; NðtÞ  NðuÞ ¼ 0g PrfNðtÞ ¼ 1g kueku ekðtuÞ u ¼ ; ktekt t

ð2:29Þ

that is a uniform distribution over ½0; t: That is, when the event was detected at time t; it occurs constantly over ½0; t: Example 2.10 (Binomial distribution) [9, p. 76] Let NðtÞ be a Poisson process with rate k: Then, from Definition 2.2, for 0  u  t, PrfNðuÞ ¼ k; NðtÞ ¼ ng PrfNðtÞ ¼ ng PrfNðuÞ ¼ kg PrfNðtÞ  NðuÞ ¼ n  kg ¼ PrfNðtÞ ¼ ng n o k ku ½ðkuÞ =k!e ½kðt  uÞnk =ðn  kÞ! ekðtuÞ ¼ ½ðktÞn =n!ekt    k  n u u nk 1 ¼ ðk ¼ 0; 1; 2; . . .; nÞ; t t k

PrfNðuÞ ¼ kjNðtÞ ¼ ng ¼

ð2:30Þ

that is a binomial distribution with parameters n and u=t: A uniform distribution sometimes appears in the cases where events occur only in the finite interval and when an event was detected at time t; without any information of its arrival time, as shown in (2.29). Example 2.11 (Inspection policy) [2, p. 113] Consider an inspection policy where the failure time of a unit is uniformly distributed over ½0; S that is called a working interval [6, p. 96]. To detect a failure, the unit is checked at planned times Tk ðk ¼ 1; 2; . . .; NÞ; where TN ¼ S (Fig. 2.2). Let c1 be the cost of one check and c2 be the cost per unit of time for the time elapsed between a failure and its detection at the next checking time. Then, the expected cost until failure detection is Tkþ1 N 1 Z X dt ½ðk þ 1Þc1 þ c2 ðTkþ1  tÞ : ð2:31Þ C1 ðNÞ ¼ S k¼0 Tk

Differentiating C1 ðNÞ with respect to Tk and setting it equal to zero, c1 Tkþ1  Tk ¼ Tk  Tk1  : c2 Fig. 2.2 Process of an inspection policy

2.2 Poisson Process

25

Solving for Tk ; Tk ¼ kT1 

kðk  1Þ c1 : c2 2

Setting TN ¼ S; Tk ¼

kS c1 þ kðN  kÞ 2c2 N

ðk ¼ 0; 1; 2; . . . ; NÞ:

From Tkþ1  Tk [ 0; S c1 ðN  2k  1Þ [ 0: þ N 2c2 When k ¼ N  1; NðN  1Þ c2 S ; \ 2 c1

NðN þ 1Þ c2 S : ð2:32Þ

2 c1 P Thus, N must be a unique minimum that satisfies Nk¼1 k c2 S=c1 : Note that the summation of integers from 1 to N plays an important rule in obtaining optimum policies for some partition models [6, p. 39] (Problem 2.13). For example, when S ¼ 100 and c1 =c2 ¼ 7; the checking number is N ¼ 5 and checking times are 34, 61, 81, 94, 100. Next, consider the backup policy with checkpoint times in which we execute the rollback operation to the latest checkpoint [6, p. 96] and reconstruct the consistent state. Let c2 be the cost per unit of time for the backup operation to the latest checking time Tk when the failure was detected between Tk and Tkþ1 (Fig. 2.3). The other assumptions are the same as those in the inspection policy. Then, the expected cost until the backup operation is done to the latest checking time when a unit has failed at time t is C2 ðNÞ ¼

i.e.;

Tkþ1 N1 Z X k¼0

dt ½kc1 þ c2 ðt  Tk Þ : S

ð2:33Þ

Tk

Differentiating C2 ðNÞ with respect to Tk and setting it equal to zero, c1 Tkþ1  Tk ¼ Tk  Tk1  : c2 Therefore, the problem of optimizing checkpoint intervals of the backup operation corresponds to that of the inspection policy when the failure time has a uniform distribution. This is the reason, as failures occur uniformly over any interval. Fig. 2.3 Process of backup policy

26

2 Poisson Processes

Fig. 2.4 Excess cost and shortage cost of backward time T

Example 2.12 (Backward time problem) [6, p. 88] Suppose that when a unit has failed, and its failure time is unknown, we go back to time T ð0  T  tÞ from time t to detect its failure and call T a planned backward time. Then, we introduce the following costs: Cost c1 ðxÞ is the excess cost for the time x from a failure to the backward time, c2 ðxÞ is the shortage cost for the time x from the backward time to a failure, and c0 ðTÞ is the cost required for the backward time with c0 ð0Þ  0 (Fig. 2.4). It is assumed that a failure is distributed uniformly over ½0; t based on the result obtained in (2.29). Then, the expected cost for the backward time T is Z T Z tT 1 CðTjtÞ ¼ c1 ðxÞdx þ c2 ðxÞdx þ c0 ðTÞ: ð2:34Þ t 0 0 In particular, when ci ðtÞ ¼ ci t ði ¼ 0; 1; 2Þ; the expected cost is i 1h 2 CðTjtÞ ¼ c1 T þ c2 ðt  TÞ2 þ c0 T: 2t

ð2:35Þ

Differentiating CðTjtÞ with respect to T and setting it equal to zero, T¼

c 2  c0 t: c 2 þ c1

ð2:36Þ

Therefore, an optimum T  ð0\T  \tÞ that minimizes (2.35) is given by (2.36) for c2 [ c0 : If c2  c0 ; then T  ¼ 0; i.e., we should not go back at all because the backward cost is high (Problem 2.14). This result in (2.29) is generalized as follows [4, p. 71, 8, p. 37]: The kth arrival times Sk ðk ¼ 1; 2; . . .; nÞ; given that NðtÞ ¼ n; has the probability kh1 ekh1 . . .khn ekhn ekðth1 h2 hn Þ ½ðktÞn =n!ekt n! ¼ n h1 h2 . . .hn t

Prftk \Sk \tk þ hk ; k ¼ 1; 2; . . .; njNðtÞ ¼ ng ¼

2.2 Poisson Process

27

for any 0\t1 \t2 \    \tn \tnþ1 ¼ t and tk þ hk \tkþ1 for small hk ; and hence, its density function is f ðt1 ; t2 ; . . .; tn jNðtÞ ¼ nÞ ¼

n! : tn

ð2:37Þ

That is, n arrival times S1 ; S2 ; . . .; Sn given that NðtÞ ¼ n are independent and identically distributed uniformly over the interval ½0; t: In addition [10, p. 15], from Example 2.10, PrfSk  u; NðtÞ ¼ ng PrfNðtÞ ¼ ng PrfNðuÞ k; NðtÞ ¼ ng ¼ PrfNðtÞ ¼ ng Pn j¼k Prf NðuÞ ¼ jg PrfNðtÞ  NðuÞ ¼ n  jg ¼ PrfNðtÞ ¼ ng    nj  j n X n u u : ¼ 1 t t j j¼k

PrfSk  ujNðtÞ ¼ ng ¼

ð2:38Þ

Clearly (Problem 2.15), EfSk jNðtÞ ¼ ng ¼

kt ; nþ1

EfSk  Sk1 jNðtÞ ¼ ng ¼

t : nþ1

ð2:39Þ

2.3 Nonhomogeneous Poisson Process Suppose that a unit begins to operate at time 0: If the unit fails, then it undergoes minimal repair and begins to operate again. It is assumed that the time for repair is negligible. Let us denote 0  S0  S1      Sn1  Sn  . . . be the successive failure times of the unit, and Xn  Sn  Sn1 ðn ¼ 1; 2; . . .Þ be the times between failures (Fig. 2.1) with FðtÞ  PrfX1  tg and FðtÞ  1  FðtÞ: We define to make minimal repair at failure as follows [3, p. 96]: Definition 2.3 The unit undergoes minimal repair at failures if and only if PrfXn  xjSn1 ¼ tg ¼

Fðt þ xÞ  FðtÞ FðtÞ

ðn ¼ 2; 3; . . .Þ

ð2:40Þ

for x [ 0 such that FðtÞ\1: The function ½Fðt þ xÞ  FðtÞ=FðtÞ is called the failure rate and represents the probability that a unit with age t fails in the interval ðt; t þ x: The definition in reliability models means that the failure rate remains undisturbed by any minimal repair of failures, i.e., a unit after each minimal repair has the same failure rate as

28

2 Poisson Processes

previous failure. Imperfect PM (Preventive Maintenance) models in which the failure rate after PM reduces were considered and their optimum policies were discussed analytically [3, p. 171]. A variety of failure rate modelling was extensively collected [12]. Assume that FðtÞ has a density function f ðtÞ and hðtÞ  f ðtÞ=FðtÞ is continuous. The function hðtÞ is also called the instantaneous failure rate or simply the failure rate and has the same monotone property as ½Fðt þ xÞ  FðtÞ=FðtÞ [2, p. 23]. Rt Moreover, HðtÞ  0 hðuÞdu is called the cumulative hazard function and satisfies the relation FðtÞ ¼ eHðtÞ that decreases with t: Because, from the definition of hðtÞ;  0    FðtÞ d log FðtÞ ¼ ¼ hðtÞ: dt FðtÞ By integration under the initial condition Fð0Þ ¼ 1; log FðtÞ ¼ 

Zt

hðuÞdu ¼ HðtÞ;

i.e.;

FðtÞ ¼ eHðtÞ :

ð2:41Þ

0

We derive the distribution of the nth failure time Sn ; 1 n1 X X ½HðtÞk HðtÞ ½HðtÞk HðtÞ Gn ðtÞ  PrfSn  tg ¼ e e ¼1 k! k! k¼n k¼0

ðn ¼ 1; 2; . . .Þ: ð2:42Þ

By the mathematical induction, G1 ðtÞ ¼ PrfX1  tg ¼ FðtÞ ¼ 1  eHðtÞ ; Z1 Gnþ1 ðtÞ ¼ PrfXnþ1  t  ujSn ¼ ugdGn ðuÞ 0

¼

Zt

FðtÞ  FðuÞ ½HðuÞn1 dFðuÞ ðn  1Þ! FðuÞ

0

¼1

n1 X ½HðtÞk k¼0

¼1

k!

e

HðtÞ

Zt

½HðuÞn1 hðuÞdu ðn  1Þ!

0

n X ½HðtÞk k¼0

e

HðtÞ

k!

eHðtÞ

ðn ¼ 1; 2; . . .Þ:

The density function of Gn ðtÞ is gn ðtÞ  G0n ðtÞ ¼

½HðtÞn1 HðtÞ ½HðtÞn1 e f ðtÞ; hðtÞ ¼ ðn  1Þ! ðn  1Þ!

2.3 Nonhomogeneous Poisson Process

29

and hence, we have the relation Gn ðtÞ ¼

Zt 0

1 X ½HðuÞn1 ½HðtÞk HðtÞ dFðuÞ ¼ e : ðn  1Þ! k! k¼n

ð2:43Þ

From the above results, EfSn g 

Z1 0

n1 Z X ½HðtÞk HðtÞ e Gn ðtÞdt ¼ dt; k! k¼0 1

0

EfXn g ¼ EfSn g  EfSn1 g ¼

Z1

ð2:44Þ ½HðtÞn1 HðtÞ e dt: ðn  1Þ!

0

Next, we prove that when hðtÞ increases (decreases), EfXn g decreases (increases) with n and converge to 1=hð1Þ as n ! 1 [3, p. 98]. Using the relation (Problem 2.16) ½HðtÞn ¼ n!

Zt

½HðuÞn1 hðuÞdu; ðn  1Þ!

0

and from the assumption that hðtÞ increases, it follows that 9 8 Z1
¼

0

Z1

½HðtÞn1 HðtÞ dt; e ðn  1Þ!

0

and hence, EfXn g decreases with n: Furthermore, from (2.43), Z1 0

½HðtÞn HðtÞ dt ¼ e n!

Z1

½HðtÞn hðtÞ HðtÞ dt e n! hðtÞ

0

1

hð1Þ

Z1 0

½HðtÞn 1 dFðtÞ ¼ : n! hð1Þ

30

2 Poisson Processes

On the other hand, for any 0\T\1; Z1

½HðtÞn HðtÞ dt ¼ e n!

ZT

0

½HðtÞn HðtÞ dt þ e n!

½HðtÞn HðtÞ dt e n!

T

0



Z1

ZT

½HðtÞn HðtÞ 1 e dt þ : n! hðTÞ

0

Thus, because Z1 lim

n!1

½HðtÞn HðtÞ 1 e dt  ; n! hðTÞ

0

and T is arbitrary, Z1 lim

n!1

½HðtÞn HðtÞ 1 : dt ¼ e hð1Þ n!

0

Similarly, when hðtÞ decreases, we can prove that EfXn g increases with n and converge to 1=hð1Þ as n ! 1; where we set that 1=hð1Þ  0 for hð1Þ ¼ 1 and 1=hð1Þ  1 for hð1Þ ¼ 0: Note that when hðtÞ increases strictly to 1; EfXn g decreases strictly to 0: The failure rate hðtÞ and the cumulative hazard Rt function HðtÞ  0 hðuÞdu in reliability theory are called an intensity function and a mean value function in stochastic processes, respectively. Furthermore, let NðtÞ be the number of failures of a unit in ½0; t; i.e., NðtÞ  maxn fSn  tg: Then, using the relation with NðtÞ and Sn in (2.16), Pn ðtÞ  PrfNðtÞ ¼ ng ¼ PrfSn  t\Snþ1 g ¼ Gn ðtÞ  Gnþ1 ðtÞ ½HðtÞn HðtÞ e ðn ¼ 0; 1; 2; . . .Þ; n! EfNðtÞg ¼ VfNðtÞg ¼ HðtÞ: ¼

ð2:45Þ

In particular, when FðtÞ ¼ 1  ekt ; i.e., HðtÞ ¼ kt; PrfSn  tg ¼ Gn ðtÞ ¼ 1 

n1 X ðktÞk k¼0

PrfXn  tg ¼ 1  ekt ;

k!

ekt ;

1 EfXn g ¼ ; k

n EðSn Þ ¼ ; k

EfNðtÞg ¼ VfNðtÞg ¼ kt;

that agree with those of the Poisson process with rate k: Referring to the above results and expanding Definition 2.1, we can generalize the Poisson process with the parameter of time t [4, p. 77, 8, p. 46]:

2.3 Nonhomogeneous Poisson Process

31

Definition 2.4 If (i) (ii) (iii) (iv)

Nð0Þ ¼ 0; the process fNðtÞg has independent increments, PrfNðt þ hÞ  NðtÞ ¼ 1g ¼ kðtÞh þ oðhÞ; PrfNðt þ hÞ  NðtÞ 2g ¼ oðhÞ;

then a counting process fNðtÞg is said to be a nonhomogeneous or nonstationary Poisson process that eliminates the stationary from a Poisson process. Using this definition, we derive the probability Pn ðuÞ  PrfNðt þ uÞ  NðtÞ ¼ ng ¼

½Hðt þ uÞ  HðtÞn ½HðtþuÞHðtÞ e n!

ðn ¼ 0; 1; 2; . . .Þ

ð2:46Þ

for a fixed t: From (ii), (iii), and (iv), P0 ðu þ hÞ ¼ PrfNðt þ u þ hÞ  NðtÞ ¼ 0g ¼ PrfNðt þ uÞ  NðtÞ ¼ 0; Nðt þ u þ hÞ  Nðt þ uÞ ¼ 0g ¼ PrfNðt þ uÞ  NðtÞ ¼ 0g PrfNðt þ u þ hÞ  Nðt þ uÞ ¼ 0g ¼ P0 ðuÞ½1  kðt þ uÞh þ oðhÞ: Hence, P0 ðu þ hÞ  P0 ðuÞ oðhÞ ¼ kðt þ uÞP0 ðuÞ þ : h h Letting h ! 0; we have the differential equation P00 ðuÞ ¼ kðt þ uÞP0 ðuÞ;

i.e.;

dP0 ðuÞ=du ¼ kðt þ uÞ: P0 ðuÞ

By integration and setting P0 ð0Þ ¼ 1; log P0 ðuÞ ¼ 

Zu

kðt þ xÞdx ¼ ½Hðt þ uÞ  HðtÞ;

0

that implies P0 ðuÞ ¼ e½HðtþuÞHðtÞ : Similarly, by the same method used in the Poisson process, we can prove (2.46). Clearly, EfNðt þ uÞ  NðtÞg ¼

1 X ½Hðt þ uÞ  HðtÞn HðtþuÞHðtÞ e n : n! n¼0

¼ Hðt þ uÞ  HðtÞ:

32

2 Poisson Processes

Finally, the distribution of the interarrival time X1 ¼ S1 ; given that there was an event in ½0; t; is [4, p. 78], for u  t; PrfNðuÞ ¼ 1; NðtÞ  NðuÞ ¼ 0g PrfNðtÞ ¼ 1g PrfNðuÞ ¼ 1g PrfNðtÞ  NðuÞ ¼ 0g ¼ PrfNðtÞ ¼ 1g

PrfS1  ujNðtÞ ¼ 1g ¼

¼

HðuÞeHðuÞ e½HðtÞHðuÞ HðuÞ : ¼ HðtÞ HðtÞeHðtÞ

ð2:47Þ

Example 2.13 (Expected number of failures) A random variable Y  HðXÞ; that represents the expected number of failures until the failure time, has the following distribution

 ð2:48Þ PrfY  tg ¼ PrfHðXÞ  tg ¼ Pr X  H 1 ðtÞ ¼ 1  et ; where H 1 is the inverse function of H: Thus, Y has an exponential distribution with mean 1; and EfHðXÞg ¼ V fHðXÞg ¼ 1: In other words, by transforming the failure times S1 ; S2 ; . . . to HðS1 Þ; HðS2 Þ; . . .; a nonhomogeneous Poisson process corresponds to a Poisson process with rate 1: Moreover, x1 that satisfies Hðx1 Þ ¼ 1 is called a characteristic life in the probability paper of a Weibull distribution. This represents the mean lifetime that about 63.2 % of units have failed until time x1 ; as shown in (1) of Sect. 2.1.1. In general, xn that satisfies Hðxn Þ ¼ n ðn ¼ 1; 2; . . .Þ represents the time that the expected number of failures is n; when failures occur in a nonhomogeneous Poisson with a mean value function HðtÞ: From Hðxn Þ ¼ n; Zxn

hðtÞdt ¼ 1

Zxn or

xn1

hðtÞdt ¼ n

ðn ¼ 1; 2; . . .Þ:

0

In particular, when FðtÞ is IFR, xn  xn1 

1 ; hðxn1 Þ

i.e.;

xn 

1 þ xn1 : hðxn1 Þ

Thus, denoting that xn 

1 þ xn1 hðxn1 Þ

ðn ¼ 2; 3; . . .Þ;

xn  xn : Similarly, xn 

1

xn1 : hðxn Þ

ð2:49Þ

2.3 Nonhomogeneous Poisson Process

33

Thus, denoting xn that satisfies xn 

1 ¼ xn1 hðxn Þ

ðn ¼ 2; 3; . . .Þ;

xn xn : The function 1=hðtÞ is called an instantaneous MTBF [11, p. 71]. Furthermore, from Theorem 4.4 of [2, p. 27] when F is IFR (DFR) and l  EfXg; FðlÞ ðÞe1 : Thus, HðlÞ  ð ÞHðx1 Þ; i.e., l  ð Þx1 : Example 2.14 (Weibull distribution) Suppose that HðtÞ ¼ kta for a [ 0; i.e., the failure time has a Weibull distribution FðtÞ ¼ 1  expðkta Þ with a shape parameter a: Then, the failure rate is hðtÞ ¼ kata1 ; and decreases for 0\a\1; constant for a ¼ 1; and increases for 1\a\1: So that, the Weibull distribution represents a typical failure distribution that has three patterns in the bath-tub curve (Fig. 2.5). In this case, the process is sometimes called a Weibull process. Furthermore, from (2.49), xn ¼ ðn=kÞ1=a ðn ¼ 1; 2; . . .Þ and l ¼ ð1=kÞ1=a Cð1 þ 1=aÞ: Therefore, (i) x1 \l for 0\a\1; (ii) x1 ¼ l for a ¼ 1; and x1 [ l for a [ 1: Table 2.4. presents xn ; xn and xn when x1 ¼ x1 ¼ x1 ; and EðSn Þ from (2.44) for a ¼ 2 and k ¼ 0:01: This indicates that xn [ EðSn Þ; however, xn are almost the same as EðSn Þ for n 2: Many Weibull models were summarized [13]. Let 0  S0 \S1 \S2 \. . . be the successive times of a nonhomogeneous Poisson process with a mean value function HðtÞ: Then, we have the relation, from Example 2.13, fSnþ1  Sn [ tg , fHðSnþ1 Þ [ HðSn þ tÞg: In addition, because HðSnþ1 Þ  HðSn Þ is independent of S1 ; S2 ; . . .; Sn and has an exponential distribution with mean 1, PrfSnþ1  Sn [ tjS1 ; S2 ; . . .; Sn g ¼ PrfHðSnþ1 Þ [ HðSn þ tÞjS1 ; S2 ; . . .; Sn g ¼ PrfHðSnþ1 Þ  HðSn Þ [ HðSn þ tÞ  HðSn Þg ¼ e½HðSn þtÞHðSn Þ :

Fig. 2.5 Bath-tub curve of the failure rate

34

2 Poisson Processes

Table 2.4 Comparisons of times of expected number when a ¼ 2 and k ¼ 0:01

n

xn

xn ðx1 ¼ x1 Þ

xn

EðSn Þ

1 2 3 4 5 6 7 8 9 10

10.00 14.14 17.32 20.00 22.36 24.49 26.46 28.28 30.00 31.62

10.00 15.00 18.33 21.06 23.43 25.56 27.52 29.34 31.04 32.65

10.00 13.66 17.07 19.84 22.25 24.41 26.39 28.23 29.95 31.59

8.86 13.29 16.62 19.39 21.81 23.99 25.99 27.85 29.59 31.23

The interarrival times for a nonhomogeneous Poisson are not independent, however, still have conditionally exponential distribution [5, p. 97].

2.4 Applications to Reliability Models 2.4.1 Replacement at the Nth Failure A new unit begins to operate at time 0; and when it fails, only minimal repair is made. It is assumed that the repair and replacement times are negligible. The failure times of the R 1unit have a density function f ðtÞ and a distribution FðtÞ with finite mean l  0 FðtÞdt; failure rate hðtÞ  f ðtÞ=FðtÞ; and cumulative hazard Rt rate HðtÞ  0 hðuÞdu: Suppose that the unit is replaced only at the Nth failure. Then, the mean time to the Nth failure is, from (2.44), N 1 Z X

1

EfSN g ¼

k¼0

Pk ðtÞdt;

0

where Pk ðtÞ  f½HðtÞk =k!geHðtÞ ðk ¼ 0; 1; 2; . . .Þ: Consider one cycle from the beginning of operation to the replacement. Let c1 be the cost of minimal repair and c2 be the replacement cost at the Nth failure. The expected cost rate is [3, p. 106] CðNÞ 

Expected cost until replacement c1 ðN  1Þ þ c2 ¼ PN1 R 1 Mean time to replacement k¼0 0 Pk ðtÞdt

ðN ¼ 1; 2; . . .Þ: ð2:50Þ

2.4 Applications to Reliability Models

35

When the failure rate hðtÞ increases strictly to hð1Þ; we derive an optimum number N  that minimizes CðNÞ in (2.50). From the inequality CðN þ 1Þ  CðNÞ 0; PN1 R 1 c2 k¼0 0 Pk ðtÞdt R1 ðN ¼ 1; 2; . . .Þ: ð2:51Þ  ðN  1Þ

c1 P ðtÞdt N 0 Letting LðNÞ be the left-hand side of (2.51), " 1 N Z X LðN þ 1Þ  LðNÞ ¼ Pk ðtÞdt R 1 k¼0

0

0

# 1  R1 [ 0; PNþ1 ðtÞdt 0 PN ðtÞdt 1

R1 because 0 Pk ðtÞdt decreases strictly from (2.44) to 1=hð1Þ when hðtÞ increases strictly. Thus, if a solution to (2.51) exists, then it is unique, and the resulting cost rate is R1 0

c1 PN  1 ðtÞdt

\CðN  Þ  R 1 0

c1 : PN  ðtÞdt

ð2:52Þ

Furthermore, we have the inequality (Problem 2.17) LðNÞ [ R 1 0

l PN ðtÞdt

ðN ¼ 2; 3; . . .Þ:

Therefore, if lim R 1

N!1

0

l c2

; PN ðtÞdt c1

i.e., if hð1Þ c2 =ðlc1 Þ; then a solution to (2.51) exists, and it is unique. In addition, if N is a unique minimum that satisfies Z1

PN ðtÞdt 

lc1 ; c2

0

then N   N: Example 2.15 (Optimum number) Suppose that the failure time of a unit has a Weibull distribution, i.e., FðtÞ ¼ expðkta Þ for a [ 1: Then, hðtÞ increases strictly from 0 to infinity, and (Problem 2.18) Z1 0

1 CðN þ 1=aÞ ; PN ðtÞdt ¼ 1=a a k CðN þ 1Þ

N 1 Z X

1

k¼0

0

Pk ðtÞdt ¼

CðN þ 1=aÞ : k1=a CðNÞ

Thus, there exists a finite and unique minimum N  that satisfies (2.51) which is given by

36

2 Poisson Processes



c2  c1 N ¼ ða  1Þc1 

1=a þ 1;

ð2:53Þ

where ½x denotes the greatest integer contained in x:

2.4.2 Software Reliability Model In software reliability modeling [14], it is assumed that HðtÞ is the expected number of faults detected by time t and the number of faults detected at any time is proportional to the current number of faults. Let a [ 0 be the total number of faults, and bðtÞ be the error detection rate per fault at time t: Then, because the number of faults at time t is a  HðtÞ; we have the differential equation dHðtÞ ¼ bðtÞ½a  HðtÞ: dt Solving this equation under the initial condition Hð0Þ  0 (Problem 2.19), 8 2 39 Zt = < ð2:54Þ HðtÞ ¼ a 1  exp4 bðuÞdu5 : ; : 0

This is called a software reliability growth model. In particular, when bðtÞ ¼ b [ 0;   HðtÞ ¼ a 1  ebt ; and Hð1Þ ¼ a: A variety of generalized software reliability growth models were summarized extensively [11, 15], and several interesting new topics were cited [16].

2.5 Compound Poisson Process Consider a standard cumulative damage model [17, p. 16, 18]: Shocks occur in a Poisson Process with rate k: A unit is subjected to shocks and suffers some damage due to shocks. Let random variables Xk ðk ¼ 1; 2; . . .Þ denote a sequence of interarrival times between successive shocks, and random variables Wk ðk ¼ 1; 2; . . .Þ denote the damage produced by the kth shock, where W0  0 (Fig. 2.6). It is assumed that the sequence of fWk g is nonnegative, independently, and identically distributed according to PrfWk  xg  GðxÞ with finite mean, and furthermore, Wk is independent of Xj ðj 6¼ kÞ:

2.5 Compound Poisson Process

37

Fig. 2.6 Process for a standard cumulative damage model

Let NðtÞ denote the total number of shocks up to time t: Then, define a random variable ZðtÞ 

NðtÞ X

Wk

ðNðtÞ ¼ 0; 1; 2; . . .Þ;

ð2:55Þ

k¼0

where ZðtÞ represents the total damage at time t: This is called a compound Poisson Process and will be discussed in detail in Chap. 7. The distribution of ZðtÞ defined in (2.55) is ( NðtÞ ) X Wj  x PrfZðtÞ  xg ¼ Pr j¼0

¼

1 X k¼0

¼ ¼

(

Pr

k X j¼0

) Wj  x NðtÞ ¼ k PrfNðtÞ ¼ kg

1 X

ðktÞk kt e PrfW0 þ W1 þ    þ Wk  xg k! k¼0

1 X

ðktÞk kt GðkÞ ðxÞ e : k! k¼0

ð2:56Þ

Letting G ðsÞ be the LS transform of GðxÞ; the LS transform of (2.56) is Z1 0

esx d PrfZðtÞ  xg ¼

1 X

½G ðsÞk

k¼0

ðktÞk kt e k!

¼ expfkt½1  G ðsÞg: Because EfNðtÞg ¼ VfNðtÞg ¼ kt in (2.18),

ð2:57Þ

38

2 Poisson Processes

( ( NðtÞ )) X ¼ EfNðtÞgEfWk g ¼ ktEfWg; EfZðtÞg ¼ E E Wk NðtÞ k¼1

 VfZðtÞg ¼ E Z 2 ðtÞ  ½EfZðtÞg2 ( ( NðtÞ )) NðtÞ X X  ½EfZðtÞg2 ¼E E Wk Wj NðtÞ j¼1 k¼1

 ¼ VfNðtÞg½EfWg2 þ EfNðtÞgVfWg ¼ ktE W 2 :

ð2:58Þ

When a random variable X has the LS transform function F  ðsÞ; i.e., F  ðsÞ  R 1 sx d PrfX  xg; the above results are derived by (Appendix A) 0 e dF  ðsÞ d2 F  ðsÞ 2 EfXg ¼  ; EfX g ¼ : ds s¼0 ds2 s¼0 Next, it is assumed that the unit fails when the total damage has exceeded a failure level K ð0\K\1Þ: Usually, a failure level K is statistically estimated and is already known. We are interested in a random variable defined by Y  mint fZðtÞ [ Kg; i.e., PrfY  tg represents the distribution of the failure time and EfYg is the MTTF of the unit. In stochastic processes, PrfY  tg is called the firstpassage distribution to some event. Because the events of fY  tg and fZðtÞ [ Kg are equivalent, from (2.56), UðtÞ  PrfY  tg ¼ PrfZðtÞ [ Kg 1 h 1 i X X ðktÞj kt GðkÞ ðKÞ  Gðkþ1Þ ðKÞ ¼ e ; j! k¼0 j¼kþ1

ð2:59Þ

and its LS transform is 

U ðsÞ 

Z1 0

est dUðtÞ ¼

1 h i k kþ1 X GðkÞ ðKÞ  Gðkþ1Þ ðKÞ : sþk k¼0

Thus, the MTTF and variance are 1 dU ðsÞ 1X 1 EfYg ¼  ¼ GðkÞ ðKÞ ¼ ½1 þ MG ðKÞ; ds s¼0 k k¼0 k 1 2  d U ðsÞ 2X VfYg ¼  ðEfYgÞ2 ¼ 2 ðk þ 1ÞGðkÞ ðKÞ  ðEfYgÞ2 ; 2 ds k k¼0 s¼0

ð2:60Þ

P ðkÞ where MG ðKÞ  1 k¼1 G ðKÞ represents the expected number of shocks before the total damage exceeds a failure level K: Furthermore, the failure rate of the unit with a failure level K is

2.5 Compound Poisson Process

U0 ðtÞ ¼ rðtÞ  1  UðtÞ

39

i h P1  ðkÞ ðkþ1Þ ðKÞ kðktÞk =k! k¼0 G ðKÞ  G h i : P1 ðkÞ k k¼0 G ðKÞ ðktÞ =k!

ð2:61Þ

Example 2.16 (MTTF) Suppose that shocks occur in a Poisson process with rate k and each damage due to shocks is exponential with mean 1=h; i.e., FðtÞ ¼ 1  ekt and GðxÞ ¼ 1  ehx : Then, from (2.57), Z1 esx d PrfZðtÞ  xg ¼ ekt½s=ðsþhÞ : 0

By inversion [2, p. 80, 17, p. 23], 2

3 Zx p ffiffiffiffiffiffi ffi p ffiffiffiffiffiffiffiffiffi PrfZðtÞ  xg ¼ ekt 41 þ kht ehu u1=2 I1 ð2 khtuÞdu5;

ð2:62Þ

0

where Ii ðxÞ is the Bessel function of order i for the imaginary argument defined by 1  2kþi X x 1 : Ii ðxÞ  2 k!ðk þ iÞ! k¼0 Thus, from (2.62), 2

3 ZK  pffiffiffiffiffiffiffiffiffi p ffiffiffiffiffiffi ffi PrfY  tg ¼ 1  ekt 41 þ kht ehu u1=2 I1 2 khtu du5:

ð2:63Þ

0

Furthermore, from (2.58) and (2.60) (Problem 2.20), kt hK þ 1 ; EfYg ¼ ; h k 2kt 2hK þ 1 ; V fZðtÞg ¼ 2 ; VfYg ¼ h k2 EfZðtÞg ¼

ð2:64Þ

where note that EfZðtÞg increases linearly with time t: In addition, we have the relation EfZðtÞg t ¼ : K þ 1=h EfYg Finally, we give two modified damage models in reliability theory:

40

2 Poisson Processes

(1) Independent Damage Model [17, p. 21, 19]. Suppose that the total damage is not additive, i.e., any shock does no damage unless its amount has not exceeded a failure level K: If the damage due to some shock has exceeded for the first time a failure level K; then the unit fails. The same assumptions as those of the previous model are made except that the total damage is additive. A typical example of this model is the fracture of brittle materials such as glasses, and semiconductor parts that have failed by some over current of fault voltage. In this model, the probability that the unit fails exactly at the ðk þ 1Þth shock ðk ¼ 0; 1; 2; . . .Þ is ½GðKÞk ½GðKÞkþ1 : Thus, the distribution of time to failure is PrfY  tg ¼ 1  expfkt½1  GðKÞg;

ð2:65Þ

that is an exponential distribution with parameter k½1  GðKÞ; and hence, the MTTF and the failure rate are EfYg ¼

1 ; k½1  GðKÞ

rðtÞ ¼ k½1  GðKÞ ¼

VfYg ¼

1 k ½1  GðKÞ2 2

;

1 ; EfYg

ð2:66Þ

that is constant for any t 0: (2) Random Failure Level [17, p. 29, 19]. Suppose that a failure level K is not constant and is a random variable with a distribution LðxÞ  PrfK  xg; where Lð0Þ  0: Then, for the cumulative damage model, the distribution of time to failure is, from (2.59)

PrfY  tg ¼

1 1 Z h X k¼0

0

1 i X ðktÞj kt e ; GðkÞ ðxÞ  Gðkþ1Þ ðxÞ dLðxÞ j! j¼kþ1

ð2:67Þ

and the MTTF and the failure rate are, respectively, 1 1X EfYg ¼ k k¼0

Z1 0

GðkÞ ðxÞdLðxÞ;

h i  P1 R 1  ðkÞ ðkþ1Þ ðxÞ dLðxÞ kðktÞk =k! k¼0 0 G ðxÞ  G h i : rðtÞ ¼ P1 R 1 ðkÞ k k¼0 0 G ðxÞdLðxÞ ðktÞ =k!

ð2:68Þ

2.5 Compound Poisson Process

41

For the independent 1 damage model, 1 Z n 1 o X X ðktÞj kt PrfY  tg ¼ e ; ½GðxÞk ½GðxÞkþ1 dLðxÞ j! k¼0 j¼kþ1 0

1 Z 1X

ð2:69Þ

1

EfYg ¼

k k¼0

½GðxÞk dLðxÞ:

0

Example 2.17 (Exponential distribution) (Problem 2.21) Suppose that FðtÞ ¼ 1  ekt ; GðxÞ ¼ 1  ehx and LðxÞ ¼ 1  ebx : Then, for the independent damage model, 1 1 PrfY  tg ¼ 1  expðktehK Þ; EfYg ¼ ¼ ehK : rðtÞ k For the random failure level, Z1h i GðkÞ ðxÞ  Gðkþ1Þ ðxÞ dLðxÞ ¼ 0

Z1

est d PrfY  tg ¼

0

By inversion,

bhk ðh þ bÞkþ1

;

kþ1 1  X k bhk kb : ¼ kþ1 s þ k sðh þ bÞ þ kb ðh þ bÞ k¼0

  kbt ; PrfY  tg ¼ 1  exp  hþb 1 hþb ¼ ; EfYg ¼ rðtÞ kb

ð2:70Þ

  hþb 2 VfYg ¼ : kb

ð2:71Þ

In addition, for the independent damage model, PrfY [ tg ¼

Z1

expðktehx Þbebx dx

0

Z 1 X ðktÞk

1

¼

k¼0

k!

1 1 ¼ EfYg ¼ rðtÞ k

beðbþkhÞx dx ¼

k¼0

0

Z1

( hx

bx

e be 0

1 X ðktÞk

dx ¼

k!

b ; b þ kh

b kðbhÞ

ðb [ hÞ;

1

ðb  hÞ:

(3) Nonhomogeneous Poisson process. Suppose that shocks occur in a nonhomogeneous Poisson process with an intensity function hðtÞ and a mean value function HðtÞ: Then, because the probability of NðtÞ is given in (2.45),

42

2 Poisson Processes

PrfZðtÞ  xg ¼

1 X

GðkÞ ðxÞ

k¼0

EfYg ¼

1 X

ðkÞ

½HðtÞk HðtÞ e ; k!

G ðKÞ

k¼0

Z1

EfZðtÞg ¼ EfWgHðtÞ;

½HðtÞk HðtÞ e dt: k!

0

For the independent damage model, PrfY  tg ¼ 1  e

½1GðKÞHðtÞ

;

EfYg ¼

Z1

e½1GðKÞHðtÞ dt:

0

The failure rate is rðtÞ ¼ ½1  GðKÞhðtÞ; that has the same property as that of an intensity function hðtÞ:

2.6 Problems 2 2.1 For any negative function gðtÞ in Example 2.3 [5, p. 84], show that ( ) Z1 1 X gðSn Þ ¼ k gðtÞdt: E n¼1

0

2.2 Show that a continuous failure distribution FðtÞ that satisfies the memoryless properly, i.e., Fðt þ uÞ ¼ FðtÞFðuÞ for any t; u 0; is only an exponential distribution [8, p. 24]. 2.3 Let XðkÞ ðk ¼ 1; 2; . . .; nÞ be the kth failure time of n units in Example 2.4. Then, noting that Xð1Þ be the failure time of a series system in which when one of n units fails, the system fails, and XðnÞ be the failure time of a parallel system when all of n units have failed, the system fails. Then, show that the failure distributions of the two systems are, respectively,



Pr Xð1Þ  t ¼

Zt

nkenku du ¼ 1  enkt ;

0

 Pr XðnÞ  t ¼

Zt

nð1  ekt Þn1 kekt dt ¼ ð1  ekt Þn ;

0

EfXð1Þ g ¼

1 ; nk

EfXðnÞ g ¼

n 1X 1 : k k¼1 k

2.6 Problems 2

43

2.4 When FðtÞ is IFR in Example 2.4, show that the MTTF ln of a parallel system with n units has the inequality [2, p. 65, 6, p. 9] l1  ln  l1

n X 1 k¼1

k

:

2.5 Check numerically in Example 2.4 that when the failure time has a Weibull a distribution FðtÞ ¼ 1  et ða 1Þ; the MTTF of a parallel system is given by approximately [20], n X 1

k k¼1

!1=a ;

ðC þ log nÞ1=a :

2.6 When the acquisition cost of one unit is c1 and the replacement cost of a failed system is c2 in Example 2.4, show that the expected cost rate is [6, p. 8] CðnÞ ¼

nc1 þ c2 P ð1=kÞ nk¼1 ð1=kÞ

ðn ¼ 1; 2; . . .Þ;

and derive analytically an optimum n that minimizes CðnÞ: 2.7 When the failure time is exponential, obtain the MTTF of a k-out-of-n system. 2.8 A harmonic series in (2.8) appears in a record time [8, p. 47]: Let X1 ; X2 ; . . . be independent and identically distributed random variables. It is said that a record occurs at time n if Xn [ maxfX1 ; X2 ; . . .; Xn1 g: In addition, let Nn denote the total number of records until time n: Then, show [8, p. 286] that   n n X X 1 1 1 EfNn g ¼ ; VfNn g ¼ 1 : k k k k¼1 k¼1 This is applied to a breaking test [7]: Suppose that there are n wooden beams. To know the strength of which beams can bear a minimum burden, we make the following experiment: We record a burden B1 on which the first beam breaks. Next, we bear B1 on the second beam. If the beam does not break, then B2 [ B1 : Conversely, if it breaks, we record a burden B2 : Similarly, we bear minfB1 ; B2 g on the third beam. If the beam does not break, then B3 [ minfB1 ; B2 g; and conversely, if it breaks, we record B3 : Then, in the same method of a record time, Pn show that the expected number of broken beams by such experiment is k¼1 ð1=kÞ: Compute the expected number of broken beams when n ðn ¼ 10; 100; 1000Þ experiments are made. 2.9 Show that the limiting distribution of binomial distribution is a Poisson distribution [4, p. 40]. 2.10 For a n units parallel system in Example 2.6, where each unit has an identical failure distribution FðtÞ; the expected cost rate is [6, p. 84]

44

2 Poisson Processes

CðnÞ ¼ c2 þ ðc1  c2 Þ

Z1

WðtÞd½FðtÞn þ c0 n ðn ¼ 0; 1; 2; . . .Þ:

0 

An optimum number n is given by a unique minimum that satisfies Z1 c0 ½FðtÞn FðtÞdWðtÞ  : c1  c2 0

In particular, when WðtÞ ¼ 1  ext and FðtÞ ¼ 1  ekt ; the above equation is n   X x c0 n :  ð1Þk k x þ ðk þ 1Þk c1  c2 k¼0 Derive the above equations, compute optimum n that minimize CðnÞ for k=x and c0 =ðc1  c2 Þ; and compare them with Table 2.3. 2.11 Suppose that failed units from two factories arrive at a repair shop according to Poisson processes with rates k1 and k2 in Example 2.8. Show how many number of spare units should be provided in order to assure with probability a that the system will remain operating in time t in Example 2.8. 2.12 Suppose that failures are roughly classified into two kinds of failures; minor failure and major failure in Example 2.9. Minor and major failures occur at the rates of 0.90 and 0.10, respectively. When kt ¼ 1; 2; 3; compute the probabilities that the number of minor failures is k ðk ¼ 0; 1; 2; 3Þ: 2.13 When the objective function in Example 2.11 is [6, p. 41], CðNÞ ¼ AN þ

B N

or

CðNÞ ¼ AN B1=N

ðN ¼ 1; 2; . . .Þ

for parameters A; B [ 0; show that an optimum N  is a unique minimum that satisfies N X B k : 2A k¼1 2.14 Consider the multiple backward times Tk ðk ¼ 1; 2; . . .; NÞ; where Sk  Pk j¼1 Tj ; S0  0; and SN  t in Example 2.12 [21]. 2.15 Prove (2.39). When 10 failures have occurred in 100 h, derive the probabilities that kðk ¼ 0; 1; 2Þ failures have occurred in 10 h in Example 2.10, and the third failure has occurred until 50 h in (2.38). Rt 2.16 Prove in p. 29 that when HðtÞ  0 hðuÞdu; i.e., hðtÞ  H 0 ðtÞ; ½HðtÞn ¼ n!

Zt 0

½HðuÞn1 hðuÞdu ðn  1Þ!

ðn ¼ 1; 2; . . .Þ:

2.6 Problems 2

45

2.17 Prove in (2.51) that when hðtÞ increases strictly, PN1 R 1 l k¼0 0 Pk ðtÞdt R1  ðN  1Þ [ R 1 0 PN ðtÞdt 0 PN ðtÞdt where Pk ðtÞ ¼ f½HðtÞk =k!geHðtÞ and l  2.18 Derive in Example 2.15 that Z1 0

1 CðN þ 1=aÞ ; PN ðtÞdt ¼ 1=a a k CðN þ 1Þ

N 1 Z X

R1 0

ðN ¼ 2; 3; . . .Þ;

eHðtÞ dt:

1

k¼0

Pk ðtÞdt ¼

0

CðN þ 1=aÞ : k1=a CðNÞ

2.19 Prove in Sect. 2.4.2 that the solution of the differential equation dHðtÞ ¼ bðtÞ½a  HðtÞ; dt is given by (2.54). 2.20 Derive EfZðtÞg; VfZðtÞg from (2.57), and EfYg; VfYg from (2.60) in Example 2.16. 2.21 Derive all equations in Example 2.17.

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18.

Kingman JFC (1993) Poisson processes. Oxford University Press, Oxford Barlow RE, Proschan F (1965) Mathematical theory of reliability. Wiley, New York Nakagawa T (2005) Maintenance theory of reliability. Springer, London Osaki S (1992) Applied stochastic system modeling. Springer, Berlin Çinlar E (1975) Introduction to stochastic processes. Prentice-Hall, Englewood Cliffs Nakagawa T (2008) Advanced reliability models and maintenance policies. Springer, London Havil J (2003) GAMMA: exploring Euler’s constant. Princeton University Press, Princeton Ross SM (1983) Stochastic processes. Wiley, New York Beichelt FE, Fatti LP (2002) Stochastic processes and their applications. CRC Press, Boca Raton Tijms HC (2003) A first course in stochastic models. Wiley, Chichester Yamada S (1994) Software reliability models: fundamentals and applications. JUSE Press, Tokyo Finkelstein M (2008) Failure rate modelling for reliability and risk. Springer, London Murthy DNP, Xie M, Jiang R (2004) Weibull models. Wiley, Hoboken Goel AL, Okumoto K (1979) Time-dependent error-detection rate model for software reliability and other performance measures. IEEE Trans Reliab R-28:206–211 Pham H (2006) System software reliability. Springer, London Pham, H (ed) (2008) Recent advances in reliability and quality in design. Springer, London Nakagawa T (2007) Shock and damage models in reliability theory. Springer, London Cox DR (1962) Renewal theory. Methuen, London

46

2 Poisson Processes

19. Esary JD, Marshall AW, Proschan F (1973) Shock models and wear processes. Ann Probab 1:627–649 20. Yun WY, Nakagawa T (2010) Note on MTTF of a parallel system. In: 16th ISSAT International Conference on Reliability and Quality in Design, pp 235–237 21. Chen M, Mizutani S, Nakagawa T (2010) Optimal backward and backup policies in reliability theory. J Oper Res Soc Jpn 53:101–118

Chapter 3

Renewal Processes

In Chap. 2, we have defined a Poisson process and shown that the interarrival times are distributed exponentially. However, in general, such times have arbitrary distributed random variables, that is called a renewal process. In most reliability systems, they may deteriorate with age and break at last, and become like new by suitable replacement or maintenance. A renewal process arises from the study of self-renewing aggregates, and a renewal theory plays an important role in the analysis of stochastic models with the total sum of independent nonnegative random variables. The results can be applied to a wide variety of both theoretical and practical probability problems in engineering, natural, economical, and social sciences. A renewal process in this chapter is the most fundamental process, and a renewal theory is an important theory in stochastic processes. Many basic stochastic systems form renewal processes essentially. Renewal processes are the largest and most crucial chapter in stochastic processes for studying reliability theory because learning a reliability theory is learning a renewal theory: In Sect. 3.1, we define renewal processes by generalizing Poisson processes. Renewal processes are characterized by renewal functions. In Sect. 3.2, it is shown that a renewal function is obtained by solving a renewal equation, and its properties are given in the name of a renewal theory. Some theorems in a renewal theory are cited as theorems without mathematical proof because it needs more mathematical preparation. But, it would be sufficient on theorem forms to apply them to reliability theory. The most fundamental policy in maintenance models is a replacement policy. In Sect. 3.3, we show how to apply a renewal theory to standard replacement policies. Using the techniques and results of a renewal theory, optimum policies for some replacement models are discussed analytically. In Sect. 3.4, we introduce an alternating renewal process that repeats on and off, up and down, or between operating and failure states. A one-unit system with repair maintenance which is the most basic reliability system is given as a typical example, and its properties are examined and the interval reliability is given by using the results of an alternating renewal process. Finally, in Sect. 3.4, we take up a geometric and a discrete renewal processes as examples of modified renewal processes. A geometric process is the process whose

T. Nakagawa, Stochastic Processes, Springer Series in Reliability Engineering, DOI: 10.1007/978-0-85729-274-2_3, Ó Springer-Verlag London Limited 2011

47

48

3 Renewal Processes

interarrival times decrease or increase geometrically, and a discrete process is defined to be a renewal process with discrete interarrival times.

3.1 Definition of Renewal Process We define renewal processes by generalizing interarrival times Xk in a Poisson process: Consider a sequence of independent and nonnegative random variables fX1 ; X2 ; . . .g in Fig. 2.1, where it is assumed that PrfXk ¼ 0g\1 for all k because of avoiding trivial cases. Suppose that X2 ; X3 ; . . . have an arbitrary identical distribution FðtÞ with finite mean l; however, X1 possibly has a different distribution F1 ðtÞ with finite l1 from FðtÞ; in which both F1 ðtÞ and FðtÞ are not degenerate at time t ¼ 0 and F 1 ðtÞ  1  F1 ðtÞ: P We introduce the total sum of random variables Sn  nk¼1 Xk ; where S0  0 for convenience, and define NðtÞ  maxn fSn  tg that represents the number of renewals in the time interval ½0; t; where the process fNðtÞg is generally called a counting process. Clearly, from (2.16), fSn  tg , fNðtÞ  ng: It is said that the nth renewal occurs at time Sn : We call epochs at which Sn ends renewal points, renewal times or regeneration points, and the time from the beginning of Sn to its end a renewal cycle with time interval Xn : We use properly two words of renewal point and regeneration point in the right place. Note that we have the memoryless property only at renewal points except F1 ðtÞ and FðtÞ are exponential distributions. In other words, the process starts newly at each renewal point. We will use the terms of events and renewals interchangeably. We define renewal processes according to three types of F1 ðtÞ: Definition 3.1 (i) If FðtÞ ¼ F1 ðtÞ; i.e., all random variables are identically distributed, then the counting process fNðtÞg is called an ordinary renewal process or renewal process for short. (ii) If F1 ðtÞ and FðtÞ are not the same, then the counting process fNðtÞg is called a modified or delayed renewal process. Rt (iii) If F1 ðtÞ  0 F 1 ðuÞdu=l that will be given in (3.24), then the counting process fNðtÞg is called an equilibrium or stationary renewal process. Example 3.1 (Reliability systems) Consider a unit that is replaced with a new one upon failures: A unit begins to operate immediately after the replacement whose time is negligible. Suppose that the failure distribution of each new unit is FðtÞ: If a new unit is installed at time t ¼ 0; then all failure times have the same distribution, and hence, we have an ordinary renewal process. On the other hand, if an old unit is in use at time t ¼ 0; then X1 represents its residual lifetime and could be different from the failure time of a new unit, and hence, we have a modified renewal process. In particular, if the time origin point at which we start observing

3.1 Definition of Renewal Process

49

the process is sufficiently large after the installation of an initial operating unit, it may be considered that the process has already entered the stationary state and Rt the failure distribution of X1 is given by 0 F 1 ðuÞdu=l; then we have an equilibrium renewal process. We may consider an operating unit that is repaired upon failures and whose repair time may be assumed to be negligible because they are small usually compared with failure times. If we observe only the operating states that repeat successively, then it may be said that most reliability systems can form renewal processes. Therefore, renewal processes are the most fundamental stochastic process in reliability theory. In general, if two positive random variables X and Y are independent and have the respective distributions FðtÞ  PrfX  tg and GðtÞ  PrfY  tg; then the sum of X and Y has the distribution PrfX þ Y  tg ¼

Zt

Fðt  uÞdGðuÞ ¼

0

Zt

Gðt  uÞdFðuÞ  FðtÞ  GðtÞ;

0

and its LS transform is, from Appendix A, Z1

est d PrfX þ Y  tg ¼ EfesðXþYÞ g ¼ EfesX gEfesY g

0

¼

Z1

est dFðtÞ

0

Z1

est dGðtÞ ¼ F  ðsÞG ðsÞ;

0

i.e., Z1

est d½FðtÞ  GðtÞ ¼ F  ðsÞG ðsÞ;

0

where the asterisk is called the pairwise Stieltjes convolution. Denoting F ð0Þ ðtÞ  1 for t  0 and F ðnÞ ðtÞ  PrfX2 þ X3 þ    þ Xnþ1  tg Zt ¼ PrfX3 þ X4 þ    þ Xnþ1  t  ugd PrfX2  ug 0

¼

Zt

F ðn1Þ ðt  uÞdFðuÞ

0

¼ FðtÞ      FðtÞ

ðn ¼ 1; 2; . . .Þ;

ð3:1Þ

50

3 Renewal Processes

F ðnÞ ðtÞ is called the n-fold Stieltjes convolution of FðtÞ with itself. That is, F ðnÞ ðtÞ represents the distribution of the total sum of independent and identical random P ðnÞ variables nþ1 k¼2 Xk : In addition, the LS transform of F ðtÞ is, from Appendix A, Z1 est dF ðnÞ ðtÞ ¼ ½F  ðsÞn ðn ¼ 0; 1; 2; . . .Þ; ð3:2Þ 0

R1

where note that 0 est dF ð0Þ ðtÞ ¼ 1 for any ReðsÞ  0: Furthermore, from the relation of Sn and NðtÞ; PrfNðtÞ ¼ 0g ¼ PrfX1 [ tg ¼ 1  F1 ðtÞ; PrfNðtÞ ¼ ng ¼ PrfSn  t and Snþ1 [ tg ¼ PrfSn  tg  PrfSnþ1  tg ¼ F1 ðtÞ  F ðn1Þ ðtÞ  F1 ðtÞ  F ðnÞ ðtÞ

ðn ¼ 1; 2; . . .Þ:

ð3:3Þ

3.2 Renewal Function We want to know properties of the number of renewals in ½0; t and define its expected number as MðtÞ  EfNðtÞg; that is called the renewal function, and the derivative of MðtÞ is mðtÞ  dMðtÞ=dt; that is called the renewal density. From (3.3) (Problem 3.1), the renewal function is MðtÞ  ¼

1 X n¼1 1 X

n PrfNðtÞ ¼ ng ¼

1 X

PrfNðtÞ  ng

n¼1

PrfSn  tg ¼

n¼1

1 X

F1 ðtÞ  F ðn1Þ ðtÞ:

ð3:4Þ

n¼1

It is fairly easy to show that MðtÞ is finite for all t  0 because PrfXk ¼ 0g\1 (Problem 3.1). Furthermore, from the notations of the convolutions in (3.1) and (3.4), MðtÞ ¼ F1 ðtÞ þ

1 Z X n¼1

mðtÞ ¼ f1 ðtÞ þ

Zt

t

F ðnÞ ðt  uÞdF1 ðuÞ ¼

0

Zt 0

½1 þ M0 ðt  uÞdF1 ðuÞ; ð3:5Þ

m0 ðt  uÞf1 ðuÞdu;

0

where M0 ðtÞ is the renewal function of an ordinary renewal process with distriP P1 ðnÞ ðnÞ bution FðtÞ; i.e., M0 ðtÞ  1 ðtÞ; and n¼1 F ðtÞ; m0 ðtÞ  dM0 ðtÞ=dt ¼ n¼1 f f ðtÞ and f1 ðtÞ are the respective density functions of FðtÞ and F1 ðtÞ: Such types of equations as (3.5) are called renewal-type equations and appear frequently in the

3.2 Renewal Function

51

analysis of reliability models, because most systems are renewed after maintenance as shown in Sect. 3.3. It is of great importance in reliability analysis to search for renewal points and set up renewal equations. Generally, solutions of renewal equations can be computed by only numerical methods, however, they are obtained explicitly in the form of LS transforms that will be shown in Sect. 3.2.3. The LS transform of MðtÞ is, from (3.4) or (3.5), Z1 F1 ðsÞ  est dMðtÞ ¼ M ðsÞ  : ð3:6Þ 1  F  ðsÞ 0

Thus, MðtÞ is determined by F1 ðtÞ and FðtÞ: When F1 ðtÞ and FðtÞ are identical, i.e., the process forms an ordinary renewal process, MðtÞ ¼ M0 ðtÞ; and (3.6) implies F  ðsÞ ¼ M  ðsÞ=½1 þ M  ðsÞ: Therefore, FðtÞ and MðtÞ determines uniquely with each other because the LS transform determines its original function uniquely. In particular, when the process forms a stationary renewal process, from (iii) of Definition 3.1, F1 ðsÞ

¼

Z1

est dF1 ðtÞ ¼

1  F  ðsÞ ; ls

0

and hence, from (3.6) and Table A.2 in Appendix A, M  ðsÞ ¼

1 ; ls

i.e.;

t MðtÞ ¼ : l

After this, it is assumed that F1 ðtÞ ¼ FðtÞ; i.e., we deal with only an ordinary renewal process. Example 3.2 (Gamma and normal distributions) Suppose that FðtÞ is a gamma  Pk1  distribution, i.e., FðtÞ ¼ 1  j¼0 ðktÞj =j! ekt ðk ¼ 1; 2; . . .Þ: In particular, when k ¼ 1; from Sect. 2.1.2, k M  ðsÞ ¼ ; s

i.e.,

MðtÞ ¼ kt;

that is derived directly, as shown in (2.18). When k ¼ 2; FðtÞ ¼ 1  ð1 þ ktÞekt ; and  2 k  F ðsÞ ¼ ; sþk and hence, from (3.6), M  ðsÞ ¼

k2 k k : ¼  sðs þ 2kÞ 2s 2ðs þ 2kÞ

52

3 Renewal Processes

Using the LS transform method in Appendix A such that Z1 e

st

Z1

a dðatÞ ¼ ; s

0

est dð1  eat Þ ¼

a ; sþa

0

the inverse LS transform of M  ðsÞ is MðtÞ ¼

 kt 1  1  e2kt : 2 4

In general, because the probability of n renewals in ½0; t for a renewal process is equal to the probability of either nk; nk þ 1; . . .; or nk þ k  1 events in ½0; t for a Poisson process with rate k (Fig. 3.1) [1, p. 57], PrfNðtÞ ¼ ng ¼

nkþk1 X j¼nk

ðktÞj kt e ; j!

and hence, because mðtÞdt is the probability of a renewal in ðt; t þ dt; " # 1 X ðktÞnk1 kt kdt: e mðtÞdt ¼ ðnk  1Þ! n¼1 Thus, a renewal function is MðtÞ ¼

Zt 0

¼

1 Z X kðkuÞnk1 ku mðuÞdu ¼ e du ðnk  1Þ! n¼1 t

0

1 X 1 X ðktÞj n¼1 j¼nk

j!

ekt :

ð3:7Þ

On the other hand, from Sect. 2.1.2 and (3.6), the LS transform of MðtÞ for a gamma distribution with order k is M  ðsÞ ¼

kk ðs þ kÞk  kk

;

and inverting its LS transform [1, p. 57, 2, p. 121, 3]

Fig. 3.1 Relationship between Poisson and renewal processes for a gamma distribution with order k

3.2 Renewal Function

53

MðtÞ ¼

k1 i kt 1 X hj h 1  ektð1hj Þ ; þ k k j¼1 1  hj

ð3:8Þ

where hj  e2pji=k ðj ¼ 1; 2; . . .; k  1Þ is the distinct root of sk ¼ 1 and i is an imaginary order. In particular, when k ¼ 3 [4, p. 162] (Problem 3.2), pffiffiffi 

3 1 2 p kt þ ; MðtÞ ¼ kt  1 þ pffiffiffi e1:5kt sin 2 3 3 3 and when k ¼ 4; MðtÞ ¼



 pffiffiffi 1 3 1 p kt  þ e2kt þ 2ekt sin kt þ : 4 2 2 4

Figure 3.2 draws the renewal functions MðtÞ for k ¼ 1; 2; 3 when k ¼ 1: Rt Suppose generously that FðtÞ ¼ 0 ½kðkxÞa1 ekx =CðaÞdx for parameters k [ 0 and a [ 0: Because the sum of two random variables with gamma distributions with the respective parameters ða; kÞ and ðb; kÞ has a gamma distribution with parameter ða þ b; kÞ (Problem 3.3), Zt kðkxÞna1 kx e dx; F ðnÞ ðtÞ ¼ ð3:9Þ CðnaÞ 0

and hence, 1 Z X kðkxÞna1 kx e dx; MðtÞ ¼ CðnaÞ n¼1 t

0

that agrees with (3.7) when a ¼ k ðk ¼ 1; 2; . . .Þ: Fig. 3.2 Renewal functions MðtÞ for k ¼ 1; 2; 3

ð3:10Þ

54

3 Renewal Processes

When Xk ðk ¼ 1; 2; . . .Þ is normally distributed with mean l and variance r2 for l r; because Sn is normally distributed with mean nl and variance nr2 ; from (3.4),   1 1 X X t  nl ; ð3:11Þ PrfSn  tg ¼ U pffiffiffi MðtÞ ¼ nr n¼1 n¼1 where UðxÞ is a standard normal distribution with mean 0 and variance 1 (Problems 3.4 and 3.5). P Next, we derive the variance V fNðtÞg [5, p. 89]: Because n2 ¼ 2 nj¼1 j  n; EfNðtÞ2 g ¼

1 X

n2 PrfNðtÞ ¼ ng ¼ 2

n¼1 1 X

¼2

1 X

n PrfNðtÞ  ng  MðtÞ

n¼1

n PrfSn  tg  MðtÞ ¼ 2

n¼1

1 X

nF ðnÞ ðtÞ  MðtÞ:

n¼1

Taking the LS transform on both sides above, Z1

1 n o X est dE NðtÞ2 ¼ 2 n½F  ðsÞn M  ðsÞ n¼1

0



F  ðsÞ 2 F  ðsÞ ¼2 þ 1  F  ðsÞ 1  F  ðsÞ ¼ 2½M  ðsÞ2 þM  ðsÞ:

ð3:12Þ

Inverting (3.12), n o E NðtÞ2 ¼ 2MðtÞ  MðtÞ þ MðtÞ; and hence, V fNðtÞg ¼ 2MðtÞ  MðtÞ þ MðtÞ  ½MðtÞ2 :

ð3:13Þ

Because MðtÞ  MðtÞ  ½MðtÞ2 ; VfNðtÞg  MðtÞ þ ½MðtÞ2 : In particular, when FðtÞ ¼ 1  ekt ; V fNðtÞg ¼ kt; that agrees with (2.18). Furthermore, when FðtÞ ¼ 1  ð1 þ ktÞekt ; from Example 3.2 and (3.13) (Problem 3.6), V fNðtÞg ¼ 

kt kt 1 þ ð1  e2kt Þ þ ð1  e4kt Þ: 4 2 16

3.2 Renewal Function

55

We summarize some important limiting theorems and results of renewal theory [5–7]: R1 Theorem 3.1 If l2  0 t2 dFðtÞ\1 and r2  l2  l2 ;  2  t r 1 MðtÞ ¼ þ  þ oð1Þ as t ! 1; ð3:14Þ 2l2 2 l R1 and if l3  0 t3 dFðtÞ\1;  4  r2 t 5r 2r2 3 2l3 þ þ  V fNðtÞg ¼ 3 þ þ oð1Þ as t ! 1; ð3:15Þ 4l4 l2 l 4 3l3 where oðhÞ=h ! 0 as h ! 0: Proof Expanding F  ðsÞ with respect to s; F  ðsÞ ¼ 1  ls þ

   1 2 1 r þ l2 s2  l3 s3 þ o s3 : 2 3!

Substituting F  ðsÞ for (3.6) and arranging them,  2  1 r  l2  þ oð1Þ; M ðsÞ ¼ þ 2l2 sl    4  1 1 r2  l 2 3r r2 3 l3  þ þ oð1Þ: ½M  ðsÞ2 ¼ 2 2 þ þ þ l3 4l4 2l2 4 3l3 sl s

ð3:16Þ

Inverting M  ðsÞ follows (3.14). Furthermore, substituting (3.16) for (3.12) and inverting it, we have (3.15) from (3.13) and (3.14) (Problem 3.6). h From this theorem, MðtÞ and mðtÞ are approximately given by t r2 1 1 þ 2  ; mðtÞ ; l 2l 2 l r2 t 5r4 2r2 3 2l3 V fNðtÞg 3 þ 4 þ 2 þ  3 4l l l 4 3l MðtÞ

ð3:17Þ

for large t: Furthermore, if r  l; then MðtÞ

t 1  : l 2

Theorem 3.2 If l and r2 are finite, MðtÞ 1 ! ; t l

1 mðtÞ ! ; l

V fNðtÞg r2 ! 3 l t

as t ! 1:

ð3:18Þ

This theorem that is easily derived from Theorem 3.1 is called an elementary renewal theorem and is the most important one in a renewal process. This is also

56

3 Renewal Processes

applied not only in the reliability theory but in also many other practical fields. Another proof of Theorem 3.2 that is more direct than that of the above was shown [1, p. 55]. A random variable X with distribution FðtÞ is said to be lattice if there exists P h [ 0 such that 1 n¼0 PrfX ¼ nhg ¼ 1: We give Blackwell’s theorem [8, p. 347]: Theorem 3.3 (Blackwell’s theorem) If X is not lattice, then for h [ 0; h lim ½Mðt þ hÞ  MðtÞ ¼ : l

t!1

This theorem shows that the expected number of renewals in ½t; t þ h tends to h=l approximately as t ! 1; i.e., it is proportional to the time duration where the proportional constant is 1=l: Furthermore, we introduce a renewal reward process [6, p. 77]. For instance, if we consider the total reward produced by the successive production of some system, then the process is said to form a renewal reward process, where the successive production can be described by a renewal process and the total rewards caused by production may be additive. Chapter 6 will deal in detail with a cumulative process that is another name of a renewal reward process. Define that some reward Wk is earned at the kth renewal point ðk ¼ 1; 2; . . .Þ: When a sequence of pairs fXk ; Wk g is independent and identically distributed, PNðtÞ ZðtÞ  k¼1 Wk is the total reward earned in ½0; t: Theorem 3.4 If EfWg  EfWk g\1; EfZðtÞg EfWg ! t l

as t ! 1:

This is proved by intuition because if NðtÞ and W are independent, EfZðtÞg ¼ EfNðtÞgEfWg ¼ MðtÞEfWg: Thus, using Theorem 3.2, EfZðtÞg MðtÞ EfWg ¼ EfWg ! t t l

as t ! 1:

The detailed proof of Theorem 3.4 was shown [6, p. 78]. This theorem shows that if one cycle is denoted by the time interval between renewals, the expected reward per unit of time for an infinite time span is equal to the expected reward per cycle, divided by its mean time, i.e., lim

t!1

EfZðtÞg Expected reward of one renewal cycle ¼ : t Mean time of one renewal cycle

ð3:19Þ

3.2 Renewal Function

57

This function is adopted by appropriate objective functions of maintenance models where the reward of maintenance is given by costs. There will exist many optimization problems with aim to minimize them in reliability theory [1], as shown in Sect. 3.3.2 Applying the asymptotic formulae of (3.17) to the usual central limit theorem, we have a so-called central limit theorem for a renewal process: Theorem 3.5 (

NðtÞ  t=l lim Pr pffiffiffiffiffiffiffiffiffiffiffiffiffi  x t!1 r2 t=l3

)

1 ¼ pffiffiffiffiffiffi 2p

Zx

eu

2

=2

du;

ð3:20Þ

1

i.e., NðtÞ is asymptotically normally distributed with mean t=l and variance r2 t=l3 for large t: The detailed proof was found in [5, p. 93, 6, p. 62]. Example 3.3 (Asymptotic normal distribution) We compute approximately the number n when the probability that the number of failures is less than n in t ¼ 10; 000 h of operation is given by 0.90. The failure distribution of each unit is unknown, but, it is estimated from the sample data that the mean and variance are l ¼ 1; 000 h and r2 ¼ 200; 000 ðhÞ2 ; respectively. Then, from Theorem 3.5, when t ¼ 10; 000; NðtÞ  t=l Nð10; 000Þ  10 pffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ 2 r2 t=l3 is approximately normally distributed with mean 0 and variance 1. Thus,

 Nð10; 000Þ  10 n  10 pffiffiffi PrfNð10; 000Þ  ng ¼ Pr  pffiffiffi 2 2 pffiffi ðn10Þ= 2 Z 1 2

pffiffiffiffiffiffi eu =2 du ¼ 0:90: 2p 1

pffiffiffiffiffiffi R u0 u2 =2 e du ¼ 0:90; Because u0 ¼ 1:28 that satisfies ð1= 2pÞ 1 pffiffiffi n 10 þ 1:28 2 ¼ 11:81: In this case, it can be said that with probability more than 0.90, the system with 12 units, i.e., the MTTF is 1:2 104 h, including the first unit, will continue to operate for 104 hours without any supply (Problem 3.7). Theorem 3.6 (Key or Fundamental renewal theorem) If gðtÞ is bounded in ½0; 1Þ and FðtÞ is not lattice, then [2, p. 123, 5, p. 95, 9, p. 295],

58

3 Renewal Processes

Zt lim

t!1

1 gðt  uÞdMðuÞ ¼ l

0

Z1

ð3:21Þ

gðtÞdt: 0

This is proved intuitively that from Theorem 3.3 [6, p. 65], Mðt þ hÞ  MðtÞ dMðtÞ 1 ¼ lim ¼ ; t!1 dt h l Zt Z1 Zt 1 1 gðt  uÞdMðuÞ ¼ lim gðt  uÞdu ¼ gðtÞdt: lim t!1 x!1 l l lim lim

h!0 t!1

0

0

0

Recall that mðtÞ ¼ dMðtÞ=dt is called the renewal density and represents the expected number of renewals per unit of time.

3.2.1 Age and Residual Life Distributions Let dðtÞ  t  SNðtÞ and cðtÞ  SNðtÞþ1  t that denote the shortage and excess random variables (Fig. 3.3), respectively. In reliability theory, dðtÞ represents the current age and cðtÞ represents the residual or remaining life of an operating unit. Thus, bðtÞ  dðtÞ þ cðtÞ ¼ SNðtÞþ1  SNðtÞ ¼ XNðtÞþ1 represents the total life of the unit. Theorem 3.7 (i) PrfcðtÞ  xg ¼ Fðt þ xÞ 

Zt

Fðt þ x  uÞdMðuÞ:

ð3:22Þ

0

(ii)

PrfdðtÞ  xg ¼

FðtÞ 

R tx 0

Fðt  uÞdMðuÞ 1

for x  t; for x [ t:

ð3:23Þ

(iii) The limiting distribution is 1 lim PrfdðtÞ  xg ¼ lim PrfcðtÞ  xg ¼ t!1 t!1 l

Zx FðuÞdu: 0

Fig. 3.3 Age dðtÞ and residual life cðtÞ in a renewal process

ð3:24Þ

3.2 Renewal Function

59

Proof From Fig. 3.3 [1, p. 58, 5, p. 97],   PrfcðtÞ  xg ¼ Pr SNðtÞ  t\SNðtÞþ1  t þ x 1 X Prft\Skþ1  t þ x; NðtÞ ¼ kg ¼ k¼0

¼

1 X

PrfSk  t\Skþ1  t þ xg

k¼0

¼

1 Z X k¼0

¼

Prft  u\Xkþ1  t þ x  ugd PrfSk  ug

0

1 Z X k¼0

t

:

t ðkÞ

½Fðt þ x  uÞ  Fðt  uÞdF ðuÞ

0

¼ Fðt þ xÞ  FðtÞ þ

Zt

½Fðt þ x  uÞ  Fðt  uÞdMðuÞ

0

¼ Fðt þ xÞ 

Zt

Fðt þ x  uÞdMðuÞ:

0

Furthermore, because both events fdðtÞ [ xg and fcðt  xÞ [ xg mean that there is no renewal in ½t  x; t; we have the equality PrfdðtÞ [ xg ¼ Prfcðt  xÞ [ xg; that implies (3.22) from (3.23). Applying Theorem 3.6 to (3.22) and (3.23), 1 lim PrfcðtÞ  xg ¼ 1  t!1 l

Z1

1 Fðt þ xÞdt ¼ l

0

Zx FðuÞdu; 0

and lim PrfdðtÞ  xg ¼ lim Prfcðt  xÞ  xg ¼ lim PrfcðtÞ  xg;

t!1

t!1

t!1

that follows (3.24). This means that when the process started at t ¼ 1; looking backwards and forwards from the present time t; the age and the residual life have the same distribution in a renewal process. h From (i) of Theorem 3.7 (Problem 3.8), 2 3 Zt Z1 EfcðtÞg ¼ 4FðxÞ þ Fðx  uÞdMðuÞ5dx t

0

¼ l½1 þ MðtÞ  t;

ð3:25Þ

60

3 Renewal Processes

that is also easily derived because   EfcðtÞg ¼ E SNðtÞþ1  t ¼ l½1 þ MðtÞ  t: It is also of interest that the mean of the above limiting distribution in (3.24) is (Problem 3.8) 1 l

Z1

tFðtÞdt ¼

l l2  l2 l r2 ¼ þ ; þ 2l 2 2 2l

ð3:26Þ

0

that is greater than half of the mean interval time l: In general [2, p. 128, 6 p. 69] (Problem 3.9),

lim PrfbðtÞ  xg ¼

t!1

Zx

1 l

udFðuÞ:

ð3:27Þ

0

Example 3.4 (Exponential distribution) When FðtÞ ¼ 1  ekt ; PrfdðtÞ  xg ¼ PrfcðtÞ  xg ¼ lim PrfdðtÞ  xg t!1

¼ lim PrfcðtÞ  xg ¼ 1  ekx ; t!1

and its mean time is 1=k: In addition, PrfbðtÞ  xg ¼ PrfdðtÞ þ cðtÞ  xg ¼ lim PrfbðtÞ  xg t!1

¼ 1  ð1 þ kxÞe

kx

;

i.e., the failure time of the unit that is operating at time t has a gamma distribution with order 2, and its mean life is twice as much as the usual unit. Example 3.5 (Uniform distribution) Suppose that FðtÞ is distributed uniformly on ½0; 10; i.e., FðtÞ ¼ t=10 for 0  t  10 and l ¼ 5: In this case, from (3.24), 1 lim PrfcðtÞ  xg ¼ t!1 5

Zx 

1

u x x2 du ¼  10 5 100

0

for 0  x  10; and its mean is 10=3 [ l=2 ¼ 2:5: From (3.27), lim PrfbðtÞ  xg ¼

t!1

for 0  x  10 and its mean is 20=3:

x2 100

3.2 Renewal Function

61

3.2.2 Expected Number of Failures The renewal function MðtÞ in a renewal process and the cumulative hazard function HðtÞ in a nonhomogeneous Poisson process play a great role in reliability theory. We summarize and compare the properties of MðtÞ and HðtÞ: Suppose that the unit begins to operate at time 0 and fails according to a failure distribution FðtÞ with finite mean l and a density function f ðtÞ  dFðtÞ=dt: 3.2.2.1 Properties of MðtÞ The unit is repaired immediately at failures and operates again like new. (a) Renewal function MðtÞ represents the expected number of failures in ½0; t and is, from (3.5), Zt

MðtÞ ¼ FðtÞ þ

Mðt  uÞdFðuÞ

0 1 X

¼

F ðnÞ ðtÞ ¼ FðtÞ þ FðtÞ  FðtÞ þ    :

n¼1

The LS transform is, from (3.6), M  ðsÞ ¼

F  ðsÞ : 1  F  ðsÞ

Thus, FðtÞ and MðtÞ determine uniquely with each other. However, it is not easy to compute MðtÞ from FðtÞ except FðtÞ is a gamma distribution (Example 3.2). (b) Renewal density is mðtÞ ¼ dMðtÞ=dt; and mðtÞdt represents the probability of a renewal occurring in ½t; t þ dt; that is given by a renewal equation mðtÞ ¼ f ðtÞ þ

Zt

mðt  uÞf ðuÞdu;

0

¼

1 X

f ðnÞ ðtÞ ¼ f ðtÞ þ f ðtÞ  f ðtÞ þ    :

n¼1

The Laplace transform is 

m ðsÞ ¼

Z1 0

where f  ðsÞ 

R1 0

est f ðtÞdt:

est mðtÞdt ¼

f  ðsÞ ; 1  f  ðsÞ

62

3 Renewal Processes

3.2.2.2 Properties of HðtÞ The unit is repaired at failures and undergoes minimal repair defined in Sect. 2.3. (a) Cumulative hazard function HðtÞ represents the expected number of failures in ½0; t and is, from Sect. 2.3, FðtÞ ¼ 1  eHðtÞ ;

HðtÞ ¼  log FðtÞ:

i.e.,

Hence, ½HðtÞ2 ½HðtÞ3 þ þ ; 2! 3! ½FðtÞ2 ½FðtÞ3 þ þ : HðtÞ ¼ FðtÞ þ 2 3 FðtÞ ¼ HðtÞ 

Clearly, 0

hðtÞ  H ðtÞ ¼ f ðtÞ½1 þ FðtÞ þ FðtÞ2 þ   : (b) Hazard rate or failure rate function is hðtÞ  dHðtÞ=dt; and hðtÞdt represents the probability of a failure occurring in ½t; t þ dt; given that the age is t; and the unit undergoes minimal repair at each failure that is given by Rt f ðtÞ  hðuÞdu ; i.e., hðtÞ ¼ FðtÞ ¼ 1  e 0 : 1  FðtÞ When FðtÞ ¼ 1  ekt ; MðtÞ ¼ HðtÞ ¼ kt;

mðtÞ ¼ hðtÞ ¼ k:

3.2.2.3 Inequalities of MðtÞ and HðtÞ It is important to investigate MðtÞ and HðtÞ for maintenance models [11, p. 5] when FðtÞ is IFR, i.e., hðtÞ increases with t: (a) When FðtÞ with finite mean l and variance r2 is IFR [1, p. 54] (Problem 3.10), l  r; MðxÞ  Mðt þ xÞ  MðtÞ; x MðxÞ  lim ½Mðt þ xÞ  MðtÞ ¼ ; t!1 l t V½NðtÞ  MðtÞ  ; l t t tFðtÞ t  1  MðtÞ  R t  :  1Rt l l FðuÞdu FðuÞdu 0 0

3.2 Renewal Function

63

Because mðtÞdt is the probability of a renewal in ½t; t þ dt; mðtÞ ¼

Zt

hðxÞd PrfdðtÞ  xg:

0

Thus, when FðtÞ is IFR, mðtÞ  hðtÞ;

i.e.;

MðtÞ  HðtÞ:

In particular, when f ðtÞ ¼ k2 tekt ; for t [ 0;  k k2 t 1  e2kt \hðtÞ ¼ ; 2 1 þ kt kt 1 MðtÞ ¼  ð1  e2kt Þ\HðtÞ ¼ kt  logð1 þ ktÞ: 2 4 mðtÞ ¼

(b) Because ea  1 þ a; and from (a) of Sect. 3.2.2.2, HðtÞ FðtÞ ;  FðtÞ  HðtÞ  1 þ HðtÞ FðtÞ and hence, when FðtÞ is IFR, HðtÞ FðtÞ :  FðtÞ  MðtÞ  HðtÞ  1 þ HðtÞ FðtÞ Furthermore, we have the inequality HðtÞ FðtÞ Rt ; t 0 FðuÞdu

ð3:28Þ

that is proved as follows: Let QðtÞ  HðtÞ

Zt

FðuÞdu  tFðtÞ;

0

we have that Qð0Þ ¼ 0 and 0

Q ðtÞ ¼ hðtÞ

Zt

FðuÞdu  FðtÞ  FðtÞ½thðtÞ  HðtÞ:

0

Furthermore, Q0 ð0Þ ¼ 0 and 00

0

Q ðtÞ ¼ h ðtÞ

Zt 0

½FðtÞ  FðuÞdu þ f ðtÞ

Zt 0

½hðtÞ  hðuÞdu  0;

64

3 Renewal Processes

that implies (3.28). Therefore (Problem 3.11), MðtÞ FðtÞ HðtÞ FðtÞ : Rt  hðtÞ  R 1  t t t FðuÞdu 0 FðuÞdu

ð3:29Þ

3.2.2.4 Mean Time between Failures (MTBF) Let Xk be the failure time between the ðk  1Þth and the kth failures ðk ¼ 1; 2; . . .Þ: Then, for a renewal process, Z1 EfXk g ¼ FðtÞdt ¼ l 0

for all k  1; and for a nonhomogeneous Poisson process, from (2.44), EfXk g ¼

Z1

½HðtÞk1 HðtÞ dt e ðk  1Þ!

ðk ¼ 1; 2; . . .Þ:

0

Thus, when FðtÞ is IFR, from Sect. 2.3, l ¼ EfX1 g  EfX2 g     ; and l  EfXk g !

1 hð1Þ

as k ! 1:

Furthermore, x1 that satisfies Hðx1 Þ ¼ 1 is called characteristic life in the probability paper of a Weibull distribution, i.e., it represents the mean lifetime that about 1  e1 63:2% of units have failed until time x1 , as shown in (1) of Sect. 2.1.1. When FðtÞ is IFR, t1 that satisfies Mðt1 Þ ¼ 1 is greater than x1 because MðtÞ  HðtÞ: This means that more than 63.2% of units have failed until time t1 at which the expected number of failures is 1 for a renewal process.

3.2.3 Computation of Renewal Function In general, it is not so easy to compute a renewal function except the interarrival distribution FðtÞ is a gamma distribution. The following three methods of computing a renewal function MðtÞ are presented [12, p. 310, 13]: 3.2.3.1 Gamma Approximation Suppose that the distribution FðtÞ of a random variable X is approximately replaced by a gamma distribution as follows: From (a) of Sect. 3.2.2.1,

3.2 Renewal Function

65

MðtÞ ¼

1 X

F ðnÞ ðtÞ

n¼1

k X

F ðnÞ ðtÞ þ

n¼1

1 X

ðnÞ

FG ðtÞ

ð3:30Þ

n¼kþ1

for k ¼ 2; 3; . . .; and FG ðtÞ is a gamma distribution, i.e., 1 FG ðtÞ ¼ CðaÞ

Zt

ka xa1 ekx dx

for k [ 0 and a [ 0;

0 ðnÞ

and FG ðtÞ is given in (3.9). Because the infinite series is truncated, this computation can be established by choosing that EfXg ¼ na=k and VfXg ¼ na=k2 and ðkÞ using the stopping criterion for the first k such that FG ðtÞFG ðtÞ=F G ðtÞ\e for some specified e [ 0: 3.2.3.2 Laplace Inversion The LS transform of MðtÞ is given in Example 3.2 and is obtained by inverting it. The inversion formula for computing the renewal density is Zbþic

1 mðtÞ ¼ lim c!1 2pi

est M  ðsÞds;

ð3:31Þ

bic

pffiffiffiffiffiffiffi where i ¼ 1; b [ maxfr; 0g, and r is a radius of convergence. But, (3.31) would be hard to use actually. Several methods of computing (3.31) are summarized [12, p. 462, 13]: The typical method is, applying the Fourier series to (3.31), 2ebt mðtÞ ¼ p

Z1

ReðM  ðb þ iuÞÞ cosðxtÞdx

0 bt

2e ¼ p

Z1

ð3:32Þ ImðM  ðb þ iuÞÞ sinðxtÞdx:

0

3.2.3.3 Discretization Algorithm From (3.5), the renewal function MðtÞ is computed numerically by solving the renewal equation MðtÞ ¼ FðtÞ þ

Zt 0

Mðt  uÞdFðuÞ:

66

3 Renewal Processes

Xie [14] proposed the discretization algorithm for solving the above Volterra integral equation: Divide the time interval ½0; t by n line segments with equal length d; and denote Mi ¼ MðidÞ and Ki ¼ FðidÞ ði ¼ 0; 1; 2; . . .; nÞ at each point i ¼ 0; d; 2d; . . .; nd ¼ t; where Fi  Fðði  0:5ÞdÞ: Then, the following recursive formula gives the approximation of the renewal function Mn ; starting from M0  0; " # i1 X 1 Ki þ ðMj  Mj1 ÞFijþ1  Mi1 F1 : Mi ¼ ð3:33Þ 1  F1 j¼1 This formula is easy to program and surprisingly shows accurate results. The renewal function of a Weibull distribution was presented in [12, p. 312] (Problem 3.4).

3.3 Age Replacement Policies 3.3.1 Renewal Equation We can generalize the renewal equation in (3.5) to the following equation form [5, p. 95, 6, p. 65] gðtÞ ¼ hðtÞ þ

Zt

gðt  uÞdFðuÞ;

ð3:34Þ

0

where hðtÞ and FðtÞ are known, and gðtÞ is unknown. This is called the renewaltype equation. Taking the LS transform of (3.34), 

g ðsÞ 

Z1

est dgðtÞ ¼

h ðsÞ ¼ h ðsÞ½1 þ M  ðsÞ: 1  F  ðsÞ

0

Thus, inverting the LS transform g ðsÞ; gðtÞ ¼ hðtÞ þ

Zt

hðt  uÞdMðuÞ:

ð3:35Þ

0

Such modified types of equations appear well in reliability theory, as shown in (3.22) and (3.23). We show how to set up renewal-type equations for a variety of reliability models by paying close attention to renewal points and making some suitable modifications: Consider an age replacement model where a unit is replaced at time Tð0\T  1Þ after its installation or at failure, whichever occurs first. The event fT ¼ 1g represents that no planned replacement is done at all. It

3.3 Age Replacement Policies

67

is assumed that failures are instantly detected and each failed unit is replaced with a new one, where its replacement time is negligible, and so, a newly installed unit begins to operate instantly. Furthermore, suppose that the failure time Xk ðk ¼ 1; 2; . . .Þ of each operating unit is independent andR has an identical failure distri1 bution FðtÞ  PrfXk  tg with finite mean l  0 FðtÞdt\1 throughout this section. A new unit is installed at time t ¼ 0: Then, an age replacement procedure generates a renewal process as follows: Let fXk g1 k¼1 be the failure times of successive operating units. Define a new random variable Zk  minfXk ; Tgðk ¼ 1; 2; . . .Þ: Then, fZk g1 k¼1 represents the intervals between replacements caused by either failures or planned replacements (Fig. 3.4). In other words, because PrfZk ¼ Xk g ¼ PrfXk  Tg ¼ FðTÞ; PrfZk ¼ Tg ¼ PrfXk [ Tg ¼ FðTÞ; such trials corresponds to a Bernoulli trial [5, p. 34] whose outcome can be classified as either success or failure with the respective probabilities p  FðTÞ and q  FðTÞ: A sequence of random variables fZk g1 k¼1 ; where Zk is independently and identically distributed, forms a renewal process with an interarrival distribution

FðtÞ for t  T; PrfZk  tg ¼ 1 for t [ T: Thus, the mean time between replacements is EfZk g ¼

ZT

tdFðtÞ þ TFðTÞ ¼

0

ZT FðtÞdt: 0

First, let Y1 be the first time at which the unit is replaced before time T by failure, and G1 ðtÞ  PrfY1  tg: Then, by paying attention to the renewal point of replacement times, G1 ðtÞ ¼

Zt 0

DðuÞdFðuÞ þ

Zt

G1 ðt  uÞFðuÞdDðuÞ;

0

where DðtÞ is a degenerate distribution at time T; i.e., DðtÞ  0 for t\T and 1 for t  T: Taking the LS transform and solving it for G1 ðsÞ; Fig. 3.4 Process of age replacement with planned time T

68

3 Renewal Processes



G1 ðsÞ 

Z1 e

st

0

R 1 st R T st DðtÞdFðtÞ e dFðtÞ 0R e dG1 ðtÞ ¼ ¼ 0 sT : 1 st 1  0 e FðtÞdDðtÞ 1  e FðTÞ

ð3:36Þ

Similarly, letting Y2 be the first time at which the unit is replaced before failure by the planned replacement, G2 ðtÞ  PrfY2  tg ¼

Zt

FðuÞdDðuÞ þ

Zt

0

G2 ðt  uÞDðuÞdFðuÞ;

0

and its LS transform is 

G2 ðsÞ 

Z1

est dG2 ðtÞ ¼

0

esT FðTÞ : RT 1  0 est dFðtÞ

ð3:37Þ

Therefore, the mean times of Y1 and Y2 are, respectively, ZT 1  G1  ðsÞ 1 l1  EfY1 g ¼ lim FðtÞdt; ¼ s!0 s FðTÞ 0

1  G2  ðsÞ 1 l2  EfY2 g ¼ lim ¼ s!0 s FðTÞ

ð3:38Þ

ZT FðtÞdt: 0

It can be easily seen that l1 decreases from 1=hð0Þ to l when FðtÞ is IFR, l2 increases from 0 to 1; and when FðTÞ ¼ 1=2; l1 ¼ l2 (Problem 3.12). The above results are easily derived by solving the respective renewal equations ZT l1 ¼ ðT þ l1 ÞFðTÞ þ tdFðtÞ; 0

l2 ¼ TFðTÞ þ

ZT

ðt þ l2 ÞdFðtÞ;

0

and

 EfZk g ¼ 1=

 1 1 : þ l1 l2

It is of interest that 2EfZk g is given by the harmonic mean of l1 and l2 : Next, let M1 ðtÞ be the expected number of replacements due to failures in ½0; t: We have a renewal equation for M1 ðtÞ M1 ðtÞ ¼

Zt 0

½1 þ M1 ðt  uÞDðuÞdFðuÞ þ

Zt 0

M1 ðt  uÞFðuÞdDðuÞ:

3.3 Age Replacement Policies

69

Forming the LS transform and solving for M1 ðtÞ; 

M1 ðsÞ ¼

Z1

st

e

RT dM1 ðtÞ ¼ R T0 0

0

est dFðtÞ

sest FðtÞdt

:

ð3:39Þ

Similarly, let M2 ðtÞ be the expected number of replacements due to the planned replacement in ½0; t: We have a renewal equation for M2 ðtÞ M2 ðtÞ ¼

Zt

M2 ðt  uÞDðuÞdFðuÞ þ

0

Zt

½1 þ M2 ðt  uÞFðuÞdDðuÞ:

0

Thus, 

M2 ðsÞ ¼

Z1

est dM2 ðtÞ ¼ R T 0

0

est FðTÞ sest FðtÞdt

:

ð3:40Þ

Then, by using a Tauberian Theorem in Appendix A.3, lim

t!1

M1 ðtÞ FðTÞ 1 ¼ ; ¼ lim sM1  ðsÞ ¼ R T s!0 t FðtÞdt l1 0

M2 ðtÞ FðTÞ 1 ¼ : lim ¼ lim sM2  ðsÞ ¼ R T t!1 s!0 t FðtÞdt l2

ð3:41Þ

0

Therefore, letting MðtÞ be MðtÞ  M1 ðtÞ þ M2 ðtÞ that is the expected number of replacements in ½0; t; lim

t!1

MðtÞ 1 1 1 1 ¼ þ : ¼ RT ¼ t FðtÞdt EfZk g l1 l2 0

Three limiting results of M1 ðtÞ; M2 ðtÞ and MðtÞ satisfy the elementary renewal theorem.

3.3.2 Optimum Replacement Policies We give optimum policies for age and periodic replacements as applied examples of renewal processes [1, p. 84, 11, p. 70]. 3.3.2.1 Age Replacement We take up the age replacement model where the unit is replaced at time T or at failure, whichever occurs first, as shown in Sect. 3.3.1. The model forms a renewal process RT with an interarrival distribution PrfZk  tg and its mean time EfZk g ¼ 0 FðtÞdt:

70

3 Renewal Processes

We consider the problem of minimizing the expected cost per unit of time for an infinite time span. Introduce the following replacement costs: Cost c1 is incurred for each failed unit that is replaced; this includes all costs resulting from the failure and its replacement. Cost c2 ð\c1 Þ is incurred for each exchange of nonfailures at time T: Let M1 ðtÞ and M2 ðtÞ denote the expected replacement numbers of failures and b nonfailures and the expected cost CðtÞ denote c1 M1 ðtÞ þ c2 M2 ðtÞ in ½0; t: Then, from (3.41), the expected cost per unit of time for an infinite time span is

b M1 ðtÞ M2 ðtÞ CðtÞ CðTÞ  lim þ c2  lim c1 t!1 t t!1 t t ð3:42Þ c1 c2 c1 FðTÞ þ c2 FðTÞ ¼ þ ¼ : RT l 1 l2 0 FðtÞdt We consider the renewal cycle from one replacement to the next replacement. Then, the pairs of time and cost on each cycle are independently and identically distributed, and both have finite means. The expected cost of one cycle is c1 PrfXk  Tg þ c2 PrfXk [ Tg ¼ c1 FðTÞ þ c2 FðTÞ; RT and the mean time of one cycle is EfZk g ¼ 0 FðtÞdt: Thus, the expected cost per unit of time is easily given in (3.42) from (3.19). We call CðTÞ the expected cost rate and generally adopt it as the objective function of the optimization problem for age replacement with an infinite time span [11, p. 71]. We derive an optimum planned replacement time T  that minimizes the expected cost rate CðTÞ in (3.42). It is assumed that the failure rate hðtÞ  f ðtÞ=FðtÞ is differentiable and increases strictly, i.e., h0 ðtÞ  dhðtÞ=dt [ 0; where f ðtÞ is a density function of FðtÞ: It is clearly seen that Cð0Þ  lim CðTÞ ¼ 1; T!0

Cð1Þ  lim CðTÞ ¼ T!1

c1 : l

Thus, there exists a positive T  ð0\T   1Þ: In addition, differentiating CðTÞ with respect to T and setting it equal to zero, ZT hðTÞ

FðtÞdt  FðTÞ ¼

c2 : c 1  c2

0

Letting Q1 ðTÞ be the left-hand side of (3.43), Qð0Þ ¼ 0 and Q1 ð1Þ ¼ lhð1Þ  1: Differentiating Q1 ðTÞ with respect to T; dQ1 ðTÞ ¼ h0 ðTÞ dT

ZT FðtÞdt [ 0: 0

ð3:43Þ

3.3 Age Replacement Policies

71

Therefore, we have the optimum policy when K  c1 =½lðc1  c2 Þ: (i) If hð1Þ [ K; then there exists a finite and unique T  ð0\T  \1Þ that satisfies (3.43), and the resulting cost rate is CðT  Þ ¼ ðc1  c2 ÞhðT  Þ: (ii) If hð1Þ  K; then T  ¼ 1; i.e., the unit is replaced only at failure, and the expected cost rate is Cð1Þ ¼ c1 =l:

3.3.2.2 Three Periodic Replacement We take up three periodic replacements where the unit is replaced at periodic times kT ðk ¼ 1; 2; . . .Þ: These replacement models form the simplest renewal process with a constant time T ð0\T  1Þ: First, we consider the block replacement where each failed unit between planned replacements is instantly detected and is replaced with a new one. That is, the model also forms a renewal process with a renewal function MðtÞ between planned replacements. Then, because the expected number of failures in one planned replacement cycle is MðTÞ; the expected cost rate is, from (3.19), C1 ðTÞ ¼

c1 MðTÞ þ c2 ; T

ð3:44Þ

where c1 ¼ cost of replacement at each failure and c2 ¼ cost of planned replacement at time T: Clearly, from (3.18), limT!1 C1 ðTÞ ¼ c1 =l: Compare the expected costs with age and block replacements: Letting AðTÞ  c1 FðTÞ þ c2 FðTÞ and BðTÞ  c1 MðTÞ þ c2 ; we have the renewal equation (Problem 3.13) BðTÞ ¼ AðTÞ þ

ZT

BðT  tÞdFðtÞ;

or

BðTÞ ¼ AðTÞ þ

0

ZT

AðT  tÞdMðtÞ;

0

i.e., AðTÞ and BðTÞ determine each other. We seek an optimum planned replacement time T  that minimizes C1 ðTÞ in (3.44). It is assumed that MðtÞ is differentiable and define mðtÞ  dMðtÞ=dt: Then, differentiating C1 ðTÞ with respect to T and setting it equal to zero, TmðTÞ  MðTÞ ¼

c2 : c1

ð3:45Þ

This is a necessary condition that there exists a finite T  ; and the resulting cost rate is C1 ðT  Þ ¼ c1 mðT  Þ:

72

3 Renewal Processes

Next, we consider the periodic replacement where each failed unit between replacements undergo minimal repair, as shown in Sect. 2.4.1. It is assumed that the repair times are negligible and the failure rate remains undisturbed by any minimal repair. That is, the replacement model also forms a nonhomogeneous Poisson process with a cumulative hazard function HðtÞ between planned replacements, where FðtÞ ¼ 1  eHðtÞ and H 0 ðtÞ ¼ hðtÞ: Then, because the expected number of failures in one planned replacement cycle is HðTÞ from (2.45), the expected cost rate is, from (3.19), C2 ðTÞ ¼

c1 HðTÞ þ c2 ; T

ð3:46Þ

where c1 ¼ cost of minimal at each failure, and c2 is given in (3.44). When FðtÞ is IFR, C2 ðTÞ  C1 ðTÞ for the same replacement costs from HðTÞ  MðTÞ: We seek an optimum planned time T  that minimizes C2 ðTÞ in (3.46). Differentiating C2 ðTÞ with respect to T and setting it equal to zero, c2 ThðTÞ  HðTÞ ¼ ; ð3:47Þ c1 and the resulting cost rate is C2 ðT  Þ ¼ c1 hðT  Þ: It is shown that if the failure rate hðtÞ increases strictly, then the left-hand side of (3.47) also increases strictly [11, p. 102]. So that, if a solution T  to (3.47) exists, then it is unique. Finally, we consider the periodic replacement where each failed unit remains in a failed state for the time interval from failure to its planned replacement. Then, the mean time from a failure to the replacement is ZT

ðT  tÞdFðtÞ ¼

0

ZT FðtÞdt: 0

Thus, the expected cost rate is [11, p. 120] RT c1 0 FðtÞdt þ c2 C3 ðTÞ ¼ ; T

ð3:48Þ

where c1 ¼ downtime cost per unit of time elapsed between a failure and its replacement, and c2 is given in (3.44). Differentiating C3 ðTÞ with respect to T and setting it equal to zero, TFðTÞ 

ZT 0

c2 FðtÞdt ¼ ; c1

ZT or

tdFðtÞ ¼

c2 : c1

ð3:49Þ

0

Thus, if l [ c2 =c1 ; then there exists an optimum time T3  that satisfies uniquely (3.49), and the resulting cost rate is

3.3 Age Replacement Policies

73

C3 ðT3  Þ ¼ c1 FðT  Þ: In general, the results of three periodic replacements are summarized as follows: The expected cost rate is [11, p. 125] RT c1 0 uðtÞdt þ c2 ; ð3:50Þ CðTÞ ¼ T where uðtÞ is hðtÞ; mðtÞ; and FðtÞ; respectively. Differentiating CðTÞ with respect to T and setting it equal to zero, TuðTÞ 

ZT

c2 uðtÞdt ¼ ; c1

0

ZT or

tduðtÞ ¼

c2 : c1

ð3:51Þ

0

If there exists T  that satisfies (3.51), then the expected cost rate is CðT  Þ ¼ c1 uðT  Þ: Conversely, if a finite T  does not exists, i.e., T  ¼ 1; and the unit does not undergo any planned replacement, then Cð1Þ ¼ c1 uð1Þ:

3.4 Alternating Renewal Process Alternating renewal processes are the process that combines two renewal processes fXk g and fYk g generated alternately (Fig. 3.5) [7]. That is, each sequence of fXk g and fYk g forms a renewal process, and the sequence of the sum of fXk þ Yk g also forms a renewal process. In this case, Xk and Yk are allowed to be dependent. Most redundant systems with repair maintenance that repeat up and down states alternately, generate alternating renewal processes. Furthermore, the process is a special case of Markov renewal processes in Chap. 5.

3.4.1 Ordinary Alternating Renewal Process We consider a simple system with one unit that repeats on and off states. Suppose that fXk g and fYk g ðk ¼ 1; 2; . . .Þ are the respective on and off times of the kth renewal cycle and have the distribution FðtÞ  PrfXk  tg with finite mean l and Fig. 3.5 Realization of alternating renewal process

74

3 Renewal Processes

the distribution GðtÞ  PrfYk  tg with finite mean b: To analyze the system, we define the following states: State 0: Unit is on. State 1: Unit is off. When the unit starts to State 0 at time 0; the unit repeats States 0 and 1 alternately (Fig. 3.5). Let Mij ðtÞ ði; j ¼ 0; 1Þ denote the expected number of State j in ½0; t when the unit goes into State i at time 0; where the first visit to State j is not counted when i ¼ j: Then, we have the following renewal equations: M01 ðtÞ ¼ FðtÞ þ

Zt

M11 ðt  uÞdFðuÞ;

0

M10 ðtÞ ¼ GðtÞ þ

Zt

M00 ðt  uÞdGðuÞ;

ð3:52Þ

0

M11 ðtÞ ¼

Zt

M01 ðt  uÞdGðuÞ;

M00 ðtÞ ¼

0

Zt

M10 ðt  uÞdFðuÞ:

0

Thus, taking the LS transforms of both sides of the above equations and solving them, F  ðsÞ G ðsÞ  ; M ; ðsÞ ¼ 10 1  F  ðsÞG ðsÞ 1  F  ðsÞG ðsÞ     ðsÞ ¼ G ðsÞM01 ðsÞ; M00 ðsÞ ¼ F  ðsÞM10 ðsÞ; M11  M01 ðsÞ ¼

ð3:53Þ

where the R 1 asterisk of the function denotes the LS transform with itself, i.e., U ðsÞ  0 est dUðtÞ for any function UðtÞ: Next, let ZðtÞ denote the state of the process at time t; and define Pij ðtÞ  PrfZðtÞ ¼ jjZð0Þ ¼ ig ði; j ¼ 0; 1Þ: The probability Pij ðtÞ is called the transition probability from State i to State j and represents the probability that the unit is in State j at time t if it starts from State i at time 0: By the similar method for obtaining (3.52), P00 ðtÞ ¼ FðtÞ þ

Zt

P10 ðt  uÞdFðuÞ;

0

P11 ðtÞ ¼ GðtÞ þ

Zt

P01 ðt  uÞdGðuÞ;

ð3:54Þ

0

P10 ðtÞ ¼

Zt 0

P00 ðt  uÞdGðuÞ;

P01 ðtÞ ¼

Zt 0

P11 ðt  uÞdFðuÞ:

3.4 Alternating Renewal Process

75

Thus, again taking the LS transforms, P00 ðsÞ ¼ P10 ðsÞ

1  F  ðsÞ ; 1  F  ðsÞG ðsÞ

¼G



ðsÞP00 ðsÞ;

P11 ðsÞ ¼ P01 ðsÞ

1  G ðsÞ ; 1  F  ðsÞG ðsÞ

¼F



ð3:55Þ

ðsÞP11 ðsÞ:

Thus, comparing (3.53) and (3.55), we have the following relations: Zt

P01 ðtÞ ¼

Gðt  uÞdM01 ðuÞ ¼ M01 ðtÞ  M00 ðtÞ;

0

Zt

P10 ðtÞ ¼

Fðt  uÞdM10 ðuÞ ¼ M10 ðtÞ  M11 ðtÞ:

0

These relations between renewal functions and transition probabilities would be useful for the analysis of more complex systems. Next, it is assumed that distributions FðtÞ and GðtÞ have finite variances rF 2 and rG 2 ; respectively. Then,  1 F  ðsÞ ¼ 1  ls þ l2 þ r2F s2 þ oðs2 Þ; 2  1 G ðsÞ ¼ 1  bs þ b2 þ r2G s2 þ oðs2 Þ: 2 Substituting these equations for (3.53),  M01 ðsÞ ¼

1 1 l 1 r2 þ r2G þ oð1Þ:  þ þ F l þ b s l þ b 2 2ðl þ bÞ2

 ðsÞ gives, for large t; The LS inversion of M01

M01 ðtÞ ¼

t l 1 r2 þ r2G þ oð1Þ:  þ þ F l þ b l þ b 2 2ðl þ bÞ2

M00 ðtÞ ¼

t 1 r2 þ r2G  þ F þ oð1Þ; l þ b 2 2ðl þ bÞ2

P00 ðtÞ ¼

l þ oð1Þ; lþb

ð3:56Þ

Similarly,

P01 ðtÞ ¼

b þ oð1Þ: lþb

ð3:57Þ

Clearly, lim

t!1

Mij ðtÞ 1 ¼ t lþb

ðj ¼ 0; 1Þ;

l EfXk g lim Pi0 ðtÞ ¼ 1  lim Pi1 ðtÞ ¼ ¼ ; t!1 t!1 l þ b EfXk g þ EfYk g

ð3:58Þ

76

3 Renewal Processes

that are independent of an initial State i in the limit. Note that (3.58) represents the steady-state availability of a one-unit system [11, p. 9]. Example 3.6 (One-unit system with repair) Suppose that an operating unit is repaired when it fails, and it begins to operate again. In this case, States 0 and 1 represent that the unit is operating and under repair, respectively. It is assumed that the failure time has an exponential distribution FðtÞ ¼ 1  ekt and the repair time has an exponential distribution GðtÞ ¼ 1  eht : Then, it is easy to invert the LS transforms as follows: i k h P01 ðtÞ ¼ 1  eðkþhÞt ; kþh  2 h i kht k 1  eðkþhÞt : þ M01 ðtÞ ¼ kþh kþh Thus, for large t; P01 ðtÞ

k ; kþh

M01 ðtÞ

 2 kh k : tþ kþh kþh

Next, when FðtÞ ¼ 1  ð1 þ ktÞekt and the time for repair is constant b; P01 ðsÞ ¼

k2 ð1  ebs Þ

;

 M01 ðsÞ ¼

b ; 2=k þ b

M01 ðtÞ

2

ðs þ kÞ 

k2 ebs

k2 2

ðs þ kÞ  k2 ebs

;

and for large t; P01 ðtÞ

t : 2=k þ b

3.4.2 Interval Reliability Interval reliability is defined as the probability that at a specified time, the unit is operating and will continue to operate for an interval duration [1, p. 7, 11, p. 11]. Then, the interval reliability Rðx; T0 Þ for an interval of duration x starting at time T0 is Rðx; T0 Þ  PrfZðuÞ ¼ 0; T0  u  T0 þ xg;

ð3:59Þ

and its limit of Rðx; T0 Þ as T0 ! 1 is called the limiting interval reliability. When the unit begins to operate at time 0 in Example 3.6, the interval reliability is

h k ðkþhÞT0 kx kx Rðx; T0 Þ  P00 ðT0 Þe e ; þ e ¼ ð3:60Þ kþh kþh

3.4 Alternating Renewal Process

77

and its limiting interval reliability is RðxÞ  lim Rðx; T0 Þ ¼ T0 !1

h kx e : kþh

ð3:61Þ

Suppose that the unit has a failure distribution FðtÞ with mean l and a repair distribution GðtÞ with mean b in Sect. 3.4.1. When the unit begins to operate at time 0; the interval reliability is [11, p. 48] Rðx; T0 Þ ¼ FðT0 þ xÞ þ

ZT0

FðT0 þ x  uÞdM00 ðuÞ;

ð3:62Þ

0

where M00 ðtÞ is given in (3.53). Hence, its Laplace transform is R1 Z1 esx x est FðtÞdt ; esT0 Rðx; T0 ÞdT0 ¼ R ðx; sÞ  1  F  ðsÞG ðsÞ

ð3:63Þ

0

and its limiting interval reliability is 

RðxÞ  lim Rðx; T0 Þ ¼ lim sR ðx; sÞ ¼ T0 !1

s!0

R1 x

FðtÞdt : lþb

ð3:64Þ

Next, suppose that we set the preventive maintenance (PM) at time T ð0\T  1Þ for the operating unit [11, p. 141]. However, the PM is not done in the interval ½T0 ; T0 þ x even if it is time for PM. It is assumed that the distribution of time to the PM completion is the same as the repair distribution GðtÞ: Let DðtÞ be a degenerate distribution placing unit mass at T; i.e., DðtÞ  0 for t\T and 1 for t  T: Then, in a similar way of obtaining (3.52), M00 ðtÞ ¼

Zt

M10 ðt  uÞ½DðuÞdFðuÞ þ FðuÞdDðuÞ;

0

M10 ðtÞ ¼

Zt

½1 þ M00 ðt  uÞdGðuÞ:

0

Taking the LS transforms,

h i RT G ðsÞ 1  0 sest FðtÞdt  h i: ðsÞ ¼ M00 RT 1  G ðsÞ 1  0 sest FðtÞdt

ð3:65Þ

By the similar method of obtaining (3.62), the interval reliability is RðT; x; T0 Þ ¼ FðT0 þ xÞDðT0 Þ þ

ZT0 0

FðT0 þ x  uÞDðT0  uÞdM00 ðuÞ:

ð3:66Þ

78

3 Renewal Processes

Taking the Laplace transform of (3.66) and substituting it for (3.65), 

R ðT; x; sÞ 

Z1

esT0 RðT; x; T0 ÞdT0

0

ð3:67Þ

R Tþx

est FðtÞdt x ; ¼ RT 1  G ðsÞ þ G ðsÞ 0 sest FðtÞdt esx

that agrees with (3.63) when T ¼ 1 (Problem 3.14). Thus, the limiting interval reliability is RðT; xÞ  lim RðT; x; T0 Þ ¼ lim sR ðT; x; sÞ T0 !1 s!0 R Tþx FðtÞdt ¼ R Tx ; 0 FðtÞdt þ b

ð3:68Þ

that also agrees with (3.64) when T ¼ 1 (Problem 3.15). We seek an optimum PM time T  that maximizes RðT; xÞ for a fixed x [ 0: Let kðt; xÞ  ½Fðt þ xÞ  FðtÞ=FðtÞ for t  0 and kð1; xÞ  limt!1 kðt; xÞ  1: Then, note that both kðt; xÞ and the failure rate hðtÞ  f ðtÞ=FðtÞ have the same property [1, p. 23, 11, p. 6]. It is assumed that kðt; xÞ increases strictly with t: Then, differentiating RðT; xÞ in (3.68) with respect to T and setting it equal to zero, 2 T 3 Z ZT kðT; xÞ4 FðtÞdt þ b5  ½Fðt þ xÞ  FðtÞdt ¼ b; ð3:69Þ 0

0

whose left-hand side QðT; xÞ increases strictly from Qð0Þ ¼ bFðxÞ to Qð1; xÞ ¼ kð1; xÞðl þ bÞ 

Zx FðtÞdt: 0

Therefore, if Qð1; xÞ [ b; then there exists a finite and unique T  ð0\T  \1Þ that satisfies (3.69), and the resulting interval reliability is RðT  ; xÞ ¼

FðT  þ xÞ ; FðT  Þ

that represents the probability that the unit with age T  does not fail in ½T  ; T  þ x: Example 3.7 When FðtÞ ¼ 1  ð1 þ ktÞekt ;   kx ekx ; kðt; xÞ ¼ 1  1 þ 1 þ kt

3.4 Alternating Renewal Process

79

that increases strictly with t to ð1  ekx Þ: Thus, if Q ð1; xÞ [ b; i.e., x [ b; an optimum T  is a unique solution of the equation kTðx  bÞ  xð1  ekT Þ ¼ ð1 þ kxÞb; and the interval reliability is RðT  ; xÞ ¼

 1þ

 kx ekx : 1 þ kT 

In this case, an optimum T  decreases with x to a unique solution of T

1  ekT ¼ b: k

Conversely, if x  b; then T  ¼ 1 and the interval reliability in (3.68) is given in (3.64).

3.4.3 Off Distribution Let NðtÞ and DðtÞ be the number of on states and the total amount of time spent in off states in ½0; t; respectively. It is said in reliability theory that DðtÞ is called a downtime [1, p. 79, 11, p. 45]. Then, from the formula of the total sum of independent random variables, PrfY1 þ Y2 þ    þ Yn  xjNðtÞ ¼ ng PrfNðtÞ ¼ ng ¼ PrfY1 þ Y2 þ    þ Yn  xg PrfX1 þ    þ Xn  t  x\X1 þ    þ Xnþ1 g h i ¼ GðnÞ ðxÞ F ðnÞ ðt  xÞ  F ðnþ1Þ ðt  xÞ : Thus, the downtime distribution is

P1 ðnÞ ðnÞ ðnþ1Þ ðt  xÞ n¼0 G ðxÞ½F ðt  xÞ  F PrfDðtÞ  xg ¼ 1

for t [ x; for t  x: ð3:70Þ

The above equation can be written as PrfDðt þ xÞ  xg ¼

1 X n¼0

(P PrfDðtÞ [ xg ¼

0

h i GðnÞ ðxÞ F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ ;

1  ðnÞ n¼0 G ðxÞ

  Gðnþ1Þ ðxÞ F ðnþ1Þ ðt  xÞ

for t [ x; for t  x:

Next, we investigate the properties of DðtÞ: Denote Ld  mint fDðtÞ [ dg as the first time that the total time spent in off states, i.e., the total downtime, exceeds a threshold level d: We have the following relation:

80

3 Renewal Processes

PrfLd  tg ¼ PrfDðtÞ [ dg 1 h i X ¼ GðnÞ ðdÞ  Gðnþ1Þ ðdÞ F ðnþ1Þ ðt  dÞ:

ð3:71Þ

n¼0

Thus, the mean time that the total downtime first exceeds d is EfLd g ¼ d þ l

1 X

GðnÞ ðdÞ:

ð3:72Þ

n¼0

The mean downtime in ½0; t is EfDðtÞg ¼

Zt

xd PrfDðtÞ  xg:

ð3:73Þ

0

In general, it is very difficult to compute (3.73). It is noted easily from Sect. 3.4.1 that EfDðtÞg ¼

Zt

P01 ðuÞdu;

ð3:74Þ

0

and its LS transform is Z1

est P01 ðtÞdt ¼

F  ðsÞ½1  G ðsÞ : s½1  F  ðsÞG ðsÞ

ð3:75Þ

0

Thus, from (3.57) EfDðtÞg lim ¼ lim P01 ðtÞ ¼ lim s t!1 t!1 s!0 t

Z1

est P01 ðtÞdt ¼

b : lþb

ð3:76Þ

0

Example 3.8 (One-unit system with repair) We consider the same one-unit system in Example 3.6. The downtime distribution is [1, p. 80] PrfDðtÞ [ xg ¼

1 X ðkxÞn n¼0

n!

ekx

1 X ½hðt  xÞk hðtxÞ e ; k! k¼nþ1

or 2

3 Zx  pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi p ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi PrfDðtÞ  xg ¼ ehðtxÞ41þ khðtxÞ eky y1=2 I1 2 khðtxÞy dy5; 0

3.4 Alternating Renewal Process

81

where I1 ðxÞ is the Bessel function of order 1 for the imaginary argument defined by I1 ðxÞ 

1 X ðx=2Þ2jþ1 j¼0

j!ðj þ 1Þ!

:

In addition, from (3.72) and (3.74), 1 EfLd g ¼ d þ ð1 þ hdÞ; k i k k h t 1  eðkþhÞt : EfDðtÞg ¼ 2 kþh ðk þ hÞ Takács [10] also proved the following important theorem: Theorem 3.8 If r2F \1 and r2G \1; then 9 8 > > > > Zx = < DðtÞ  bt=ðl þ bÞ 1 2 r ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi p ffiffiffiffiffi ffi ¼  x eu =2 du: lim Pr h i t!1 > > 2p > > ; : ðbrF Þ2 þðlrG Þ2 t=ðl þ bÞ3 1

ð3:77Þ

That is, if the means and variances of FðtÞ and GðxÞ are statistically estimated, then the probability of the total time spent in off states, i.e., downtime DðtÞ is approximately computed for large t; by using a standard normal distribution. Example 3.9 (Asymptotic downtime distribution) [11, p. 50] We wish to compute the time d when the probability that the system is down more than d in t ¼ 10; 000 hours of operation is given by 0.90. The failure and repair distributions are unknown, but from the sample data, the estimates of means and variances are l ¼ 1; 000;

rF 2 ¼ 100; 000;

b ¼ 100;

rG 2 ¼ 400:

Then, from Theorem 3.8, when t ¼ 10; 000; DðtÞ  bt=ðl þ bÞ Dð10; 000Þ  909:09 rhffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ i 102:56 ðbrF Þ2 þðlrG Þ2 t=ðl þ bÞ3 is approximately normally distributed with mean 0 and variance 1: Hence,

 Dð10; 000Þ  909:09 d  909:09 [ PrfDðtÞ [ dg ¼ Pr 102:56 102:56 Z1 1 2

pffiffiffiffiffiffi eu =2 du ¼ 0:90: 2p ðd909:09Þ=102:56

82

3 Renewal Processes

pffiffiffiffiffiffi R 1 2 Because u0 ¼ 1:28 such that ð1= 2pÞ u eu =2 du0 ¼ 0:90; d 909:09  102:56 1:28 ¼ 777:81; i.e., the system is down more than 778 h in t ¼ 10; 000 h with probability 0:90 (Problem 3.16).

3.4.4 Terminating Renewal Process Consider the first time that an amount of the time spent in off states exceeds a ek  Yk ; threshold level U with distribution WðtÞ: We define that if Yk  U then Y e and conversely, if U\Yk then U k  U ðk ¼ 1; 2; . . .Þ: The process ends with the first event of fU\YN g that is called the terminating process e 1 ; X2 ; Y e2 ; . . .; XN1 ; Y eN1 ; XN ; U e g; and its interest quantities are the total fX1 ; Y PN1 N ek Þ þ XN þ U e N ðN ¼ 1; 2; . . .Þ and its sum of random variables ZN  k¼1 ðXk þ Y P1 distribution LðtÞ  N¼1 PrfZN  tg: The probability that Yk is not greater than U and Yk  t is BðtÞ  PrfYk  U; Yk  tg ¼

Zt WðuÞdGðuÞ; 0

and the probability that Yk is greater than U and U  t is e  PrfU\Yk ; U  tg ¼ BðtÞ

Zt GðuÞdWðuÞ: 0

Thus, from the formula of the total sum of independent random variables, LðtÞ ¼

1 X

PrfZN  tg ¼

N¼1

1 X

e F ðkÞ ðtÞ  BðkÞ ðtÞ  FðtÞ  BðtÞ:

ð3:78Þ

k¼0

Therefore, the LS transform of LðtÞ is 

L ðsÞ 

Z1

est dLðtÞ ¼

e  ðsÞ F  ðsÞ B ; 1  F  ðsÞB ðsÞ

ð3:79Þ

0

and its mean time is l

Z1 0

R1 1  L ðsÞ l þ 0 GðtÞWðtÞdt ¼ R1 : tdLðtÞ ¼ lim s!0 s 0 WðtÞdGðtÞ

ð3:80Þ

3.4 Alternating Renewal Process

83

In particular, when a random variable U is constant cðc [ 0Þ; i.e., WðtÞ  0 for t\c; and 1 for t  c; Rc l þ 0 GðtÞdt F  ðsÞesc GðcÞ  Rc L ðsÞ ¼ ; l¼ ; ð3:81Þ 1  F  ðsÞ 0 est dGðtÞ GðcÞ where c is called a critical allowed time for maintenance [15]. Example 3.10 (Repair limit policy) [11, p. 51] Consider a one-unit system that is repaired or replaced if it fails. Let l denote the mean failure time of the unit and GðtÞ denote the repair distribution of the failed unit with finite mean b: When the unit fails, its repair is started immediately, and when the repair is not completed within time T ð0  T\1Þ; that is called the repair limit time, it is replaced with a new one. If Zk  minfYk ; Tg, then the sequence of fXk ; Zk g ðk ¼ 1; 2; . . .Þ forms an alternating renewal process. Let c1 be the replacement cost of a failed unit, caused by failure and replacement, and cr ðtÞ be the expected repair cost during ½0; t; that includes all costs incurred due to repair and downtime during ½0; t: Consider one cycle from the beginning of an operating unit to the repair or replacement completion. Then, the expected cost of one cycle is ½c1 þ cr ðTÞGðTÞ þ

ZT

cr ðtÞdGðtÞ ¼ c1 GðTÞ þ

ZT

0

GðtÞdcr ðtÞ;

0

and the mean time of one cycle is l þ TGðTÞ þ

ZT

tdGðtÞ ¼ l þ

0

ZT GðtÞdt: 0

Thus, from (3.19), the expected cost rate is RT c1 GðTÞ þ 0 GðtÞdcr ðtÞ CðTÞ ¼ : RT l þ 0 GðtÞdt

ð3:82Þ

Clearly, c1 Cð0Þ  lim CðTÞ ¼ ; T!0 l

R1 Cð1Þ  lim CðTÞ ¼ T!1

0

GðtÞdcr ðtÞ ; lþb

that represent the expected cost rates with only replacement and only repair maintenance, respectively. Suppose that cr ðtÞ ¼ at for a [ 0: Then, the expected cost rate is, from (3.82), RT c1 GðTÞ þ a 0 GðtÞdt CðTÞ ¼ : ð3:83Þ RT l þ 0 GðtÞdt

84

3 Renewal Processes

We find an optimum repair limit time T  that minimizes CðTÞ: It is assumed that rðtÞ  gðtÞ=GðtÞ; where gðtÞ is a density function of GðtÞ: Then, differentiating CðTÞ with respect to T and setting it equal to zero, 2 3 ZT al ð3:84Þ rðTÞ4l þ GðtÞdt5 þ GðTÞ ¼ : c1 0

Letting QðTÞ be the left-hand side of (3.84), Qð0Þ  lrð0Þ þ 1;

Qð1Þ ¼ ðl þ bÞrð1Þ:

Thus, if rðtÞ decreases strictly and Qð0Þ [ al=c1 [ Qð1Þ; there exists uniquely a finite and positive T  ð0\T  \1Þ that satisfies (3.84), and the resulting cost rate is C ðT  Þ ¼ a  c1 r ðT  Þ: If Qð0Þ  al=c1 then T  ¼ 0; i.e., no repair should be done. If Qð1Þ  al=c1 then T  ¼ 1; i.e., no replacement should be done (Problem 3.17).

3.5 Modified Renewal Processes This section takes up two modified renewal processes.

3.5.1 Geometric Renewal Process Suppose that a renewal process is defined by a sequence of random variables fX1 ; X2 =a; . . .; Xk =ak1 ; . . .g for a [ 0 with distribution Fðak1 tÞ  PrfXk =ak1  tg; where FðtÞ has finite mean l and variance r2 [16]. It is shown in a one-unit system that each operating time decreases geometrically with the number of repairs for a [ 1: Then, the mean time and variance are



 Xk l Xk r2 E k1 ¼ k1 ; V k1 ¼ 2ðk1Þ : a a a a Pn When Sn  k¼1 Xk =ak1 ;   r2 a2  a2ðn1Þ lða  anþ1 Þ EfSn g ¼ ; ; V fSn g ¼ a2  1 a1 and for a [ 1; EfSn g !

la ; a1

V fSn g !

r2 a 2 a2  1

as n ! 1:

3.5 Modified Renewal Processes

85

Furthermore, a renewal function Mðt; aÞ  Zt

Mðt; aÞ ¼ FðtÞ þ

P1

n¼1

PrfSn  tg is, from (3.5),

M ðaðt  uÞ; aÞdFðuÞ;

ð3:85Þ

0

because PrfSn  tg ¼

Zt

PrfSn1  aðt  uÞgdFðuÞ

ðn ¼ 1; 2; . . .Þ:

0

The distributions of the age dðtÞ and the residual life cðtÞ at time t in (3.22) and (3.23) are rewritten as, respectively, 1 Z t X PrfcðtÞ  xg ¼ Fðt þ xÞ  F ðan ðt þ x  uÞÞdF ðnÞ ðuÞ;

PrfdðtÞ  xg ¼

8 <

n¼1

FðtÞ 

:

0

X1 Z n¼1

tx

Fðan ðt  uÞÞdF ðnÞ ðuÞ

for x  t;

1

for x [ t:

:

0

ð3:86Þ In general, it is very difficult to derive Mðt; aÞ explicitly even if FðtÞ is exponential, and so, there is nothing but to compute it numerically. For example, when FðtÞ ¼ 1  ekt [17, p. 156] (Problem 3.18), " # n n X Y   1 F ðnÞ ðtÞ ¼ 1  ð3:87Þ exp ak1 kt ðn ¼ 1; 2; . . .Þ: kj 1  a k¼1 j¼1;j6¼k The renewal process with interarrival times fX1 ; aX1 ; a2 X1 ; . . .g is also called the quasi-renewal process [18, p. 53]. Example 3.11 (Replacement policy) [15, p. 201] It is assumed in Sect. 2.4.1 that the time between the ðk  1Þth and kth failures is denoted by Xk =ak1 and its mean time is l=ak1 ðk ¼ 1; 2; . . .Þ for a [ 1: Then, the expected cost rate is (Problem 3.19) C1 ðNÞ ¼

c1 ðN  1Þ þ c2 ðN ¼ 1; 2; . . .Þ; lða  aNþ1 Þ=ða  1Þ

where costs c1 and c2 C1 ðN þ 1Þ  C1 ðNÞ  0;

are

given

aNþ1  a c2  ðN  1Þ  c1 a1

in

(3.44).

From

ðN ¼ 1; 2; . . .Þ:

ð3:88Þ the

inequality

ð3:89Þ

86

3 Renewal Processes

It is easily seen that the left-hand side of (3.89) increases strictly from a to 1: Thus, there exists a finite and unique N  ð1  N  \1Þ that satisfies (3.89). If a  c2 =c1 ; then N  ¼ 1: In addition, because the left-hand side of (3.89) also increases with a; N  decreases with a. Next, consider an alternating renewal process defined by two sequences of fXk =ak1 g and fYk =bk1 g for a; b [ 0: Example 3.12 (One-unit system) Consider a one-unit system in Example 3.6 where the failure time for the kth operation is Xk =ak1 with mean time l=ak1 for a  1 and the repair time for the kth repair is Yk =bk1 with mean time b=bk1 for 0\b  1: That is, the mean failure times decrease with their number, and conversely, the mean repair times increase with their number. Suppose that the unit is replaced with a new one at the Nth failure. Then, the expected cost rate is (Problem 3.20) c1 ðN  1Þ þ c2 ðN ¼ 1; 2; . . .Þ; PN1 k1 Þ þ k1 Þ k¼1 ðl=a k¼1 ðb=b

C2 ðNÞ ¼ PN

ð3:90Þ

where c1 ¼ cost of repair and c2 ¼ replacement cost at the Nth failure. Then, from C2 ðN þ 1Þ  C2 ðNÞ  0;  PN1   PN  k1 þ k¼1 b=bk1 c2 k¼1 l=a  ðN  1Þ  ðN ¼ 1; 2; . . .Þ; ð3:91Þ N N1 c1 l=a þ b=b P where 01  0: Thus, if l b l b þ k1 [ kþ1 þ k ; k a b a b i.e., the mean times between failures decrease, then there exists a finite and unique N  ð1  N  \1Þ that satisfies (3.91). If l=ðl=a þ bÞ  c2 =c1 ; then N  ¼ 1:

3.5.2 Discrete Renewal Process Until now, it has been assumed in a renewal process that the sequence of random variables fXk g has an identical continuous distribution FðtÞ: However, the times to some event in actual fields are often measured by the number of cycles, might not be recorded at its exact instant time and be observed statistically per day, per month, per year, and so on. In failure studies, the failure times of units such as tires of jet fighters, switching devices, railroad tracks, and ball bearings might be measured discretely. In general, the conversion from continuous models to discrete ones is possible. However, the results are often too complicated to be useful, unless skillful treatment is used. This section simply shows how to make the conversion.

3.5 Modified Renewal Processes

87

Consider the time over an infinitely long cycle j ðj ¼ 1; 2; . . .Þ and renewal cycle k ðk ¼ 1; 2; . . .Þ: Suppose that Xk has an identical probability function pj ðj ¼ 1; 2; . . .Þ with finite mean l and a probability distribution Pj ; i.e., pj  P PrfXk ¼ jg; Pj  ji¼1 pi ðj ¼ 1; 2; . . .Þ; and 1 X

l

jpj ¼

j¼1

1 X ð1  Pj Þ\1; j¼1

where p0  P0  0: The generating function of a probability function is defined as P j p ðzÞ  1 j¼1 z pj for jzj  1 from Appendix A.4, and hence, P ðzÞ 

1 X

zj Pj ¼

j¼1

Let the total sum of random variables Sn  ðnÞ pj

 PrfSn ¼ jg; where ð2Þ

ð1Þ pj

¼ pj and

ð0Þ pj

p ðzÞ : 1z

Pn

k¼1

ð3:92Þ

Xk and its probability function

 1: Then,

¼ PrfS2 ¼ jg ¼ PrfX1 þ X2 ¼ jg

pj

¼

j X

PrfX2 ¼ j  ijX1 ¼ ig PrfX1 ¼ ig ¼

i¼1

j X

pji pi ;

i¼1

and its generating function is 1 X

ð2Þ

z j pj

¼

j¼1

j 1 X X zj pji pi ¼ ½p ðzÞ2 : j¼1

i¼1

Similarly, by using the mathematical induction, 1 X

ðnÞ

z j pj

¼ ½p ðzÞn

ðn ¼ 0; 1; 2; . . .Þ:

ð3:93Þ

j¼1 ðnÞ

The generating function of Pj 1 X

ðnÞ

zj Pj

¼

j¼1

ð0Þ

 Prf Sn  jg; where Pj

j 1 X X ½p ðzÞn ðnÞ zj pi ¼ 1z j¼1 i¼1

 1; is

ðn ¼ 0; 1; 2; . . .Þ:

Thus, the probability function of Nj  maxn f Sn  jg is   Pr Nj ¼ 0 ¼ Prf X1 [ jg ¼ 1  Pj ;   Pr Nj ¼ n ¼ Prf Sn  j and Snþ1 [ jg ¼ Prf Sn  jg  Prf Snþ1  jg ¼

ðnÞ Pj



ðnþ1Þ Pj

ð3:94Þ

ð3:95Þ

ðn ¼ 0; 1; 2; . . .Þ:   The renewal function Mj  E Nj ðj ¼ 1; 2; . . .Þ that represents the expected number of renewals in cycle j is

88

3 Renewal Processes

Mj ¼

1 X

1   X ðnÞ n Pr Nj ¼ n ¼ Pj

n¼1

¼ Pj þ

n¼1 j X

ð3:96Þ

ðj ¼ 1; 2; . . .Þ;

pi Mji

i¼1

and its generating function is, from (3.92), M ðzÞ 

1 X

zj Mj ¼

j¼1

P ðzÞ 1 p ðzÞ ¼ : 1  p ðzÞ 1  z 1  p ðzÞ

ð3:97Þ

In particular, when pj ¼ pqj1 ðj ¼ 1; 2; . . .Þ; where q  1  p; p ðzÞ ¼

pz ; 1  qz

P ðzÞ ¼

1 pz ; 1  z 1  qz

and hence, from (3.97), M  ðzÞ ¼

pz ð1  zÞ

2

¼p

1 X

jzj ;

i.e.,

Mj ¼ jp ¼

j¼1

j ; E fX k g

that is the same result as the exponential case of the continuous renewal process shown in Example 3.2. Example 3.13 (Age replacement policy) [11, p. 80] Consider the unit with a failure probability function pj that should be operating for the time over an infinitely long cycle j ðj ¼ 1; 2; . . .Þ: The unit is replaced at a planned cycle N ðN ¼ 1; 2; . . .Þ after its installation or at failure, whichever occurs first. Letting Zk  minfXk ; N g ðk ¼ 1; 2; . . .Þ; a sequence of fZk g1 k¼1 forms a discrete renewal process with an independent and identical distribution

Pj for j  N; i¼1 pi Prf Zk  jg ¼ 1 for j [ N; and its mean time is E fZk g ¼

N X j¼1

jpj þ N

1 X j¼Nþ1

pj ¼

N X 1 X

pj :

i¼1 j¼i

Thus, by the similar method of obtaining (3.42), the expected cost rate is c1 PrfXk  N g þ c2 PrfXk [ N g EfZk g P P c1 Nj¼1 pj þ c2 1 j¼Nþ1 pj ðN ¼ 1; 2; . . .Þ; ¼ PN P1 i¼1 j¼i pj

CðNÞ ¼

where replacement costs are given in (3.42).

ð3:98Þ

3.5 Modified Renewal Processes

89

We seek an optimum replacement cycle N  that minimizes CðNÞ: Let hj  P pj = 1 i¼j pi ðj ¼ 1; 2; . . .Þ be the discrete failure rate of the probability function pj with finite mean l: Suppose that hj increases strictly with j: From the inequality CðN þ 1Þ  CðNÞ  0;

hNþ1

N X 1 X i¼1 j¼i

pj 

N X

pj 

j¼1

c2 c1  c 2

ðN ¼ 1; 2; . . .Þ:

ð3:99Þ

Letting LðNÞ be the right-hand side of (3.99),

LðN þ 1Þ  LðNÞ ¼ ðhNþ2  hNþ1 Þ

N þ1 X 1 X

pj [ 0;

i¼1 j¼i

Lð1Þ  lim LðNÞ ¼ lh1  1: N!1

Therefore, if Lð1Þ [ c2 =ðc1  c2 Þ; i.e., h1 [ K; where K  c1 =½lðc1  c2 Þ; then there exists a finite unique minimum N  ð1  N  \1Þ that satisfies (3.99), and the resulting cost rate is ðc1  c2 ÞhN   CðN  Þ\ðc1  c2 ÞhN  þ1 : Conversely, if h1  K; then N  ¼ 1; i.e., no planned replacement should be done. Note that 0\hj  1 from the definition of the failure rate hj : Thus, if K  1; then N  ¼ 1: In particular, assume that the failure distribution is a negative binomial distribution pj ¼ jp2 qj1 ðj ¼ 1; 2; . . .Þ; where q  1  p ð0\p\1Þ: Then, l ¼ ð1 þ qÞ=p and hj ¼ jp2 =ðjp þ qÞ that increases strictly from p2 to p: Therefore, from the above results, if c1 q [ c2 ð1 þ qÞ; an optimum N  ð1  N  \1Þ is given by a finite and unique minimum that satisfies ðN þ 1Þpð1 þ qÞ þ qNþ2 c1  : Np þ 1 c1  c 2 For example, when c1 ¼ 10; c2 ¼ 1; and l ¼ 9; i.e., p ¼ 1=5; N  ¼ 4 and Cð4Þ ¼ 0:92: In this case, Cð1Þ ¼ c1 =l ¼ 1:11: Consider the discrete alternating renewal process with fXk g and fYk g that have the respective probability functions pj  PrfXk ¼ jg and qj  PrfYk ¼ jg; Pj  Pj Pj i¼1 pi ; and Qj  i¼1 qi : Other assumptions are the same as those of Sect. 3.4. Let Mij ðnÞ ði; j ¼ 0; 1Þ denote the expected number of visits to State j in n cycles, starting from State i at cycle 1; and its generating function is Mij ðzÞ ¼ P1 n n¼1 z Mij ðnÞ for jzj\1: Then, the results in Sect. 3.4 are rewritten as

90

3 Renewal Processes

M01 ðnÞ ¼ Pn þ

n X

pk M11 ðn  kÞ;

M10 ðnÞ ¼ Qn þ

k¼1

M11 ðnÞ ¼

n X

n X

qk M00 ðn  kÞ;

k¼1

qk M01 ðn  kÞ;

M00 ðnÞ ¼

k¼1

n X

pk M10 ðn  kÞ:

k¼1

Forming the generating functions as the above equations and solving them for Mij ðzÞ;   ðzÞ ¼ p ðzÞM10 ðzÞ; M00   M11 ðzÞ ¼ q ðzÞM01 ðzÞ;

1 p ðzÞ ; 1  z 1  p ðzÞq ðzÞ 1 q ðzÞ  : M10 ðzÞ ¼ 1  z 1  p ðzÞq ðzÞ  M01 ðzÞ ¼

ð3:100Þ

Next, let ZðnÞ denote the state of the process at cycle n; and define Pij ðnÞ ¼ PrfZðnÞ ¼ jjZð1Þ ¼ ig: Then, from (3.54), the results in continuous time are rewritten as n X P00 ðnÞ ¼ 1  Pn þ pk P10 ðn  kÞ; k¼1

P11 ðnÞ ¼ 1  Qn þ

n X

qk P01 ðn  kÞ;

k¼1

P10 ðnÞ ¼

n X

qk P00 ðn  kÞ;

P01 ðnÞ ¼

k¼1

P10 ðzÞ ¼ q ðzÞP00 ðzÞ;

pk P11 ðn  kÞ;

k¼1

and their generating functions Pij ðzÞ  P01 ðzÞ ¼ p ðzÞP11 ðzÞ;

n X

P1

n¼1

zn Pij ðnÞ are

1 1  p ðzÞ ; 1  z 1  p ðzÞq ðzÞ 1 1  q ðzÞ : P11 ðzÞ ¼ 1  z 1  p ðzÞq ðzÞ P00 ðzÞ ¼

ð3:101Þ

Clearly, P01 ðnÞ ¼ M01 ðnÞ  M00 ðnÞ;

P10 ðnÞ ¼ M10 ðnÞ  M11 ðnÞ:

Example 3.14 (One-unit system with repair) Consider a one-unit system with repair maintenance whose failure and repair distributions are geometric, i.e., pk ¼ pqk1 and qk ¼ abk1 ðk ¼ 1; 2; . . .Þ; where p þ q ¼ 1 and a þ b ¼ 1: Then, the results in Example 3.6 are rewritten as pz ; ð1  zÞ½1  ðb  pÞz pz 1  bz  M01 ðzÞ ¼ : 1  z ð1  zÞ½1  ðb  pÞz P01 ðzÞ ¼

3.5 Modified Renewal Processes

91

Inverting the generating functions for 1 [ b  p [ 0 (Problem 3.21), p ½1  ðb  pÞn ; pþa

p p½1  ðb  pÞn  ; na þ M01 ðnÞ ¼ pþa pþa P01 ðnÞ ¼

and as n ! 1; P01 ðnÞ !

p ; pþa

M01 ðnÞ pa ! : n pþa

3.6 Problems 3 3.1 Prove that MðtÞ\1 for all t\1 [6, p. 57, 9, p. 287]. 3.2 In Example 3.2, show that when k ¼ 3 and k ¼ 4 [4, p. 162], respectively, pffiffiffi 

3 1 2 1:5kt p p ffiffi ffi sin kt þ ; e MðtÞ ¼ kt  1 þ 2 3 3 3

 pffiffiffi 1 3 1 p : MðtÞ ¼ kt  þ e2kt þ 2ekt sin kt þ 4 2 2 4 3.3 Prove that when X1 and X2 have a gamma distribution with the respective parameters ða; kÞ and ðb; kÞ; X  X1 þ X2 has a gamma distribution ða þ b; kÞ: a 3.4 When FðtÞ ¼ 1  eðktÞ ; show that (       2 ) 1 1 1 2 1 EfXg ¼ C 1 þ ; VfXg ¼ 2 C 1 þ  C 1þ ; k a a a k and using the discretization algorithm, compute MðtÞ for a = 1, 1.5, 2.0, 2.5, 3.0 and t = 1, 2, ......, 10 when k = 1, and compare them with t/E{X} given by the approximation of an elementary renewal theorem in Theorm 2.3 3.5 For a two-unit parallel system with an exponential failure distribution ð1  ekt Þ; show that the failure distribution of the system is FðtÞ ¼ ð1  ekt Þ2 and a renewal function is [24, p. 161]

2 1 3kt kt  ð1  e Þ : MðtÞ ¼ 3 3 3.6 When FðtÞ ¼ 1  ð1 þ ktÞekt ; derive VfNðtÞg and (3.15) in Theorem 3.1. 3.7 In Example 3.3, compute the number n of units when the probability that the number of failures is less than n in t ¼ 104 hours is given by a ¼ 0:95; 0:99:

92

3 Renewal Processes

3.8 Show (3.25) and (3.26) [19, p. 259]. 3.9 Prove (3.27). 3.10 In inequalities of MðtÞ and HðtÞ; prove that t tFðtÞ  1  MðtÞ  R t : 0 FðuÞdu 0 FðuÞdu

Rt

3.11 In (3.29), prove when F(t) is IFR, FðtÞ Rt

FðuÞdu

 hðtÞ  1 R

FðtÞ

:

FðuÞdu

t

0

3.12 In (3.38), prove that when FðtÞ is IFR, l1 decreases from 1=hð0Þ to l; and l2 increases from 0 to 1: 3.13 In Sect. 3.3.2.2, prove that when AðTÞ ¼ c1 FðTÞ þ c2 FðTÞ and BðTÞ  c1 MðTÞ þ c2 ; BðTÞ ¼ AðTÞ þ

ZT

BðT  tÞdFðtÞ; BðTÞ ¼ AðTÞ þ

Z

T

AðT  tÞdMðtÞ: 0

0

3.14 Show that the interval reliability in (3.67), when T0 is a random variable with an exponential distribution ð1  eat Þð0\a\1Þ; is RðT; x; aÞ ¼ aR ðT; x; aÞ; and derive an optimum policy that maximizes RðT; x; aÞ [11, p. 143] when the failure rate increases strictly. 3.15 If the time for PM has a distribution G2 ðtÞ with mean b2 and the time for repair has G1 ðtÞ with mean b1 ; then show that the limiting interval reliability in (3.68) is [11, p. 143] R Tþx FðtÞdt x RðT; xÞ ¼ R T : 0 FðtÞdt þ b1 FðTÞ þ b2 FðTÞ 3.16 In Example 3.9, compute the time d when the probability that the system is down more than d in t ¼ 10; 000 h of operation is given by a ¼ 0:95; 0:99: 3.17 In Example 3.10, derive optimum policies when (i) cr ðtÞ ¼ at; GðtÞ ¼ pffi 1  eh t ; and (ii) cr ðtÞ ¼ at2 ; GðtÞ ¼ 1  eht : 3.18 Derive (3.87). 3.19 Derive (3.88) and (3.89). 3.20 Derive (3.90) and (3.91). 3.21 In Example 3.14, derive P01 ðnÞ and M01 ðnÞ: 3.22 A generalized renewal process in which Snþ1 ¼ Sn þ Anþ1 Xnþ1

ðn ¼ 0; 1; 2; . . .Þ

is applied to a imperfect repair model [20]. When An ¼ a; derive PrfXnþ1  xjSn ¼ tg:

3.6 Problems 3

93

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20.

Barlow RE, Proschan F (1965) Mathematical theory of reliability. Wiley, New York Prabhu NU (2007) Stochastic processes. World Scientific, Singapore Parzen E (1962) Stochastic process. Holden-Day, San Francisco Beichelt FE (2006) Stochastic processes in science, engineering and finance. Chapman & Hall, Boca Raton Osaki S (1992) Applied stochastic system modeling. Springer, Berlin Ross SM (1983) Stochastic processes. Wiley, New York Cox DR (1962) Renewal theory. Methuen, London Feller W (1968) An introduction to probability theory and its application, vol I, 3rd edn. Wiley, New York Çinlar E (1975) Introduction to stochastic processes. Prentice-Hall, Englewood Cliffs Takács L (1960) Stochastic processes. Wiley, New York Nakagawa T (2005) Maintenance theory of reliability. Springer, London Tijms HC (2003) A first course in stochastic models. Wiley, Chichester Dohi T, Kaio N, Osaki S (2002) Renewal processes and their computational aspects. In: Osaki S (ed) Stochastic models in reliability and maintenance. Springer, Berlin Xie M (1989) On the solution of renewal-type integral equations. Commun Statist B 18:281– 293 Calabro SR (1962) Reliability principles and practices. McGraw-Hill, New York Lam Yeh (2007) Geometric process and its applications. World Scientific, Singapore Nakagawa T (2007) Shock and damage models in reliability theory. Springer, London Wang H, Pham H (2006) Reliability and optimal maintenance. Springer, London Rausand M, Høyland A (2004) System reliability theory. Wiley, Hoboken Kijima M (2002) Generalized renewal processes and general repair models. In: Osaki S (ed) Stochastic models in reliability and maintenance. Springer, Berlin

Chapter 4

Markov Chains

In Sect. 3.4, we consider the system that repeats up and down such as operating and failed states alternately. Next, as one example of extended models, we take up the system with repair maintenance: The system begins to operate at time 0 and undergoes repair according to two types of failures such as minor and major failures. After the repair completion, the system becomes like new and begins to operate again. To form the stochastic modeling of the above system, we define the following three states: State 0: System is operating. State 1: System is under repair according to minor failure. State 2: System is under repair according to major failure. In general, the repair time and cost required for major failure might be greater than those for minor failure. The system repeats transitions between the states of 0 ! 1 ! 0 or 0 ! 2 ! 0 (Fig. 4.1). We explain simply and clearly Markov processes on the subject of the system: Suppose that the failure occurs according to a general distribution FðtÞ, and when it has occurred, the probability of minor failure is a ð0  a  1) and the probability of major failure is 1  a, where both probabilities are independent of failure times. The distribution of time to the repair completion of minor and major failures are G1 ðtÞ and G2 ðtÞ, respectively. If we pay no regard to transition times and are concerned only with transition probabilities among three states, we express the process in the following matrix form: 2 1 00 1 0 0 a 1a P ¼ 1 @1 0 0 A: 2 1 0 0

T. Nakagawa, Stochastic Processes, Springer Series in Reliability Engineering, DOI: 10.1007/978-0-85729-274-2_4, Ó Springer-Verlag London Limited 2011

ð4:1Þ

95

96

4 Markov Chains

Fig. 4.1 Process of the system with two types of failures

The stochastic process with discrete-state space such as (4.1) is called a discretetime Markov chain. Using the transition matrix in (4.1), we can obtain stochastic behaviors of the system in Sect. 4.1. Next, assume that FðtÞ ¼ 1  ekt and Gj ðtÞ ¼ 1  ehj t (j ¼ 1; 2). We investigate the states of the system at time t. Let ZðtÞ denote the state of the system at time t when it starts from State 0, and define that Pj ðtÞ  PrfZðtÞ ¼ jg (j ¼ 0; 1; 2), that is called a transition probability. Note that the process has the memoryless property irrespective of elapsed operating and repair times because of exponential distribution in Sect. 2.1.1. Thus, we have the following equations: For small h [ 0, P0 ðt þ hÞ ¼ P0 ðtÞekh þ P1 ðtÞð1  eh1 h Þ þ P2 ðtÞð1  eh2 h Þ þ oðhÞ; P1 ðt þ hÞ ¼ P1 ðtÞeh1 h þ P0 ðtÞað1  ekh Þ þ oðhÞ; P2 ðt þ hÞ ¼ P2 ðtÞeh2 h þ P0 ðtÞð1  aÞð1  ekh Þ þ oðhÞ; where oðhÞ=h ! 0 as h ! 0. From these equations, P0 ðt þ hÞ  P0 ðtÞ 1  ekh 1  eh1 h 1  eh2 h oðhÞ þ P1 ðtÞ þ P2 ðtÞ þ ¼ P0 ðtÞ ; h h h h h P1 ðt þ hÞ  P1 ðtÞ 1  eh1 h 1  ekh oðhÞ þ P0 ðtÞa þ ¼ P1 ðtÞ ; h h h h P2 ðt þ hÞ  P2 ðtÞ 1  eh2 h 1  ekh oðhÞ þ P0 ðtÞð1  aÞ þ ¼ P2 ðtÞ : h h h h As h ! 0, we have the differential equations P00 ðtÞ ¼ kP0 ðtÞ þ h1 P1 ðtÞ þ h2 P2 ðtÞ; P01 ðtÞ ¼ h1 P1 ðtÞ þ akP0 ðtÞ; P02 ðtÞ ¼ h2 P2 ðtÞ þ ð1  aÞkP0 ðtÞ: In a matrix form, 0

1 0 P00 ðtÞ k h1 P0 ðtÞ  @ P01 ðtÞ A ¼ @ ak h1 P02 ðtÞ ð1  aÞk 0

1 h2 0 APðtÞ: h2

ð4:2Þ

4 Markov Chains

97

Therefore, by solving the above differential equations mathematically and numerically, we can obtain the probabilities Pj ðtÞ (j ¼ 0; 1; 2). The stochastic process, whose conditional probability of transition to the future states depends only on the present state and is independent of the past history, is called a Markov property. The stochastic process with Markov property at any time is a Markov process. Many real-world phenomena can be modeling stochastically by Markov processes because we may observe the phenomena only at Markov property points without keeping track of the time elapsed since the last point. In other words, if we know the state for any specific value of time, we can predict the stochastic behavior beyond the point. Stochastic processes with independent increments defined in Sect. 2.2 always have the Markov property. Markov processes can be analyzed by forming differential equations such as (4.2) and are also called a continuous-time Markov chain. More detailed discussions for Markov processes are done in Sect. 4.2.

4.1 Discrete-Time Markov Chain 4.1.1 Transition Probabilities Consider a discrete-time stochastic process fXn ; n ¼ 0; 1; 2; . . .g with a finite discrete-state set f0; 1; 2; . . .; mg. Note that the event fXn ¼ jg represents that the process is in State j (j ¼ 0; 1; . . .; m) at time n (n ¼ 0; 1; 2; . . .). That is, the process moves among (m þ 1) states at some unit of time, time unit, or jump according to a given probability law. Then, we want to know mainly in which state the process is at time n and converges as n becomes larger. If we suppose that PrfXnþ1 ¼ jjX0 ¼ i0 ; X1 ¼ i1 ; . . .; Xn ¼ in g ¼ PrfXnþ1 ¼ jjXn ¼ in g

ð4:3Þ

for all i0 ; i1 ; . . .; in and all n  0, then the process fXn ; n ¼ 0; 1; 2; . . .g is said to be a Markov chain (Markov 1806). This property indicates that, given the value of Xn , the future value of Xnþ1 does not depend on the value of Xk for 0  k  n  1. If the probability of Xnþ1 being in State j, given that Xn is in State i, is independent of n, i.e., PrfXnþ1 ¼ jjXn ¼ ig  Pij ; then the process has a stationary or homogeneous (one-step) transition probability, i.e., the process transits from State i to State j with probability Pij and remains in State i with probability Pii regardless of n. In other words, the transition probability from State i to State j does not change even if time varies. We restrict ourselves only to discrete-time Markov chains with stationary transition probabilities and a finite state space, because most reliability systems form such processes. A matrix

98

4 Markov Chains

P  fPij g is called a transition probability matrix, and manifestly, Pij satisfies P Pij  0 and m j¼0 Pij ¼ 1 for any i, j. Let P denote the transition matrix given by 0

P00 B P10 B . B . B . P  ðPij Þ ¼ B B Pi0 B . @ . . Pm0

P01 P10 .. . Pi1 .. . Pm1

1 P0m P1m C .. C C . C C: . . . Pim C .. C A . . . . Pmm ... ...

Example 4.1 (Alternating renewal process) Consider an alternating renewal process in Sect. 3.4, where a unit begins to operate at time 0 and undergoes repair at failures. After the repair completion, a unit becomes like new and begins to operate again. Then, we define State 0: unit is operating. State 1: unit is under repair. The process has the simplest Markov chain with two states f0; 1g, and a transition probability matrix is     P00 P01 0 1 P : ¼ P10 P10 1 0 The process repeats alternately between operating and repair. When the repair is made, the unit operates like new with probability 0:8, however, it undergoes repair again because of its imperfectness with probability 0:2. Then, a transition probability matrix is   0 1 P¼ : 0:8 0:2 It is helpful to understand the process whose behavior is illustrated by using state transition diagram as shown in Figs. 4.1 and 4.2, where the number circled denotes a state and the number appearing on an arc is a transition probability or a transition rate (Fig. 4.3). It is indispensable to draw a state transition diagram and to make clear of the behavior for modeling reliability systems stochastically. Example 4.2 (One-unit system) Consider a one-unit system with repair in Example 3.6, where the failure and repair times have exponential distributions

Fig. 4.2 Diagram of oneunit system

4.1 Discrete-Time Markov Chain

99

Fig. 4.3 Diagram of oneunit system with exponential failure and repair times

ð1  ekt Þ and ð1  eht Þ, respectively. To know the behavior of the system, we check the unit at periodic times kT (k ¼ 1; 2; . . .). If the unit has failed at some check, it undergoes repair immediately, and otherwise, leaves alone. Similarly, if the repair is completed at some check, it begins to operate, and otherwise, continues to make the repair. Then, because of the memoryless property of exponential distributions, the transition matrix is    kT  P00 P01 1  ekT e P¼ ¼ ; P10 P10 1  ehT ehT and the state transition diagram is shown in Fig. 4.3. When T ! 1, and ekT ¼ 0 and ehT ¼ 0:2, the transition matrices correspond to those in Example 4.1, respectively. A Markov chain is completely specified by the transition probabilities Pij and an initial probability PrfX0 ¼ ig. Let Pnij denote the probability that the process goes from State i to State j in n transitions that is called the n-step transition probability, i.e., Pnij  PrfXnþk ¼ jjXk ¼ ig P0ij P0ii

0

ðn ¼ 1; 2; . . .Þ;

for i 6¼ j;

 1;

P n 1 n Pnij  0; m j¼0 Pij ¼ 1 for all n  0, and Pij  Pij for any i, j. Note that Pij is independent of the present time k and depends only on the time duration n. Then, Pnij ¼ PrfXnþk ¼ jjXk ¼ ig ¼

m X

PrfXnþk ¼ j; Xrþk ¼ ljXk ¼ ig

l¼0

¼ ¼

m X l¼0 m X

PrfXnþk ¼ jjXrþk ¼ lgPrfXrþk ¼ ljXk ¼ ig Pril Pnr lj

ðr ¼ 0; 1; 2; . . .; nÞ;

ð4:4Þ

l¼0

that is called the Chapman–Kolmogorov equation. In a matrix form, from (4.4),   PðnÞ  Pnij ¼ Pð1Þ  Pðn1Þ ¼ P  Pðn1Þ ¼    ¼ Pn ;

ð4:5Þ

100

4 Markov Chains

i.e., PðnÞ can be computed by the nth power of matrix P. Thus, (4.4) is written in the following matrix form Pn ¼ Pr  Pnr

ðr ¼ 0; 1; 2; . . .; nÞ;

ð4:6Þ

where note that Pð0Þ ¼ P0 ¼ I is the identity matrix where all diagonal elements are 1. The Chapman–Kolmogorov equation in (4.4) plays a great role in analyzing Markov chains that will be shown in (4.25). Therefore, the probability that the process is in State j at time n, irrespective of an initial state, is pj ðnÞ  PrfXðnÞ ¼ jg ¼

m X

PrfXðnÞ ¼ j;

Xð0Þ ¼ ig

i¼0

¼ ¼

m X i¼0 m X

PrfXðnÞ ¼ jjXð0Þ ¼ igPrfXð0Þ ¼ ig pi ð0ÞPnij

ðj ¼ 0; 1; 2; . . .; mÞ;

ð4:7Þ

i¼0

P where note that m j¼0 pj ðnÞ ¼ 1 for all n  0. Letting pðnÞ  fp0 ðnÞ; p1 ðnÞ; . . .; pm ðnÞg, (4.7) is written in a matrix form pðnÞ ¼ pð0ÞPðnÞ

ðn ¼ 0; 1; 2; . . .Þ:

ð4:8Þ

Example 4.3 In Example 4.2, setting a  1  ekT and b  1  ehT [1, p. 101],   1a a ; P¼ b 1b ! ð1  aÞa þ ð1  bÞa ð1  aÞ2 þ ab 2 : P ¼ ð1  aÞb þ ð1  bÞb ab þ ð1  bÞ2 In general (Problem 4.1), 1 P ¼ aþb n



b b

a a



ð1  a  bÞn þ aþb



a b

 a : b

If the initial state is pð0Þ ¼ ð1; 0Þ, i.e., the unit starts to operate at time 0, 1 ð1  a  bÞn ða; aÞ ðb; aÞ þ aþb aþb   b þ að1  a  bÞn a½1  ð1  a  bÞn  ; : ¼ aþb aþb

pðnÞ ¼

For example, when kT ¼ 0:1 and hT ¼ 0:2, i.e., a ¼ 0:095 and b ¼ 0:181, pðnÞ ¼ ð0:656 þ 0:344ð0:724Þn ; 0:344½1  ð0:724Þn Þ

4.1 Discrete-Time Markov Chain

101

that goes to pð1Þ  limn!1 pðnÞ ¼ ð0:656; 0:344Þ, and the values of 0.656 and 0.344 are approximately equal to the availability h=ðk þ hÞ and the unavailability k=ðk þ hÞ in Example 4.2.

4.1.2 Classification of States Two States i and j communicate if and only if there exist integers n  0 and k  0 such that Pkij [ 0 and Pnji [ 0. By using a communication relation, all states are classified into several classes. Furthermore, we define the first-passage distribution as Fijn  PrfXn ¼ j; Xk 6¼ j; k ¼ 1; 2; . . .; n  1jX0 ¼ ig ðn ¼ 1; 2; . . .Þ; that is the probability that starting in State i, the first transition into State j occurs at P n the nth transition, where Fij0  0 for all i, j. In addition, we define Fij  1 n¼1 Fij that represents the eventual transition probability from State i to State j. All states in a finite Markov chain are classified into two kinds of states: (i) If Fii ¼ 1, then State i is said to be recurrent, (ii) If Fii \1, then State i is said to be transient. A recurrent set of states is one in which all states communicate and which cannot be left, once it is entered. A transient set of states is one in which all states communicate and which might be left even if it is entered. We have the following relation between a transition probability Pnij and a firstpassage distribution Fijn Pnij ¼

n X

Fijk Pnk jj

ðn ¼ 1; 2; . . .Þ;

ð4:9Þ

k¼0

because Pnij is denoted by the probability that the process transits from State i to State j for the first time at the kth step (k ¼ 1; 2; . . .; n) and returns to State j at the ðn  kÞth step, where note that P0ij ¼ 0 for i 6¼ j, and P0jj ¼ 1. Let P ðzÞ and Fij ðzÞ be the generating functions of Pnij and Fijn , i.e., Pij ðzÞ  P1 n n P1 n ijn  n¼0 z Pij and Fij ðzÞ  n¼0 z Fij for jzj\1 from Appendix A.4. Then, (4.9) is rewritten by the generating function, for i 6¼ j, Pij ðzÞ ¼ ¼

1 X n¼0 1 X k¼0

zn Pnij ¼

1 X n¼0

zk Fijk

1 X n¼0

zn

n X

Fijk Pnk ij

k¼0

zn Pnjj ¼ Fij ðzÞPjj ðzÞ;

ð4:10Þ

102

4 Markov Chains

and for i ¼ j, Pii ðzÞ ¼ 1 þ Fii ðzÞPii ðzÞ;

Pii ðzÞ ¼

i.e:;

1 : 1  Fii ðzÞ

ð4:11Þ

Next, let Mijn denote the expected number of visits to State j in the nth transition if the process starts from State i, not including the first at time 0. Then, clearly  Pn k ðn ¼ 1; 2; . . .Þ; k¼1 Pij Mijn ¼ ð4:12Þ 0 ðn ¼ 0Þ: Forming the generating function of (4.12), for i 6¼ j, Mij ðzÞ 

1 X

zn Mijn ¼

Pij ðzÞ ; 1z

zn Miin ¼

Pii ðzÞ  1 : 1z

n¼0

Mii ðzÞ 

1 X n¼0

ð4:13Þ

Using Theorem A.5 of Appendix A, when State i is recurrent, lim z!1

1 X n¼0

zn Fiin ¼ lim Fii ðzÞ ¼ z!1

1 X

Fiin ¼ 1;

n¼0

and hence, from (4.11), lim Pii ðzÞ ¼ lim z!1

z!1

1 X

zn Pnii ¼ 1:

n¼0

Thus, using Theorem A.5, 1 X

Pnii ¼ 1:

n¼0

P1

n n¼0 Pii

¼ 1, we assume that Fii \1. Then, from (4.11), Next, when limz!1 Pii ðzÞ\1 using Theorem A.5. In addition, using Theorem A.5 again, P1 n P1 n n¼0 Pii \1 that contradicts the assumption. Thus, if n¼0 Pii ¼ 1, then Fii ¼ 1. When State i is transient, we can make similar discussions. Therefore, we have the following results: Theorem 4.1 P1 n (i) State i is recurrent if and only if P ¼ 1. P1n¼0 nii (ii) State i is transient if and only if P n¼0 ii \1. The above results show that for any recurrent State i, the process can revisit State i infinitely. For any transient State i, the process transits to states of another class after it revisits to State i finitely, and cannot return to State i.

4.1 Discrete-Time Markov Chain

103

Example 4.4 In Example 4.3, it is assumed that 0\a\1 and b ¼ 0, i.e., any failed units remain a failed state without making repair. Then, a transition probability matrix is   1a a P¼ ; 0 1 and 1 X

1 Pn00 ¼ 1 þ ð1  aÞ þ ð1  aÞ2 þ    ¼ ; a n¼0

1 X

Pn11 ¼ 1 þ 1 þ    ¼ 1;

n¼0

that implies that State 0 is transient and State 1 is recurrent. In general, the transition probability of a finite Markov chain can be expressed in a matrix form [1, p. 124, 2, p. 36]

(4.14)

where C1 ; C2 ; . . .; Ck are all the sets of recurrent classes and T is a set of transient states.

4.1.3 Limiting Probabilities P n Recall that if State i is recurrent, then Fii ¼ 1 and 1 conversely, n¼0 Pii ¼ 1, andP P1 n n if State i is transient, then Fii \1 and n¼0 Pii \1. Define that lj  1 n¼1 nFjj is the mean recurrence time for State j (j ¼ 0; 1; . . .; m). Theorem 4.2 (i) If State jis recurrent, then Pnjj !

1 [0 lj

as n ! 1:

ð4:15Þ

(ii) If State jis transient, then Pnjj ! 0 as n ! 1:

ð4:16Þ

104

4 Markov Chains

(iii) If State jis recurrent, then Pnij !

Fij : lj

ð4:17Þ

Proof From (4.12), Mjjn  Mjjn1 ¼ Pnjj !

1 lj

as n ! 1:

P n Furthermore, because m j¼0 Pjj ¼ 1 for any n  0, 1=lj [ 0. If State j is transient, P1 n n n¼1 Pjj \1, and hence, Pjj ! 0 as n ! 1. If a Markov chain starts from State i, visits State j in a recurrent class with probability Fijn and results infinitely State j. Thus, noting that Fij represents the eventual transition probability from State i to State j, we get easily (iii). h Theorem 4.3 If States i and j (i; j ¼ 0; 1; . . .; m) are in the same recurrent class, pj  limn!1 Pnij [ 0, that is a unique and positive solution to satisfy pj ¼

m X

pi Pij

ðj ¼ 0; 1; . . .; mÞ;

i¼0

m X

pj ¼ 1:

ð4:18Þ

j¼0

Proof From the assumption that both States i and j are in the same recurrent class, Fij ¼ 1, and hence, limn!1 Pnij ¼ 1=lj [ 0 from (i) and (iii) of Theorem 4.2. Furthermore, from (4.7), pj ðnÞ ¼

m X

pi ðn  1ÞPij ;

i¼0

m X

pj ðnÞ ¼ 1 for all n  1:

j¼0

Letting n ! 1, we easily have (4.18).

h

Two methods of solving (4.18) were proposed [3, p. 107]; direct method and iterative method. However, because the number of states is not so large in most reliability models, it would be sufficient to solve (4.18) by the direct method. Note that (4.18) is expressed in a matrix form p  ðp0 ; p1 ; . . .; pm Þ ¼ pP:

ð4:19Þ

n Therefore, we summarize the liming probabilities P1 ij  limn!1 Pij of a finite Markov chain in (4.14)

4.1 Discrete-Time Markov Chain

105

1 ði; j 2 Cl l ¼ 1; 2; . . .; kÞ; lj Fij ¼ ði 2 T; j 2 Cl l ¼ 1; 2; . . .; kÞ; lj

P1 ij ¼ P1 ij

P1 ij ¼ 0 ði 2 Cl ; j 2 Cr

ð4:20Þ

l 6¼ rÞ;

P1 ij ¼ 0 ði; j 2 TÞ; P1 ij ¼ 0 ði 2 Cl

l ¼ 1; 2; . . .; k; j 2 TÞ:

Example 4.5 When kT ¼ 0:1 and hT ¼ 0:2, i.e., a ¼ 0:095 and b ¼ 0:181 in Example 4.3, a transition probability matrix is 0 P¼ 1



0 0:905 0:181

1  0:095 ; 0:819

that consists of one recurrent class. Then, the limiting probabilities p0 and p1 are, from Theorem 4.3,   0:905 0:095 : ðp0 ; p1 Þ ¼ ðp0 ; p1 Þ 0:181 0:819 Thus, solving two equations p0 ¼ 0:905p0 þ 0:181p1 ;

p0 þ p1 ¼ 1;

p0 0:656 and p1 0:344 that are equal to pð1Þ in Example 4.3. In a transition probability matrix in (4.1), the limiting transition probabilities are given by solving simultaneous equations p1 ¼ p0 a;

p2 ¼ p0 ð1  aÞ;

p0 þ p1 þ p2 ¼ 1:

Thus, p0 ¼ 1=2, p1 ¼ a=2, and p2 ¼ ð1  aÞ=2 (Problem 4.2). Example 4.6 Suppose that a transition probability matrix is given by

106

4 Markov Chains

Then, the recurrent classes are C1 ¼ f0g, C2 ¼ f1; 2g, C3 ¼ f3; 4g, and a transient class is T ¼ f5; 6g. The transition diagram among seven states is shown in Fig. 4.4 For a recurrent class C2 ¼ f1; 2g, the transition probability matrix is 1 0 1 1 C B P2 ¼ @ 2 2 A; 1 3 4 4 and the limiting probabilities are, from (4.18), 1 p1 ¼ ; 3 Similarly, C3 ¼ f3; 4g,

0

2 P ¼ @3 1

1 1 3 A; 0

2 p2 ¼ : 3

3 p3 ¼ ; 4

1 p4 ¼ : 4

We derive the first-passage distributions Fij (i 2 T; j 2 Ck (k ¼ 2; 3)) defined in Sect. 4.1.2 [1, p. 127]. Theorem 4.4 The first-passage distribution Fij from State i (i 2 T) to State j (j 2 Ck ) ðk ¼ 1; 2; . . .Þ satisfies X X Pil þ Pil Flj ði 2 T; j 2 Ck Þ; ð4:21Þ Fij ¼ l2Ck

l2T

and its matrix form is ðFij Þ ¼ ðI  QÞ1 Rk  1;

ð4:22Þ

where 1 is a column vector where all elements are 1. Proof The first-passage distribution Fij has two possibilities: one is to transit from State i to State l (l 2 Ck ) directly, and the other is to transit from State i to State l ðl 2 T) once including itself and then to transit to State j (j 2 Ck ). Thus, we have (4.21). Rewriting (4.21) in a matrix form, Fig. 4.4 Transition diagram of Example 4.6

4.1 Discrete-Time Markov Chain

107

Fij ¼ Rk  1 þ QðFij Þ; i.e., ðFij Þ ¼ ðI  QÞ1 Rk  1: Example 4.7 In Example 4.6,   0 35 ; Q¼ 1 0 3

h  ðI  QÞ ¼

i.e:;

Hence, the inverse matrix is ðI  QÞ1 ¼



5 4 5 12

3 4 5 4

1  13

  35 : 1

 :

Thus, using Theorem 4.4, 0 5 B ðI  QÞ1 R1  1 ¼ @ 4 5 12 0 5 B ðI  QÞ1 R2  1 ¼ @ 4 5 12 0 5 B ðI  QÞ1 R3  1 ¼ @ 4 5 12

10 1 1

0 5 1 CB 5 C B 8 C A@ Að1Þ ¼ @ A; 1 5 17 2 4 24 0 1 1 1 3 10   1 1 0A B C 4 C@ ¼ @ 8 A; A 10 1 5 1 0 0 24 4 1 011 3 10 1 0 1 C B C 4 CB 10 ¼ @ 4 A: A@ A 1 1 5 1 0 6 4 4 3 4

Consequently, 5 F50 ¼ ; 8 17 F60 ¼ ; 24

1 ðj ¼ 1; 2Þ; 8 1 F6j ¼ ðj ¼ 1; 2Þ; 24 F5j ¼

where note that 58 þ 18 þ 14 ¼ 1 and

17 24

1 4 1 F6j ¼ 4

F5j ¼

ðj ¼ 3; 4Þ; ðj ¼ 3; 4Þ;

1 þ 24 þ 14 ¼ 1. Therefore, from (4.20),

5 1 1 1 P1 ; P1 50 ¼ ; 51 ¼ ¼ 8 8 3 23 1 3 3 1 1 1 ; P1 ; P1 53 ¼ ¼ 54 ¼ ¼ 4 4 16 4 4 16 17 1 1 1 P1 ; P1

¼ ; 60 ¼ 61 ¼ 24 24 3 72 1 3 3 1 1 1 1 1 P63 ¼ ¼ ; P64 ¼ ¼ ; 4 4 16 4 4 16 P4 P 4 1 where note that j¼0 P1 j¼0 P6j ¼ 1. 5j ¼

1 2 1 P1 ; 52 ¼ ¼ 8 3 11

P1 62 ¼

1 2 1

¼ ; 24 3 36

108

4 Markov Chains

Collecting these results, a matrix of limiting probabilities is

4.1.4 Absorbing Markov Chain We are sometimes interested in stochastic behaviors before system failure. An absorbing Markov chain consists of one in which all recurrent states are absorbing. The transition probability matrix of an absorbing Markov chain with k absorbing states and transient states can be expressed, from (4.14),

(4.23)

where 0 is zero matrix in which all elements are zero. We obtain the expected number of visits to State j, starting from State i (i; j 2 T), i.e., ði; j 2 TÞ: Mij ¼ EfNj ð1ÞjXð0Þ ¼ ig Theorem 4.5 The expected number of visits from State i to State j is [2, p. 46]   Mij ¼ ðI  QÞ1 : ð4:24Þ Proof Letting Ujn be the function that takes 1 when the process is State j at time n, P n and otherwise, takes 0, it can be easily seen that Nj ð1Þ ¼ 1 n¼0 Uj . Thus,

4.1 Discrete-Time Markov Chain

  Mij ¼

( E

1 X n¼0

¼ ¼

109



Ujn Xð0Þ

)! ¼i

¼

1 n o X E Ujn jXð0Þ ¼ i n¼0

1 h   i X ðnÞ ðnÞ Pij 1 þ 1  Pij 0

! ¼

n¼0 1 X

1 X

!

! ðnÞ Pij

n¼0

Qn ¼ ðI  QÞ1 :

n¼0

Note that at least one of sums of each row in Q is \1 because Q is a matrix of transient states, and hence, the inverse of a matrix ðI  QÞ exists. h Example 4.8 In Example 4.7, ðI  QÞ1

0

5 5B 4 ¼ @ 5 6 12

1 3 4C 5 A: 4

Thus, the expected numbers of visits to transient States 5 and 6 before absorbing are 5=4 þ 3=4 ¼ 2 from State 5 and 5=12 þ 5=4 ¼ 5=3 from State 6, where note that 2 and 5=3 are the mean times from States 5 and 6 to an absorbing state, respectively, because each transition time is a unit of time in a Markov chain. Example 4.9 (Two-unit parallel system with spare units) Consider a two-unit parallel system with no repair, however, spare units in inventory: Two units begin to operate at time 0. If one of operating unit fails before time T, the spare unit is ordered immediately and is placed into service as soon as it is delivered after a lead time L. If two operating units do not fail up to time T, the spare unit is ordered at time T and is delivered after time L. In this case, when two units have failed, the spare unit is put into service and the spare unit is ordered immediately. When only one unit has failed, the spare unit is put into service, and when two units are operating, the spare unit is put into inventory. It is assumed that each unit fails according to an identical exponential distribution ð1  ekt Þ. We define the states: State State State State State State

0: 1: 2: 3: 4: 5:

Two units are operating and there is no spare. Two units are operating and spare is ordered. One unit is operating and spare is ordered. Two units are operating and spare is delivered. One unit is operating and spare is delivered. Two units have failed and spare is delivered.

We draw a state transition diagram of the above system in Fig. 4.5. Referring to the diagram, transition probabilities are P02 ¼ 1  e2kT ; P13 ¼ e2kL ; P01 ¼ e2k T ; kL kL kL 2 P14 ¼ 2e ð1  e Þ; P15 ¼ ð1  e Þ ; P24 ¼ ekL ; P25 ¼ 1  ekL ; P30 ¼ P40 ¼ P52 ¼ 1;

110

4 Markov Chains

Fig. 4.5 Diagram of twounit parallel system with spare units in inventory

and its matrix form is 1 00 0 0 e2kT 1B 0 B0 2B 0 0 P¼ B 3B 1 0 B 4 @1 0 5 0 0

2 1  e2kT 0 0 0 0 1

3 0 e2kL 0 0 0 0

4 5 1 0 0 2ekL ð1  ekL Þ ð1  ekL Þ2 C C ekL 1  ekL C C: C 0 0 C A 0 0 0 0

When kT ¼ 0:1 and kL ¼ 0:05, the transition probability matrix is 0 1 2 P¼ 3 4 5

1 00 0 0:8187 B0 0 B B0 0 B B1 0 B @1 0 0 0

2 0:1813 0 0 0 0 1

3 0 0:9048 0 0 0 0

4 5 1 0 0 0:0928 0:0024 C C 0:9512 0:0488 C C: 0 0 C C 0 0 A 0 0

Suppose that State 5 is in a failed state of the system and is absorbing. Striking out row 5 and column 5 of I  P and inverting it, we can compute the expected number of visits to State j before system failure 0 1 2 3 4

0 1 2 3 4 1 488:944 400:303 88:360 392:387 95:557 B 488:800 401:185 88:604 393:241 95:549 C C B B 484:079 396:319 88:749 388:483 95:597 C : C B @ 488:944 400:303 88:630 393:387 95:557 A 488:944 400:303 88:630 392:387 96:557 0

Thus, the mean time from Sate 0 to State 5 is 1; 465:551 ð1=kÞ by summing the number of column 0. For example, when 1=k ¼ 100, the mean time is 146, 555:1 (Problem 4.3).

4.2 Continuous-Time Markov Chain

111

4.2 Continuous-Time Markov Chain 4.2.1 Transition Probabilities Consider a continuous-time stochastic process fXðtÞ; t  0g with a finite state set f0; 1; 2; . . .; mg. Note that the event fXðtÞ ¼ jg represents that the process is in State j (j ¼ 0; 1; 2; . . .; m) at time t. Then, we want to know mainly in which state the process is at time t and the process converges as t becomes larger. If we suppose that PrfXðtÞ ¼ jjXðt0 Þ ¼ i0 ; Xðt1 Þ ¼ i1 ; . . .; Xðtn Þ ¼ in g ¼ PrfXðtÞ ¼ jjXðtn Þ ¼ in g for all i0 , i1 , . . ., in and 0  t0 \t1 \    \tn \t, then the process fXðtÞg is said to be a continuous-time Markov chain. This property indicates that, given the value of Xðtn Þ, the future value of XðtÞ for t [ tn does not depend on the value of Xðtn1 Þ for any tn1 \tn and depends only on the present state. If the probability of Xðt þ uÞ being in State j, given that XðuÞ is in State i, is independent of u  0, i.e., PrfXðt þ uÞ ¼ jjXðuÞ ¼ ig  Pij ðtÞ; then the process has a stationary or homogeneous transition probability that depends only on the time difference t. We restrict ourselves to continuous-time Markov chains with stationary transition probabilities and a finite state space f0; 1; 2; . . .; mg like discrete-time Markov chains. A continuous-time Markov chain is a stochastic process in which the process moves among states in accordance with a discrete-time Markov chain, however, its sojourn time spent in each state to visit the next state is independent and distributed exponentially. A matrix PðtÞ ¼ fPij ðtÞg is called a transition probability matrix at time t, and P satisfies Pij ðtÞ  0 and m j¼0 Pij ðtÞ ¼ 1 for any t  0 and i, j. In a similar way of obtaining (4.4), Pij ðt þ uÞ ¼ PrfXðt þ uÞ ¼ jjXð0Þ ¼ ig m X ¼ PrfXðt þ uÞ ¼ j; XðuÞ ¼ kjXð0Þ ¼ ig k¼0

¼ ¼

m X k¼0 m X

PrfXðt þ uÞ ¼ jjXðuÞ ¼ kgPrfXðuÞ ¼ kjXð0Þ ¼ ig Pik ðuÞPkj ðtÞ

ð4:25Þ

k¼0

for t, u  0, that is called the Chapman–Kolmogorov equation. In a matrix form, (4.25) is rewritten as Pðt þ uÞ ¼ PðtÞPðuÞ:

ð4:26Þ

112

4 Markov Chains

However, we cannot calculate the transition probability by the matrix operation as shown in Sect. 4.1.1.

4.2.2 Pure Birth Process and Birth and Death Process Recall that a Poisson process is a counting process in which the process has stationary and independent increments, and the interarrival times have an identical exponential distribution with mean 1=k. Expanding the Poisson process, the law of probability of arrival events depends on the total number of events that has already arrived, i.e., the interarrival times are independent and distributed exponentially with mean 1=kk that depends on the present State k. In this case, states are denoted by the total number of events. Definition 4.1 If fXðtÞg is a continuous-time Markov chain with (i) Xð0Þ ¼ 0, (ii) PrfXðt þ hÞ  XðtÞ ¼ 1jXðtÞ ¼ kg ¼ kk h þ oðhÞ, (iii) PrfXðt þ hÞ  XðtÞ  2jXðtÞ ¼ kg ¼ oðhÞ, then the process fXðtÞg is said to be a pure birth process with parameters kk and transits only from k to k þ 1. Using this definition, we derive the stationary transition probability Pk ðtÞ  PrfXðtÞ ¼ kjXð0Þ ¼ 0g ¼ PrfXðtÞ ¼ kg

ðk ¼ 0; 1; . . .; mÞ;

that represents that k events occur in the time interval ½0; t. By the similar method of obtaining (2.25), from Definition 4.1, P0 ðt þ hÞ ¼ PrfXðt þ hÞ ¼ 0g ¼ PrfXðtÞ ¼ 0gPrfXðt þ hÞ  XðtÞ ¼ 0jXðtÞ ¼ 0g ¼ P0 ðtÞ½1  k0 h þ oðhÞ: Hence, as h ! 0, dP0 ðtÞ ¼ k0 P0 ðtÞ: dt By integration and setting P0 ð0Þ ¼ 1 from (i), P0 ðtÞ ¼ ek0 t :

ð4:27Þ

4.2 Continuous-Time Markov Chain

113

Similarly, Pk ðt þ hÞ ¼ PrfXðt þ hÞ ¼ kg ¼ PrfXðtÞ ¼ k  1; Xðt þ hÞ  XðtÞ ¼ 1g þ PrfXðtÞ ¼ k; Xðt þ hÞ  XðtÞ ¼ 0g þ

k X

PrfXðtÞ ¼ k  j; Xðt þ hÞ  XðtÞ ¼ jg

j¼2

¼ Pk1 ðtÞ½kk1 h þ oðhÞ þ Pk ðtÞ½1  kk h þ oðhÞ þ oðhÞ: Hence, as h ! 0, dPk ðtÞ ¼ kk Pk ðtÞ þ kk1 Pk1 ðtÞ; dt i.e., ekk t ½P0k ðtÞ þ kk Pk ðtÞ ¼

d½ekk t Pk ðtÞ ¼ kk1 ekk t Pk1 ðtÞ: dt

By integration under Pk ð0Þ ¼ 0 for k  1, Pk ðtÞ ¼ kk1 ekk t

Zt

ekk u Pk1 ðuÞdu ðk ¼ 1; 2; . . .Þ:

ð4:28Þ

0

Therefore, from (4.27) and (4.28), we can compute Pk ðtÞ successively. Furthermore, interarrival times are independent and have an exponential distribution with mean 1=kk (k ¼ 0; 1; 2; . . .; m), as shown in Sect. 2.2. Example 4.10 (Parallel system) Consider a parallel redundant system with n identical units each of which has an independent exponential distribution ð1  ekt Þ. Then, setting that kk  ðn  kÞk (k ¼ 0; 1; . . .; n  1), the system forms a pure birth process. Solving (4.28) under P0 ðtÞ ¼ enkt ,   n ð1  ekt Þk eðnkÞkt ðk ¼ 0; 1; . . .; nÞ; Pk ðtÞ ¼ k that is easily derived by using a binomial distribution (Problem 4.4). In addition, it is shown that the mean time to system failure is given in (2.8). Next, consider the stochastic process that states increase or decrease according to arrivals or departures of events. Definition 4.2 If the process fXðtÞg is a continuous-time Markov chain with (i) Xð0Þ ¼ i, (ii) PrfXðt þ hÞ  XðtÞ ¼ 1jXðtÞ ¼ kg ¼ kk h þ oðhÞ,

114

4 Markov Chains

(iii) PrfXðt þ hÞ  XðtÞ ¼ 1jXðtÞ ¼ kg ¼ lk h þ oðhÞ, (iv) PrfjXðt þ hÞ  XðtÞj [ 1jXðtÞ ¼ kg ¼ oðhÞ, then the process fXðtÞg is said to be a birth and death process with parameters fkk ; lkþ1 g and transits only from k to k  1 or k þ 1, where kk and lkþ1 (k ¼ 0; 1; 2; . . .) are called the birth and death rates, respectively. It follows easily from (ii–iv) that PrfXðt þ hÞ  XðtÞ ¼ 0jXðtÞ ¼ 0g ¼ 1  k0 h þ oðhÞ; PrfXðt þ hÞ  XðtÞ ¼ 0jXðtÞ ¼ kg ¼ 1  ðkk þ lk Þh þ oðhÞ

ðk ¼ 1; 2; . . .Þ:

Using this definition, we derive the stationary transition probability Pij ðtÞ  PrfXðtÞ ¼ jjXð0Þ ¼ ig

ði; j ¼ 0; 1; . . .; mÞ:

From the Chapman–Kolmogorov equation in (4.25), Pij ðt þ hÞ ¼

m X

Pik ðtÞPkj ðhÞ:

k¼0

Thus, from (ii–iv), Pij ðt þ hÞ ¼ Pij1 ðtÞPrfXðt þ hÞ  XðtÞ ¼ 1jXðtÞ ¼ j  1g þ Pij ðtÞPrfXðt þ hÞ  XðtÞ ¼ 0jXðtÞ ¼ jg þ Pijþ1 ðtÞPrfXðt þ hÞ  XðtÞ ¼ 1jXðtÞ ¼ j þ 1g m X Pik ðtÞPrfXðt þ hÞ  XðtÞ ¼ j  kjXðtÞ ¼ kg þ k¼0;k6¼j1; j; jþ1

¼ Pij1 ðtÞ½kj1 h þ oðhÞ þ Pij ðtÞ½1  ðkj þ lj Þh þ oðhÞ þ Pijþ1 ðtÞ½ljþ1 h þ oðhÞ þ oðhÞ: Hence, as h ! 0, dPij ¼ kj1 Pij1 ðtÞ  ðkj þ lj ÞPij ðtÞ þ ljþ1 Pijþ1 ðtÞ dt

ðj ¼ 1; 2; . . .; m  1Þ: ð4:29Þ

In particular, for the respective cases of j ¼ 0 and j ¼ m, dPi0 ðtÞ ¼ k0 Pi0 ðtÞ þ l1 Pi1 ðtÞ; dt dPim ðtÞ ¼ km1 Pim1 ðtÞ  lm Pim ðtÞ: dt

ð4:30Þ

Therefore, from (4.29) and (4.30), we can compute Pij ðtÞ under the condition of Pii ð0Þ ¼ PrfXð0Þ ¼ ig ¼ 1;

Pij ð0Þ ¼ 0

for i 6¼ j:

4.2 Continuous-Time Markov Chain

115

2λ

Fig. 4.6 Transition diagram among three states

0

λ 1

θ

2 θ

Example 4.11 (two-unit parallel system) Consider a two-unit parallel system with one repair person, i.e., only one failed unit is repaired by the repair person. It is assumed that two units begin to operate at time 0 and fail according to an identical exponential distribution ð1  ekt Þ. Failed units are repaired by one repair person according to an exponential distribution ð1  eht Þ, become like new ones, and start to operate again immediately upon the repair completion. Then, we define the states: State 0: Two units are operating. State 1: One unit is operating and the other is under repair. State 2: One unit is under repair and the other waits for repair. Note that states represent the number of failed units. Setting that k0 ¼ 2k, k1 ¼ k, and l1 ¼ l2 ¼ h, the transition diagram among three states is shown in Fig. 4.6. Setting that P00 ð0Þ  1, (4.29) and (4.30) for the system become P000 ðtÞ ¼ 2kP00 ðtÞ þ hP01 ðtÞ; P001 ðtÞ ¼ 2kP00 ðtÞ  ðk þ hÞP01 ðtÞ þ hP02 ðtÞ; P002 ðtÞ ¼ kP01 ðtÞ  hP02 ðtÞ: From Appendix A, using the Laplace–Stieltjes (LS) transformation of Z1 est dP0ij ðtÞ ¼ s½Pij ðsÞ  Pij ð0Þ; 0

the above differential equations are sP00 ðsÞ  s ¼ 2kP00 ðsÞ þ hP01 ðsÞ; sP01 ðsÞ ¼ 2kP00 ðsÞ  ðk þ hÞP01 ðsÞ þ hP02 ðsÞ; sP02 ðsÞ ¼ kP01 ðsÞ  hP02 ðsÞ: Solving these equations, s2 þ ðk þ 2hÞs þ h2 ; s2 þ ð3k þ 2hÞs þ 2k2 þ 2kh þ h2 2ks þ 2kh ; P01 ðsÞ ¼ s2 þ ð3k þ 2hÞs þ 2k2 þ 2kh þ h2 P00 ðsÞ ¼

116

4 Markov Chains

P02 ðsÞ ¼

2k2 ; s2 þ ð3k þ 2hÞs þ 2k2 þ 2kh þ h2

where note that P00 ðsÞ þ P01 ðsÞ þ P02 ðsÞ ¼ 1. Expanding the right-hand side of P00 ðsÞ in partial fractions, P00 ðsÞ ¼ C þ

As Bs þ ; s þ w1 s þ w2

where h2 ; 2k2 þ 2kh þ h2 ðs þ w1 Þðs þ w2 Þ ¼ s2 þ ð3k þ 2hÞs þ 2k2 þ 2kh þ h2 ; C

i.e., w1 

pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 3k þ 2h þ k2 þ 4kh ; 2

w2 

pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 3k þ 2h  k2 þ 4kh : 2

From the following identity equation Cðs þ w1 Þðs þ w2 Þ þ Asðs þ w2 Þ þ Bsðs þ w1 Þ ¼ s2 þ ðk þ 2hÞs þ h2 ; A

2kðw1  k  hÞ ; w1 ðw1  w2 Þ

B

2kðw2  k  hÞ : w2 ðw2  w1 Þ

Thus, using the inverse LS transform of Z1

est dewt ¼

s ; sþw

0

the transition probability is   h2 2k w1  k  h w1 t w2  k  h w2 t þ e  e : P00 ðtÞ ¼ 2 w1 w2 2k þ 2kh þ h2 w1  w2 Similarly,   2kh 2k w1 þ h w1 t w2 þ h w2 t þ e  e ; w1 w2 2k2 þ 2kh þ h2 w1  w2   2k2 2k2 1 w1 t 1 w2 t þ e  e ; P02 ðtÞ ¼ 2 w2 2k þ 2kh þ h2 w1  w2 w1 P01 ðtÞ ¼

where note that P00 ðtÞ þ P01 ðtÞ þ P02 ðtÞ ¼ 1. Thus, the limiting probabilities Pj  limt!1 Pij ðtÞ are

4.2 Continuous-Time Markov Chain

P0 

117

h2 ; 2k2 þ 2kh þ h2

P1 

2kh ; 2k þ 2kh þ h2 2

and P2 ¼ 1  P0  P1 that represents the unavailability for a two-unit parallel system with one repair person. Next, suppose that there exists two repair persons. In this case, any units are repaired immediately upon failures. Then, setting k0 ¼ 2k, k1 ¼ k, l1 ¼ h, and l2 ¼ 2h, P000 ðtÞ ¼ 2kP00 ðtÞ þ hP01 ðtÞ; P001 ðtÞ ¼ 2kP00 ðtÞ  ðk þ hÞP01 ðtÞ þ 2hP02 ðtÞ; P002 ðtÞ ¼ kP01 ðtÞ  2hP02 ðtÞ: By the same method of the previous model, s2 þ ðk þ 3hÞs þ 2h2 ; ðs þ 2k þ 2hÞðs þ k þ hÞ 2kðs þ 2hÞ ; P01 ðsÞ ¼ ðs þ 2k þ 2hÞðs þ k þ hÞ

P00 ðsÞ ¼

P02 ðsÞ ¼

2k2 : ðs þ 2k þ 2hÞðs þ k þ hÞ

Forming the inverse LS transforms of P0j ðsÞ,  2 h þ keðkþhÞt ; kþh i 1 h i 2k h P01 ðtÞ ¼ 1  eðkþhÞt h þ keðkþhÞt ; kþh kþh  i 2 k h ðkþhÞt 1e P02 ðtÞ ¼ kþh P00 ðtÞ ¼

(Problem 4.5). Example 4.12 We compute explicitly Pj ðtÞ (j ¼ 0; 1; 2) given in (4.2). Forming the LS transforms of (4.2), sP0 ðsÞ  s ¼ kP0 ðsÞ þ h1 P1 ðsÞ þ h2 P2 ðsÞ; sP1 ðsÞ ¼ h1 P1 ðsÞ þ akP0 ðsÞ; sP2 ðsÞ ¼ h2 P2 ðsÞ þ ð1  aÞkP0 ðsÞ:

118

4 Markov Chains

Solving these equations, ðs þ h1 Þðs þ h2 Þ ; þ sðk þ h1 þ h2 Þ þ ð1  aÞkh1 þ akh2 þ h1 h2 ak P1 ðsÞ ¼ P ðsÞ; s þ h1 0 ð1  aÞk  P2 ðsÞ ¼ P ðsÞ: s þ h2 0 P0 ðsÞ ¼

s2

Forming the inverse LS transforms of Pj ðsÞ, h1 h2 ðw1  h1 Þðw1  h2 Þ w1 t e þ ð1  aÞkh1 þ akh2 þ h1 h2 w1 ðw1  w2 Þ ðw2  h1 Þðw2  h2 Þ w2 t e þ ; w2 ðw2  w1 Þ akh2 akðw1  h2 Þ w1 t P1 ðtÞ ¼ e  ð1  aÞkh1 þ akh2 þ h1 h2 w1 ðw1  w2 Þ akðw2  h2 Þ w2 t e  ; w2 ðw2  w1 Þ ð1  aÞkh1 ð1  aÞkðw1  h1 Þ w1 t P2 ðtÞ ¼ e  ð1  aÞkh1 þ akh2 þ h1 h2 w1 ðw1  w2 Þ ð1  aÞkðw2  h1 Þ w2 t e  ; w2 ðw2  w1 Þ P0 ðtÞ ¼

where

 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 w1  k þ h1 þ h2 þ ðk þ h1 þ h2 Þ2  4½ð1  aÞkh1 þ akh2 þ h1 h2  ; 2  qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 k þ h1 þ h2  ðk þ h1 þ h2 Þ2  4½ð1  aÞkh1 þ akh2 þ h1 h2  : w2  2

From (4.29) and (4.30), the transition probability matrix satisfies the differential equation P0 ðtÞ ¼ PðtÞA; where the initial condition of Pð0Þ ¼ I, where 0 k0 k0 0 0 B l1 ðk1 þ l1 Þ k 0 1 B B 0 ðk þ l Þ k l 2 2 2 2 B AB . .. .. .. B .. . . . B @ 0 0 0 0 0 0 0 0

ð4:31Þ

I is an identity matrix and ... ... ...

0 0 0 .. .

0 0 0 .. .

1

C C C C C: C ... C . . . ðkm1 þ lm1 Þ km1 A ... lm lm

4.2 Continuous-Time Markov Chain

119

Solving (4.31), PðtÞ ¼ eAt ¼ I þ

1 X An t n

n!

n¼1

:

ð4:32Þ

In general, we can compute the transition probabilities from (4.32), however, it is difficult to do so except in simple cases.

4.2.3 Limiting Probabilities If all states communicate with each other, i.e., the process consists of one recurrent class, then there exists the limiting probabilities Pj  lim Pij ðtÞ [ 0

ðj ¼ 0; 1; 2; . . .; mÞ;

t!1

Pm

Pj ¼ 1, as shown in Sect. 4.1.3.

lim P0ij ðtÞ ¼ 0:

ð4:33Þ

that are independent of the initial State i and Furthermore,

j¼0

t!1

It can be clearly seen that there exits limt!1 P0ij ðtÞ from (4.29) because the limiting probabilities exists. Thus, if P0ij ðtÞ 6¼ 0, then there exists T [ 0 such that P0ij ðtÞ  c;

or

P0ij ðtÞ   c

for t  T and c [ 0. If P0ij ðtÞ  c, then ZT0

P0ij ðtÞdt  cðT0  TÞ

T

for T0 [ T, i.e., Pij ðT0 Þ  Pij ðTÞ  cðT0  TÞ; that leads to the mathematical contradiction because the right-hand side goes to infinity as T0 ! 1. Therefore, (4.33) is proved. It is also proved in the case of P0ij ðtÞ   c. Using the above results, from (4.29) and (4.30), the limiting probabilities satisfy the simultaneous equations, k0 P0 þ l1 P1 ¼ 0; kj1 Pj1  ðkj þ lj ÞPj þ ljþ1 Pjþ1 ¼ 0 km1 Pm1  lm Pm ¼ 0:

ðj ¼ 1; 2; . . .; m  1Þ;

ð4:34Þ

120

4 Markov Chains

Summing for i ¼ 0; 1; 2; . . .; j  1 in (4.34), kj1 kj1 kj2 Pj1 ¼ Pj2 ¼    ¼ Pj ¼ lj lj lj1 Recalling that

Pm

j¼0

j Y kk1 k¼1

lk

! P0 :

Pj ¼ 1, Qj

Pj ¼ where

Q0

k¼1

ðkk1 =lk Þ Pmk¼1Qj 1 þ j¼1 k¼1 ðkk1 =lk Þ

ðj ¼ 0; 1; . . .; mÞ;

ð4:35Þ

 1.

Example 4.13 (n-unit parallel system) Consider an n-unit parallel system with one repair person. Then, setting that kk ¼ ðn  kÞk and lkþ1 ¼ h (k ¼ 0; 1; . . .; n) in (4.35), the limiting probabilities are ðk=hÞj =ðn  jÞ! Pj ¼ Pn k k¼0 ðk=hÞ =ðn  kÞ!

ðj ¼ 0; 1; 2; . . .; nÞ:

In particular, when n ¼ 2, Pj agree with the limiting probabilities Pj of a two-unit parallel system with one repair person in Example 4.11.

4.3 Problems 4 4.1 Prove in Example 4.3 that by the mathematical induction,     a a b a 1 ð1  a  bÞn Pn ¼ ; þ aþb aþb b a b b   b þ að1  a  bÞn a½1  ð1  a  bÞn  ; : pðnÞ ¼ aþb aþb When a þ b ¼ 1, show that for n  1,   1a a n ; pðnÞ ¼ ð1  a; aÞ: P ¼ 1a a 4.2 The unit begins to operate at time 0, and when it fails, it is replaced with a new one [4, p. 113]. In addition, if the unit has not failed until Nth period, it is also replaced with a new one. It is assumed that the unit has a failure probability pn at time n. Then, the probability that the unit with age n fails at time n is pn rn  P1

k¼n

pk

;

4.3 Problems 4

121

that is called a discrete failure rate [5, p. 18, 6, p. 9]. Show that the transition probabilities are given by Pi1 ¼ 1  Piiþ1 ¼ ri

ði ¼ 1; 2; . . .; N  1Þ;

PN1 ¼ 1;

and derive the limiting probabilities pj (j ¼ 1; 2; . . .; N). 4.3 In Example 4.9, compute P and ðI  PÞ1 by striking out row 5 and column 5 when kt0 ¼ 0:9 and kL ¼ 0:1. 4.4 In Example 4.10, prove that by using the binomial distribution with n and p  1  ekt ,   n ð1  ekt Þk eðnkÞkt : Pk ðtÞ ¼ k 4.5 Derive the transition probabilities of two-unit standby system with one and two repair persons, respectively.

References 1. 2. 3. 4. 5. 6.

Osaki S (1992) Applied stochastic system modeling. Springer, Berlin Kemeny JG, Snell JL (1960) Finite Markov chains. Nostrand Co, Princeton Tijms HC (2003) A first course in stochastic models. Wiley, Chichester Ross SM (1983) Stochastic processes. Wiley, New York Barlow RE, Proschan F (1965) Mathematical theory of reliability. Wiley, New York Nakagawa T (2005) Maintenance theory of reliability. Springer, London

Chapter 5

Semi-Markov and Markov Renewal Processes

We consider the same one-unit system in Chap. 4. If FðtÞ and Gj ðtÞ (j ¼ 1; 2) are not exponential, the process does not have the Markov property at any time t and has its property at only epochal times of transition to states. Let ZðtÞ denote the state of the process at time t; Nj ðtÞ (j ¼ 0; 1; 2) denote the number of times that the process visits State j in ½0; t when the system begins to operate at time 0, and define that Pj ðtÞ  PrfZðtÞ ¼ jg;

Fj ðtÞ  PrfNj ðtÞ [ 0g;

where Fj ðtÞ is called a first-passage distribution to State j: Then, using the techniques of forming renewal equations in Sect. 3.3, F0 ðtÞ ¼ aFðtÞ  G1 ðtÞ þ ð1  aÞFðtÞ  G2 ðtÞ; F1 ðtÞ ¼ aFðtÞ þ ð1  aÞFðtÞ  G2 ðtÞ  F1 ðtÞ; F2 ðtÞ ¼ ð1  aÞFðtÞ þ aFðtÞ  G1 ðtÞ  F2 ðtÞ;

ð5:1Þ

where the asterisk denotes the pairwise Stieltjes convolution, i.e., aðtÞ  bðtÞ  Rt 0 bðt  uÞdaðuÞ: Furthermore, using Fj ðtÞ; the transition probabilities are P0 ðtÞ ¼ 1  FðtÞ þ F0 ðtÞ  P0 ðtÞ; P1 ðtÞ ¼ F1 ðtÞ  ½1  G1 ðtÞ þ G1 ðtÞ  P1 ðtÞ;

ð5:2Þ

P2 ðtÞ ¼ F2 ðtÞ  ½1  G2 ðtÞ þ G2 ðtÞ  P2 ðtÞ: Thus, using Fj ðtÞ in (5.1), we can obtain the transition probabilities Pj ðtÞ: The process fZðtÞg is called a semi-Markov process because it has the Markov property only at regeneration points at which the state transitions take place. We investigate mainly the properties of the transition probabilities Pj ðtÞ of a semi-Markov process in Sect. 5.2 as an example of a standby redundant system.

T. Nakagawa, Stochastic Processes, Springer Series in Reliability Engineering, DOI: 10.1007/978-0-85729-274-2_5, Ó Springer-Verlag London Limited 2011

123

124

5

Semi-Markov and Markov Renewal Processes

Finally, define that renewal functions Mj ðtÞ  EfNj ðtÞg (j ¼ 0; 1; 2) that represent the expected number of visits to State j in ½0; t: In a similar way of obtaining (5.2), M0 ðtÞ ¼ F0 ðtÞ  ½1 þ M0 ðtÞ; M1 ðtÞ ¼ F1 ðtÞ  ½1 þ G1 ðtÞ  M1 ðtÞ; M2 ðtÞ ¼ F2 ðtÞ  ½1 þ G2 ðtÞ  M2 ðtÞ:

ð5:3Þ

Thus, using Fj ðtÞ in (5.1), we can obtain Mj ðtÞ from (5.3) (Problem 5.1). The process fNj ðtÞg is called a Markov renewal process that combines a renewal process in Chap. 3 and a Markov chain in Chap. 4. We investigate mainly the properties of Mj ðtÞ for a Markov renewal process in Sect. 5.3. However, there is no clear difference between semi-Markov and Markov renewal processes. Markov chain, Markov process, Semi-Markov process, and Markov renewal process are called Markov process in one word, i.e., Markov processes are a general term of the stochastic process with the Markov property. State space is usually defined by the number of units that are working satisfactorily. As far as the applications to reliability theory is concerned, we consider only a finite number of states, contrast with a queueing theory. We mention only the theory of stationary Markov processes with a finite-state space. It is shown that transition probabilities, first-passage distributions, and renewal functions are given by forming renewal equations. Furthermore, some limiting properties are summarized when all states communicate.

5.1 Markov Process Consider a stochastic process with a finite state space f0; 1; 2; . . .; mg that makes transitions from state to state in accordance with a Markov chain with stationary transition probabilities. However, in the process, the amount of time spent in each state until the next transition is not always constant and exponential, but has a random variable shown in Definition 5.1. Consider a stochastic process fXðnÞ; Sn ; n ¼ 0; 1; 2; . . .g where Sn denotes the random variable of the nth arrival time at which the process just moves from one state to another, and XðnÞ denotes the state of the process at time Sn : Definition 5.1 The stochastic process fXðnÞ; Sn g is called a Markov renewal process [1, p. 167, 2, p. 313] if PrfXðn þ 1Þ ¼ j; Snþ1  Sn  tjXð0Þ; Xð1Þ; . . .; XðnÞ ¼ i; S0 ; S1 ; . . .; Sn g ¼ PrfXðn þ 1Þ ¼ j; Snþ1  Sn  tjXðnÞ ¼ i; Sn g  Qij ðtÞ for all i; j ¼ 0; 1; 2; . . .; m and t  0:

ð5:4Þ

5.1 Markov Process

125

The probability Qij ðtÞ denotes that after making a transition into State i; the next process makes a transition into State P j; in an amount of time less than or equal to t: Clearly, we have Qij ðtÞ  0 and m j¼0 Qij ð1Þ ¼ 1; where Qij ð1Þ represents the probability that the next process makes a transition into State j; given that the process goes into State i; i.e., Qij ð1Þ forms a discrete-time Markov chain in Sect. 4.1. We call the probability Qij ðtÞ a mass function or a one-step transition probability [1, p. 167]. Letting Gij ðtÞ ¼

Qij ðtÞ Qij ð1Þ

for Qij ð1Þ [ 0;

then Gij ðtÞ represents the conditional probability that the process makes a transition in an amount of time less than or equal to t; given that the process goes from State i to State j at the next transition. In other words, Gij ðtÞ ¼ PrfSnþ1  Sn  tjXðnÞ ¼ i; Xðn þ 1Þ ¼ jg is the distribution of the sojourn time that the process takes in State i given that the next visiting state is j: Because XðnÞ denotes the state of the process immediately after the nth transition has occurred for n  1; the stochastic process fXðnÞ; n ¼ 0; 1; 2; . . .g is called an embedded Markov chain with transition probabilities Qij ð1Þ: If the process makes a transition from one state to another with one unit of time, i.e., Gij ðtÞ ¼ 0 for t\1; and 1 for t  1; then an embedded Markov chain becomes a discrete-time Markov chain in Sect. 4.1. Furthermore, if an amount of time spent in State i depends only on State i and is distributed exponentially independent of the next state; Gij ðtÞ ¼ 1  eki t for constant ki [ 0; the process is said to be a continuous-time Markov chain in Sect. 4.2. In addition, the process becomes a renewal process in Chap. 3 if it has only one state and becomes an alternating renewal process in Sect. 3.4 if it has two states. If we let ZðtÞ denote the state of the process at time t; then the stochastic process fZðtÞ; t  0g is called a semi-Markov process. Let Nj ðtÞ denote the number of times that the process visits State j in ½0; t: It follows from the renewal theory that with probability 1, Nj ðtÞ\1 for t  0: The stochastic process fN0 ðtÞ; N1 ðtÞ; N2 ðtÞ; . . .; Nm ðtÞg is called a Markov renewal process, and EfNj ðtÞg are called Markov renewal functions. The event fNj ðtÞ ¼ kg represents the number of k visits to State j in ½0; t: An embedded Markov chain records the state of the process at each transition point, a semi-Markov process records the state of the process at each time point t; and a Markov renewal process records the total number of times that each state has been visited in ½0; t: Letting Hi ðtÞ denote the distribution of an amount of time spent in State i until the process makes a transition to the next state, Hi ðtÞ 

m X

Qij ðtÞ;

j¼0

that is called the unconditional distribution for State i: We suppose that Hi ð0Þ\1 for all i: Denoting

126

5

gi 

Z1

Semi-Markov and Markov Renewal Processes

tdHi ðtÞ and

lij 

0

Z1

tdGij ðtÞ;

0

it is easily seen that gi ¼

m X

Qij ð1Þlij ;

j¼0

which represents the mean time spent in State i: We denote transition probabilities, first-passage distributions, and Markov renewal functions as, respectively, Pij ðtÞ  PrfZðtÞ ¼ jjZð0Þ ¼ ig; Fij ðtÞ  PrfNj ðtÞ [ 0jZð0Þ ¼ ig; Mij ðtÞ  EfNj ðtÞjZð0Þ ¼ ig: We have the following renewal equations that show the relationships for Pij ðtÞ; Fij ðtÞ and Mij ðtÞ in terms of the mass functions Qij ðtÞ: Pii ðtÞ ¼ 1  Hi ðtÞ þ

m Z X k¼0

Pij ðtÞ ¼

t m Z X k¼0

t

Pki ðt  uÞdQik ðuÞ;

0

ð5:5Þ

Pkj ðt  uÞdQik ðuÞ for i 6¼ j;

0

Zt m X

Fij ðtÞ ¼ Qij ðtÞ þ

k¼0;k6¼j

Mij ðtÞ ¼ Qij ðtÞ þ

m Z X k¼0

Fkj ðt  uÞdQik ðuÞ;

ð5:6Þ

0 t

Mkj ðt  uÞdQik ðuÞ:

ð5:7Þ

0

Therefore, the mass functions Qij ðtÞ determine Pij ðtÞ; Fij ðtÞ and Mij ðtÞ uniquely. Furthermore, Pii ðtÞ ¼ 1  Hi ðtÞ þ

Zt

Pii ðt  uÞdFii ðuÞ;

0

Pij ðtÞ ¼

Zt 0

Pjj ðt  uÞdFij ðuÞ

ð5:8Þ for i 6¼ j;

5.1 Markov Process

127

Mij ðtÞ ¼ Fij ðtÞ þ

Zt

Mjj ðt  uÞdFij ðuÞ:

ð5:9Þ

0

Taking the LS transforms of the above equations, Pii ðsÞ ¼

1  Hi ðsÞ ; 1  Fii ðsÞ

Pij ðsÞ ¼ Fij ðsÞPjj ðsÞ

for i 6¼ j;

Mij ðsÞ ¼ Fij ðsÞ½1 þ Mjj ðsÞ;

ð5:10Þ ð5:11Þ

where the asterisk denotes the LS transform of the function with itself. Thus, if the first-passage distributions Fij ðtÞ are obtained, then the transition probabilities Pij ðtÞ and renewal functions Mij ðtÞ are given by (5.10) and (5.11), respectively. Consider the process in which all states communicate, i.e., Gii ð1Þ ¼ 1; lii \1 for all i; and each Gii ðtÞ is a nonlattice distribution. It is said that the process consists of one positive recurrent class. Then, Gij ð1Þ ¼ 1;

lij \1; n X Qik ð1Þlkj : lij ¼ gi þ k¼0;k6¼j

Furthermore, lim Mij ðtÞ ¼

Z1

t!1

Pij ðtÞdt;

0

Mij ðtÞ 1 ¼ \1; t!1 t ljj lim

lim Pij ðtÞ ¼

t!1

gj \1: ljj

In this case, there exist limt!1 Mij ðtÞ=t and limt!1 Pij ðtÞ: From a Tauberian theorem in Appendix A.3 that if for some nonnegative integer n; lims!0 sn U ðsÞ ¼ C; then limt!1 UðtÞ=tn ¼ C=n!; Mij ðtÞ ¼ lim sMij ðsÞ; s!0 t Zt 1 Pij ðuÞdu ¼ lim Pij ðsÞ: lim Pij ðtÞ ¼ lim t!1 t!1 t s!0

lim

t!1

0

Example 5.1 (Alternating renewal process) An alternating renewal process in Sect. 3.4.1 has the mass functions Q01 ðtÞ ¼ FðtÞ and Q10 ðtÞ ¼ GðtÞ: Then, from (5.5),

128

5

P00 ðtÞ ¼ 1  Q01 ðtÞ þ

Zt

Semi-Markov and Markov Renewal Processes

P10 ðt  uÞdQ01 ðuÞ

0

¼ FðtÞ þ

Zt

P10 ðt  uÞdFðuÞ;

0

P11 ðtÞ ¼ 1  Q10 ðtÞ þ

Zt

P01 ðt  uÞdQ10 ðuÞ

0

¼ GðtÞ þ

Zt

P01 ðt  uÞdGðuÞ;

0

P01 ðtÞ ¼

P10 ðtÞ ¼

Zt

P11 ðt  uÞdQ01 ðuÞ ¼

Zt

0

0

Zt

Zt

P00 ðt  uÞdQ10 ðuÞ ¼

0

P11 ðt  uÞdFðuÞ;

P00 ðt  uÞdGðuÞ;

0

that agree with (3.54). Furthermore, from (5.7), M00 ðtÞ ¼

M11 ðtÞ ¼

Zt

M10 ðt  uÞdQ01 ðuÞ ¼

Zt

0

0

Zt

Zt

M01 ðt  uÞdQ10 ðuÞ ¼

0

M01 ðtÞ ¼ Q01 ðtÞ þ

M10 ðtÞ ¼ Q10 ðtÞ þ

M10 ðt  uÞdFðuÞ;

M01 ðt  uÞdGðuÞ;

0

Zt

M11 ðt  uÞdQ01 ðuÞ ¼ FðtÞ þ

Zt

0

0

Zt

Zt

0

M00 ðt  uÞdQ10 ðuÞ ¼ GðtÞ þ

M11 ðt  uÞdFðuÞ;

M00 ðt  uÞdGðuÞ;

0

that agrees with (3.52). Example 5.2 (Intermittent fault) [3, p. 113] Faults occur in a communication system according to an exponential distribution ð1  ekt Þ and are hidden. When the duration time X in hidden fault exceeds an upper limit time Y; faults become permanent failures eternally, and otherwise, they get out of a hidden state. That is, if the event fX  Yg occurs, then hidden faults disappear, and conversely, if the event fX [ Yg occurs, then they become permanent failure. It is assumed that both

5.1 Markov Process

129

random variables X and Y are independent and have exponential distributions PrfX  tg ¼ 1  elt and PrfY  tg ¼ 1  eht ; respectively. We define the following states of intermittent faults: State 0: No fault occurs and the system is in a normal condition. State 1: Hidden fault occurs. State 2: Permanent failure occurs. We have the following mass functions Qij ðtÞ (i ¼ 0; 1; j ¼ 0; 1; 2) from State i to State j in time t from Fig. 5.1: Q01 ðtÞ ¼ 1  ekt ; Q10 ðtÞ ¼ PrfX  Y; X  tg ¼

Zt

ehu l elu du ¼

i l h 1  eðlþhÞt ; lþh

0

Q12 ðtÞ ¼ PrfX [ Y; Y  tg ¼

Zt

elu h ehu du ¼

ð5:12Þ

i h h 1  eðlþhÞt : lþh

0

Let Pij ðtÞ denote the transition probabilities from State i at time 0 to State j at time t: Then, we have the following renewal equations: P00 ðtÞ ¼ 1  Q01 ðtÞ þ Q01 ðtÞ  P10 ðtÞ; P01 ðtÞ ¼ Q01 ðtÞ  P11 ðtÞ; P10 ðtÞ ¼ Q10 ðtÞ  P00 ðtÞ; P11 ðtÞ ¼ 1  Q10 ðtÞ  Q12 ðtÞ þ Q10 ðtÞ  P01 ðtÞ;

ð5:13Þ

P02 ðtÞ ¼ Q01 ðtÞ  P12 ðtÞ; P12 ðtÞ ¼ Q12 ðtÞ þ Q10 ðtÞ  P02 ðtÞ: Taking the LS transforms of (5.13) and rearranging them, 1  Q01 ðsÞ ; 1  Q01 ðsÞQ10 ðsÞ   Q01 ðsÞ 1  Q10 ðsÞ  Q12 ðsÞ  P01 ðsÞ ¼ ; 1  Q01 ðsÞQ10 ðsÞ   Q ðsÞ 1  Q01 ðsÞ ; P10 ðsÞ ¼ 10  1  Q01 ðsÞQ10 ðsÞ 1  Q10 ðsÞ  Q12 ðsÞ : P11 ðsÞ ¼ 1  Q01 ðsÞQ10 ðsÞ P00 ðsÞ ¼

Fig. 5.1 Transition diagram of intermittent fault

ð5:14Þ

130

5

Semi-Markov and Markov Renewal Processes

Thus, substituting (5.12) for (5.14) and taking the inverse LS transforms, the transition probabilities from State i to State j (i; j ¼ 0; 1; 2) are 1 ½ðl þ h  x2 Þex2 t  ðl þ h  x1 Þex1 t ; x1  x2 k l ðex2 t  ex1 t Þ; P10 ðtÞ ¼ ðex2 t  ex1 t Þ; P01 ðtÞ ¼ x1  x2 x 1  x2 1 P11 ðtÞ ¼ ½ðk  x2 Þex2 t  ðk  x1 Þex1 t ; x1  x2 P02 ðtÞ ¼ 1  P00 ðtÞ  P01 ðtÞ; P12 ðtÞ ¼ 1  P10 ðtÞ  P11 ðtÞ; P00 ðtÞ ¼

ð5:15Þ

where  qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 x1  k þ l þ h þ ðk þ l þ hÞ2  4kh ; 2  qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 x2  k þ l þ h  ðk þ l þ hÞ2  4kh : 2 Next, let Mij ðtÞ (i; j ¼ 0; 1) be the expected number of visits to State j from State i before State 2 in time t: Then, we have the following renewal equations: M00 ðtÞ ¼ Q01 ðtÞ  M10 ðtÞ; M11 ðtÞ ¼ Q10 ðtÞ  M01 ðtÞ;

M01 ðtÞ ¼ Q01 ðtÞ  ½1 þ M11 ðtÞ; M10 ðtÞ ¼ Q10 ðtÞ  ½1 þ M00 ðtÞ:

ð5:16Þ

Taking the LS transforms of (5.16) and rearranging them,  M00 ðsÞ ¼

Q01 ðsÞQ10 ðsÞ ; 1  Q01 ðsÞQ10 ðsÞ

 ðsÞ ¼ M01

Q01 ðsÞ : 1  Q01 ðsÞQ10 ðsÞ

ð5:17Þ

Thus, substituting (5.12) for (5.17) and taking the inverse LS transforms,   kl 1 1 x2 t x1 t ð1  e Þ  ð1  e Þ ; M00 ðtÞ ¼ x 1  x 2 x2 x1   ð5:18Þ k x1  l  h x2  l  h x1 t x2 t ð1  e Þ ð1  e Þ : M01 ðtÞ ¼ x1  x2 x1 x2 Example 5.3 (ARQ model) ARQ (Automatic-Repeat-Request) models with intermittent faults that are mainly employed in data transmission to achieve high reliability of communication [3, p. 110]: ARQ strategies are adopted when data transmission due to faults, fail. However, the data throughput decreases significantly if retransmissions are repeated without limitation. To keep the level of data throughput, when all numbers N of transmissions have failed, the system is inspected and maintained. We repeat the above procedure until the data transmission succeeds.

5.1 Markov Process

131

It is assumed in Example 5.2 that when the system is in State 0, the data transmission succeeds with probability 1, when the system is in State 1, it succeeds with probability q and fails with probability p  1  q; and when the system is in State 2, it fails with probability 1. When the data transmission fails, the retransmission incurs in time T1 : When N retransmissions have failed consecutively including the first one, the system is inspected and preventively maintained, and the same data transmission begins new after time T2 : Suppose that the system is in State 0 at time 0. Then, the probability that the data transmission succeeds until the Nth number is PðNÞ ¼ 1  pP01 ðT1 Þ  P02 ðT1 Þ þ pP01 ðT1 Þ½1  pP11 ðT1 Þ  P12 ðT1 Þ

N 2 X

½ pP11 ðT1 Þj ;

j¼0

and the probability that all N retransmissions have failed is ( ) N2 X j N1 ; PðNÞ ¼ P02 ðT1 Þ þ pP01 ðT1 Þ P12 ðT1 Þ ½ pP11 ðT1 Þ þ½ pP11 ðT1 Þ j¼0

where PðNÞ þ PðNÞ ¼ 1: We call the time from the beginning of transmission to N failures or success as one period. Then, the mean time of one period is ( l1 ¼ T1 1  pP01 ðT1 Þ  P02 ðT1 Þ þ pP01 ðT1 Þ½1  pP11 ðT1 Þ  P12 ðT1 Þ  (

N2 X

) j

ðj þ 2Þ½ pP11 ðT1 Þ þNPðNÞ

j¼0

  pP01 ðT1 ÞP12 ðT1 Þ ¼ T1 1 þ ðN  1Þ P02 ðT1 Þ þ 1  pP11 ðT1 Þ

) 1  pP11 ðT1 Þ  P12 ðT1 Þ 1  ½ pP11 ðT1 ÞN1 þ pP01 ðT1 Þ : 1  pP11 ðT1 Þ 1  pP11 ðT1 Þ

Thus, the mean time in which the data transmission succeeds is l2 ðNÞ ¼ l1 þ PðNÞ½T2 þ l2 ðNÞ; i.e., l1 þ T2 PðNÞ PðNÞ T1 A þ ðN  1ÞB n o  T2 þ ¼ 1  pP11 ðT1 Þ C þ D 1  ½ pP11 ðT1 ÞN1

l2 ðNÞ ¼

ðN ¼ 1; 2; . . .Þ; ð5:19Þ

132

5

Semi-Markov and Markov Renewal Processes

where T1 ½P00 ðT1 Þ þ qP01 ðT1 Þ ; A  T1 þ T 2  1  pP11 ðT1 Þ   pP01 ðT1 ÞP12 ðT1 Þ ; B  T1 P02 ðT1 Þ þ 1  pP11 ðT1 Þ C  P00 ðT1 Þ þ qP01 ðT1 Þ;

D  pP01 ðT1 Þ

P10 ðT1 Þ þ qP11 ðT1 Þ : 1  pP11 ðT1 Þ

From the inequality l2 ðN þ 1Þ  l2 ðNÞ  0; 1 þ C=D ½1  pP11 ðT1 Þ½ pP11 ðT1 Þ

N1

 ðN  1Þ 

A 1 þ B 1  pP11 ðT1 Þ

ðN ¼ 1; 2; . . .Þ: ð5:20Þ

Letting LðNÞ be the left-hand side of (5.20), Lð1Þ  lim LðNÞ ¼ 1; N!1

LðN þ 1Þ  LðNÞ ¼

Lð1Þ ¼

1 þ C=D ; 1  pP11 ðT1 Þ

1 þ C=D  ½pP11 ðT1 ÞN [ 0: ½P11 ðT1 ÞN

Therefore, an optimum N  (1  N  \1) that minimizes l2 ðNÞ in (5.19) is given by a finite and unique minimum that satisfies (5.20). If C=D  ðA=BÞ½1  pP11 ðT1 Þ; then N  ¼ 1 (Problem 5.2).

5.2 Embedded Markov Chain Consider a standby redundant system with n þ 1 repair persons and one operating unit supported by n identical spare units. Each unit fails according to a general distribution FðtÞ with finite mean l and undergoes repair immediately. When failed units are repaired, they become as good as new units and rejoin spare ones. It is assumed that the repair time of each failed unit is an independent random variable with an identical exponential distribution ð1  eht Þ with finite mean 1=h: Let denote by ZðtÞ the number of units undergoing repair at time t: We say that the system is in State j at time t if ZðtÞ ¼ j (j ¼ 0; 1; 2; . . .; n þ 1). We also say that system failure occurs when the system is in Sate n þ 1: Denote by s0 ; s1 ; . . .; sk ; . . . the failure times of operating units. For simplicity of computations, we restrict ourselves to the case of s0  0: If we denote Zk  Zðsk  0Þ for k ¼ 1; 2; . . .; then Zk represents the number of units undergoing repair immediately before the kth failure occurrence, where Z0  0: It is noted that the process fZk ; k ¼ 0; 1; 2; . . .g

5.2 Embedded Markov Chain

133

forms an embedded Markov chain and fZðtÞ; t  0g forms a semi-Markov process with stationary transition probabilities and a finite state space.

5.2.1 Transition Probabilities We derive the transition probabilities by using binomial moments [4] and the expected number of failures before system failure: Let denote the transition probabilities as ðkÞ

Pij ðtÞ ¼ PrfZk ¼ j; sk  tjZ0 ¼ ig

ði; j ¼ 0; 1; 2; . . .; nÞ

for k ¼ 1; 2; . . .: By similar arguments of obtaining the Chapman–Kolmogorov equations of (4.4) and (4.25), ðkÞ Pij ðtÞ

¼

n Z X l¼0

t ð1Þ

ðk1Þ

Plj ðt  uÞdPil

ðuÞ:

ð5:21Þ

0

Furthermore, because Z1 has a binomial distribution with parameters i þ 1 and ehu under the given conditions, PrfZ1 ¼ jjs1  s0 ¼ u; Z0 ¼ ig  8 < i þ 1 ð1  ehu Þiþ1j ejhu ¼ j : 0

ði ¼ 0; 1; . . .; n  1; j  i þ 1Þ; ðj [ i þ 1Þ:

Dropping the condition s1  s0 ¼ u; i.e., the operating unit fails in ðu; u þ du; 8 Z t  iþ1 < ð1  ehu Þiþ1j ejhu dFðuÞ ði ¼ 0; 1; . . .; n  1; j  i þ 1Þ; ð1Þ Pij ðtÞ ¼ j 0 : 0 ðj [ i þ 1Þ: ð5:22Þ If Z0 ¼ n; then ð1Þ Pnj ðtÞ

¼

Zt

ð1Þ

ðn þ 1Þh eðnþ1Þhu Pn1j ðt  uÞdu:

ð5:23Þ

0 ðkÞ

Next, introduce the rth binomial moment of the LS transform of Pij ðtÞ as follows:

134

5

Semi-Markov and Markov Renewal Processes

1

ðkÞ Uir ðsÞ



n  Z X j j¼r

r

ðkÞ

est dPij ðtÞ

ðr ¼ 0; 1; 2; . . .; nÞ

0

for k ¼ 1; 2; . . .: Then, taking the LS transforms of (5.21)–(5.23) and arranging them,      iþ1 n þ 1 ðs þ rhÞdin ð1Þ  ; Uir ðsÞ ¼ F  ðs þ rhÞ s þ ðn þ 1Þh r r  ðkÞ ðk1Þ ðk1Þ Uir ðsÞ ¼ F  ðs þ rhÞ Uir ðsÞ þ Uir1 ðsÞ    nþ1 s þ rh ðk1Þ  ðsÞ ðr ¼ 0; 1; 2; . . .; nÞ; U s þ ðn þ 1Þh in r

ð5:24Þ

where din is the Kronecker delta, i.e., din  0 for i 6¼ n and din ¼ 1 for i ¼ n: From ðkÞ (5.24), Uir ðsÞ can be determined recursively for every k and r: It is more convenient to introduce the generating function in Appendix A.4 Uir ðz; sÞ 

1 X

ðkÞ

zk Uir ðsÞ

for jzj  1:

k¼1

Taking the generating function of (5.24),   iþ1  Uir ðz; sÞ ¼ zF ðs þ rhÞ Uir ðz; sÞ þ Uir1 ðz; sÞ þ r

  nþ1 s þ rh ½Uin ðz; sÞ þ din  :  s þ ðn þ 1Þh r Solving for Uir ðz; sÞ;   iþ1 zF  ðs þ rhÞ Uir ðz; sÞ ¼ Uir1 ðz; sÞ þ 1  zF  ðs þ rhÞ r  

nþ1 s þ rh  ½Uin ðz; sÞ þ din  : s þ ðn þ 1Þh r Denote Cr ðz; sÞ 

r Y

zF  ðs þ lhÞ 1  zF  ðs þ lhÞ l¼0

ðr ¼ 0; 1; . . .; nÞ;

C1 ðz; sÞ  1:

ð5:25Þ

5.2 Embedded Markov Chain

135

Then, dividing both side of (5.25) by Cr ðz; sÞ;   iþ1 Uir ðz; sÞ Uir1 1 ¼ þ Cr ðz; sÞ Cr1 ðz; sÞ Cr1 ðz; sÞ r   nþ1 s þ rh Uin ðz; sÞ þ din  C r r1 ðz; sÞ s þ ðn þ 1Þh

ðr ¼ 1; 2; . . .; nÞ;

Ui0 ðz; sÞ Uin ðz; sÞ þ din ¼1s : s þ ðn þ 1Þh C0 ðz; sÞ

ð5:26Þ

ð5:27Þ

Writing down (5.26) for r  1; r  2; . . .; 1; adding them, and from (5.27),   r  r  X iþ1 nþ1 Uir ðz; sÞ X 1 s þ jh Uin ðz; sÞ þ din ¼  ðz; sÞ ðz; sÞ s þ ðn þ 1Þh Cr ðz; sÞ C C l l r1 l1 l¼0 l¼0 ðr ¼ 0; 1; . . .; nÞ:

ð5:28Þ

Setting r ¼ n in (5.28) and rearranging it, Piþ1  iþ1 =Cl1 ðz; sÞ Uin ðz; sÞ þ din ¼ Pnþ1 l¼0 : l nþ1 s þ ðn þ 1Þh ðs þ lhÞ=Cl1 ðz; sÞ l¼0

ð5:29Þ

l

Thus, from (5.28), " Uir ðz; sÞ ¼ Cr ðz; sÞ

 r  X iþ1

1 ðz; sÞ C l l1 l¼0 # Pr  nþ1   iþ1 X iþ1 ðz; sÞ ðs þ lhÞ=C 1 l1 l¼0 l ;  P  nþ1 Cl1 ðz; sÞ nþ1 l ðs þ lhÞ=Cl1 ðz; sÞ l¼0 l¼0 l ð5:30Þ

that is given in terms of the binomial moment. By inverting the above binomial moment,

Pij ðz; sÞ



1 X k¼1

k

Z1

z

e 0

st

ðkÞ dPij ðtÞ

¼

n   X r r¼j

j

ð1Þrj Uir ðz; sÞ:

ð5:31Þ

136

5

Semi-Markov and Markov Renewal Processes ðkÞ

Therefore, the transition probabilities Pij  PrfZk ¼ jjZ0 ¼ ig for the process fZk ; k ¼ 0; 1; . . .g are given by the generating function Pij ðzÞ



1 X

ðkÞ zk Pij

¼

k¼1

lim Pij ðz; sÞ s!0

n   X r

¼

r¼j

j

ð1Þrj Uir ðz; 0Þ:

ð5:32Þ

The total expected number of visits to State j in ½0; t immediately before the P ðkÞ operating unit fails, initially starting from State i; is 1 k¼1 Pij ðtÞ: In particular, the P ðkÞ total expected number of system failures in ½0; t is 1 k¼1 Pin ðtÞ:

5.2.2 First-Passage Distributions We derive the first-passage distributions using the results of transition probabilities in Sect. 5.2.1: Let denote the first-passage distribution ðkÞ

Fij ðtÞ  PrfZk ¼ j; Zl 6¼ j for l ¼ 1; 2; . . .; k  1; sk  tjZ0 ¼ ig ði; j ¼ 0; 1; . . .; nÞ for k ¼ 1; 2; . . . : Then, by similar methods of obtaining (4.9), ð1Þ

ð1Þ

Pij ðtÞ ¼ Fij ðtÞ; ðkÞ Pij ðtÞ

¼

ðkÞ Fij ðtÞ

þ

k1 Z X l¼1

t ðklÞ

Pjj

ðlÞ

ðt  uÞdFij ðuÞ

ð5:33Þ

0 ðkÞ

for k ¼ 2; 3; . . . : The first-passage distribution Fij ðtÞ can be calculated recursively from (5.33) and are uniquely determined. Introduce the following notation

Fij ðz; sÞ



1 X k¼1

k

Z1

z

ðkÞ

est dFij ðtÞ for jzj  1:

0

Then, taking the generating functions and the LS transforms of (5.33), Fij ðz; sÞ ¼

Pij ðz; sÞ : 1 þ Pjj ðz; sÞ

ð5:34Þ

5.2 Embedded Markov Chain

137

Substituting (5.31) for (5.34), Pn  r rj Cr ðz; sÞ r¼j j ð1Þ hP  P  nþ1 r iþ1 =Cl1 ðz; sÞ nþ1 ðs þ lhÞ=Cl1 ðz; sÞ  l¼0 l¼0 l l i Piþ1  iþ1 Pr  nþ1 ð5:35Þ  l¼0 l =Cl1 ðz; sÞ l¼0 l ðs þ lhÞ=Cl1 ðz; sÞ : Fij ðz; sÞ ¼ P  jþ1 jþ1 =Cl1 ðz; sÞ l¼0 hP l  i Pnþ1  nþ1 r n rj C ðz; sÞ ðz; sÞ ð1Þ ðs þ lhÞ=C  r l1 r¼j j l¼rþ1 l Therefore, the first-passage distribution ðkÞ

Fij  PrfZk ¼ j; Zl 6¼ k for l ¼ 1; 2; . . .; k  1; jZ0 ¼ ig is given by the generating function Fij ðzÞ 

1 P

ðkÞ

zk Fij

k¼1

hP  P n rj r iþ1 C ðzÞ ð1Þ =Cl1 ðzÞ nl¼0 j =Cl ðzÞ r r¼j l¼0 l i Pr1  n Piþ1  iþ1 ðzÞ ðzÞ =C =C  l¼0 l1 l l¼0 l l hP  i ; ¼P  Pn  n r jþ1 jþ1 n rj ðzÞ C ðzÞ ðzÞ =C ð1Þ =C l1 r l l¼0 r¼j j l¼r l l Pn  r j

ð5:36Þ

where Cr ðzÞ  Cr ðz; 0Þðr ¼ 1; 0; 1; . . .Þ: In particular, for i\j; Fij ðzÞ

Piþ1  iþ1

l ¼ Pl¼0 jþ1  iþ1 l¼0

l

=Cl1 ðzÞ =Cl1 ðzÞ

:

Next, let lij be the mean first-passage time from State i to State j: Then, from (5.35), for i\j; jþ1 X 1  Fij ð1; sÞ ¼l s!0 s l¼1

lij  lim



jþ1 l



 

iþ1 l



1 ; Cl1

ð5:37Þ

Q where Cr  rl¼1 fF  ðlhÞ=½1  F  ðlhÞg (r ¼ 1; 2; . . .) and C0  1: In particular, when i ¼ 1 and j ¼ n; l1n agrees with [5]. Note that State -1 means the initial condition that one unit begins to operate and n units are in standby at time 0.

138

5

Semi-Markov and Markov Renewal Processes

5.2.3 Expected Numbers of Visits to States We obtain two reliability quantities from the first-passage distributions; (1) the expected number of visits to State j before system failure and (2) the expected number of failures and repairs before system failure. First, we derive the probabilities of visits to State j at least once before returning ðkÞ to State i: Let denote fm;ij (k ¼ 1; 2; . . .) for an embedded Markov chain fZk ; k ¼ 0; 1; 2; . . .g the probability of first-passage from State i to State k without ðkÞ passing through State m: In general, fm;ij is known as taboo probability for a Markov chain. We evidently have, for j 6¼ m; ð1Þ

ð1Þ

ðkÞ

ðkÞ

Fij ¼ fm;ij ; Fij ¼ fm;ij þ

k1 X

ðklÞ ðlÞ

ðk ¼ 2; 3; . . .Þ;

Fmj fj;im

ð5:38Þ

l¼1

because, the first-passage from State i to State j for a Markov chain fZk ; k ¼ 1; 2; . . .g occurs without passing through State m (6¼ j) or by passing through State m: By exchanging j and m in (5.38), (5.38) is hold and uniquely deterðkÞ mine fm;ij : Introduce the following generating function  Fm;ij ðzÞ 

1 X

ðkÞ

zk fm;ij

for jzj  1:

k¼1

Then, from (5.38), for j 6¼ m;  ðzÞ ¼ Fm;ij

  Fij ðzÞ  Fim ðzÞFmj ðzÞ :   1  Fjm ðzÞFmj ðzÞ

ð5:39Þ

 ðzÞ and the probability Substituting (5.36) for (5.39), we can obtain Fm;ij

fm;ij 

1 X k¼1

ðkÞ

 fm;ij ¼ lim Fm;ij ðzÞ z!1

ð5:40Þ

of the first-passage from State i to State j without passage through State m: Furthermore, because fi;ij ¼ 1  fj;ii (Problem 5.3), from (5.37) and (5.39), ," #   X  j jþ1  X 1 jþ1 ri r ð1Þ fi;ij ¼ 1 Cr for i\j  n; ð5:41Þ i l Cl1 r¼i l¼rþ1 that is the probability of visits to State j at least once before returning to State i:

5.2 Embedded Markov Chain

139

Next, we give two reliability quantities from the probability fj;jn : Let denote by ðkÞ Mj

(j ¼ 0; 1; . . .; n  1) the number of visits to State j for a Markov chain fZk ; k ¼ 1; 2; . . .g; including the first before system failure. Clearly, ðkÞ

PrfMj

X

¼ lg ¼

ðk Þ ðk Þ

ðk

Þ ðk Þ

fn;jj1 fn;jj2 fn;jjj1 fj;jnj

ðk; l ¼ 1; 2; . . .Þ:

k1 þk2 þ þkj ¼k;k1 ;k2 ;...;kj [ 0

ð5:42Þ By operating the convolution in (5.42), 1 X

ðkÞ

  ¼ lg ¼ ½Fn;jj ðzÞl1 Fj;jn ðzÞ;

ðkÞ

¼ lg ¼ ð1  fj;jn Þl1 fj;jn :

zk PrfMj

ð5:43Þ

k¼1

and from (5.40) and (5.43), 1 X

PrfMj

k¼1

Thus, the expected number Mj of visits to State j before system failure is Mj ¼

1 X 1 X 1 ðkÞ l PrfMj ¼ lg ¼ ; fj;jn l¼1 k¼1

and from (5.41), Mj ¼

n X r¼j

rj

ð1Þ

  X  nþ1  1 r nþ1 Cr : j l C l1 l¼rþ1

ð5:44Þ

Let M denote the total expected number of failures before system failures, when one unit starts to operate and n units are in standby as spare ones at time 0: Then,

M ¼1þ

n1 X

Mj ;

j¼0

because each failure causes one visit before system failure and the last failure causes system failure. From (5.44),  nþ1  X 1 nþ1 M¼ : j Cj1 j¼1 The expected number of repairs before system failure is M  ðn þ 1Þ: Note that lM is also the mean time to system failure that agrees with l1n in (5.37).

140

5

Semi-Markov and Markov Renewal Processes

5.2.4 Optimization Problems Consider the total cost of the operation for the system before system failure. It is assumed that the cost of each unit is c0 ; the repair cost incurred for each failed unit is c1 ; and the cost incurred for the system failure is c2 : Then, the total expected cost including the cost of system failure is ðn þ 1Þc0 þ ½M  ðn þ 1Þc1 þ c2 ; and hence, the expected cost rate is CðnÞ ¼

ðn þ 1Þðc0  c1 Þ þ Mc1 þ c2 : lM

ð5:45Þ

We can easily determine an optimum number of spare units that minimizes the total expected cost before system failure. If the total cost C before system failure is previously given, we can find an upper bound of the number of spare units such that ðn þ 1Þc0 þ ½M  ðn þ 1Þc1 þ c2  C:

ð5:46Þ

We can determine an optimum number of spare units that minimizes (5.45) within this number. In this case, because CðnÞ in (5.45) decreases to c0 =l (Problem 5.4), an optimum n is given by a unique maximum that satisfies (5.46).

5.3 Markov Renewal Process with Nonregeneration Points This section explains unique modifications of Markov renewal processes and applies them to redundant repairable systems including some nonregeneration points [6]. It has already been shown that such modifications give powerful plays for analyzing two-unit redundant systems [7] and communication systems [8]. It is assumed that the Markov renewal process under consideration has only one positive recurrent class, because we restrict ourselves to applications of reliability models. Consider the case where epochs at which the process enters some states are not regeneration points. Then, we partition a state space S into S ¼ S [ Sy (S \ Sy ¼ ;), where S is the portion of the state space such that the epoch entering State i (i 2 S ) is not a regeneration point, and Sy is such that the epoch entering State i (i 2 Sy ) is a regeneration point, where S and Sy are assumed not to be empty. Define the mass function Qij ðtÞ from State i (i 2 Sy ) to State j (j 2 S) by the probability that after entering State i; the process makes a transition into State j; in an amount of time less than or equal to t: However, it is impossible to define mass functions Qij ðtÞ for i (i 2 S ), because the epoch entering State i is not a regenðk ;k2 ;...;kl Þ

eration point. We define the new mass function Qij 1

ðtÞ that is the probability

5.3 Markov Renewal Process with Nonregeneration Points

141

that after entering State i (i 2 Sy ), the process next makes transition into State k1 ; k2 ; . . .; kl (k1 ; k2 ; . . .; kl 2 S ), and finally, enters State j (j 2 S), in an amount of time less than or equal to t:

5.3.1 Type 1 Markov Renewal Process Consider a Markov renewal process with m þ 1 states that consists of Sy ¼ f0g and S ¼ f1; 2; . . .; mg in Fig. 5.2. The process starts in State 0, i.e., Zð0Þ ¼ 0; and makes transitions into State 1; 2; . . .; m; and comes back to State 0. Then, from straightforward renewal arguments, the first-passage distributions are F01 ðtÞ ¼ Q01 ðtÞ; ð1;2;...;j1Þ

F0j ðtÞ ¼ Q0j

ð1;2;...;mÞ

F00 ðtÞ ¼ Q00

ðtÞ ðj ¼ 2; 3; . . .; mÞ;

ð5:47Þ

ðtÞ;

the renewal functions are ð1;2;...;mÞ

ðtÞ  M01 ðtÞ M01 ðtÞ ¼ Q01 ðtÞ þ Q00 ¼ Q01 ðtÞ þ F00 ðtÞ  M01 ðtÞ; ð1;2;...;j1Þ

M0j ðtÞ ¼ Q0j

ðtÞ þ F00 ðtÞ  M0j ðtÞ

M00 ðtÞ ¼ F00 ðtÞ þ F00 ðtÞ  M00 ðtÞ; and the transition probabilities are Fig. 5.2 State transition diagram for Type 1 Markov renewal process

ðj ¼ 2; 3; . . .; mÞ;

ð5:48Þ

142

5 ð1Þ

ð1;2;...;mÞ

P01 ðtÞ ¼ Q01 ðtÞ  Q02 ðtÞ þ Q00 ¼ P0j ðtÞ ¼

Semi-Markov and Markov Renewal Processes

ðtÞ  P01 ðtÞ

ð1Þ Q01 ðtÞ  Q02 ðtÞ þ F00 ðtÞ  P01 ðtÞ; ð1;2;...;j1Þ ð1;2;...;jÞ Q0j ðtÞ  Q0jþ1 ðtÞ þ F00 ðtÞ

 P0j ðtÞ

ðj ¼ 2; 3; . . .; mÞ;

P00 ðtÞ ¼ 1  Q01 ðtÞ þ F00 ðtÞ  P00 ðtÞ; ð1;2;...;mÞ

ð5:49Þ

ð1;2;...;mÞ

where Q0mþ1 ðtÞ ¼ Q00 ðtÞ ¼ F00 ðtÞ: Taking the LS transforms on both sides of (5.48) and (5.49),  M01 ðsÞ ¼

Q01 ðsÞ  ðsÞ ; 1  F00 ð1;2;...;j1Þ

Q0j ðsÞ  1  F00 ðsÞ  F00 ðsÞ  M00 ðsÞ ¼  ðsÞ ; 1  F00  ðsÞ ¼ M0j

ðj ¼ 2; 3; . . .; mÞ;

ð5:50Þ

and ð1Þ

P01 ðsÞ ¼

Q01 ðsÞ  Q02 ðsÞ ;  ðsÞ 1  F00 ð1;2;...;j1Þ

ð1;2;...;jÞ

ðsÞ  Q0jþ1  ðsÞ 1  F00  1  Q01 ðsÞ P00 ðsÞ ¼  ðsÞ ; 1  F00 P0j ðsÞ ¼

Q0j

ðsÞ

ðj ¼ 2; 3; . . .; mÞ;

ð5:51Þ

P  where note that m j¼0 P0j ðsÞ ¼ 1: From (5.50) and (5.51), the renewal functions and the transition probabilities are computed explicitly upon inversion, however, this might not be easy except in simple cases. Example 5.4 (Parallel system) Consider an n-unit parallel redundant system: When at least one of n units is operating, the system is operating. When all units are down simultaneously, the system fails and will begin to operate again immediately by replacing all failed units with new ones. Each unit operates independently and has an identical failure distribution FðtÞ: The states are denoted by the total number of failed units. When all units begin to operate at time 0, the mass functions are Q01 ðtÞ ¼ 1  ½FðtÞn ; n   X n ð1;2;...;j1Þ Q0j ½FðtÞi ½FðtÞni ðtÞ ¼ i i¼j

ðj ¼ 2; 3; . . .; nÞ:

Because the system begins to operate immediately at system failure,

5.3 Markov Renewal Process with Nonregeneration Points

ð1;2;...;nÞ

Q00

ð1;2;...;n1Þ

ðtÞ ¼ Q0n

143

ðtÞ ¼ FðtÞn :

If the system operates again after a maintenance time Y with distribution GðtÞ  PrfY  tg at system failure, then States 0 and n are regeneration points and ð1;2;...;nÞ ðtÞ ¼ FðtÞn  GðtÞ ¼ F00 ðtÞ: Thus, substituting the above Qn0 ðtÞ ¼ GðtÞ; Q00 equations for (5.50) and (5.51), we can obtain the renewal functions and the transition probabilities.

5.3.2 Type 2 Markov Renewal Process Consider a Markov renewal process with Sy ¼ f0g and S ¼ f1; 2; . . .; mg in Fig. 5.3. The process starts in State 0 and is only permitted to make a transition into one State j 2 S ; and then return to State 0. The LS transforms of the first-passage distributions, the renewal functions, and the transition probabilities are  ðsÞ ¼ F00

m X

ðiÞ

Q00 ðsÞ;

i¼1  ðsÞ F0j

¼

,"

Q0j ðsÞ

1

m X i¼1;i6¼j

Fig. 5.3 State transition diagram for Type 2 Markov renewal process

# ðiÞ Q00 ðsÞ

ð5:52Þ ðj ¼ 1; 2; . . .; mÞ;

144

5

Semi-Markov and Markov Renewal Processes

 F00 ðsÞ  ðsÞ ; 1  F00 Q0j ðsÞ  ðj ¼ 1; 2; . . .; mÞ; M0j ðsÞ ¼  ðsÞ 1  F00 P  1 m j¼1 Q0j ðsÞ  P00 ðsÞ ¼ ;  1  F00 ðsÞ  M00 ðsÞ ¼

ðjÞ

P0j ðsÞ

Q0j ðsÞ  Q00 ðsÞ ¼  ðsÞ 1  F00

ð5:53Þ

ð5:54Þ

ðj ¼ 1; 2; . . .; mÞ:

The process corresponds to a special one of Type 1 when m ¼ 1: That is, it is the simplest state space with one nonregeneration point. The process takes two alternate States 0 and 1. When the epoch entering State 1 is also a regeneration point, the process becomes an alternating renewal process described in Sect. 3.4. Example 5.5 (Inventory model) We give the inventory model of spare units [6] as one example of Markov renewal processes with nonregeneration points: A unit begins to operate at time 0. If the operating unit fails before time T; the spare unit is ordered immediately and is placed into service as soon as it is delivered after a lead time L: If the operating unit does not fail up to time T; the spare one is ordered at time T and is delivered after the lead time L: In this case, the spare unit is put into service when the original unit has failed or is put into inventory when it does not fail. Define the following states: State 0: Unit is operating and spare one is not delivered. State 1: Unit is operating and spare one is ordered. State 2: Unit is operating and spare one is in inventory. State 3: Operating unit has failed during the order of spare unit. State 4: Spare unit is ordered because of the failure of operating unit before time T: The system with the above five states forms a Markov renewal process with regeneration point f0g and nonregeneration points f1; 2; 3; 4g (Fig. 5.4).

Fig. 5.4 State transition diagram of spare part inventory

5.3 Markov Renewal Process with Nonregeneration Points

145

Assume that the failure times of each operating unit are identical and have a general distribution FðtÞ with mean time l: In addition, assume that PrfT  tg  AðtÞ and PrfL  tg ¼ BðtÞ are the degenerate distributions placing unit masses at T and L; respectively. Then, the mass functions of the above process are Q01 ðtÞ ¼

Zt FðuÞdAðuÞ; 0

ð1Þ

Q02 ðtÞ ¼

Zt

FðuÞd½AðuÞ  BðuÞ;

0 ð1;2Þ

Q00 ðtÞ ¼

Zt

½AðuÞ  BðuÞdFðuÞ;

0 ð1Þ

Q03 ðtÞ ¼

Zt

½AðuÞ  BðuÞdFðuÞ;

0

ð1;3Þ

Q00 ðtÞ ¼

8 Z t
Q04 ðtÞ ¼

:

0

9 =

½FðuÞ  FðvÞdAðvÞ dBðu  vÞ; ;

Zt AðuÞdFðuÞ;

Q40 ðtÞ ¼ BðtÞ:

0

Thus, the LS transforms of the above mass functions are Q01 ðsÞ ¼ esT FðTÞ; ð1Þ

Q02 ðsÞ ¼ esðTþLÞ FðT þ LÞ; Z1 ð1;2Þ est dFðtÞ; Q00 ðsÞ ¼ TþL ð1Þ Q03 ðsÞ

¼

Z1

st

e

dFðtÞ 

T ð1;3Þ Q00 ðsÞ

est dFðtÞ;

TþL sðTþLÞ

¼e

Q04 ðsÞ ¼

Z1

ZT

½FðT þ LÞ  FðTÞ;

est dFðtÞ;

0

Q40 ðsÞ ¼ esL :

146

5

Semi-Markov and Markov Renewal Processes

The LS transform of the recurrence time distribution to State 0 is ð1;2Þ

 H00 ðsÞ ¼ Q00

¼

Z1 e

ð1;3Þ

ðsÞ þ Q00

ðsÞ þ Q04 ðsÞQ40 ðsÞ

st

sðTþLÞ

dFðtÞ þ e

sL

½FðT þ LÞ  FðTÞ þ e

ZT

TþL

est dFðtÞ;

0

 where note that H00 ð0Þ ¼ 1: Thus, the mean recurrence time to State 0 is

l00 

Z1

ZTþL

tdH00 ðtÞ ¼ l þ

FðtÞdt: T

0

Using the above results, the LS transforms of the transition probabilities are P00 ðsÞ ¼

1  Q01 ðsÞ  Q04 ðsÞ ;  ðsÞ 1  H00

P01 ðsÞ ¼

Q01 ðsÞ  Q02 ðsÞ  Q03 ðsÞ ;  ðsÞ 1  H00

P02 ðsÞ ¼

Q02 ðsÞ  Q00 ðsÞ ;  ðsÞ 1  H00

ð1Þ

ð1Þ

ð1Þ

ð1;2Þ

ð1Þ

ð1;3Þ

Q03 ðsÞ  Q00 ðsÞ ;  ðsÞ 1  H00 Q ðsÞ½1  Q40 ðsÞ ; P04 ðsÞ ¼ 04  ðsÞ 1  H00 P03 ðsÞ ¼

where note that P00 ðsÞ þ P01 ðsÞ þ P02 ðsÞ þ P03 ðsÞ þ P04 ðsÞ ¼ 1: Thus, the limiting probabilities Pj  limt!1 P0j ðtÞ ¼ lims!0 P0j ðsÞ are 1 P0 ¼ l00

ZT FðtÞdt;

1 l00

Z1 FðtÞdt; TþL

ZTþL FðtÞdt; T

0

P2 ¼

1 P1 ¼ l00

2 TþL 3 Z 1 P3 ¼ 4 ½FðTÞ  FðtÞdt5; l00 T

LFðTÞ ; P4 ¼ l00 where also note that P0 þ P1 þ P2 þ P3 þ P4 ¼ 1 (Problem 5.5).

5.3 Markov Renewal Process with Nonregeneration Points

147

Furthermore, the LS transforms of the expected number of visits to State j are  ðsÞ ¼ M00

 H00 ðsÞ ;  ðsÞ 1  H00

 ðsÞ ¼ M01

Q01 ðsÞ ;  ðsÞ 1  H00

 ðsÞ ¼ M03

Q03 ðsÞ ;  1  H00 ðsÞ

ð1Þ

Q02 ðsÞ ;  1  H00 ðsÞ Q04 ðsÞ  M04 ðsÞ ¼  ðsÞ ; 1  H00  ðsÞ ¼ M02

ð1Þ

and the limiting expected number of visits to State j per unit of time, i.e., Mj   ðsÞ are limt!1 M0j ðtÞ=t ¼ lims!0 sM00 1 FðTÞ ; M1 ¼ ; l00 l00 FðT þ LÞ  FðTÞ M3 ¼ ; l00

M0 ¼

FðT þ LÞ ; l00 FðTÞ M4 ¼ ; l00 M2 ¼

where note that M0 ¼ M2 þ M3 þ M4 and M1 = M2 + M3 (Problem 5.6).

5.4 Problems 5 5.1 Derive Fj ðsÞ; Pj ðsÞ and Mj ðsÞðj ¼ 0; 1; 2Þ by taking the LS transforms of (5.1)–(5.3). 5.2 It is assumed in Example 5.3 that when the system is in State i (i ¼ 0; 1; 2), the data transmission succeeds with probability qi and fails with probability pi  1  qi ; where 0  p0  p1  p2  1 [3, p. 116]. Then, obtain the mean time l3 ðNÞ in which the data transmission succeeds and derive analytically an optimum number N  that minimizes l3 ðNÞ: 5.3 Prove that fi;ij ¼ 1  fj;ii . 5.4 Prove in (5.45) that CðnÞ decrease to c0 =l for c0  c1 : 5.5 Derive Pj and Mj (j ¼ 0; 1; 2; 3; 4) in Example 5.4. 5.6 It is assumed in Example 5.5 that c1 is the expedited order cost of a spare unit when the original unit fails before time T; c2 (\c1 ) is the ordinary order cost at time T; c4 is the inventory cost per unit of time in State 2, and c3 ([c4 ) is the system failure cost per unit of time in States 3 and 4. Then, when FðtÞ ¼ 1  ekt ; derive an optimum time T  that minimizes the expected cost rate CðTÞ ¼ c1 M4 þ c2 M1 þ c3 ðP3 þ P4 Þ þ c4 P2 :

148

5

Semi-Markov and Markov Renewal Processes

References 1. 2. 3. 4. 5.

Osaki S (1992) Applied stochastic system modeling. Springer, Berlin Çinlar E (1975) Introduction to stochastic processes. Prentice-Hall, Englewood Cliffs Nakagawa T (2008) Advanced reliability models and maintenance policies. Springer, London Takács L (1962) Introduction to the theory of queues. Oxford University Press, New York Srinivasan VS (1968) First emptiness in the spare parts problem for repairable components. Oper Res 16:407–415 6. Nakagawa T, Osaki S (1976) Markov renewal process with some nonregeneration points and their applications to reliability theory. Microelectron Reliab 15:633–636 7. Nakagawa T (2002) Two-unit redundant models. In: Osaki S (ed) Stochastic models in reliability and maintenance. Springer, Berlin, pp 165–185 8. Yasui K, Nakagawa T, Sandoh H (2002) Reliability models in data communication systems. In: Osaki S (ed) Stochastic models in reliability and maintenance. Springer, New York, pp 281–306

Chapter 6

Cumulative Processes

Failures of units or systems are generally classified into two failure modes: Catastrophic failure in which units fail by some sudden shock, and degradation failure in which units fail by physical deterioration suffered from some damage. In the latter case, units fail when the total damage due to shocks has exceeded a critical failure level. This is called a cumulative damage model or shock model with additive damage and can be described theoretically by a cumulative process [1] in stochastic processes. Damage models can be applied to actual units that are working in architecture, industry, service, information, and computers, and were summarized [2]. Let fNðtÞg be a counting process that represents the total numbers of events up to time t defined in Chaps. 2 and 3. Next, let Wk be some amount of increments due to the kth event. Then, define ZðtÞ 

NðtÞ X

Wk

ðNðtÞ ¼ 0; 1; 2; . . .Þ;

ð6:1Þ

k¼0

that is the total amount of increments at time t: The stochastic process fZðtÞg is a cumulative process [1] or is called a jump process [3] or doubly stochastic process [4] (Problem 6.1). In Sects. 6.1 and 6.2, we investigate the properties of the process fZðtÞg when increments due to each event are additive or not, respectively. We apply such results to damage models and obtain their reliabilities, mean times to failure, and failure rates. Furthermore, Sect. 6.3 proposes four modified damage models. In Sect. 6.4, we summarize compactly replacement policies for damage models where a unit is replaced at a planned time, a shock number or a damage level. The continuous damage model where the total damage increases with time will be dealt in Chap. 7 by applying a Brownian motion and a Lévy process. In this way, cumulative processes can be applied to not only reliability models but also stochastic models such as queueing, insurance, finance, shot noise, and stochastic duel [2, p. 170].

T. Nakagawa, Stochastic Processes, Springer Series in Reliability Engineering, DOI: 10.1007/978-0-85729-274-2_6, Ó Springer-Verlag London Limited 2011

149

150

6 Cumulative Processes

6.1 Standard Cumulative Process Consider a cumulative process in the context of Cox [1]: Let random variables Xk ðk ¼ 1; 2; . . .Þ denote a sequence of interarrival times between successive events that form Poisson or renewal processes introduced in Chaps. 2 and 3. Next, suppose that an increment Wk ðk ¼ 1; 2; . . .Þ such as damage, wear, fatigue, cost, and risk is associated with the kth event, where W0  0: Two random variables Xk and Wk might be possibly dependent. It is assumed that the sequence of fWk g is nonnegative, independently, and identically distributed, and is independent of Xj ðk 6¼ jÞ: Then, consider the cumulative process fZðtÞg defined in (6.1). Example 6.1 (Replacement cost) Suppose that each failed unit is replaced immediately at failures with a new one, i.e., Xk is the failure time between replacements. If Wk is the replacement cost for the kth failure, then ZðtÞ is the total replacement cost up to time t: Furthermore, if Xk is less than a planned replacement time T; Wk is the replacement cost c1 after failure, and conversely, if Xk is greater than T; Wk is the replacement cost c2 before failure, then ZðtÞ is the total b replacement cost for an age replacement policy in Sect. 3.3 and ZðtÞ ¼ CðtÞ in (3.42). On the other hand, suppose that an operating unit is replaced only at periodic times kT and each failed unit undergoes some repair, i.e., Xk is always equal to T: If Wk is the sum of repair and replacement costs in the kth replacement interval, ZðtÞ is the total cost up to time t for periodic replacement policies in Sect. 3.3.2. Example 6.2 (Cumulative damage model) A unit is subjected to shocks and suffers some damage, and each damage is additive. It is assumed that the unit fails when the total damage has exceeded a failure level K ð0\K\1Þ for the first time (Fig. 6.1).

Fig. 6.1 Process for a standard cumulative damage model

6.1 Standard Cumulative Process

151

We are interested in the following quantities of the cumulative process fZðtÞg : (i) PrfZðtÞ  xg; the distribution of the total increment at time t: (ii) EfZðtÞg; the total expected increment at time t: In addition, let K be the threshold level of the total increment. Defining Y  mint fZðtÞ [ Kg; (iii) PrfY  tg; the first-passage distribution to a critical point. (iv) EfYg; the mean time to a critical point. Suppose that the sequences of fXk g and fWk g are non-negative, independently, and identically distributed, and both of Xk and Wk are independent of each other. It is assumed that FðtÞ  PrfXk  tg with finite mean EfXk g  l and GðxÞ  PrfWk  xg with finite mean EfWk g  x for any t; x  0: Then, from (3.3), the number NðtÞ of events in ½0; t has the following probability PrfNðtÞ ¼ ng ¼ F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ

ðn ¼ 0; 1; 2; . . .Þ;

where uðnÞ ðtÞ denotes the n-fold Stieltjes convolution of any function uðtÞ with itself, and uð0Þ ðtÞ  1 for t  0: Thus, ( NðtÞ ) ( ) n X X Wk  x; NðtÞ ¼ n ¼ Pr Wk  xjNðtÞ ¼ n PrfNðtÞ ¼ ng Pr k¼0

k¼0 ðnÞ

¼ G ðxÞ½F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ: Therefore, the distribution of ZðtÞ defined in (6.1) is ( NðtÞ ) X Wk  x PrfZðtÞ  xg ¼ Pr k¼0

¼ ¼

1 X n¼0 1 X

(

Pr

n X

) Wk  xjNðtÞ ¼ n PrfNðtÞ ¼ ng

k¼0

GðnÞ ðxÞ½F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ;

ð6:2Þ

n¼0

and the survival probability is PrfZðtÞ [ xg ¼

1 X ½GðnÞ ðxÞ  Gðnþ1Þ ðxÞF ðnþ1Þ ðtÞ:

ð6:3Þ

n¼0

The total expected increment at time t is EfZðtÞg ¼

Z1

xd PrfZðtÞ  xg

0

¼x

1 X n¼1

F ðnÞ ðtÞ ¼ xMF ðtÞ;

ð6:4Þ

152

6 Cumulative Processes

P ðnÞ where MF ðtÞ  EfNðtÞg ¼ 1 n¼1 F ðtÞ is a renewal function with distribution FðtÞ and represents the expected number of events occurred in ½0; t; and its properties have been investigated in Sect. 3.2. It can be intuitively known that EfZðtÞg is given by the product of the average amount of increment suffered from each event and the expected number of events in time t: This is useful for estimating the total expected increment at time t: R1 Furthermore, from Theorem 3.1, if li  0 ti dFðtÞ\1ði ¼ 2; 3Þ and r2F  l2  l2 ; then as t ! 1;  2  t rF 1  þ oð1Þ; MF ðtÞ ¼ þ 2l2 2 l  4  r2 t 5rF 2r2F 3 2l3 þ þ VfNðtÞg ¼ F3 þ  þ oð1Þ; 4l4 l2 l 4 3l3 where oðhÞ=h ! 0 as h ! 0: Thus, when FðGÞ has finite mean lðwÞ and variance r2F ðr2G Þ; approximately, for large t (Problem 6.2), ( ( NðtÞ )) X Wk jNðtÞ ¼ EfNðtÞgEfWk g EfZðtÞg ¼ E E 

k¼1

 t r2F  l2 x þ ; 2l2 l VfZðtÞg ¼ EfZ 2 ðtÞg  ðEfZðtÞgÞ2 ( ( NðtÞ )) NðtÞ X X ¼E E Wk Wj jNðtÞ  ðEfZðtÞgÞ2 k¼1

ð6:5Þ

j¼1

¼ VfNðtÞgðEfWk gÞ2 þEfNðtÞgVfWk g   2     t rF r2G 5r4F 2r2F 3 2l3 r2G r2F  þ þ þ þ þ  1 :  x2 4l4 l2 2 l2 l l2 x2 4 3l3 Moreover, because EfZðtÞg x ¼ ; lim t!1 t l

  VfZðtÞg x2 r2F r2G ¼ ; þ t!1 l l2 x 2 t lim

ð6:6Þ

by applying Theorem 3.5 to the cumulative process fZðtÞg; we have the following asymptotic distribution of ZðtÞ given by a normal distribution: ( ) Zx ZðtÞ  xt=l 1 2  2   x ¼ pffiffiffiffiffiffi lim Pr eu =2 du: ð6:7Þ 2 2 2 2 t!1 ðx t=lÞ rF =l þ rG =x 2p 1

Example 6.3 (Cumulative damage model) In the cumulative damage model given in Example 6.2, we wish to estimate the total damage when the probability that is

6.1 Standard Cumulative Process

153

more than z in t ¼ 30 days of operation is given by 0.90. The distributions FðtÞ and GðxÞ of shock times and the amount of damage are unknown, but from sample data, the following estimations of means and variances are made: l ¼ 2 days;

r2F ¼ 5 ðdaysÞ2 ;

x ¼ 1;

r2G ¼ 0:5:

In this case, from (6.5), EfZð30Þg  15:125: Then, from (6.7), when t ¼ 30; ZðtÞ  xt=l Zð30Þ  15 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼  2  5:12 ðx2 t=lÞ rF =l2 þ r2G =x2 is approximately normally distributed with mean 0 and variance 1. Hence,

Zð30Þ  15 z  15 PrfZð30Þ [ zg ¼ Pr [ 5:12 5:12 Z1 1 2 eu =2 du ¼ 0:90:  pffiffiffiffiffiffi 2p ðz15Þ=5:12

pffiffiffiffiffiffi R 1 2 Because u0 ¼ 1:28 satisfies ð1= 2pÞ u0 eu =2 du ¼ 0:90; z ¼ 15  5:12  1:28  8:45: Thus, the total damage is more than 8.45 in 30 days with probability 0.90. Next, when a failure level is known as K ¼ 10;

Zð30Þ  15 10  15 [ PrfZðtÞ [ 10g ¼ Pr 5:12 5:12 Z1 1 2  pffiffiffiffiffiffi eu =2 du  0:836: 2p 0:98

Thus, the probability that the unit with a failure level K ¼ 10 fails in 30 days is about 0:836 (Problem 6.3). Next, suppose that the process ends when the total increment exceeds a threshold level K that is called a critical point. In this case, the probability that the total increment exceeds a threshold level K at the ðn þ 1Þth event is pnþ1 ¼ GðnÞ ðKÞ  Gðnþ1Þ ðKÞ ðn ¼ 0; 1; 2; . . .Þ: Define Y  mint fZðtÞ [ Kg: Because the event of fY  tg and fZðtÞ [ Kg are equivalent, the first-passage distribution to a critical point is, from (6.3), UðtÞ  PrfY  tg ¼ PrfZðtÞ [ Kg 1 X ½GðnÞ ðKÞ  Gðnþ1Þ ðKÞF ðnþ1Þ ðtÞ; ¼ n¼0

ð6:8Þ

154

6 Cumulative Processes

and its Laplace-Stieltjes (LS) transform from Appendix A is

U ðsÞ 

Z1

est dUðtÞ ¼

1 X

½GðnÞ ðKÞ  Gðnþ1Þ ðKÞ½F ðsÞnþ1 ;

ð6:9Þ

n¼0

0

where u ðsÞ denotes the LS transform of any function uðtÞ; i.e., u ðsÞ  R 1 st duðtÞ: Thus, the mean time to a critical point is 0 e   Z1 dU ðsÞ EfYg ¼ tdPrfY  tg ¼  ds  0 ð6:10Þ s¼0 1 X ¼l GðnÞ ðKÞ ¼ l½1 þ MG ðKÞ; n¼0

P1

where MG ðKÞ  n¼1 GðnÞ ðKÞ represents the expected number of events before a critical point. Similarly, when GðxÞ has finite mean x and variance r2G ; from (3.17), approximately,   K r2G þ x2 þ EfYg  l : ð6:11Þ 2x2 x In addition, when GðxÞ has an IFR property defined in Sect. 3.2.2 (Example 6.4), it has been shown that x x  1\MG ðxÞ  ; x x from (a) of Sect. 3.2.2.3. Thus,   lK K \EfYg  l þ1 : x x In Example 6.3, EfYg is approximately 21.5 days and 20\EfYg  22: Next, suppose that events occur in a nonhomogeneous Poisson process with an Rt intensity function hðtÞ and a mean value function HðtÞ  0 hðuÞdu in Sect. 2.3. Then, from (2.43), ½HðtÞn HðtÞ ðn ¼ 0; 1; 2; . . .Þ: e n! P j HðtÞ Thus, by replacing F ðnÞ ðtÞ with 1 formally, we can obtain the j¼n f½HðtÞ =j!ge following quantities: PrfNðtÞ ¼ ng ¼

6.1 Standard Cumulative Process

PrfZðtÞ  xg ¼

EfYg ¼

155

1 X

½HðtÞn HðtÞ GðnÞ ðxÞ ; e n! n¼0

1 X

ðnÞ

G ðKÞ

n¼0

Z1

EfZðtÞg ¼ xHðtÞ;

½HðtÞn HðtÞ e dt: n!

ð6:12Þ

0

If each event occurs at a constant time T ð0\T\1Þ; i.e., FðtÞ is the degenerate distribution placing unit mass at time T; and FðtÞ  0 for t\T and 1 for t  T; then Z1 ð½t=TÞ ðKÞ; EfYg ¼ Gð½t=TÞ ðKÞdt; PrfY  tg ¼ 1  G 0

where ½t=T denotes the greatest integer less than or equal to t=T: Finally, when GðxÞ  0 for x\1 and 1 for x  1; and K ¼ n; PrfY  tg ¼ F ðnþ1Þ ðtÞ;

EfYg ¼ ðn þ 1Þl:

6.2 Independent Process Consider the independent process where the total increment is not additive, and the process ends when one amount of increment exceeds a threshold level K: In the damage model, when the damage due to some shock has exceeded for the first time a failure level K; then the unit fails (Fig. 6.2). The practical examples of such models are the fracture of brittle materials and semiconductor parts that have failed by some over-current or fault voltage (Problem 6.4). The generalized model with three types of shocks where shocks with a small level of damage cause no damage to the unit, shocks with a large level of damage result in failure, and shocks with an intermediate level result in failure only with some probability are considered [5]. In this case, the probability that the increment exceeds firstly a threshold level K at the ðn þ 1Þth event is Pnþ1 ¼ ½GðKÞn  ½GðKÞnþ1 : Thus, the first-passage distribution to a critical point is PrfY  tg ¼

1 X f½GðKÞn  ½GðKÞnþ1 gF ðnþ1Þ ðtÞ; n¼0

Fig. 6.2 Process for a standard cumulative damage model

ð6:13Þ

156

6 Cumulative Processes

and its LS transform is Z1

est dPrfY  tg ¼

½1  GðKÞF ðsÞ : 1  GðKÞF ðsÞ

ð6:14Þ

0

Hence, the mean time to a critical point is l : 1  GðKÞ

EfYg ¼

ð6:15Þ

If shocks occur in a nonhomogeneous Poisson process with a mean value function HðtÞ; then PrfY  tg ¼

1 X

f1  ½GðKÞn g

n¼0

½HðtÞn HðtÞ e n!

¼ 1  e½1GðKÞHðtÞ ; Z1 EfYg ¼ e½1GðKÞHðtÞ dt:

ð6:16Þ

0

If events occur at a constant time T; PrfY  tg ¼ 1  ½GðKÞ½t=T ; Z1 EfYg ¼ ½GðKÞ½t=T dt: 0

Example 6.4 (Failure rate) It is important to investigate the properties of failure rates practically and theoretically. Let Pn denote the probability of surviving the first n shocks ðn ¼ 0; 1; 2; . . .Þ; where P0  1; and Fn ðtÞ be the probability that n shocks occur in time t; where F0 ðtÞ  1: Then, the survival function that the unit does not fail in time t is written in the following general form: From (6.2), UðtÞ ¼ PrfY [ tg ¼

1 X n¼0

Pn PrfNðtÞ ¼ ng ¼

1 X

Pn ½Fn ðtÞ  Fnþ1 ðtÞ:

n¼0

Let fn ðtÞ be a density function of Fn ðtÞ: The failure rate is rðtÞ 

P1 0 U ðtÞ ðPn  Pnþ1 Þfnþ1 ðtÞ : ¼ P1n¼0 P UðtÞ n¼0 n ½Fn ðtÞ  Fnþ1 ðtÞ

ð6:17Þ

6.2 Independent Process

157

In particular, when shocks occur in a Poisson process with rate k; i.e., P j kt Fn ðtÞ ¼ 1 ; j¼n ½ðktÞ =j!e 1 X ðktÞn kt UðtÞ ¼ Pn ð6:18Þ e : n! n¼0 The probabilistic properties of UðtÞ were extensively investigated [6, 7]. The failure rate is, from (6.17), P1

Pnþ1 ½ðktÞn =n!  k: rðtÞ ¼ k 1  Pn¼0 1 n n¼0 Pn ½ðktÞ =n! When Pn ¼ qn ; i.e., q ¼ GðKÞ and the total damage is not additive, from (6.18), UðtÞ ¼ ekð1qÞt ;

rðtÞ ¼ kð1  qÞ;

that is constant. Any distribution FðtÞ is said to have the property of IFR (Increasing Failure Rate) if and only if ½Fðt þ xÞ  FðtÞ=FðtÞ increases with t for x [ 0 and FðtÞ\1 [8, p. 23], where FðtÞ  1  FðtÞ: Furthermore, it has been proved that FðtÞ is IFR 0 if and only if hðtÞ  F ðtÞ=FðtÞ increases with t: The failure rate has been discussed in Sect. 2.3. In this model, the following properties (i) and (ii) were proved [9]: (i) The failure rate rðtÞ in (6.17) increases with t if ðPn  Pnþ1 Þ=Pn increases with n: When Pnþ1 =Pn decreases with n and fnþ1 ðtÞ ¼ ½ðktÞn =n!ekt ; we prove that rðtÞ increases with t: Differentiating rðtÞ in (6.17) with respect to t; " # 1 1 X k2 ðktÞn X ðPnþ1 Pkþ1  Pnþ2 Pk Þ : ½UðtÞ2 n¼0 n! k¼0 The bracket is   1 1 X X ðktÞn ðktÞk Pkþ1 Pnþ1 Pnþ1 Pk  n! k! Pk Pnþ1 n¼0 k¼0 "    # n k 1 nþ1 1 X ðktÞ X X ðktÞ Pkþ1 Pnþ2 ðktÞk Pkþ1 Pnþ2 ¼ þ Pnþ1 Pk  Pk  n! k! k! Pk Pnþ1 Pk nþ1 n¼0 k¼0 k¼nþ1   1 nþ1 X X ðktÞn ðktÞk Pkþ1 Pnþ2 Pnþ1 Pk  ¼ n! k! Pk Pnþ1 n¼0 k¼0   1 nþ1 X X ðktÞnþ1 ðktÞk1 Pnþ2 Pkþ1 Pnþ1 k Pk  þ ðn þ 1Þ! k! Pnþ1 Pk n¼0 k¼0   n k 1 nþ1 X ðktÞ X ðktÞ Pkþ1 Pnþ2 ðn þ 1  kÞ  0; Pnþ1 Pk  ¼ ðn þ 1Þ! k! Pk Pnþ1 n¼0 k¼0 that completes the proof.

158

6 Cumulative Processes

In addition, when the total damage is additive, from (6.8), UðtÞ ¼

1 X

GðnÞ ðKÞ

n¼0

ðktÞn kt e : n!

ð6:19Þ

Rt (ii) The failure rate average 0 rðuÞdu=t increases with t because ½GðnÞ ðxÞ1=n decRt reases with n: Note that if rðtÞ increases, then 0 rðuÞdu=t also increases. In P j hK particular, when Pn ¼ GðnÞ ðKÞ ¼ 1 ; Pnþ1 =Pn decreases j¼n ððhKÞ =j!Þe strictly with n from ð1  ehK Þ to 0. Thus, the failure rate rðtÞ increases strictly from kehK to k (Example 6.5). In general, for the cumulative damage model with a failure level K; the failure rate is, setting that Pn ¼ GðnÞ ðKÞ and Fn ðtÞ ¼ F ðnÞ ðtÞ in (6.17), P1 ½GðnÞ ðKÞ  Gðnþ1Þ ðKÞf ðnþ1Þ ðtÞ : ð6:20Þ rðtÞ  Pn¼0 1 ðnÞ ðnÞ ðnþ1Þ ðtÞ n¼0 G ðKÞ½F ðtÞ  F When shocks occur in a nonhomogeneous Poisson process with an intensity function hðtÞ and a mean value function HðtÞ [7], from (6.12), UðtÞ ¼

1 X n¼0

Pn

½HðtÞn HðtÞ e : n!

ð6:21Þ

(iii) The failure rate rðtÞ increases if both hðtÞ and ðPn  Pnþ1 Þ=Pn increase. Rt (iv) The failure rate average 0 rðuÞdu=t increases if both rðtÞ=t and ðPn  Pnþ1 Þ=Pn increase. When the total damage is additive, (6.19) is UðtÞ ¼

1 X n¼0

GðnÞ ðKÞ

½HðtÞn HðtÞ : e n!

ð6:22Þ

Then, properties (iii) and (iv) are rewritten as (Problem 6.5): (v) The failure rate rðtÞ increases if both hðtÞ and ½GðnÞ ðKÞ  Gðnþ1Þ ðKÞ=GðnÞ ðKÞ increase. Rt (vi) The failure rate average 0 rðuÞdu=t increases if both HðtÞ=t and ½GðnÞ ðKÞ  Gðnþ1Þ ðKÞ=GðnÞ ðKÞ increase. Such results were compactly summarized [10]. Moreover, when shocks occur in the birth process [11], in the counting process [12], and in the Lévy process [13, 14], similar results were obtained. Example 6.5 (Exponential damage and shock times) Suppose that FðtÞ ¼ 1  ekt and GðxÞ ¼ 1  ehx ; i.e., shocks occur in a Poisson process with rate k and each

6.2 Independent Process

159

damage due to shocks is exponential with mean 1=h: In this case, both nonhomogeneous Poisson and renewal processes form the same Poisson process, i.e., 1 X ½HðtÞk

Fn ðtÞ ¼ F ðnÞ ðtÞ ¼

k!

k¼n

eHðtÞ ¼

1 X ðktÞk k¼n

k!

ekt

ðn ¼ 0; 1; 2; . . .Þ:

In the cumulative damage model, from (6.2), Z1

esx d PrfZðtÞ  xg ¼ ekts=ðsþhÞ :

0

By inversion [8, p. 80], 2

3 Zx p ffiffiffiffiffiffi ffi p ffiffiffiffiffiffiffiffiffi PrfZðtÞ  xg ¼ ekt 41 þ kht ehu u1=2 I1 ð2 khtuÞdu5; 0

where Ii ðxÞ is the Bessel function of order i for the imaginary argument defined by Ii ðxÞ 

1 2jþi X x j¼0

2

1 : j!ðj þ iÞ!

Thus, from (6.8), 2

3 ZK p ffiffiffiffiffiffi ffi p ffiffiffiffiffiffiffiffiffi UðtÞ ¼ PrfY  tg ¼ 1  ekt 41 þ kht ehu u1=2 I1 ð2 khtuÞdu5: 0

Furthermore, from (6.4), (6.6), and (6.10), EfZðtÞg ¼

kt ; h

VfZðtÞg ¼

2kt ; h2

EfYg ¼

hK þ 1 : k

The failure rate is, from (6.17), pffiffiffiffiffiffiffiffiffiffi kekthK I0 ð2 khtK Þ rðtÞ ¼ pffiffiffiffiffiffiffi R K pffiffiffiffiffiffiffiffiffi : 1 þ kht 0 ehu u1=2 I1 ð2 khtuÞdu Because GðnÞ ðKÞ  Gðnþ1Þ ðKÞ ðhKÞn =n! ¼ P 1 j GðnÞ ðKÞ j¼n ðhKÞ =j! increases with n from ehK to 1 (Problem 6.6), rðtÞ increases from kehK to k: In the independent damage model, from (6.13),   PrfY  tg ¼ 1  exp ktehK ;

160

6 Cumulative Processes

and from (6.15), EfYg ¼

1 1 ¼ ehK : rðtÞ k

Example 6.6 (Replacement policy) Consider a replacement policy with two kinds of damage: Shocks occur in a renewal process according to FðtÞ with finite mean l, and the amount Wk of damage 1 due to the kth shock has an identical distribution GðxÞ  PrfWk  xg with finite mean x: Each damage 1 is additive, and the unit fails when the total damage exceeds a failure level K: Suppose that the unit is replaced at time T ð0\T  1Þ or at failure, whichever occurs first. Then, the probability that the unit is replaced before failure at time T is, from (6.8), UðTÞ ¼

1 X

GðnÞ ðKÞ½F ðnÞ ðTÞ  F ðnþ1Þ ðTÞ;

n¼0

and the probability that it is replaced at failure is UðTÞ ¼

1 X ½GðnÞ ðKÞ  Gðnþ1Þ ðKÞF ðnþ1Þ ðTÞ: n¼0

Thus, the mean time to replacement is

TUðTÞ þ

ZT 0

tdUðtÞ ¼

1 X n¼0

ðnÞ

G ðKÞ

ZT

½F ðnÞ ðtÞ  F ðnþ1Þ ðtÞdt:

0

Introduce the replacement costs: cK is the replacement cost at failure and cT is the replacement cost at time T; where cK [ cT : Then, from (3.19), the expected cost rate is P ðnÞ ðnÞ ðnþ1Þ cK  ðcK  cT Þ 1 ðTÞ n¼0 G ðKÞ½F ðTÞ  F e CðTÞ ¼ : RT P1 ðnÞ ðnÞ ðnþ1Þ ðtÞdt n¼0 G ðKÞ 0 ½F ðtÞ  F

ð6:23Þ

Next, that another damage 2 occurs in a non homogeneous Poisson process with Rt an intensity function hðtÞ and a mean value function HðtÞ  0 hðuÞdu; i.e., the probability of n occurrences of damage 2 in ½0; t is given in (2.43). It is assumed that damage 2 occurs independently for damage 1 and its damage is not additive. When damage 2 occurs, the unit undergoes only minimal repair and its amount of damage 2 becomes zero. Thus, the expected number of occurrences of damage 2, i.e., minimal repair, before the replacement is, from (2.45),

6.2 Independent Process

HðTÞ

1 X

161

Z 1 X ðTÞþ ½GðnÞ ðKÞGðnþ1Þ ðKÞ HðtÞdF ðnþ1Þ ðtÞ T

ðnÞ

ðnÞ

G ðKÞ½F ðTÞF

ðnþ1Þ

n¼0

¼

1 X

n¼0

GðnÞ ðKÞ

n¼0

ZT

0

½F ðnÞ ðtÞF ðnþ1Þ ðtÞhðtÞdt:

0

Therefore, letting cM be the minimal repair cost and adding the total minimal e repair cost before replacement to CðTÞ in (6.23), P ðnÞ ðnÞ ðnþ1Þ cK  ðcK  cT Þ 1 ðTÞ n¼0 G ðKÞ½F ðTÞ  F R T ðnÞ P1 ðnÞ ðnþ1Þ þcM n¼0 G ðKÞ 0 ½F ðtÞ  F ðtÞhðtÞdt CðTÞ ¼ : RT P1 ðnÞ ðnÞ ðnþ1Þ ðtÞdt n¼0 G ðKÞ 0 ½F ðtÞ  F

ð6:24Þ

Clearly, Cð0Þ  lim CðTÞ ¼ 1; T!0

Cð1Þ  lim CðTÞ ¼

cK þ cM

P1

n¼0

T!1

R1 GðnÞ ðKÞ 0 ½F ðnÞ ðtÞF ðnþ1Þ ðtÞhðtÞdt ; l½1 þ MG ðKÞ

whose denominator represents the mean time to failure in (6.10). Thus, there exists a positive T ð0\T  1Þ that minimizes CðTÞ in (6.24). We find an optimum time T that minimizes CðTÞ: Differentiating CðTÞ with respect to T and setting it equal to zero, ( ðcK  cT Þ rðTÞ

1 X

ðnÞ

G ðKÞ

n¼0

ZT

½F ðnÞ ðtÞ  F ðnþ1Þ ðtÞdt

0

1 X  ½GðnÞ ðKÞ  Gðnþ1Þ ðKÞF ðnþ1Þ ðTÞ

)

n¼0

þ cM

1 X n¼0

ðnÞ

G ðKÞ

ZT

½F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ½hðTÞ  hðtÞdt ¼ cT ;

ð6:25Þ

0

where rðTÞ is given in (6.20). It can be clearly seen that if rðtÞ increases strictly and hðtÞ increases, or rðtÞ increases and hðtÞ increases strictly (Problem 6.7), then the left-hand side of (6.25) increases strictly from 0 to

162

6 Cumulative Processes

ðcK  cT8Þflrð1Þ½1 þ MG ðKÞ  1g 9 Z1 1 = < X þ cM lhð1Þ½1 þ MG ðKÞ  GðnÞ ðKÞ ½F ðnÞ ðtÞ  F ðnþ1Þ ðtÞhðtÞdt : ; : n¼0 0

ð6:26Þ Thus, if (6.26) is greater than cT ; then there exists a finite and unique T ð0\T \1Þ that satisfies (6.25). When HðtÞ ¼ at; i.e., hðtÞ ¼ a ða [ 0Þ; from (6.26), if lrð1Þ½1 þ MG ðKÞ [ cK =ðcK  cT Þ then there exists a finite T that satisfies (6.25). In P j hK ; addition, when FðtÞ ¼ 1  ekt and GðnÞ ðKÞ ¼ 1 j¼n ½ðhKÞ =j!e rðTÞ ¼

k

P1

n¼0 ½G

ðKÞ  Gðnþ1Þ ðKÞ½ðkTÞn =n! n ðnÞ n¼0 G ðKÞ½ðkTÞ =n!

ðnÞ

P1

increases strictly from kehK to k; as shown in Example 6.5. Thus, if hK [ cT =ðcK  cT Þ; then there exists a finite and unique T ð0\T \1Þ that satisfies (6.25).

6.3 Modified Damage Models We give the following four modified damage models [2, p. 28]: Damage model with imperfect shock where some shock may produce no damage, a failure level K is a random variable with a general distribution LðxÞ; the total damage decreases exponentially with time, and the total damage increases with time. A variety of shock models were widely summarized [14].

6.3.1 Imperfect Shock It has been assumed that shocks occur in a renewal process with FðtÞ and each damage due to shocks is distributed generally with GðxÞ: However, it may be considered the case where some shocks do not produce any damage. Suppose that each shock does some damage with probability p (0\p  1Þ and no damage with probability q  1  p: Then, the distribution of time where some shocks does the damage in time t is F1 ðtÞ  ½1 þ qFðtÞ þ qFðtÞ qFðtÞ þ  pFðtÞ; and its LS transform is F1 ðsÞ ¼

pF ðsÞ : 1  qF ðsÞ

ð6:27Þ

6.3 Modified Damage Models

163

Hence, the mean time to the first damage due to shocks is EfY1 g 

Z1

l tdF1 ðtÞ ¼ : p

0

Thus, by replacing FðtÞ with F1 ðtÞ; we can get the results in the cases where shocks are imperfect. In particular, when FðtÞ ¼ 1  ekt ; F1 ðtÞ ¼ 1  epkt and EfY1 g ¼ 1=ðpkÞ; and PrfNðtÞ ¼ ng ¼

ðpktÞn pkt e n!

ðn ¼ 0; 1; 2; . . .Þ:

Similarly, when shocks occur in a non homogeneous Poisson process with a mean value function HðtÞ; ½pHðtÞn pHðtÞ e ðn ¼ 0; 1; 2; . . .Þ; n! EfNðtÞg ¼ VfNðtÞg ¼ pHðtÞ:

PrfNðtÞ ¼ ng ¼

Therefore, substituting F1 ðtÞ in (6.27) for FðtÞ in (6.2), (6.4), (6.8), (6.10) and (6.20), PrfZðtÞ  xg; EfZðtÞg; PrfY  tg; EfYg; and rðtÞ are given. In particular, from (6.9) and (6.10), Z1 e 0

st

  1 X pF ðsÞ nþ1 ðnÞ ðnþ1Þ dPrfY  tg ¼ ½G ðKÞ  G ðKÞ ; 1  qF ðsÞ n¼0 l EfYg ¼ ½1 þ MG ðKÞ: p

The corresponding results for the independent damage model are, from (6.14) and (6.15), Z1

est dPrfY  tg ¼

p½1  GðKÞF ðsÞ ; 1  ½q þ pGðKÞF ðsÞ

0

EfYg ¼

l : p½1  GðKÞ

Example 6.7 (Parallel system) Consider a parallel redundant system with n identical units, each of which fails at shocks with probability p; where q  1  p; and shocks occur in a renewal process with mean interval l (Problem 6.8). Let Wk be the total number of units that fail at the kth ðk ¼ 1; 2; . . .Þ shock. Then, because P the probability that one unit fails until the kth shock is kj¼1 pqj1 ¼ 1  qk ; the mean time to system failure is [2, p. 62]

164

6 Cumulative Processes 1 X

kl PrfW1 þ W2 þ þ Wk1  n  1 and W1 þ þ Wk ¼ ng

k¼1

¼

1 X

kl½ð1  qk Þn  ð1  qk1 Þn 

k¼1 1 X

n   X n 1 ð1Þkþ1 ¼l ½1  ð1  q Þ  ¼ l : 1  qk k k¼0 k¼1 k n

6.3.2 Random Failure Level Most units have individual variations in their ability to withstand shocks and are operating in different environments. In such cases, a failure level K is not constant and would be random. Consider the case where a failure level K is a random variable with a general distribution LðxÞ such that Lð0Þ ¼ 0: Then, for the cumulative damage model, the distribution of time to failure is, from (6.8), PrfY  tg ¼

1 X

F

ðnþ1Þ

ðtÞ

Z1

n¼0

½GðnÞ ðxÞ  Gðnþ1Þ ðxÞdLðxÞ;

ð6:28Þ

0

and its mean time is, from (6.10),

EfYg ¼ l

Z1

½1 þ MG ðxÞdLðxÞ:

ð6:29Þ

0

The failure rate is, from (6.20), P1 ðnþ1Þ R 1 ðnÞ f ðtÞ 0 ½G ðxÞ  Gðnþ1Þ ðxÞdLðxÞ R : rðtÞ ¼ Pn¼0 1 ðnÞ ðnþ1Þ ðtÞ 1 GðnÞ ðxÞdLðxÞ n¼0 ½F ðtÞ  F 0

ð6:30Þ

For the independent damage model,

PrfY  tg ¼

1 X

F

ðnþ1Þ

n¼0

EfYg ¼ l

0

1 1 Z X n¼0

Z1 ðtÞ f½GðxÞn  ½GðxÞnþ1 gdLðxÞ;

0

ð6:31Þ n

½GðxÞ dLðxÞ:

6.3 Modified Damage Models

165

For the cumulative model with imperfect shock, Z1 e

st

0

 Z1 1  X pF ðsÞ nþ1 dPrfY  tg ¼ ½GðnÞ ðxÞ  Gðnþ1Þ ðxÞdLðxÞ: ðsÞ 1  qF n¼0

ð6:32Þ

0

Example 6.8 (Random failure level) Suppose that all random variables are exponential. When FðtÞ ¼ 1  ekt ; F1 ðsÞ ¼ pk=ðs þ pkÞ; i.e., F1 ðtÞ ¼ 1  epkt by inversion. Thus, substituting k for pk; we can obtain the corresponding results. When GðxÞ ¼ 1  ehx and a failure level K also has an exponential distribution LðxÞ ¼ 1  ebx ; Z1

½GðnÞ ðxÞ  Gðnþ1Þ ðxÞdLðxÞ ¼

0

bhn ðb þ hÞnþ1

:

Thus, from (6.28), Z1

st

e 0

nþ1 1  X k bhn kb : dPrfY  tg ¼ ¼ nþ1 sþk sðb þ hÞ þ kb ðb þ hÞ n¼0

By inversion, 

 kbt PrfY  tg ¼ 1  exp  ; bþh   1 1 h ¼ þ1 : EfYg ¼ rðtÞ k b For the independent damage model,

PrfY [ tg ¼

Z1

  exp ktehx bebx dx

0

Z 1 X ðktÞn

1

¼

n¼0

EfYg ¼

n!

1 1 ¼ rðtÞ k

beðbþnhÞx dx ¼

0

Z1 0

ehx bebx dx ¼

1 X ðktÞn n¼0

(

n!

b ; b þ nh

b kðbhÞ

ðb [ hÞ;

1

ðb  hÞ:

166

6 Cumulative Processes

6.3.3 Damage with Annealing The total damage in the usual reliability models is additive and does not decrease. In some materials, annealing, i.e., lessening the damage, can take place in materials such as rubber, fiber reinforced plastics, and polyurethane. Takac´s [15] considered the following damage model: If a unit suffers some damage W due to shock, then its damage after time duration t is reduced to Weat ð0\a\1Þ: Define ZðtÞ ¼

NðtÞ X

Wn exp½aðt  Sn Þ;

ð6:33Þ

n¼1

P where Sn  nk¼1 Xk (Fig. 6.3). Suppose that shocks occur in a Poisson process with rate k: Then, Uðt; xÞ  PrfZðtÞ  xg forms the following renewal equation [15, p. 105] 8 9 Zx < = dUðt; xÞ ¼ k Uðt; xÞ  G½ðx  yÞeat dy Uðt; yÞ ; : ; dt 0

and its LS transform is dU ðt; sÞ ¼ k½1  G ðseat ÞU ðt; sÞ; dt R1 R1 where U ðt; sÞ  0 esx dUðt; xÞ and G ðsÞ  0 esx dGðxÞ: Solving the differential equation, 8 9 < Zt = U ðt; sÞ ¼ exp k ½1  G ðseau Þdu ; : ; ð6:34Þ 0  dU ðt; sÞ kxð1  eat Þ EfZðtÞg ¼  ¼ : ds s¼0 a Fig. 6.3 Process for a cumulative damage model with annealing

6.3 Modified Damage Models

167

In addition, when x\1; limt!1 PrfZðtÞ  xg exits and its LS transform is 9 8 < k Z1 1  G ðsuÞ = ð6:35Þ du : U ð1; sÞ ¼ exp  ; : a u 0

We derive (6.34) by another method: When shocks occur in a Poisson process, from (2.29) and (2.37), unordered random variables of n shock times S1 ; S2 ; . . .; Sn ; given that NðtÞ ¼ n; are independent and identically distributed uniformly over the interval ½0; t; i.e., t  Sk ðk ¼ 1; 2; . . .Þ are also distributed uniformly over ½0; t: Thus, when NðtÞ ¼ 1; ZðtÞ ¼ W1 exp½aðt  S1 Þ and its LS transform is Z1 e

sx

1 dPrfZðtÞ  xg ¼ EfG ½s expðaS1 Þg ¼ t

0

Zt

G ðseau Þdu:

0

Generally, when NðtÞ ¼ n; Z1

2 t 3n Z 1 esx dPrfZðtÞ  xg ¼ ðEfG ½s expðaS1 ÞgÞn ¼ 4 G ðseau Þdu5 : t

0

0

So that, using the law of total probability, Z1

2

1 X ðktÞn

3n

Zt

1 ekt 4 G ðseau Þdu5 n! t n¼0 0 8 9 t Z < = ¼ exp k ½1  G ðseau Þdu : : ;

est dPrfZðtÞ  xg ¼

0

0

Furthermore, (6.34) is also derived directly as follows [16, p. 40]:

EfZðtÞjNðtÞ ¼ ng ¼ E

( NðtÞ X

) Wk exp½aðt  Sk ÞjNðtÞ ¼ n

k¼1

¼

n X    E Wk exp½aðt  Sk ÞNðtÞ ¼ n k¼1

n X       ¼ E Wk NðtÞ ¼ n E exp½aðt  Sk ÞNðtÞ ¼ n k¼1

¼ EfWk geat

n X    E expðaSk ÞNðtÞ ¼ n : k¼1

168

6 Cumulative Processes

Recalling that Sk is distributed uniformly over ½0; t; ( ) n X    expðaSk Þ E ZðtÞNðtÞ ¼ n ¼ xeat E k¼1

¼ xeat

n t

Zt

eax dx ¼

xn ð1  eat Þ: at

0

So that, because EfNðtÞg ¼ kt;

xNðtÞ kx at ð1  e Þ ¼ ð1  eat Þ: EfZðtÞg ¼ E at a Example 6.9 (1) When GðxÞ ¼ 1  ehx , 



U ðt; sÞ ¼

s þ heat sþh

m

ekt ;

where m  k=a: Thus, by inversion [2, p. 32],  1  1 X X ðhxeat Þi mþj1 kt PrfZðtÞ  xg ¼ e expðhxeat Þ: ð1  eat Þj j i! i¼j j¼0 In a similar way,  m h U ð1; sÞ ¼ ; sþh

lim PrfZðtÞ  xg ¼

Zx

t!1

hðhuÞm1 hu e du; CðmÞ

0

that is a gamma distribution with mean m=h: (2) When GðxÞ  0 for x\x and 1 for x  x; i.e., the damage due to each shock is constant and its amount is x: From the results [15, p. 129], 0 1  m Z1 su 1 e duA; exp@m U ð1; sÞ ¼ u sxc x

where c  eC ¼ 1:781072. . . and C  0:577215. . . that is Euler’s constant. By inversion, Rx P  j j m ðjÞ xm þ 1 j¼1 ð1Þ ðmÞ =j! jx ðx  uÞ I ðuÞdu lim PrfZðtÞ  xg ¼ ; t!1 ðxcÞm Cð1 þ mÞ where IðuÞ is uniform over ½0; x:

6.3 Modified Damage Models

169

6.3.4 Increasing Damage with Time Consider the cumulative damage model with two kinds of damage (Fig. 6.4). One of them is caused by shock and is additive, and the other increases proportionately with time. That is, the total damage is accumulated subject to shocks and time at the rate of constant a ða [ 0Þ; independent of shocks. A unit fails whether the total damage exceeds with time or has exceeded a failure level K at some shocks, and its failure is detected only at the time of shocks. Such models would be the life of dry and storage batteries. The replacement policy where the unit is replaced before failure will be discussed of Chap. 7. P in Example 7.4 P Suppose that Sn  nk¼1 Xk and Zn  nk¼1 Wk ; where S0 ¼ W0 ¼ 0: Because PrfSn  tg ¼ F ðnÞ ðtÞ and PrfZn  xg ¼ GðnÞ ðxÞ; the distribution of time to detect a failure at some shock is [2, p. 35], PrfY  tg ¼

1 X

PrfZn þ aSn \K  Znþ1 þ aSnþ1 ; Snþ1  tg

n¼0

¼

8 tu t 1 Z
0

:

0

9 =

½GðnÞ ðK  auÞ  Gðnþ1Þ ðK  aðu þ xÞÞdFðxÞ dF ðnÞ ðuÞ; ; ð6:36Þ

where note that GðnÞ ðxÞ ¼ 0 for x\0: Thus, the mean time to detect a failure at some shock is

Fig. 6.4 Process for a cumulative damage model with two kinds of damages

170

6 Cumulative Processes

EfYg ¼

8 1 1 1 Z
¼l

0

:

0

K=a 1 Z X n¼0

9 = h i ðt þ xÞ GðnÞ ðK  atÞ  Gðnþ1Þ ðK  aðt þ xÞÞ dFðxÞ dF ðnÞ ðtÞ ;

GðnÞ ðK  atÞdF ðnÞ ðtÞ:

ð6:37Þ

0

Finally, if the total damage increases exponentially, i.e., ZðtÞ ¼

NðtÞ X

Wn exp½aðt  Sn Þ;

ð6:38Þ

n¼1

then by arguments similar to those of Sect. 6.3.3, when FðtÞ ¼ 1  ekt ; 9 8 = < Zt U ðt; sÞ ¼ exp k ½1  G ðseau Þdu ; ; : 0

kxðeat  1Þ ; EfZðtÞg ¼ 2a 1 3 Z k 1  G ðsuÞ 5 U ð1; sÞ ¼ exp4 du : a u

ð6:39Þ

1

6.4 Replacement Models Suppose that shocks occur in a renewal process with distribution FðtÞ and each causes a random amount of damage with distribution GðxÞ to a unit. These damages are additive, and the unit fails when the total damage has exceeded a failure level K: When the failure during actual operation may be costly or dangerous, it is of great importance to avoid such terrible situation. It would be wise to exchange the unit before failure at a lower cost. If we have no information on the condition of the unit, its maintenance should be done at a planned time. On the other hand, if we could get the number of shocks up to now and the amount of damage at shock times, its maintenance should be done at a pre-specified number of shocks or at a damage level. A variety of replacement models subjected to shocks were summarized [2]. This section summarizes the replacement policies for cumulative damage models using the theories of cumulative processes: In Sect. 6.4.1, the unit is replaced before failure at a planned time T; at a shock number N; or a damage level Z; whichever occurs first. Introducing the respective replacement costs for T; N and Z; we obtain the expected cost rate CðT; N; ZÞ: In Sect. 6.4.2, we derive analytically optimum T ; N and Z that minimize the expected cost rates for three policies. Furthermore, we discuss optimum N and Z that minimize the expected

6.4 Replacement Models

171

cost rate CðN; ZÞ and give a numerical example. Finally, we consider the periodic replacement with minimal repair at failures for a cumulative damage model and obtain the expected cost rate.

6.4.1 Three Replacement Policies As per the preventive replacement policies, the unit is replaced before failure at time T ð0\T  1Þ; at shock number N ðN ¼ 1; 2; . . .Þ; or at damage level Z ð0  Z  KÞ; whichever occurs first. In addition, it is assumed that the unit is replaced at K or Z without considering to replace it at N when the total damage has exceeded K or Z at shock N: The probability that the unit is replaced at time T is PT ¼

N 1 h i X F ðnÞ ðTÞ  F ðnþ1Þ ðTÞ GðnÞ ðZÞ; n¼0

the probability that it is replaced at shock N is PN ¼ F ðNÞ ðTÞGðNÞ ðZÞ; the probability that it is replaced at damage Z is PZ ¼

N 1 X

F

ðnþ1Þ

ðTÞ

n¼0

ZZ

½GðK  xÞ  GðZ  xÞdGðnÞ ðxÞ;

0

and the probability that it is replaced at a failure level K is ZZ N 1 X ðnþ1Þ F ðTÞ GðK  xÞdGðnÞ ðxÞ; PK ¼ n¼0

0

where note that PT þ PN þ PZ þ PK ¼ 1 (Problem 6.9). Thus, the mean time to replacement is ZT N1 h i X ðnÞ ðnþ1Þ ðnÞ ðNÞ F ðTÞ  F ðTÞ G ðZÞ þ G ðZÞ t dF ðNÞ ðtÞ EfYg ¼ T n¼0

þ

þ

T N 1 Z X n¼0 0 T N1 XZ n¼0

¼

N 1 X n¼0

t dF ðnþ1Þ ðtÞ

ZZ

0

½GðK  xÞ  GðZ  xÞdGðnÞ ðxÞ

0

t dF ðnþ1Þ ðtÞ

0

GðnÞ ðZÞ

ZT h 0

ZZ

GðK  xÞdGðnÞ ðxÞ

0

i F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ dt:

172

6 Cumulative Processes

For the above replacement model, we introduce the following replacement costs: Costs cT ; cN ; cZ ; and cK are the respective replacement costs at time T; at shock N; at damage Z; and at level K; where cK is higher than the three costs cT ; cN and cZ : Then, the expected cost rate is, from (3.19) in Sect. 3.2, CðT; N; ZÞ ¼

cT PT þ cN PN þ cZ PZ þ cK PK EfYg i XN1 h ðnÞ ðnþ1Þ cK  ðcK  cT Þ F ðTÞ  F ðTÞ GðnÞ ðZÞ n¼0

 ðcK  cN ÞF ðnÞ ðTÞGðNÞ ðZÞ RZ PN1 ðnþ1Þ  ðcK  cZ Þ n¼0 F ðTÞ 0 ½GðK  xÞ  GðZ  xÞdGðnÞ ðxÞ ¼ : RT PN1 ðnÞ ðnÞ ðnþ1Þ ðtÞdt n¼0 G ðZÞ 0 ½F ðtÞ  F ð6:40Þ When the unit is replaced only after failure, the expected cost rate is, from (6.10), CðT; N; ZÞ ¼ C  lim T!1 N!1 Z!K

cK : l½1 þ MG ðKÞ

ð6:41Þ

The unit might be working as long as possible in the case where the replacement cost after failure is not so high. In this case, the unit should be replaced before failure at time T; shock number N; or damage level Z; whichever occurs last. Then, the probability that the unit is replaced at time T is PT ¼

1 h X

ih i F ðnÞ ðTÞ  F ðnþ1Þ ðTÞ GðnÞ ðKÞ  GðnÞ ðZÞ ;

n¼N

the probability that it is replaced at shock N is h i ðNÞ PN ¼ F ðTÞ GðNÞ ðKÞ  GðNÞ ðZÞ ; the probability that it is replaced at damage Z is PZ ¼

1 X

F

ðnþ1Þ

ðTÞ

n¼N

ZZ

½GðK  xÞ  GðZ  xÞdGðnÞ ðxÞ;

0

and the probability that it is replaced at a failure level K is PK ¼

1 h i X ðnÞ ðNÞ ðNÞ F ðnÞ ðTÞ  F ðnþ1Þ ðTÞ G ðKÞ þ F ðTÞG ðKÞ n¼N

þ

1 X n¼N

F

ðnþ1Þ

ðTÞ

ZZ 0

GðK  xÞdGðnÞ ðxÞ;

6.4 Replacement Models

173

where note that PT þ PN þ PZ þ PK ¼ 1 (Problem 6.9). Thus, the mean time to replacement is T

1 h ih i X F ðnÞ ðTÞ  F ðnþ1Þ ðTÞ GðnÞ ðKÞ  GðnÞ ðZÞ n¼N

þ

Z1

h i t dF ðNÞ ðtÞ GðNÞ ðKÞ  GðNÞ ðZÞ

T 1 Z X

1

þ

t dF

ðnþ1Þ

ðtÞ

n¼N

þ

T T Z 1 X n¼0

þ

n¼0

þ

0 1

N 1 Z X

½GðK  xÞ  GðZ  xÞdGðnÞ ðxÞ

0

h

t dF ðnþ1Þ ðtÞ GðnÞ ðKÞ  Gðnþ1Þ ðKÞ

i

h i t dF ðnþ1Þ ðtÞ GðnÞ ðKÞ  Gðnþ1Þ ðKÞ

T

1 1 Z X n¼N

ZZ

t dF ðnþ1Þ ðtÞ

T

¼ l½1 þ MG ðKÞ 

ZZ

GðK  xÞdGðnÞ ðxÞ

0 1 h X

ðnÞ

ðnÞ

G ðKÞ  G ðZÞ

n¼N

i Z1h

i F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ dt:

T

Therefore, by the similar method of obtaining (6.40), the expected cost rate is   P  ðnÞ ðnþ1Þ ðTÞ GðnÞ ðKÞ  GðnÞ ðZÞ cK  ðcK  cT Þ 1 n¼N F ðTÞ  F h i ðNÞ  ðcK  cN ÞF ðTÞ GðNÞ ðKÞ  GðNÞ ðZÞ RZ P ðnþ1Þ ðcK  cZ Þ 1 ðTÞ 0 ½GðK  xÞ  GðZ  xÞdGðnÞ ðxÞ n¼N F R 1  ðnÞ  : CðT;N;ZÞ ¼ P  ðnÞ ðnÞ ðnþ1Þ ðtÞ dt l½1þMG ðKÞ  1 n¼N G ðKÞ  G ðZÞ T F ðtÞ  F ð6:42Þ Clearly, when T ! 1; N ! 1, or Z ! K; the expected cost rate agrees with (6.41).

6.4.2 Optimum Policies We derive analytically the optimum planned time T ; shock number N ; and damage level Z that minimize the expected cost rates.

174

6 Cumulative Processes

6.4.2.1 Optimum T* Suppose that the unit is replaced only at time T or at failure, whichever occurs first. Then, the expected cost rate is, from (6.40), C1 ðTÞ  N!1 lim CðT; N; ZÞ Z!K

 ðnÞ  P ðnÞ ðnþ1Þ cK  ðcK  cT Þ 1 ðTÞ n¼0 G ðKÞ F ðTÞ  F ¼ : RT P1 ðnÞ ðnÞ ðnþ1Þ ðtÞdt n¼0 G ðKÞ 0 ½F ðtÞ  F

ð6:43Þ

Note that limT!0 C1 ðTÞ ¼ 1 and limT!1 C1 ðTÞ is given in (6.41). We seek an optimum time T that minimizes C1 ðTÞ in (6.43) for cK [ cT : Then, differentiating C1 ðTÞ with respect to T and setting it equal to zero, rðTÞ

1 X

ðnÞ

G ðKÞ

n¼0

¼

ZT h

1 h i i X F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ dt  GðnÞ ðKÞ  Gðnþ1Þ ðKÞ F ðnþ1Þ ðTÞ n¼0

0

cT ; cK  c T

ð6:44Þ

where rðTÞ is given in (6.20). It can be clearly seen that if rðTÞ increases strictly with T; then the left-hand side of (6.44) also increases strictly from 0 to lrð1Þ½1 þ MG ðKÞ  1 (Problem 6.10). Thus, if rð1Þ½1 þ MG ðKÞ [

cK ; lðcK  cT Þ

then there exists a finite and unique T that satisfies (6.44), and the resulting cost rate is C1 ðT Þ ¼ ðcK  cT ÞrðT Þ: Conversely, if rð1Þ½1 þ MG ðKÞ 

cK ; lðcK  cT Þ

then T ¼ 1; i.e., the unit is replaced only at failure, and the expected cost rate is given in (6.41). If a failure level K is distributed according to LðxÞ, as shown in Sect. 6.3.2, the expected cost rate becomes R1 P  ðnÞ ðnþ1Þ cK  ðcK  cT Þ 1 ðTÞ 0 GðnÞ ðxÞdLðxÞ n¼0 F ðTÞ  F : ð6:45Þ C1 ðTÞ ¼ R1 P1 R T ðnÞ ðnþ1Þ ðtÞdt ðnÞ ðxÞdLðxÞ ½ F ðtÞ  F G n¼0 0 0

6.4 Replacement Models

175

In particular, suppose that shocks occur in a nonhomogeneous Poisson process and K is distributed exponentially, i.e., LðxÞ ¼ 1  ebx : Then, replacing F ðnÞ ðtÞ  R1 F ðnþ1Þ ðtÞ with f½HðtÞn =n!geHðtÞ and setting that 0 GðnÞ ðxÞdLðxÞ ¼ ½G ðbÞn ;

C1 ðTÞ ¼

cK  ðcK  cT Þe½1G ðbÞHðTÞ ; RT ½1G ðbÞHðtÞ dt 0 e

ð6:46Þ

that corresponds to an age replacement policy by setting that FðtÞ ¼ e½1G ðbÞHðtÞ in (3.42). 6.4.2.2 Optimum N* Suppose that the unit is replaced only at shock N ðN ¼ 1; 2; . . .Þ or at failure, whichever occurs first. Then, the expected cost rate is, from (6.40), C2 ðNÞ  lim CðT; N; ZÞ T!1 Z! K

¼

cK  ðcK  cN ÞGðNÞ ðKÞ PN1 ðnÞ l n¼0 G ðKÞ

ðN ¼ 1; 2; . . .Þ:

ð6:47Þ

Forming the inequality C2 ðN þ 1Þ  C2 ðNÞ  0 to seek an optimum N that minimizes C2 ðNÞ for cK [ cN ; rNþ1

N1 X

GðnÞ ðKÞ  ½1  GðNÞ ðKÞ 

n¼0

cN cK  cN

ðN ¼ 1; 2; . . .Þ;

ð6:48Þ

where rNþ1 

GðNÞ ðKÞ  GðNþ1Þ ðKÞ : GðNÞ ðKÞ

If rN increases strictly with N; i.e., GðNþ1Þ ðxÞ=GðNÞ ðxÞ decreases strictly with N; then the left-hand side of (6.48) also increases strictly with N to r1 ½1 þ MG ðKÞ  1; where r1  limN!1 rN  1 (Problem 6.10). Thus, if r1 ½1 þ MG ðKÞ [ cK =ðcK  cN Þ; then there exists a finite and unique minimum N (1  N \1) that satisfies (6.48). In particular, when GðxÞ ¼ 1  ehx ; rNþ1 increases to 1: In this case, if hK [ cN =ðcK  cN Þ then there exists a finite and unique minimum N ð1  N \1Þ that satisfies (6.48). 6.4.2.3 Optimum Z* Suppose that the unit is replaced only at damage Z ð0  Z  KÞ or at failure, whichever occurs first. Then, the expected cost rate is, from (6.40),

176

6 Cumulative Processes

C3 ðZÞ  lim CðT; N; ZÞ T!1

h i RZ cK  ðcK  cZ Þ GðKÞ  0 GðK  xÞdMG ðxÞ ¼ : l½1 þ MG ðZÞ N!1

ð6:49Þ

We seek an optimum level Z that minimizes C3 ðZÞ for cK [ cZ : Differentiating C3 ðZÞ with respect to Z and setting it equal to zero, ZK cZ ½1 þ MG ðK  xÞdGðxÞ ¼ ; ð6:50Þ cK  cZ KZ

whose left-hand side increases strictly from 0 to MG ðKÞ: Thus, if MG ðKÞ [ cZ =ðcK  cZ Þ; then there exists a finite and unique Z ð0\ Z \KÞ that satisfies (6.50), and its resulting cost rate is C3 ðZ Þ ¼

ðcK  cZ ÞGðK  Z Þ : l

In particular, when GðxÞ ¼ 1  ehx ; (6.50) is simplified as hZ ehðKZÞ ¼

cZ : c K  cZ

6.4.2.4 Optimum N* and Z* We can consider optimum policies ðN ; Z Þ and ðT ; Z Þ that minimize the expected cost rates C4 ðN; ZÞ and C5 ðT; ZÞ (Problem 6.11) from CðT; N; ZÞ in (6.40), respectively. In particular, we derive optimum policies N and Z that minimize C4 ðN; ZÞ : The expected cost rate is, from (6.40), C4 ðN; ZÞ  lim CðT; N; ZÞ T!1

cZ þ ðcN  cZ ÞGðNÞ ðZÞ PN1 R Z ðnÞ þ ðcK  cZ Þ n¼0 0 ½1  GðK  xÞdG ðxÞ ¼ : PN1 l n¼0 GðnÞ ðZÞ Differentiating C4 ðN; ZÞ with respect to Z and setting it equal to zero, " # N1 gðNÞ ðZÞ X ðnÞ ðNÞ ðcN  cZ Þ PN1 G ðZÞ  G ðZÞ ðnÞ n¼1 g ðZÞ n¼0 Z N1 Z X ½GðK  xÞ  GðK  ZÞdGðnÞ ðxÞ ¼ cZ ; þ ðcK  cZ Þ n¼0

0

ð6:51Þ

ð6:52Þ

6.4 Replacement Models

177

where gðnÞ ðxÞ  dGðnÞ ðxÞ=dx: From the inequality C4 ðN þ 1; ZÞ  C4 ðN; ZÞ  0; " # N1 GðNÞ ðZÞ  GðNþ1Þ ðZÞ X ðnÞ ðNÞ  ðcN  cZ Þ G ðZÞ þ G ðZÞ GðNÞ ðZÞ n¼0 2 PN1 ðnÞ ZZ G ðZÞ ½1  GðK  xÞdGðNÞ ðxÞ þ ðcK  cZ Þ4 n¼0ðNÞ G ðZÞ 0 3 Z Z N1 X ½1  GðK  xÞdGðnÞ ðxÞ5  cZ : ð6:53Þ  n¼0

0

Substituting (6.52) for (6.53), " # GðNÞ ðZÞ  GðNþ1Þ ðZÞ gðNÞ ðZÞ þ PN1  ðcN  cZ Þ ðnÞ GðNÞ ðZÞ n¼1 g ðZÞ 1  ðcK  cZ Þ ðNÞ G ðZÞ

ZZ

½GðK  xÞ  GðK  ZÞdGðNÞ ðxÞ  0:

ð6:54Þ

0

Thus, if cN  cZ then there exists no finite N that satisfies (6.54) for any Z: It is assumed that cK [ cZ [ cN and GðxÞ ¼ 1  ehx : Then, (6.52) and (6.54) are simplified as, respectively, ( ) N1 X 1 1 ðhZÞN1 =ðN  1Þ! X ðhZÞj hZ X ðhZÞj hZ e e   ðcZ  cN Þ PN2 j j! j! j¼N n¼0 j¼n j¼0 ðhZÞ =j! þ ðcK  cZ Þ

N1 X 1 X ðhZÞj hK e ¼ cZ ; j! n¼0 j¼nþ1

ð6:55Þ

and (

ðhZÞN =N! ðhZÞN1 =ðN  1Þ! ðcZ  cN Þ P1 þ PN2 j j j¼N ðhZÞ =j! j¼0 ðhZÞ =j! P1 j hK j¼Nþ1 ðhZÞ =j!e  ðcK  cZ Þ P1  0: j hZ j¼N ðhZÞ =j!e

)

ð6:56Þ

P j hZ to 1, the left-hand side of Because ½ðhZÞn =n!= 1 j¼n ½ðhZÞ =j! increases from e (6.56) goes to cZ  cN as N ! 1: Thus, there exists a finite N ð1  N \1Þ that satisfies (6.56). Table 6.1 presents the optimum shock number N and damage level Z ; and expected cost rate C4 ðN ; Z Þ for K ¼ 10; 20; 30; 40; 50 when h ¼ 0:5; l ¼ 1; cN ¼ 1; cZ ¼ 10 and cK ¼ 100: In case of N ¼ 1; C4 ð1; ZÞ decreases with Z and

178

6 Cumulative Processes

Table 6.1 Optimum N and Z ; and expected cost rate C4 ðN ; Z Þ when h ¼ 0:5; l ¼ 1; cN ¼ 1; cZ ¼ 10 and cK ¼ 100 K 10 20 30 40 50 N Z C4 ðN ; Z Þ

1 1 1.606

3 18.2 0.421

6 26.7 0.204

9 36.2 0.128

12 45.8 0.091

C4 ð1; 1Þ  1:606: Both optimum N and Z increase with K for K  20: Note that the ratio of Z to K are approximately 90%; i.e., Z =K  0:90; and N ð1=hÞ=K increase slowly from 0.20 to 0.48 with K for K  10: Furthermore, by the regression analysis, we have the regression line N ¼ 0:28K  2:2 for K  10 (Problem 6.12).

6.4.2.5 Periodic Replacement It is assumed in Sect. 6.4.1 that the total damage due to shocks is additive when it has not exceeded a failure level K; however, it is not additive at any shock after it has exceeded K (Fig. 6.5). In this case, the minimal maintenance is done at each shock and the damage level remains in K: Suppose that the unit is replaced at periodic times kT ðk ¼ 1; 2; . . .Þ for a planned time T ð0\T  1Þ (Problem 6.14). Then, the expected number of minimal maintenances, i.e., the expected number of shocks in the case where the total damage remains in K when it has reached K; is 1 h X

ðnÞ

G ðKÞ  G

ðnþ1Þ

n¼0

¼

ZT h 1 iX i ðKÞ ðk þ 1Þ F ðkÞ ðT  tÞ  F ðkþ1Þ ðT  tÞ dF ðnþ1Þ ðtÞ k¼0

1 X

0

½1  GðnÞ ðKÞF ðnÞ ðTÞ:

n¼1

Let cM be the cost of minimal maintenance. Then, by the similar method of obtaining (6.40), the expected cost rate is ( ) 1 X 1 ðnÞ ðnÞ F ðTÞ½1  G ðKÞ ; ð6:57Þ CðTÞ ¼ c T þ cM T n¼1 where cT is the replacement cost at time T: We seek an optimum time T that minimizes CðTÞ: Differentiating CðTÞ with respect to T and setting it equal to zero, 1 X n¼1

½1  GðnÞ ðKÞ½Tf ðnÞ ðTÞ  F ðnÞ ðTÞ ¼

cT ; cM

6.4 Replacement Models

179

Fig. 6.5 Process for PM at time T

i.e., 1 h X

1  GðnÞ ðKÞ

i ZT

n¼1

td f ðnÞ ðtÞ ¼

cT ; cM

ð6:58Þ

0

where f ðnÞ ðtÞ  dF ðnÞ ðtÞ=dt: Thus, if 1 X ½1  GðnÞ ðKÞT½f ðnÞ ðTÞ0 [ 0; n¼1

and 1 X

ðnÞ

½1  G ðKÞ

n¼1

Z1

td f ðnÞ ðtÞ [

cT ; cM

0

then there exists a finite and unique T (0\T \1) that satisfies (6.58). In particular, when FðtÞ ¼ 1  ekt ; (6.58) is 1 X ðkTÞn n¼1

n!

ekT

n X cT ½GðkÞ ðKÞ  GðnÞ ðKÞ ¼ ; c M k¼1

ð6:59Þ

whose left-hand side increases strictly from 0 to MG ðKÞ: Therefore, if MG ðKÞ [ cT =cM then there exists a finite and unique T ð0\T \1Þ that satisfies (6.59).

6.5 Problems 6 6.1 Search for examples of cumulative processes on your daily life and investigate some examples [2]. 6.2 Derive (6.5).

180

6 Cumulative Processes

6.3 In Example 6.3, compute the total damage when a ¼ 0:95; 0.99 and the probability that the unit with K ¼ 15; 20 fails in 30 days. 6.4 Give some examples of independent damage models [2, p. 21]. 6.5 Prove (v) and (vi) in Example 6.4. P k 6.6 Prove in Example 6.5 that ½ðhKÞn =n!= 1 k¼n ½ðhKÞ =k! increases with n from ehK to 1. 6.7 Prove in (6.25) that if rðtÞ increases strictly and hðtÞ increases, or rðtÞ increases and hðtÞ increases strictly, the left-hand side of (6.25) increases strictly from 0 to (6.26). 6.8 In Example 6.7, consider the optimization problem in which the system is replaced when N ðN ¼ 1; 2; . . .; nÞ units have failed as the preventive replacement policy [2, p. 62]. 6.9 Prove that PT þ PN þ PZ þ PK ¼ 1 in Sect. 6.4.1 6.10 Prove in Sect. 6.4.2 that if rðTÞ and rN increase strictly with T and N; respectively, then (6.44) and (6.48) increase strictly. 6.11 Derive in Sect. 6.4.2.4 optimum T and Z that minimize C5 ðT; ZÞ  limN!1 CðT; N; ZÞ when cT ¼ cZ and shocks occur in a Poisson process [2, p. 53]. 6.12 In Table 6.1, derive the regression line N ¼ 0:28K  2:2: 6.13 In Sect. 6.4.2.5, obtain the expected cost CðT; NÞ where the unit is replaced at time T or shock number N; whichever occurs first.

References 1. Cox DR (1962) Renewal theory. Methuen, London 2. Nakagawa T (2007) Shock and damage models in reliability theory. Springer, London 3. Abdel-Hameed M (1984) Life distribution properties of devices subject to a pure jump damage process. J Appl Probab 21:816–825 4. Grandell J (1976) Doubly stochastic poisson process. Lecture notes in mathematics 529. Springer, New York 5. Finkelstein MS, Zarudnij VI (2001) A shock process with a non-cumulative damage. Relib Eng Syst Saf 71:103–107 6. Gaver DP Jr (1963) Random hazard in reliability problems. Technometrics 5:211–226 7. Barlow RE, Proschan F (1975) Statistical theory of reliability and life testing. Halt, Rinehart & Winston, New York 8. Barlow RE, Proschan F (1965) Mathematical theory of reliability. Wiley, New York 9. Esary JD, Marshall AW, Proschan F (1973) Shock models and wear processes. Ann Probab 1:627–649 10. Shaked M (1984) Wear and damage processes from shock models in reliability theory. In: Abdel-Hameed M, Çinlar E, Quinn J (eds) Reliability theory and models. Academic, Orlando 11. Abdel-Hameed M, Proschan F (1975) Shock models with underlying birth process. J Appl Probab 12:18–28 12. Block HW, Savits TH (1978) Shock models with NBUE survival. J Appl Probab 15:621–628

References

181

13. Abdel-Hameed M (1984) Life distribution properties of devices subject to a Lévy wear process. Math Oper Res 9:606–614 14. Nikulim MS, Limnios N, Balakrishnan N, Kahle W, Huber-Carol C (2010) Advances in degradation modeling. Birhäuser, Boston 15. Takács L (1960) Stochastic processes. Wiley, New York 16. Ross SM (1983) Stochastic processes. Wiley, New York

Chapter 7

Brownian Motion and Lévy Processes

Wiener (1923) and Lévy (1939) mathematically gave the theoretical foundation and construction of the Brownian Motion based on the phenomenon of small particles observed by Brown (1827) and Einstein (1905). This is called the Brownian Motion or Wiener process, and is an example of a Markov process with continuous time and state space. This is now one of the most useful stochastic processes in applied sciences such as physics, economics, communication theory, biology, management science, and mathematical statistics, and recently, it is also used to make several valuable models in finance. Section 7.1 describes a brief outline of a Brownian Motion process and considers its application to a continuous damage level. The total damage might be estimated only at periodic times or with time roughly. Section 7.2 takes up three damage models and compares numerically their optimum replacement policies. Section 7.3 introduces a Lévy process and gives two damage models to show how to apply it to reliability models.

7.1 Brownian Motion and Wiener Processes We define a Brownian Motion process or a Wiener process [1, p. 184, 2, p. 340]: Definition 7.1 A stochastic process fXðtÞg is said to be a Brownian Motion process if (i) Xð0Þ ¼ 0; (ii) the process fXðtÞg has stationary and independent increment, (iii) the process fXðtÞg is normally distributed with mean 0 and variance r2 t for any t [ 0: For any t; u [ 0; the increments Xðt þ uÞ  XðuÞ and XðtÞ  XðuÞ for t [ u have a normal density with mean 0, and variance r2 t and r2 ðt  uÞ; respectively, i.e., the density f ðx; tÞ of PrfXðt þ uÞ  XðuÞ  xg ¼ Prf XðtÞ  xg is

T. Nakagawa, Stochastic Processes, Springer Series in Reliability Engineering, DOI: 10.1007/978-0-85729-274-2_7, Ó Springer-Verlag London Limited 2011

183

184

7 Brownian Motion and Lévy Processes

1 2 2 f ðx; tÞ ¼ pffiffiffiffiffiffiffi ex =ð2r tÞ ; 2ptr

ð7:1Þ

and its Laplace transform is (Problem 7.1) Z1

esx f ðx; tÞdx ¼ es

2 2

r t=2

ð7:2Þ

:

1

When r ¼ 1; BðtÞ  XðtÞ=r is called a standard Brownian Motion because VfXðtÞ=rg ¼ t: Thus, any Brownian Motion can convert to the standard process. For any 0  t0 \t1 \    \tn \t; from the properties of independent and stationary increments, PrfXðtÞ  xjXðt0 Þ ¼ x0 ; . . .; Xðtn Þ ¼ xn g ¼ PrfXðtÞ  xjXðtn Þ ¼ xn g ¼ PrfXðtÞ  Xðtn Þ  x  xn g; that states that the process has a Markov property, and its distribution function is, from (7.1), 1 PrfXðtÞ  Xðtn Þ  x  xn g ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2pðt  tn Þr

xxn Z

eu

2

=½2r2 ðttn Þ

du;

ð7:3Þ

1

i.e., XðtÞ  Xðtn Þ has a normal density with mean 0 and variance r2 ðt  tn Þ for any t  tn ; that does not depend on tn : When ZðtÞ ¼ lt þ rBðtÞ; where BðtÞ is a standard Brownian Motion, ZðtÞ is called a Brownian Motion with drift parameter l and variance r2 [1, 2]. Example 7.1 (Continuous damage model) The total damage or wear ZðtÞ usually increases swayingly with time t from Zð0Þ ¼ 0; and the unit fails when ZðtÞ has exceeded a failure level K (Fig. 7.1). It is assumed that ZðtÞ ¼ AðtÞt þ rBðtÞ for AðtÞ  0 and BðtÞ is a standard Brownian Motion. Then, the reliability RðtÞ at time t; in other words, the probability that the total damage has not exceeded K in time t; is RðtÞ ¼ PrfZðtÞ  Kg ¼ PrfAðtÞt þ rBðtÞ  Kg; R1 and the MTTF is given by 0 RðtÞdt:

ð7:4Þ

(1) When AðtÞ  l (constant), K ¼ k (constant), and BðtÞ is normally distributed with mean 0 and variance t;   k  lt pffi ; RðtÞ ¼ PrfrBðtÞ  k  ltg ¼ U ð7:5Þ r t where UðxÞ is the standard normal distribution with mean 0 and variance 1, pffiffiffiffiffiffi R x 2 i.e., UðxÞ  ð1= 2pÞ 1 eu =2 du (Problem 7.2).

7.1 Brownian Motion and Wiener Processes

185

Fig. 7.1 Process for a continuous damage model

(2) When BðtÞ  0; K ¼ k and AðtÞ is normally distributed with mean l and variance r2 =t;   k  lt pffi ; ð7:6Þ RðtÞ ¼ PrfAðtÞ  k=tg ¼ U r t that becomes equal to (7.5). (3) When AðtÞ  l; BðtÞ  0 and K is normally distributed with mean k and variance r2 ;   k  lt RðtÞ ¼ Prflt  Kg ¼ U : ð7:7Þ r When K is normally distributed with mean k and variance r2 t; RðtÞ is equal to (7.5) and (7.6). pffiffiffiffiffi Replacing a  r= lk and b  k=l in (7.5) or (7.6), " rffiffiffi rffiffiffi!# 1 b t ;  RðtÞ ¼ U a t b which is called the Birnbaum–Saunders distribution [3]. This is widely applied to fatigue failure for material strength subject to stresses [4, 5]. When ZðtÞ ¼ AðtÞt þ rBðtÞ; if AðtÞ; BðtÞ; and K are deterministic, i.e., AðtÞ  l; BðtÞ ¼ 1; and K ¼ k; then the unit fails at time ðk  rÞ=l: By fitting appropriate probability distributions to AðtÞ; BðtÞ; and K, and estimating their parameters for actual units, the function ZðtÞ can be used as a continuous damage or wear function in cumulative damage models [6, p. 26]. Furthermore, the process defined by XðtÞ  eZðtÞ for t  0 is called a geometric Brownian Motion, and its state space is the interval ½0; 1Þ [1, p. 193]. The process

186

7 Brownian Motion and Lévy Processes

is applied to models of share prices, interest rates, and stock options in finance. When ZðtÞ ¼ lt þ BðtÞ defined in (7.1), from (7.2),    n o 1 E esZðtÞ ¼ exp st l þ sr2 ; 2 and hence, setting that s ¼ 1; n o 2 EfXðtÞg ¼ E eZðtÞ ¼ eðlþr =2Þt :

ð7:8Þ

Example 7.2 [7, p. 379]. Suppose that a unit begins to operate at time 0 and a failed unit undergoes repair, where the repair time is negligible. It is assumed that YðtÞ  c1 ½XðtÞ  1 is the total maintenance cost due to failures over the time interval ½0; t and is given by n o h i 2 cðtÞ  EfYðtÞg ¼ c1 E eZðtÞ  1 ¼ c1 eðlþr =2Þt  1 : Suppose that the unit is replaced at time T: The expected cost rate is, from Sect. 3.3.2.2,

CðTÞ 

cðTÞ þ c2 ¼ T

h i 2 c1 eðlþr =2ÞT  1 þ c2 T

;

ð7:9Þ

where c2 is the replacement cost at time T: Clearly, lim CðTÞ ¼ lim C1 ðTÞ ¼ 1:

T!0

T!1

Thus, there exists an optimum T  that minimizes (7.9). Differentiating CðTÞ with respect to T and setting it equal to zero, 

 h i c r2 2 2 2 lþ Teðlþr =2ÞT  eðlþr =2ÞT  1 ¼ ; 2 c1

ð7:10Þ

whose left-hand side increases from 0 to 1: Therefore, there exists a unique optimum T  (0\T  \1) that satisfies (7.10), and the resulting cost rate is (Problems 7.3 and 7.4)   r2 ðlþr2 =2ÞT  : ð7:11Þ e CðT  Þ ¼ c1 l þ 2

7.2 Three Replacements of Cumulative Damage Models

187

7.2 Three Replacements of Cumulative Damage Models We consider three cumulative damage models: A unit is subjected to shocks and suffers some damage due to it (see Sect. 7.2.1.1); the amount of damage due to shocks is measured only at periodic times (see Sect. 7.2.1.2), and the total damage increases linearly with time (see Sect. 7.2.1.3). The total damage is additive and the unit fails when the total damage has exceeded a failure level K (0\K\1) for three models. Models 2 and 3 would be practically used as the approximation ones of Model 1 and Models 1 and 2, respectively. As per the replacement policy, the unit is replaced at a planned time T: The expected cost rates of each model are obtained, and optimum policies that minimize them are derived analytically. Furthermore, optimum replacement times of each model are compared numerically when each model has the same mean amount of damage at any periodic times.

7.2.1 Three Damage Models We consider the following three damage models for an operating unit [8].

7.2.1.1 Standard Model The unit is subjected to shocks and suffers some damage at each shock. Let random variables Xk (k ¼ 1; 2; . . .) denote a sequence of interarrival times between successive shocks, and random variables Wk (k ¼ 1; 2; . . .) denote the damage produced by the kth shock, where W0 ¼ 0 (Fig. 6.1). In addition, let NðtÞ denote the random variable that is the total number of shocks up to time t: Then, define a random variable ZðtÞ ¼

NðtÞ X

Wk

ðNðtÞ ¼ 0; 1; 2; . . .Þ;

ð7:12Þ

k¼0

that represents the total damage at time t: It is assumed that FðtÞ  PrfXk  tg with finite mean l and variance r2F and GðxÞ  PrfWk  xg with finite mean x and variance r2G for t; x  0: Then, from (6.3), (6.5) and (6.8), PrfZðtÞ  xg ¼

1 X

GðkÞ ðxÞ½F ðkÞ ðtÞ  F ðkþ1Þ ðtÞ;

k¼0

EfZðtÞg ¼ xMF ðtÞ;

  x2 t r2F r2G VfZðtÞg þ ; l l2 x 2

ð7:13Þ

188

7 Brownian Motion and Lévy Processes

where /ðkÞ ðtÞ (k ¼ 1; 2; . . .) denotes the k-fold Stieltjes convolution of any funcP ðkÞ tion /ðtÞ with itself and /ð0Þ ðtÞ  1 for t  0; and M/ ðtÞ  1 k¼1 / ðtÞ: Suppose that the unit fails when the total damage has exceeded a failure level K: As per the replacement policy, the unit is replaced at time T or at failure, whichever occurs first. Letting cK be the replacement cost at failure and cT (cT \cK ) be the replacement cost at time T; the expected cost rate is, from (6.23) and (6.43), P ðkÞ ðkÞ ðkþ1Þ cK  ðcK  cT Þ 1 ðTÞ k¼0 G ðKÞ½F ðTÞ  F : ð7:14Þ C1 ðTÞ ¼ RT P1 ðkÞ ðkÞ ðkþ1Þ ðtÞdt k¼0 G ðKÞ 0 ½F ðtÞ  F We find an optimum time T1 that minimizes C1 ðTÞ: Let f ðtÞ be a density function of FðtÞ and P1 ðkþ1Þ f ðTÞ½GðkÞ ðKÞ  Gðkþ1Þ ðKÞ : Q1 ðTÞ  Pk¼0 1 ðkÞ ðkÞ ðkþ1Þ ðTÞ k¼0 G ðKÞ½F ðTÞ  F By the method similar to Example 6.6, if Q1 ðTÞ increases strictly with T and Q1 ð1Þ½1 þ MG ðKÞ [ cK =½lðcK  cT Þ; then there exists a finite and unique T1 (0\T1 \1) that satisfies (Problem 7.5) Q1 ðTÞ

1 X k¼0



1 X

ðkÞ

G ðKÞ

ZT

½F ðkÞ ðtÞ  F ðkþ1Þ ðtÞdt

0

F ðkþ1Þ ðTÞ½GðkÞ ðKÞ  Gðkþ1Þ ðKÞ ¼

k¼0

cT : cK  cT

ð7:15Þ

7.2.1.2 Periodic Model Each amount Wn (n ¼ 1; 2; . . .) of damage is measured only at periodic times nT0 (n ¼ 1; 2; . . .) for given T0 (0\T0 \1) and has an identical distribution GT ðxÞ  PrfWn  xg with mean xT and variance r2T : Then, ðnÞ

PrfZðnT0 Þ  xg ¼ GT ðxÞ; EfZðnT0 Þg ¼ nxT ;

VfZðnT0 Þg ¼ nr2T :

ð7:16Þ

Suppose that the unit is replaced at time NT0 or at failure, whichever occurs first. Then, the expected cost rate is, from (6.47), ðNÞ

C2 ðNÞ ¼

cK  ðcK  cT ÞGT ðKÞ PN1 ðnÞ GT ðKÞ T0 n¼0

ðN ¼ 1; 2; . . .Þ:

ð7:17Þ

7.2 Three Replacements of Cumulative Damage Models

189

We find an optimum number N  that minimizes C2 ðNÞ: From the inequality C2 ðN þ 1Þ  C2 ðNÞ  0; Q2 ðN þ 1Þ

N1 X

ðnÞ

ðNÞ

GT ðKÞ  ½1  GT ðKÞ 

n¼0

cT ; c K  cT

ð7:18Þ

where ðN1Þ

Q2 ðNÞ 

GT

ðNÞ

ðKÞ  GT ðKÞ ðN1Þ

GT

ðKÞ

:

Thus, if Q2 ðNÞ increases strictly with N and Q2 ð1Þ½1 þ MGT ðKÞ [ cK =ðcK  cT Þ; then there exists a finite and unique minimum N  (1  N  \1) that satisfies (7.18) (Problem 7.6).

7.2.1.3 Continuous Model The total damage ZðtÞ usually increases with time t: Suppose that ZðtÞ ¼ AðtÞt where AðtÞ has a distribution LA ðtÞ with mean a and variance r2A =t: Then, PrfZðtÞ  xg ¼ PrfAðtÞ  x=tg ¼ LA ðx=tÞ; EfZðtÞg ¼ at;

VfZðtÞg ¼ r2A t:

ð7:19Þ

Suppose that the unit is replaced at time T or at failure, whichever occurs first. Then, the expected cost rate is C3 ðTÞ ¼

cK  ðcK  cT ÞLA ðK=TÞ : RT 0 LA ðK=tÞdt

ð7:20Þ

We find an optimum time T3 that minimizes C3 ðTÞ: Let lA ðtÞ be a density function of LA ðtÞ and rA ðtÞ be the failure rate of LA ðtÞ; i.e., rA ðtÞ  lA ðtÞ=LA ðtÞ: Then, if rA ðtÞ increases strictly, then there exists a finite and unique T3 ð0\T3 \1Þ that satisfies (Problem 7.7) rA ðK=TÞ

ZT

LA ðK=tÞdt  ½1  LA ðK=TÞ ¼

cT : cK  c T

ð7:21Þ

0

7.2.2 Numerical Examples We compute optimum policies numerically for the three models: It is assumed that FðtÞ ¼ 1  ekt and GðxÞ ¼ 1  ex=x for Model 1, GT ðxÞ ¼ 1  ex=xT for Model 2, and AðtÞ has a normal distribution NðxA ; r2A =tÞ for Model 3.

190

7 Brownian Motion and Lévy Processes

In addition, we set the mean and variance of the total damage at any time nT0 as equal for all models, i.e., EfZðnT0 Þg ¼ kxnT0 ¼ nxT ¼ xA nT0 ;

VfZðnT0 Þg ¼ 2kx2 nT0 ¼ nx2T ¼ r2A nT0 :

Hence, k¼

2 ; T0

x¼

xT ; 2

xA ¼

xT ; T0

r2A ¼

x2T : T0

When T0 ¼ 1 and xT ¼ 1; k ¼ 2; x ¼ 1=2; xA ¼ 1 and r2A ¼ 1: Optimum policies are, for Model 1, from (7.15), P1

T j j j Z 1 X ð2KÞj 2K X j¼0 ½ð2TÞ ð2KÞ =j!j! e P1 Pj j k j! k¼0 j¼0 ½ð2KÞ =j! k¼0 ½ð2TÞ =k! j¼0 0

2



1 X ð2TÞj j¼1

j!

e2T

j1 X ð2KÞk

k!

k¼0

e2K ¼

ð2tÞk 2t e dt k!

1 ; cK =cT  1

ð7:22Þ

for Model 2, from (7.18), N 1 X 1 N1 k X K N =N! K j K X N K 1 P1 e e  ;  k =k!Þ ðK j! k! =c c K T 1 k¼N k¼0 j¼k k¼0

ð7:23Þ

and for Model 3, from (7.21), pffiffiffiffi pffiffiffiffi ZT      pffiffiffiffi ðK=T þ 1Þ/ðK= T  T Þ K pffi K pffiffiffiffi pffiffiffiffi pffiffiffiffi U pffi  t dt  1  U pffiffiffiffi  T t T 2 T UðK= T  T Þ 0

1 ¼ ; cK =cT  1

ð7:24Þ

pffiffiffiffiffiffi R x pffiffiffiffiffiffi 2 2 where /ðxÞ  ð1= 2pÞex =2 and UðxÞ  ð1= 2pÞ 1 eu =2 du: Table 7.1 presents optimum T1 and C1 ðT1 Þ for cK =cT ¼ 5; 10; 20; 50 and K ¼ 5; 10; 15; 20: This indicates that T1 decreases when cK =cT and 1=K increase. It is of interest that C1 ðT1 Þ have their minimum values at K ¼ 15 for cK =cT  10: Table 7.2 presents optimum N  T0 and C2 ðN  T0 Þ for the same parameters cK =cT and K as Table 7.1. This shows the similar tendencies to Table 7.1, however, C2 ðN  Þ decreases with K and cT =cK : Table 7.3 presents optimum T3 and C3 ðT3 Þ: This shows the same tendencies as Table 7.2 for both T3 and C3 ðT3 Þ: These tables indicate that T1 is equal to N  T0 and N  T0 [ T3 for large cK =cT and K; and T1 =T3 changes from 1:0 to 2:2 in this case. It is natural that T1 ; N  T0 and T3 decrease when cK =cT and 1=K increase. The values of T3 are too small compared with N  T0 : If T0 and 1=k would be small, T3 would approach N  T0 :

7.2 Three Replacements of Cumulative Damage Models

191

Table 7.1 Optimum T1 and C1 ðT1 Þ when T0 ¼ 1 and lT0 ¼ 1 cK =cT

K 5

5 10 20 50

10

15

20

T1

CðT1 Þ

T1

CðT1 Þ

T1

CðT1 Þ

T1

CðT1 Þ

2.9 2.2 1.7 1.3

0.520 0.668 0.840 1.127

5.9 5.0 4.3 3.6

0.212 0.247 0.284 0.336

9.3 8.2 7.3 6.4

0.126 0.141 0.157 0.178

13.3 12.2 11.5 11.0

0.110 0.145 0.207 0.385

Table 7.2 Optimum N  T0 and C2 ðN  Þ when T0 ¼ 1 and lT0 ¼ 1 cK =cT

K 5

10 

5 10 20 50





15 



20 

N T0

C2 ðN Þ

N T0

C2 ðN Þ

N T0

C2 ðN Þ

N  T0

C2 ðN  Þ

3.0 2.0 2.0 1.0

0.508 0.684 0.887 1.330

6.0 5.0 4.0 4.0

0.213 0.253 0.299 0.377

9.0 8.0 7.0 6.0

0.128 0.146 0.164 0.190

13.0 12.0 10.0 9.0

0.089 0.100 0.110 0.122

Table 7.3 Optimum T3 and C3 ðT3 Þ when T0 ¼ 1 and lT0 ¼ 1 cK =cT

K 5

5 10 20 50

10

15

20

T3

CðT3 Þ

T3

CðT3 Þ

T3

CðT3 Þ

T3

CðT3 Þ

2.5 2.0 1.8 1.6

0.483 0.559 0.626 0.704

6.0 5.1 4.6 4.1

0.198 0.219 0.237 0.260

9.2 8.7 7.8 7.1

0.119 0.130 0.139 0.149

13.1 12.0 11.3 10.6

0.084 0.090 0.095 0.102

7.3 Lévy Process We briefly explain a Lévy process appearing in stochastic processes [9]. Definition 7.2 A stochastic process fXðtÞg is said to be a Lévy process if (i) Xð0Þ ¼ 0; (ii) the increments Xðt1 Þ  Xðt0 Þ; Xðt2 Þ  Xðt1 Þ; . . .; Xðtn Þ  Xðtn1 Þ are independent for all 0  t0 \t1 \    \tn ; (iii) the distribution of increment Xðtk Þ  Xðtk1 Þ depends only the difference tk  tk1 : A Lévy process is a stochastic process with independent and stationary increments. For the strict definition of a Lévy process, we have to add two regularity conditions, however, we omit them to exceed mathematically the limit of the book

192

7 Brownian Motion and Lévy Processes

[10, 11]. From this definition, the increment XðtÞ  XðuÞ is independent of the process fXðxÞ; 0  x  ug and has the same distribution of Xðt  uÞ for t  u  0: Special cases of Lévy processes are the Brownian Motion with continuous increment and the Poisson process with jump increment. In other words, the Poisson process is the jump Lévy process and the Brownian Motion is the continuous Lévy process. The Lévy process has been applied to various fields such as physics, finance, and statistics [11]. However, there have been few papers treated with reliability models using Lévy processes: When damage and wear models form a Lévy process, the damage models have been discussed analytically [12–14]. Example 7.3 (Continuous damage model) It is assumed in Sect. 7.2.1.3 that pffi ZðtÞ ¼ xt þ Bt ; where Bt has an exponential distribution 1  ex=r t : That is, the total damage usually increases linearly with time t; however, it undergoes positively some change according to an exponential distribution. Then, pffi PrfZðtÞ  xg ¼ PrfBt  x  xtg ¼ 1  exp½ðx  xtÞ=r t; pffi EfZðtÞg ¼ xt þ r t;

ð7:25Þ

2

VfZðtÞg ¼ r t:

Thus, when the unit is replaced at time T; the expected cost rate is, from (7.20), pffiffiffiffi cK  ðcK  cT Þf1  exp½ðK  xTÞ=r T g : C1 ðTÞ ¼ RT pffi 0 f1  exp½ðK  xtÞ=r tgdt

ð7:26Þ

Differentiating C1 ðTÞ with respect to T and setting it equal to zero, pffiffiffiffi ZT pffi ðK=T þ xÞ exp½ðK  xTÞ=r T  pffiffiffiffi pffiffiffiffi f1  exp½ðK  xtÞ=r tgdt 2r T f1  exp½ðK  xTÞ=r T g 0

pffiffiffiffi  exp½ðK  xTÞ=r T  ¼

ð7:27Þ

1 : cK =cT  1

Table 7.4 presents optimum T  and expected cost rate C1 ðT  Þ=cT for cK =cT ¼ 5; 10; 20; 50 and K ¼ 5; 10; 15; 20 when r2 ¼ 1 and x ¼ 1: Compared to Table 7.3, optimum T  are less than T3 because the average amount of damage is greater than that in the continuous model (Sect. 7.2.1.3). However, if the damage level would be estimated at a lowest level and additional damage suffered from environment change and frequent disasters would be caused randomly, this would be applied to such models (Problem 7.8). On the other hand, if ZðtÞ ¼ xt  Bt ; i.e., the total damage changes negatively according to an exponential distribution, then

7.3 Lévy Process

193

Table 7.4 Optimum T  and C1 ðT  Þ=cT when x ¼ r2 ¼ 1 K cK =cT 5 5 10 20 50

10

15

20

T

C1 ðT  Þ=cT

T

C1 ðT  Þ=cT

T

C1 ðT  Þ=cT

2.2 1.6 1.1 0.8

0.7619 1.1029 1.3361 1.8131

4.8 3.8 3.1 2.3

0.2920 0.3648 0.4461 0.5683

7.8 6.4 5.3 4.3

0.1696 0.2045 0.2424 0.2983

T 10.9 9.2 7.8 6.5

pffi PrfZðtÞ  xg ¼ PrfBt  xt  xg ¼ exp½ðxt  xÞ=r t; pffi EfZðtÞg ¼ xt  r t; VfZðtÞg ¼ r2 t:

C1 ðT  Þ=cT 0.1163 0.1371 0.1594 0.1918

ð7:28Þ

Thus, the expected cost rate is (Problem 7.9) pffiffiffiffi cK  ðcK  cT Þ exp½ðxT  KÞ=r T  C2 ðTÞ ¼ : RT pffi 0 exp½ðxt  KÞ=r tdt

ð7:29Þ

Example 7.4 [6, p. 35] Consider the damage model with two kinds of damage in Sect. 6.3.4. One of them is caused by shock and is additive in Sect. 6.3.1, and the other increases proportionately with time t; i.e., the total damage is YðtÞ  at þ ZðtÞ; where ZðtÞ is given in (7.12). The unit fails whether the total damage exceeds a failure level with time or has exceeded in both cases, at some shock, and the failure is detected only at time of shocks. Denote that Sn  X1 þ X2 þ    þ Xn ; Zn ¼ W1 þ W2 þ    þ Wn ðn ¼ 1; 2; . . .), where l  EfXn g: The distribution of time to detect a failure at some shock is, from (6.36), 1 X

PrfZn þ aSn \K  Znþ1 þ aSnþ1 ; Snþ1  tg

n¼0

¼

8 tu t 1 Z
0

:

0

9 = i GðnÞ ðK  auÞ  Gðnþ1Þ ðK  aðu þ xÞÞ dFðxÞ dF ðnÞ ðuÞ: ; ð7:30Þ

The mean time to detect a failure at time shock is, from (6.37), 1 Z X

K=a

a

n¼0

GðnÞ ðK  atÞdF ðnÞ ðtÞ:

ð7:31Þ

0

Next, suppose that the unit is replaced at time T or at failure, i.e., when the total damage has exceeded K at some shock, whichever occurs first [6, p. 35, 15]. Then, the probability that the unit is replaced because failure is detected at some shock is, from (7.30),

194

7 Brownian Motion and Lévy Processes 1 X

PrfZn þ aSn  K; Znþ1 þ aSnþ1 [ K; Snþ1  Tg

n¼0 T 1 Z h i X GðnÞ ðK  atÞ  Gðnþ1Þ ðK  atÞ dF ðnþ1Þ ðtÞ ¼ n¼0

þ

0

8 Tt T 1 Z
0

:

0

9 = i GðnÞ ðK  atÞ  Gðnþ1Þ ðK  aðt þ xÞÞ dFðxÞ dF ðnÞ ðtÞ; ; ð7:32Þ

and the probability that it is replaced at time T before failure is 1 X

PrfZn þ aSn  K; Sn  T  Snþ1 g

n¼0 1 Z X

ð7:33Þ

T

¼

n¼0

ðnÞ

ðnÞ

FðT  tÞG ðK  atÞdF ðtÞ;

0

where (7.32) ? (7.33) = 1. The mean time to replacement is, from (7.32) and (7.33), 1 Z X

T

T

n¼0

FðT  tÞGðnÞ ðK  atÞdF ðnÞ ðtÞ

0

T 1 Z h i X þ t GðnÞ ðK  atÞ  Gðnþ1Þ ðK  atÞ dF ðnþ1Þ ðtÞ n¼0

0

8

9 = i ðt þ xÞ GðnÞ ðK  atÞ  GðnÞ ðK  aðt þ xÞÞ dFðxÞ dF ðnÞ ðtÞ þ : ; n¼0 0 0 2 3 T Z Tt 1 Z X 4 ¼ FðxÞdx5GðnÞ ðK  atÞdF ðnÞ ðtÞ: T Tt 1 Z
n¼0

0

h

0

ð7:34Þ Thus, the expected cost rate is, from (7.33) and (7.34), P RT ðnÞ ðnÞ cK  ðcK  cT Þ 1 n¼0 0 FðT  tÞG ðK  atÞdF ðtÞ h i ; CðTÞ ¼ P1 R T R Tt FðxÞdx GðnÞ ðK  atÞdF ðnÞ ðtÞ n¼0 0 0

ð7:35Þ

where cK is the replacement cost at failure, and cT is the replacement cost at time T; where cK [ cT :

7.3 Lévy Process

195

Table 7.5 Optimum T  when 1=k ¼ 1=h ¼ 1:0 a K ¼ 100 cK =cT 0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.5 3.0 4.0 5.0 6.0

K ¼ 200 cK =cT

2

5

10

100

2

5

10

100

81 69 59 52 47 42 39 36 33 31 29 25 22 18 15 13

72 62 54 48 43 39 36 33 31 29 27 23 21 17 14 12

68 58 51 45 41 37 34 32 29 28 26 22 20 16 14 12

59 51 45 40 37 33 31 29 27 25 24 21 18 15 13 11

169 142 123 108 97 87 80 74 68 63 59 51 45 36 30 26

157 133 115 102 92 83 76 70 65 61 57 49 43 35 29 25

151 128 112 99 89 81 74 68 64 59 56 48 43 34 29 25

138 118 103 92 83 76 69 64 60 56 53 46 40 33 28 24

We find an optimum T  that minimizes CðTÞ when shocks occur in a Poisson P k kt process, i.e., F ðnÞ ðtÞ ¼ 1 ðn ¼ 0; 1; 2; . . .Þ: Then, differentiating k¼n ½ðktÞ =k!e CðTÞ with respect to T and setting it equal to zero,   Z T Z T BðTÞ kcT kt kt 1þ e dBðtÞ  e BðtÞ dt ¼ ; ð7:36Þ RT c K  cT 1 þ 0 B ðtÞ dt 0 0 where BðtÞ  k

1 X ðktÞn n¼0

n!

Gðnþ1Þ ðK  atÞ:

Next, it is assumed that GðxÞ ¼ 1  ehx : Then, the average damage per unit of time is a þ k=h: So that, the mean time until the total damage exceeds a failure level K is K=ða þ k=hÞ: Table 7.5 presents the optimum T  that satisfies (7.36) for K ¼ 100; 200; cK =cT ¼ 2; 5; 10; 100 and a changes from 0 to 6 when 1=k ¼ 1=h ¼ 1:0: For example, when K ¼ 200; cK =cT ¼ 5; a ¼ 0:2; K=ða þ k=hÞ ¼ 167 and T  ¼ 133; i.e., the unit with the meantime 167 to failure should be replaced at time 133. In other words, if the operating time is ð133=167Þ 100 80% of the mean time to failure, the unit should be replaced before failure. This indicates that optimum T  decreases when both cK and a increase. This means that if a is large, the total damage increases mainly with time, and hence, we should adopt the scheduled replacement time T (Problem 7.10). Therefore, the rate a is an important factor for deciding the optimum policy (Problem 7.11).

196

7 Brownian Motion and Lévy Processes

7.4 Problems 7 7.1 When the density is 1 2 2 f ðx; tÞ ¼ pffiffiffiffiffiffiffi ex =ð2r tÞ ; 2ptr show that Z1

esx f ðx; tÞdx ¼ es

2 2

r t=2

:

1

7.2 In (1) of Example 7.1, compute the reliability RðtÞ for t ¼ 5; 10 when l ¼ r ¼ 1 and K ¼ 10; and the MTTF. 7.3 Prove in Example 7.2 that an optimum T  ð0\T  \1Þ is a finite and unique solution of (7.10), and the resulting cost rate is given in (7.11). 7.4 In Example 7.2, CðTÞ in (7.9) is approximately given by #  " 2 ðl þ r2 ÞT r2 c2 e þ : CðTÞ ¼ c1 l þ 1þ 2 2 T

7.5

7.6

7.7 7.8 7.9 7.10 7.11

e e e that minimizes CðTÞ: Derive CðTÞ and an optimum T Furthermore, show e numerically that T becomes a good approximation to T  : Prove that if Q1 ðTÞ increases strictly with T and Q1 ð1Þ½1 þ MG ðKÞ [ cK = ½lðcK  cT Þ; then there exists a finite and unique T1 ð0\T1 \1Þ that satisfies (7.15). Prove that if Q2 ðNÞ increases strictly with N and Q2 ð1Þ½1 þ MGT ðKÞ [ cK = ðcK  cT Þ; then there exists a finite and unique minimum N  ð1  N  \1Þ that satisfies (7.18). Prove that if rA ðtÞ increases strictly to infinity, then there exists a finite and unique T3 that satisfies (7.21). In Example 7.3, rewrite all results when Bt has a Weibull distribution  p ffi 

PrfBt  xg ¼ 1  exp  xa =r t for a  1: Derive an optimum T  that minimizes C2 ðTÞ in (7.29). Explain the physical meaning of why optimum T  decreases with a: In Example 7.4, consider the damage model where the unit is replaced at shock number N or at failure, and discuss an optimum N  that minimizes the expected cost [15].

References 1. Ross SM (1983) Stochastic processes. Wiley, New York 2. Karlin S, Taylor HM (1975) A first course in stochastic processes. Academic Press, New York 3. Birnbaum ZW, Saunders SC (1969) A new family of life distributions. J Appl Probab 6:319– 327

References

197

4. Owen WJ, Padgett WJ (2003) Accelerated test models with the Birnbaum–Saunders distribution. In: Pham H (ed) Handbook of reliability engineering. Springer, London, pp 429– 439 5. Park C, Padgett WJ (2005) New cumulative damage models for failure using stochastic processes as initial damage. IEEE Trans Reliab 54:530–540 6. Nakagawa T (2007) Shock and damage models in reliability theory. Springer, London 7. Beichelt FE (2006) Stochastic processes in science, engineering and finance. Chapman & Hall, Boca Raton 8. Ito K, Nakagawa T (2008) Comparison of three cumulative damage models. In: Sheu SH, Dohi T (eds) Advanced reliability modeling III. McGraw-Hill, Taipei, pp 332–338 9. Prabhu NU (2007) Stochastic processes. World Scientific, Singapore 10. Bertoin J (1996) Lévy processes. Cambridge University Press, Cambridge 11. Sato K (2001) Basic results on Lévy processes. In: Bandorff-Nielsen O, Mikosch T, Resnick S (eds) Lévy processes: theory and applications. Birkhäuser, Boston 12. Zuckerman D (1978) Optimal replacement policy for the case where the damage process is a one-sided Lévy process. Stoch Process Appl 7:141–151 13. Abdel-Hameed M (2010) Degradation process: an overview. In: Nikulin MS, Limnios N, Balakrishnan N, Kahle W, Huber-Carol C (eds) Advances in degradation modeling. Birkhäuser, Boston, pp 17–25 14. Lehmann A (2010) Failure time models based on degradation processes. In: Nikulin MS, Limnios N, Balakrishnan N, Kahle W, Huber-Carol C (eds) Advances in degradation modeling. Birkhäuser, Boston, pp 209–233 15. Satow T, Nakagawa T (1997) Replacement policies for a shock model with two kinds of damages. In: Osaki S (ed) Stochastic modeling in innovative manufacturing. Springer Lecture notes in economics and mathematical systems, vol 445. Springer, Berlin, pp 188–195

Chapter 8

Redundant Systems

High system reliability can be achieved by providing redundancy and maintenance. It is not too much to say that learning reliability is analyzing redundant systems and deriving optimum maintenance policies. In the preceding chapters, we have already given many useful examples of reliability models to understand naturally stochastic processes, and conversely, to apply the theory of stochastic processes to actual reliability models. As final examples of reliability models, we take up standard redundant systems and show systematically how to use well the techniques of stochastic processes to analyze them and to obtain their reliability properties theoretically. This would be greatly helpful for understanding stochastic processes and learning reliability theory [1, p. 160]. In Sect. 8.1, we analyze one-unit systems with no repair, that is, the simplest and fundamental reliability system, using Poisson, nonhomogeneous Poisson, and renewal processes. This section aims not only to make differences among three processes, but also to apply renewal processes to standard replacement policies. Furthermore, we obtain several reliability measures of a one-unit system with repair, using an alternating renewal process. In Sect. 8.2, we take up two-unit systems with repair. When both failure and repair times have exponential distributions, we obtain transition probabilities, using a continuous-time Markov chain. Further, when failure and repair times have general distributions, we analyze the system, using a Markov renewal process. This indicates that Markov renewal processes are the most powerful tool for analyzing reliability systems. It has been presented until now that states are well defined in stochastic processes and their behaviors are derived analytically. In Sect. 8.3, it can be shown as an example of a standby system with spare units that the system is analyzed and its several reliability measures are derived by forming renewal equations directly, and by taking their Laplace-Stieltjes (LS) transforms and forming generating functions. This would be useful for analyzing simple noncomplicated reliability models.

T. Nakagawa, Stochastic Processes, Springer Series in Reliability Engineering, DOI: 10.1007/978-0-85729-274-2_8, Ó Springer-Verlag London Limited 2011

199

200

8 Redundant Systems

8.1 One-Unit System 8.1.1 Poisson Process As the simplest redundant system, we consider a one-unit system where a unit begins to operate at time 0 and has a failure distribution ð1  ekt Þ. When the unit fails, it is repaired, becomes like new, and begins to operate again. The repair time is very small compared to its operating time and is assumed to be negligible. In other words, the unit is assumed to be replaced immediately with a new one at each failure and its replacement time is negligible. Let Xk (k ¼ 1; 2; . . .) be the failure times of the kth operating unit that are independent and identically distributed random variables with an exponential distribution ð1  ekt Þ, and Sn  X1 þ X2 þ    þ Xn (n ¼ 1; 2; . . .). From Chap. 2, the distribution of Sn is a gamma distribution with order n. Thus, the number NðtÞ of failures in ½0; t is, from (2.12) and (2.16), PrfNðtÞ ¼ ng ¼ PrfSn  t\Snþ1 g ¼ PrfSn  tg  PrfSnþ1  tg 1 1 X X ðktÞk kt ðktÞk kt e  e ¼ k! k! k¼n k¼nþ1 ¼

ðktÞn kt e n!

ðk ¼ 0; 1; 2; . . .Þ;

ð8:1Þ

that is a Poisson distribution with mean kt. It is said that failures occur in a Poisson process with rate k, i.e., the failures occur randomly independent of its age and operating time, and its residual life at any time t also has the same exponential distribution (1  ekt ) because of its memoryless property.

8.1.2 Nonhomogeneous Poisson Process Suppose that the unit undergoes minimal repair at failures and its failure rate hðtÞ remains unchanged by minimal repair given in Definition 2.3 of Sect. 2.3. The unit begins to operate again after the completion of minimal repair, where the time for minimal repair is negligible. Then, the expected number EfNðtÞg of failures in Rt ½0; t is HðtÞ  0 hðuÞdu, and the number NðtÞ of failures in [0, t] is, from (2.42), PrfNðtÞ ¼ ng ¼

½HðtÞn HðtÞ e n!

ðn ¼ 0; 1; 2; . . .Þ;

ð8:2Þ

that is a Poisson distribution with mean HðtÞ. It is said that failures occur in a nonhomogeneous Poisson process with an intensity function hðtÞ and a mean value function HðtÞ. If FðtÞ is IFR, i.e., hðtÞ increases with time t, failures are apt to fail with age. Some variations for minimal repair and preventive maintenance are

8.1 One-Unit System

201

well-known as the imperfect repair [2] and preventive maintenance, and were summarized [3, p. 96].

8.1.3 Renewal Process Suppose that the unit has a general failure distribution FðtÞ with finite mean l. Then, letting F ðnÞ ðtÞ be the n-fold Stieltjes convolution of FðtÞ with itself, from (3.3), (3.4), and (3.18), PrfSn  tg ¼ F ðnÞ ðtÞ; MðtÞ 1 ¼ ; t!1 t l lim

PrfNðtÞ ¼ ng ¼ F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ; 1 X F ðnÞ ðtÞ: EfNðtÞg ¼ MðtÞ ¼

ð8:3Þ

n¼1

It is said that failures occur in a renewal process with an interarrival distribution FðtÞ. Most reliability systems fail in accordance with renewal processes. The renewal function MðtÞ plays an important role in analyzing reliability systems. However, in general, it is practically impossible to explicitly express MðtÞ for a general distribution FðtÞ because it is very difficult to compute the n-fold Stieltjes convolution F ðnÞ ðtÞ of FðtÞ explicitly except exponential and gamma distributions given in Example 3.2. Thus, we derive the Laplace-Stieltjes (LS) transform of MðtÞ from Z 1 F  ðsÞ M  ðsÞ ¼ est dMðtÞ ¼ ; ð8:4Þ 1  F  ðsÞ 0 and compute MðtÞ explicitly only by inverting M  ðsÞ numerically, as shown in Sect. 3.2.3. A unit is planned to be replaced at periodic times kT (k ¼ 1; 2; . . .) and failures occur in a renewal process between replacements, i.e., each failed unit is replaced with a new one. Then, from (3.44), the expected cost rate is C1 ðTÞ ¼

c1 MðTÞ þ c2 ; T

ð8:5Þ

where c1 ¼ cost of replacement at each failure and c2 ¼ cost of planned replacement at time T. Similarly, suppose that failures occur in a nonhomogeneous Poisson process with a mean value function HðtÞ, i.e., each failed unit undergoes minimal repair. Then, from (3.46), the expected cost rate is C2 ðTÞ ¼

c1 HðTÞ þ c2 ; T

ð8:6Þ

where c1 ¼ cost of minimal repair at each failure and c2 is given in (8.5). Next, suppose that a unit is replaced at time T or at failure, whichever occurs first. Then, defining Zk  min fXk ; Tg, the process fZk g has a renewal process with interarrival distribution

202

8 Redundant Systems

 PrfZk  tg ¼

FðtÞ t  T; 1 t [ T;

and hence, EfZk g ¼

Z

T

t dFðtÞ þ TFðTÞ ¼

0

Z

T

FðtÞdt; 0

where uðtÞ  1  uðtÞ for any function uðtÞ. Then, from (3.42), the expected cost rate is C3 ðTÞ ¼

c1 FðTÞ þ c2 FðTÞ ; RT 0 FðtÞdt

ð8:7Þ

where c1 ¼ replacement cost at failure and c2 ¼ replacement cost at time T:

8.1.4 Alternating Renewal Process It is assumed in Sect. 8.1.3 that each repair time is not negligible and has an identical distribution GðtÞ with finite mean b. Let Yk (k ¼ 1; 2; . . .) be the repair times of the kth failed unit that are independent and identically distributed random variables with GðtÞ. We define the following states: State 0: Unit is operating. State 1: Unit is under repair. The unit repeats operating and repair alternately in Figs. 3.5 and 4.2. It is said that the unit forms an alternating renewal process with States 0 and 1. Clearly, the process fXk þ Yk g has a renewal process with interarrival distribution Rt FðtÞ  GðtÞ ¼ 0 Gðt  uÞdFðuÞ and mean ðl þ bÞ: Suppose that the unit begins to operate at time 0. Let Nj ðtÞ be the number of visits to State j in ½0; t, where the first visit to State 0 at time 0 is not counted, and Mj ðtÞ  EfNj ðtÞg that is called the renewal function. Then, from (3.53), Z 1 F  ðsÞG ðsÞ  ; est dM0 ðtÞ ¼ M0 ðsÞ ¼ 1  F  ðsÞG ðsÞ 0 ð8:8Þ Z 1 F  ðsÞ  st M1 ðsÞ ¼ e dM1 ðtÞ ¼ : 1  F  ðsÞG ðsÞ 0 Let ZðtÞ be the process at time t, and Pj ðtÞ  PrfZðtÞ ¼ jg (j ¼ 0; 1). Then, from (3.55), P0 ðsÞ ¼

1  F  ðsÞ ; 1  F  ðsÞG ðsÞ

P1 ðsÞ ¼

F  ðsÞ½1  G ðsÞ : 1  F  ðsÞG ðsÞ

ð8:9Þ

8.1 One-Unit System

203

Thus, the limiting probabilities Pj  limt!1 Pj ðtÞ are P0 ¼ lim P0 ðsÞ ¼ s!0

l ¼ 1  P1 ; lþb

ð8:10Þ

that represents the steady-state availability of the unit. Furthermore, the interval reliability is the probability that at a specified time, the unit is operating and will continue to operate for an interval of duration [3, pp. 48, 140]. Then, the interval reliability Rðx; tÞ for an interval of duration x starting at time t is, from (3.62) to (3.64), Z t Rðx; tÞ ¼ Fðt þ xÞ þ Fðt þ x  uÞdM0 ðuÞ; 0

and its Laplace transform in Appendix A is R1 Z 1 esx x est FðtÞdt  st e Rðx; tÞdt ¼ : R ðx; sÞ ¼ 1  F  ðsÞG ðsÞ 0 Thus, the limiting interval reliability is 

RðxÞ  lim Rðx; tÞ ¼ lim sR ðx; sÞ ¼ t!1

s!0

R1 x

FðtÞdt : lþb

ð8:11Þ

Next, suppose that an operating unit undergoes repair at time T or at failure, whichever occurs first, from the beginning of its operation, and define Zk  min fXk ; Tg. Then, the process fZk ; Yk g forms an alternating renewal process. On the other hand, when the repair is not completed until time T, it is replaced with a new one, that is called a repair limit policy in Example 3.7. Then, defining Zk  min fYk ; Tg, the process fXk ; Zk g forms an alternating renewal process. In this RT case, because EfZk g ¼ 0 GðtÞdt, the expected cost rate is, from (3.82), RT c1 GðTÞ þ 0 GðtÞdcr ðtÞ CðTÞ ¼ ; ð8:12Þ RT l þ 0 GðtÞdt where c1 ¼ replacement cost of a failed unit when its repair is not completed until time T and cr ðtÞ ¼ repair cost during ½0; t (Problem 8.1).

8.2 Two-Unit Standby System A two-unit standby redundant system with a single repair facility is one of the most fundamental and important redundant systems in reliability theory [4]. The system consists of two units, where one unit is used for operation and the other is in standby as an initial condition. If an operating unit fails, then it undergoes repair immediately and the other standby unit takes over its operation. Either of the two

204

8 Redundant Systems

units is alternately operating. It is said that system failure occurs when both units are down simultaneously. The transition probabilities, the first-passage distributions, and the renewal functions are derived by using the techniques of Markov processes.

8.2.1 Exponential Failure and Repair Times An operating unit has an exponential failure distribution (1  ekt ), and a failed unit also has an exponential repair distribution (1  eht ). If an operating unit fails and the other unit is in standby, the failed unit undergoes repair immediately, and the standby unit takes over its operation immediately. However, if an operating unit fails while the other unit is under repair, the failed unit has to wait until the repair facility become empty. This means that system failure has occurred. It is also assumed that a failed unit recovers its functioning upon repair completion and becomes as good as new. The unit in standby neither deteriorates nor fails in the standby interval. Two units are used alternately for its operation, as described above. Even if system failure occurs, the system can operate again upon repair completion. That is, the system repeats up and down alternately. All random variables considered here are independent of each other. To analyze the above system, we define the following three system states that represent the states of the process: State 0: One unit is operating and the other unit is in standby. State 1: One unit is operating and the other unit is under repair. State 2: One unit is under repair and the other unit is waiting for repair. The system states defined above are regeneration points because the failure and repair times are exponential. Let fXðtÞ ¼ jg denote that the process is in State j (j ¼ 0; 1; 2) at time t. Then, the process fXðtÞg forms a continuous-time Markov chain with finite state set f0; 1; 2g. Let Pij ðtÞ  PrfXðtÞ ¼ jjXð0Þ ¼ ig be the transition probabilities from State i to State j at time t. By the similar method in Example 4.11, P000 ðtÞ ¼ kP00 ðtÞ þ hP01 ðtÞ; P001 ðtÞ ¼ kP00 ðtÞ  ðk þ hÞP01 ðtÞ þ hP02 ðtÞ; P002 ðtÞ ¼ kP01 ðtÞ  hP02 ðtÞ: Taking the LS transformations of the above equations, sP00 ðsÞ  s ¼ kP00 ðsÞ þ hP01 ðsÞ; sP01 ðsÞ ¼ kP00 ðsÞ  ðk þ hÞP01 ðsÞ þ hP02 ðsÞ; sP02 ðsÞ ¼ kP01 ðsÞ  hP02 ðsÞ:

8.2 Two-Unit Standby System

205

Solving these equations, s2 þ ðk þ 2hÞs þ h2 ; s2 þ 2ðk þ hÞs þ k2 þ kh þ h2 kðs þ hÞ ; P01 ðsÞ ¼ s2 þ 2ðk þ hÞs þ k2 þ kh þ h2

P00 ðsÞ ¼

P02 ðsÞ ¼

k2 : s2 þ 2ðk þ hÞs þ k2 þ kh þ h2

Taking the inverse LS transforms,   h2 k x1  k  h x1 t x2  k  h x2 t ; P00 ðtÞ ¼ 2 þ e  e x1 x2 k þ kh þ h2 x1  x2   kh k x1 þ h x1 t x2 þ h x2 t ; P01 ðtÞ ¼ 2 þ e  e 2 x1  x2 x1 x2 k þ kh þ h   k2 k2 1 x1 t 1 x2 t P02 ðtÞ ¼ 2 þ e  e ; ð8:13Þ x2 k þ kh þ h2 x1  x2 x1 where P00 ðtÞ þ P01 ðtÞ þ P02 ðtÞ ¼ 1 (Problem 8.2), and x1  k þ h þ

pffiffiffiffiffi kh;

x2  k þ h 

pffiffiffiffiffi kh:

8.2.2 General Failure and Repair Times An operating unit has a general failure distribution FðtÞ with finite mean l, and a failed unit has a general repair distribution GðtÞ with finite mean b. The other assumptions are the same ones in Sect. 8.2.1. However, State 0 is not a regeneration point except when an operating unit fails for the first time from the beginning of its operation at time 0, and the failure time is exponential. From this reason, we define an initial time instant: State -1: One unit begins to operate and the other unit is in standby. The other states are the same as those in Sect. 8.2.1. An epoch at which the process makes a transition into State 1 is a regeneration point. However, the epochs at which the system makes a transition into State j (j ¼ 0; 2) are not regeneration points, except the failure and repair times are exponential (Fig. 5.3). We analyze the system by using the techniques of a Markov renewal process in Sect. 5.3. Define a mass function Qij ðtÞ from State i (i ¼ 1; 1) to State j (j ¼ 0; 1; 2) by the probability that after making a transition into State i, the process next makes a transition into State j, in a smaller amount of time than

206

8 Redundant Systems

time t in a Markov renewal process. Then, from Sect. 5.3.2, we have the following mass functions: Q11 ðtÞ ¼ FðtÞ; Z t FðuÞdGðuÞ; Q10 ðtÞ ¼

Q12 ðtÞ ¼

0

Z

t

GðuÞdFðuÞ:

ð8:14Þ

0

It is impossible to define the mass function Q01 ðtÞ and Q21 ðtÞ because the epochs for States 0 and 2 are not regeneration points. Thus, define a new mass ð0Þ ð2Þ function Q11 ðtÞ ðQ11 ðtÞÞ that is the recurrence time distribution for State 1 via ð0Þ

ð2Þ

State 0 (State 2). That is, Q11 ðtÞ ðQ11 ðtÞÞ is the probability that after making a transition into State 1, the process next makes a transition into State 0 (State 2) and returns to State 1 in time t. Then, from Sect. 5.3.2, Z t Z t ð0Þ ð2Þ Q11 ðtÞ ¼ GðuÞdFðuÞ; Q11 ðtÞ ¼ FðuÞdGðuÞ: ð8:15Þ 0

0

Therefore, the LS transforms of the mass functions are Q11 ðsÞ ¼ F  ðsÞ; Z 1  est FðtÞdGðtÞ; Q10 ðsÞ ¼ Z0 1 ð0Þ Q11 ðsÞ ¼ est GðtÞdFðtÞ; 0

Q12 ðsÞ ð2Þ

¼

Q11 ðsÞ ¼

Z

1

Z0 1

est GðtÞdFðtÞ;

ð8:16Þ

est FðtÞdGðtÞ:

0

8.2.2.1 First-Passage Distribution We obtain the first-passage distributions and their mean times by using the mass functions. Let Hij ðtÞ denote the first-passage distributions from State i (i ¼ 1; 1) to State j (j ¼ 0; 1; 2). First, suppose that the process starts in the epoch for State 1. Then, from (8.15), the recurrence time distribution H11 ðtÞ for State 1 is ð0Þ

ð2Þ

H11 ðtÞ ¼ Q11 ðtÞ þ Q11 ðtÞ ¼ FðtÞGðtÞ; that is the probability that the process comes back to State 1 via State 0 or State 2, whichever occurs first, and its LS transform is, from (8.16), Z 1 Z 1 Z 1  st st H11 ðsÞ ¼ e dH11 ðtÞ ¼ e FðtÞdGðtÞ þ est GðtÞdFðtÞ: ð8:17Þ 0

0

0

Thus, the mean recurrence time l11 for State 1 is Z 1 t dH11 ðtÞ ¼ l þ b  c; l11 ¼ 0

ð8:18Þ

8.2 Two-Unit Standby System

207

R1 where c  0 FðtÞGðtÞdt. The first-passage distributions H1j ðtÞ (j ¼ 0; 2) are given by solving the following renewal-type equation: ð2Þ

H10 ðtÞ ¼ Q10 ðtÞ þ Q11 ðtÞ  H10 ðtÞ;

ð8:19Þ

where the asterisk mark represents the Stieltjes convolution, i.e., aðtÞ  bðtÞ  Rt 0 bðt  uÞdaðuÞ for any functions aðtÞ and bðtÞ. The first term at the right-hand side in (8.19) represents the probability that the process goes to State 0 directly, and the second term is the probability that the process goes to State 0 after returning to State 1 via State 2. Similarly, ð0Þ

H12 ðtÞ ¼ Q12 ðtÞ þ Q11 ðtÞ  H12 ðtÞ:

ð8:20Þ

If the process starts from State -1, then, H1j ðtÞ ¼ Q11 ðtÞ  H1j ðtÞ

ðj ¼ 0; 2Þ:

Therefore, taking the LS transforms of (8.20)–(8.22) and solving them, R1 Q ðsÞQ ðsÞ F  ðsÞ 0 est FðtÞdGðtÞ  R1 H10 ðsÞ ¼ 11 ð2Þ10 ¼ ; 1  0 est FðtÞdGðtÞ 1  Q11 ðsÞ R1 Q11 ðsÞQ12 ðsÞ F  ðsÞ 0 est GðtÞdFðtÞ  R : ¼ H12 ðsÞ ¼ 1 ð0Þ 1  0 est GðtÞdFðtÞ 1  Q11 ðsÞ

ð8:21Þ

ð8:22Þ

The mean first-passage times l1j (j ¼ 0; 2), starting from State -1, are  1  H10 ðsÞ b ; ¼ l þ R1 s!0 s FðtÞdGðtÞ 0 " #  1  H12 ðsÞ 1 ¼ l 1 þ R1 ; ¼ lim s!0 s GðtÞdFðtÞ 0

l10 ¼ lim l12

ð8:23Þ

that represent the mean times to system recovery and system failure, respectively. Furthermore, the mean time of the busy period of a repair facility is  1  H10 ðsÞ b : ¼ l10  l ¼ R 1 s!0 s 0 FðtÞdGðtÞ

l10 ¼ lim

The mean downtime ld after system failure is Z 1 Z 1 ð2Þ ld ¼ t dQ11 ðtÞ  t dQ12 ðtÞ 0 Z0 1 FðtÞGðtÞdt ¼ b  c: ¼ 0

208

8 Redundant Systems

8.2.2.2 Expected Number of Visits to States Consider the expected number of visits to State j (j ¼ 0; 1; 2) during the interval ½0; t. Let the renewal function Mij ðtÞ be the expected number of occurrences of State j in ½0; t, starting in the epoch for State i (i ¼ 1; 1) at time 0, where the first visit to j is not counted if i ¼ j. First, when the process starts from State 1 at time 0, we have the following renewal-type equation: ð0Þ

ð0Þ

ð2Þ

M10 ðtÞ ¼ Q10 ðtÞ  Q11 ðtÞ þ Q11 ðtÞ  ½1 þ M10 ðtÞ þ Q11 ðtÞ  M10 ðtÞ ¼ Q10 ðtÞ þ H11 ðtÞ  M10 ðtÞ;

ð8:24Þ

where the first term is the probability that the process goes to State 0 directly and then remains in State 0, the second term is the expected number that the process returns to State 1 via State 0 and goes to State 0, and finally, the third term is the expected number that the process returns to State 1 via State 2 and then goes to State 0. Similarly, ð0Þ

ð2Þ

M11 ðtÞ ¼ Q11 ðtÞ  ½1 þ M11 ðtÞ þ Q11 ðtÞ  ½1 þ M11 ðtÞ ¼ H11 ðtÞ  ½1 þ M11 ðtÞ; ð2Þ

ð0Þ

ð2Þ

M12 ðtÞ ¼ Q12 ðtÞ  Q11 ðtÞ þ Q11 ðtÞ  M12 ðtÞ þ Q11 ðtÞ  ½1 þ M12 ðtÞ

ð8:25Þ

¼ Q12 ðtÞ þ H11 ðtÞ  M12 ðtÞ: The expected numbers of visits to State j (j ¼ 0; 1; 2) in ½0; t, starting from State -1, are M11 ðtÞ ¼ Q11 ðtÞ  ½1 þ M11 ðtÞ; ð8:26Þ M1j ðtÞ ¼ Q11 ðtÞ  M1j ðtÞ ðj ¼ 0; 2Þ: Taking the LS transforms of (8.24)–(8.26) and solving them, R1 Q11 ðsÞQ10 ðsÞ F  ðsÞ 0 est FðtÞdGðtÞ  ; M10 ðsÞ ¼  ðsÞ ¼  ðsÞ 1  H11 1  H11 Q11 ðsÞ F  ðsÞ  ðsÞ ¼ M11 ¼ ;   ðsÞ 1  H11 ðsÞ 1  H11 R1 Q11 ðsÞQ12 ðsÞ F  ðsÞ 0 est GðtÞdFðtÞ  : M12 ðsÞ ¼  ðsÞ ¼  ðsÞ 1  H11 1  H11

ð8:27Þ

In addition, if FðtÞ and GðtÞ are non-lattice and their mean times are finite, from Theorem 3.2, there exist the limits Mj  limt!1 Mij ðtÞ=t (j ¼ 0; 1; 2) of expected numbers of visits to State j per unit of time in the steady-state that are independent of an initial State i. Thus, by applying a Tauberian theorem in Appendix A.3 to (8.27), respectively,

8.2 Two-Unit Standby System

209

Z 1 1 M0 ¼ ¼ FðtÞdGðtÞ; l11 0 Z 1 1  ðsÞ ¼ GðtÞdFðtÞ; M2 ¼ lim sM12 s!0 l11 0 1  ðsÞ ¼ ¼ M0 þ M2 : M1 ¼ lim sM11 s!0 l11  lim sM10 ðsÞ s!0

A quantity of some interest is the total number of units that have failed and have been repaired in ½0; t, respectively. Letting Mif ðtÞ and Mir ðtÞ be the respected number of failed and repaired units in ½0; t, when the process starts from State i (i ¼ 1; 1), ð2Þ

M1f ðtÞ ¼ Q12 ðtÞ  Q11 ðtÞ þ H11 ðtÞ  ½1 þ M1f ðtÞ ¼ FðtÞGðtÞ þ H11 ðtÞ  ½1 þ M1f ðtÞ;

ð8:28Þ

ð0Þ

M1r ðtÞ ¼ Q10 ðtÞ  Q11 ðtÞ þ H11 ðtÞ  ½1 þ M1r ðtÞ ¼ FðtÞGðtÞ þ H11 ðtÞ  ½1 þ M1r ðtÞ: If the process starts from State -1, then M1f ðtÞ ¼ Q11 ðtÞ  ½1 þ M1f ðtÞ;

ð8:29Þ

M1r ðtÞ ¼ Q11 ðtÞ  M1r ðtÞ: Thus, combining (8.28) and (8.29),  M1f ðsÞ ¼

 M1r ðsÞ ¼

h i ð2Þ Q11 ðsÞ 1 þ Q12 ðsÞ  Q11 ðsÞ  ðsÞ 1  H11

h i ð2Þ Q11 ðsÞ Q10 ðsÞ þ Q11 ðsÞ  ðsÞ 1  H11

; ð8:30Þ

:

Clearly, Mf  lim

t!1

M1f ðtÞ M1r ðtÞ ¼ Mr ¼ lim ¼ M1 : t!1 t t

8.2.2.3 Transition Probabilities Denote that Pij ðtÞ is the transition probability that the process is in State j (j ¼ 0; 1; 2) at time t, starting from State i (i ¼ 1; 1) at time 0. Then, we have the following renewal equations:

210

8 Redundant Systems

ð0Þ

P10 ðtÞ ¼ Q10 ðtÞ  Q11 ðtÞ þ H11 ðtÞ  P10 ðtÞ ¼ FðtÞGðtÞ þ H11 ðtÞ  P10 ðtÞ; P11 ðtÞ ¼ 1  Q10 ðtÞ  Q12 ðtÞ þ H11 ðtÞ  P11 ðtÞ

ð8:31Þ

¼ FðtÞGðtÞ þ H11 ðtÞ  P11 ðtÞ; ð2Þ

P12 ðtÞ ¼ Q12 ðtÞ  Q11 ðtÞ þ H11 ðtÞ  P12 ðtÞ ¼ FðtÞGðtÞ þ H11 ðtÞ  P12 ðtÞ: Furthermore, if the process starts from State -1, then P10 ðtÞ ¼ 1  Q11 ðtÞ þ Q11 ðtÞ  P10 ðtÞ ¼ FðtÞ þ Q11 ðtÞ  P10 ðtÞ; ðj ¼ 0; 2Þ: P1j ðtÞ ¼ Q11 ðtÞ  P1j ðtÞ

ð8:32Þ

Thus, the LS transforms of P1j ðtÞ (j ¼ 0; 1; 2) are h i ð0Þ Q11 ðsÞ Q10 ðsÞ  Q11 ðsÞ

P10 ðsÞ ¼ 1  Q11 ðsÞ þ  ðsÞ 1  H11   Q ðsÞ 1  Q10 ðsÞ  Q12 ðsÞ P11 ðsÞ ¼ 11 ;  1  H11 ðsÞ h i ð2Þ Q11 ðsÞ Q12 ðsÞ  Q11 ðsÞ : P12 ðsÞ ¼  ðsÞ 1  H11

; ð8:33Þ

It is evident that because P10 ðsÞ þ P11 ðsÞ þ P12 ðsÞ ¼ 1, P10 ðtÞ þ P11 ðtÞ þ P12 ðtÞ ¼ 1. Note that P11 ðtÞ ¼ M11 ðtÞ  M10 ðtÞ  M12 ðtÞ. If FðtÞ and GðtÞ are non-lattice and their means are finite, there exist the limiting probabilities Pj  limt!1 Pij ðtÞ (j ¼ 0; 1; 2) that are independent of an initial State i. Thus, by applying a Tauberian theorem to (8.33), b ; l11 lþb ; P1 ¼ lim P11 ðsÞ ¼ 1 þ s!0 l11 l P2 ¼ lim P12 ðsÞ ¼ 1  : s!0 l11 P0 ¼ lim P10 ðsÞ ¼ 1  s!0

It is noted that AðtÞ ¼ P10 ðtÞ þ P11 ðtÞ represents the pointwise availability of the system at time t, and AðtÞ ¼ P12 ðtÞ represents the pointwise unavailability at time t, given that two units are initial good. It is also noted that A ¼ P0 þ P1 ¼ l=l11 represents the steady-state availability and A ¼ P2 represents the steady-state

8.2 Two-Unit Standby System

211

unavailability (Problem 8.3). In other words, the so-called availability represents the ratio of the mean operating time to the total of mean operating and repair times [3, p. 9].

8.3 Standby System with Spare Units Consider the system with a main unit and n spare units [5]. It is assumed that the spare units are statistically not identical to the main unit, but have the same functions as the main unit, if they take over the system operation. The system operates in such a way that if the main unit fails then it undergoes repair immediately and one of the spare units replaces it. As soon as the repair of the main unit is completed, it begins to operate and the operating spare unit is available for further use. A failed spare unit is scrapped. The system can operate until the nth spare unit fails. In other words, the system fails when the last spare unit fails while the main unit is under repair. The operating behaviors of this model require the following knowledge: (1) Distribution of time and mean time to first system failure, given that n spare units are provided at time 0. (2) Probability that the number of failed spare units is exactly equal to n and its expected number in ½0; t. These quantities are derived from renewal equations and used for two optimization problems and to determine an initial number of spare units to stock.

8.3.1 First-Passage Time to System Failure Assume that the failure time of the main unit has a general distribution FðtÞ with mean l and its repair time has a general distribution GðtÞ with mean b. That is, the main unit forms an alternating renewal process that repeats operating and failed states. It is also assumed that the failure time of each operating spare unit has a general distribution FS ðtÞ with mean lS , even if it has been used before, i.e., the life of spare units that have already been in service is not affected by the past. All random variables considered here are independent. Unless otherwise stated, all units are good at time 0, and any failures are instantaneously detected and repaired or scrapped. Each switchover is perfect and each switchover time is instantaneous. Using the modified methods of an alternating renewal process in Sect. 3.4, we obtain the reliability properties of the system. First, suppose that n spare units are provided at time 0. Let Hk ðtÞ (k ¼ 1; 2; . . .; nÞ denote the first-passage distribution

212

8 Redundant Systems

to system failure when k spare units are provided at time 0. Then, we have the following renewal equation: Z t ðnÞ GðuÞdFS ðuÞ Hn ðtÞ ¼ FðtÞ  0 ) Z th n1 i X ðkÞ ðkþ1Þ þ Hnk ðtÞ  FS ðuÞ  FS ðuÞ dGðuÞ ðn ¼ 1; 2; . . .Þ; 0

k¼0

ð8:34Þ ðkÞ

where the asterisk mark represents the convolution, and FS ðtÞ (k ¼ 1; 2; . . .) ð0Þ

represents the k-fold Stieltjes convolution of FS ðtÞ with itself and FS ðtÞ  1 for t  0. The first term of the bracket on the right-hand side in (8.34) is the distribution of the time that all of n spare units fail before the first repair completion of the failed main unit. The second term is the distribution of the time that k (k ¼ 0; 1; . . .; n  1) spare units fail exactly until the first repair completion of the main unit, and then, the system with n  k spare units operates again. The first-passage distribution Hn ðtÞ can be calculated recursively and uniquely determined from (8.34). Introduce the notation of the generating function of the LS transform: Z 1 1 X  n z est dHn ðtÞ H ðz; sÞ  n¼1

0

for jzj  1. Thus, taking the LS transform of (8.34) and forming the generating function H  ðz; sÞ, R P ðnÞ n 1 st F  ðsÞ 1 GðtÞdFS ðtÞ n¼1 z 0 e  h i : ð8:35Þ H ðz; sÞ ¼ R P n 1 st F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ dGðtÞ 1  F  ðsÞ 1 S S n¼0 z 0 e Moreover, let ln denote the mean first-passage time to system failure. Then, the renewal equation for ln is easily given, from (8.34), ln ¼ l þ

Z

1h 0

ðnÞ 1FS ðtÞ

i

GðtÞdt þ

n1 X k¼0

Z lnk 0

1h

ðkÞ

ðkþ1Þ

FS ðtÞFS

i ðtÞ dGðtÞ

ðn ¼ 1; 2; . . .Þ;

ð8:36Þ

and hence, its generating function is i R 1h P ðnÞ lz=ð1  zÞ þ 1 zn 0 1  FS ðtÞ GðtÞdt n¼1 i l ðzÞ  z n ln ¼ : R h P n 1 F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ dGðtÞ 1 1 z n¼1 S S n¼0 0 1 X

ð8:37Þ

8.3 Standby System with Spare Units

213

8.3.2 Expected Number of Failed Units We derive the expected number of failed spare units in ½0; t. Let Mn ðtÞ (n ¼ 0; 1; . . .) be the that the total number of failed spare units in ½0; t Pprobability 1 are n, and MðtÞ  n¼1 nMn ðtÞ. Then, in a similar way of obtaining (8.34),  M0 ðtÞ ¼ FðtÞ þ FðtÞ  F S ðtÞGðtÞ þ M0 ðtÞ  ( h i ðnÞ ðnþ1Þ ðtÞ GðtÞ Mn ðtÞ ¼ FðtÞ  FS ðtÞ  FS þ

n X

Mnk ðtÞ 

k¼0

Z th 0

ðkÞ FS ðuÞ



Z

t

F S ðuÞdGðuÞ ;

0

ðkþ1Þ FS ðuÞ

i

) dGðuÞ

ðn ¼ 1; 2; . . .Þ: ð8:38Þ

Introduce the notation M  ðz; sÞ 

1 X

zn

n¼0

Z

1

est dMn ðtÞ:

0

Then, from (8.38),

nh i o R1 P ðnÞ ðnþ1Þ 1  F  ðsÞ 1  1 zn 0 est d FS ðtÞ  FS ðtÞ GðtÞ n¼0 h i : M  ðz; sÞ ¼ R P n 1 st F ðnÞ ðtÞ  F ðnþ1Þ ðtÞ dGðtÞ 1  F  ðsÞ 1 S S n¼0 z 0 e ð8:39Þ It can be easily shown that limz!1 M  ðz; sÞ ¼ 1. The LS transform of the expected number of failed spare units in ½0; t is Z 1 1 X dM  ðz; sÞ  M ðsÞ  n est dMn ðtÞ ¼ lim z!1 dz 0 n¼1 R 1 F  ðsÞ 0 est GðtÞdMS ðtÞ ; ð8:40Þ ¼ 1  F  ðsÞG ðsÞ P ðnÞ where note that MS ðtÞ  1 n¼1 FS ðtÞ is the renewal function for spare units. Furthermore, the expected number of failed spare units per unit of time in the steady-state is MðtÞ M  lim ¼ t!1 t

R1 0

MS ðtÞdGðtÞ : lþb

ð8:41Þ

214

8 Redundant Systems

Equation (8.41) can be intuitively explained because the numerator represents the total expected number of failed spare units during the repair time of the main unit, and the denominator represents the mean time from the beginning of the operating main unit to its repair completion. Further, we obtain the expected number mn of failed main units until system failure for the system with n spare units. Because Z

1

0

n1 Z X

ðnÞ

GðtÞdFS ðtÞ þ

k¼0

1h

ðkÞ

ðkþ1Þ

i ðtÞ dGðtÞ ¼ 1;

ðkþ1Þ

i ðtÞ dGðtÞ:

FS ðtÞ  FS

0

and by the similar method of obtaining (8.36), mn ¼ 1 þ

n1 X

Z

1h

mnk

ðkÞ

FS ðtÞ  FS

0

k¼0

Thus, the generating function of mn is m ðzÞ 

1 X n¼1

z n mn ¼

1

z=ð1  zÞ i : R 1 h ðnÞ ðnþ1Þ n ðtÞ dGðtÞ n¼0 z 0 FS ðtÞ  FS

P1

ð8:42Þ

Example 8.1 (Exponential repair time) Suppose that GðtÞ ¼ 1  eht . Then, from (8.37), the mean time to system failure, given that n pare units are provided at time 0, is (Problem 8.4)   1 1  FS ðhÞ ln ¼ l þ n l þ : h FS ðhÞ Note that, by adding one spare unit to the system, the mean time to system increases by constant A, independent of the number of the former spare units, where A  ðl þ 1=hÞ½1  FS ðhÞ=FS ðhÞ. Furthermore, the LS transform of the expected number of failed spare units in ½0; t is, from (8.40), F  ðsÞFS ðs þ hÞ  ; M  ðsÞ ¼   1  FS ðs þ hÞ f1  ½h=ðs þ hÞF  ðsÞg and its limit per unit of time is, from (8.41) (Problem 8.4), M¼

1 FS ðhÞ 1 ¼ :  l þ 1=h 1  FS ðhÞ A

In a similar way, the expected number of failed main units until system failure is, from (8.42) (Problem 8.4), mn ¼ 1 þ n

1  FS ðhÞ ; FS ðhÞ

8.3 Standby System with Spare Units

215

where note that mn ¼

ln þ 1=h : l þ 1=h

8.3.3 Expected Cost and Optimization Problems We derive the expected cost rate by introducing costs incurred for each failed main unit and failed spare units when infinite spare units are provided at time 0. This expected cost is easily deduced from the expected number of failed units. Then, we compare the expected cost rates for the system with both main unit and spare units, and for the system with only spare units, and determine which of them is more economical. Cost c1 is incurred for each failed main unit that is repaired, and cost cS is incurred for each failed spare unit that is replaced and scrapped. Letting CðtÞ be total expected cost incurred in ½0; t, CðtÞ  c1 MR ðtÞ þ cS MðtÞ; where MR ðtÞ represents the expected number of repairs for the failed main unit, and recall that MðtÞ is the expected number of failed spare units in ½0; t. It is easily seen in Sect. 3.4 that MR  lim

t!1

MR ðtÞ 1 ¼ : t lþb

Thus, the expected cost rate is, from (3.19) and (8.41), CðtÞ ¼ c1 MR þ cS M tR 1 c1 þ cS 0 MS ðtÞdGðtÞ ; ¼ lþb

C  lim

t!1

ð8:43Þ

that is also equal to the total expected cost of one cycle from the beginning of the operating main unit to its repair completion. On the other hand, if only a spare unit is allowed to operate, then the expected cost rate is CS ¼

cS : ls

Therefore, comparing (8.43) and (8.44), C  ðÞCS if and only if  Z 1 lþb c1  ðÞcS  MS ðtÞdGðtÞ : ls 0

ð8:44Þ

ð8:45Þ

216

8 Redundant Systems

In particular, when GðtÞ ¼ 1  eht , (8.45) becomes  l þ 1=h FS ðhÞ  : c1  ðÞcS ls 1  FS ðhÞ

ð8:46Þ

In this case, if the mean time ln to system failure is required to be greater than DT , then we can provide a minimum number of spare units that satisfies n

DT  l 1  FS ðhÞ : l þ 1=h FS ðhÞ

If the expected cost until the mean time to system failure that is given by  1  FS ðhÞ cS n þ c1 mn ¼ c1 þ n cS þ c1 FS ðhÞ is less than DC , we can provide a maximum number of spare units that satisfies n

cS þ c1



DC  c1 

1  FS ðhÞ =FS ðhÞ

(Problem 8.5).

8.4 Problems 8 8.1 In (8.12), derive an optimum repair limit time T  that minimizes CðTÞ when cr ðtÞ ¼ at for a [ 0 [3, p. 52]. 8.2 Derive the transition probabilities of a two-unit standby system with standby failure, i.e., when the unit in standby has the failure rate kS , and the other assumptions are the same ones in Sect. 8.2.1 [4]. 8.3 Consider the same two-unit standby system in Sect. 8.2.2 where each unit has a failure distribution FðtÞ with mean l and a failed unit has a repair distribution G1 ðtÞ with mean b1 . We adopt the following PM policy for an operating unit. When an operating unit works for a specified time T without failure, we stop its operation for PM. It is assumed that the time to PM completion has a general distribution G2 ðtÞ with mean b2 , and any unit becomes as good as new upon repair or PM completion. Furthermore, we make the following two assumptions. (a) PM of an operating unit is done only if the other unit is in standby. (b) An operating unit that has forfeited PM because of (1) undergoes PM just upon repair or PM completion of the other unit. Under the above assumptions, obtain the availability and derive an optimum PM time that maximizes it [3, p. 144].

8.4 Problems 8

217

8.4 In Example 8.1, derive ln , mn and M. 8.5 Consider the PM policy in Sect. 8.3 where the operating main unit undergoes PM at time T (0\T  1) after its installation or is repaired at failure, whichever occurs first. Spare units are infinite and work temporarily during the interval of the repair or PM time of the main unit. It is assumed that the PM time has the same distribution GðtÞ as the repair time. The main unit is as good as new upon repair or PM and begins to operate immediately. The costs incurred for each failed main unit and each spare unit are c1 and cs , respectively. The PM cost c2 with c2 \c1 is incurred for each non-failed main unit. Then, obtain the expected cost rate and derive an optimum PM time T  that minimizes it [5].

References 1. 2. 3. 4.

Birolini A (1999) Reliability engineering theory and practice. Springer, New York Brown M, Proschan F (1983) Imperfect repair. J Appl Prob 20:851–859 Nakagawa T (2005) Maintenance theory of reliability. Springer, London Nakagawa T (2002) Two-unit redundant models. In: Osaki S (ed) Stochastic models in reliability and maintenance. Springer, Berlin, pp 165–185 5. Nakagawa T, Osaki S (1976) Reliability analysis of a one-unit system with unrepairable spare units and it optimization applications. Oper Res Q 27:101–110

Appendix A: Laplace Transform

This appendix presents some results of Laplace and Laplace–Stieltjes (LS) transforms and give useful tools used in reliability theory [1–4].

A.1 Laplace and Laplace–Stieltjes Transforms Let f ðtÞ be a function of t specified for t  0 and s be a complex number. Then, the Laplace transform of f ðtÞ is defined by 

Lff ðtÞg  f ðsÞ ¼

Z1

est f ðtÞdt;

ðA:1Þ

0

that is a function of the complex number s. Although s is defined as the complex number, we do not need the knowledge of complex analysis and may use the Laplace transform as if it is a real number in reliability theory. If the above integral (A.1) converges for ReðsÞ [ s0 , then it always exists for ReðsÞ [ s0 . For example, suppose that f ðtÞ is a function of t  0 such that jf ðtÞj  Meat for some M and a. Then, f  ðsÞ exists for all ReðsÞ [ a. The function f ðtÞ is called the inverse Laplace transform of f  ðsÞ and is written f ðtÞ ¼ L1 ff  ðsÞg:

ðA:2Þ

When f ðtÞ is a probability density of a nonnegative random variable X, sX

Efe

g¼

Z1

est f ðtÞdt ¼ f  ðsÞ:

ðA:3Þ

0

Similarly, let FðtÞ be a function of t  0: The Laplace–Stieltjes (LS) transform of FðtÞ is defined by

219

220

Appendix A: Laplace Transform

F  ðsÞ 

Z1

est dFðtÞ

ðA:4Þ

0

for ReðsÞ [ s0 . When FðtÞ is a probability distribution of a random variable X, the LS transform of FðtÞ is EfesX g ¼

Z1

est dFðtÞ ¼ F  ðsÞ;

ðA:5Þ

0

that agrees with the Laplace transform f  ðsÞ in (A.3). From (A.5), the nth moment of a random variable X is  dF  ðsÞ 1  F  ðsÞ EfXg ¼  ¼ lim ;  ds s¼0 s!0 s  dn F  ðsÞ ðn ¼ 1; 2; . . .Þ; EfX n g ¼  dsn s¼0 VfXg ¼ EfX 2 g  ðEfXgÞ2 : There is no theoretical difference greatly between Laplace and LS transforms, because 

F ðsÞ ¼

Z1 e 0

st

dFðtÞ ¼ s

Z1

est FðtÞdt:

ðA:6Þ

0

In other words, the LS transform can be obtained by s times the Laplace transform. It would be more convenient to use LS transforms because we deal mainly with distribution forms more than density ones in reliability theory. However, we sometimes use the Laplace transform for some functions in p. 61, p. 77, p. 78, p.184 and p. 196.

A.2 Properties We always use LS transforms, except for some special functions through this book and summarize their brief properties. LS transforms play significant, important roles in making integral and differential calculuses, and in calculating distributions of the sum of random variables. Let F  ðsÞ be the LS transform of FðtÞ in (A.5).

Appendix A: Laplace Transform

1. The LS transform of

221

Rt

FðuÞdu is 3 Zt Z1 Z1 1 st 4 5 e d FðuÞdu ¼ est FðtÞdt ¼ F  ðsÞ: s 0

2

0

0

ðA:7Þ

0

2. The LS transform of F 0 ðtÞ is Z1

est dF 0 ðtÞ ¼ s

0

Z1

est F 0 ðtÞdt ¼ s½F  ðsÞ  Fð0Þ:

ðA:8Þ

0

Rt 3. If the function HðtÞ is given by the convolution HðtÞ ¼ 0 Gðt  uÞdFðuÞ, and the LS transforms of GðtÞ and FðtÞ are G ðsÞ and F  ðsÞ, respectively, the LS transform of HðtÞ is 2 t 3 Z1 Z est d4 Gðt  uÞdFðuÞ5 H  ðsÞ ¼ 0

¼

Z1

0 st

e

Z1 dGðtÞ

0

est dFðtÞ

ðA:9Þ

0

¼ G ðsÞF  ðsÞ: This is easily obtained by another method: If two random variables X and Y are independent and have distributions GðtÞ and FðtÞ, respectively, the LS transforms of distribution of Z  X þ Y is EfesZ g ¼ EfesðXþYÞ g ¼ EfesX gEfesY g ¼ G ðsÞF  ðsÞ: Table A.1 presents some properties of LS transforms and Table A.2 presents LS transforms of some functions [1, 4]

A.3 Abelian and Tauberian Theorems We show two theorems that are very useful for performing limiting operations with transforms: Theorem A.1 (An Abelian Theorem) If lim

t!1

FðtÞ C for a  0; ¼ ta Cða þ 1Þ

ðA:10Þ

222

Appendix A: Laplace Transform

Table A.1 Properties of LS transforms

Table A.2 LS transforms of functions

FðtÞ

F  ðsÞ

F1 ðtÞ þ F2 ðtÞ aFðtÞ Fðt  aÞða [ 0Þ FðatÞ eat FðtÞ ða [ 0Þ F 0 ðtÞ ¼ dFðtÞ=dt tF 0 ðtÞ Rt FðuÞdu R0t 0 F2 ðt  uÞd F1 ðuÞ limt!0 FðtÞ limt!1 FðtÞ

F1 ðsÞ þ F2 ðsÞ aF  ðsÞ esa F  ðsÞ F  ðs=aÞ ½s=ðs þ aÞF  ðs þ aÞ s½F  ðsÞ  Fð0Þ sdF  ðsÞ=ds F  ðsÞ=s F1 ðsÞF2 ðsÞ lims!1 F  ðsÞ lims!0 F  ðsÞ

FðtÞ

F  ðsÞ

1 t t n ðn ¼ 1; 2; . . .Þ ta ða [ 1Þ eat ða [ 0Þ teat ða [ 0Þ tn eat ða [ 0Þ

1 1=s n!=sn Cða þ 1Þ=sa s=ðs þ aÞ s=ðs þ aÞ2 sn!=ðs þ aÞnþ1

tb eat ða [ 0; b [ 1Þ cos at sin at log t

sCðb þ 1Þ=ðs þ aÞbþ1 s2 =ðs2 þ a2 Þ sa=ðs2 þ a2 Þ C  log sy

y C Euler’s constant and C = 0.577215…

then lim sa F  ðsÞ ¼ C; s!0

where CðaÞ 

R1 0

ðA:11Þ

ta1 et dt is a gamma function defined in (2.13).

TheoremR A.2 (A Tauberian Theorem) If FðtÞ increases and its LS transform 1 F  ðsÞ ¼ 0 est dFðtÞ exists for ReðsÞ [ 0, and if lim sa F  ðsÞ ¼ C s!0

for a  0;

ðA:12Þ

then lim

t!1

FðtÞ C ¼ : ta Cða þ 1Þ

ðA:13Þ

The readers refer to Example 4.11, Problems 4.5, and Sect. 8.2.1 as applications to differential equations of LS transforms.

Appendix A: Laplace Transform

223

In general, it is very difficult to making the inversion of LS transforms. However, with rapid progress of personal computers and a significant development of Laplace inversion methods, it is not so difficult to compute numerically some probabilities and other quantities in reliability models. Tijms [2] summarized some useful methods of Laplace inversion (see Sect. 3.2).

A.4 Generating Function LS transforms are used for continuous functions, while generating functions [2, p. 449, 5, p. 446] are used for discrete probability distributions and functions with variables n (n ¼ 0; 1; 2; . . .; or n ¼ 1; 2; . . .Þ as shown in Sects. 3.5.2, 4.1.2 and 8.3. Let an (n ¼ 0; 1; 2; . . .) be a sequence of real number. If a ðzÞ 

1 X

an z n

ðA:14Þ

n¼0

exists for jzj  1, then a ðzÞ is called the generating function of the sequence  fan g1 n¼0 . The discrete function can be recovered from a0 ¼ a ð0Þ and  1 dn a ðzÞ an ¼ n! dzn z¼0

ðn ¼ 1; 2; . . .Þ:

ðA:15Þ

When pn (n ¼ 0; 1; 2) is a probability function of random variable X,  1 X dp ðzÞ EfXg ¼ npn ¼ ; dz z¼1 n¼1  1 X d2 p ðzÞ nðn  1Þpn ¼ ; EfXðX  1Þg ¼ dz2 z¼1 n¼1 VfXg ¼ EfX 2 g  ðEfXgÞ2 : For example, when a probability distribution is geometric distribution, i.e., pn ¼ pqn1 (n ¼ 1; 2; . . .), the generating function is p ðzÞ ¼

1 X

zn pqn1 ¼

n¼1

pz ; 1  qz

and    dp ðzÞ p  ¼   2 dz z¼1 ð1  qzÞ 

z¼1

1 ¼ ; p

  d2 p ðzÞ 2pq  ¼  dz2 z¼1 ð1  qzÞ3 

z¼1

¼

2q : p2

224

Appendix A: Laplace Transform

Thus, the mean and variance of a geometric distribution are 1 2q 1 1 q EfXg ¼ ; VfXg ¼ 2 þ  2 ¼ 2 : p p p p p P The sequence cn ¼ nk¼0 ak bnk (n ¼ 0; 1; 2; . . .) where the generating functions of fan g and fbn g are a ðzÞ and b ðzÞ for jzj  1, respectively, the generating function of fcn g is c ðzÞ 

1 X n¼0

zn

n X

ak bnk ¼ a ðzÞb ðzÞ

k¼0

for jzj  1, that corresponds to the property of (A.9) in continuous case. For example, when both probability functions are Poisson distributions, i.e., an ¼ ½ðk1 tÞn =n!ek1 t and bn ¼ ½ðk2 tÞn =n!ek2 t ðn ¼ 0; 1; 2; . . .Þ, their generating functions are a ðzÞ  ek1 tð1zÞ ;

b ðzÞ  ek2 tð1zÞ ;

and hence, c ðzÞ ¼ exp½ðk1 þ k2 Þtð1  zÞ; that shows the properties of superposition and decomposition of a Poisson process as shown in Examples 2.8 and 2.9, respectively. Abelian and Tauberian theorems are rewritten in discrete case [5 p. 447, 6, p. 418]: Theorem A.3 (An Abelian Theorem) If limn!1 an ¼ a, then limð1  zÞ z!1

1 X

an zn ¼ a:

n¼0

Theorem A.4 (A Tauberian Theorem) If limz!1 ð1  zÞ limn!1 nðan  an1 Þ ¼ 0, then

P1

n¼0

an zn ¼ a and

lim an ¼ a:

n!1

Theorem A.5 Let fan g1 n¼0 be a nonnegative sequence of real numbers with generating function in (A.14). For constant a and b, 1 X

an ¼ a

n¼0

lim

m!1

m 1X an m n¼0

,

lim a ðzÞ ¼ a;

,

limð1  zÞa ðzÞ ¼ c:

z!1

! ¼c

z!1

Extending such a generating function, the universal generating function is introduced and is applied to reliability analysis of multi-state systems [7–9].

Appendix B: Answers to Selected Problems

Chapter 2 2.1 From Sn has a gamma distribution with order n in (2.12) [10, p. 84,], EfgðSn Þg ¼

Z1

gðtÞd PrfSn  tg ¼

0

Z1 gðtÞ

kðktÞn1 kt e dt: ðn  1Þ!

0

Hence, E

( 1 X

) gðSn Þ

1 Z X

1

¼

n¼1

n¼1

kðktÞn1 kt gðtÞ e dt ¼ k ðn  1Þ!

0

Z1 gðtÞdt: 0

2.2 See [11, p. 24]. 2.3 EfXð1Þ g ¼

Z1

enkt dt ¼

1 ; nk

0

EfXðnÞ g ¼

Z1



0

n  1X 1 : 1  ð1  enkt Þ dt ¼ k k¼1 k a

2.5 Compute numerically when FðtÞ ¼ 1  et [12], MTTF ¼

Z1



 a 1  ð1  et Þn dt;

0

and compare it with

Pn

k¼1

1=k

1=a

and ðC þ log nÞ1=a .

225

226

Appendix B: Answers to Selected Problems

2.6

From [13, p. 9], an optimum number n (1  n \1) is given by a finite and unique minimum that satisfies nþ1 X nþ1

kþ1 k¼1 2.7

From [13, p. 12], MTTF ¼



n  Z X n j¼k

j

c2 c1

1

ðn ¼ 1; 2; . . .Þ:

ejkt ð1  ekt Þnj dt ¼

0

n 1X 1 : k j¼k j

2.8 From [11, p. 286], let Ik ¼ 1 if there is a record at time t, Ik ¼ 0 otherwise. Then, because PrfIk ¼ 1g ¼ ðk  1Þ!=k! ¼ 1=k; n n n X X X 1 Ik ; EfNn g ¼ EfIk g ¼ Nn ¼ ; k k¼1 k¼1 k¼1  n X 1

 1 VarfIk g ¼ 1 : VarfNn g ¼ k k k¼1 k¼1 n X

When n ¼ 10; 100; 1000, respectively, n X 1 k¼1

k

¼ 2:92897;

5:18738;

7:48547:

2.9 See [1, p. 40]. 2.11 The number of spare units is given a minimum number that satisfies n X ðk1 þ k2 Þk k¼0

k!

eðk1 þk2 Þ  a:

2.12 Compute the probabilities, for p ¼ 0:90; 0:10 and kt ¼ 1; 2; 3; ðpktÞk pkt e k!

ðk ¼ 0; 1; 2; 3Þ:

2.13 From the inequality CðN þ 1Þ  CðNÞ  0; B NðN þ 1Þ  ; A

i.e.,

N X k¼1

k

B : 2A

2.14 Consider the multiple backward times Tk (k ¼ 1; 2; . . .; N), where Sk  Pk j¼1 Tj , S0  0 and SN  t [14]. If we cannot find the actual failure time until time Sk1 from time t by the backward operation, we execute again the next backward time with Tk . If we can find the failure time, i.e., the system fails in ðt  Sk ; t  Sk1 , then the process ends. Finally, if we cannot find the failure time until SN1 , we make the final backward with TN ¼ t  SN1 , and can find it certainly.

Appendix B: Answers to Selected Problems

227

It is assumed that c1 þ c2 Tk is the cost for the kth backward. When the failure time is detected at the kth backward, the expected cost is 1 ðc1 k þ c2 Sk Þ½Fðt  Sk1 Þ  Fðt  Sk Þ; FðtÞ and hence, the total expected cost is CðT1 ; T2 ; . . .; TN Þ ¼

N1 1 X ðc1 k þ c2 Sk Þ½Fðt  Sk1 Þ  Fðt  Sk Þ: FðtÞ k¼1

We find optimum times Tk ðk ¼ 1; 2; . . .; NÞ that minimize CðT1 ; T2 ; . . .; TN Þ for a given t [ 0: Differentiating CðT1 ; T2 ; . . .; TN Þ with respect to Sk and setting it equal to zero, Skþ1  Sk ¼

Fðt  Sk1 Þ  Fðt  Sk Þ c1  c2 f ðt  Sk Þ

ðk ¼ 1; 2; . . .; N  1Þ:

Setting that xk  t  Sk ; xk  xkþ1 ¼

Fðxk1 Þ  Fðxk Þ c1  c2 f ðxk Þ

ðk ¼ 1; 2; . . .; N  1Þ;

ðB:1Þ

where xN  0 and x0  t. Then, the total expected cost is Cðx1 ; x2 ; . . .; xN Þ ¼

N 1 X ½c1 k þ c2 ðt  xk Þ½Fðxk1 Þ  Fðxk Þ: FðtÞ k¼1

ðB:2Þ

Therefore, we solve the simultaneous equations in (B.1) for given N  1 and compute the total expected cost in (B.2). Finally, comparing the total expected cost for N  1, we can get an optimum number N  and backward times Sk ¼ t  xk (k ¼ 1; 2; . . .; N  ). 2.15 t k1   Z j k1   X X u u nj kt n n j!ðnjÞ! EfSk jNðtÞ ¼ ng ¼ 1 du ¼ t ¼ ; j j ðnþ1Þ! nþ1 t t j¼0 j¼0 0

EfSk  Sk1 jNðtÞ ¼ ng ¼

kt ðk  1Þt t  ¼ : nþ1 nþ1 nþ1

Compute when k ¼ 0; 1; 2;  PrfNð10Þ ¼ kjNð100Þ ¼ 10g ¼ PrfS3  50jNð100Þ ¼ 10g ¼

10



k 10 X 10 k¼3

k

10 100 

k 

50 100

1

10 100

10k ;

k   50 10k 1 ¼ 0:945: 100

228

Appendix B: Answers to Selected Problems

2.16 Differentiate ½HðtÞn =n! with t and integrate it from 0 to t, where note that Hð0Þ  0. R1 2.17 Noting that l ¼ 0 P0 ðtÞdt; 1 PN1 R 1 N 1 Z X 1 k¼1 0 Pk ðtÞdt R1 ½Pk ðtÞ  PN ðtÞdt [ 0;  ðN  1Þ ¼ R 1 0 PN ðtÞdt 0 PN ðtÞdt k¼1 0

R1

because when hðtÞ increases strictly, 0 Pk ðtÞdt decreases strictly from (2.44) for N  2. 2.19 Setting that KðtÞ  1  HðtÞ=a, solve the differential equation by using the separation of variables method. 2.20 From (2.57),  d exp½kts=ðs þ hÞ kt EfZðtÞg ¼   ¼ h; ds s¼0   2 d exp½kts=ðs þ hÞ kt 2kt VfZðtÞg ¼  ¼ 2;  2 ds h h s¼0

and from (2.60), 1 EfYg ¼ ð1 þ hKÞ; k  2 1 1 X 2X ðhKÞj hK 1 2hK þ 1 ðk þ 1Þ  ð1 þ hKÞ ¼ : VfYg ¼ 2 e j! k k k¼0 k2 j¼k

Chapter 3 3.2

Considering that the inverse LS transform of 1=½ðs þ aÞ2 þ b2  is given by " pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi  # a2 þ b2 at 1 1 b e sin bt þ tan 1 ; b a2 þ b2 a derive when k ¼ 3; 4; respectively, M  ðsÞ ¼ M  ðsÞ ¼

k3 2

2

s½ðs þ 3k=2Þ þ 3k =4

¼

k4 sðs þ 2kÞ½ðs þ kÞ2 þ k2 

k ks þ 3k2 ;  3s 3½ðs þ 3k=2Þ2 þ 3k2 =4

¼

k k k2   : 4s 4ðs þ 2kÞ 2½ðs þ kÞ2 þ k2 

3.3 Use the LS transform of a gamma distribution in (2.14).

Appendix B: Answers to Selected Problems

229 a

Table B.1 MðtÞ and t=EfXg when FðtÞ ¼ 1  eðktÞ and k ¼ 1 t a

1 2 3 4 5 6 7 8 9 10 10/E{X}

3.4 3.5

1.0

1.5

2.0

2.5

3.0

1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 10.00

0.84 1.95 3.05 4.16 5.27 6.38 7.49 8.59 9.70 10.81 11.08

0.75 1.89 3.02 4.15 5.28 6.41 7.54 8.86 9.79 10.92 11.28

0.70 1.84 2.97 4.10 5.23 6.35 7.48 8.61 9.74 10.86 11.27

0.67 1.80 2.93 4.05 5.17 6.29 7.41 8.53 9.65 10.77 11.20

Table B.1 is computed. From Example 2.4, FðtÞ ¼ ð1  ekt Þ2 , and hence, Z1 2k2  ; est dFðtÞ ¼ F ðsÞ ¼ ðs þ kÞðs þ 2kÞ 0

M  ðsÞ ¼ 3.7

F  ðsÞ 2k2 ¼ : 1  F  ðsÞ s2 þ 3ks

Use the inverse LS transform in Table A.2. pffiffiffi n  10 þ 1:64 2 ¼ 12:32 for a ¼ 0:95; pffiffiffi n  10 þ 2:33 2 ¼ 13:29 for a ¼ 0:99:

3.8

From (3.25), 2 3 Z1 Zt 4FðxÞ þ Fðx  uÞdMðuÞ5dx t

0

¼

Z1

FðxÞdx 

0

¼ltþ

Zt

2 3 Z1 Z t FðxÞdx þ 4 Fðx  uÞdMðuÞ5dx

2 t 0 3 Z t Z1 Ztu FðxÞdx þ 4 FðxÞdx  FðxÞdx5dMðuÞ 0

Zt 0

¼ l  t þ lMðtÞ þ

0

Zt 0

¼ l½1 þ MðtÞ  t:

0

FðxÞdx 

0

Zt 0

Mðt  xÞFðxÞdx

230

Appendix B: Answers to Selected Problems

From (3.26), 1 l

Z1

1 tFðtÞdt ¼ 2l

0

Z1

1 FðtÞdt ¼ 2l 2

0

Z1

t2 dFðtÞ ¼

l2 þ r2 : 2l

0

3.9 From [3, p. 128] and Fig. B.1, PrfbðtÞ  xg ¼ PrfXNðtÞþ1  xg ¼

Zt

½FðxÞ  Fðt  uÞdMðuÞ:

tx

Thus, by Theorem 3.6, 1 lim PrfbðtÞ  xg ¼ t!1 l

Zx

1 ½FðxÞ  FðuÞdu ¼ l

0

Zx udFðuÞ: 0

3.10 Let NA ðtÞ and NB ðtÞ be the number of replacements of units for the age and block replacements with planned replacement time T in ½0; t of Sect. 3.3.2. Then, from [16, p. 67], EfNB ðtÞg  EfNA ðtÞg; lim

t!1

EfNB ðtÞg MðTÞ þ 1 EfNA ðtÞg 1 ¼  lim ¼ RT : t!1 t T t FðtÞdt 0

Thus, t  1: 0 FðuÞdu

MðtÞ  R t

e A ðtÞ and N e B ðtÞ be the number of failures of units for the age and Next, let N block replacements in ½0; t. Then, from [16, p. 69], when FðtÞ is IFR, e A ðtÞg; e B ðtÞg  Ef N Ef N e B ðtÞg MðTÞ e A ðtÞg Ef N Ef N FðTÞ lim ¼  lim ¼ RT : t!1 t!1 t T t FðtÞdt 0

Fig. B.1 Process of the life in a renewal process

Appendix B: Answers to Selected Problems

231

Thus, tFðtÞ : 0 FðuÞdu

MðtÞ  R t 3.12 1 lim T!0 FðTÞ

ZT

1 ; FðtÞdt ¼ hð0Þ

1 lim T!1 FðTÞ

0

ZT

FðtÞdt ¼ l;

0

where if hðtÞ ! 0 as t ! 0 then 1=hð0Þ is assumed to be infinity. In addition, when FðtÞ is IFR, 0 @ 1 FðTÞ

ZT

10 FðtÞdtA ¼

0

1 FðTÞ½FðTÞ2

" RT 0

FðtÞdt

FðTÞ

RT 0

FðtÞdt

#  hðTÞ  0:

Similarly, 1 lim T!0 FðTÞ

ZT

FðtÞdt ¼ 0;

1 lim ¼ T!1 FðTÞ

0

Clearly,

RT 0

ZT

FðtÞdt ¼ 1:

0

FðtÞdt=FðTÞ increases even when FðtÞ is not IFR.

3.13 AðTÞ þ

ZT

BðT  tÞdFðtÞ

0

¼ c1 FðTÞ þ c2 FðTÞ þ

ZT

½c1 MðT  tÞ þ c2 dFðtÞ

0

¼ c1 MðTÞ þ c2 ; AðTÞ þ

ZT

AðT  tÞdMðtÞ

0

¼ c1 FðTÞ þ c2 FðTÞ þ

ZT

½c1 FðT  tÞ þ c2 FðT  tÞdMðtÞ

0

¼ c1 MðTÞ þ c2 : 3.14 When T0 has an exponential distribution ð1  eat Þ, from (3.67),

232

Appendix B: Answers to Selected Problems

RðT; x; aÞ ¼

R Tþx

eat FðtÞdt : RT 1  G ðaÞ þ aG ðaÞ 0 eat FðtÞdt aeax

x

Clearly, Rð0; x; aÞ ¼ 0;

R1 aeax x eat FðtÞdt Rð1; x; aÞ ¼ : 1  G ðaÞF  ðaÞ

Differentiating RðT; x; aÞ with respect to T and setting it equal to zero, 2 3 ZT kðT; xÞ41  G ðaÞ þ aG ðaÞ eat FðtÞdt5 0

 aG ðaÞ

ZT

eat ½FðtÞ  Fðt þ xÞdt ¼ 1  G ðaÞ;

ðB:3Þ

0

where kðt; xÞ 

Fðt þ xÞ  FðtÞ : FðtÞ

Note that kðt; xÞ and hðtÞ have the same property. Letting LðTÞ be the lefthand side of (B.3), Lð0Þ ¼ FðxÞ½1  G ðaÞ  1  G ðaÞ; Lð1Þ ¼ ½kð1; xÞ  1½1  G ðaÞF  ðaÞ Z1   ax þ 1  G ðaÞ þ aG ðaÞe eat FðtÞdt: x

If the failure rate increase strictly, then 2 L0 ðTÞ ¼ k0 ðT; xÞ41  G ðaÞ þ aG ðaÞ

ZT

3 eat FðtÞdt5 [ 0:

0

Thus, we have the following optimum policy: (i) If Lð1Þ [ 1  G ðaÞ then there exists a finite and unique T  ð0\T  \1Þ that satisfies (B.3), and the interval reliability is RðT  ; x; aÞ ¼

FðT  þ xÞ : G ðaÞFðT  Þ

(ii) If Lð1Þ  1  G ðaÞ then T  ¼ 1.

Appendix B: Answers to Selected Problems

233

3.16 From Example 3.5, when a ¼ 0:95 and 0:99, respectively, d  909:09  102:56 1:64 ¼ 740:89; d  909:09  102:56 2:33 ¼ 670:13: pffi 3.17 When GðtÞ ¼ 1  eh t , (3.84) is pffiffiffi

alpffiffiffiffi 1 hl T  1  eh T ¼ ; c1 h 2

ðB:4Þ

and when cr ðtÞ ¼ at2 and GðtÞ ¼ 1  eht , from (3.82), T

1  ehT c1 h ¼ : hðlh þ 1Þ 2a

ðB:5Þ

In both cases, there exist optimum T  that satisfies (B.4) and (B.5). 3.18 From [17, p. 156], the LS transform of an exponential distribution Fk ðtÞ ¼ ð1  ekk t Þ is kk =ðs þ kk Þ, and hence, the LS transform of the nth convolution F ðnÞ ðtÞ of Fk ðtÞ is F

ðnÞ

ðsÞ ¼

Z1

est dF ðnÞ ðtÞ ¼

0

n Y kk : s þ kk k¼1

Inverting F ðnÞ ðsÞ, ðnÞ

F ðtÞ ¼ 1 

n X k¼1

! kj ekk t : k  k j k j¼1;j6¼k n Y

In particular, when kk ¼ ak1 k, ðnÞ

F ðtÞ ¼ 1 

n X k¼1

! 1 expðak1 ktÞ: kj 1  a j¼1;j6¼k n Y

3.19 The mean time to replacement is N X l lða  aNþ1 Þ ¼ : k1 a a1 k¼1

3.20 The mean operating and repair times until the unit is replaced at the Nth failure are, respectively, N X l ; k1 a k¼1

N 1 X b : k1 b k¼1

234

Appendix B: Answers to Selected Problems

3.21  pz p 1 1 ¼  ð1  zÞ½1  ðb  pÞz p þ a 1  z 1  ðb  pÞz 1 p X ¼ ½1  ðb  pÞn zn ; p þ a n¼0  pz 1  bz p a p z ¼ þ 1  zð1  zÞ½1  ðb  pÞz p þ a 1  z 1  ðb  pÞz 1  z 1 1 X p X ½a þ pðb  pÞn zn zk ¼ p þ a n¼0 k¼1  1  p X p½1  ðb  pÞn  n ¼ na þ z : p þ a n¼0 pþa 3.22 PrfXnþ1  xjSn ¼ tg ¼

Fðx þ atÞ  FðatÞ : 1  FðatÞ

Chapter 4 4.1 P Pn ¼



1a b

a 1b



   b a 1 ð1  a  bÞn a þ aþb aþb b a b

¼ Pnþ1 :   1a b þ að1  a  bÞn a½1  ð1  a  bÞn  pðnÞ P ¼ ; aþb aþb b

a b 

a 1b

¼ pðn þ 1Þ: 4.2

P¼

1 2 .. . N1 N

(

1 r1 r2 .. .

2 1  r1 0 .. .

3 0 1  r2 .. .

... ... ...

N1 0 0 .. .

N 0 0 .. .

rN1 1

0 0

0 0

... ...

0 0

1  rN1 0

)

:



Appendix B: Answers to Selected Problems

From p ¼ pP, p1 ¼

N X

235

pj r j ;

j¼1

pjþ1 ¼ pj ð1  rj Þ

ðj ¼ 1; 2; . . .; N  1Þ:

Thus, Pj1 ð1  ri Þ i; pj ¼ P hi¼1 N j1 j¼1 Pi¼1 ð1  ri Þ where rN  1 and P0i¼1  1; 4.4 Let FðtÞ be the identical failure probability of each unit at time t. Then, the probability that k units have failed and n  k units have not failed independently at time t is   n ½FðtÞk ½1  FðtÞnk ðk ¼ 0; 1; 2; . . .; nÞ: k 4.5 For one repair person, the LS transforms of the differential equations are ðs þ kÞP00 ðsÞ ¼ s þ hP01 ðsÞ; ðs þ k þ hÞP01 ðsÞ ¼ kP00 ðsÞ þ hP02 ðsÞ; ðs þ hÞP02 ðsÞ ¼ kP01 ðsÞ: Solving these equations for P0i ðsÞ ði ¼ 0; 1; 2Þ and inverting them,   h2 k x1  k  h x1 t x2  k  h x2 t ; þ e  e P00 ðtÞ ¼ 2 x1 x2 k þ kh þ h2 x1  x2   kh k x1 þ h x1 t x2 þ h x2 t P01 ðtÞ ¼ 2 þ e  e ; x1 x2 k þ kh þ h2 x1  x2   k2 k2 1 x1 t 1 x2 t þ e  e P02 ðtÞ ¼ 2 ; x2 k þ kh þ h2 x1  x2 x1 where x1  k þ h þ

pffiffiffiffiffi kh;

x2  k þ h 

pffiffiffiffiffi kh:

For two repair persons,   2h2 k x1  k  2h x1 t x2  k  2h x2 t ; þ e  e x1 x2 k2 þ 2kh þ 2h2 x1  x2   2kh k x1 þ 2h x1 t x2 þ 2h x2 t ; þ e  e P01 ðtÞ ¼ 2 2 x1  x2 x1 x2 k þ 2kh þ h   k2 k2 1 x1 t 1 x2 t P02 ðtÞ ¼ 2 ; þ e  e x2 k þ 2kh þ 2h2 x1  x2 x1 P00 ðtÞ ¼

236

Appendix B: Answers to Selected Problems

where

pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

1 2k þ 3h þ 4kh þ h2 ; 2 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

1 x2  2k þ 3h  4kh þ h2 : 2

x1 

Chapter 5 5.1 F0 ðsÞ ¼ F  ðsÞ½aG1 ðsÞ þ ð1  aÞG2 ðsÞ; aF  ðsÞ ð1  aÞF  ðsÞ  ; F ; ðsÞ ¼ 2 1  ð1  aÞF  ðsÞG2 ðsÞ 1  aF  ðsÞG1 ðsÞ Fj ðsÞ½1  Gj ðsÞ 1  F  ðsÞ  P0 ðsÞ ¼ ðsÞ ¼ ; P ðj ¼ 1; 2Þ; j 1  F0 ðsÞ 1  Fj ðsÞGj ðsÞ

F1 ðsÞ ¼

M0 ðsÞ ¼

F0 ðsÞ ; 1  F0 ðsÞ

Mj ðsÞ ¼

Fj ðsÞ 1  Fj ðsÞGj ðsÞ

ðj ¼ 1; 2Þ:

5.2 Let Qij be the probability that when the system is in State i ði ¼ 0; 1Þ; the data transmission has failed at j times (j ¼ 1; 2; . . .; N) consecutively. Then, using (5.15), we have the following equations related to Qij [13, p. 116]: Q0j ¼ P00 ðT1 Þp0 Q0j1 þ P01 ðT1 Þp1 Q1j1 þ P02 ðT1 Þpj2 ; Q1j ¼ P10 ðT1 Þp0 Q0j1 þ P11 ðT1 Þp1 Q1j1 þ P12 ðT1 Þpj2 : Forming the generating functions of Q0j and Q1j , solving them, and inverting their generating functions, 1  fx1  p1 ½P11 ðT1 Þ  P01 ðT1 Þgx1j x1  x2   fx2  p1 ½P11 ðT1 Þ  P01 ðT1 Þgx2j

Q0j ¼

þ

p2 x1  x2

fx1 P02 ðT1 Þ  p1 ½P11 ðT1 ÞP02 ðT1 Þ  P01 ðT1 ÞP12 ðT1 Þg

x1j  p2j  fx2 P02 ðT1 Þ  p1 ½P11 ðT1 ÞP02 ðT1 Þ  P01 ðT1 ÞP12 ðT1 Þg x1  p2 ! x2j  p2j ðj ¼ 0; 1; 2; . . .; NÞ; x2  p2

Appendix B: Answers to Selected Problems

237

where

 1 p0 P00 ðT1 Þ þ p1 P11 ðT1 Þ 2 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi þ ½p0 P00 ðT1 Þ  p1 P11 ðT1 Þ2 þ 4p0 p1 P01 ðT1 ÞP10 ðT1 Þ ;  1 x2  p0 P00 ðT1 Þ þ p1 P11 ðT1 Þ 2 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi  ½p0 P00 ðT1 Þ  p1 P11 ðT1 Þ2 þ 4p0 p1 P01 ðT1 ÞP10 ðT1 Þ :

x1 

Thus, the mean time l3 ðNÞ is PN1 T1 j¼0 Q0j þ T2 Q0N l3 ðNÞ ¼ 1  Q0N

ðN ¼ 1; 2; . . .Þ:

From l3 ðN þ 1Þ  l3 ðNÞ  0,

Q0N

N 1 X Q0N T2 ð1  Q0N Þ  Q0j  :  Q0Nþ1 T1 j¼0

ðB:6Þ

Let LðNÞ be the left-hand side of (B.6). If Q0jþ1 =Q0j increases strictly, then LðNÞ increases strictly. Therefore, if Lð1Þ [ T2 =T1 , then there exists a finite and unique minimum N  ð1  N  \1Þ that satisfies (B.6). 5.3 fi;ij is the probability of first-passage from State i to State j without returning to State i, and fj;ii is the probability of returning from State i to State i without passing through State j, and hence, fi;ij þ fj;ii ¼ 1. 5.4 From (5.44) and (5.45), ðn þ 1Þðc0  c1 Þ þ c2   lCðnÞ ¼ þ c1 : Pnþ1 n þ 1 C j1 j¼1 j Thus, because  nþ1  n X n   nþ1 n 1 X 1 1X 1   0; n þ 1 j¼1 Cj1 n j¼1 j¼1 j Cj1 j c0 lim CðnÞ ¼ ; n!1 l CðnÞ decreases to c0 =l. 5.6 CðTÞ ¼ c1 M4 þ c2 M1 þ c3 ðP3 þ P4 Þ þ c4 P2 R TþL R1 c1 FðTÞ þ c2 FðTÞ þ c3 T FðtÞdt þ c4 TþL FðtÞdt : ¼ R TþL l þ T FðtÞdt

238

Appendix B: Answers to Selected Problems

Clearly, Cð0Þ ¼

c 2 þ c3

RL

R1 FðtÞdt þ c4 L FðtÞdt ; RL l þ 0 FðtÞdt

0

Cð1Þ ¼

c1 þ c3 L : lþL

Differentiating CðTÞ with respect to T and setting it equal to zero, 8 9 2 3 ZT < = LðTÞ  kðT; LÞ c3 l þ c4 4L þ FðtÞdt5  c1 FðTÞ  c2 FðTÞ : ; 0 2 3 ZTþL þ ½ðc1  c2 ÞhðTÞ  c4 4l þ FðtÞdt5 ¼ 0;

ðB:7Þ

T

where hðtÞ  f ðtÞ=FðtÞ and kðt; LÞ  ½Fðt þ LÞ  FðtÞ=FðtÞ. If hðtÞ increases strictly, then kðt; LÞ increases strictly with t, and hence, LðTÞ increases strictly with T from Lð0Þ to Lð1Þ, where 2 Lð0Þ ¼ FðLÞðc3 l þ c4 L  c2 Þ þ 4l þ

ZL

3 FðtÞdt5½ðc1  c2 Þhð0Þ  c4 ;

0

Lð1Þ ¼ kð1; LÞðc3 l  c1 Þ þ ðl þ LÞfðc1  c2 Þhð1Þ  c4 ½1  kð1; LÞg: Therefore, we have the following optimum policy: (i) If Lð0Þ  0 then T  ¼ 0, i.e., the order is made at the same time as the beginning of operation. (ii) If Lð0Þ\0 and Lð1Þ [ 0 then there exists a finite and unique T  ð0\T  \1Þ that satisfies (B.7). (iii) If Lð1Þ  0 then T  ¼ 1.

Chapter 6 6.3 From Problem 3.7 and 3.16, when a ¼ 0:95, 0:99, respectively, z ¼ 15  5:12 1:64 ¼ 6:60; z ¼ 15  5:12 2:33 ¼ 3:07; PrfZð30Þ [ 15g ¼ 0:500;

PrfZð30Þ [ 20g ¼ 0:164:

Appendix B: Answers to Selected Problems

239

6.6 1 1 ðhKÞnþ1 X ðhKÞk ðhKÞn X ðhKÞk  ðn þ 1Þ! k¼n k! n! k¼nþ1 k! " # n X 1 1 X ðhKÞ ðhKÞkþ1 ðhKÞk ¼  ðn þ 1Þ [ 0: ðn þ 1Þ! k¼n k! k! k¼nþ1

6.7

Differentiating (6.25) with respect to T, 0

0

½ðcK  cT Þr ðTÞ þ cM h ðTÞ

1 X n¼0

6.8

ðnÞ

G ðKÞ

ZT

½F ðnÞ ðtÞ  F ðnþ1Þ ðtÞdt:

0

From [17, p. 62], the probability that the system is replaced after failure is N   k   X X 1 n k Pn ¼ ; ð1Þk pnk ð1Þi k i 1  qni i¼0 k¼0 the probability that it is replaced before failure, i.e., the total number of failed units is N þ 1; N þ 2; . . .; n  1, is n1   k   X X 1 n k k nk ; PN ¼ ð1Þ p ð1Þi k i 1  qni i¼0 k¼Nþ1 and the mean time to replacement is  N   N k  X X 1 n nk lNþ1 ¼ l ð1Þk k i 1  qni i¼0 k¼0

ðN ¼ 0; 1; 2; . . .; n  1Þ:

Thus, the expected cost rate is CðNÞ 

cn Pn þ cN PN lNþ1

P   P   cN þ ðcn  cN Þ Nk¼0 nk ð1Þk pnk ki¼0 ki ð1Þi ½1=ð1  qni Þ ¼ P   PNk  nk  l Nk¼0 nk ð1Þk i¼0 ½1=ð1  qni Þ i ðN ¼ 0; 1; 2; . . .; n  1Þ; where cn ¼ replacement cost of a failed system and cN ¼ replacement cost before system failure with cN \cn . When the number n of units is given, we can obtain an optimum number n that minimizes CðNÞ.

240

Appendix B: Answers to Selected Problems

6.10 Differentiating the left-hand side of (6.44) with respect to T, r0 ðTÞ

1 X

GðnÞ ðKÞ

n¼0

ZT

½F ðnÞ ðtÞ  F ðnþ1Þ ðtÞdt:

0

From (6.48), rNþ2

N N1 X X GðnÞ ðKÞ  ½1  GðNþ1Þ ðKÞ  rNþ1 GðnÞ ðKÞ þ ½1  GðNÞ ðKÞ n¼0

n¼0

¼ ðrNþ2  rNþ1 Þ

N X

GðnÞ ðKÞ:

n¼0

6.11 When cT ¼ cZ and shocks occur in a Poisson process, i.e., F ðnÞ ðtÞ ¼ P1 j kt , from (6.40), j¼n ½ðktÞ =j!e C5 ðT; ZÞ  lim CðT; N; ZÞ N!1 RZ P ðnþ1Þ cT þ ðcK  cT Þ 1 ðTÞ 0 GðK  xÞdGðnÞ ðxÞ n¼0 F P ¼ : ðnþ1Þ ðTÞGðnÞ ðZÞ ð1=kÞ 1 n¼0 F Differentiating C5 ðT; ZÞ with respect to T and setting it equal to zero, QðT; ZÞ

1 X

F ðnþ1Þ ðTÞGðnÞ ðZÞ

n¼0



1 X

F

ðnþ1Þ

ðTÞ

n¼0

ZZ

GðK  xÞdGðnÞ ðxÞ ¼

cT ; cK  c T

ðB:8Þ

0

and differentiating C5 ðT; ZÞ with respect to Z and setting it equal to zero, GðK  ZÞ

1 X

F ðnþ1Þ ðTÞGðnÞ ðZÞ

n¼0



1 X

F

ðnþ1Þ

ðTÞ

n¼0

ZZ

GðK  xÞdGðnÞ ðxÞ ¼

cT ; c K  cT

ðB:9Þ

0

where P1 QðT; ZÞ 

RZ ðTÞ  F ðnþ1Þ ðTÞ 0 GðK  xÞdGðnÞ ðxÞ P1 : ðnÞ ðnþ1Þ ðTÞGðnÞ ðZÞ n¼0 ½F ðTÞ  F

n¼0 ½F

ðnÞ

Thus, there does not exist both T  ð0\T  \1Þ and Z  ð0\Z  \KÞ that satisfy (B.8) and (B.9) simultaneously, because QðT; ZÞ\GðK  ZÞ. As further studies, when cZ [ cT , obtain both optimum T  and Z  that minimize

Appendix B: Answers to Selected Problems

241

P ðnÞ ðnþ1Þ cK  ðcK  cT Þ 1 ðTÞGðnÞ ðZÞ n¼0 ½F ðTÞ  F P1 ðnþ1Þ  ðcK  cZ Þ n¼0 F ðTÞ RZ 0 ½GðK  xÞ  GðZ  xÞdGðnÞ ðxÞ C5 ðT; ZÞ ¼ RT P1 ðnÞ ðnÞ ðnþ1Þ ðtÞdt n¼0 G ðZÞ 0 ½F ðtÞ  F

:

6.13 Suppose that the unit is replaced at time T or at shock number N, whichever occurs first [17, p. 90]. The probability that the unit is replaced at time T is PN1 ðnÞ F ðTÞ, the probability that it is replaced at shock number N is Pn¼0 1 ðnÞ n¼N F ðTÞ, and the expected number of minimal maintenance is N1 X

½F ðnÞ ðTÞ  F ðnþ1Þ ðTÞ

n¼1

n X

½1  GðiÞ ðKÞ

i¼1

1 N 1 X X ½F ðnÞ ðTÞ  F ðnþ1Þ ðTÞ ½1  GðiÞ ðKÞ þ n¼N

¼

N1 X

i¼1

½1  GðnÞ ðKÞF ðnÞ ðTÞ:

n¼1

The mean time to replacement is T½1  F

ðNÞ

ðTÞ þ

ZT tdF 0

ðNÞ

ðtÞ ¼

ZT

½1  F ðNÞ ðtÞdt:

0

Thus, the expected cost rate is PN1 PN1 ðnÞ cN þ ðcT  cN Þ n¼1 F ðTÞ þ cM n¼1 ½1  GðnÞ ðKÞF ðnÞ ðTÞ : CðT; NÞ ¼ RT ðNÞ ðtÞdt 0 ½1  F

Chapter 7 7.1 Z1 1

f ðx; tÞdx ¼ e

s2 r2 t=2

Z1 1

" # 1 ðx þ sr2 tÞ2 2 2 pffiffiffiffiffiffiffi exp  dx ¼ es r t=2 : 2r2 t 2ptr

242

Appendix B: Answers to Selected Problems

pffiffiffi 7.2 When t ¼ 5, Rð5Þ ¼ Uðð10  5Þ= 5Þ ¼ 0:987, and 2 3  Z1  Z1 Zt pffiffi 2 10  t 1 MTTF ¼ dt ¼ 4 pffiffiffiffiffiffiffi eð10xÞ =2 x dx5dt  10:5: U pffi t 2pt 0

0

0

When t ¼ 10, Rð10Þ ¼ Uð0Þ ¼ 0:500. 2 7.3 Letting LðTÞ be the left-hand side of (7.10), where A  elþr =2 ; Lð0Þ ¼ 0; Lð1Þ ¼ 1 and 

L0 ðTÞ ¼ A2 TeAT [ 0;

CðT  Þ ¼ c1 AeAT :

7.4 Using ea  1 þ a þ a2 =2, from (7.9),   r2 ðl þ r2 =2ÞT c2 e 1þ þ : CðTÞ ¼ c1 l þ 2 T 2 e e e Clearly, Cð0Þ ¼ Cð1Þ ¼ 1. Differentiating CðTÞ with respect to T and setting it equal to zero, rffiffiffiffiffiffiffi 1 2c2 e T ¼ l þ r2 =2 c1 e and T  from (7.10), and compare them, and assure that T  \ T e, Compute T 2 because for T [ 0 and l þ r =2 [ 1; ATeAT  ðeAT  1Þ [

ðATÞ2 : 2

7.5 Letting LðTÞ be the left-hand side of (7.15), Lð0Þ ¼ 0; 0

L ðTÞ ¼

Lð1Þ ¼ lQ1 ð1Þ½1 þ MG ðKÞ  1;

Q01 ðTÞ

1 X

ðkÞ

G ðKÞ

k¼0

ZT

½F ðkÞ ðtÞ  F ðkþ1Þ ðtÞdt:

0

7.6 Letting LðNÞ be the left-hand side of (7.18), Lð1Þ ¼ Q2 ð1Þ½1 þ MGT ðKÞ  1; LðN þ 1Þ  LðNÞ ¼ ½Q2 ðN þ 2Þ  Q2 ðN þ 1Þ

N X n¼0

7.7 Letting LðTÞ be the left-hand side of (7.21), Lð0Þ ¼ 0;

Lð1Þ ¼ 1; ZT 0 0 L ðTÞ ¼ rA ðK=TÞ LA ðK=tÞdt: 0

ðnÞ

GT ðKÞ [ 0:

Appendix B: Answers to Selected Problems

243

7.8 (7.26) and (7.27) are rewritten as, respectively, pffiffiffiffi cK  ðcK  cT Þf1  exp½ðK  xTÞa =r T g CðTÞ ¼ ; RT pffi a 0 f1  exp½ðK  xtÞ =r tgdt pffiffiffiffi ðK  xTÞa1 ½K þ ð2a  1ÞxT exp½ðK  xTÞa =r T  pffiffiffiffi pffiffiffiffi 2rT T 1  exp½ðK  xTÞa =r T  ZT pffiffiffiffi pffi f1  exp½ðK  xtÞa =r tgdt  exp½ðK  xTÞa =r T  0

¼

1 : cK =cT  1

7.9 From (7.29), C2 ð0Þ ¼ 1 and C2 ð1Þ ¼ R 1 0

cK pffi : exp½ðxt  KÞ=r tdt

Differentiating C2 ðTÞ with respect to T and setting it equal to zero, K þ xT pffiffiffiffi 2rT T

ZT

pffiffiffiffi pffi exp½ðxt  KÞ=r tdt þ exp½ðxT  KÞ=r T  ¼

cK ; c K  cT

0

whose left-hand decreases strictly to 0. Hence, T  ¼ 1.

Chapter 8 8.1 When cr ðtÞ ¼ at, (8.12) is [18, p. 53] RT c1 GðTÞ þ a 0 GðtÞdt CðTÞ ¼ : RT l þ 0 GðtÞdt R1 If 0 GðtÞdt  b then Cð1Þ ¼ ab=ðl þ bÞ and C(0)=c1/l. Let rðtÞ  gðtÞ=GðtÞ, where gðtÞ is a density function of GðtÞ. Differentiating CðTÞ with respect to T and setting it equal to zero, 2 3 ZT al rðTÞ4l þ GðtÞdt5 þ GðTÞ ¼ : ðB:10Þ c1 0

244

Appendix B: Answers to Selected Problems

If rðtÞ decreases strictly then the left-hand side of (B.10) decreases from lrð0Þ þ 1 to ðl þ bÞrð1Þ. Therefore, we have the following optimum policy where k

al  c1 ; c1 l

K

al : c1 ðl þ bÞ

(i) If rð0Þ  k then T  ¼ 0. (ii) If rð0Þ [ k and rð1Þ\K then there exists a finite and unique T  ð0\T  \1Þ that satisfies (B.10). (iii) If rð1Þ  K then T  ¼ 1. 8.2 P000 ðtÞ ¼ ðk þ ksÞP00 ðtÞ þ hP01 ðtÞ; P001 ðtÞ ¼ ðk þ ksÞP00 ðtÞ  ðk þ hÞP01 ðtÞ þ hP02 ðtÞ; P002 ðtÞ ¼ kP01 ðtÞ  hP02 ðtÞ: Taking the LS transforms of the above equations, solving and inverting them,  h2 k þ ks x1  k  h x1 t x2  k  h x2 t ; þ e  e P00 ðtÞ ¼ x1 x2 ðk þ hÞðk þ ksÞ þ h2 x1  x2  hðk þ ksÞ k þ ks x1 þ h x1 t x2 þ h x2 t þ e  e P01 ðtÞ ¼ ; x1 x2 ðk þ hÞðk þ ksÞ þ h2 x1  x2  kðk þ ksÞ kðk þ ksÞ 1 x1 t 1 x2 t ; þ e  e P02 ðtÞ ¼ x2 ðk þ hÞðk þ ksÞ þ h2 x1  x2 x1 where qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 x1  ½2k þ 2h þ ks þ 4kh þ k2s ; 2 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 x2  ½2k þ 2h þ ks  4kh þ k2s : 2 8.3 State State State State

0: 1: 2: 3:

One One One One

unit unit unit unit

is is is is

operating and the other is on standby. operating and the other is under repair. operating and the other is under PM. under repair or PM and the other waits for repair.

Appendix B: Answers to Selected Problems

245

From [18, p. 149], the limiting probabilities Pj (j ¼ 0; 1; 2; 3) of transition probabilities are 1 P0 ¼ c11 Pj ¼

1 cjj

ZT 0

ZT

1 P3 ¼ c11

1 G1 ðtÞFðtÞdt þ c22

Gj ðtÞFðtÞdt

ZT

G2 ðtÞFðtÞdt;

0

ðj ¼ 1; 2Þ;

0

ZT 0

1 G1 ðtÞFðtÞdt þ c22

ZT

G2 ðtÞFðtÞdt;

0

where 2

3 R 1 G1 ðtÞdFðtÞ  b1 þ G1 ðtÞFðtÞdt þ 4b2 þ G2 ðtÞFðtÞdt5 TR 1 ; 1  T G2 ðtÞdFðtÞ 0 0 2 3 R1 ZT ZT 1 G2 ðtÞdFðtÞ  b2 þ G2 ðtÞFðtÞdt þ 4b1 þ G1 ðtÞFðtÞdt5 R 1T : G 1 ðtÞdFðtÞ T ZT

c11

c22

ZT

0

0

The availability is AðTÞ  P0 þ P1 þ P2 h i iR  h RT R1 RT 1 c1 þ 0 G1 ðtÞFðtÞdt 1  T G2 ðtÞdFðtÞ þ c2 þ 0 G2 ðtÞFðtÞdt T G1 ðtÞdFðtÞ i iR ¼h ;  h RT R1 RT 1 b1 þ 0 G1 ðtÞFðtÞdt 1  T G2 ðtÞdFðtÞ þ b2 þ 0 G2 ðtÞFðtÞdt T G1 ðtÞdFðtÞ

where ci 

Z1

Gi ðtÞFðtÞdt;

qi 

0

Z1

Gi ðtÞdFðtÞ

ði ¼ 1; 2Þ:

0

Let QðtÞ 

q1 G2 ðtÞ  q2 G1 ðtÞ ; q1  q2

k

q2 ; q1 c2  q2 c1

K

q1 : lðq1  q2 Þ

Suppose that G1 ðtÞ\G2 ðtÞ for 0\t\1 and the failure rate increases strictly.

246

Appendix B: Answers to Selected Problems

(i) If hð1Þ [ K; q1 c2 [ q2 c1 , and hð0Þ\k or hð1Þ [ K and q1 c2  q2 c1 , then there exists a finite and unique T  ð0\T  \1Þ that satisfies 2 T 3 Z1 Z ZT QðtÞFðtÞdt5  QðtÞdFðtÞ hðTÞ4 QðtÞFðtÞdt þ 0

¼

q1

0

R1

QðtÞdFðtÞ þ q2 q1  q2

0

RT 0

0

QðtÞdFðtÞ :

(ii) If hð1Þ\K then T  ¼ 1. (iii) If q1 c2 [ q2 c1 and hð0Þ [ k, then T  ¼ 0. 8.4 When GðtÞ ¼ 1  eht , P j n  lz=ð1  zÞ þ ð1=hÞ 1  n¼1 z ½1  FS ðhÞ  l ðzÞ ¼ 1  ½1  FS ðhÞ=½1  zFS ðhÞ   1 1 X 1 1  FS ðhÞ X n ¼ lþ nz þ l zn ; h FS ðhÞ n¼1 n¼1 z=ð1  zÞ 1  ½1  FS ðhÞ=½1  zFS ðhÞ 1 1 X 1  FS ðhÞ X n nz þ zn ; ¼ FS ðhÞ n¼1 n¼1

m ðzÞ ¼

M¼

1 1 FS ðhÞ : MS ðhÞ ¼ l þ 1=h l þ 1=h 1  FS ðhÞ

8.5 From [19], the total expected cost of one cycle from the beginning of the operation of the main unit to repair or PM completion is 2 FðTÞ4c2 þ cS

Z1

3

2

MS ðtÞdGðtÞ5 þ FðTÞ4c1 þ cS

Z1

0

3 MS ðtÞdGðtÞ5;

0

and the mean time of one cycle is

FðTÞðl þ TÞ þ FðTÞl þ

ZT

tdFðtÞ ¼ l þ

0

where MS ðtÞ 

P1

n¼1

ðnÞ

ZT FðtÞdt; 0

FS ðtÞ. The expected cost rate is CðTÞ ¼

e c 1 FðTÞ þ ec 2 FðTÞ ; RT l þ 0 FðtÞdt

Appendix B: Answers to Selected Problems

247

where e c 1  c1 þ cS

Z1 0

MS ðtÞdGðtÞ;

e c 2  c 2 þ cS

Z1

MS ðtÞdGðtÞ:

0

Using the results in Sect. 3.3.2.1, we can obtain an optimum policy.

References 1. Osaki S (1992) Applied stochastic system modeling. Springer, Berlin 2. Tijms HC (2003) A first course in stochastic models. Wiley, Chichester 3. Beichelt FE, Fatti LP (2002) Stochastic process and their applications. CRC Press, Boca Raton 4. Rausand M, Høyland A (2004) System reliability theory. Wiley, Hoboken 5. Bhat UN, Miller GK (2002) Elements of applied stochastic processes. Wiley, Hoboken 6. Feller W (1968) An introduction to probability theory and its application, vol I, 3rd edn. Wiley, New York 7. Gnedenko B, Ushakov I (1995) Probabilistic reliability engineering. Wiley, New York 8. Levitin L, Lisnianski A (2003) Multi-state systems reliability analysis and optimization. In: Pham H (ed) Handbook of reliability engineering. Springer, London 9. Levitin G (2005) The universal generating function in reliability analysis and optimization, Springer, London 10. Çinlar E (1975) Introduction to stochastic processes. Prentice-Hall, Englewood Cliffs 11. Ross SM (1983) Stochastic processes. Wiley, New York 12. Yun WY, Nakagawa T (2010) Note on MTTF of a parallel system. In: 16th ISSAT International conference on reliability and quality in design, pp 235–237 13. Nakagawa T (2008) Advanced reliability models and maintenance policies. Springer, London 14. Chen M, Mizutani S, Nakagawa T (2010) Optimal backward and backup policies in reliability theory. J Oper Res Soc Jpn 53:101–118 15. Prabhu NU (2007) Stochastic processes. World Scientific, Singapore 16. Barlow RE, Proschan F (1965) Mathematical theory of reliability. Wiley, New York 17. Nakagawa T (2007) Shock and damage models in reliability theory. Springer, London 18. Nakagawa T (2005) Maintenance theory of reliability. Springer, London 19. Nakagawa T, Osaki S (1976) Reliability analysis of a one-unit system with unrepairable spare units and it optimization applications. Oper Res Q 27:1019–110

Index

A Abelian theorem, 221–222, 224 Age, 58–78 Age replacement, 3, 66–70, 88, 175 Allowed time, 83 Alternating renewal process, 3, 47, 73–83, 86, 98, 125, 127–128, 199, 202–203, 211 Automatic repeat request (ARQ), 130–132 Availability, 7, 76, 101, 203, 210

Cumulative hazard, 28–30, 34, 44, 61–63, 72 Cumulative process, 1, 4, 18, 149–179 Cumulative damage, model, 4, 8, 36–41, 149–179, 185, 187–189

B Backup policy, model, 7, 25–26 Backward time, 3, 26–27, 44 Bathtub curve, 10, 33 Bernoulli trial, 67 Binomial distribution, 17, 23–24, 43, 113, 121, 133 Birth process, 112–114, 158 Birth and death process, 112, 114–119 Blackwell’s theorem, 56 Block replacement, 71 Branching process, 5 Brownian motion, 1, 4–5, 18, 149, 183–186, 192

E Elementary renewal theorem, 55, 69 Embedded Markov chain, 125, 132–139 Expected cost, rate, 12, 24–26, 34, 43, 70–73, 83, 85, 88, 140, 147, 160, 170, 172–178, 180, 186–189, 192–194, 196, 201–203, 215–217 Expected number of visits, failures, 102, 108, 110, 124, 130, 136, 138, 147, 208

C Central limit theorem, 57, 152–153 Chapman-Kolmogorov equation, 99, 111, 114, 133 Checkpoint, 25 Communication system, 128–130, 140 Compound Poisson process, 3, 8, 36–41 Computer, system, 5, 11 Counting process, 16, 19, 31, 48, 112, 149, 158

D Degenerate distribution, 67, 77, 145, 155 Doubly stochastic process, 4, 149 Downtime, 4, 79–83

F Failure rate, 7, 9–11, 14, 16–17, 27–28, 30, 33–35, 38, 40, 42, 62, 70, 72, 78, 89, 93, 121, 149, 156–159, 164, 189, 200, 216 Fault tolerance, 2 Finance, 1, 5, 149 First-passage time, distribution, 137, 206–207, 211–212

G Gamma distribution, 14–15, 21, 51–53, 60, 64–65, 91, 168, 200–201

249

250

G (cont.) Gamma function, 16 Gaussian process, 5 Generating function, 87–90, 101, 134–137, 212–214 Geometric distribution, 86 Geometric process, 47, 84–86

H hazard rate, function, 9, 10 harmonic series, 13, 43

I Imperfect maintenance, 28 Imperfect repair, 93, 201 Increasing failure rate (IFR), 10, 12, 33, 43, 62–63, 68, 72, 92, 154, 157, 200 Independent increment, 19, 112, 183, 191 Inspection model, policy, 3, 7, 24–25 Insurance, 1, 5, 149 Intensity function, 30, 154, 158, 200 Intermittent fault, 128–130 Interval reliability, 47, 76–79, 203 Inventory model, cost, 144–147

Index Markov property, 3, 97, 123, 184 Markov renewal process, 4, 73, 123–147, 199, 205 Martingale, 5 Mass function, 125 Mean time to failure (MTTF), 4, 7–13, 38–43, 57, 113, 149, 156, 164, 184, 207, 211, 214 Mean time between failures (MTBF), 64, 86 Mean value function, 30, 33, 154, 158, 160, 163, 200 Memoryless property, 13, 42, 48, 96, 99, 200 Minimal repair, maintenance, 7, 27, 34, 62, 72, 160, 171, 178–180, 200 Minor failure, 3, 44, 95–96

N Negative binomial, 89 Normal distribution, 51, 54, 57, 81, 152, 184, 189 Nonhomogeneous Poisson process, 3, 7, 27, 34, 41, 61, 64, 175

O One-unit system, 2–4, 47, 76, 80, 83–84, 86, 90, 98, 123, 199, 200

J Jump process, 4, 149

K k-out-of-n system, 43

L Laplace transform, 3–74, 184, 196, 219–222 Laplace–Stieltjes (LS) transform, 5, 8, 15, 38, 49–53, 61, 66–67, 74–78, 80, 82, 115–117, 127, 129, 130, 142–148, 154–156, 167, 201, 204–209, 212–214, 219–222 Laplace inversion, 65, 76, 205 L´evy process, 1, 4–5, 18, 149, 158, 183–186 Lifetime, 32

M Maintenance, policy, 2, 4, 7, 47, 51 Major failure, 3, 44, 95 Markov chain, 1, 3–4, 95–119, 124–125, 138–139, 199, 204 Markov process, 1, 3, 119, 124–132

P Parallel system, 3–5, 13, 43, 92, 109–110, 113, 115–118, 120, 142, 163 Periodic replacement, 3, 69, 71–73, 150, 178–179 Poisson distribution, 7, 17, 21, 22, 43, 200, 224 Poisson process, 1–44, 47, 52, 61, 64, 112, 154, 156–159, 163, 166–167, 175, 180, 192, 200–201, 224 Preventive maintenance, 2, 11–12, 28, 77, 200–201

Q Queueing, 1, 5, 124, 149

R Regeneration point, 4, 48, 123, 140, 147 Renewal density, 50, 58, 61, 65 Renewal equation, 3–4, 47, 51, 61, 65–66, 68–69, 71, 74, 123–124, 126, 129–130, 166, 209–212

Index Renewal function, 3–4, 47, 50–53, 61, 64–66, 71, 75, 85, 87, 92, 124–127, 141–143, 152, 201–202, 204, 208, 213 Renewal point, 48, 51, 56, 66–67 Renewal process, 1, 3–4, 47–52, 55–89, 92–93, 98, 124–127, 140–144, 150, 159–160, 163, 170, 199, 201–203, 205–206, 211 Renewal theory, 3, 47, 125 Repair, limit, 2–5, 47, 73, 76–77, 83, 86, 95, 98–99, 117, 132, 186, 199, 202–204, 211 Replacement model, policy, 2, 7, 34, 42, 47, 84, 86, 149–150, 160, 170–178, 186–195, 199, 201–203 Residual life, time, 13, 48, 58–59, 85, 200 Redundant unit, system, 2, 5, 15, 73, 113, 123, 132, 140, 142, 163, 199–216 Reward process, 56

S Sample path, 19 Scheduling, time, 17 Semi-Markov process, 4, 123–125, 133 Series system, 42 Shock, model, 3–4, 8, 36–41, 149–180, 187, 193–196 Software reliability, 7, 36

251 Spare unit, part, 2, 7, 17, 44, 109, 110, 132, 140, 144, 199, 211–217, 226 Standby system, 4, 7, 15, 17–18, 199, 203–205, 211–216 Stationary, increment, 7, 19, 22, 97, 111, 124, 133, 183, 191 Stationary process, 5

T Tauberian theorem, 69, 127, 208, 210, 221–222, 224 Transition probability, 3–4, 74, 75, 95–130, 133–136, 142–146, 199, 204, 210, 216 Two types of failures, 95–96 Two-unit system, 2, 4–5, 92, 109, 115–117, 120–121, 140, 199, 203, 211, 216

U Uniform distribution, 24–25, 60 Up time, 4

W Wear model, 5, 192 Weibull distribution, 32–33, 35, 43, 64, 66, 196 Wiener process, 4, 183