PerforlJ1a~ce
Analysis .of Transaction .. :.Proc~ssin9 Syste~s
PerlOrmanceAnalysh of Transaction' . Processing Systems Wilbur H. Highleyman
Prentice Hall, Englewood Cliffs, New Jersey 07632
To my parents-
Peach and Bud
v
. Contents
PREFACE
1
xvii
INTRODUCTION
1
PERFORMANCE MODELING, 2 USES OF PERFORMANCE MODELS, 5 THE SOURCE OF PERFORMANCE PROBLEMS, 7 THE PERFORMANCE ANALYST, 7 THE S'IRUCI'URB OF nus BOOK, 8 SYMBOLOGY, 10
2 TRANSACTION-PROCESSING SYSTEMS COMPONENT SUBSySTEMS. 14 C'ommmricarion Network. 15 Processors. 15 Memory. 15 Application Processes. 16 Data Base. 16 Other Peripherals. 17 SYSTEM ARCHITECTURES. 17 Expandability, 18
12
Contents
viii
Distributed Systems, 18 Fault Tolerance, 20 Summary, 22 TRANSPARENCY, 22 The Process, 22 110 Processes, 24 Interprocess Communications, 26 Process Names, 28 Process Mobility, 29 Summary, 30 PROCESS STRUCTURE, 30 Process Functions, 30 Interprocess Messages, 31 Addressing Range, 33 Process Code Area, 33 Process Data Area, 33 The Stack, 35 Summary, 36 PROCESS MANAGEMENT, 36 Shared Resources, 36 Process Scheduling, 37 MechaDisms for Scheduling, 38 Memory Management, 40 Paging, 40 Managing Multiple Users, 43 Additional Considerations, 44 Summary,4S SURVIVABnlTY, 4S Hardware Quality, 46 Data-Base Integrity, 47 Software Redundancy, 49 Transaction Protection, SO Synchronization, 53 Backup Processes, 55 Message Queuing, 56 Chedcpoiating, 59 SOFI'WARE ARCHITECTURE, 63 Bottlenecks, 63 Requestor-Server,64 DyDamic Servers, 64
3 PERFORMANCE MODELING BO'ITLENECKS,67
67
Contents
ix
QUEUES, 68 THE RELATIONSHIP OF QUEUES TO BOTTLENECKS, 69 PERFORMANCE MEASURES, 70 THE ANALYSIS, 73 Scenario Model, 73 Traffic Model, 74 Performance Model, 77 Model Results, 83 Analysis Summary, 8S
4 BASIC PERFORMANCE CONCEPTS
87
QUEUES-AN INTRODUcrION, 88 Exponential Service Times, 90 Constant Service Times. 91 General Distributions, 91 Uniform Service Times, 92 Discrete Service TUDeS, 92 Summary, 92 CONCEPTS IN PROBABDJTY AND OTHER TOOLS, 94 Random Variables, 94 Discrete Random Variables, 94 Continuous Random Variables, Case 1, 99 Case 2, 100 Permutations and Combinations, 104 Series. 105 The Poisson Distribution. 106 The Exponential Distribution. 110 Random Processes Summarized. 111 CHARACTBRlZING QUEUING SYSTEMS, 113 INFINrI'E POPULATIONS, 113 Some Propenies of IDfiDite Populations, 114 Dispersion of Response Time, 114 Gt.11IImIl distribution, 115 Central Limit Theorem, 116 VQriQnce of response tiIMs, 117 Properties of MlMJI Queues, 120 Properties of MIG/I Queues, 122 SiDgIe-QamIel Server with Priorities. 123
Nonpreeinptive SeTver. 124 Preemptive server, 125 Multip1e-Channel Server (MIMIc). 126 Multiple-Owmel Server with Priorities, 127 Nonpreemptive server, 127 Preemptive server, 128
.
x
Contents
FINITE POPULATIONS, 128 Single-Server Queues (MlMllImlm), 130 Multiple-Server Queues (MJMIclmJm), 131 Computatiooal Considerations for Finite Populations; 132 COMPARISON OF QUEUE TYPES, 132 SUMMARY, 136
5 COMMUNICATIONS
1.
PERFORMANCE IMPACT OF COMMUNICATIONS, 140 COMMUNICATION CHANNELS, 141 Dedicated Lines, 141 Dialed. Lines, 141 Virtual Circuits, 142 Satellite Channels, 143 Local Area Networks, 144 Multiplexers and Concentrators, 144 Modems,147 Propagation Delay, 148 DATA TRANSMISSION, 149 Character Codes, 149 Asynchronous Communication, 150 Synchronous Communication, 152 Euor Pelfonnance, 152 Euor Protection, 154 Half-Duplex Channels, 155 Full-Duplex Cbaunels, 156
PROTOCOLS, 157 Message Identification aDd Protection, 157 Message TraDSfer, 158 Half-duple% mastlge transfer. 158 Full-duple%-mastlge transfer. 158 Channel Allocation, 160 Bit Syncbronous Protocols, 160 BITS, BYTES, AND BAUD, 163
LAYERED PROTOCOLS, 164 !SOIOSI, 165 Applictztion kzyer. 165 Presenttztion kzyer, 166 Session layer. 167 Transport kzyer. 167 Network layer, 167 Data link layer. 168 Physicollllyer. 168 SNA,I69 X.25, 170
Contents
xi
MESSAGE TRANSFER PERFORMANCE, 172 Half-Duplex Message Transfer Efficiency, 172 Full-Duplex Message Transfer Efficiency, 176 Message Transit Time, 178 Message Transfer Example, 180 ESTABUSHMENT/TERMINATION PERFORMANCE, 181 Point-To-Point Contention, 182 Multipoint PolllSelect, 18S
LOCAL AREA NETWORK. PERFORMANCE, 189 Multipoint Contention: CSMAlCD, 189 Token Rings, 192 SUMMARY, 196
6 PROCESSING ENVIRONMENT
197
PHYSICAL RESOURCES, 198 Processors, 199 Cache Memory, 199 110 System, 200 Bus, 201 Main Memory, 202 Processor Performance Factor, 203 Traffic Model, 204 Performance Tools, 20S Performance Model, 206 W-Bus, 206 Memory, 207 R-Bus,207 Memory queuefull, 208 Model Summary, 208 Model Evaluation, 208 OPERATING SYSTEM, 213 Task Dispatching, 214 Interproc:ess Messaging, 218 Global message netWork, 218 Directed message paths, 219 File system, 219 Mailboxes, 219 Memory Management, 220 110 Transfers, 220 OIS Initiated Actions, 221 Locks,222 Thrashing, 222 SUMMARY, 224
xii
Contents
7 DATA-BASE ENVIRONMENT
226
THE FILE SYSTEM, 227 Disk Drives, 228 Disk Controller, 228 Disk Device Driver, 229 Cache Memory, 230 . File Manager, 231 File System Perfonnance, 232
FILE ORGANIZATION, 234 Unstructured Ftles, 234 Sequential Ftles, 237 Random Ftles, 238 Keyed Ftles, 238 Indexed Sequential Ftles, 243 Hashed Files, 245 DISK CACHING, 245 OTHER CONSIDERATIONS, 249 Overlapped Seeks, 250 Alternate Servicing Order, 250 Data Locking, 251 Mirrored Ftles, 252 Multiple Ftle Managers, 253
File m.tZ1IIZger per disk volume, 253 Multiple file m.tZ1IIZgers per disk volume, 253 Multiple file m.tZ1IIZgers per multiple volumes, 255
AN EXAMPLE, 255
8 APPUCATION ENVIRONMENT PROCESS PERFORMANCE, 260 Overview, 260 Process Time, 265 Dispatch Tune, 266 Priority, 266 Operating System Load, 267 Messaging, 267 Queuing, 268
PROCESS STRUCTURES, 269 Monoliths, 269 Requestor-SeIver, 270
271 Servers, 27.2 File m.tZ1IIZgers, 274 Multitasking, 274 Reque~rs,
Dynamic SeIvers, 278 Asynchronous 110, 280
259
xiii
Contents AN EXAMPLE, 281
SUMMARY, 296
9
FAULT TOLERANCE
298
TRANSACI'ION PROTECI'ION, 300 SYNCHRONIZATION, 303 ~AGEQUEmNG,~
CHECKPOINTlNG, 307 DATA-BASE INTEGRITY, 308 AN EXAMPLE, 308
10 THE PERFORMANCE MODEL PRODUCT
312
REPORT ORGANlZATION, 313 Executive Summary, 313 Table of CoDteDtS, 313 System Description, 314 Transaction Model, 314 Traffic Model, 314 Performance Model, 315 Model Summary, 31S Scenario, 316 Model Computation, 316 Results, 317 Conclusions and RecommendatioDS, 317 PROGRAMMING THE PERFORMANCE MODEL, 317 Input Parameter Entry and Edit, 317 Input Variable Specification, 318 Report Specification, 319 Parameter Storage, 319 Dicticmary, 319 Help, 320 Model Calculation, 320 Report, 320 TUNING, 321
QUICK AND DIRTY, 322
11 A CASE STUDY
323
PERFORMANCE EVALUATION OF THE SYNTREX GEMINI SYSTEM, 325 Executive Summary. 326
xiv
Contents
-
Table of Contents, 327 1. Inttoduction,328 2. Applicable Documents, 329 3. System Description, 329 3.1 General, 329 3.2 AqUlJrius Communication Unes, 330 3.3 AqUlJrius Interface (AI), 331 3.4 Shmed Memory, 334 3.5 File Manager, 335 3.6 File System, 336 4. Transaction Model, 338 5. Traffic Model, 340 6: Performance Model, 342 6.1 Notation, 34i 6.2 Average Transaction Time, 343 6.3 AqUarius Terminal, 344 6.4 Communication Line, 344 6.5 Aquqrius Interface, 346 6.6 File Manager, 350 6.7 Disk Management, 353 6.8 Buffer Overjlow, 355 7. Scenario, 357 8. Scenario Time, 359 9. Model Summary, 360 10. Results, 365 10.1 Benchmark Comparison, 365 10.2 Component Analysis, 368 11. Recommendations, 371 References, 373
APPENDIX 1 GENERAL QUEUING PARAMETERS
APPENDIX 2 QUEUING MODELS
375
377
APPENDIX 3 KHINTCHINE-POLLACZEK EQUATION FOR MlG!1 QUEUING SYSTEMS
383
APPENDIX 4 THE POISSON DISTRIBUTION
APPENDIX 5 MIRRORED WRITES A. DUAL LATENCY TIMES, 389 B. SINGLE DISK SEEK TIME, 391 C. DUAL DISK SEEK TIME, 392
389
_
Contents
xv
APPENDIX 6 PROCESS DISPATCH TIME
397
A. INFINITE POPULATION APPROXIMATION ERROR, 397 B. DISPATCHING MODEL, 399 C. SINGLE PROCESSOR SYSTEM, 402 D. MULTIPROCESSOR SYSTEM, 40S E. AN APPROXIMATE SOLUTION, 409
APPENDIX 7 PRIORITY QUEUES APPENDIX 8 BIBLIOGRAPHY
411
414
Preface
This book provides the tools necessary for predicting and improving the performance of real-time computing systems, with special attention given to the rapidly growing field of on-line traDsaCtion-processing (OLTP) systems. It is aimed at two audiences:
1. The system analyst who thorougbly understands the concepts of modem operating systems and appJic:ation stnlCtIIres but who feels lacking in the mathematical tools necessary for performance evaluatiOD. 2. The mathematician who has a good grasp of probability and queuing theory but who would-like to gain a better undastandiDg of the technology bebind today's compotiDg systems so tbat these tools might be effectively applied._
OLTP systems are mpidly becomiDg a part of our evayday Hfe. Mercbants pass our mdit cards thmugb slot readers so that rem.ote systems can check our czedit. Every commercial aiEpJane ride and hotel stay is plamled and ttacked by these systems. When we are critically ill, OLTP systems monitor oar critical sigDs. They control our facrories and power plants. We obtain cash from ATMs, play the horses and lotteries, and inveSt in stocks and bonds tbaDks to OLTP systems. _ _ No wonder 1beir performance isbecomiDg a growing concem. -A poorly performing system may simply frustrate us wbile we wait for its response. Even worse, it can be life-tlnea:rening to a b1!sjness-or even to our loved ones. - We define the perfOlillaDce of an OLTP system as -~ time required to receive a xesponse from it once we have sent it a traDSaCtion. Our traDsadion must wait its tum over xvii
XVIii
Preface
.
..
and over again as it passes from one setvice point to another in the OLTP system, since it is just one of many transactions that the system is trying to process simultaneously. These service points may be processors, disks, critical programs, commUDication lines-any common resource shared among the transactions. As the system gets busier, the delays at each service point get longer; and the system may bog down. The study of the behavior of these delays as a function of transaction volume is the subject of the rapidly expanding field of queuing theory. .. ApeIformance aDiiyit is'One'who has an intimate knowledge of the ~ of . these systemS and who can apply the practical tools available from queuing theory and other mathematical disciplines to make IeasODable statements about the expected performance of a system. The system may be one still being conceived, an existing system in trouble, or a system undergoing euhancement. This book is intended to train performance analystS. For the system analyst who may be a bit intimidated by higher mathematics, it presents mathematical tools that have practical use. Mmeover, the derivations of these tools ate for the most part explored, perlJaps not always rigorously, but in a manner designed to give a full understanding of the meaning of the equations that represent the tool. For the most part, only a knowledge of simple algebra is required. For the practicing mathematician, there is an in-depth description of each OLTP system component that may have a performance impact. These components include commuDication lines, processors. memories, buses, operating systems, file systems, and softwue application architectures. System extensions for fault tolerance, an important attribute of OL'IP systems, ate also covered.. This book is organized so that the reader may skip easily over material that is already familiar and focus instead on material of direct inteEest. The book does not present a "cookbook" approach to perfoImance analysis. Radler, it stresses the understanding of the use of basic tools to solve a variety of problems. To this end, many examples ate given during the discussion of' each topic to hone the mader's ability to use the appIopriate tools. As of the dare of this writiDg, the title "perfoImance analyst" has DOt been accepted as a usual job c1escriptioD. This is certaiDly DOt due to a pa:eeption that pe!f0!DJlllCe analysis is unnecessmy but pedJaps instead to the pa:eeption that meaningful pe!formance analysis is somewbat that of a mythical art. If this book can advance the accepamce of the pmcticiDg perfOl'lllaDCe aualyst, it will have achieved its goal.
A wodc of this magnitDde is the result of the efforts of maDy. I would like to take this opportunity to
tbaDk:
• All my anonymous reviewers, whose in-depth criticisms played a majorrole in the ~rijzation of the book.
Preface
xvix
• -My partner, Burt Liebowitz, who often challenged my fuzzy thinking to enforce a clarity and accuracy of presentation. • My daughter Leslie for the typing and seemingly endless retyping of the manuscript as it progressed through its various stages. • My wife, Janice, a professional writer in her own right, for turning my writing into real English. • Charles Reeder, who prepared the illustrations for this book, often with •'realtime" responses. • My many customers, who have provided the real-life testing ground for the methodology presented in the book, and especially to Concurrent Computers Corp. and Syntrex Inc., for their kind permission to use the studies presented in chapters 6 and 11, respectively. • Last, but not least, my editors, Paul Becker and Karen Winget, for their encour"" agement and guidance in the mechanics and business issues of bringing a new book to press. ABOUT THE AUTHOR Dr. Higbleyman bas over 30 years' experience in the development of real-time on-line data processing systems, with particular emphasis on high perfOIDWlce multiprocessor faulttolerant systems and large communications-or systems. Other application areas include intelligent terminals, editorial systems, process control, and business applications. Major accomplisbments include the first computerized tota1i7JJtor system for racetrack wagering installed for the New York Racing Association, the first automation of floor trading for the Chicago Board of Trade, the intemalional telex switching systems utilized by ITT World Communic:atiODS, the fault-tolerant data-base management system used by the New York Daily News, a 6000-terminallottery system for state lotteries, an electroDic billing data collection system for the telephone opezating c:ompaDies, and message switching systems for international cablegram and private network services. Dr. Higbleyman is fOUllder and ChaimJan of1be Sombers Group, a company which has provided tumkey software packages for such systems since 1968. He is also founder and c:bahman of MiniDasa Services, a company using miDicompuier technology to bring data processing services to small businesses. Prior to these activities, he was founder and vice-presideDt of Data TEeDds, Inc., a turnkey developer of real-time Systems since 1962. Befen that, he was employed by Bell Telephone Laboratories, where be was responsible for the development of the 103 and 202 data sets, and by LincoJn Laboratories, whe!e he worked on the first UIDsistorized computer. In addition to his management activities at The Sombers Group, Dr. Higbleyman is currently active in:
• performance modeling of multiprocessor systems. • fault-tolerant CODSiden.tions of multiprocessor systems.
xx
Preface • architectural design of hardware and software for real-time and multiprocessor
systems. • functional specifications for data -processing systems. He has performed analytical perfonnance modeling on many systems for seveml clients including: A. C. Nielson
Autotote Bunker Ramo Concurrent Computer Digital Equipment Corp. FlISt National Bank of Chicago FTC Communications G.E. Credit Corp. Harris Hewlitt Packard
m
World Communications MACOMDCC PYA/Monarch Smith Kline Stratus Syntrex
Systeme
Tandem Telesciences Time
Dr. Highleyman received the D.E.E. degree from Brooldyn Polytechnic Institute in 1961, the S.M.E.E. degree from Massachusetts Institute of Technology in 1957, and the B.E.E. degree from Rensselaer Polytechnic Institute in 1955. He holds four U.S. patents and bas published extensively in the fields of data communications, pattern recognition, computer applications, and fault-tolerant systems. He also sits 01' bas sat on several boards, including: • The Sombers Group (Chairman) • Science Dynamics, Inc. • MiniData Services, Inc. (Chairman)
• Intematicmal Tandem User's Group (Past PIesident) • Vertex Industries
1 Introduction
Ubiquitous. mysterious. WODderful--and sometimes aggravating-computers. They
truly are becoming IIlOle and more involved in what we do today. What they do often affects our quality of Jife, from the success of our businesses to the enjoyment of our free time to our comfons and CODVaUeDces. Businesses enter 1DDSaCtioDs as they occur and obtain up-to-tbe-:mimlte status information for decision-makiDg. Banks are extending on-liDe financial services to their corporate customeIs for interactive money transfers and account S1atUs. giving COIpO!3te money mauagers the uJrimate in cash-lIow ~ CaD your telephoDe company or credit cani company about a bill. and your cbarge and paymeat history appears OIl the SCIeen for jmnwtiate action. See your tmvel agent for airline tickets. and the CODIpIUer p1eseats all optioDs, mabsyour zesemdiODS, aud issues
your tickets.
.
TIlDe for fun? Buy tickets tbIOugh your local ticket outlet from the inventory kept OIl COI:DpUt£r. Play the hones or the state lottery-ifyou'te a lucky wilmer, the computer will c:a1cuJate your payoff. Not feeling well? Computers will mauge your hospital stay, will order your tests, and, of course. will prepme your bills. Other computers will monitor your specimens as they llow tbrougb the cliDicallabolatory, thus eusnring the quality and ac:curacy of the test results. ,ten
.
Need to COThinllDicate? Computers-will carry your voice, yOUr'data, and your writwords rapicD.y to ~ ~0IlS. •
1
2
Introduction
Chap. 1
And Quietly in the background, computers monitor the material fabric of our daily lives, from power and energy distribution to traffic control and sewage disposal. All of the preceding examples are types of transaction-processing systems. These systems accept a transaction, process it using a base of data, and return a response. A b:ansaction may be an' inquiry, an Onter, a message, a status. The d8ta base may contaiJi customer information, inventory, orders, or system status. A response may be a status display, a ticket, a message, or a command. For instance, in a wagering system, the wager information is a transaction that is stored in the data base, and the reply is a ticket. Subsequently, the ticket becomes the ttansac:tion as the system compares it to the data base to see if it is a winner. The reply is the payoff amount. In the control system for an electrical power netwOIk, the transactions are status changes such as power loading, transformer temperatures, and circuit failures. The data base is the current network configuration and status. The new status change updates the data base, and a command to alter the configuration may be issued as a response (e.g., reduce the voltage to prevent a power overload, thereby avoiding a brownout). In a message-switching system, the transaction is an incoming message (text, data, facsimile, or even voice). The data base is the set of messages awaiting delivery and the routing procedmes. The response is the J:eCeipt and delivery acknowledgments returned to the sender and the message routed to the destination. In an airline or ticket IeSerVation system, the transaction is first an iDquiJy. Schedule status is re4m hed. from the data base, which also holds an inventory of available seats. A subsequeDt ttaDSaCtioD. is the Older, which will update the inventory. A ticket will be issued in IeSpoDSe. No wonder we become affected--even aggravated-by the performance of these systems. Have you ever wai1ed sevaal minutes 08 the telephone for the cledc on the other end to get your biDing status? Have you ever watched the anxiety and the anger of a racehorse player1ryiDg to get the bet in befo!e the beD goes off, only to be frustrated by an ever-slowiDg Hne at the wiDdow? Have you ever watched a mmclumt get impatient waiting for a credit cud validation, give up, and mab the sale without authorization, thus risking a possible loss? Have you ever ••• ? The liSt goes 08. , And that is wbat tbis book is an about: the prediction and control of the perl'onnanc:el of these U3DSaCIioa-proc systems, which are weaving their way into our lives. •
w
an know tbatas a computer sysrem. becomes loaded, it ""bogs down." Respcmse times to user requests get longer and leading to iDc:reased and aggravaDOD of .. . '.. .longer, ... .. fiuslmUon _.. . .. We
,
_-_._----
IOf coarse, system avaiJahiIity is an eqaaDy impodIDl cciDI:em. Ifa\oe you ever beeIl1Dld Ibat you CIIl't pc a ticket • Ibis time because ""die c:uoupaIei is doWD"? The zeliability aaalysis of1hese sysrcms is DOt a tcpic: for dIis book. Howew:r, perfarmall'OB cIegaIdIIion clue to ~ tabla by die sySIIe.ms to _ nIiable oper-
atioa is a CODCenl aDd is CCM:leCI. TecbDiques for die IdiabJlity aaaI.ysis of ~ sysImDS may be fouDd in ~ ,witz [17].
Chap. 1
Performance Modeling
3
.
"
the user population. A measure of the capacity of the system is the greatest load (in transactions per hour, for instance) at which the response time remains marginally accept-
able. Deterioration of response time is caused by bottlenecks within a system. These bottlenecks are common system resources that are required to process many simultaneous" transactions; therefore, transactions must wait in line in order to get access to these resources. As the system load increases, these lines, or queues, become longer, processing delays increase, and responsiveness suffers. Examples of such common system resources are the processor itself, disks, communication lines, and even certain programs within the system. One can represent the flow of each major transaction through a system by a model that identifies each processing step and highlights the queuing points at which the pr0cessing of a transaction may be delayed. 'Ibis model can then be used to create a mathematical expression for the time that it takes to process each type of transaction, as well as an average time for all transactions, as a function of the load imposed on the system. This processing time is, of course, the response time that a user will see. The load at which response times become unacceptable is the capacity of the system. Performance modeling concerns itself with the prediction of the response time for a system as a function of load and, consequently, of its capacity. A simple example serves to illustIate these points. FIgUre l-1a shows the standard symbol used throughout this book for a resource and the attendant queue of transactions awaiting servicing by tbat IeSOurCe. The "service time" for the resource is often shown near it (T" in this case). The service time is the average time it takes the resource to process a transaction queued to it. Figure I-1b is a simple view of a transaction-processing computer system. TrausactioDs mive &om a variety of incoming c:onnnunk:ation lines and are queued to a program (1) tbat processes these inbound requests. 'Ibis program requires an average of 80 mjUisemnds (msec.) to process a traDsaction, which is then sent to the disk server (2) to read or write dal:a to a disk. The disk server ~ these am other requests and requires an average of SO 1DSeC. per request. Once it has completed all disk walk, it forwards a respouse to an output program (3), which retams these and other respoases to the comDlUDication lines. 1be output pogram requires 20 1DSeC. on the average to process each
respoase. Since the programs and the disk system me serviDg' multiple sources, queues of As the servers get busier, the queues wID get loDger, the time a 1raDsactioil spends ill the system wID get longer, and the system's respoase time will get slower. , One huplied queue not shown in.Figure l-Ib is the processor queue. We assume in this system that many pograms are rmming-many more dian are shown. But t:he!e is only one processor. Therefore, wilen a program has woat to do, it must wait ill line with other programs before it can be giveo access to the processor and actuilly run. Let us DOW do a IiUle performance aoalysis using Figu!e 1-1b (wbiCb, by the way, will later be called a "traffic moder~). If the system is idle, no queues wID build;, and an average transaction will work its way tbrougb the system ill 80 + SO +20 = ISO msee.
ttaDsac:tioDs awaiting service may build ill froat of eich of these serYeIS.
Introduction
4
--- "I I
TRANSACTION
I
1----.1
Chap. 1
RESPONSE
RESOURCE
1 - - -.....
QUEUE TS
RESOURCE MODEL (a)
(I)
~----. III
PROGRAM PROCESS
TRANSACT~
INPUT
/
\(
BOmsec
(2)
(3)
50msec
:= I. . -~__. .
1-----'
OUTPUT.
-
20msec SIMPlE COMPUTER SYSTEM (b)
·(the sum of the iDdividual service times for each server in its path). Not a bad
respoase
time. Now let us look at the msponse time in a more normally loaded system in which the queue leDgtbs for all servas, inducting the processor, aVerage 2. Tbat is, on the average, any RqUest for service will find 2 Rquests in from of it--one being serviced and one waiting for service. The DeWly eatencl mquest will be the seconcl Rquest in line, DOt COUIdiDg the mquest being serviceclat!be time. (As willbe shown.later, resomce loads of ~ will oflauesult in ~ leDgIbs of2. That is, jfa server is busy 67 pen:ent of the time on the average, its avezage queue leagth will be 2.) With these queue leDgtbs, each traDSaCtion must wait 2 service times before being serviced, and then each traDsaCtion must be serviced. Sounds like the nisponse time should triple £.rom 150 1DSeC. to 450 1DSeC~, right? WIODg. The:response time degrada!:ion is eveu worse since each progmm must wait in a queue for the processor. Let us assume . that the average time the processor spends nmiIiDg a prognm1 is 50 1DSeC. Let us also
Chap. 1
Uses of Performance Models
5
assume that the disk server is an intelligent device that does not use the processor and so is not slowed by baving to wait in the processor queue. Remember that the processor is 67 percent loaded, so its queue length is 2. When the input program wants to run, it must wait in the processor queue for 2 x 50 = 100 msec. and then must spend 80 msec. serving the incoming transaction. So far as the incoming transaction is concerned, the service time of this program is 180 msec., not 80 IDSeC. Likewise, the effective service time of the output program in the loaded pr0cessor is 2 x 50 + 20 = 120 msec. rather than 20 msec. The disk server time remains at 50 msec. since it doesn't rely on the processor. An average queue length of2 in front of each server now causes the response time to triple relative to the effective service times. Thus, at a load of~, system response time degrades from ISO IDSeC. to 3(180 + SO + 120) = 1050 msec! Note that we could plot a curve of response time versus load if we just knew bow to relate queue size to load. Then, given the maximum acceptable response time, we could determine that load representing the capacity of the system, as shown in Figure 1-2. 'Ibis is what performance modeling is all about.
USES OF PEllFORlMtNCE .ODELS Ideally, a performance model should be validated and tuned. Its results should be c0mpared to measured~. If they are sigDificandy diffeJent, the IeaSOIIS should be understood and the model COJrected. Usually, this results in the inclusion of certain processing steps initially deemed trivial or in the determiDation of more accmate parameter values. A performance model, no matter how deIailed it may be, is neverdleless a simplification of a very complex process. As such, it is subject to the inacc:uracies of simplification. However, experieDc:e has shown these models to be smprisinglyaccurate. Moreover, the tteads pleCticted are even IDOIe ac:cmate and can be used as very efJective decision tools in a variety of cases, such as
1. Perjomuznce Predictitm. The perfomumc:e of a pJanued system can be predicted before it is built. This is an ememely valDable tool durlDg the design phase; as bottleDecks can be idemified aDd CODeCCed before implementaIion (often zequiring signifiCant 'architec:tuia'i ch3nges), and pertormanc:e goalS' Can be' W:mied.-' 2. Perjomuznce TIIIIing. Once a system is ~, it may DOt perform as expected. The performance model can be used along with actual performance measDl&ments to "look inside" the system and to help locate the problems by comparing actual pm"SSiDg and delay times to those tbat are expected. 3. CostIBeMjit of EnhIlncements. When it is planned to modify or eahance the system, the performance model can be used to esriJnate the performance impact of the proposed change. This iDformation is invaluable to the evaluation of the proposed change. If the change is being made strictly for petfonnance purposes, then the costIbenefit of the change, can be accurately determmed.
Introduction
6
Chap. 1
2.0
1.8
1.6
1.4
u
MAXIMUM jACCEPTABLE RESPONSE nME
1&1 !! 1.2 1&1
:IE
i=
., z .,0 1&1
--.------- ---
1.0
I I
Q.
III
a::
.8
I
CAPACITY
:/
.6
.4
o
.2
.4
.6
.8
1.0
LOAD
. . 4. System Co'lffiguration. As a product is introduced to the madcetplace, it often has several options that can be used to tailor the'system's perfonaance to the user's Jleeds-;.for example. the number of disks, the power of the processor. or comDlUIIic:ation line speeds. The performance model can be packaged as a sales tool to help configure-new systems and,to give the costomer some CODfidenc:e in the system's capacity and performance.
Chap. 1
The Performance Analyst
7
THE SOURCE OF PERFORIlllANCE PROBLEMS
It is interesting to speculate about the main cause of the poor peIformance that we see in practice. Is it the hardware-the processors, memories, disks, and communication systems of which these systems are built? Is it the software-the operating systems and applications that give these systems their intelligence? Is it the users of the systems, through inexperienced or hostile actions? The data base organization? The system managers?
Actually, it is none of these. The second most common cause of poor peIformance is that the designers of the system did not understand peIformance issues sufficiently to ensure adequate peIformance. The first most common cause is that they did not understand that they did not understand. , If this book does nothing more than focus attention on those issues, that are important to peIformance so that designers may seek help when they need it, the book will have achieved a major part of its goal. If it allows the designers to make design decisions that lead to an adequately peIforming system, it will have achieved its entiIe goal. THE PERFORIIANCE ANALYST The effective prediction of peIformance has been a little-used art-and for a very good reason. h IeqUires the joining of two disciplines that are inheIently contradictory. One is that of the mathematical disciplines-from probability and statistical theory to the LapIace-Stieltjes transforms, generating functions, and birth-death processes of queuing theory. The odler discipline is that of 1he system aaalyst-dara-base design, communication protocols, process stnICtUIe, and softwaIe arcbitecIure. Qualified system analysts are cenamIy adept at a1gebm. They most likely, are a little rusty atelememaly calcalus, pmbabiIity theory, and statistics. Do they know how to solve differential-cliffele'we equations, apply Bessel functions, or uDdersraud the attributes of ergodicity? Pmbably not. They don't need to know, and 1hey p!Obably don't want to know. Practicing applied madJematicians. for the most part, have not been exposed to the inner WOIkiDgs of CODfeD4KDIy data porasingsystems. 1bird-normal form, pipelining, aad SNA may be not much !DOle tban WOlds to theni. ADd when they do uncIenrand a system's performance problem. it is difficWt for 1hem to iDake ~ on it because the 8SShiI'I4ioDs teq1Iiied for reasoaab1e ca1cuJations ofIeD divage so far from the real world tbat the mathematicians' oldco11ege professcD-aDd'cerIaiDlythe pat body of coatemporary coDeagues-woald never appiove. . . Performance analysts are in an awkwaJ:d position. They must be JeaSODably accomplished in system aaalysis to 1IIlderstand the nature of the system they are auilyzing. yet they must be practical enough to make those assumptions and approXimations necessary to solve the problem. Likewise. they mustUDdersrand the application of some fairly straightforwani malbematieal principles to the solution of the problem without being SO embarrassed by the accompanying assumptions as to render themselves ineffective. In short,
8
Introduction
Chap. 1
performance analysts must be devout impeIfectionists. The only caveat is that they must set forth clearly and succinctly the impeIfections as part of the analysis. A performance model is a very simple mathematical characterization of a very complex physical process. At best, it is only an approximate statement of the real world (though actual results have been surprisingly useful and accurate). But isn't ,it better to be able to make some statement about anticipated system peIformance than none? Isn't it better to be able to say that response time should be about 2 seconds (and have some confidence that you are probably within 20% of the actual time) than to design a system for a I-second response time and achieve 5? Based on actual experience, the most honest statement that can be made without a performance analysis is of the form "The system will probably have a capacity of 30 transactions per second, but I doubt it." The pmpose of this book is to eliminate the phrase "but I doubt i~. "
THE STRUCTURE OF THIS BOOK To that end, this book takes on both system analysis and mathematics. It is designed to give applied mathematicians the background they need to understand the structure of contemporary transaction-processing systems so they can bring their expertise to bear on the analysis of their performance. It is also designed to give system designeIS the mathematical tools tequked to predict performance. In either case, we have created perl'ormance analysts. It is the author's strong contention that a tool is most useful if the users are so familiar with it that they can make it themselves. 'Ibis applies to the various relationships and equations which are used in this book. Most will be derived, at least heuristically if not rigoJ:ously; in this way, the assnmprions that go mto the use of the particular tool are inade .clear. Most derivations are simple, requiring only basic algebra, a little CalculuS perbaps, and an elementary knowledge of probability and SllDstics. The derivatioDs are included in die main body of the text. MOle complex. derivations are left for the appendixes. Just oc:casioDaD.y a derivation is so complex that only a reference is given. The book is stn1CIDIed to support the system analyst seWcing better matbematical tools, the mathematician seeking. a better system UDderstanding, . . either one seeking anythjng in between. 'Ibis structure is shown in Figme 1-3. Chapter 2 isa major!eView oldie c:on1m&JlOl8Iy tecbnology involved in ttansactionp1'OC"SSing systems. It wiD be most useful to those seeking a better undmtaDding of the ~ of'these systems, but this cb8Pter could be skhnmecl or bypassed by system analysts knowledgeable in ttansaction processing. Chapter 3 gives a sDnple but iD-deptb example of performance modeling based on chapter 2 material extended by some elementary m~ introduced as die modeling progresses. This chapter ~ a preview of the rest of the book. A thorough underS1aDding of chapters 2 and 3 will equip the leader with the tools necessary to perform elementary performance analyses. The rest of the book then hones the tool:s developed in ~ two chapters.
Chap. 1
The Structure of This Book
9
CHAPTER 2
CHAPTER 4
SYSTEM
MATHEMATICAL
BACKGROUND
BACKGROUND
!
'!
CHAPTER 3
CHAPTERS 5-8
~
A LOOK AT PERFORMANCE MeDEUNG
COMMUNICATIONS
~ ..1
CHAPTER 10
/
DOCUMENTm'lON
-
( DISK)
PROCESSOR
OPERATING SYSTEM I-APPUCATION PROGRAMS DATA BASE
CHAPTER 9 FAULT
TOLERANCE
CHAPTER II CASE STUDY
Chapter 4 peseIdS a variety of mattM:mar;ca1 tools that m useful in cerram situ': While the bulk: of the chapLer is areview of queaiDg theoJy, which allows us to !elate queue 1eagtbs to mouit:e loads, many other useful tools m~. including basic coacepts in probability and statisdcs and expmsion of IISeful series. Cbapters 5 tbrougb 8 expand concepts relative to the major building blocks of a ttaDSadion-pr0c:essiD8· system. These building blocks include the oommnnjeation network (cbapter 5), the processor and opexating system (chapter 6), the data base (chapter 7), and the· application programs (chapter 8). Chapter 9 extends 1hese concepts to fault~leraDt systems. These cbaptm's COIdiin iDsigbt for both the aulyst and the matbemati~
tioas.
Introduction
10
Chap. 1
ciano System concepts are explored in more depth, and the application of the mathematical tools to these areas is illustrated. Chapters 10 and 11 are summary chapters. Chapter 10 discusses the organization and components of the formal performance model so that it will be useful and maintainable as the system grows and is enhanced. Chapter 11 includes a complete example of a perfounance model. References are given following chapter 11 and are organized alphabetically. Appendix 1 summarizes the notation used, and Appendix 2 summarizes the major queuing relationships. Further appendixes give certain derivations that will be of interest to the more serious student. The book is not intended to be exhaustive; the rapidly progressing technology in TP systems prevents this anyway. Nor is it intended to provide a cookbook approach to perfonnance modeling. The subject is far too complex for that. Rather, it is intended to provide the tools requiIed to attack. the performance problems posed by new system architectures and concepts. The book is also not intended to be a programming guide for performance model implementation. Though the complexity and interactive nature of many models will require them to be programmed in order to obtain IeSUlts, it is assumed that the performance analyst is either a qualified programmer or has access to qualified staff. No progranmring examples are given; however, useful or critical algorithms are occasionally presented. The author highly recommends that the serious student read references [19] and [24]. James Martin and Thomas Saaty both present highly J:eadable and in-depth presentations of many oftbe concepts necessazy in performance analysis. Also. Lazowska [16] provides an int.eJ:esting approach by which many performance problems can be attacked using fairly simple and straightforwani techniques.
SVlfBOLOGY One final Dote on symbology before embaddDg on this subject. The choice of parameter symbols, I fiDel. is ODe of the most frustrating and mun.daDe parts of performance analysis. 0fteD. symbols for several hundred parameters must be iDveJDd; there are just not enough cbaracters to go around without eDODDOUS subscripting. Therefore, the choice of symbols is often not !dec:tive of what they lep:eseDt. 'Ibis is a poblem we live with. The symbols used in the body of this book are summarized in Appeu.cIix·l for easy &efeIeDce. NotWitbstanmng the.pmblems of naming CODVeDtioDs me.atioDed above, the author does iDipose certain &eSttictioDs: . .,
1. All symbols _ at most ODe chalacterplus subscripts. Thus, TS is never used for service time; but T$ may be. This pteveuts poduc:ts . of symbols from being ambiguous. The only exception is var(x), used to rep:esent 1he varlaDc:e of x. 2. Oaly characl:erSfrom the Arabic alphabet are used (uppmcase and lowercase A through Z,
IlUDleials 0 through 9).
There are two reasons for doing this:
Chap. 1
Symbology
11
a. Most of the typewriters, word processors, and programming languages I use provide little if any support for the Greek alphabet. b. More important, performance models are most useful if understood and used by middle and upper technical management. I usually find this level of user to be immediately and terminally u,timidated by strings of Greek letters. This convention can be particularly disturbing to the applied mathematician who is used to p, A, and JL as representing occupancy, mival rates, and service rates, respectively. Instead, in this text he or she will findL. R. and T as representing load, mival rate, and service time. 'Rather than p = AlJL, he will find L = RT. However, the first time he or she tries to program a model in Greek, he or she will quickly learn to translate. The only exception to this is delta. a is used to indicate a small increment, and 8 is used to represent the delta (impulse) function in certain derivations. They never appear in resulting expressions. One other convention used is the door U) and ceiling
2 Transaction-Processing Systems
This chapter explores the basic mchitecture of traDSaction-processiDg systems. Special attention is given to process structure and maoagement. a thorough knowledge of which is mandatory for an appreciation of peIfonuance issues. Strategies for fault tolerance &Ie also paalted. 1"he tam transaction-processing system is a ratber general teml whose definition in this book has importance ODly in terms of the applicability of the material p1eseutecl in the following chapters. It can be defined as readily by what it is as it can be by what it is . DOt.
A TP system is an on-line _-time multiuser system that accepts requests and retums IeSpODSeS to those zequests. The act of geaentiDg a tapODSe usually implies acw:ssiq a base of data maintaiaed by the TP systeQ1. 'Ibis cIata base is often a complex set of iDteaelatecl information maintaintA on disk files by a sophisticated clata-base manager. In someTP applicatioDs. simple memory-Ieside:Dt tables may be sufficient. On-liM II'Je8DS 1bat the user bas a direct CODDeCtion to die system. A user who calls a bDliDg clakis DOt on-line, though the bilJiDg clak with uerminaJ is. Asystem mwhich requests me eataecl via punched cards is DOt oa-JiDe.Systems mwhich users iDIaact with the system via tennjnaJs cxmnected by private or dialedcom"'uDication lines lie on-line, as me systems in which IemOte season communicate status over c:onummicaIion lines and n=ive commands in response to status .cbaDges. Martin [19] defines real time as follows:
12
Chap. 2
Transaction-Processing Systems
13
A real-time conlputer system may be defined as one which controls an environment by receiving data, processing them, and raking action or retuming results sufficiently quickly to qJJect the functioning of the environment at that time. (Emphasis added.)
"Sufficiently quickly" is a matter of the application. In a TP system, this is satisfied if IeSpODSeS are received in a time that appears short to the USer, i.e., the system appears responsive to the user. For a human user, this usually means one or two seconds. For a real-time control application, acceptable response times might be much shorter or perhaps even much longer. A single word that means "on-line real time" is interactive. If the user can communicate directly with a system, and the system responds quickly, then the user is in a position to interact with the system. He or she can ask questions and get responses without having to preplan his session with the system. His next request can be a function of the system's previous response. All TP systems are interactive. If the system's response slows to the point that intemction is not feast"le or is overly frustrating, the TP system loses its value. Thus, the importance of performance analysis. Conversely, a 1P system is not a batch system in which data is accumulated over a long period of time and then processed as one long contiguous file (say once per day). However, there is an impo.ttant connection between batch and TP systems. Most TP systems have a batch component. There are certain functions that are simply more efficient in batch mode than if done interactively. A prlme.example is data-base updating. An interactive update involves finding one or more indivicIual reconIs in a la!ge file, updating them, returning them, and modifying any key files used to locate them. The system may also have to update audit files Iequired to recover the data base in the event of a system failure during the update. If, instead, updates are batched, sorted periodically, and passed against the file in an ordered manner, the total system time required for d:lese updates can be sjgnjficandy less. Qapter 7 explores this in more-delail. 'Ibemore, many TP systems will do interactively 1hose updates which are IeQUiIed to suppott" interactive JeqUests and will batch updates and zeports tbat can be delayed. Often, batch pl'CnSsing will run in backpouDd mode while the system is supporting iDtenctive taffic. 11ms, the conteDtion for resources (prlmarlly processor, meDlOlY, and disk) betweeo sUnnJtaneously opaating batch and TP systems must be understood and accouDted for. Also, a TP system is nota sdentific number-crunching application, such as a weather-predidion system (or a perfOl'llllDCe model, for that matter). These Systems are charader:ized by very longprocessiDg times (minutes-to hours) befme IeSUlts are available. A 'IP system is DOt an iDtelligent termiDal, a ciR:uit switch, or a packet switch which massages data and passes it on. Fmally, for pmposes of this book, a 'IP system is not a single.user system, such as a persoaal c:ompurer, since peIfomJance degmdation due to loading does DOt occur. A network of persoaalcottlpllters accessing a common data base, ~, is very mnch a TP system.
Transaction-Processing Systems
14
Chap. 2
Though the performance concepts discussed in this book do bave some relation to non-TP systems, they are primarily pertinent to interactive data-base systeriIs.
COIIPONENT SUBSYSTEMS TP systems bave the following subsystem components, as shown in Figure 2-1: . . • <;Ommunication netwOIk
• • • • •
Processors MemoIy Application programs Data base Other peripherals
The functions of each of these subsystems are desCIibed below.
I
.
USER TERMINALS
I
PROCESS \
" M E M
o R y
PATH ..
'a ~ Inquiries Updates
---~-DATA BASE MANAGER
DATA FILES Figure 2-1 ComponeDts of a TP system..
Chap. 2
Component Subsystems
15
Communication Networlc A COIDIDIlDication network intercoDDeCts the users with the system. This network could include point-to-point and multidrop lines, dialed ~es, packet networks, local area networks, and satellite links. These links could utilize a variety of protocols, both half duplex and full duplex. The communication subsystem also includes the cominunication bardware that interfaces the communications network with the processor and the software that manages the communication netwOIk (noted as the communication manager in FJ.gUre 2-1). The communication subsystem is IeSpODSible for doing everytbing necessaty to get a request to the application programs and to return a reply to the user. However, it has no knowledge, nor need it bave any. of the content of a request or reply message.
One or more processors provide the processing power required for the communication manager, application programs. and data-base manager. If multiple processors are pr0vided. there must be some means to coordinate·their activities and to share work between them. For purposes of performance analysis, the concept of a processor is extended beyond the hardware to encompass the operating system. That is to say. the processor provides the enviIomnent in which the application programs, CQD111111Irication 1DlIIUlget. and database manager opeme. This includes memory management (swapping pages or overlays), task dispatching. priority management, and interprogram messaging. For systems c0mprising multiple processors, this also iDcludes the mechanisms for load sharing. interp:ocessor communication, and fault recovery (if any).
IIemory A JlII!JtfJOr! system suppcx1S the fundioDs of the processor. For a single processor system, the memory is jurimately associaIed.with the processor and is considered to be ODe and the Same with the processor forpelforiiilftCe aaa1ysis ~ union of a processor with its iDputIOU1pUt ports and dedicat.ed JDeIDOIY is called a computer. (Note: in general, a computer bas a processor. manary. buJk stomge, and periphenl devices, iDcludiDg printers and commnnicaDon lines. However. a processor with 110 ports and memory is sufficient to pedoml useful functioas and represents a computer in many applic:atioas.) A system with many proc:esscxs, each with its own memory. is called a mullicompurer system. Memory as a separate subsystem loses its ideDtity in this case. However, in a system with seveml processors sbaring the same memory. the memory is a common resource for which the" various processOIS compete and must be viewed as a separate subsystem. Such systems are caDed maltipIoc:essor systems and are distinct from multicomputer systems. Such systems, by the way, often bave small high-speed CtlChe 1!UfII'IOries ~ with each processor. These ~ memories are used to retain
imPoses.
Transaction-Processing ~ystems
16
Chap. 2
the most recently used data and instructions (i.e., those likely to be reaccessed in the near future) in an effort to unload common memory.
Of coone, a multicomputer system may also have associated with it a common memory to wbich all computers in the system have access. This case, though, is no from any other external data device, such as disk units, and is treated in the way. Multiprocessor and multicomputer systems are described in some greater detail later in this chapter.
different
same
Application Processes
Application programs process requests and issue replies..To be more accurate, let us define more precisely a program. A program is something created by a programmer, who has written it by specifying some data structures and some procasing procedures in some programming language. (This is called the source code.) The programmer has then compiled that source code by using a language compiler (another program), which translates the source code into object code that is executable by the processor. That object code is bundled with common system library routines into an object module and sits as a file on a disk unit somewhere in the system. It is this object module that is the physical embodiment of the program. h contains all data structures and computation procedures but is DOt actually executing. In order to execute, the object module must be loaded into the memory accessible by a processor and then executed by that processor. Most contemporaIy systems provide a multiuser environment. This meaDS that the same program may be loaded many times into the computer to service different users simultaneously. (In pmctice, usually only one copy of the procedures is loaded, as it can be shared by all users; a copy of the data structllJ.'es is individually loaded for each user.) To keep these diffaem instan1iatioDs of the same progxlDl ideDtUied, they are given independent DameS. A program rmmiDg in a compIIter with a system-uDique 1UI1ne is called a process. Thus, to be JDOJe accurate, the appJic:atioD subsystem c:omprlses ODe or more pro.Cesses diat accept requests from the cO'IPlimllication manager (as shown iii Figure 2-1): make tequesIS for data or updates to the da1a-base maaager, formulate Ieplies, and letum these replieS to the user via the C(l!Ju1mnW:ation mauage:r. Note a1sotbat the rornmmrication mIDager aDd data-base manager usually 1'UD as processes in the 1P system. Though the operatiDg.system may be implemeDted as a set of COIlCIIDmtly execnring pmc::esses, it is oftea ~ ftoma peEfunnaP:e viewpoint to not Ueat it as such. Radler, the ope[atiDg ~. in coqjuDc:tion with tbe hardwme, provides the envimmDeDt wi1bin which the processes execute.
same
DataBase The data base contains the data upon which· mmsaction replies are based. .1bis data is ~Y stored on disk in ~ systems but may be stored in other bulk star8ge.
Chap. 2
System Architectures
17
devices, such as large RAMs (random access semiconductor memories), bubble memories, or drums. The data-base subsystem includes the following: • One or more physical storage media and associated controllers, such as multiple disks and their controllers. • The computer 110 channels used to transfer data between the storage system and the computer's memory. This data transfer is usually made directly to or from computer memory (DMA, or direct memory access), with the processor being notified (via an interrupt) upon completion of the transfer. • The data-base manager, wbich runs as a process or a set of cooperating processes in the computer. Its job is to execute (readlwritelupdate) requests and other data management CQTIJTII8IIds (openIclose file, locklunloclc record, etc.) received from the application subsystem. It knows everything about the details for storing and retrieving information but knows nothing about the content of that information.
Other Peripherals Other periphera1s may be needed in the TP application. Most other periphera1s that one might find on a computer system, such as tapes, printeIs, and card readers, are DOt used for intemcI:ive processing. One exception might be a printer used to log status changes and to :request CODeCtive action by the operator. However, in contemporary systems, status logs are usually written to disk; the operator interfaces to the system via a CRT terminal, with the operator's actions also logged to disk.
SYSTEIf ARCHITECTURES A COhill!i()Jl cbaracte:ristic ofTPapplicaIions is that they grow. Gmw1b occurs asa IeSUlt of two fadoJs. One is simply the growth in transaction volume. As a system is suCcessfully used and finds accepIIDCe. amcmg the user CQIIlnpmjty, more and more useD • fOUDd·for the sysIaD.. Tnmsadionvolume grows-and .grows-somethlle5 with DO appmalt limit in sight. Home banking is an excellent example of this. What is the size of the potential user populatioD. for a home banking system? And what load will it impose OIl the system? No one knows in advance (at least, not anile time-of this writing). So hoW big a sys1Iml should a bank pmdlase to provide home banking? A little one in case the service doesn't catch hold? A big ODe in case it does? How big is big? The second factor that causes growth is inaased functioDality. A racetrack, for instance, buys a tota1irator system to handle wagering. This system is ~ to accept bets, maintain and display pools, calculate payoffs, and cash winning tickets. But once ~ system is succ:essfaIly DIs1aIled and nmning, management sees new and expanded ~
Transaction-Processing Systems
18
Chap. 2
for it. Exotic new wagers such as the Reinvested Quadrifecta (whatever that is). Telephone account betting. Off-track betting. The racing secretary's selection procedures and calendar. State audit reports. Maybe even payroll and general ledger. Many of these enhancements must operate during active wagering and therefore impose additional load on the primary TP functions. Oearly, the system needs more power than that called for by the initial specifications. Just as clearly, the "tote" system, having been procured under competitive bidding procedures, is configured very tightly. It has no excess capacity. What to do? .
Expandability The answer is to design the initial TP system with expandability in mind. Not with a hieruchy of "boxes" such that the user "trades in" (that is, tries to sell on the used marlcet) a CIl1'IeDt box for a bigger box, with all the attendant conversion effort required to get full advantage of the enbana:d features of the bigger box. Rather. expandabili.ty is
obtained today by choosing a multiprocessor or multicomputer architecture that allows expansions to system capacity to be made by simply adding processors or computers, diSks. and cnmnnlnication lines as volume warrants. And in today's art, such expansion can be achieved ~ DO softwaIe changes whatsoever-a paramount consideration.
This is not to say that there are not a lot of 1P systemS in use based on sing:1e-c:omput technology. There are. But understanding the performance of multiprocessor an4 multicomputer systems (distributed systems, as we will call them) allows one to easily model a sing1e-computer System since it;is simply a degenelate example of either. The basic structure of expandable systems is shown in Figmes 2-2 and 2-3. A multicomputer system (FJgUre 2-2) camprlseS two or IDOIe c:ompur.er systemS interc0nnected by a high-speed bus (typically with a capacity. of 2 to 30 megabyteslsecond). The bus is used primarily to pass messages between processes. This arcbitecture is called loosely cwpled because the coupliDg between componen1S·is oaly at the message·level (i.e., proc:esses being I'UD by c:tiffaeDt processors com""Jniane with each other by . ~manging messages). BUS
MEMORY 110
PROCESSOR MEMORY 110
PROCESSOR
MEMORY 110
Chap. 2
System Architectures
19
MEMORY
r-------i--CACHE
----PROCESSOR
BUS
CACHE
CACt£
PROCESSOR
PROCESSOR
I/O
A muJ.tiproc:esso system (FIgUre 2-3) comprises two or more processCn CODIleCted via a high-speed bus to a common memory. Each processor may bave its own small local memory (called a cache memmy) to try tom;Jrimjze main memory acc:essing. In this case, the bus speed is typically 10 to 80 megabyteslsecond because of its higher utilization in a bigher speed enviroDment. Such systems are termed tightly coupled because coupling is at the data level (i.e., processes being run by different processors communicate with each otber by &baring common data in memory). In eitber case, iDputIODIpUt devices are distributed across the processors or computers to divide up the I/O load appropriately. The multicomputer and the multiprocessor aICbitec:tures bave advantages and disadvantages ~ to each odler. A multicomputer.system requires more memmy than the multiprocessor system since the opetad:ug system and common process code must be mplicated in each processor iDstead of being sbaIed in a common memory. Intaprocess cx.u""maicatioD is also slower in a multicomputer system since such coomulDication is based on messages beiag passed tbIough a messaging system (typically, a few mj1Hsemads is xeqaired to sead a message to anotberprocess). WIth the availability of common accessible memory, a multiprocessor system C8D pass messages between ~ cesses in times measured in microsecoDds. FiDaJly, o.ace again from a perf~ viewpoint, a mu1Dprocessor system has iDhe.talt load balancing properties, since processors can operate off a single task queue maimained in COUlftW)D memory. For examp1e# a process may be ready to run and will be processed by p!ocessor 3 UD1il a disk can is made. It is thea paused aDd,..upon complelion of the data transfer, is requeued to the 1aSk queue. Wbea it works its way to the head of the queue, processor 6 is the next processor that becomes avaiJable, and it processes the next step in that process. Thus, a process will be passed at each step to aDOtber processor, and processors will be Opt 100% busy as long as there is WOJk to do. No such provision exists for the processors in a multicomputer system. Each is peasslgned a fixed set of tasks and typically keeps busy 50 percent to 90 peECeDt of the time. If a processor gets very-:busy, there is DO way for a less busy processor to share its load.
an
an
Transaction-Processing Systems
20
Chap. 2
On the other band, the multiprocessor architecture can suffer from a severe potential fault mechaDism. A sick processor (one with one of a set of specific hardware malfuuctiODS, especially in its memory interface circuits) or a sick. process may IUD amok aud CODtaminate critical parts of memory. The system has then crashed. A process in a multicomputer System, OD the other band, may protect itself from the faults of others by ensuring that all messages received from other processes at least appear to be sane and will Dot crash that process. Thus, a multicomputer system trades reliability for performance relative to a multiprocessor system. The common memory that gives the multiprocessor system its superior performance advautages is its Achilles heel when system reliability is cousidered. AD interestiDg modification to the above architecture is the hybrid approach ShoWD in Figure 2-4. In this architecture, a series of processing modules are inteICODDec:ted via a high-speed bus. "Ibis is a loosely coupled arcmtectme, since each processing module is a full computer, and proc:essiDg modules communicate via messages. A processing module, however, comprises a tightly coupled architecture of several processors executing tasks off a common queue in common memory. Hybrid architedmes have the potential of achieving the benefits of both loosely and tightly coupled architedmes while giviDg up little in terms of either peUODDaIlce or reliability.
.-------.,.-- - --- ----, BUS
PROCESSING MODULE
P
M
P
ij
P p
0
R
I
o
I
o
Y
I Fault Tolerance These expaDdable 8ldDtec mtes have ODe additioDa1 vay impcWmt capability. With a lia:le added ~ (atleast·concepmally). they may be made to be SIII'ViVable. That is, they will contiDue to maintain full fanctioDality in tbe presea.ce of-any Single (and often multiple)haJ:dwme fault. A commoa an:bitectwe applicable to lOosely coupled, tightly coupled, and hybrid architectures is shown in Figure 2-5. Basically. only two hardware eMancemeDtS need be made: 1. The bus. CODDeCIiag the processors or computeIs is replicated so that communication between processes can CODtiDue in the event of a bus failure. If both buses
Chap. 2
System Architectures
21
r-----' For I MEMORY I-Multi-processor I
I
L..-r----i 1 I
DUAL BUS
PROCESSOR
I
On~
I I 1
PROCESSOR
PROCESSOR
'or Computer
Comm Lines
are operatioDal, they share the oomnnmication load. If one fails, the othei assumes the fall load. 2. AU 110 CODttollers are dual-ported and CODIleCt to two processors. Thus, there is a path to every peripheml device even in the case of a pmc:essor or computer failme. In addition, physical disks may be dually ported to dual CODIrOllers so that full access to files is guamnteed even in the event of a ctiSk comroller failure.
Note that memory in a closely coupled system is geuezaJly DOt zepJic:amd. It would be to DO avail, since a sick processororproc:ess would comanrinate boCh memories anyway. However, memory in these ~ is panitionecl so that if a memoIy partition fails, it can be CODfigmed out of the system and the renurining memoIY used to run the system. (Sequoia Systems is an exc:eption in that it provides a system with replic:aled meJIlO!Y
protected by fault-tolerant processors.) In addition to the bardware configuration shown in Figure 2-5, sigDDicant operating system eDbancemeDU must be made to support fault tolerance and IeCOVeiry. These are described in later sections in tbis chqter.
22
Transaction-Processing Systems
Chap. 2
Sammary
In terms of contemporary offerings, fault-tolerant loosely coupled systems have been offered by such vendors as Tandem Computers, Tolerant Systems, Digital Equipment COIp., and Concurrent Computer. Tighdy coupled fault-tolerant systems are offered by Sequoia Systems and Concmrent Computer. Fault-tolerant hybrid systems are offered by; Stratus and Parallel Computers. Other distributed-system vendors include Arete, Enmasse, IBM, Nixdorf, and NCR, among others. . Another architecture not described is that of a dually redundant data-base system. . supporting multiple intelligent worlc-stations. SuCh systems are offered by No Halt Computers ofFanningdale, Long Island, and Symrex qf Eatontown, N.J. (Synttex.' s offerii:lg is basically a word-processing system.) . Note that a fundamental property of distributed systems, as these architectmes are collectively called, is that of trfJ1lSp(lTency. Since a process can be nm in any processor in the system, it must be able to communicate with any other process or peripheral device, no matter where that component currendy resides_ Thus, the configuration of the system. must be logically transparent to the process. It cares not where the other processes and 110 devices are. It simply wants to be able to communicate with them. In the following sections, the concept of a process, its management, the management of memory in which it xesides, and its role in transparency and survivability will be discussed further. Excellent in-depth coverage of distributed systems is given by Uebowitz and Carson [17].
TRANSPARENCY
In oider for a distributed·sysrem to be completely genera1-purpose, the first and foIemost requi!ement is that the user must not be mquired to be aware that it is a distributed system. Wilen peparing progmms, the user should not be concemed with the processor to be used or to which pmcesso!S the various perlphenls to be used are connected. 1be opemdng system must povide all the featmes xequiIed for any program l'UDDing • _.:.I. jpbaal.a-..:_ --.1.._ • on one processo.rto ca"'ntnmcate w&u& any ~ \lWYI\iIW or any U&UIOiI pIOgEam nmnmg on my other processor. Tbis chaDctaistic is called t1'tl1ISpQ1'ency. It hides the in1ric:acies and complexities of the disIribured environment from. the user, who thus can treat the system as if it weze a single computer. .
711eProcess The concept of transparency is illustrated in Figure 2-6. Let us coasider a simple data-base inquiry program. To the user, tlUs is a program that accepts an inquiry from the user termmal, accesses a data base stoled on disk, and retums a response as shown in Figure 2-68. In a siDgle-processor system, FIgUre 2-68 might be a good repesemation of the J'hysica1 patbs involved in the application.
Chap. 2
Transparency
23
REQUEST
•
INQUIRY
fafLlCAT!ON
Path WreBlBNOf (b)
Fipre u
Distriba1ed system U8IISp8IeIICY.
In a distributed system, FlgUIe 2-6a rqxesents how we would want to view the' applicalion logically. However, the physical representation might be quite different, as shown jn Figure 2-6b. Here the application is IUDDing in a four-c:omputer system. The opemtor's termiDal is physically connected to computer 1, the blquUy process is physically rmmiDg in computer 3, and the disk is physically CODDeCted to computer, 4. Via the iDtetprocessor bus, however, the termiDal is logi.cally COIIIleCted to the inquiry process, which is logically CODDeCted to die disk. It is die logical CODDeCtiODs that are apparent to the user (Figure 2-6&); the physical CODDeClions (FlgDl'e 2-&) are tlauspaieat in that the user doesn't necessarlly bow whidl computers are iDvolved in a specific application. Thus, the operatiDg system provides traaspareacy ifthe application pogram teqU.i:les DO advance knowledge of the system c:oafigmalion. Tbe idemical iDquity pogram shown in Figure 2-& would support a aa:miDal CODDeCted to CUiiipidm 2 c:ommnnicatiDg with a disk on computer 1 wbile it itself ran as a process in comp11Ia" 4, 8Ild so on. Since the physical compotas are immaterial in the design of an application for a truly traDSpareDt system, let us resttudUre our tbjnkjng a little by redefining the followiDg terms: • A computer (or CPU) is a physical piece ofbantware comprising logic, JDeD1OIY,
and 110• • A program is a physical set of object code, probably residing on·a disk connected to some computer.
24
Transaction-Processing Systems
Chap. 2
• A process is a program running in a processor. (There is nothing to restrict multiple copies of a program from numing in one or more computers, thus creating several like processes, each perhaps handling a different terminal but otherwise providing the same application as its companion processes.)
• Plrysical means the way things Ie8lly are. • Logicol means the way things appear to the user. Thus, a process is the logical IeSUlt of a physical program running in a physical computer. The user sees the application provided by the process but is not aware of wbich physical computer is being used nor of where the physical program resides. A process, then, is the basic logical unit within a distributed system. Each logical task is handled. by a process. To the user, there are two types of tasks to be performed and therefore two types of processes: application processes written by the user and devicehandling, or 110, processes provided. by the operating system.
I/O PlVCesses We have already talked about application processes. Device-hIlndling processes, or I/O processes, are typically considered part of the operating system but are in fact identical in structure to application processes. Their job is that of the classical device handler: they handle traDSfers to and from their IeSpeCtive devices (writes and reads), as well as other control functions, in IeSpODSe to requestS from appli~on processes. They ctiffer from application processes only with regard to certain restrictions: • An I/O process must reside in the same computer to wbich its coae&pOnCliDg device controller is auached. • An I/O process CIIIIlOt origiDare c:ommunicaIion to another process (except to its backup, as described later); it can only respond to requestS from other processes. • An I/O process can execute I/O iDstructions, whereas an application process can-
not. Thus, the inquiry application of Figure 2-6 does not involve just the application process. It also involves a trmrinaJlIO process anela disk 110 process. The logical structure of the application is tbeEefore better ~ by Figme 2-7. Here, the physicai intmprocessor bus has been replaced with a logical inIaprocess bus. Application pr0cesses are shown above the bus and 110 processes below the bus. Figure 2-7b shows a DlO1'e extended application in which 0Iders are entered from several tenninals. The Older entry processes (one perterminal) access various files on disk to verify and build the order on disk. 0Dce an Older is complete, the common invoice process is informed so that it can print an invoice on the primer. It reads the disk-IesideDt invoice file and prints the requesred invoices. (Note: AD application process can be ,designed to handle a single termiDal, as above, or multiple terminals.)
Chap. 2
Transpanency
25
APPLICATION
PROCESSES
I/O PROCESSES
INQUIRY APPUCAT10N (a)
ORDER ENTRY APPUCATlON (b)
Fipre2-7
~assetsofpilCc 1111.
Transaction-Processing Systems
26
Chap. 2
The example of Figure 2-Th shows -that application processes not only communicate with 110 processes but also with other application processes. (An 110 process will never communicate directly with another 110 process since an 110 process cannot initiate a communication; it can only respond to one.) In fact, except/or irs inrenuzl processing, the only thing an application process in a multicomputer environment can do is to communicate with another process. As we shall see, it is the simplicity of this statement of the role of a process that leads.to the elegance of the multicom.pu~ structure and forms the basis of the .transparency and survivability functions. A process in a multiprocessor enviromnent is not similarly restrained but still forms the basis for transparency. This description of processes is exemplary of contemporary systems today. No bard-and-fast rules are followed, and the propenies of processes in different systems may be somewhat different from those described here. However, the principles described are sound and will form a satisfactory basis for understanding the nature of a process and its management for ped'ormance analysis purposes. . Interprocess CoIIImunications
mterprocess
Let us now look in more detail at c:ommunica1lons. One process communicares with another by sending it an interproceSS message. To do so, it maely provides to the operating system the name of the process, the content and length of the message, and whether a response is required. Note that this leads to three types of intelprocess messages, which we will desigaate
as follows: • Write. A message is sent to another process with DO IeSponse xequixecl (except for completion SWUS, i.e., success or euor condition). • Read. A null message (i.e•• DO data, leDgtb of zero) is sent to another process with a zesponse expected. • Wrireread. A message is sent to another process. and a response is expected. In !be example ofF1gU1'e 2-7b. an order entry process might mum infOImation to its tmninaJ and wait for tile next opemtOJ entry. This is a WRlT.EREAD iDteJ:pocess message. When it JeCeives data from !be opentm. it may verify or expand certain infODDalion, such as part number, by senctiDg READ messages to the disk process, which will lead tequeSted clara from !be disk and mum it to tile order entry process. . The order entry process may tbeD write invoice data to disk by ~ WRITE messages to the disk process and will tbeD inform the invoice process that the. invoice is n:ady by sending it a WRITE message. The invoice process will READ the invoice from the disk process and wID print it by sending WRITE messages to the printer process. Note again that 110 processes never send inteIptocess messages; they can only IeSpOJld to them. .
Chap. 2
Transparency
27
In order to support interprocess messages, the system must provide tbIee facilities: • a hardware path interconnec:tiDg all computers, with supporting software.
• a provision to assign a unique name to each process. • an operating system capability to know in which computer each process is currently operating. The hardware path can be implemented in any one of a number of ways, as shown in Figure 2-8. Each has its own advantages and disadvantages: • The bus is simple but must resolve contention and carry the entire multicomputer load. • The ring can use different paths simultaneously but places a load on computers not involved in the message. .
• A star can use cWferent paths simultaneously but requires a4ditional hardware: the switch. The switch must carry the entire multicomputer load. • A fully CODDeCted system provides maximum capacity bot requires N-l bus connections at each computer for an N-computer system. . • A partially ccmnected system is app1ication-depend.
oEjDEJ BUS
RING
STAR
FULLY CONNECTED
PARTIALLY CONNECTED
. . Transaction-Processing Systems
28
Chap. 2
No matter which network is chosen, the important fact is that it is transparent to the
user. We do not care bow an interprocess message physically gets from one process to another as long as it'logica1ly gets there. In most contemporary cases, the bus netWork is used. However, DEC's VAXcluster is a star network; and Stratus uses a modified star/ring network.
Process Nalll8S The next facility required is that of process 1Ul1DeS. A process is named when it is created. A process is created at tbe request of an operator command or at the request of another process. Creating a process causes the operating system, in the CPU in which the process is to run, to schedule that program (i.e., object code on disk) to be rolled into memory and run as the named process. A typical command to run a process might be RUN ORDER / NAME ORDERl, CPU 3, PRI 150 I.
This command would cause the program with the physical object file name ORDER (on disk) to be run on physical computer 3 as a logical process named ORDERl at priority ISO. (If the CPU weze not specified, the process would run on the same CPU as the creating termiDal or process. If the Dame were not speci1ied, an arbitrary UDique name would be assigned. But no process other than the creator would know this name; therefore, no process other than the creator could COIDIIlUDicate with it. If the priority were DOt specified, a default priority would be used.) . Finally, tbe operating systeDi must know in which computer each process is oper~ ating. Whenever a process is aeated, aU computers me informed of this event (via tbe bus). Each computer maintains a process diIectoIy that contaiDs the name of each process and the CPU in which it is cmreDtly running. I/O processes are Cleated at system generation time and are pe.nnanent entties in the process c:tinc:toJ:y. An exception to this is disk files. A disk I/O process COD1!Ols one physical disk controller. This COId!Oller may be connec:red to seveml physical disk uDits. Many named disk files me on these disks and are tbe:refore accessible tbrougb a common disk process. However, the application pmgmm. wiD. waDt to Jefere.nc:e a file by its name; it doesn't need to know the name oftbe disk process with which it must comnmnicate·jn order to reach the pbysic:al disk containing that file. Disk files are typic:aUy cqaniml into volumes, each volume br;iDg bandled by a specific disk process. The name assjgned to a disk process is, jn fact, the name of tbe volume it is JvmdHng. When a zequest is made to lad ore write a disk file in a given volume, the process dkectoly is sem:hed to cI.etermine the process and
CPU handUng that disk volume. . Thus, wbeaa process wants to send an inteIprocess message to anotber process, its local operating system looks up the process name in its process directory, ~the CPU in wbichtbat process is running, and, if it is in another compurer, sends the ~ over the iure&pioc:essOr bus to the receiving process. If a teSpODSe is expected•. it WiD. zetum that response to the ttansmitting process when it is received. .. Note that. the process itself neied have DO knowledge whatsoever of where other processes IeSide nor bow to get a message to other processes. These fandions have been . totally handled by the openting system, thus providing the traDspmency we desUe.
Chap. 2
Transparency
29
The fact that a process may be mobile (i.e., move from one computer to another) leads to two important characteristics of multicomputer systems:
'. Load sharing. The creator of a process can request the status of all computers and can then create the process in the least-loaded computer. It is common practice among programmers to display the status of all computers before deciding in which computer to run a compile, edit, or other utility. • SurviVflbility. Process mobility allows us to re-create a process (or switch to a backu.p process) if another computer that was nmning that proceSs bas failed. This subject will be dealt with later in depth.
Of course, in mu1t:iprocessor systems, no such mobility is required, as the system. is automatically load balancing; a failed processor simply does not execute any tasks off the
common queue. A final point should be made about system. transparency. As we have said, in a ttuly ttanspareDt system., the user is UDawate of wheIe the various peripheIals and processes are iii a distributed system bec:aDse he or she is unaware of the mechau;sm involved in routing a message from. one process to another. This mechanism involves the bigh-speed inter-
processor bus. Though the high speed of the bus is necessary to achieve good system througbput, tbat is the primII:y le8S01l for its speed. 'I'hele is no logical reason whypart of the bus could not be a slower coml!VJDicaDon line, allowing geographical separation of gmups of c0mputers. The system of Figure 2-6b might be distributed as shown in Figure 2-9. 'Ibis pmvides a geographical disb:ibution for a Ialge system, which can offer many advantages:
,/
."",-------------- .......... -',
INTER~aus
,
\
\
\ '--_-:::3=
COr.PUtERs
~
Sf. LOUIS NODE
~
CHICAGO NODE
4
Transaction-Processing Systems
30
Chap. 2
• The use of a common corporate data base without excessive line costs. • Load balancing in a large, geograpbically distributed system by running large jobs at the least-loaded node. • Printing of reports at the user's site even if the data base is remote. • Message delivmy among all users of the diStributed networlc. • Local control over local processing capability while having access to the networlc for load sharing and data-base access. This distributed system requiIes DO substantial effort on the part of the user, for the
same transparency conceptS of inteIp.rocess communication hold as described previously. It is only necessary for the operating system to be able to know the geograpbical node, as well as the computer in which a process is running via the process ~.
Sulllmary We have seen that distributed system transparency has been achieved by considering an application as a COIDID1lDications networlc. The nodes of the networlc are the processes which communicate with each other via inteIprocess messages over an intetprocess bus.
As. in any COIDID1lDications system. node or path failures may cause loss or duplication of messages. This is one of the major consideIations when we discuss survivability. Also, communication path delays and queuing at the nodes impose a limit on the capacity of the system. 'lbis will become the basis of our tbroughput aualysis.
PROCESS S17lUCTURE We haVe described the process as the fuDdamenta1logical UDit of a distributed system from the applications viewpoiDt. A process is sbnply a progIam naming in a compurer. There are two types of pmc:esses: application processes, which peIform a portion or all of the app1ication-orient logic, anclIlO processes. each of which is a device bandler for a speci1ic device or device group. An application is implemented by a set of appIopriare processes that COIDIDUIIicate among themselves via iDtap.roc:ess messages. In Older to understand the ~ used to achieve survivability and to be able to analyze aDd enhance the throughput of a distlibuteci system, we must understand a process in mo.te detail. In this section, we will describe a typical structme for a process and then explore how it is managed by the operadDg system: .
Process Functio_ Since a process is a program-io the classical sense-nmning in a computer, it has some of the characteristics we natmally associate with programs. That is, it comprises a set of .jnsttuctions, or code, that operates on data. However, seven! restrictions are p~ on the
Chap. 2
Process Structure
31
code and data at this point that make a process a subset of what we generally think of as a program. In fact, a process.is allowed to do only two things:
• Perform its assigned logical duty by operabng on its own local data (this includes communicating with and controlling its device in the case of an 110 process). • Exchange interprocess messages with other processes. These messages are used to pass data and control functions from one process to another. A process is specifically DOt allowed, for instance, to modify its own code, to access or manipulate the data of another process, or to directly interface with a device Dot under its control as an 110 process. The strudllIe of a process is shown in Figme 2-10. We will characterize it as comprising three parts: • A process control block (PCB), which is used by the operating system to schedule and control the process. • The code that implements the functions of the process. • The data local to the process. upon wbich the code interacts. PROCESS
QUEUE OF RECEIVED INTERPROCESS MESSAGES SYSTEM
S'I"STEM '-- CALLS CODE , - - - - -
-------OPTIONAL. RESPONSE
...
CODE
INTER-
~s
TRANSMiTTED INTERPROCeSS MESSMES _ _ _ _ _ _ _ -SErtr: OPTICINAL RESPONSE DATA
Iipre 2-18 SUacIIa of a pocess.
As shown in FIgUIe 2-10, the process is allowed to do only two things in addition to ~
its own local da!a. h may exchange messages with other processes and may
Transaction-Processing Systems
32
Chap. 2
make certain calls to routines (not processes) contained within the operating system (these calls are considered part of its assigned logical duty). Only three types of"interprocess messages may be sent by a process:
• A message to another application process, perhaps with a response. These messages are used to pass control functions and data between application processes. • Read, write, and control requests to I/O processes for peripheral device activity. These messages are similar in all IeSpectS to messages sent to" other application
processes. • Data to its backup process, if any, to worm the backup process of the cuuent status of the data in case a failure should cause the backup to have to take over procesgng. These "checkpointing" messages will be desaibed later. The first two message types may be sent by application processes ODly. As we have said before, they cannot be origiDatecl by I/O processes. However, any process-application or I/O-may send a ~ message to its backup process. 'Ibis forms the basis of one of the survivability strategies, which will be descn"bed later. In addition to these interproc:ess messages. a process (application or 110) may receive messages tbat appear to be interproc:ess messages but are in fact generated by the operating system. These "messages are generally status messages conceming situations, such as processor down or up events or the stopping or aborting of a process that was cteated by this process. " As shown in Figme 2-10, messages sent to a process need not necessarily be pr0cessed i1TlTlWiiately. Rather, they are queued to the process (via links in the PCB) until the process is ft:8dy to lead and process them. This queuing facility will become impch1aDt in our discussionS of survivability, since there may be no way for the backup process to know about messages queued but"DOt read by its primary process. DOr may theIe be any way for 1be opemtiDg system to knoW tbat a message has beentead and processed but DOt aaswaM. Lost and duplicate messages in the event of a takeover are as importaDt in multic:wlputei survivability consickntions as they are in any communicatioDs a e t w o r k . " " The system calls tbat a process can make to tbe.operatingsystem generally fan into two classes:
"
• CaDs to routiDes ~ are sbDply common"routiDes supplied by the operaIing system but which in reality are logically part of the process's code. For the DlOst part,
these lOlltiDes" impIemeat1be "transmjssjon _ DCeption of the variQUS "tYPes of interprocess messages. They are fODDally part of the operating system because they use system-level tables and common buffer pools tbat must be "bidden" from the application programmer. • SchednUng zequests for itself and other processes. A process can IeqUest that it be suspended until anyone of a set of events occurs (discussed later), or it can request tbat another process be created or stopped. "
Chap. 2
"Process Structure
Addressing Range
::tbe rather formal structure of a process, as described above, bas important implications with respect to addressing within the process. In the process structure defined above, there" is DOt a requirement that any memory location in the system be available from any memory location within a process. Conversely: • The code cannot modify itself. Therefore, a data instruction can access only the data area, and a jump, branch, or routine call instruction can refer only to the code area. Thus, code and data spaces are mutually exclusive. In physical memory, the code and data areas are separate and identifiable . • A process can access only its own local data area. No addressing range is necessary or allowed to access the data within the local range of another process.
Note that one process does not have direct access to the data of another process. This is not an arbiuary restrictiEDl but is necessary to the ftmdamental requirement of survivability. It is of paramount importance that the data area of a process not be corrupted by another process tbat is "sick" because of faulty hardware or software. If a faulty process had access to the data areas of other processes, it could indeed COIrUpt those areas and cause a system failure. However, processes bave access to the data of other processes only through the medWJism of inteIprocess messages. 'Ibis gives a process the ability to edit and reject improper accesses to its data area, thus ensuring to any arbitrary extent the integxity of its own data. 'Ibis may not be ttue of multiprocessor systemS that share a common memory.
For purposes of some of the following discussions, particularlywitb tegard to survivability, it is impo:rtint to understand in more detail the SI:rUCtDre of the code and data in a process. The code area comprises a collection of p:ocedures. A procedme is a routine that bas an entranCe and an exit (or perhaps multiple entraDces and exits).·One procedme is . designated the main ~ It is given control whentbe process is first: IUD and bas DO exit except to stop the process. It ucalls" other p:ocedures, which may in turn call other proc:ecbues, and so on. When a procedure exits, it Ietums CODtrol to the procedure that called it at the point in the code ionOwing the call. There is DO other way for a procedme to exit (short of termiDa1ing the process). FlgUIe 2-11 shows a typical code mea and its associated data mea
areas
as processing' progresses through several stages.
"
Prvcess Data AnNt
.J'he data mea is separated into two distinctly differalt areas, global data and the proCess
:local data, the Jatteroften implemtmted via a stack, as will be described. The global dab!-
Transaction-Processing Systems
34
MAIN PROCEDURE
~DURE PROCICURE
~-l PROCEDURE
PROCEDURE " 'DATA C CODE
T
STACK STACK
IWLI MAIN DATA
~
r-
MAIN DATA
~-------
~ I}-
~~S
(b)
(0)
MAIN alTA
t--'------
MAIN CALLS A PROCEDURE
MAIN EXECUTING
GLOBAL DATA
P-u..i'SSED
GLOBAL. DATA
GLOBAL. DATA
Chap. 2
GIsf.PlL
'\R'"
GLOBAL DATA
MAIN DATA
MAIN DATA
MAIN DATA
1------ r----- 1-----1----- 1------ 1------
A PARA. MAlNSM
A DATA
8 PARA.
B PARA
BMRA.
MAINSM
MAlNSM
MAIN SM
8 DATA
BDATA
BDATA
CPARA ~---BSM
1-----CDA'Ia
MAIN CAUSA
A EXITS 10 MAIN
B
MAIN
CALLS 8
CALLSC
C EXITS
TOB
B EXITS
10 MAIN
PROCEDURE EXECUTION (e)
fipre 2-11 Proc:essiDg cycle.
'is available to all procecluRS within the process. The stack is used to hold the ctara: that is local to the procecluRS currently invoked and to COD1rOl the aestiDg of those procecluRS. Tbae is some similarity between processes and procedures. Just as a process bas exclusi~ access to its daIa mea, a procedme has local daIa to which OIlly it bas access. Mcnover, just as a process can pass daIa to another process via an iaterp.&ocess message, so can a procedme pass daIa to another procedme when it caDs it. Tberefoie, a procedme has access to 0Dly dnee types of.~
• The 'global daIa of the process, which is accessible by all pmcedures witbiD tbat
process. • Its own local data. • Dam passed to it by the procedme that c:alled it.
Chap. 2
Process Structure
35
The Stack A stack allows data items to be pushed onto it and then popped from it in inverse order (last-in, first-out). The last two types of data listed above are aeated in the stack area and are then deleted as procedures are entered and exited. Therefore, this data requires addressable meDlOIY space only when it is meaningful, i.e., when the procedure is invoked. The data simply does not exist when the procedure is not bemg used. The use of a stack for procedure data thus brings a degree of efficiency to memory utilization. Figure 2-11 shows an example of stack usage as a process executes. When the process is first created, it is given a data area containing its global data and the local data for the main procedure (since the main procedure always exists, so does its local data). The code and data areas.at process creation time are shown in Figure 2-11a. As the main procedure executes in-IiDe code, the data area remains in this connguration. However, when the main procedure decides to call another procedure, it does so by pushing aprocetlure data set onto the staclc. The procedure data set comprises all of the data needed by the called procedure and, in fact, is the only data available to that procedure (except, of course, that the global data is available to all procedures). The procedure data set comprises three types of data, as shown in Figure 2-11b. • The parameterS being passed to the called procedure by the calling procedure. These parameters can include pointers to data in the global area or pointers to local data of the calling procedure. '
• A stoc1c 1IUIT1rD definjng how the procedure is to exit. The stack marker contaiDs the address in the calling procedure to which the called procedure is to retllm (the addless following the procedure call) and the macbiDe enviromnent that is to be IeStOIed following the called procedure's mum. • The local data for the called procedure. The called procedure then executes, operating on the pammeters passed to it and the global data pertineDt to it. When it bas c:ompleted. it deletes its data set from the stack and IetumS to the calling procedure accoJding to the stack matter. Of course, a procedure can call another- procedure, which can call aaotber, and so OD. In this case, the stade grows, with the 'nesting of pwcedwes being defined by the sequence of SlaCk JDalbrs. . Figure 2-11c shOws the stack COIIfigmaDoil as the maiD procedure executes. The main pmcedw:e fiIst calls procedure A, which executes and IetUms. The main procedure ~ caDs procedure B, which in tum caDs procedure C. At this point, the stack CODtaiDs two procedure data sets, one for B and one for c. Procedure C then completes, zetm:Ding to B, which itself completes. zetuming to the main procedure with a clean stack. A vital point that will fODD one of the foundations of our survivability discussion is ~ the data area with its stack completely defines the state of the process at any ~ in,
36
Transaction-Processing Systems
Chap. 2
its processing cycle. If we were to interrupt a process and then could somehow move its data area as it then stood and its code area to another processor and restart execution at the interrupted instruction, the process would continue to IUD as if nothing had happened (baving, of course, copied the machiDe environment in terms of memory maps, condition codes and so forth). More about this later.
Summary In S1lIDIIlaIY. a process is the basic logical entity at the applications level in a distributed system. It can do only two things: perform its own assigned processing functions (including process scheduling) and exchange inte!process messages with other processes. An application process can send interpIocess messages to other application processes, to I/O processes, and to its own backup. An 110 process can respond to messages received from application processes and can send messages to its own backup. A process comprises a data area and umnoditiable code that implements a series of procedureS. The data area contains global data, which is available to all procedureS. and a stack. The stack holds all local data pertinent to the procedureS CUIIeDtly invoked and controls their nesting. The data area and the CUDeDt machine enviromnent totally define the c:mrent state of the
process.
PROCESS IfAlVAGEIIENT Having described the process as the basic logical element in a disuibuted system, and baving explO!ed its struc:mre and the way in wbich it interacts with other processes to implement an application, we must DOW tum our attemion to how the process shares its envhoDmeDt c:oopem!ively with other proCesses Competing for the same resources. Obviously. in a large distributedsyste!D, many things go on simultaneously. Many users peIfOlJD their various 1aSks wbile siUing at many terminals while operatodess backgrouDd 1aSks may be at wade npdaring data bases, maintaining statistics, and doing sundry other tasks. Many processes me active simultaneously.
Shared R...".. Processes in ODe coiiiputer geaaally nm DldependeDtly of processes in other computers. However, .that JlOUP of p.rocesses assigDecl to a giveD. compure:r (aD processes in a multiprocessor system) must sbm:e its IeSOUices in suc:h a way that all processes may Complete • their tasks in a timely maDIle1'. 'Ibere. lie two domiDant computer resources to be CODSideIed: time and space. The time RSOUtCe is processor time. Thougb it appears to the user that multiple processes lie l'UDIliDg COIlCUl'IeDt1y in a processor, we know. in fact. that at any instant only one process is actually numing.lt will run until it decides to reJinquish control of the processor. at which time another process will be given the PJOCeSSOl. Therefore, there :must be a way to know which processes are. awaiting the processor and·to transfer control
Chap. 2
Process Management
37
of the processor from one process to another in an orderly and efficient manner. This is called process scheduling. A necessaxy efficiency in process scheduling is that processes Carry priorities and that, in fact, the highest-priority process desiring the processor bas it. The space resource is memory. The sum total of the memory requUements of all the processes that are to run in a given computer could well exceed that computer's memory size. This is particularly true with mobile processes in a multicomputer system, for we do not necessarily know in advance which processes are going to run where. Therefore, we must have a way to allocate the available memory to those processes cuaendy requiring memory. This is called memory 1M1'IIlgement. The functions of process scheduling and memory management in a distributed system do not differ sigDiftcandy from,the way in which they are bandied in any one of many modem-day single processor operating systems; the reader who is familiar with these operating systems will already understand the concepts to be discussed. However, they are important for our following discussions of survivability and peIformance and will theIefore be described so that we will all be talking the same language. Process Scheduling Let us first discuss process scheduling. We assume that seveml processes have been aeared in a computer and are currently taking tums running in that computer. A process can be in one of three Slates: • Waiting. The process is currently idle but is waiting for one or more events to occur. Until one of these events occurs, theI:e is notbing the process can do; therefme, it is not a candidate for srJteAuling. These events may typically be one of three types: (1) the receipt of an intelpJ:oc:ess message; (2) a device interrupt sigDjfying an action has been completed and IeqUiIes process action (for 110 processes ODly); (3) a time iD1erval specified by tbe process has elapsed
• Reody. An event bas oc:cwted, one tbat must be bandled by the process. The process is ready to make use of the COIIIJIUteI:,wben it is given the chance. • Active. The process is CUI'Ia1tly the process 1'UIIDing in the compaIet. (Note thatthae is DO such thing as a cIonmmt process. Once a process is ~ it is waiting, leidy or active. If it decides to stop itself, it DO loager 'exists. The physical program that was used to Clate it still exists, but tbe logical process has disappeared from the system.) . The active process will coatinue to run and to consume all processor time (except for intenupt and cycle-stealjng activities, which are transparent to it) until one of the follow-
ing situations occurs:
,
• It decides it must wait for an event (interplocess message or response, intellupt. and/or time interval). .
Transaction-Processing Systems
38
Chap. 2
• It decides to stop itself. (The process is of no further interest to us. since it is no longer a candidate for scheduling.) • A higher-priority process becomes ready (it will preempt the active process).
In the first case (waiting for an event). the process goes into the waiting stare. In the second case (termination), the process disappears. In the third case (higher priority pr0cess). the process returns to the ready state. In any event. the highest priority ready process, if any, is given control of the processor. This switcbing of processes proceeds , indefinitely and is the basis of process scheduling within a computer.
Mechaniams for Scheduling Processor schedu1ing is handled via two fairly straightforward mechanisms that we will call the ready list and the timer list. The ready list is a list ofall processes in the ready state, in order of priority (within the same priority. the processes are orclerecl on a first-come basis). The timer list is a list of all processes awaiting time-outs, in order of their time-out interval. A process may be on the timer Jist. on the ready list, or on neither. Processes are maintained in a list via links in their PCBs. Figure 2-12 shows a typical process scbeduJjng sequence. In Figure 2-12a. 10 Cleated processes. named A through J. are running in a computer. Process A is c:unently the active process; it is the one that is actually nmniDg in the computer. Processes B. C. D. and E are all ready to run but are lower priority dian process A (priority I is the highest. to simplify tbis example). 1beIefore. they are linked in the ready list via thcir PCBs, with process B being at the head of the list, since it is the highest-priority ready process. Processes F. G. H. I. and J are waitiDg for some event to occur. and processes F. G, ancI H are faab«more being CODtroIlecl by time-outS. For example, if an event does DOt occur within six seconds, ~ G will become ready myway. (A process can also wait simply for the pmpose of timing aad DOt for any other event but its own time-out). Processes F, G. and H are liDked via their PCBs into the timer list, with process F beiDg at the head of the list, since it bas the shortest timo-oat. TJIDeoOUt values in the PCBs for tbis example are iDaemeotaJ. values to be added to those abead Of them on the Jist. 'lbns, processes F, G, and Hare waitiDg 1,6 and 16 seconds, respectively. In Figme 2-12b, process Z has been czeated and has been added by the operatiDg system to the zeady list. Since it bas bigberpriority than any oCber process in the ready list, it is pIacecl at the head. of that list. . In Figure 2-12c, process A bas completed and has ~ itselfumil it receives another evem. h is DOt timiDg on that evem. Process Z. being thebigbest-prlority ready process, is JeJDoved fiom the ready list and made the active process. In Figure 2-12d. process Z bas suspeDded itself to await an event and bas IeqUested an eigbt-second 1ime-ouL It is themoIe pJacecl in the waitiDg state and added to the timer list between processes G and H, wbich have smaller and jrearer 1iine-outs. IeSpeCtively (process H's time-out is adjusted by Z's incremeDt). Note that F timed out and bas been added to the ready list. Also note that procesS J bas xeceived an event and bas
Process
Chap. 2
rj:l-
READY UST
TIMER LIST
loicHSil--II!c1
~
I~IIP~I
WAITING PROCESSES - - -
INITIAL CONDITION (a)
~tfY fZjlZ]1ZJ1;sl-GrJ •
READY PROCESSES
I
Irs~R ~11!cl I~I
[!l
WAITING PROCESSES
PROCESS Z IS CREATED (b)
~--tE!J-I=: B
READY LIST~~~ I
I 1;j:1 ~
READY PROCESSES
~~ 1K)~lttl-W
I~ I 0 IT]
WAITING PROCESSES
PROCESS Z RUNS (e)
READY~~ ~ eLI ~ le&.,gJ IeaYJgJ
LIST
READ\' PROCESSES
nMER'~ LIST
lzkj
h&J
~
~ 1P<21 ~
WAITING PROCESSES
PROCESS
,I) SUSPENDS
READt
LIST
TIMER~ LIST W IukI
I
PCI \ IPCBII A I·
WAmNG PROCESSES
PROCESS (~) USURPS Figure 20-12 Pzocess schedn1mg.
I
39
40
Transaction-Processing Systems
Chap. 2
therefore been scheduled by being added to the ready list. Since process Z is now suspended, process B is removed from the head of the ready list and is made active. In Figure 2-12e, process Z bas received an event prior to its time-out. It is removed from the timer list, and process H's time-out is readjusted to include what had been Z's time-out interval. Process Z bas a higher priority than the currently active process. Theii~ fore, it is not placed on the ready list. Instead, the cuuently active process B is returned to the head of the ready list, and process Z is made active, thus usmping control of the processor from process B. 'Ibis is called preemptive scheduling. The foregoing example shows that processes share the processor by simply moving from active to waiting to ready and back to the active state again. Via the timer and ready lists and via appropriate actions in response to events by the operating system, the cuuent state of each process is known and controlled. Process scheduling gives a computer a multitasking capability because. in effect, several differeat tasks may be running CODCUDeIltly.
IIeInory IIanageIIIent TumiDg DOW to memory m.aoagement, the problem is how to stuff five pounds of potatoes into a two-pound bag. Since in a distributed system we generally do not know in advance what processes will be IUDDiDg. much less in which computer, we must be prepared to support a set of processes that have memory requirements exceMing the memmy availability of the computer in which they are ruDDing. perhaps by a wide margin. 1be requirements of a single process might even be bigger than the memory on its computer.
Paging The fust step in memory maDagemeDt is to realize tbatall processes oftenC8DDOt fit into memory simu1taDeously. but somehow the COD1eDts of their data areas must be maintained. Therefore, an image of the process's data area is created on disk wIleD the process itself is aeated. (Code areas never cbange, and their images are already available on disk.) This disk image repleseDts the c:anent CODteDts of all meiDOry locations in the. data mea tbat are not CUD'eDtly in the physic:a1 memozy of the compurer. . It is the msponsibility of memolY management to easm:etbat if a section of a pr0cess's data area is to lose its place in memory to ~ poc:ess, its disk image must fust be updated. Since the disk image of a process (code aad data)ismaely a IepIesmtatioo of the true process, which exists only when it is adUally in memory, it is called 'VinlIIzl 1II!f11ID1'Y. By proper management of vhtual memory, theoompatec can be made to tI[1ptIflT to have a memory as big as is needed to hold all proCesses. . Virtual memory may be mauaged via a mechanism known as pagefrJulting. Physical memory is organized into "pages," typically O.SK to 4K bytes in size. When a process adchesses a WOld in its memory (either in its daIa area or in its code area). the processor (by bardware) eosures that the page containing tbat word is,.in fact, in memory; if not, obvi~usly , .it c:aDIlOt be accessed. .
Chap. 2
Process Management
41
If the processor finds that the page currently being referenced is not in memory. it generates a page-fault indication. This means that the processor must bring this page into memory from its disk image before the C1D'reDt process can continue. In order to do that. the processor must select a memory page that it is willing to overwrite with the new process page (note that every memory page contains valid code or data from some process or another). The algorithm used to select a memory page for destruction varies from system to system but is generally based on two considerations: • Age. When was this page last referenced? If the contents of a page have not been used for awhile. the page is considered dormant and a candidate for overwrit-
ing• • Modqication . ..Dirty" pages are data pages (never code pages) that have been modified since they were last read from their disk images. If a dirty page is to be overwritten. it must first be written out to update its disk image. whereas a clean page does not need to be written out. Clean pages. therefore. have priority over dirty pages as candidates for overwriting. since less system resources will be
used. Once the processor bas selected a page for overwriting. it will write it to its disk image if it is dirty (bas been modified) and will then read in to that same physical page in memory the process page being accessed. At this point. the process may continue. Some systems use overJay amIS to swap process Portions between memory and disk: However, the basic concepts of page faulting and overlays are similar, except tbat overlays are of variable size and usually larger and may be controlled by the process itself. Though vhtual memory significantly expands the capabilities ofa computer, it must be empbasized mat page faultiDg is an extremely hazardous event with JeSpect to system perIomumce. It can W1uaJly SIall a processor if allowed to get out ofhand. Wben a page fault occurs, DOt much else can go on in the sys1em. Certainly, anotberpage faultc:aunot be honored muil the.first bas been processed. Otherwise, one could fiDd so many page faults in propss mat wbea. one page fault completed, 1hat page would aiready bave been reseized by a later zequest (an ·exueme but possible samario). 'Iherefore, dmiDg page fan1ring,1be processor bec:omes vecy limited in what it does 8Ild may sIall, dms drastically reducing its tbroughpat. . To alleviate this, one may be able to ""fix" part or all of the code or data of critical processes iDto pb.ysical memory. These pages ·are never subject to paging. T1~y will always be in memory wIleD needed but willleduce 1be amount of IIleDlO!Y available for
paging. One of the obvious cbaractaistic:s of virtual memory as descr:ibed above is that one never knows what physical page in memory CUIreDtly contains a particular logical page of a process. In fact, a logical process page migrates randomly through physical memory.as page fanlting occurs. SomIAimes it is 1bere, and sometimes it isn't. And wheoever it ~, it pops up somewhere else.
-
Transaction-Processing Systems
Chap. 2
Keeping track of wbich logical page is currently in wbich physical page is called memory mapping. Like page faulting, it is largely a hardware-supported function since it must be performed on-the-fty with instruction execution. To accomplish this, the processor typically. has a set of hardware memory maps. One may be used, for instance, to map the code area and one to map the data. area. These maps can be loaded by software and then used by the hardware as a process executes. The contents of a memOIy map are illustrated in Figure 2-13. LOGICAL PROCESS PHYSICAL MEMORY PAGE ADDRESS PAGE ADDRESS
LOGICAL PROCESS PHYSICAL MEMORY PAGE ADDRESS PAGE ADDRESS
0
113
0
16
I
I
2
27 -(97)
2
17 79
3 4
72
4
13
• • •
• • •
• • •
-
• •
• CODE MAP
-
3
DATA MAP
i Likewise, each process has a code map and a data map associated with it. When it is rmming, its maps are loaded into the processor's hardware maps. Let us say that the example of Figure 2-13 shows the maps for the .cmrently nmuing process and d1at the process is cmrently executing code in its fiISt logical page (page 0). 'Ibis code is found in physical page 113 by the processor, and the processor is thus executing code located in physical page 113. If the process's code execution falls through to page 1 or lmmches to page 4, execution proceeds llOI!D8lly since these pages are in physical meJDOIY. However, jfit should attempt to execute an insttuc:tion in its lOgical page 2, the processor will not find that page to be CUDeDdy in physical meDlOIY. It.will page fault, find an old page to overwrite (say physical page 97) and will lead the cunent process's logical page 2 into physical page !17. Arr.y maps !efaenciag the old c:onteDts of ph,sicalpage !17 must bemodiied to showtbat logical page to be DO longer in physical memory. The process may DOW pnx:eed to execute code in its logical page 2 (physical page 97). A similar sceaario holds for dara-page accesses. In the mnnp1e ofFIgUre 2-13, data. accesses to logical pages 0, 1, 2, or 4 will be executed normally. A data access to page 3 will cause a page fault. Note that under memory mapping. the code or data area of a process is not necessarily (and probably is not) mapped into contiguous physical memory. 1bough memory space appears logically to be contiguous to the user, it is in faa: spread quite mndomly throughout physical memory. We bave c1escribed separate memory maps. for code and data. areas, for these areas .. ~ totally exclusive and may each DDge over the entire add!ess space.
Chap. 2
Process Management
43
.anaging .ultiple Users One additional fallout from these types of operating systemS should be noted. As we have said, a process is a program I'IJlUUng in a computer and comprises a code area and a data area. The code area is never modified. Tbe!e is no :reason that in that same computer, another process could not be created. using the same program but simply carrying another name. In fact, since its code area would be identical to that of the first process, the code area would not have to be duplicated; both processes would map into the same code area in physical IDeIDOJ:Y, ~ shown in F~ 2-~4.,
'~A.
TERMINAL I
PROGRAM INO
TERMIIIAL
2
TERMINAL. N
Fipre 2-14 MIIIIiuser opeadaL
Let us incmduce one IDCR concept, 1hat of the home termintzl. When a process is aeated by an operator. it is Cleated by that operator typing in a rormmmd at a terminal. The process tbeD lcnowS that tamjnaJ as i1s home terminal aDd may Jeter to it as such. Consider a progtam written to pezform a task (maybe an inquiry task) to support ODe and ODly ODe tfmrinaJ, which is i1s home tamjnaJ. The program file is ,stcnd as an object file on disk and is IIiImed INQ. " ODe operator could walk up to tmninaJ 1 (in FIgIR 2-14) and type RUN lNQINAME INQAI and tile Process INQA would be Cleated.' A copy of tile code ccmespondiDg to program lNQ would be made available to p.roc:ess INQA, and a data area image for process INQA would be created on disk. The operator can now use process INQA for inquiry pirposes. As die process executes. pages requD:ed by it Will be paged in aDd mapped as pmiously described.
Transaction-Processing Systems
44
Chap. 2
A second operator can walk up to terminal 2 and enter a similar command: RUN lNQINAME lNQBI and process INQB will be created as was lNQA, except that a second copy of lNQ's code does not need to be paged into physical memory if lNQB is running in the same computer as lNQA (that is, the terminals are on the same computer, or the CPU is specified in the RUN command). Process lNQB will use the same code area as lNQA. Of course, a separate data area is aeated for process lNQB. This can continue vinDally without limit, allowing more and more simultaneous users of the same single-user program by creaIing more and more uniquely named pr0cesses using the same program. All such processes lUDDing in a common computer will use a common memory-residen.t copy of the code, but each will have its own data area. In fact, the sum total of memory used by many sing1e-user processes is about the same as the JDeIDOr)r used by one lirge rimltiUser process since the separate data areas are· required anyway. The duplication of common data areas and the additional multiprocess oved1ead (e.g., PCBs) are periJaps offset by the simpler code requiIed.
Additional ConsideratioIts The creation of a laIge number ofprocesses does not come without problems, and there are some instances in wbich it is desiIable to design a process to handle a number of users. Among these considerations are the following: -
• Each process requires a PCB and other system stmctu.res, such as file control blocks, that ue aIlocared from vecy valuable common buffer space in the system data area. 'Ibis imposes a pacticallimit cmthe number of prOcesses that can exist in one computer at any one time. Typical CODtempOImy systemS can support from 100 to a few bundred COJlCUDeIlt processes. • If large buffers ue RqUiIed for each user, II users will mquire II buffers if each process is siDglo-user. This could result in excessive page faulting. A multiuser process could get by with a smaller buffer pool, wbich it c:J.ynamically allocates to users as needed, so page faultiag would be redaced or etinrinated. • IDterproc:ess messages ue time·cons"""ng in multicomputer sysIeJiIs. Typical interpIoc:ess message times mage from 1 JDSeC. to SO JDSeC. In some applications. it is possible to batch traDsactiODs iD one process before sencfing.tbem. to 8DOCber proceSs, thus saviDgmany intetpIocess messages. In this case, it may be desirable to have a· piQcess ccmtto1 many users so it can aa:amulaie a batch in a Ielatively short period of time. Orherwise, data-base updating and othetfuneticms dependent on these traDsactiODs might drag. • The act ofprocessfi'itcbing addsopaating system ovedlead. Typical overl1eads run from·O.s DJSeC. to 10 DJSeC. When processes are switched, the old active piOc:ess must be switched to waiting and pedJaps added to the timer list, the new active process must be removed from the =rdy list, and maps must be ~hed.
Chap. 2
Survivability
45
A single-user process must be switched after every transaction. A multiuser pr0cess must be switched only when it has processed all transactions pending from all its users.
Su...",.ry This description of processes and process management highlights another important feature of contemporary distributed systems: simple software development. Let us look at what this type of operating system allows us to do. A programmer can write a program designed to support a single user and, with virtually DO additional effort, find that the program has the ,following attributes: • It is a multiuser program, one that can be used by many users simultaneously. • It is a program that will run in a multitasking enviromnent (i.e., many other tasks may be numing CODCU1IeDtly). • It is a program that will run on a computer regardless of the amount of memory it requires or the amount of memory available to that computer. • It is a program that will run in a distributed enviromnent in which the programmer can be unaware of the particular computer that will be used or to which computers the programmer's various periphe:ral devices are connected. Thus, the problem of writing multiuser, multitasJdng. distributed programs has been reduced to the writing of a single-thread, single-user progEaDl-the simplest possible solution. UDfortuDately, this has not come without peaalty. The penalty is in system capacity because of the high overhead of this do-all operating system. But tbis is the trade-off we will be willing to make as hardware costs go down and as software costs go up-more ,hardware for less software. The critical subject of system capacity and responsiveness thus becomes even more important and creates a stronger focus 'on the need for performance analysis.
SURVNABIUTY The simple expedieat of formally defining the process as the basic logicalUDit in a system, bouudiDg its capabDities, and then building an operatiDgsystem that effectively mauages such processes leads us to a powerful progtli iliiring envD:om:oeDt. We can now write multiuser applicatioDs that run in a multitaslc:ing, distributed eavironment wbile concerning omselves 0Dly with the problems of a single-user, smgle-tbread applicatioD. The distributed aspect allows us a further exteDSion of thesecapabiliti.es. Since the system DOW has at least two p.roc:esso1'S, and since we bave the optiOn of ,adding two of anything else that might be critical, we bave tile OPPOdDDity to make the system bigbly faD1t.,toleJ:ant. We can C!eate a system that will survive any single failure (and many cases
Transaction-Processing Systems
46
Chap. 2
of multiple failures) in that it will continue to perform functionally in the same way in the face of these failures. The user may notice a loss of capacity or responsiveness but will oat lose any of the system's capabilities. We will see that the structure and management of processes. playa big role in achieving this goal.
Hardware Duality The first step in achieving survivability is hardware duality. If a critical hardware component fails, there must be at least one other identical component that can be used immediately. Equally important are the paths to all c:omponents. If one path fails, there must be an altemate path to that component. For instance, if the processor to which a User's tenDinal is connected fails, and there is DO means to connect the terminal to another processor, tben so far as the user is eoncemed, the system has failed. This leads to the concept of dual-ported devices, in which each device controller has two ports, each ofwhieh can be connected to a separate eompnter. At any one time, one of the ports is being actively used, and the other is dormant, playing a backup role. The computer eonneeted to the active port is said to "own" the device. As an example, Figme 2-15 shows a dual-ported printer. The printer (a normal, everyday, sing1e-ported printer) is connected to a controller that can be driven either by processor A or processor B via two independent ports. Each of these processors runs an 110 process, which controls the printer via its CODDected port (remember that an 110 process must reside in the same computer to which the device is connected). Processor A cur.rently owns the printer, and the operating system knows this. 'Ibelefore, intetpioeess messages containiog data to be printed on the printer are routed to the printer 110 process in
processor A. . Theze are seveml failmes that could cause this path to the printer to fail. Specifically, processor A could fail, or logic in the printer controller port conneered to processor .A could fail. In the foJmer case, the operatiag system would zealize the failure of pro-
.
"".-------
Chap. 2
Survivability
47
cessor A and would transfer ownership of the printer to processor B. In the latter case, the printer 110 process would detect the fault in the controller and would transfer ownership to processor B. In either event, subsequent interproc:ess printer messages are sent to the printer 110 process in processor B. Therefore, the failure of a processor or a device port is indeed ttansparent to the user insofar as the user's access to that device is concerned (providing the application process is written to reissue an 110 message in the event of an em>r). UnfOItUDately, in all devices there are simplex points of failure that will totally remove that device from service. For instance, the failure of a printer motor or even common logic, such as a line driver in the printer controller, will disable the printer. This can be overcome only by totally duplicating the device and its controller and by making provision in the system for rerouting work away from the failed device to an altemate device. In the case of a printer, for instance, a sophisticated spooler queues work for all printers on the system. If a printer fails, it simply becomes UDavailable to the spooler, which continues to despool all work to the remaining printers. In many cases, the duplication of a device is not economically justifiable; and work for a failed device is simply held until the device once again becomes available. There are two cases in which the continued operation of the device is every bit as imponant as the processors themselves. One case involves the inteIproc:essor bus; without it, all paths but local paths within a processor are lost. Therefore, this bus most be duplexed. The other case is that of disk files containing critical data bases and system files (program and process images).
Data Base Integrity If a disk containing a critical file goes down, and there is DO alternate, the system goes down. Totally. Furthermore, just having a backup disk is DOt satisfactOIy. It most COD~ completely updated files, i.e., it most be a mirror image of its partner. As data comes in that updates one disk of a "miDomd pair," it must also update the other disk. FIgUre 2-16 shows the c:onfigmation for a miIIored disk pair. Three levels of mirroring may be used:
• One c:onttoller and two disks (Figure 2-168). However, if CODtroller logic c0mmon to both disks fails, then access to both disks is lost. • One ~ per disk (Figure 2-16b). No single failure will prevent ac:c:ess to the data. • Dual-ported disk devices COIl1leCted to dual-ported controllers (Figule 2-16c). This adds an additional level of redundancy to the IIIiIIoml pair. There is an important utility that most be available to support min:ored files if they are to be truly effective: an on-line disk copy utility to be used when a disk that is part of a mirrored pair is to be Ietumed to service. When a disk unit fails, the files are handled by the mnajning simplex disk. When the disk is repaired or replaced with a spare and is to be
Transaction-Processing Systems
48
SINGLE CONTROLLER (Q>
DUAL CONTROLLERS (b)
DUAL-PORTED DISKS (c)
Chap. 2
Chap. 2
Survivability
49
put back in the system, it must be brought back to its minored condition (i.e., containing an exact copy of the other disk) even while further modifications are received for that data. That is the job of the on-line disk copy utility: to copy one disk to another while at the same time ensuring that file updates are kept current. To summarize hardware duality, Figure 2-5, which was previously discussed, shows a simple system with communication lines, printer, and mirrored disk pair. That figure shows bow these peripherals might be configured physically. Figure 2-17 shows the logicalllO process configuration as it would interact with an application process. Primary paths are indicated as solid lines, backup paths as dashed. One final point should be made about hardware duality in a survivable system. Duality is fruitless if a failed device cannot be Iepaired and IetUmed to service while the system is running (this led to the need of an on-line disk copy utility for mirrored files). Therefore, any piece ofbardware-including processors, buses, device controllers, power supplies, even fans-must be capable of being mnoved, repaiIed, or replaced and plugged baclc in while power is still on the system and without inducing' 'glitches" in the system operation.
Software Redundancy There are several approaches to software redundancy in contemporary distributed systems. These approaches can be classified acconting to the following geaeral methods, wbich are discussed in the following paragraphs.
~
¢
I I . I I
Transaction-Processing Systems
50 • • • •
Chap. 2
Transaction protection Synchronization Message queuing CheckpointiDg
Software redundancy pIeSeDtS different problems. depending upon whether a multiprocessor or multicomputer (including bybrid) arcbitecture is used.
Transaction Protef:tion Using trausaction protection, no attempt is made to keep a backup process updated. Rather, each transaction is logged to disk in a set of audit files in such a way that transactions that were in progress at the time of failure can be backed out of the system. The user is then requested to teenter that ttansac:tion. In this case, the failure is not transparent to the user, though its impact is mjnimal and usually acceptable. However, recovery may take sevezal minutes as the audit files are played back and the data files couected. Transaction protection is used ~vely by multiprocessor systems; as it must be assumed that a fault bas cootaTDimrted memory and that no other recovery mechanism is available. It is also used in certain malticomputer system offerings and in maay single computer systems for which it is the only means of IeCOvery. Note that ~ processes are wmecessary if ttaasaction protection is used as the fault-recovery strategy since they need DOt be kept up-to-date anyway. Should a fault occur in a processor, the system will IeCOnSttuCt its data base via the audit files and then IeCreate the failed processes in a surviviDgpocessor before restarting the system. In fact, in a uue load-sharing multiprocessor system, the concept of a backup process is meaningless, since the process is IepI'esented by control strucbJ:Ies (the Process Conttol Block, for instance), a code area, and a data Ilea in common memory. It survives even if a processor fails and needs to be JeCJeated 0Dly if a memory partition failure destroys some control infoJ:mation peItiDeDt to that process. The traDSaCtion-p:otection procedure developed by the former Syuapse CorporaDon of Milpitas, Califomia, for their multiprocessor system is a very good example of Ibis tecImique. BasicaDy, an chaDges to files are logged in separate audit files as traDsaCtions me processed. In the event of.failme, the system is paused, and an files me ietumed to a peviouslyestablished "CODSisUmcy point, ,. which eusm:es the iutegtity of the data base. 1iaDsactions that have been completed followiDg the CODSisteDcy point me "'J:olled forwani" from the log, or IeCOIDp1eted.. TI'8IISaCtiODS that have DOt been completed are "'J:olled baclcwani," or deleted. The system is then xestartecl, and uncompleted transactions must be n:enteled. Figme 2-18 shows the anctiting activities that Syuapse used in order to tecODSIrUCt its data base foUowiDg a sYstem fault. All cbaaged data me written to two logs: the history log and the tempotalY log. The histoty log contains an before. and after IeCOId images of o.cbaDged data. These IeCOIds must be pbysically written to the history log before a ttaDS-
Chap. 2
Survivability
51
r
\.
APPLICATION
PROCESS
DATA BASE
)
f HIS10RY \. LOG
BEFORE/AFTER RECORD IMAGES
TEMPORARY \. LOG
BEFORE IMAGES SINCE LAST CP
LOGGING - HISTORY LOG BEFORE/AFTER RECORD IMAGES PHYSICAL DISC WRITE GUARANTEED AT COMMIT TIME BEFORE PROCESS CAN PROCEED
- TEMPORARY LOG BEFORE IMAIES OF CHANGED DATA NO DATA CAN BE PHYSICALLY WRITTEN UNTIL TEMPORARY LOG IS WRITTEN
- CONSISTENCY POINT (CP) EVERY FEW (2) MINUTES-USER SPECIFIED HISTORY L.06. WRmEN 10 DISC TEMPORARY LOG DATA DELETED ALL CHANGED DATA WRITTEN 10 DISC
1....... (. ... • d") -n.-.c: . . to be ~ I.e., lS CQlh""1ttK • "'JIW,&~OIe, at traDsaCtlOIl oommit time, the user is brie1ly paused UDtil an of the changecl data is written to the bistoly
.. . . .
8CUon IS
:~~
c:oDSaaw.&...u
log. The tempOraIy log CODfa;ns 0Dly before images of changed datL No data can be written to a data-base file UDtil the before image bas been physic:ally written to the remporaIY log. Periodically (typicaUy every·two minutes), the entire system pauses to establish a CODSisteDcy point (CP). At this point, all pending data are written to the history log. the contents of the tempo1'8Iy log are deleted, and aD .changed data are written to the· appropriate.data-base files. At the CODSistency point, the entire set of application processes are
Transaction-Processing Systems
52
Chap. 2
correctly represented on disk. All that is required in the event of a failure is to restore the system to this consistency point, redo transactions since completed, and restart the users. This recovery process is done via the history and tempormy logs. Since the temporary log was cleared at CP time and contains only the before images of any data-base changes since that time, the data base is rolled back to its CP state via the temporary log. The history log is then used to replay transaction activity, as shown in Figure 2-19. There are seven possible tranSaction scenarios, as shoWD. Transactions that started before the coDSistency point but never completed before the time of failUre or that were aborted (rolled back) per an operator request follOwing the consistency point are rolled back (types 1 and 2). A transaction that started before the consistency point and successfully completed before the failure is reconstructed by rolling forward from the history log (type 3). Transactions that started after the consistency point and either didn't complete (type 4) or were rolled back at the request of the user (type 5) are ignored. Transactions that staned after the consistency point and that completed prior to the failure are reconstructed by rolling forward from the history log (type 6). No action is required :for any transaction that completed before the consistency point (type 7). RECOVERY - REESt'ABUSH SYSTEM AT CP VIA TEMPORARY LOG
- RECONSTRUCT' SUBSEQUENT ACTMTY VIA HIS1ORY' LOG INCUJDING 1"" Ua<S - ROI LBACK INCCMPI.ETED TRANSACTDG VIA HIS'TORY LOG
-RESULT PC!
LaACK TYPES 1.2
RCIU.FORWARD TYPES 3,S
acoRE TYPES 4,5,7
TYPE
2 3 4 5 S 7
S S S
-
..-
,. i
I I
i
I I I I
S-RJC
FAUpE
CP
I I I
S S S
I
R
C
-
R
C
s- SfART R. OPERATOR ABORT 'ItOU8ACK) c· COMPLETED
TRANSACTION
Figure 2-D Recowr:y
Chap. 2
Survivability
53
This procedure recovers, via software, activity on a transaction basis. If a failure should occur, the user may see a delay measured as a few minutes and may be asked to reenter information that already bad been entered. Beyond this, the procedure ensures full data integrity of the data base and continued operation of the system.
Synchronization In a synchronizing system, every process is replicated in a different processor. All pr0cesses execute, i.e., they all process the transaction and periodically compare results. This synchronizing point is usually an operating system function and may typically occur at 110 request points or completion points. FJgUIe 2-20a shows a dually-redundant synchronizing system. When one process is ready to sync:bronize, it will wait for the other process to catch up and itself be ready for syncbronization. Certain interprocess checks are made, and if everytbing compmes, the processes proceed. If there is an euor, diagnostic procedures are invoked to determine the faulty process, after wbich the surviving process continues with the transaction (after, pe!baps, creating another backup process to maintain survivability). One way to provide diagnostic checking to determine the identity of the failed processor is to have more than two processors (say, tine or four) involved in the transaction. If there is then a failure, a simple vote of all processors will determine the culprit (assuming only single-point failures). Such voting systems (Figme 2-2Ob) are used for ultrahigh-reliability IeqUhements such as the space program. August Systems (Salem, 0Ieg0n) offers a triplexed voting system for process control applications (such as nuclear power plants), in which the concept of voting is even applied to the digital and analog inputs and outputs of the system (for aualog sigDals, the median sigaal is taken). An impol1ant example of a synchronizing system is the product iDtroduced by Stratus (Marlboro, Massachusetts). This system is a quadraplexed votiDg system. in which SynchroDizing is done by hardware at the system clOCk fIequency (typically seYeral tilDeS per miclOsecond). The four processors are grouped as pairs of dual processors (Figme 2-2Oc:), with all four executiDg in "lock step, .. i.e., all doing exactly the same operaIioD at each system clock time. If one processor fails, it will DOt agme with its comPanion, and both are taken out of service. The other pair contiDues 1ID1lffected. Tbis is the 0Dly medlod to dale that requires no softwam support whaIsoever (except, of course, to diagnose the problem). . A modified approach tbat is somewhat similar to that· developed by St:ra!us is·taken by A1T in its 3B20Dproduct (FIgme 2-2Od). 1bis is a dual compurer sysImD in which ODe operates while the other provides a hot SIandby. As data areas in memory are modified in die primaIy system, a bardwme tiDk updates those areas in the memory of the standby system. Should the primary fail, the SIandby uses the process control blocks (wbich, of course, are part of the memory-resident data that was kept updated) to load the active processes and to continue execution. One might also consider this system. as a class of checkpointiDg systems, described later. Note that all synchronizing systemS requiJ:e at least two processors to do the job of one..· However, leCC)Vety is vUmaIly insamtaneous.
Tran~action-Processing
54
Systems
REQUEST
Process
COMPARE
REPLY
REQUEST
REPlY
SYSTEM BUS
Proces.o.~
LOCK STEP (c)
~.-----....,
---_
MEMORY
00 I
10
Executing
Processes ME~ UPDATE
d)
. Fipre 2-21 SyucbroaiziDg systems. :.~
.
Chap. 2
Chap. 2
Survivability
55
Backup Processes
Transaction recovery, as described above, IeqWres no backup process. If there is a failure, failed processes are recreated in a surviving computer, incomplete transactions are backed out, and the user IeeDters the last request if it bad not been complet¢. This procedure could take several minutes. SyncbroDizing systems require two or mOle fully active processors executing in parallel but give nearly instantaneous recovery. 'Ibus, these two approaches represent the extremes of the cost-performance tradeoff: minimal haniware and long recovery times (transaction recovery) versus maximum hardware and almost instantaneous recovery times (syncbronization). The software redundancy teclmiques to be discussed next-message queuing and checkpointing-are aimed at getting near-instantaneous (i.e., seconds) recovery while using little additional processing power to keep the backup process (which is dormant) updated. BefOle describing these methods, we must first discuss the mecbanism for creadng and managing the backup process. Just as each bardware unit must bave a backup. so must each software unit, or process. Should the computer in wbich a process is runoing fail, then that process will cease to exist (and the capabilities it provides will be lost to the user) unless a spare process can be "switched in... This requUes two capabilities of the process: • It must be able to aeare a backup copy of itself in another computer whenever it is aeated or bas taken over from a failed process . • It must be able to keep its backup.infomled of what it is cummtly doing (for instaDc:e. what transaction it is cuaeatly working on) so that the backup can continue its work UDinrmupted should the primary process fail.
Let us first CODsider the aeaDon and maaagement of the backup process. Assume that process A bas been created. One of.the first things process A does is to Iequest the operating sysaem to CIeate a backup copy of itself in another computer. We will call this backup process A'. It bas the same DaIIIe as process. A but is aeated by and runs in a diffeteat computer. A' detects that it is the backup (because it can sease that its c0mpanion al1eady exisIs) and iyrmwtiately calls a monitor procedme that is !eSpODSible for IDODitoring the primmy process and taking ovec in the event of primary failme. The monitor procedme is provided by the operatiDg system. Just as the opmatiDg system. must know of process A so !bat it can route inte.rpJ:oc:ess messages to it, so must ~ know about process A'. As we disc:ussed earlier. it knows about process A and all other processes in the system via the process dUectOIy. a copy of which .is maintained in each physical processor. Let US DOW extend the concept of the process diRctory to what we will caD. the process pair directDry (PPD). The PPD contains the name of each process. the computer in which the primary is numiDg. and the computer in wbich the backup, if any. is running. Figuxe 2-21 shows a part of a typical·PPp.
Transaction-Processing Systems
56 PROCESS NAME
PRIMARY PROCESSOR
INQUIRE
Chap. 2
BACKUP
PROCESSOR 2
MAl NT
2
REPORTA
4
• • •
rJglll'e 2-21 ""
Process pair directory.
i
"
"!
As process A is perfomliDg its duties, the operating system routes all intexprocess messages destined for process A to it. However, should process A fail (most likely because of a processor failure but possibly because of a software fault tbat causes the
operating system to abort process A), the operating system will look in the PPD and find tbat process A, in fact, bas a backup. It will send process A' what appears to be an inteIprocess message indicating that process A bas failed and the reason for its failure (processor failure, abort, orwhatever). This causes dleoperating system in the process A' computer to schedule process A' (i.e., put it on the ready list). Further iDteIpIocess messages for process A are DOW routed to process A' for processing, and the system survives. At this point, process A' may create its own backup to protect itselffrom further failure. Fxgme 2-22 shows a typical life of a process in the presence of a computer failure. A dm:e-computer system is shown in which process A is Cleated in computer 1. It creates its backup, A' , in computer 2. Later, computer 1 fails; and process A takes over, creating its backup, A" , in CODOq1Uter 3. Subsequently, computer 1 is repaDed. The system could be left as is. However, in this case, it is desiIed to Jeestablish load baJancing. 1'herefole. prOcess A' stops its backup, A", and reaeates a backup A in computer 1. It tbeD switches COJdml to process A, resetting the system to its initial CODfigmatiOD. As can be seen, a variety of strategies can be emPJoyed to eDsate system survival in a degmdiag system. Load sbariDg should be an· impor13Dt" CODSi
Message Queuing
Having a backup process to switch to in case the primary process fails is essential to ,surviving. However, iftbe system. is to perform fuctionaUy in the same way it would have
Chap. 2
Survivability
Q)
A IS CREATED
b) A CREATES BACKUP
~
COMPUTER I FAILS BECOMES PRIMARY
c)
a
d)
Ii CREATES
BACKUP
If'
I
f)
Ii
,II
STOPS ITS BACKUP
I 9) :
I
CREATES A BACKUP
',~I
'-,
021
PPO
~311~1~IBI
1..-1
I
I
II ~ II t:;\ 2
~D
311
IB,I
PROCE A SSl2P ..._~.. • • • PPD
h) d SWITCHES CONTROL
TOA
bad DO failme occaneci, the backup process must take over whe!e the primary left off!" This means it must know wb3t the primary is doing. . One way to accomplish tbis is to queue all intecpIocess messages received by the primary process to its cformant backup. so that the backup proci:ss may ""catch up" if it bas to take over (see Figure 2-23). Message queuing was fIrstinttoduced in a distributed
sysremby Aungen (Fort Lee, New Jemey). The basic concept as impleiDeDl:ed by Auragen is for a process to send each interprocess message to duee destinatious: 1. The primary process that is the intended destiDatiOD. 2. The baclcup process for the iDteDded destination process •. 3. The baclcup process for the sending process • .Separate actioas are taken by each of these receMng processes'
Transaction-Processing Systems
58
Chap. 2
BUS Intel PI ocess Message - IPM
FipI'e ~23 Message queuing.
• The primary destinari9n process processes the·message as is appxopriate, with DO CODSic1eration given to the other JeCeiviDg proc:esses. • The backup process for the iDIr.:Dded des1iDation process does nothing. It simply keeps the message in its input message queue. • The backup of the sending process notes that a message has been sent by incrementing a counter and then discards that message. Thus, each backup process has a queue of an messages that have been JeCeived by its active COUDte1p8rt and knows how many messages its active half has sent. Should a failure terminate an active process, its backup will take over and begin doing what is DatDral: process its message queue. Thus, it will redo an processing that the foaner active process bad done, zesulring in a data space ideaticaI to that of the pxevious active process at the time of its failuIe. Futtbetmore, by using the count of messages ttaDsmitted by its other half, it will pmveDt the sending of duplicate messages. such as disk updates, tamjnaJ displays, etc. ~ of comse, implies that the ~atiDg sysrem guarantees that the order of messages in the backup's JeCeive queue is precisely identical to the order of message execation by the prlmaEy process.) One ftaw in this strategy as preseDted so far is that the badaIp process's iDput queue will beccIDe adXtladly lcmg wDh time, thus CCMs.."ring an ammatD.y large amount of stmage space and requidDg aD arbittadly loDg recovery 1ime. This problem is solved by perlodically usyncbroDiziDg"1be backup process widl its pimary. Syncb[oaiziag in ~ sease is acc:omplisbed simply byfcxciDg an clinydala l*PS of·the prlma!y process to disk; i.e., iDvokiDg 1hepage fault mecJumism of the pocesSor on bebaJf Of this proc;ess to wrire an IDoctified data pages in memoIY to disk.. . .' . At this ~ if tbebackap took over, it would page-in up-to-date memory. Thus, its receive queue can be. ~, aad its coum of messages· sent cali be cleared to zero. SyaclumizariOll poiIII:s lie geDDlly basecl on queue leDgth 01' time. Th.eytypic:ally OCCUr every mimIre or so with active processes, aad !eCOvery should take a few (5 to 10) seconds. The time between syncbroDiziDg poiD1s is clearly a comp:omise. By making the time shorter~ the page faulting load on the system iDcteases. By maldng the 1ime loDger, ~ recovery time iDaeases.
Chap. 2
Survivability
59
Though message queuing systems requD:e software support to provide recovery from failure, this support is appJications-iDdependeDt and can be made a function of the operiting system. Thus, as with a lock-step, synchroDizecl system, the application program-" mer can be totally 1IDCOIlCelDed with the problems of survivability.
Checkpointing CheckpoiDting in the sease used here was first iDtroducecl by Tandem Computers (Cupertino, Califomia) for use in its Non-Stop 'nI series of fault-tolerant systems. 'Ibis type of checkpointiDg takes advantage of the fact that it only is necessary for the data area of the backup process to be idemical with that of the primary process at certain critical points in the process's execution. Going back to om' discussion of process SIn1CtU.re, checkpninring is done quite simply as the IeSUlt of one of the ~ of the structure of a process. A process c:onvrins a code area and a data IleL The data area comprises global data and a SlaCk that is used to nest pIOCedmes. to pass parameters between proc:eclmes, and to hold temporaIy data needed locaDy by a proc:edwe. The stare of a process is cJetermmed by the stare of its data area, i.e., give two like processes (two pIocesses with the same code area) the same data area and enviroameDt. aDd they will perform ideDtical functioDs. Therefore, if we could somehow m:rinaajn 1he data area of1he backup process so !bat it was ideDtical to the data area oftbe primatyprocess, it would behave exactly 1he same as if it were the primary process at the time of a prlmaty process failure. UDfortuDately. 1he system load that would be iwposed by CODSIantly updatiDg the backup's data area precludes such an app!08Ch. However, it only is necessary that the data areas be ideDtieal at ce:nain criDcal points in the process's execution. For iDstance. if the data axeas were ,made idemicaJ. jmmediately following the receipt of a traDsacQon to pr0cesS, then if the primary failed afIa' panially processiDg tbe ttaDsaCtion, the bac1cDp would start at tbe point at which tbe 1raII-=ticm was teCeivecl and would mproc:ess it. In many applicatioDs. 1bis would be acc:eptab1e. The backup's data area is updahri via a mechanism called checkpoinzing. 'Ibis is simply an iDtecpocess message seat by tbe pimaLy process to its backDp, tbe CODteiDts of which me tbe cam:at COD1mtS of tbe data area. Like other iDtaplocess messages. tbe !eceipt of1ilis ~ ~age by the backDpprocess causes it to be scheduled to IUD. As desaibed·~, tbe backap process aeca1eS the JDODitol procedwe. This proc:edme will mcave the checkpoiDt message aacl will stale it in its data mea, thus updating the data area as desired. Updadag.1be data _ includes updaring !be stack.. '. . Should abe primary ~ sDbseqae:ady fail, the zeceipit of a ~ messagedesci.rib:iug that fai11R (as c1escribed pMously) wiD cause !be moDitor pmc:edwe to IebIm to 1he main low of tbe Process at.1he last received cbeckpojDt. That is to say, the backup process takes over aDd starts exec :ijI he accouting to tbe last c:beckpoiDt. But how does the JDODiror know specifically which iDstruc:tion it is to start execut~? The program COUDter is not part oftbe data _ and is 1hel'efore DOt sent over as ~
Transaction-Processing Systems
60
Chap. 2
of the checkpoint message. This is bandJed via the last stack malicer iii the data area's stack. Each stack marker indicates the place at which the c:mrent procedm:e was called and Causes the cummt procedure to return to the iDstruction following its call when it bas completed. Upon the procedm:e's retum, the stack marker also contains the processor enviroDment to IeStore. When a primary process wants to send a checlcpomt message, it does so by c:aDiDg an operating system procedure that takes care of actually issuing the iDterpIocess message. Thetefore, the last stack marker on the checkpoiDted stack.was placed there as a result of the call to the checkpoint procedm:e by the primary process; it points to the iDsttuction in the application progmmfollowiDg the checkpoint call. When the JDODitorprocedm:e wants to tum em the baclalp process. it simply executes a procedm:e IetUm according to the last stack marlcer, i.e., the backup process will tum on at the iDstmction following the checkpoint call as if it weze the primmy pnx:ess exiting the checkpoint procedm:e. This is illustrated in Figure 2-24. The checkpoint proc:edure also zetDrDS with a status CODditiOD. This status condition normally indicates success when the primary process is rmmiDg. However, if the backup process is mmed on. the D1OJlitor proc:edure foIces an error staDls that indicates that the primary failed and why. This allows tbe backup process to perform special takeover logic, should any be ~ In actual padice, it is usually .1JDDeN$saty to checkpoint the entile data area. The :global data often mntaiDs a laIge data base, whereas the stack is typically small. S~ PRIMARY PROCESS
l
r- RECEIVE TRANSACTION
:
J
BACKUP PROCESS
I
RECEIVE
rONI
+ t Trr---+-~rr
I I I
EDIT
TRANsAcnoN
t---AULURE
: POST TO lRANSACTIClN FILE
RECEIPT OF
EaT TRANSACTION
~..
l
PCST 1\) TRANSACTION FILE
I I I I
RETURN RESPONSE
: . ! IWAIT
FOR NEXT TRANSACTION
L _ _ _ .J
RETURN RESPONSE
~
WAIT FeR NEXT TRANSACTION
L _ _ _ --I
Chap. 2
Survivability
61
large (multi-Kword) cbeckpoint messages would Iep1esent a large bus and processor load, it is advantageous to send over just that part oftbe data mea that bas changed since the last checkpoint. This often is simply c:enain elements of the global data plus the stack. In some cases, it may not even be necessary to checlcpoint the stacie. If cenam messages simply update intema1 parameterS, it ODly is necessary to process 'the message, update those parameten, and then checlcpoint tbe changed data. In this case, the stack is not c:heckpointed, so backup processing will resume at the last point at which the stack was checkpoiDted rather than at noncritical c:heckpoints. The basic concepts of checkpojntiDg are quite sttaightforward CIeate backup pr0cesses and keep them iDfo.tmed at critic:al processing points via chec:kpoiDt messages. CheclcpoimiD.g does repRSeIlt a sigDificant system load and should DOt be used casually. Each checlcpoint is, in effect, an iDtapIocess message reqoirlDg several miUisecxmds, as discussed earlier. CheckpoiDtiDg is an impotIant cxmsideration in pelfODDaDce analysis. CbeckpoiDtiDg sa:aregies should be carefully thought out in terms of mjnjmjzing the number of checlcpoint messages and checlcpoint lengths wbile achieving the degIee of fault ttaDSp81eDCY desired in the system. More important, these strategies must be established as part of the design of the process. It is insufficient to implement simplex processes iDitially without giving thought to survivability and then worry later about whe!e to put the c:heckpoints. This can lead to a process organization in wbich the cbeC'kpcriDting task burden is so large it c:aDDOt Iea1istically be cmied. Let us explore various levels of checkpoiDtiDg. As we have said, the level of checkpoiDt:iDg should be CQ!1,naswate wiIh the level of fault ttaDSp81eDCY desimL Consider an iDquiIy application in which the operator enters an iDquily, a file is searched, and data is JetWued to the operator. In many situations, it may be quite xeasoaable to ask the openror to xeeoter the inquiry in the rare oc:caneuce of a system fault that bas iDtea:upted the inquiry process. No c:beckpointing Deed be doae at an. Should the backup process find it has taken over, it migbt simply sead a lepeat RqUeSt to an opaatms. DOt knowiDg ~ ones had active "iDqujries. An even better siluatioD thaD the above is one in which the terminals buffer the inquiry aad pass it to the system. in !espouse to a poll. In tis case, the DeW primary process Deed 0Dly poll an 1mDiDals; dlose wiIh 1IDIIISWered iDquirles will Hmllsmit those inquiries fO!' lep!occssiag. Full fault U3IISp8reDCy bas beea acbieved without checlcpoint~.
.
However, if it is 1IDCIesinIble to teqUeSt again 1be traDsaCtioa. from the operatOr once it has been JeCeived, the process can checlcpoint it as soon as it teceives it and then process the inquiry. In tis case, if the process fails, 1be backup has the traDsactioD and will leproc:ess it without having to zequest it again. The ope!3tOI will IeCeive a mspcmse without ever knowiDg tb.eIe was a fault. In tis case, if a failme 0CC1IlS after the system bad IeSpODded to 1be last inquiry and before it bas obIaiDed the next one,1be IeSpODSe to the last inquiry will be lettansmitted to the operator since the ttaDSaction is being totally leprcx:essed. If this is UDdesirable. a second checlcpoint is RqUirecl following the retum of the iDquiJ:y IeSpODSe. U~y, applicatioDs vsuaUy lIeD't tbis simple. Typically, a traDsaCtion is
62
Transaction-Processing Systems
Chap. 2
used to update a file. The simplest case of this is when the transaction is simply logged to a transaction file for later processing. In this case, all of the aforementioned strategies . hold. If the operator or terminal can be requested to resubmit the transaction, DO checkpointing is required. Otherwise, the transaction should be checkpointed when it is received. " In this example, a failure could cause the transaction to be logged twice. Often, the processing programs can handle the case of duplicate transactions (transactions may cmy serial numbers, for instance). If this is intolerable, then (as above) the process should checkpoint following the logging of the transaction to cause the backup to pick up at this point. Often, however, the transaction is used to perform an on-line update of a file. In this case, a record must be read, modified according to the transaction, and then rewritten. Whether or not the transaction was checlcpointed when it was received, it is imperative that it and the record be c:heclcpointed when the read of the record has been completed. 0therwise, a double update could occur (unless this is allowed). Consider a transaction that contains a count that is to be added to a field in a record. If the transaction is simply reprocessed following a failure, it would be added again to the field if the first transaction had completed. However, by c:heclcpointing the read record and by assuming a failure after the transaction had been completed, the backup process would continue from the point at which the read had completed. It would add the trans-" action count to the origiDal field and would retum the record, overwriting identical data left behind by the primary process. So far as transaction processing is conc:emed, this is the case of most general inta:est and is shown in Figure 2-25. The first cbeckpoint is needed only if the transaction cannot be requested again from the user. The middle cbeckpoint is needed only ifa double update is DOt allowed.Tbis same checkpoint can be used to protect multiple updares, provided all data is read, the c:heckpoint is sent, and then files are updated. The last checkpoint is needed 0Dly if a Iepeat :teSpODSe cannot be tolemted. Most processes. whether they deal with transaction processjng, extema1 event c0ntrol, c:onnmmicatioa switchiDg, or whatever, can be fnmed as subsets of FJgUre 2-25. TheL'efOle, we can see that the usual wont case for a transaction is three checkpoints. k is impoI1aDt to mjninrin, the Il1IDlber of cbeckpoiDts because they. create system ovedlead. Some1imes DO checkpoints are requUed. k is 1iequeDtly possible to design the system so that DO !IlClR than one c:heckpoint per traDsaCtion is 1eQ1liIed. However, CODsideratioDs to allow this often range throughout the system. from operating procedmes (reenteriDg a traDsadioD) to termma1s (block ttansmit. ignoring unexpected respoases) to processing functions (defecting and ignoring du.plicate transactions). 'IbelefoIe, the determination of checkpoint strategies belongs in the very early stages of design and is not a candidate for afterdlought. . The above discussion has concerned itself with a software mechanism for checkpointing. The ATI 3B2OD, discussed earlier and shown in Figure 2-2Od, can be considered a system. in .which checkpoin1ing is accomplished via hardware. As memory in the primary system. is mocfified. a hardware chedcpointing mecba~. Dism updates c:cmespoDding meDlOIY in the standby system. As opposed to·the check-
Chap. 2
PRIMARY PROCESS
~CEIVE TRANSACTION
l l
CHECKPOINT TRANSACTION
I BACKUP PROCESS :
--.l.~? I ~crlON
CHECKPOINT RECORD
I I•
UPDATE RECORD
I
READ DISK RECORD
l
l
l
REWRITE RECORD RETURN R!SPONSE
l
CHECKPOINT STACK
l
63
Software Architecture
STARTING HERE
RE-UPDATES
I ~~DISK
I
I I I I.. r~RHERE I TRANSACTI~
WAIT FOR NEXT TRANSACTION ,
L____ ..1I
•;
I'ipre 2-25 Geaeal uansaction c:hecqK'4nting.
Pointing tedmique described above, however, this hardwme approach requires that ~ memory CODteDts of both the primary and standby systems be identical. Therefore, both systemS must be dedic:ated to the same tasks; in this sease, tbe ATr system is more akin to a synchronizing system. .
SOFIWARE ARCHITECTURE The pedClJh'gc:e of a TP system is 0Dly partly derermiDed by its hardware CODfigmation. The impact of the softwaIe is usuaBy even more cmcial. This impact mages fiom tbe efficieacy with which programs me written to laDguage CODSideratioDs. opeiatiDg system characteristic: and application softwaIe arcbito:uae.
In almost eve:ry TP system, there is a softwme bottleneck somewhere that may limit ultimare system capacity DO mauer how much hardwme is added. The most rommou bottleneck is the data-base mauager. Since it must coordiaar.e the activity of many application processes that wish to access the data base simultaneously, all data-base requests must be funneled tbrough it. This is largely because of tbe problem of simult8neous updates to a recant. Let us say that process ":-- and process B·both want to
64
Transaction-Processing Systems
Chap. 2
read a record and perhaps update it. For both to read the record is DO problem. But DOW let us assume that both modify the record and want to rewrite it. If process A is the first to rewrite its record, then the updated record submitted by process B will overwrite process A's changes, which are DOW lost. By fuJmelling all requests through a common data-base manager, mechanisms can be established to prevent such confticts. Typically, a process that wishes to update a record will request that record (or file, or field within a record, depending upon the system) be locked and thus be made unavailable to any other proces& desiring to modify that record until it bas been updated by the requesting process (it is, however, usually available for reading by other processes). If another process makes a request to lock this record while it is locked by another record, the new request is either rejected or the requesting process is placed. in a queue for that record. In some systems, multiple data-base manager paths are provided to mjnimjze the data-base bottleneck effect. This can be done sometimes by segregating unrelated files or, altematively, by segregating unrelated operations (open/closes versus readIwrites, for instance).
Another bottleneck that can exist is a common log. If, for instance, all system actions must be logged to a common log file or printer, that log becomes a bottleneck. The audit files required for transaction recovel)', described earlier, are a good example of such a potential bottleneck. Requestor-Server
One common software architecture that is found in many TP systems is the requestorserver model, shown in FlgUIe 2-26. In this model architecture, xequestor processes each service one or men user 1r!nnina1s. When a requestor receives a request from a user, it evaluates the request and passes "it to an appropriate server process that is designed to bancIle that request~. Figure 2-26, beJDg vel)' simplistic, "shows two 1;ypes of servers: one for bndling inquiries aDd one for bandHng updates. A server will do wbatever it bas to do to satisfy the request. This usually involves iDtetac1;ing wi1h the data-base III8II8geI' to gain aa:ess to-or update" data in the data base.' It theD formuJara a reply and IetUms it to the user via its nquestor process.
Requestor processes are pezmaueudy crearecl aDd assipM to support a 'fixed set of user termjnaJs. However, server processes me, often dynanricaIly allocated according to the load OIl the system. For instaDce. let us assume that the volume of updates becomes vel)' high in the aftemoon. Seeing tbat queues of waiting update IeqUeStS me begiDning to foun, thuS slowing down IeSpODSe time (i.e., the system is begimring to appear sluggish to the user), the system will spawn additiODal update server processes aDd will spread the waiting transactions over them to achieve a degI:ee of paraJ1el processing. If this is a distributed system. new servers can be crearecl in the least-beavily loaded J:omputeIs. If the system bas multiple disk spindles, system pedonnance might be nearly
Chap. 2
Software Architecture
65
CONTEXT
USERS
Fipre 2-~ Requestor-servu model.
,
;
. I
proportiODal to the number of servers up to the point, at least, that all servers are being kept busy. Of course, the disk system will ultimately be a bottleneck. limiting system pedor-
manc:e. As the load on a particular server class ctiminjshes, servers are killed off 1II1til 0Dly one remains. A common queue may be maint;ained for all servers in a class, with the next server that becomes free getting the next request; or each server may have its own queue, with requests being distribatedto the servers ac:conting to some algorltbm (round robin, or
to the shortest queue, for insIanc:e). Note tbat aD imporIaDt cbaDcreristic of the requestor-server model bas a signjficam design impact. Since there can be multiple servers in a class, there is 110 guarantee tbat a request Will be routed to a particular server. Many ttansadiODS comprise a sequ.eace of IeqUeStlMp1y iDlaactioas with the opentoI, with the next request depending upon information from che previous reply. This inf01mation is 1mIled the conte%t of the zeqaest, i.e., it is the CODteXt in which che Iequest is to be intelpeted. This CODIeXt is ~.1ogically IequUed to be carried mthe IeqUest; in priDeipal, at least, it is known to the system. For instance, let us assume tbat the user has asked to see billing information for a particular customer tbat, as part of that request, bas supplied a customer number. However, tb.eIe is m.ore reply iDfomIaIion thin can fit on the scteea, so a pard81zeply is tetwued. To see the next ••page" Of this information, the user should. be able simply to send a ·'next" message, as tile system knows which function (billing iDquiry, in this case) is being exacised on wbich customer and wbich of many pages is cummtly being viewed. This continuing information-tbe function, the customer, and the page-is the context of the user's next RqUeSt. But w~ can Ibis context be kept? Not in the senter, because it is not known in advance, which server witbjn a class will get the next request. Ii can be maintNned in the
.s
66
Transaction-Processing Systems
Chap. 2
requestor; but perhaps the requestor services many terminals. Moreover, if the context data size is large for the worst-case request,1arge amounts of memory may be required for a requestor to store context. So much, in fact, that it may have to keep context areas on disk (as shown in Figure 2-26). And these extra disk accesses are going to have an impact on system performance. Therefore, an element of good TP design is to minimize request context. One excellent way in today's art is through the use of intelligent terminals, which can store the context at the user's site. Then the context can simply be"added to each request as that request is sent to the TP system. Of course, this compounds another in the ODgoing saga of performance' issues: communication-line loading. So we see that achieving optimum pedormance is an unending search for best compromises. How could we ever do that effectively without pedormance modeling? The answer is simple-we don't! In the next chapter, we introduce the basic concepts of pedormance modeling. This will set the stage for the more detailed discussions to follow.
3 Performance Modeling
The degIadation of performance as a TP system gets busier is caused by two related factors: queues and bott1eneclcs. Queues of traDSaCtioDs awaiting service will form in a busy TP systein for each shared resource. As the system becomes busier, these queues will get longer, causing processing delays and performance degradation. The system resource with the lowest capacity will limit the ultimate tbroughput of the system and is the
system's bottleneck. The primary role of a performance model .is simply to identify the queues and bottlenecks in a system and to evaluate tbeir impact on system perfODDaDce. The system designers will then know what to expect from the system and which areas widJin the system provide the most fertile opportDDities for performanc:e eDbancemem In 1bis chapter. we explole the cbaracteristicsof queues and bottleDecks. with a simple yet talistic and cOmpiete perfOI'l:llalK:eaualysis. As DOted in the Introduction, a tbo.rougb uadeIstandiDg of the CODteDts of 1bis chapter provides the system aaalyst not otherwise interested in bemming a perf01'lDlDCe "specialist" with the tools necessary for e1emen1aly perfonmmce analysis•.
BOTTLENECKS A boUleDeek. is nothing more than a common system resource that can run out of capacity Any common resource within the TP system is a ~date for a bottleneck. The disk is the most common bottleneck in TP systems; but
befoIeother COJJIQlOD systemresomces do.
67
68
Performance Modeling'
Chap. 3
given enough disk spindles to provide multiple access paths in order to spread data' uniformly (in terms of access requirements) over the disks, this bottleneck can be broken. The processor (or processors) might run out of capacity before the disks and thus become the bottleneck. Or memory could be the bottleneck if there is not enough to provide common data structures (such as 110 buffers) or if there is not enough to accommodate the bulk of process memory requirements. This leads to excessive o~y swapping or page faulting.
The bus that connects multiple processors or computers in a distributed system can be a bottleneck, as can communication lines when many user teIminals share a common line (multidropped lines or local mea networks). Fmally, processes themselves can become the bottleneck. This is often true of the data-base manager, since data-base consistency requirements often preclude multiple instantiations of the data-base manager. Log servers and other nondyuamic server precesses are other examples of a potential process bottleneck, as discussed in the previous chapter.
Sometimes, a bottleneck is built into the system unintentionally. More tban one designer has brought a system to its knees by deciding to log unsuccessful transactions on the console printer for opemtor action. If the printer is a 3O-character-per-second printer, and if the failed transaction requires 300 characters to describe it, it will take 10 seconds to print. Let us say that the TP system was designed to handle SO transactions per secondor SOO in 10 seconds. If cmIy 0.2% of all transactions are unsuccessful, the console printer becomes the boUleneck!
QUEUES
Queues are much more exciting than bottle.aecks. Howevez-, we wouldn't have queues if we didn't have bottlenecks, for a queue is simply a line of Rquests for service by a c:ommnn system IeSOUrCe that, of course, is a candidate for a boa:leDeck In fact, the closer it is to being a system boCdeDeck, the loDger in general will be the queue waitiDg for it. Some system. JaOUnleS don't proVide' a queuing capability. Oar infamous console printer could be an ex"",le oftbat if e8ch pvcess WIIdiDg to pdDt a message bas ~ sejze it, prim its message, and tbeD Jelease it. If the c:ciJsOle printer bas abeady been seized, then the process must back off and try again JaIer. In tbis case, traDsactions back up in the queues of, the zequestiDg processes, provicIiDg they support queues of awaiting traDsactions, or back up to pevious queues, or eventDaJly even to the user. Of course, the CODSOle printer sync1rome can be cured by providing a CODSOle process that drives the printer. Other processes will send their messages to the queue' of the console process, which will then leismely print them without holding up the other pre-
cesses.
A more critical situation is c:baracteristic of common memory. If the common memory subsystem of a mulIiprocessor system does not pzovide queuing, there is DO, way for ,S9ftware to provide it because of the very high-speed Datme oftbis subsystem. In fact, by
Chap. 3
The Relationship of Queues to Bottlenecks
69
definition, memory speed must be comparable to the instruction execution speed of the processors that it supports, since it is one of the limiting factoIs in execution speeds. Therefore, some processors oontending for common memory will have to back off and wait while the CUIIeD.t processor is being serviced. . Some common memory systems do provide a queue implemented via hardware buffers. Sophisticated processors with look-ahead capability can anticipate their data and instruction needs and can therefore queue memory requests while they continue pr0cessing. Of course, being a very high-speed and expensive b~, the common memory queue is usually very short. Once filled, processors will have to wait as if there were DO queue. We have DOW seen tbree simple classes of queues: none, limited size, and infinite (at least practically so). Queues ate explored in great depth in the next chapter, where it is seen that maximum queue length is only one of several attributes characterizing the behavior of a queue. The time required for processing by a common resource is the sum of two c0mponent times: the time spent waiting in the queue and the time spent actually being serviced. These ate called the wait time and the service time, respectively. The sum of these two times is called the delay time, i.c., the time that the transaction is delayed due to a service requirement by that resource: . Delay Time = Wait Tune
+ Service Time.
For a large class of queuing situations (randomly distributed mivals and service times in a steady-stare system), the delay time is given by (see chapter 4): Delay =
T r=z.
(3-1)
Here T is the average service time of the resource (or the server as it is called in queuing theory, not to be confused with the use of server in the !eCpJeStor-servel model, which is also a server in the queuing sease. Oh. well!). L is the load imposed on the resource (the p!qJOl1ion of time that it is busy). L is also called the 0CCDpIDCY of the server. The term 1 ~ L can be thought of as a ..stretching factor." h Stletcbesoat the service time of 1he ieSOUrte as the load on that resource inaeases to account for the queue waiting time. In fact, the service time is sttetcbed without bound, i.e.• die queue grows ubiUarlly large as die load approacbes l00eJ, (L= 1). THE RELADONSHIP OF QUEUES TO BOTTLENECKS. We can illustrate the relationship between bottlenecks and queues with a simple example. Assume that a request requi1'es six process dispatches to service: once for die requestor process to IeCeive the request. once for the server process to receive· it, once for the data-base mauage:r to receive it and to initiate a disk request, and once for each of the above processes to receive and return the reply. Notice that this request also IeqUires one disk~. ~ assume that there are several requestor aad server processes handHng
70
Performance Modeling
Chap. 3
a volume of like transactions, each vying with the others for access to the data-base manager and each competing with the other and with the data-base manager for the use of the processor. Using equation ~ 1, we can predict the perf~ce of this system. Let tp be the average processing time requjrecl for a process dispatch, td be the average disk access time, and R be the total system transaction rate. Then the load on the processor is 6Rtp , and the load on the disk is Rtd. Since the total response time that the operator will see is the sum of the processor and disk delays (since each step must be performed in series, one after the other), we can use equation 3-1 to write 'D _ _ &-1:"'-
time
=
6tp + td 1 - 6Rtp 1 - Rtd·
(3-2)
This expression shows the process waiting in line and being serviced six times by the processor and being serviced once by the disk. 1 Let's put some numbers into tbis equation and plot the results. We will CODSidertwo cases. For case I, let the average processing time requiIed for each process dispatch, tp , be five mUlisecnnds (msec); and let the disk access time, td, be SO DlSeC. Response time as a function of system load, R, is plotted in FIgUre 3-1a for this case, along with the component delay times for the processor and disk. Note that the disk capacity is 20 traDsactioos per second (11.05 seconds) and that the processor capacity is 33.3 transactions per minute (11(6 x .005) seconds). Thus, the disk is the bottleneck; and theresponse-time curve re1lects tbis. Now let us double the proc:essjng time to 10 JDSeC. The resulting response-time curves are shown in Figure 3-1b. The processor now becomes the bottleneck, with a capacity of 16.7 transactions per second, as compaIed to 20 for the disk. As the system load approaches the bottleneck capacity, the IeSpODSe time grows quite large. In fact, note that at around ~ of system capacity (12 and 10 traDsactions per second, respec:dvely), the respcmse time iDa:eases dmmatically with small increases in volume. This is the poim at which users may really become frusIrated and which empbasizes a basic rule in the design of 1'e8l-time systemS: never load a component to mom than 60% to 70% of its capacity. 'UDfoduuately, the use of this rule is often the sole attempt at perfcmumc:e aualysis.
Befcxe we can get seriously into pedOa "umC:e modeUug. we must 1iIst come to apeement on wbat we are measudng. Sblce TP systems are interactive, and since the most common ~ oompJajnt is system slugg;sbness. it seems reasoaable, that response time be
om:
lJn Ibis example, qgeaes baildiDg at die data-base DIIII8F' aad die pnlmses are igDcnd to simplify die c:ak:aJaIioIIs The zesaltiDg CODCepIS at Ibis pojDt are IDCIIe hupOilaat 1baD is die proper modeJiDg ec:Imique. which wm be ldDecl sboIdy.
Chap. 3
Performance Measures
71 BOTTLNECKS
0) fp = 5 msec. fd =50 msec.
Processor -
"
I ,
I -Disk
I I
~.3 i= IIJ
oz .2 ~
13
II: .1
/
L..-_-- - - ---
"",-'
"
.",
/~
o
o
5
10
I
I I I
J
DISK
PROCESSOR
I
I
'y
,,/
/
I"" I
------------
.5
I
, I I I
_-
--- -- ..... --+...., 15
20
25
30
35
TRANSACTIONS/SECOND
= 10 msec. td" 50 msec.
b) tp
5
10
15
20
25
TRANSACTIONS/SECOND
PrimarY candidate. We have ~ explored the concept of respcmse time in some detaii in the previous section• . However, response time COIDeS in many varieties. What we have described so far is the average. or mean, respcmse time. In Figure 3-1a, for a load of 10 tnmsactiODS per ~d, the average respcmse time is .143 ~.
72
Performance Modeling
Chap. 3
In many cases, the system operator's concern is not with the average customer but with the iIate customer. Therefore, the operator wants to specify a maximDm response time-Iet's say of two seconds-that ensures a level of performance that may keep eveI]'body happy. . Unfortunately, the specification of an absolute cap on response time is quite umeasonable, because the wOl'St-case. response time will occur only for the very UDlikely (in fact, nearly impossible) case of simultaneous receipt of transactions from all users. For instance, assume for the case ofFlgUre 3-1a that there are 600 customers using the system. If, on the average, each generated a transaction once a minute during peak activity, the average transaction load on the 'system would be 10 transactions per second, giving an average response time of 143 msec. A pretty swift system. Nevertheless, the WOl'St case would indeed occur if all transactions mived simultaneously. Then, since the disk would be the bottleneck and would process only 20 transactions per second, the last transaction would be completed in 30 seconds. Pretty horrible. And also very umealistic:. Taken to its conclusion, only 40 users could be supported and still guarantee a maximum response time of two seconds. A more realistic statemcmt would be to ~y "99.99% of all ttansactons will finish in less than two seconds," or some equivalent statement. This still would leave an occasional iIate customer, but the level of hassling could certainly be controlled (1 out of 10,000 may Dot be too bad). To achieve the level ofperfOJmaDCe described in that statement, DOt only do we need to know the average value ofn:sponse time, but we also need to know its distribution. We must be able to state that with probability p, all responses will be received in less than t seconds, whatever p and t are specified to be. This, then, is the second performance IDe8S1R (average response time being the first). Analyzing the distribution of respcmse time is a much more difficult problem tban estimating its mean and is solved for only a few cases. Fortunately, for the general case, thete is an elegant and simple approximate solution that we will often use. Formaayofthe queues we will analyze, one can make the statemeDt that 9S% of all response times willl:!e less than tbree times the average response time, 99% will be less than five times the average, 99.9% will be less than seven times the avenge, and 99.99% of all responses will be less than nine times the average. ('Ihese staraDeIIts are based on the Gamma function, which is explcnd fUrdIa" in chapcer 4). Thus, to achieve a level of performance such that 99.99% of all response times·will be less 'than two seconds J:equi:res an average xesponse time of DO JDOIe than % = .22 seconds. From FJgUre 3-11, tbis allows a 1raIIsadion rate of 15 U3DSaC:IioDs per second. The difference between 99.99% and 100% satisfaction is the c:IiffeIence between 40 users and 900 users! Lesson: Tum a deaf ear to absolute maxima: We have, mc;ctemaUy, just introduced the thhd performance measure: ctI[IQdtj. As we have seen, the capacity of a TP ~ is the sumiJwl ttansaction tate tbat prodD<:es the maximum acceptable response time, wbether that response time is stated a mean ~ time or as the probability that a response time will not exceed a certain limit.
as
Chap. 3
The Analysis
73
In summary, the performance measures with which we will be concerned are the following: • Mean response time. The average respoDSe time to a transaction seen by a user, . taken as a function of system load. • Maximum response time. That time such that a specified proportion of all responses will occur in less time, also taken as a function of system load. • Capodty. MWmed in terms of system load, that which results in the minimum acceptable performance as spec:ified by a response-time requjrement.
THE ANALYSIS A proper performance analysis comprises many parts:
1. A system description of the TP system to be modeled. 2. A scenDrio model that describes the chaIacteristics of the traDsaction load being placed on the TP system. . 3. A traffic model that describes the flow of data through the system. 4. A perfornuzn&e model tiDcummt recording for posterity the above tbree items. 5. A pe7fOT'fNlllCe model program tbat Unp~ the equations of the model if they are too complex for IIWlU8l c:alcn1arion. 6. Result memoranda that give results of the model's predictions as "what if" games are played with it. Items I, 4, and 6 (system description, performance model document, and result memonmda) are discussed in the Summary cbapters 10 and 11. Item 5, the program, is DOt a topic for tbis book; the need for it is mentioned only in passing. though it is briefly
CODSidaecl in chapter 10. However, items 2 8Ild 3, the scenario model and the traffic model, represent the bUlk of the performance analyst's work and will be imroduced here. First, the scenario model.
Scenario IIofIeI The sceuario model deals with dJaracteriring the ttaDSactioD load being placed on the system by the lISeIS. There is DOtbiDg very complicated about it-it is Ie8Ily just a case of bean counting-bat it caD be a very laIge (and adJnittMty dull) task at times. The sceuario model identifies every tnmsaction that will be Offered to the system and characterizes the ttaDSaCtioD's use of xesourc:es: CQ!DII!1!Dk:ation message lengrhs, disk accesses. processing requilemeDtS, and orber special mquhemeuts. It also establisbes the probability of each ~on oc:cuniDg.
74
Performance Modeling
Chap. 3
Using these probabilities, the load on each resource imposed by an "average transaction" can then be calculated by summing the products of each probability and the use of that resource.
Sometimes we cheat and don't take all transactions into consideration. It may be that 3 transaction types account for 95% of the load and 30 transaction types account for . 5% of the load. In this case, it is perfectly reasonable to consider only the 3 types and "add. 5% to their imposed load to ~ up the diffeIeDce. At other times, we're not so lucky and must account for many. many transaction types. As an example, let's play lucky. Let us assume that there are only two types of transactions: an inquiIy and an update. Each has its own unique request- and replymessage formats and undergoes similar processing in a requestor-server environment, except that inquiries are processed by the inquiry server and updates by the update server. Inquiries require a single disk read, and updates require a read to fetch the record to be updated and a write to return the updated record. Thirty-five percent of all transactions are inquiries, and 65% are updates. Our scenario model can then be summarized in the following table, with values added for processing times and for communication line message lengths: TABLE 3-1. SCENARIO MODEL Message size (bytes)
Traasacticm IDquiry
Update
Disk
Server
Probability
~
Reply
accesses
time(msec)
.35 .65
20 200
400 IS
1 2
10 IS
In many cases, the values in Ibis table will be expJeSSioDs. For instance, the number of disk accesses Rquired for a cashing uansactiOll in a wagering system may be a function of the DlllDberof bell OIl a ticket. The size of a COlDhmniadiOD mquest message for an iDsert tnmsadion in a ~ system may be a fanctiOD of the size of the iDsert, and so forth~ iii the example perf~ model leport ~ in cbaprer II, the scenario table is quite complex. The above example is, of course, quite simple. Traffic IIodeI The traffic model is. the fun of performauce mocteling. It is the characte:rizati of the flow of requests and Ieplies (the system ttaffic) through the system. Evqy step that may have an impact on performance shoUld be included, as we do not want to a priori what is ~t and what is not. Let the zesults of the model tell us tbat.
Judge
Chap. 3
The Analysis
75
It is often helpful to draw a traffic diagram. This diagram is only a tool; it is DOt intended to be rigorous. Only three symbols are needed, but feel free to invent additional ones if they will help you organize your thoughts. The three symbols are . _...... ... - _....-...... .. . . 1. A processing step, typically representing a process or a disk access. The step is described in the box, and a service-time expression is provided.
--_ _-
2. A message path, such as a communication line or an interprocess message. As with a processing step, the path is identified, and a service-time expression is given.
3. A queue. If the queue has a maximum size, it is shown. A queue-length expression is also shown.
10
--II~" 11"11 q
I ..
The 1raffic ctiagram is sttuctm:ed to show ill the processing of a ttaDSadion each step that is sigDificant relative to pedmnanc:e. 11 is often helpful to DUlDbertbe steps so that the ctiagram' may ,be easily desc:ribed(part of good doc1l1laeD1ation that wiJl be messed in chaprer 10). For some sysIaDS,mulIiple 1raffic ctiagrams may be teqUhecl to cbmctaize diffeJ:eDt classes of tIaIISaCtiom if their plOCeS'ing sequences are sigDi1icant1y difJeiem. The average response time can then be expiessed as the sum of the IeSpODSe times of the ciomponent processiDg aad message patbs plus the time Spent in the various queues. , . , As an example of a trafIic model, let us c:oasida" a variation on the reqUestor-server system described in cbapter 2 and as illustIatecl in FJgUIe 3-2. Requests are received by a request handler process and routed to one of two servers: an inquiIy server dedicated to haudUng inquiries and an update server dedicar.ed to bandting updates. Each issues its own dasa-base directives, to the data-base manager, as described in the scenario of the ,
,
,
.
7&
Performance Modeling
Chap. 3
REQUEST--!·~8 · ~ DATA BASE
CDI~:>
REPLY"'~E---
Fipre 3-2 A TP system.
server
Previous section. i.e., the inquiry will request the Iead of a record; the update server will request a Iead followed by a write of the updated record. Servers are single-tbreaded. Each processes ODly one transaction at a time. Each server formulates a reply to the user when it bas fiuisbed processing the transaction and mums tbat reply via the reply handler. Replies to inqujries contain the requested data; replies to updates contain a completion status. CcmmmDication-line message sizes. server processing times. and the distribution of ttaDSaCIion types are given iD Table 3-1 (the sceuario model developed iD the previous section). User tenniDals are CODDeCted to the system via point-to-poiDt 96OO-baud (960 bytes per second) asynchronous COIDJDUDication lines (one termiDal per line). All 110 is done by the requestIreply baDdlers for M1IIIIIIJDicatioas aad by the data-base mauager for disk; tbere are no separate 110 processes. An processes nm at the same priority. and iDtemJpt processiDg can be igDoIed. . A ttUfic model fortbis system is shoWn iD Figure 3-3. Refenmg to the numbeI:eci steps in pareatheses on tbatdiagEam, we fiDel a mquest beiDg ttaDsnritted from the user temrinaJ over dleCMlrmmicaI:ioD)iDe (I). requiriDg a time. tao 'Ibis request is received by the RqUeSt bandler process (2), wbicb mquhes a processiDg time of '1'1 befqle passiDg die RqUeSt via an iDtaprocess message (3)'to die queue ofthe-appopriate server (4)~ Interprocess messages IequiIe a time, It-. and die servers (5) zequiIe '. time to tota1Iy process an iDquiEy request and sead back Us teply and ,_ time to p1'OVide similar service for an update request. Avaage queue 1eagtbs for the iDqaiIyserver and updaIe server are lJi and lJ". respectively. (Note the symbology ctif6cully iD the use of the subscript i to mean input as in lei and '1'1 and inqrdTy as in lJi aad r.,. This plObIem is c:ompounded iD larger models but is a hazaId of the professiClll. Clear and unambiguous definitioD is the 0Dly solution.)
The server Diust make an avaage of lid disk calls to process a traDsactioD. For tbC iDquiry server, lid = 1 (one mad) 8Dd lid = 2 for the update server (a read followed by a write in order to update the JeCO.rd). Each requires an iDtetprocess message (6) to send a . ~ to die data-base manager's queue (7), ~ averages lJb items in length. The
Chap. 3
The Analysis
77 (10)
(2)
(I)
(3)
RECIUEST~ ~ II.PM tel·
~
. t ,pm
(9)
(4) I
JjJJ}-
'----r-'
qi
~
qu
tsu nd • I FOR INQUIRY = 2 FOR UPDATE
REPLY. comm teo Figure 3-3 Traftic model.
, !
data-base manager (8) tequiIes a prOcessing time of tb to service each request plus a disk
access time (9) of t.. Note that there is no queue for the disk. as its only user is the data-base manager. which itself provides a queue for its users. Theretum of the disk data (for a read) or status (fora write) (10) to the inquiry server or update server is time-free, as it is the READ portion of the original WRlTEREAD iDteIpIocess message which sent the IeqUest to the data-base manager and is therefore included in the origiDal message time. Disk IeSpODSeS do DOt queue for the server, since each server is siJ1g1e-1hIeade and is waitiDg for the respoase. When the server bas fiDished processing the iequest and bas formulated a reply, it sends that reply via an iaterprocess message (11) to the l1'p1y bandler queue (12), which averages qo replies in leDgtb. The reply bandler then retums the reply to the ~ (13), requjJing a processiDg time of t,. aDd a MID1D1JDicalion time of teo. Wbat is your intuitive feel for the capacity of this syStem? What is the PrecJoimnani bottleDeck? Let's see how good your guess is.
We can DOW express respoase time as a function.ofload, using DO more mathematics 1ban that of equation 3-1·and DO D1O!e sysIembowledgetban 1bat giveD in cbaprm 2. The ODly subtlety is the effective service time oftbe poc:esses, siDce they an c:ompece for a common processor. This is dealt with below. . . '.. To 1eDc:l a bit of organization to the developmem
Te
= COIIIID1IDicati line component of respoase time (sec:onds).
Ti = request handler component of n=sponse time (seconds). Ts = server component of respoDSe time, including the data-base manager time (sec. ODds).
Performance Modeling
-
78
Chap. 3
TD = reply bandler component of response time (seconds). ,
Then, T = Te + Ti + Ts + TD
(3-3)
is the system re5pODSe time, where
T = system response time (seconds).
Communications. Since there is no sharing of communication lines (they are point-fO.point lines), there are no line queues. Communication time is fixed. Let
= communication line speed (bytes per second),
s
mii = inquiry request message size (bytes),
= inquiry reply message size (bytes), m,u = update request message size (bytes),
miD
IPZut>
= update reply message size (bytes),
Pi = probability of an inquiry transaction,
p"
= probability of an update transaction.
=
Of course, Pi + p" 1. Then the total communication tnffic for an inquiry is (m;; + m;o) bytes and for an update is (m,u + 1PZut» bytes. The average message length is the sum of these weighted by their probability of oc:c:urRnce. 'Thus, the average traDSaction time contributed by the· ' communication line, Te. is '
Te = (p,(m;; + m;o) + pJ..m,,; + 1PZut»]Is
(34)
Request handler. The component of response time inttodnced 'by the JeqUeSt handler comprises the foJiowiDg items:
1. The service time for serviciDg the JeqUeSt•. 2. An iDtaprocess message time for samng the zequest to the server queue. (Where we put this is ariAtwy. For this model, we wiD load intapocess message times onto the sender.) The request handler service time mnst include 1he process dispatch time, sinCe when the RqUeStor is Ieady to service an item., it must fixst get in line with the other processes to wait for the processor. Once it has the processor, it completes its service befoIe relinquislUng it. Let td
= avemge process dispatch time (seconds).
Chap. 3
The Analysis
79
Then Ti td
= tri + td + tipm
(3-S)
will be evaluated later.
Server.
The server response-time component comprises the following steps:
1. A wait in the server queue. 2. The service time for processing the :request and formulating the reply. 3. lnterprocess message times to send data-base requests to the data-base manager queue (lid requests per transaction). 4. The data-base manager service time for each disk :request. S. An intexprocess message time to send the reply to the outgoing :requestor. Let us define the following terms:
tqi = delay time for inquiry service (queue plus service time .in seconds). tqu = delay time for update service in seconds.
Since the probability of these two types of ttansactions is Pi and P,., respectively, the server response time component, Ts, is
Ts = p;tqi + PrJ",.
(3-6)
By further defiDing Tb = data-base mauager delay time (seconds), Lsi = load on the inquiry server,
La. = load on the update server, we can expzess the response-time c:ompone:at for an inquiry traasaction, _ 'si + 2td+ Tb + 2tp 1 - L,.;
tlli -
and for an update transaction,
t",.,
tqi,
as (3-7)
as
_ t$ll + 3td + 2Tb + 3tp 1 _ L$II
tl/ll -
r:LO) \..-v
Remember that an inquiIy ~ one disk access and that an update IeqUi1'es two disk accesses. The server process must be dispatched once upon the Iec:eipt of the request and once upon the Ieceipt of each disk response. The server loads are, teSpeCtively. Lsi = P;R(tsi + 2td + Tb
+ 2t;pm)
(3-9)
(3-10)
Performance Modeling
80
Chap. 3
where R = transaction rate (transaction per second). Data·base manager. Note that the data-base manager time, Tb , becomes a part of the server service time because it is in a closed loop with the server; that is, the server must wait for each disk request to be completed before it can go on. Thus, this time becomes part of its service time. Since the data-base manager has its own queue, the server delay time is a solution to a compound queue problem but in a straightforward and obvious way. Letting
Lb = data-base manager load, then the data-base manager deJay time,
TIl>
is
(3-11)
where (3-12)
Reply handler.
Once a reply has been issued by a server, it suffers the following
further delays: 1. A wait in the reply handler queue. 2. The service time for processing the reply.
4
= reply handler load
the reply handler service time compoaeot, To, is
T. o
= tro + t"
1-4
(3-13)
where
4 = R(tro + ttD
(3-14)
Dispatch time. We DOW have expIeSSions for an parametets based on known inputs, except for the dispatch time, tdo This is the average time that a process must wait in the processor queue (the Ieady list, as described in chapter 2) before gaining access to the ~.
Chap. 3
The Analysis
81
For each transaction, the following dispatches take place: Number of dispatches
Ploc:ess
Service time
Requestor Requests Replies Server IDqairies Updates Data-base IIIIIIII&eI' Toral dispatches
t,,+ ·t.... t",
3p.
Pi + ']p. 2+3pi+SP.
From this list, one can state that the dispatch rate, Td
t61 + 21.... t.,,+ 3t.... t.
']pi
Tdo
is
= R(2 + 3Pi + 5p,,)
(3-15)
The average process time per dispatch, tpo is calculated by summjng the various process service times weighted by their probabilities of occummce for a transaction. This probability of occunence is the ratio of their fiequencies of occurrence in a transaction to the total number of dispatches per traDsaction. Thus, average process time, fp, may be ex~as
fp
= [tri
+ tTO + P~6i + PrJ"" + (1 + P.)tb + (3 + p,,)tq.JI(2 + 3Pi + 5p.,).
(3-16)
The processor load, Lp, is
Lp =
T~p
(3-17)
. . . . . Thus, average dispatcll time, t", (the time spent waiting in the processor queue bnt exclud. ing the service time) is from equation 3-1: 1
-..i..._ "P-I-Lp'fl ~ - Lp t t.d-l-Lp
(3-18)
Model summary. 'Ibis model is smnmarized in Tables 3-2 anc:l3-3. Table 3-2 lists all model paua:uerms, separated into four categories:
• Result ptl7Il1MIer. Those calcalamd parameters that are of most probable inteIest as a result (0Dly T in this case) . • Input VtIriobles. Those parameters which are most likely to be varied to play "what if" games (only R in this case). IA men ac:cmate approacb would c:ak:aIaIe diffezaIt dispatdl times for eich pocess by excluding ~ pmc:essor load of the pcoc:ess beiDg evaluarecI. See Appeadix 6 aDd a CCIIDpII8bIe example ill Qaprer 8.
Performance Modeling
82
Chap. 3
TABLE 3-2. PERFORMANCE MODEL PARAMETERS 1. Ruult PIITtIIIIdD'S T Avenge traDsadion respoase time (seconds) 2. Input VDriable.s R Avenge system 1IIIIS8dion rare (ttaDsacrions per sec:cmd) 3. Input PilTtl1MIeTS mu Avenge iDquiIy request message lengdl (bytes) 1IIj" Avenge iDqaiJy reply message lengdl (bytes) III,,; Average updare request message lengdl (bytes) m.. Avenge updare reply message lengdl (bytes) p; Probability of aD iDquiry uusaction P. Probability of aD upcIaIe traDsaCtiOD s COJIIIDDDic:adoD liDe speed (bytes per sec:cmd) t. Avenge daIa-bese IDIII8F' proc:essiDg time per disk request (seconds) Z. Avenge disk access time (secoads) IDretptocess message time (seconds) t,; Avenge request baDdler pIOCeSSiDg time for a request (seconds) t", Avenge reply haDdler proc:essiDg time for a seply (seconds) to; Average iDquiry server ptoc:essiDg time for aD iDqujry (seconds) to. Avenge update seMI' processing time for aD upcIaIe (seconds) 4. I~ PIITtIIIIdD'S L" DaIa-base IIIIIII8ger load L" Reply baDdler load Lp Pnx:essor load 1.,; IDquiIy server load L.. Updafe server load Td PIOcess dispIrch rare (ptcmm per seccmd) ~ Avenge ctispaIch time (waiting time for the processor in seconds) tp Avenge paocess time per cIispIrch (secaads) tq; Average iDqairy sene!' delay time (secaads) t.,. Average update sem:r delay time (secaads) T. Average dala-blSeJlllllllpl' delay time zequiIed to paocess a disk request (secaads) T. OcIi""",,/cmau.-liDe sespoase time CCIDIpOIlII1t (seconds) T; Request baDdler zespaase time compoaeat (secaads) T.. Reply baDdler sespoase time compoaeut (secaads) T6 Sener respaase time COIIIpOIII'JIIt (secaads)
t....
• Input ptITtI1MIerS. Those parameters wbich c:baracter.ize me'system and for which values are assumed for pmposes of the model• • IntennediIzte ptITtI1MIerS. All c:alcuJatecI. parameteIS except for leSUlt parame. ters. Table 3-3 SUiilmarizes the equaDODS, with ~ numbers to refecenc:e back to ~ . text. Note the top-dowD pn;seotation of all parameIer expessioDs in Table 3-3. The expressiOn for an intemaediate parameter is DOt presented until it appears in a previous expession. 'Ibis is an iDvaluable aid in organizing a model and in orgmrizing your own thought process.
Chap. 3
The Analysis
83
Model Results The average response time, T, as a function of the transaction load, R. is shown in Figure 3-4, using the values of input parameters largely suggested in previous sections and summarized in Table 3-4. Note the rather precipitous bIeak in the curve-much sha1per than we would expect from a T/(l-L) relationship. This is usually caused by a "sleeper" in the system: a bottleneck that contributes little delay at low loads but becomes the system bottleneck at high loads. In fact, with this response-time characteristic, the system will "bIeak" very suddenly as load is increased, and it is ~ to stay away from the knee of the curve. If the system capacity is taken as four transactions per second (67% of full capacity), we are fairly safe, with an average response time of about 0.6 seconds. Note that if we TABLE 3-3. PERFORMANCE MODEL SUMMARY 1. lWponse time T= Tc+ T/+ T.+ T.
(3-3)
.2.C~
Tc =[P,(IIIJ/ + m",) + p.c."". + m...)] s
(3-4)
3. Reqwst htwller T/=tli+t.t+z..., 4. SInen
(3-5)
T.=~+p.r..
(3-6)
_ ,. + 2t.t + T. + 2trr
(3-7)
I-La _ t. + 3t.t + 2T. + 31... z..l-L.. La =p/«.t. + 2z., + T. + 2t...> L. =p"R{t. + ~ + 2T. + 31..,) ft;-
(3-8) (3-9)
(3-10)
s. DcrIa-btlsc I/IIDItZIC7" :t _t.+t,,+z.,.
(3-11)
.- I-I. I. =R(p, + 'P.)(r. + t.t + z...) 6.
(3-12)
RIIJII11rt1n4ler r.=~
(3-13)
• I-I.. I.. =R(t,..+r,,>
(3-14)
7. DispaIt:h time t.t= 1 feL,. z,
L,. =r~
(3-18) .
_ [tli + fro + PI. + p.r. + (1 + p.)r. + (3 + p.)z.,.J Z, (2 + 3pi + 51.) r4
=R(2 + 3p, + 51.)
(3-17) (3-16) (3-15)
Performance Modeling
84
I
J
I
1.5
I I
.,, I
J
I
-I.
I
I
I
i=
I
1&1
I
Average Response nme
I
en I /
------ ... o
2
__ -
.",.
",
"
I
/
/
~
Server
COmponent
3
4
rme (tQU) 5
6
TRANSACTIONS/SECOND (R) Figare3-4 SysIem lespaase time.
TA8LE3-4. INPUT PARAMETER VALUES
Patameeer Ills
".. m.,
..... PI P.
.s It
,. t.,. tli t", tli
'.
MeaDiDg
IJIqIIi1y Ieqaest leDgIh (byres) IJIqIIi1y seply IeagIh (bytes) Update Ieqaest Ie:DgI:h (byIIes) 1JpdaIe seply IeagIh (bytes) PloIIabiIity of aD iDqaiIy PloIIabiIity of aD apcIaIe CmEhHillj.-:atioa !iDe speed (byres per secaad) DII:a-base IIIIIIIF' pocessiDg time (secaads) Disk access lime (secoads) IDImpiOCii$$ messap lime (secoads) Request baDdIer pocessiDg lime (secaads) Reply b8Ddle.r paoc:essiDg lime (secoads) IDqui:ry sener paoc:essiDg lime (secoads) Update sener ptOCeSSiDg lime (secoads)
Value 20
400 200
IS 0.35 0.65
9CiO .010
.035 .OOS .OOS .OOS .010 .015
Chap. 3
Chap. 3
The Analysis
85
should push the capacity up by 25% to 5 transactions per second, we only get a 15% increase in response time (to 0.7 seconds). However, at this point, a mere 10% increase in load gives almost a 50% increase in response time! This fairly low capacity-four transactions per second-is DOt an uncommon cbar~ acteristic ofTPsystemsas a per processor measure. One to teD transactioDS per second per processor is the common range for contemporary systems. This emphasizes the great importance of performance analysis and its companion, perfOImaDce plaDning. Note that we can use the model to "peek" into the system to analyze its performance more deeply. Not only can we plot the component response times-Te, Tit Ts, T(1' tqi, and t..-but we also can evaluate component loading-Lsi> La, Lb , ~, and Lp. This capability is our tool to determine where the system can best be modified to enhance its
performance. Let us. apply this tool to our example system. Peeking inside will identify our "sleeper," the update server. Its response time component, t.. , is also shown in FlgUIe 34. h is initially hidden by the comrmmications component, which is independent ofload because of the point~~point lines. However, it becomes the bottleneck at six transactions per second. Wbat Can we do to eDbance the system? Simply add another update server, and have doubled the capacity of the system. No extra batdware needed. In fact, the model will show the following component loads for the single server case at a load of six" transactions per second:
we
IDqairy server (h,) lJpdare server ct-) Dala-bIIse JIIIIIIIpI' (4) Reply baDCIer (J..)
PRIcessor (J..,J
0.27 0.94 0.49
0.06 0.35
By addiDg a second update server, its load is Ieduc:ed from a vety high load (0.94) to a reascmable load (0.47) at six traDsadioDs per second. Since the haniware componeitts Of the system-the disk: and processor-are carrying a 35% to SO% load, significant capacity enhancwuent could be achieved by moving into a mukiserverenviroDmeat wUbout purchasing any DeW hardware. The model could be cbaDged to Ieftect this and could be used to pIedict the optimum IlUIIIber of servers zequimd to balance the softwaIe with the bardwue. Of course, dyDamic servers, as described in chapter 2, would be self-adjUsting. The performance analysis would tbeD be diIecred to predict the performance of that system.
OaIy
Allalpis Sum...ry The above discussion bas presented, to some extent, the content of a proper performance ,analysis. The system description bas spmmed chapter 2 and this chapter, though ~
86
Performance Modeling
Chap. 3
description would normally be contained in a more formal and localized section of the performance analysis. A scenario model is developed along with a traffic model; significant explanatory text accompanies these developments. The model is summarized, and results are presented (no program was necessary to allow calculation of this simple model), as well as recommendations for system performance improvement at no additional bardware cost. In short, DOt only has an analysis been undertaken, but it also bas been thoroughly documented. Not only have we completed a real performance model, but we also have completed our first perfOI1lWlCe analysis! This analysis was completed using only a simple queuing equation, a knowledge of TP systems as presented in chapter 2, and a little native ingenuity. This is all it takes to be a performance analyst. Why, then, are there so
few of us? The remaining chapters add some tricks to our tool kit, give us a better understanding of our tools, and suess the impo.ttance of documenting our model and analysis.
4 Basic Performance Concepts
The matbematical foundation required for performance modeUng is very much a function of the depth to which the analyst is interested in diving-from sJdnmring the surface to over ODe'S head. As we saw in chapter 3. a peat deal of pedormaDc:e modeling can be acbieved with ODe queuing equation (equation 3-1). a lot of simple algebra, and some system
sease.
However, tbeIe _ many unusual and complex TP system an:bitectures that can be !DOle accurately analyzed with some better tools. 'J'bjs chapter can be ~ the ''tool kit" for the .book. 'l'.blee of the six sections of this cbapter pIOVide the basic fouadaDon of queuiDg theory for the remaining cbaptas. Below is a brief overview of 1hese tIRe sectioDs.
I. Queues-AnI7rtTDtluction gives a simple. iDI:ui1ive derivation ofpabaps the most impollaot: zelationsbip for the pafoJmance aaalyst the JCbIntchiDe-Pollaczek equation, which cbaracterizes a broad range of queue types. Equation 3-1 is a . subset of this eqaatioJL NmDlathematicians shoulcl find the iDtui1ive derivaIion UDCImtaDdabJe aDd illnnrinaring in terms Of 1IIlCIasIaDctin the behavior of queues. Those knowledgeable in queuing theory will find definitions of teIWS and ~ used 1brougbout the rest of the book. 2. Chl.arlcterizing Queuing Systems slll!ll"arizes the Kendall classification system for queues. This classification system is used tbrougbout the book.
Basic Perfonnance Concepts.
88
Chap. 4
3. Comparison of Queue Types summarizes the queuing concepts presented in this chapter by discussing the comparative behavior of queuing systems as a function of servi~ time distribution, number of servers, and population size. The othertbree sections of the chapter deal with a broad range of marhematical tools, including probability concepts, useful series, and queuing theory. Those less inclined to mathematics will be pleased to know that the rest of the book is completely understandable without a detailed knowledge of these tools.
QUEUES-AN 'NTRODUcnGN As we have seen in the simple performance examples in chapter 3, queues are all': important to the study of system performance. In the simplest of terms, a queue is a line of items awaiting service by a commoDly shared resource (which we hereiDafter will call' a server). Let us first, therefcxe, take an intuitive look at the phenomenon of queuing. Through just common sense and a little algebra, we can come close to deriving the Kbintchine-Pollaczek equation, which is one of the most important tools available for performance analysis. Its formal derivation is presented in Appendix 3, but it is not much more difficult than that presented here. Let us consider a queue with very simple characteristics. Interestingly enough, tbese characteristics are applicable as a good approximation to most queues that we will fiDei in real-Hfe systems.
These cbaractaistics are 1. 1'.bere is ODIy a single server. 2. No limit is imposed on the length to which the queue might grow. 3. Items to be serviced mive on a random basis. 4. Service order is first-in, first-out. 5. The queue is in a steady-state CODditio.D, i.e., its average length measmed Over a long enough period is CODSIaIlt
In FJgme 4-1, a server is serv:iDg a line of waiDDgttaDsadioDs. On the avemge, it takes the server T seconds to service a tmDsactioD. Some 1I3DSaCtioDs take more, some tate less, but the avemge is.T. T is called the 8DVice time of the server. The average number of 1I3DSaCtioDs in the sysIem, whlch we will call queue length, is Q tmDsaCtions. This compises W umsactioDs waiting in line plus the 1raDsaction. (if any) cummtly being serviced. Finally, transacIions are miviDg at a tate of R t:raasactions per second. The server is busy, or occupied, RT of the time. (IfttaDsadioDs are miviDg at a rate of R 1 per second, and if the server zequites T OS sec:oods to service a ttaDSaCtioD, then it is busy 50% of the time.) 'Ibis is called the·occupancy of the server or the 1Dad ~) on the server: . , (4-1) L=RT
=
=
Chap. 4
Queues-An Introduction
89
CO .1 -l 0-1
11--'
[
ARRIVAL RATE =R
I
I
SERVER
AVG. SVC. TIME • T
Tq = QUEUE nME (TIME WAITING IN LINE) Td • DELAY TIME (Tltt£ IN UNE PLUS SERVICE TIME)
L also represents the probability that an miving transaction will find the server busy. (Obviously, iftbe server is not busy, 1he!e is no waiting line.) When a tnmsaction mi~ to be serviced, it will find in front of it, on the average, W transactiODS waiting for service. W"11h probability L, it will also find a traDSac:tion in the process of beiDg serviced. The servicing of the CUJteDt traDsaction, if any, will bave been partiaUy completed. Let us say that only leT time is left to fiDish its servicing. The newly mived ttansaction will bave to wait in liDe long enough for the cummt transaction to be c:ompleted (leT seconds L of the time) and then for each traDsaction in front of it to be serviced (WT seconds). TheIefOle, it must wait in line a time Til (the queue time): Til
=WT + 1J(J'
(4-2)
From the time that it arrived in liDe to the time that its servicing begiDs, an average of W other traDsadiODS must bave mived to majntain the average line length. Since transactiODS are arriving at R ttaDSaCtioDs per secoad, tbeD . W=T"R
or Til
=WIR
(4-3)
EquadDg 4-2 and 4-3 and solviDg for 1he waiting IiDe length gives (usiDg 4-1)
1£2
W=I~
(~)
The to1allcmgth of the queue as seen by an miving ttaDsaction is the waiting line, W, plus a traDsaction being serviced L of the time:
Q =·W + L
(4-5)
L . Q = l-L [1-.(1 - k)I..]
(4-6)
or
Basic Performance ConceptS
90
Chap. 4
The delay time, Td , is detenDiDed in a similar manner. It is the total amount of time the transaction must wait in order to be serviced-its waiting time in line plus its se.Mce time. During the time tbat the transaction is in the system. (Tto, Q transactions must arrive to maintain the steady state:
Q = TdR
= TdLIT
(4-7)
Setting this equal to equation 4-6 and solving for the response time, Td , gives Td = 1 ~ L [1 - (1 - k)L]T
(4-8)
kL Tq = l-L T
(4-9)
From 4-2 and 4-4,
Equations 4-4, 4-6, 4-8, and 4-9 are the basic queuing equations for a single server. assumptions inherent in these expressions are that transactions arrive completely independendy of each other and are serviced in order of arrival. The equations axe, thelefore, very genem1 expressions of the queuing phen0menon. They axe quite accnrate when the transactions axe being generated by a number of indepeDdent sources, as long as the number of som:ces is sigaifiC8Dtly greater 1baD the average queue lengIbs. A common linrinng case of accaracy occurs when there is a small IlDIIlber of soun::es, each of which can have only one O'Iltsta1lCting transaction at a time (a common case in Computer systems). In this case, the queue length CIDIlot exceed the IlDIIlber of sow:ces; and the mival mte becomes a function of server load. (Arrivals will slow down as queues build, since sources must await servicing before geDeEating a new transaction.) Howev«, in this case, the above explessions wiD be coaservative, since they will in general pedict queue leDgtbs somewhat pealer than those that actua1ly will be experienced. (Obviously, the Jimiring case of only one sourcewDl never experience a queue, although equation 4-6 wiD predict a queue.) The parameter Ie in these equations is a function of the service-time cIistribuDon. It is the average amount of service time that is left to be expeDded on the CUIl'eIIt transaction ,being serviced when a new transaction mives. Let us look at these explessions for certain impoIIant cases of service-time distribuIions. As stated previously, the primary
For expoaeatial service times, the pobabiJity that the remaining service time wiD be greater than a given amouat t is expoaeatial (e-CI). We diseass this and other p10babiJity concepts later in this cbapter. An expoaeatial distribution bas the cbaracreristic that the remaining service time after any arbitraty delay, mllmjng that the servicing is still in pmgress, is S1ill expoaeatial, i.e., it bas DO memory. Thus, one bas the fonowing interesting situation: If the average service time of the server is T, and if the server is CWieady busy (for DO matter bow long), one is going to have to wait an average time of T for the "service to complete. Since Ie is the ptoportion of servicing J'I'!D'UIjnjng ~ a new traIIS-
Chap. 4
Queues-An Introduction
91
action enters the queue, then k = 1 for exponential service times. Thus, equations 4-6, 4-8, and 4-9 become L
Q
= l-L T
Td=T=7
(4-10) (4-11)
and
Tq
L
= 1 -L T
(4-12)
These are the forms of the queuing expzessions often seen in the literature and represent a typical worst case for service-time distributions (though it is possible to COD"struct unusual distributions that are worse than exponential). Equation 4-11 is the fomi" that was used in chapter 3 as equation 3-1.
Constant Service Times If the service time is a constant, then on the average. half of the service time will remain when a new ttaDSaCtion enters the queue. Therefore k = ~, and Q =
~ (I - ~)2 I-L
(4-13)
Td=--L(I-~)T l-L 2
(4-14)
LI2 T.q=-T l-L
(4-15)
and
These expressions show queues and delays that are somewhat less tbantbose pIedieted for exponential service times. CODSIaDt service times genaally repesent the best case.
(4-16)
where E(T)
= the mean of the service-time distribution (simply represented as Therein).
var(T)
= the variance of the service-time distribution.
Basic Performance Concepts
92
Chap. 4
and E(1'2)
= the second moment of the service-time distribution.
This JeSuit is derived in Appendix 3. We will hereafter refer to k as the distribution coefficient of the service time for the server. For exponentially distributed service times, as we will see later, the standard deviation equals the mean. Since the variance is the square of the standard deviation, then k = I, consistent with our argument above. For constant service times, the variance is zero, and k = Y.z, as also just shoWD.
Unifonn Sentice Times A service time that can range from zero to s seconds with equal probability is uniformly distributed. It has a mean of s/2 and a second moment of E(1'~
lL~Pdt = r/3 = -So
lberefore, (4-17)
This is between the cases of constant and random service time, as would be expected.
Discrete Sentice Times Often times, the service time will be one of a set of CODStant times, each with a duration of Ti and a probability of Pi. In this case, k=
! :~$
(4-18)
'Ibe above equatioDs aEe the very imPOltaut qneuing equatioDs derived by TCbjntebine and Po)]acze1c (ac:tua1ly, equatiOD 4-9 for the queue 1ime. Tt • is fOImaDy known as the
KbintcMne-PoDaczek equation, or the PoDaczek-KbintcbiDe equalion, depending upon to whom you wish to give primary credit). These are so important to pesfoJmance analYsis that we summarize them be:re: . 1cL2
W
= l-L
Q
L = -I-L [1 -
(4-4)
(I - k)L]
(4-6)
Chap. 4
Queues-An Introduction
Td
1 = l-L [1 -
Tq =
(1 - k)LJT_
kL r.:rT
(4-8) (4-9)
and (4-16)
where W
Q
= average number of items waiting to be servic:ed, = average length of the queue, including the item cmteDtly being ser· viced, if any,
Td
= average delay time for a ttaDsaction, including waiting plus service time,
Tq = average time that a transaction waits in line before being serviced, k
= distribution coefficient of the service time,
T = average service time of the server,
L = load on (oc:cupancy of) the server.
. Equation 4-8 for-the delay time is the one which will be most often used in this book. The queue length, Wor Q (equation 4-4 or 4-6), is often necessary when we are coucemed with overflowing fiDite queues. The queue time, Tq (equation 4-9), is useful when the queue contaiDs items· froin diverse somces, each with a diffem1t mean service time. In this case, the queue time is calcalatM using the weighted mix of an tnmsaction ~ times. The average delay time is calcalatrd for an item by adding the average queue time, calculated for the mix of an items mthe queue, to the average service time for that item. Thus, Td = Tq
+T
(4-19)
as is suppotted by equatioDs 4-8 and 4-9. Qrber nseful n=1atioDs between these parameters that can be deduced from the above equatioDs aDd that have already been. pesented, are Q =W+L
(4-20)
T.q-!IT L
(4-21)
and
Td=
'f
=Tq+T
(4-22)
Basic Performance Concepts
94
Chap. 4
Note that L is the average load on the server. As we shall see later, in a multiserver system with c servers, the total load on the system is d. Equations 4-20 through 4-22 still hold after substituting d for L. Equation 4-7, derived earlier, provides another important insight into queuing systems:
.
Q=RTd
(4-7)
This is known as Uttle's Law (Utt1e [18], Lazowska [16]) and states that the number of items in a system is equal to the product of the throughput, R, and the residence time of an item in the system, Td • These relationships are surprisingly valid across many more queue disciplines than the simple first-in, fiIst-out discipline discussed here. This point is explored further in Appendix 3, where it is shown that the order of servicing is not important so long as an item is not selected for service based on its characteristics. For instance, these relationships would apply to a round-robin (or polling) service algorithm but not to a service procedure that gave priority to short messages over long messages. CONCEPTS IN PROBABILITY, AND OTHER TOOLS
Performance analysis can at times require some innovative ingenuity. This typically does not require any knowledge of higher mathematics. However, a basic knowledge of probability concepts and other helpful hints can often be useful. The material presented in this section is intended to simply touch on those concepts that have proven useful to the author over the course of several performance analyses. We will launch into some detail, however, cnncmring randomness as itIelates to the Poisson and expoaential probability distributions. Not that we need these so much in our analysis efforts but because an in-depth knowledge of them is impeIative in Older to cleady understand many of the simplifying assumptions that we will often have to make in Older to obcain even approximate solutions to some problems. Excellent coverage of these topics for the p.taeticing analyst may be found in Martin [19] and GolDey [7].
Rando.. Variables Probability theory COIICel'IlS itself with tile desaiption and manipuJatioD of Tfl1IIlom variables. These variables desa:ibe real-life situations aDd may be classified into disc!ete and continuous random variables.
Discrete RandollJ Variables A discrete 'Vtl1itIble is one which can take ODly certain disCl'ete values (even though there may be an iDfinite set of these values, known as a coU1l1llbly i11jinite set). The length of a ,~is a discrete random 'Vtl1itIble; it may have no items, one item, two items, and so on
Chap. 4
Concepts in Probability, and Other Tools
95
without limit. The length of a queue with a fixed maximum length is an example of a discrete variable with a finite number of values. If we periodically sampled a queue in a real-life process, we would find that at each sample point it would contain some specific number of items, ranging from zero items to the maximum number of items, if any. If we sampled enough times and kept counts relative to the total number of samples (and assumed that the queue is in steady swe), we would find that the pIOpOItion of time that there were exactly n items in the queue would converge to some number. 'Ibis would be true for all allowable values of n. For instance, if we made 100,000 samples and found that for 10,000 times there was nothing in the queue, for 20,000 times there was 1 item. in the queue, and for 1,000 times there were 10 items in the queue, we would be fairly certain that the queue would nmmally be idle 10% of the time, have a length of 1 for 20% of the time, and have a length of 10 for 1% of the time. These values, expressed as proportions rather than as percentages, are, of course, the probabilities of their corresponc.ting events. That is, the probability in this case of a queue length of zero is .1, etc. We wiD note the probability of a discrete event as P,., wheIe n in some way describes the event. For instance, in the case of a queue, P,. is the probability of there being n items in the queue. If we were drawing balls of different colors from an urn, we might choose PI to be the probability of drawing a red baD and P2 that probability for a peen ball. Thus,
p"
= Probability of the 0CCUIrenCe of event n.
The set ofP,. that describes a random variable, n, is called the probability density function of~ . There are several important properties of discrete variables, but the most obvious are
I. Each probability must be no greater than I, since we can never be more certain than certain: '
OSp,. so 1
(4-23)
2. The probability deasity function must sum to 1. siDce 1 and 0Dly 1 event OIl each observalion is certain:
lJ,,.= 1
"
(4-24)
wheIe the SI""""rion is over an allowed values of n. 3. .Assnnring that ew:ms _ ~, the probability of a specific combination of ew:ms is the product of their p!Obabilities. Thus, if we were to draw a ball from an urn COJl1lriTring balls of several different colors, put it back, and draw another ball, and if PI were the probability of drawing a red ball and P2 the probability of drawing a green ball, then 1he probability of drawing a red ball and a green ball is PIP2. (Note that we have to put the fiISt ball back in order to avoid changing the probabilities; otherwise, the two events would not be independent.)
Basic Performance Concepts
96
Probability of ocamence
of a set of independent events
Chap. 4
(4-25)
= 1rp" "
where the product, 11', is over the specified events. Thus, "and" fmplies product (the probability of event 1 aNi event 2 is Pl]J2). 4. Assummg that events are independent, the probability that an event will be one of several is the sum of those probabilities. In the above example, the probability of drawing either a red ball or a green ball on a single draw is PI +
1'2. Probability of occunence of one of a set of independent events = ~"
(4-26)
"
where the summation is over the deslred events. Thus, "or" implies "sum" (the probability of event 1 or event 2 is PI + ]J2). S. The probability of a sequence of depeDdeDt events depends upon the conditional probabilities of those events. If we did DOt return the ball to the um, then the probabilities would change for the second dmw. The probability of a red and then greeD. draw would be the probability of a red draw times the probability of a greeD. ball given that a red ball has been drawn. Thus, the probability of red, then green = P1Pi1), where p,.(m) is the probability of event n oc:curring (a greeD ball in this case) given that event m has occurred. In general, letting p(n, m) be the probability of the sequeace of events n and m, p(n, m)
= p,.p".(n)
(4-27)
6. The average value (or mean) of a l'81ldom variable that has JllIDleric memring is the sum of each value of the variable weigbrecl by its probability. Let i be the mean of the variable with values n and probabilities p". Then (4-28) where the sum is taken over all allowed values of n. 7. h is often imponaDt to bave a feel for the ·'cfisPe.mion'· of the DDdom variable. If its mean is P, will all obemdioDs yield a value close to P (low dispersion), or will they vary widely (high dispersion)? A CODlDlOIl measme of dispeEsion is to calculate the average mean square of all obse:rvariODS Ielative to the mean. 'Ibis is called the variaDce of the lIIldom variable, deDoted VIr (n):
var(n) =
L(n-i)7" "
(4-29)
whe.re the sum is taken over all allowed values of n.. The square root of the variance ~ called its standard deviasion and, of course, bas the same dimension as the variable (i.e., items, seconds, ttaDsactiODS. etc.).
Chap. 4
Concepts in Probability, and Other Tools
97
8. The moments of the variable~ also sometimes used. The mth moment of a variable, n, is represented as TI" and is (4-30) where the summation is over all allowable n. Note that the mean is the ~ IDOIDeDt (m 1). There is also an important relation between the variance and the second moment:
=
var(n)
= L(n-iifp. n =
Ln7. n
~2nipn +
Lii?n
From equations 4-24 and 4-28, var(n) =
Til -
ii2
(4-31)
.That is, the variance of a random. variable is the diffmmc:e between its second moment and the square of its mean. (See equation 4-16 for a use of this relationship.)
9. If x is a mndom variable that is the sum. of other random variables, then the mean of x is the sum of the meaDS of its component variables, and the variance of x is the sum of the variances of its component variables. Thus, if
x=a+b+c+ .. ,
-
-
- ... x=a+b+c+
(4-32)
and var(x) = var(a)
+ var(b) +var(c) + ...
(4-33)
10. Ifx is a choice of ODe of a possible set ofvariables, thea its mean is the weigbIed avenge of those variables, and its second IIlOIDeIlt is the weighted average of the sec:oad DIOIIleDts of those variables. Thus, if a, b, c, ... are each random. variables, and if x may be a with probability P., b with probability Pin etc., then
- -
x- = ape + bpb + -CPc + ... ~ =DiP. + Jilpb + ilpc +
...
(4-34)
(4-3S)
Note that weighted second IIlOIDeDtS are added when x is a choice, whereas variances are added when x is a c:ombiDaIion. 11. The set of probabilities Pn !bat describe a I3Ddom variable may be summed up to, or beyond, some limit. This sum is called the cllmuJative disuibution func-.
Basic Performance Concepts
98
Chap. 4
tiOD of n. If the sum is up to but does Dot include the limit, then the cumulative distribution function gives the probability that n will be less than the limit. This is denoted by P(n < m), where m is the limit
< m) =
P(n
L p"
n<m
(4-36)
where the summation is over all n less than m. Note as m grows large, < m) tends toward unity. If the sum is beyond the limit (n > m), then P(n > m) is the probability that n will exceed the limit
P(n
> m) =
P(n
L P"
n>m
(4-37)
where the sum. is over all n greater than m. As m grows large, P(n > m) tends to zero. A simple example will illusttate many of these points. Figure 4-2a gives a probability deDsity function for the size of a message that may be transmitted for a terminal. Its size in bytes is distributed as follows: Message size (11)
Probability (p,,)
20 21 22
.1 .2 .3 .2
23 24
.2
Note that the probability of all messages is 1: 24
LP" ='1 _20 The mean message size is
ii =
24
L np" = 22.2 11-20
The varlanc:e of the message size is 24
var(n) = .
Its standard deViation is 1.25. The second moment is
L (n-ii)7" = 1.56 ,,-20
Chap. 4
Concepts
in Probability, and Other Tools
99
Note the relationship between variance and second moment: var(n) =
r1- - ;? = 494.4 - 22.22 = 1.56
'Ibis illustrates one potential computational pitfall. The variance calculated in this manner can be a small difference between two relatively large numbers. For that reason, the calculation should be made with sufficient accuracy. The cumulative distribution functions for this variable are shown in Figure 4-2b. As with the density function, these functions have meaning only at the disaete values of the variable. Thus, the probability !bat the message length will be greater than 22 is .4 (i.e., P23 + P24 = .4) and tbat it will be less tban 22 is .3 (i.e., P20 + P21 .3). Now, let us assume that we have a second message type with a mean of 35 bytes and a variance of3. Denote asml the first message described by the disttibution of Figure 4-2, and denote as m2 the new message just defined. Consider the following two cases:
=
Case 1. ml is a zequest message, and m2 is the zesponse. What is the average communication line usage (in cbarade.rs) and its variance for a complete transaction? In 1bis case, the communication line usage is the sum. of the message usages. The mean and variance of this total usage are the sum. of the means and variances for the individual messages. Let the total line usage per transaction be m. Then
.4 Pn
.3 .2 .1
I II I ,
I TI9202I22232425 I
MESS.. SIZE (n)
P
MESSAGE SIZE em) CUllJLATIVE DISTRIBUTION FUNCTl(JNS (b)
Basic Performance Concepts
100 m = mI
Chap. 4
+ m2_
m = iiI + m2 = 22.2 + 35 = 57.2 var(m)
= var(ml) + var(m:z) = 1.56 + 3 = 4.56
Thus, average communication traffic per traDsaction will be 57.2 bytes with a variance of 4.56 or a standaId deviation of 2.14 bytes. Case 2. Both ml and m2 are request messages. m will be ml 30 percent of the time and m2 70 percent of the time. What are the mean and variance of m? m is DOW a choice between messages. Its mean is
ii = .3 x 22.2 + .7 x 35
= 31.16
The second moment of m is found by adding the weighted second moments of ml and m2. The second moment of m2 is the sum of its variance and the square of its mean: m22 = var(m:z)
+ iil· = 3 + 39 =
1228
Then
"r = .3 x 494.4 + .7 x 1228 = 1007.92 The variance of m, then, is var(m)
= m 2 -;;p = 1007.92 -
31.1@ = 36.97
Thus, the Ieqaest messages will average 31.16 bytes in length, with a variance of 36.97 or a standaId deviation of 6.08 bytes.
The previous sec:tioD cIesc:ribed disaete random variables-those tbat take on ODly certain discrete (often integer) values, such as tbe number of items in a queue or tbe IlUIDber of bytes in a message~ But what about the number of seconds in a service? If a process mquhes somewheze betweeIl 2 and 17 msec to process a traDSaCtioD, it can vary MDf.innnusly between these limits. We can say that the probability is .15 that the service time for this process is betweeIl 10 and 11 secoDds, bat this p10bability includes service times of 10~. 10.2, and 10.25679 seccmds. The service time variable is not disc:me in this caSe. It can assume an infiDite IlUIDber of values and is dIaefore called a continuous rarulom variDble. AU of the IUleS we haw; eStabusbed for disc:me variables have a corollary foi
continuous variables, often with 1he summation sign simply replaced with an integral sign. The chaJ:acteristic and rules with which we will be conc:emed in pedmnance analyses are as fonows:
Chap. 4
Concepts in Probability, and Other Tools
101
1. The probability density function is continuous. If%is a random value,/(%) is the probability tbat x will fall within the iDfinitesimal range /(x)dx. More specifically, the probability that x will be between a and b is P(a s x
S
b) = f:/(x)dx
S
1
(4-38)
The probability density function of x is/(;c). Notice that there is no requirement tbat/(x)<1 for all values of x, only that eqnation 4-38 results in a value no greater than 1 over any range. For instance, if x has equal probability of ranging from 0 to .1,/(x) 10 for 0 S x S .1. 2. Since x must have some value, the integral of the probability density function must be one:
=
f!(%)dx = 1
(4-39)
where integration is over the allowed range of x. 3. The average, or mean, value of x is its integral weighted by its probability over its range:
L
x=
,g(x)dx
(4-40)
where the integration is over the allowed range of x. 4. The variance of x is the square of the deviation of x from its mean weighted by its probability and integrated over its range: var(x)
=
i
(x-x)2j(x)dx
(4-41)
where the integration is over the allowable range of x. 5. The mth moment of x is x"', weighted by its probability and integrated over the range of x:
x,. = Lx",/(X)dx
(4-42)
where the integration is over the allowed range ofx. Note that the variance given. in equation 4-41 ean be expanded to .
var(x)
=
L
x2/(x)dx -
L
2xX/(;c)dx
L
+ rt<x)dx
Using equation 4-39, 4-40, and 4-42. var(x)
= X2
-
x2
(4-43)
just as with disaece variables. 6. The properties of ca1c:qJaring means and variaDces for S1DDS of variables or choices of variables me the same as for disaete probabilities. That is, a. If x is ,a sum of continuous random variables %1. :%2, X3. • •• , then its mean is tbe sum of the means, and its var:iaDce is the sum oftbe variances ofXl, Xl. X3
Basic Performance Concepts
102
Chap. 4
b. If x is a choice between continuous random variables Xh X2, X3, ••• with probabilities PI, 1'2, P3, .•. , then its mean is the weighted sum of the means, and its second moment is the weighted sum of the second moments of Xl, X2, X3, ••• , where the weighting factors are the probabilities Ph 1'2, 1'3, .... 7. The cumulative distribution functions for X are
I~X)dx
(4-448)
= I~)dx
(4-44b)
p(X
and P(x>a)
Equation 4-44a is the probability that x is less than the limit a, where the integration is over all allowed values of x less than a. Equation 4-44b is the pr0bability that/(x) is greater than a, where the integration is over all allowed values of X greater than a.
Figure ~3 shows an example of a continuous probability density function and its cumulative distributions. . It is DOt often mperfonnance analysis that we are forced into the calculus of CODtinuous distribution :functions. However, there ale two prominent examples. One is the calculation of the distribution coefficient in the Kbintc:biDe-Pollaczek eqwdion for a c0ntinuous variable. Such an example was given earlier in this chapter (eqwdion 4-17). Another is the following. Consider a medium whose length is b. From one random position, Xl, what is the mean distance to any other random position, X2? 'I'hae are several physical processes conesponding to this case: • The average seek distance of a disk moving from its current track to a new track, 1.0,.--__
~
P(x)a)
/
"- P(x(a)
~.6 ....
iii
I0.4 IE
o
xora
Chap. 4
Concepts in Probability, and Other Tools
103
• The distance between two terminals on the bus of a local area networlc, • The amount of tape that must be passed in a tape search starting at one random point and moving to another.
The solution to this problem is presented here as an example tying many of the above . The probability density function that x will be at any given point on b, j(x), is
concepts together.
f(x)
= 1/b, 0 s x s b
Otherwise, f(x) = 0
Thus, f(x)dx = L"1 -dx = [x]" = 1 Lo"· ob b 0
The average distance between one random point, X10 and another, X2, is the avemge distance XI - X2 given a point, Xl, then averaged over all possible values of XI. I! X2 is greater than XI. then its value can range between XI and b. Therefore, its probability density function is
and its average distance from Xl is
-X2-XI = Lil(X2-XI)f(xi)dx2 . = b-1 .Lil(x2-XI)dx2 =(b-Xl) 2' X2 > %1
XI
.
Xl
%1
I! X2 is less than Xl, then its value can range from 0 to Xl. TheIefore, its probability deDsity function for Ibis case is f(xv
=
;1'
Xl
< XI
and its average distance from Xl is
.
..
The pJObability thatX2 will be ~ tbanXI is (b -x~)Jb, and the probability that it will be less is xI/b. Thus, the average distance between Xl and X2 for a given value of XI
is
Basic Performance Concepts
104
Chap. 4
Since Xl can :range from 0 to b, its probability density function is !(X1)
1
=b
The average distance tbatX2 is from Xl whenXl is varied over its range will be called i and
is - = Lb~
X
o
-
2bxl
2b
+ 2x12 J~(\Xl,"", \..1_ .
2
13
_ 1 [ 2bx1 2x ]b x=2b'2 ~xl--2-+T 0
i= bl3. Thus, the average distance between Xl and X2 is 113 b. This means tbat the average seek distance on a disk is 113 of all tracks. The average distance between two termiDals on a local mea aetworlt is 113 of the bus length. The avenge:random seek distance for a tape is 113 of its length.
Pelflllltations and Combinations It is sometimes useful to be able to calculate the DUIIIber of ways in wbich we can select n objects from a group of m objects. Sometimes the order in which we select them is important, and sometimes it is not. If order is important, we are talking about permuIIItions. Let there be m distinct objects in a group, and we desUe to choose n of them. How many c:ti1fereDt ways can we choose these n objects? On the first choice, we will choose one of m objects. Given n choices, the total number of di:ffeEent ways we can choose n objects is m(m - I)(m - 2) ... (m - n
+ I).
Tbis can be written as
plll= II
m! (m-n)!
(4-45a)
wbme P': is the number of pamutations of m objects taken n at a time. However, if order is DOt important, we haveCOUDted too maay possibilities in the above analysis. We·have C01IDted all of the permutations for each set of choices but are only intaested in COUD1iDg that particular combinatiODof choices once. For iDstance, if one set of choices was ABC, we have counted it as
ABC ACB
BAC BCA
CAB CBA
or six times, whereas we are ODly iDtelestecl in COUIltiDg it once. We are iDterested in the of combiMtions of objects, DOt in all of their pemmtations.
~
Chap. 4
Concepts in Probability, and Other Tools
105
The first item chosen could have occmrec:l during any of the n choices. Given that, the second item could have occurred during any one of the remaining (n - 1) choices and SO on. The same combiDation bas been counted n(n - 1) (n - 2) ... (1) times for a total of n! times. Thus, the total number of combiDations is the number of permutations divided byn!:
cm = n!(m-n)! m! n
(4-45b)
where C': is the number of combinations of m objects taken n at a time.
Series In many of the cases with wbich we will work, we will find ourselves with a summation over an infinite (or at least a very large) number of items. Often, these infinite series can be reduced to a very manaple expression. Some of the more useful ones are summarized here.
a. 1 + x + :x2 + r3 + ... ,0 so x so 1 This can be written in the form
...
1
2X'=1=0 I-x
(4-46)
The similar series, wbich is truncated on the left, directly fonows and is
x" + Xll+l + X"+ 2 + ... ,0 so x so 1 This may be written as
(4-47)
Likewise, this series truncal'tid to the right is 1 + x + :x2 + ... + r l , 0 so x S; 1 wbich may be written as ... ID ID l-xi' 2X' = 2Xi - 2X' = (1-xi')2x i = 1-0 i-O t-II 1-0 I-x
11-1
(4-48)
The doubly ttuncated series is ;xii
+;xII+l + ... +
r
1, O:s x:s 1
This may be written as 11-1.
l-xll-II
11-11-1.
2x'=;xII i-II
2 X'=X i-O
Il
XII-xII =I-x I-x
(4-49)
106
Basic Performance Concepts
b.
+ 2.%2 + 3%3 + ... ,0 S This can be expressed as
%
%~
...
~.
Chap. 4
L %
LiU:' = - i=1 (l-x'j-
(4-50)
This can be expressed as (4-51) .
Conversely,
may be expressed as (4-52)
We DOW ctiscuss the Poisson and exponential distributions in some detail. Not because we will use them in our calculations so often (though simulation studies certainly do) but because they repsesent much of the statistics of queuing tbeoIy and form an important undeJpinDing to our unc:lersranding of the use of the tools we will bring to bear on the aualysis of perfOllDlDCe problems. The Poisson distribution provides the pmbabilities tbat exactly n events may happen in a time iDterval, t, provided tbat the 0CCUDeDCe of these events is independent. That the iDdepeDdeDce of events is the only assumption made is the mason tbat this distribution is so important. Event independeDc:e simply says tbat events oc::cur completely randomly. They do DOt occur in batches. The OCCWleDCe of one event is DOt at all depeDdent on what bas 0CCIIDed in the past, nor bas it any iDftuence on what will oc::curin the fatme. The process has DO memory; it is memoryless. We will can a process tbat creates such rauc:Iom events a . random process. In queuing th.eory, thele are two important cases of a random process: 1. The mival of an item at a queue is a random event and is independent of the . arrival of any 01her item. 'IbelefOIe, mivals to a queue are random. 2. The iDstant at which the ser:viciDg of an item by a server completes is a rauc:Iom event. h is independent of the item being serviced and of any of its past service cycles. TheIefore, service completions by a server are random.
Chap. 4 . Concepts in Probability, and Other Tools
107
Note that randomness has to do with events: the event of an mival to a queue, the event of a service time completion. Let us determine the probability that exactly n random events will occur in time t. We will represent this probability by p,,(I): p,,(t)
= the probability that n random events will occur in time t
(Remember that n is a discrete random variable. Its values are the result of a random· process. These two uses of rtmdom are unrelated. Random variables also are the result of nomandom processes.) Note thatPII(I) is a probability function that depends on an additional parameter, t. As t becomes larger, the probability that n events will occur c:han.ges. This is unlike our simple probability functions described earlier. Such a process is caDed a stochlJstic pr0-
cess. The average rate of the occmrence of events is a known pammeter and is the only one we need to know. We will denote it by r: r
= average event oc:currence rate (events per second)
Thus, on the average, n events will occur in time I. Since events are c:omp1etely random, we know that we can pick a time interval, t, sufficiently small that the probability of two or more events oc:curriDg in that time inrerval can be ignored. We will DOte Ibis ubitrarily small time interval as IJ.t and will assume that the only things that can happen in IJ.t are that DO events wDI occur or that one event will
occur. Let us DOW observe a process for a time t. At the end of tbis observation time, we find that n events have occaaed. We 1ben observe it for IJ.t more time. The probability that ~ furt:be.r event will oc:car in IJ.t is i'1J.t. The probability that no further events will occur in IJ.t is (1 - rlJ.t). Thus. the probability of observing n evems in the time (I + 1J.t) is (4-53)
This equation notes that n events may occur in the iDtaval (I + 1J.t) in one of two ways. EitI:m n evems have occ:unecl in the imerval 1 IIIIIl no events have occ:uned in the subsequent iDterval.lJ.t. or n - 1 eWIlts have oc:cwed in the iDIerval 1 IIIIIl one mare event has OCCWIed in ibe subseCpJem interval, 1J.t. (Note that since the mival of an event is independent of previous mivals, all of these probabilities me indepenIJent and may be c0mbined as shown, acccmtingtoru1es 3 and 4 in 1he earlier sectioD emitled ..Discrete Random Variables. ") . If no events OCCWIecl in the interval t + 1J.t, tbis ~p is written pJ..I
+ 1J.t) =pJ..I)(l -
rAt)
(4-54)
since PII-l does not exist. 'Ibat is, the probability of DO events occming is the pmbability that.JlQ events OC:CWled in At. . the interval t and that DO events 0CCUIIed in the interval .
Basic Performance Concepts
108 Equations 4-53 and 4-54 can be P.(t
+~-
Chap. 4
reaminged as
P.(t) = -rpJ..t)
+ rp,,-l(t)
(4-55)
and pJ..t + ~
-
pJ..t) = -rpJ..t)·
(4-56)
As we let At become smaller and smaller, tbis becomes the classical definition of the derivative of P.(t) with IeSpeCt to t, dp.(t)/dl. Denote the time derivative of pJ..t) by p~(t):
p~(t) = dpJ..t) dl
We can express equations 4-55 and 4-56 as p;At)
= -rpo(t)
(4-57)
p~t)
= -rp.(t) + rp.-l(t)
(4-58)
and
This is a set of differeDtial-c:lifferem:e equations; their solution is shown in Appendix 4 to be
P.
(t)
n = (rtY'en!
(4-59)
This is the Poisson distrlbIItion.1t gives the probability 1hat exactly n events will occur in a time interval t, given onlytbat their mivaIs are random with an avenge rate T. Though the serious student is encouraged to ~ the solution to these equations in Appendix 4, the main lesson to be leamed is the simple and underlying fact 1bat the Poisson disttibution depends only 011 the randomness of event 0CCUDeDCe. All 1bis is s1llDD18lized by sayjDg 1bat the distribution of the number of DDdom events that will occur in a time interval t is given by the Poisson distribution. ID qaeuiDg theory. the nmclom eveDIS of concern are arrivals to queues and completioDs of service. Let us look at some properties of the Poisson distribution. First, the sum of the probabilities over all values of n is
f (rtY'~-n = e-n f (Ttr = e-ne n = 1 n. n.
,,-0
.-0
as would be expected (the infinite series given by equation 4-51 is used). We now derive the mean value of n for the cJistribution:
..
ii =
tn
.-0
(rt)"e- n n!
= rtf (rtY'-l e-n ._l(n-l)!
Chap. 4
Concepts in Probability, and Other Tools
109
and
where i bas been substituted for n - 1 in the summation. Thus, the mean number of events that will occur in a time ~al t is Tt, as we would expect: (4-60)
n=Tt
The second moment of n for the Poisson distribution is derived in a similar man-
ner:
and
nz = ne-n:Lell n (rt,
~-1
n-l
Letting i
=n -
(n-l)!
1,
,;z = ne-nI(i+I) ('"!Y z!
i-o
_ _ ,;z -
Tte
... (TtY] ., Tt~(._I)' + ~ z• •-1 Z •
()i-l -n[ell .!!..-.
.-0
and
Iii = (rtf + Tt
(4-61)
From equation 4-31, the variance of n is var (II)
=Iii - i?
Since tile mean II is rt, 1bea var(lI) = rt
(4-62)
lbns, both the mean and the variance of II is rt for a Poisson distribution. Note the memoryless feature of the Poisson distribution. The probability that any number of events will happen in the time interval t is a fuDction only of the mival rate, T, the number of events, II, and the time interval, t. h is completely independent of what happened in the pevious time intervals. Even if DO event bas oc:c:urred over the past several time intervals, there is DO inaeased assurance that one will occur during the next time interval.
Basic Performance Concepts
110
Chap. 4
The Exponential Distribution The exponentilJl distribution is very much related to the Poisson distribution and can be . derived from it, as will soon be shown. It deals with the probability disUibution of the time between events. Note that the Poisson distribution deals with a discrete variable: the number of events occurrlng in a time interval t. The exponential distribution deals with a continuous variable: the time between event occurrences. To derive the distribution of interevent times, we assume that events are arriving randomly at a rate of T events per second. Let us consider the probability that, given that an event has just occ:urred, one or DlOIe events will occur in the following time interval, t. This is the probability that the time between events is less than t. 1fT is the time to the next event, we can denote this probability as P(7' < t) and can express it as (4-63)
That is, the probability that the next event will occur in a time inrervalless than t is the probability that one event will occur in time tplus the probability that two events will occur in time t, and so 011. Manipulating equatioo 4-63, we have
f (rtf - 1] n.
P(7' < t)
= e- rr [
P(7' < t)
= 1 - e- rr
..-0
and (4-64)
'Ibis is a cumulative distribution for the iDtezevent time t. Its density function, p(t), is the derivative of its cumulative distribution function. That is, from equation 4-44&, P(7' < t)
=
r
p(t)dt
DifIemltiatiDg both sides with respect to t gives oJt)
y,
= C~P(7'< t) = C~(le-rr ) tit tit
wbere C must be choseD such that tbe integral oftbe density functiOn is unity (see equation 4-39). Thus,
Since
Chap. 4
Concepts in Probability, and Other Tools
111
1
Cr-= I" r
and C=1 Thus, the probability density function for the interevent time, t, is
p(t) =
re-rr
(4-65)
We can also expJeSS the altemate cumulative distlibution giving the probability that T is greater than t. From equation 4-44b, we bave
P(T> t)
= fe-rTdT
=( -~e-rr]~ and (4-66)
as would be expected from equation 4-64, since P(T < t) + P(T > t) = 1. Since t is a continuous variable, P(T = t) is zero, and can be ignored. The mean, variance, and second moment of the exponential distribution can be shown to be
= l/r vm(t) = l1r2 i
and t2 =
21r
(4-67) (4-68) (4-69)
Note once again the JDeDlOIYless feature of the expoDeDtiaJ distribution. No matter when we start waitiDg fer an eveat (even if ODe has DOt occaned fer awbile), the expected time to the next eveat is sdJlllr. Also note 1bat t has been ~hele JeJative to the way it is used in the Poisson disIributioD. In the Poisson distnDutioo, t isa fixed iDterval over wbich the probability of occuaeuc:e of II evems is expnssed In the expoNP'iaJ disI:ributioJl, t is the raadom varlable expessiDg the time betweeIl eveats.
To SlIIDJD8I'jze the above, we make tbIee statem.eDts about a nmdom. process with an avenge eve:at rate of r events per second. 1. A TtWlom process is ODe in wbich events ate gene.fated randomly and independently. The probability 1bat an eveat will occur in an l1'bib:alily small time interval III is rill, independent of the eveat history of the process.
112
Basic Performance Concepts
Chap. 4
2. The probability PII(t) that n events will occur in a timeinterva1 t is given by the
Poisson distribution: PII(t)
=
(rt'f
n~
-rr
with n=rt var(n) = rt
and
r = (rt)2 + rt 3. The probability density function p(t) for the interevent time t is the exponential function p(t)
= re-rr
with
i = 1/r var (t)
= 1/r2
and
Thus, random, Poisson, and exponential distributions all imply the same thing: a random process. 'Ibis is a process in wbich events occur randomly, the distribution of their oc:cmren.ces in a given time interval is Poisson-distribut.ed, and the distribution of times between events is expoaeadally distributed. In queuing theaIy, dIae lie two DDCIom processes with which we ftequently deal. One is· tII'7'ivtzb to tl f/IIeW. An arrlval of an item to a queue is a random evem. Arrivals are said to be Poisson-dislribul, and the iuteIanival rate is exponentially distribatecl. The statellkems random tmivtsls and Poisson tl1TivtIJs lie equivalent. The· 01b.er p:ocess is . . IIlnice time t1/ tl server. Manning the server is busy seniciDg items in iIs queue, 1IJe·compktioD·of a service is a nmciomevem. Tbedistribution of service completioDs is PoissoD-disIrib (though we don't normaUy expess this), and service times (Which are the times between completion events in this case) are exponen- . tially disIrlbuted. The statemeJdS TfI1IIlDm·service times and exponential service times are equivalent. Due to the memoryless D8bD'e of random processes, if we begin the observation of a 1'8IIdam. server with average service time t" as it is in the middle·of pmc:essiDg an item, the average time !eQ1lh'ed to complete tbis service is still '., DO matter how loDg the item bad been in service prior to our observation. 'Ibis property was used as an argument conceming the evaluation of the service time distribution coefficient for exponential service times in the derivation of equaDOIIS 4-10 tbEOugh 4-12.
Chap. 4
Infinite Populations
113
CHARACTERIZING QUEUING SYSTEIfS Kendall [13] bas defined a classification scheme for queuing systems that lends order to the various cbaracteristics these can have. A queuing system is categorized as AlBlclKlmIZ
where
A == the arrival distribution of items into the queue, B == the service time distribution of the servers,
c == the number of servers, K == the maximum queue length,
m == the size of the population wbich may enter the queue, and
Z == the type of queue discipline (Older of service of the items in the queue). Several representations for the arrival and service time distributions (A and B) have been suggested, but for our purpose we will deal with four. A or B may be M for a IaDdom (memoryless) distribution,
D for a coastant distribution (such as a fixed service time), G for a geaeral distribution, and
U for a UDifOlmdistribution (this, admittedly, is added to the list by this anthor). lbns,.MfDI3/10l4OlFlFO represeDts a queuiDg system in which anivaIs are random, service time is CODStant, and there are 3 servers serving a queue which can be no longer tban 10 items, serving a populatioa of 40 on afint-come, fint-serve basiS.' . Ifthe maximum queueleDgth is unljnrited (K = co), if the population is iDfiDi1e (m == co), and if the queue disc:ipliDe is FIFO, 1beIl the 1ast dDee terms axe dropped. TbeD, for instance, an MIMIl system is a system in which DDdom aaivals &Ie served by a siDgle server with mndom service times. This is the simplest of an queuing systems. AD MlG/l system is one in wbichraudOm arrivals are serviced by a siDgle server with geaeral service times. This is the case solved by TOUntc:biDe and Pollaczek. INRNfI'E I'OPULA11ONS . ODe of the parameters in ~'s classification scheme is the size of the population m using the queue. This is a particularly impottant pammeter for the followiDg~. If the size of the population is iDfiDite, 1beIl the rate of arrival of user$ to the queue is independem of queue leDgth and therefOle of the load on the system. That is to say, no matter how loDg1he queue, there is still an iDfiDite popuJation of users :from which the next ~ to the queue will ~.
Basic Performance Concepts
114
Chap. 4
However, if the user population is finite, then those waiting in the queue are no longer candidates for entering the queue. As the queue grows, the available population dwindles, and the mival rate falls off. As the load on the system grows, the imposed load decreases. Thus, the load on the system is an inverse function of itself (tbis is sometimes referred to as the graceful degradation of a system). The analysis of queues formed from infinite populations is quite different from. that of queues formed from. finite populations. We will first consider infinite populations, about which a great deal can be said. Some Properties of Infinite PopUItmoIlS Regarding infinite populations, there are some general properties that can be useful. These include
1. Queue input from several sources. If several random sources each feed a c0mmon queue, each with different average arrival rates, Ti, then the total mival distribution to the queue is a Poisson distribution with an mival rate T equal to the sum of the component arrival rates, Ti (Martin [20], 393). (See Flg1JI'e 44a.) 2. Output distribution ofMIMIc queues. Ifone or more identical servers with exp0nential service times service a common queue with Poisson-type mivals, then the outputs from that queue are Poisson-djstrib, with the departure rate equal to the mival rate, i.e., the depauwes have the same distribution as the mivals (Slaty [24], 12-3). (See FIgUte 4-4b.) 3. TTtl1IStICtion stTeom is splil. If a randomly distributed. transaction stream is split into multiple paIbs, the tmusac:tious in each pa!b are random streams with proponioaate arrival lites (IBM [11], 49). (See Figure 4-4c.) 4. TII1IIl8m queues. From. 2, a randomly dimibuted transaction stream passing through 1aDdem compound queues Will emerge as a DDdomly distribu1ecl stteam with the same avenge rate as wIleD it entered the system (IBM [11], SO). (See Figure 44d.) S. 0rtJg ofservice impoI:t Oft response time. The meaD queue time and mean queue leagIh as JRCIic:red by . , KJrintcbjue-Po11acalc: equation is indepeDdeDt of the cm:Ie:r in which the 'queue is saviced, so loug as that order is not dependent upon the service time. This would not be true, for iustance, if irems xequirlDg less service wae seIViced in advance of other items (Martin [19], 423). (See Figum 4-4e.)
Dispersion of Response r .....
We have ah'ead.y disc:ussecl the ueed to be able to make a stat.ement relative to the dispersion of the response time, something in the form "the ~ that respcmse time will be less than two seconds is 99.9%." We will discuss tJ:ne appmaches to this pr0blem.
Chap. 4
Infinite Populations
115
"~~:D" MULn-C!:IANNEL
SERVER
(b)
TANDEM QUEUING SYSTEMS (d)
--..JIi[]--.C!:J-.=--"~ QUEUE ~Pl.JNE
.Gamma distribution. WIthout goiDg iDto great deIail, the Gamma function is the key to tbis stalemeot. It is a more geaeral form -of a probability functioD of which the expoDeDtial ctistribution is a special case (see Martin [19], 437-439). It bas the p:operty tbatthe sam of a set of variables follows a Gamma fmlctioD if each oftbe!8riables follows
a (janmutfanCtion. In TP systemS, a tnmsaction usually passes tbIOugh a series of servers, as we have seen. Tbe respouse time of the system is the sum of the component delay times of each server. These lie often servers with exponeutially distributed service times (at least approximarely). Though the sums oftbese delay times may DOt be exponemial, they will be Gamma-disIribut, and this distribution can be used to detemUne the probability dJat the system respouse time ~ be greater tban a multiple of its mean.
Basic Performance ConceptS
116
Chap. 4
To use this tecbnique, we need to know the mean system response time and the variance of the system response time. The mean system response time is, of comse, the primary focus of the performance model. The variance of this response time is more difficult and often impossible to calculate. However, a reasonable limiting assumption to make is that the response time is random, i.e .• it is exponentially distributed. In this case, the variance is the square of the mean. Real systems will usually have a smaller variance than this, i.e., the response time will not be completely random. The Gamma distribution is used for these pmposes as follows. Fust calculate the Gamma distribution parameter, R, where
T2
R=var(T) .
(4-70)
T is the response time, and T is its mean. Then use the Gamma cumulative distn"buQon function with parameter R to determine the probability that the response time will not exceed a multiple of the mean time. Note that R= 1 for a randomly distributed response time. Real-life response time variations will probably be less random and thus have a greater value of R. Certain values of these probabilities are listed in the following table for values ofR from 1 to 10 (the range in which we would nOJlDllly be inteIested). TABLE ..1. TABLE OF k = TIT R
= f:1vu(T) PJobablJity of zapoase time T ~exceediag
leT
.95
.99 .999 .9999
1
2
3
S
10
3.0 4.7 6.9 8.9
2.4 3.4 4.6 S.7
2.1 2.8 3.8 4.5
1.9 2.4 3.0 3.S
1.6 1.9 2.3 2.6
To take the c:onservative case of R= 1, we can say that 95 percent of an services will finish in less than three mean service times, 99 pen:ent in less than five mean service times, 99.9 peKeIlt in less than seven mean service limes, aad 99.99 perceDt in less than nine mean service times. This is oftea sufficiellt to coaservaDvely vaJidaIe the performance of a system. (These are the valnes tbat wee used in the example in chapter 3.)
CentraI.LilnitTheorem.· ACCOIdiDg to the Centtal Limit TbeoIem, "the ctisttibution of the sum of sevetal random variables approaches the DODDal distdbutioD for a wide class of variables as the number of random variables in the sum beComes large. " The p!eCise test of how closely a system will approach a normal ctistribution is quite complex. HoWever, the tbeoiem has been shown to hold well in typical queuing aDalysis problems. To use this theorem, the first step is to calculate the mean and variance of the ,;:esultiDg response time by adding the delay times for each of the components:
Chap. 4
Infinite Populations
117
-T= TI- + T2 - + ... and var(T)
= var(Tl) + var(T~ + ...
Then, for a given probability, as given in Table 4-2, simply add the standard deviation, i.e., the square root of the variance, weighted by the factor p to the mean to obtain the maximum value of response time below which actual response times will fall with the
given probability. For instance, if mean response time is 4 seconds, and if its standard deviation is found to be 3 seconds, then with 99.9 percent probability, the response time will be less than 4 + 3.09 x 3 = 13.27 seconds. TABLE 4-Z. NORMAL DlSTRIBunON p
.90 .95 .99 .999 .9999
1.28 1.65 2.33 3.09 3.71
For random distributions in which the standard devWion is equal to the mean, Table 4-2 indicates that the maximum respoase time for a given probability is (p + 1) times the mean response time. Comparing Tables 4-1 and 4-2, the normal dislribulion tedmiqne equates approximately to R = 2 to 3 when random disttibutioDs are assumed.
Variance of responie times. For a queuing system in which inputs are 1311dom with arrival rate, R, and in wbich the distribution of the service time, T, is arbitrary, i.e., the KhintcJrine-PoJlaczek MIO/l case,·the variance of the delay time, Td, is given by*
or
_
_2
KI'3
RlT2 -var(T~ = 3(1-L) + 4(1-L'f + T2 - T2
(4-71)
where
T is the mean of the service time, T (Note: elsewhere, this is noted simply as T.) *'Ibis m1aIiaD ay be foaad ill Madin [19], Madin (20], aad IBM [11], each of wbich difl'ers from Ihe oIhe:ls 8JI!I comaiDs miDor emxs.
Basic Performance Concepts
118
Chap. 4
T2 is its second moment.. T3 is its third moment R is the arrival rate to the queue L is the load on (occupancy of) the server This is solved for the three following cases of interest: 1. Exponential service time. In this case,
T2 =
if2
and
T3 = 61'3 Substituting these expressions into equation 4-71 yields the delay time variance for a server with exponentially distributed service time:
1'2
var(T,u = (l-L~
(4-72)
Note that this is the square of the mean service time as to be expected (see equation 4-11). 2. Uniform service time. If the service time may fall with equal probability between two limits (disk seeking is close'to
this), then
T2
= ~T2 3
T3
= if3
and (4-73) 3. Constant service time. If the service time is ccmstant (such as a polled communication line with a fixed-leogtb message), then
T2
= f'2
T3 = 1'3 and (4-74)
A !e8SOD8bility c:beck can be made on these variances by letting the load. L. approach zero. The delay time variance should approach the variance -of the service time.
Chap. 4
Infinite Populations
119
The results of this exercise are var(T,j) ......
T2
for exponential service for unifOIDl service for constant service
var(T,j) ...... T213 var(T,j) ...... O
All of these are as to be expected, using var (T) = T2 - T2 in each case. Using the fact that the variance of a sum of random variables is the sum of the variances (equation 4-33), one can calculate the variance of the delay times, i.e., the variance of the respoase times, of a tandem. queue in which a ttaDsaction flows through a series of servers. For iDstaDce, assume a traDsactiOD is processed by a communication line with constant service time, then by an application process with exponential service time, then by a disk with mDform service time, and finally by a comnumication line with constant service time. This situation is Jeflecteci in the following table with some sample values for service times and server loads. Service time variances are calculated acconting to the previous expressions. TABLEW- EXAMPLE TANDEM QUEUE Step
CoDl",'Oicatioos
P10cess Disk
Service lime (t) diSUibadoa
Mean
Server
ofT
load
CcmsIaDt
.2 .3
.2 .4
.004 .2SO
.4 .1
.6 .2
.373 .001
ExpoNmial
UIIifoaD
Com"''''Iicatioas
CcmsIaDt
r.o
VariaDce ofT" •
:a
We see that the tandem. queue provides us with a mean respoase time of one second and a variance of .628, i.e., a sumdard deviation of .792 sec:onds. If we wished to use the Gamma disIribution to de1mDiDe the pmbability of a long IeSpODSe, we would calculate R as R = T2/var (T)
= 12/.628 = 1.6
JmapolaDng Table 4-1 for a 99.9 paaadle, we find a value for k of 5.5. Multiplying the second mean service time by tbat Il1IIDber allows us to state that 99.9 percent of transactions will be completed in less 1ban 5.5 seconds. As an aJ.temative, we could use the Ceatral LiDiit'lbecnm. At the 99.9% pezamtile, we see that we shoald move 3.09 staDdard ckMatiODs out from die meaD. TbDs, we can make the statemeat that 99.9 percem of all transactions will ~ in less 1ban (1 + 3.09 x .786) 3.4 seconds. The Gamma distribution gave us a more conservative JeSuIt (S.S seconds) than the Centtal Limit 1beorem. In general, the more c:oase.rvative JeSUlt should be used. This will be given by the Qamma function for large, normaUmi S1aDdard deviations, i.e., the Iatio of the standard deviation to its mean, and by the Centtal Limit TheoIem for small, nonnalized standard deviations (typically, less than .6).
an
ODe
=
Basic Performance Concepts
120
Chap. 4
Properties of ././1 Queues Queue lengths. Queues formed by random mivals at a server with random service times (an MIMII system) are the easiest to analyze. For an MIMII system, the probability that a queue will be a particular length can be derived tbrougb what is called a birtb.-death process. We used the birth part of this to derive the Poisson distribution. We consider an MIMII queuing ~ i.e., a single server with random mivals and random service times, in which the average arrival rate is r and the average service rate is s. The probability that the queue will bave length II is p,.. where the queue includes all items waiting in line plus the item being serviced. If we consider a very short time interval, At, then the probability that an item will mive at the queue is rAt; this is a birth. Likewise, the probability that an item will leave the queue (US'uning there is a queue) is sAt; this is a death. We observe the queue at some point in time and note with probability P,,-h P", or P,,+ 1 that there are 11-1, 11, or 11+ 1 items, IeSpeClively, in the queue. Ifwe come back at a time that is At later, we will find 11 items in the queue under the fonowing conditions:
I. If there bad been 11 items on the first observaI:ion and if there bad been DO mivals or departmes in the subsequent interval, At. Since the probability of no mival is (I-rAt), and since the probability of DO departure is (I-sAt), then this will occur with a probability P" (l-rAt)(I-sAt). 2. If there bad been 11 items on the first observaI:ion and if there bad been one mival and one depai:tme in the time interval At. This will occur with probability p,,(rAtXsAt). 3. If there bad been 11-1 items on the fimt observation and if there bad been one mival dming the interval At, with DO departmes. This will occur with probability p_lrAt(1-sAt). 4. If there bad been 11+ 1 items on the first observation and if there bad been one depauwe during the interval At, with DO mivals. 'Ibis OCCUIS with probability pII+ 1sAt(l-rAt).
IgD.oriDg terms with At2, since these will disappear as At goes to zero, we have p"
=p,/..I -
sAt -rAt) +P_trAt + P,,+lSAt
(4-75)
AmmmJating P" terms, this beooines (s + r)p"
= rp,,-l + SPII+I .
(4-76)
The load on the system, L, is L= rls
Thus, equation 4-76. can be IeWriUeD as P,,+1 = (I
+ L) p" - !P_l
(4-77)
Chap. 4
Infinite Populations
121
For n=O, there is DO p,,-J, and there can be no departure if the initial value of n is zero. Thus, equation 4-75 can be manipulated for the case of n=O to give (4-78)
PI =Lpo
Using equations 4-77 and 4-78 iteratively, we find
P2 = LZpo
P3 = L3po and
p" = L"po Since L is the load on the server, it represents the probability tbat the server is occupied. Thus, the probability that the server is unoccupied is I-L. This is the probability tbat there are DO items in the system (no queue): (4-79)
Po= I-L
The probability of the queue length being n is P"
= L,,(I-L)
(4-80)
We can per.form some checks on this result as follows. FIl'St, the sum of these probabilities should be unity: GO
Lp"
,,-0
III
-=1 L
III
= ,,-0 LL"(I-L) = (l-L)LL" = =1 _0 I-L
Next, we can calculate the average queue leagdl, Q: GO
GO
Q = LnL"(I-L) = (l-L)LnL"
,,-1
,,-0
Using equation 4-50, this becomes Q
L
L
= (I-L) (I-L)2 = l-L
(4-81)
This is just what TCbjntr;hVte-Pol1aczek prec1ic:ted (see equation 4-10). The other msults can be simjJarIy verified. Finally, the variance of the queue length is given by
var (n)
=Q + ~ = h (I-L,
(4-82)
(The derivation of this is complex; see Saaty [24], 40.) The probability tbat a queue will exceed n items, P(q > n), is III
P(Q>n) =
~ Ln(l-L) = (I-L)
,,-,,+1
III
Lril
L Ln = (l-L)n"'ril I-L
Basic Performance Concepts
122
Chap. 4
from equation 4-47. Thus, p(Q > n) = L',+l
(4-83)
Summarizing what we have just deduced about the properties of MIMII queUes, we
have Probability of queue length being n: PII
= L II(I-L)
(4-84)
Average queue lengtb: L Q= l-L
(4-85)
Variance of queue length: L
(4-86)
var(Q) = (I-L)2
Probability of queue length exceeding n: P(Q
> n) = LII+I,
(4-87)
Also, from equations 4-20 tbrough 4-22: (4-88) (4-12)
and ([l' 1 Td=-=-T L l-L
As noted by the equatioIlllUlDbers. die expressi
(4-11) lie
the same as those
We bave akeady derived the properties of queues foaDed by random mivals to a server with a known, though DOt neces$8rily IIDdom, service time ctistribuIion. These are the Kbintchine-Po~ equatiOns. which are repeated hete for convenience. kL2 W = I:-L
(4-4)
L Q=(1- (l-k)L] l-L .
(4-6)
Chap; 4
123
Infinite Populations
Td
= 1~L [I -
Tq =
(4-8)
(l-k)L]T
kL I=rT
(4-9)
k = !E(T2) 2 T2
(4-16)
Single-Channel Senter with Priorities In many of our systems, the server is serving a queue organized by priorities. When the server becomes flee, it next services the item tbat has the bighest priority and has waited the longest in its priority class. There are two t;Vpes of priority service disciplines of interest to us. One is nonpreemptive servicing. in which the service of an item is completed before the service of another item is started, even though a higher priority item may bave mived after servicing started. On the other hand, a preemptive service discipline mquin:s that the servicing of an item be suspended if a bigher priority item mives. When the higher priority item has been serviced, servicing of the original item resumes where it left off. The impact of priority service can be deduced intoitively by noting tbat, so far as an item being serviced is concemed, the capacity of a server is Ieduc:ed by the time wbich it must spend servicing higher prioritY items. Let Lir be the load imposed on the server by items ofbigherpriority than the one we are CODSidering. The amount of time left to service an item at the considen:d priority is (l-Ln) of the total time. The average amount of time required to service an item at priority p. rp, is
rp = Tp +LnTp Here Tp is the time to service an item at priority P if thae wae no higher priority interference. This equation states tbat Tp of the server's time is actDally spent servicing the item. However, during tho total time. rp, tbat the item is being serviced. the server spends Lir of tbat time tending to higher priority duties. 'I1ms, the effective time to service the item is
.
r:=-1L p l-L" In effect, the service time at priority P bas been JengtbeDed by the factor l/(l-Ln). The world has slowed down at.priority p. We fOUDd earlier (equatioD 4-9)tbat the time an item must wait in a queue for a single priority server cauying a load L isTq .= kLTl(1-L). However, if the server must also process a higher priority load of we DOW know tbat the time tbat an item will wait in the queue at priority p, Tqp. is
L".
kLT
Tqp
= (I-L)(I-L,,)
(4-89)
Basic Performance Concepts
124
Chap. 4
Note that the term kLT is, in fact, the amount of service time left for the item currently being serviced when a new item mives at the queue. Let us call this term To. Then
T. qp
To = (l-L)(l-L,,)
(4-90)
where TlIP
To
= average queue wait time at priority p. = average service time rema;,,;ng for the item being serviced when a new item arrives at the queue.
L
= load imposed on the server by items at priority p and bigher.
Lit
= load imposed on the server by items at priorities greater than p.
If we number our priorities from 1 to 1'-, with the convention that items with bigher priority IlUIIlbem have pIecedence over items with lower priority numbers, the above defiDitions for L and Lit can be expIessed as fonows: II-. L = LL,
L, = load imposed on the server by items at priority i. Equation 4-90 is quite geDeral and is, in fact, applicable to most server systemS. This result is rigorously derived by Saaty [24] and KleiDrock [15] (see Appendix 7). Let us DOW apply 1his quite general J:eSUlt to DODpEeemptive and pteemplive serveII.
Nonpreemptive...".. For a ~ve server, the average service time mnaiDing for the item beiDg serviced when a new item enters the queue is 1he average of such times over an priorities:
That is, the mnaiDing service time at priority i is kiI'" and a prlorlty i item will be in 1he server L, of the time. LetL, be the total load on .the server and T, be the service time aver8ged over an priorities. Then the probability that an item beblg serviced is at priOrity i is LlL,. .Ifk, is iDdepe.Ddent of priority, i.e., the nalUIe ofthe service is the same xegm1less of priority, then we can assign k, = k and mwrite To as ,
II-.L, To = kL,L r T/ '-=141t
Chap. 4
Infinite Populations
125
or T()
= kL;I,
Thus, kL,T, qp - (l-L)(l-Ln)
1'. -
(4-91a)
where
L, = total load imposed on the server by items at all priorities. T, = service time averaged over all priorities.
Once an item is given to the server, its service is DOt preempted. Therefore, from equation 4-22, (4-91b) where Tdp
= delay time through the server (queue wait time plus service time) at priority p.
Tp = service time at priority p.
Preemptive server.
For peemptive service, the activity at lower priorities is
traDspmeDt to an item, since the service of lower priority items is jmnwtiately suspended
and not resumed so loDg as a higher priority item is in the system. 1'helefore, using an argument similar to tbat used for nonp.reemptive serven,
or
T = service time averaged over all priorities at priority P and higber. item is given the server, it will :ras,Wheal antime at priority is service
to P
be inteaupted by higher priority activity. Tpl(l-Ln). and
kLT
T. = (l-L)(l-Ln)
(4-928)
1'.dp-1'..+ ~ I-Lit
(4-92b)
Basic Performance Concepts
126
Chap. 4
Multiple-Channel Server (MIMIc) A multiple channel queuing system comprises c cbaonels serving a single queue into wbich items are arriving at a rate R. As soon as a server channel finishes processing an item, it starts servicing the next item at the head of the queue. Each server has an average service time, T. The distribution of the queue lengths, PII, is as follows, where n is the total number of items in the system, including those being serviced (Saaty [24], 116):
PII
=pJ.eL)"ln! , I S n S
c
(4-93)
and (4-94)
. Po is calculated from LPII 11-0
= 1: c-l
p;l =
L (eL)"ln! + (eL)"Ic!(l-L)
(4-95)
11-0
In the above expressions, L is the average load on each server. The total system load is eL=KI
(4-96)
The avenge number of items waiting in the queue for service, W, is equal to the average number of items in the system in excess of the number of servem, c. From
equation 4-94:
W=
..
.
L (n-c)PII = 11-,,+1 L (n-c)p~)"e"lc! II-c+l
Oumging the summation index to % = n-c. we have W = Poe" ~%(L)Z+"
c!
z-1 .
=p.ceL'f ~%(L)Z C! z-l
From equation 4-50, W may be eJqRSSed as W- L(eL'f - c!(l-L'f Po
(4-97)
The avenge number of items in 1he system, Q. inc1ncting those being servic:ed, is, from equation 4-20,
Q = W + eL
(4-98)
The average waiting time in the queue. Til' is obtained from equation 4-21:
Til =
WT
(eL)"
CL = c(c!)(I-L'jl PoT_
(4-99)
Chap. 4
Infinite Populations
127
The average delay time through the system, i.e., queue time plus service time, is, from equation 4-22,
Ttl = Tq + T
(4-100)
The above equations apply for exponential service times. However, Martin points out (Martin [20], 461) that simulation studies have shown that the waiting line size, W, and waiting time, Tq , do vary in about the same way as does the Khintchine-Pollaczek distribution c:oefticient k, just as do single server queues. Thus, for general service time distributions,
kL(cL)C
W == c!(I-L)2 Po
(4-101)
Q ==W+cL
(4-102)
k(cL"f
Tq == c(c!)(I-L)'1. PoT
(4-103)
+ T.
(4-104)
and
Ttl == Tq
Note for c=l, equations"4-9S, 4-97, and 4-99 reduce to
Po
1
= 1 + U(l-L) = l-L
and
L2 W= l-L
LT Tq = l-L which are the siD.gle-server KbintdJine-PoDaczek equaDons 4-4 and 4-9 for k= 1. Q and Ttl also !educe to equaUcms 4-6 and 4-8 for k= 1.
Equation 4-90 is "quite general and applies to mu1tip1e-ch anneJ. servas as wen as to singlechannel servas (see Saaty [24],p. 234). Equation 4-90 S1aI:es tbat Tq = To/(l-L) for a single priority &elVer. Thus, from equation 4-99 for a multiple cbannel server,
_ (eLY To - c(c!)(I-L) PoT
(4-105)
Nonpreemptive server. For nonpIeemptive service, To is averaged over all priorities:
Basic Performance Concepts
128 _
(cL,)C
Tqp - c(c!)(l-Lr)(l-L)(l-L,JpJr Ttip
= Tqp + Tp
p;l
=I
Chap. 4 (4-106a)
(4-106b)
c-l
(cL,rln!
+ (cL,)CIc!(l-Lr)
(4-106c)
11-0
Preemptive server. For preemptive service, lower priority service is transparent, and To is therefore averaged over all priorities from the CODSidered priority and higher: (cL)C
Tqp
= c(c!)(1-L)2(1-Ln) PoT
T.tip -- T.qp+ .Ie.I-Liz
(4-107a) (4-107b)
c-l
p;l
=I
(cL)"ln!
+ (cL)CIc!(I-L)
(4-107c)
11-0
RNITE POPULATIONS As discussed pzeviously, queues formed from finite populations bave the c:batacterlstic of graceft4 degrodlztion: as the load on the system inaeases, the arrival Iate decreases because of a Jeducecl active population. In geneml, the population should be considered fiaite unless it is much larger than the expected queue 1eDgtbs. Common appJicatiODS in TP systemS include the following: • Tmninals on a multidtopped conummicaMD line which CODteDd for that line, • Multiple servers (in the teqUeStOr-server sease) accessing a data base, • Processors in a mu1tip:oc:esso.r system CODteading for main 1DImlOIY. The following is based on IBM (11), wi1h some conectiODS aDd much enhancing. In . geaenl, we t:biak of a user as doing some wed befoIe euraing the queue. This time is callecl the availability time, T•• and is assumed to be expouenrially distributed It is the time that a tmnjnaJ is used prior to biddiDg for dle line (often called '"tbiDk time," since it lep.tesents the time that the user is drinking befo.te entering the nextuausaction) or the time that a daIa-base JD8D88"'" speDds ~ a zequest befcn geuiDg in line for the disk or the time speDt actively pmcessiDg by a processor befcxe IeqUestiDg its next common
memory access. Once in line, the user must wait a time, T", prior to being serviced and then an average expouenriaUy-distri.buted service time of T. Thus, on the avenge, each user will cycle tbIOugh dle system every (T. + Tq + T) seconds. The user's availability time. T., ~ indepeDdeDt of the system, and Tis uaaffected by system load. However. as the system
Chap. 4
Rnite Populations
129
becomes loaded, the waiting time, Tq , ~, thus slowing down the mival rates of the users. This is the graceful degradation effect. Let us define a service ratio, z, as the ratio of aVailability time, Ta, to the average service time, T:
(4-108)
z=TJT
If the user is almost always available, i.e., not in the queue, the user's service ratio may be aIbitrarily large. If the user is almost always in the queue, the service ratio may approach zero. Since each user arrives at the queue every (Ta + Tq + T) seconds on the average, and since there are m users, then the arrival rate, R, of users to the queue is
R=
m
Ta+Tq+T
=
mIT z+TqIT+l
(4-109)
and the load, L, on the system is
m
L =RT= z+TqIT+l
(4-110)
As an aside, equation 4-109 can be solved for Td = Tq + T as
m
Td= R - Ttl
This is known as the Response Time law (Lazowska [16]) and Ielates the system EeSpODSe time, Td , to the individual interarrival time, mfR, and the think time, Ta. That is, the response time is the individual inteImival time minus the availability time-an intuitively obvious xe1ationsbip. From equation 4-21, Tq =
WT WT T = iii (z+TqIT+l)
(4-111)
Solving equation 4-111 for Tq gives T. = W(z+l) T
m-W
q
From equations 4-21 and 4-112,
system
L
(4-112)
load, L, can be expressed as
=!!!..::..! z+1
(4-113)
Solving for W,
W= m -(z + 1)L
(4-114)
Q=W+L=m-zL
(4-115)
From equations 4-20 and 4-114,
130
Basic Performance Concepts
Chap. 4
+T
(4-116)
and from equation 4-22, Td = Tq
Equations 4-112 through 4-116 express the queuing relationships for a finite population and for any number of servers as a function of the service ratio, z. Note that these relationships apply to both single-server and multiple-server queuing systems by substituting cL for L, wbeIe c is the number of servers, and L is the load on each server. The only assumption is that the availability time, T". and the service time, T. are exponentially distributed. However, L and Ware functions of each other, and their solution depends on the number of servers. The evaluation of these terms is discussed in the next sections. One other general relationship of interest is the probability that a user is busy, i.e., is in the queue or is being serviced. Since users mive in the queue at a combined rate of R, each user will arrive once every mlR seconds, on the average, and will spend an average time of Td seconds in the system. Thus, the probability that a single user is busy waiting for the server or being serviced is P(busy)
L = RTd -m = -mT (Tq + 7) = 1 -
zL -m
(4-117)
using equations 4-112 and 4-114 for the simplification. These :relationships will be used to study the single-server and multiple-server
cases. Single Serrtel' Queues (M/II/f/IIJ/"'J
For a sing1e-server queue with random service times serving a finite population of size m with random availability times, the probability tbat n items will be in the system (iDcJudjng the one being serviced) can be shown to be the following (IBM [11]):
rn- II PII =
(m-n)! 1ft
(4-118)
•
~; i-oj·
The server utilization (or load), L, is the probability that the server is busy, i.e., the queue length is nonze:ro: Zm-II 1ft
L =
1ft
~PII= ~
11-1
(m-n)! 1ft
J
11-1 ~!-
(4-119)
i-oj! As a sanity check, the case for z=O represents full loading. The significant terms in equation 4-119 as z approaches zero occur for n=m andj=O. In this case, L becomes 1 as would be expected. For very large values of z, the significant tenDS are for n= 1 andj=m. In this case, L ~ mlz. For an unloaded system (z infinite), L becomes zero as expected.
Chap. 4
Finite Populations
131
Equations 4-114, 4-115, 4-112, and 4-116, respectively, may be used expression for L to calculate W, Q, Tq , and Td.
with this
Multiple-Server Queues (M"'/c/m/m) The finite population system bas been solved for the finite-population, multiple-chaDnel case. This is probably the most general solution of practical usefulness that exists as of the time of this writing. As with the single-chaDnel case, it is assumed that both service time, T, and availability time, Til, are exponentially distributed. A typical (and very important) example of this type of queuing system in TP systems is the case in which multiple application programs access a data base via multiple copies of a data-base manager. In this case, the application programs are the users, and the database managers are the servers. Again, the service Iatio, Z, is defined as Z
= T,/T
Let C be the number of servers and m the number of users. Then the probability, Pro, of there being n users in the queue can be shown to be the following (IBM [11], 45-46):
Pn
Pro -_
= (~) z~ Po, 1 :S n :S c n!
I...IJ-C
Ca.
Where( ~ )is the binomial coefficient:
(m)
1 <-< n IZ i Po, c - .n - m
m! ( m) n = n!(m-n)!
(4-120)
(4-121)
(4-122)
Po is such that it satisfies 1ft
Po= 1- ~P.
.-1
(4-123)
W is the average number of users in the system exceeding the nnmber of serven, c.
Thus,
(4-124) Knowing Was a fuDction of z (the only variable in p,.) allows us to evaluate Q, Tq , and T41 from-equations 4-113 (which gives the loadL) and from equations 4-115,4-112, and 4-116, respeaively. A diffe.rent and somewbat more complex solution to this problem is given by Saa1y [241. 3~327.
Basic Performance Concepts
132
Chap. 4
Computational Considerations for Finite Populations The expressions for finite populations do not generally lend themselves to manual calculation. This is because they must be solved iteratively for many cases. This can be seen by the following reasoDing. The length of the waiting line, W, is a function of the service ratio, z. z is a function of the availability time, 141 • In many of our analyses, the average arrival I3te at the queue is fixed as R, where Rim is the transaction I3te per user. The availability time, 141, is then la = mlR - lq - 1 (see equation 4-109); Thus, la is a function of 1 9 , which is a function of W. Thus, W is a complex function of itself. Consequently, these expressions are best evaluated iteratively by computer. Typically, a choice for z will be made, and W will be calculated. Using W, 19 can be c:alculated from equation 4-112 and then la from equation 4-109. z Can DOW be calculated, and iftbis value does not equal the starting value for z, a new value for z is chosen. This process continues until it converges on a common value for z. Of course, if the model being evaluated bas a determined average availability time, 141 , then these expressions can be calculated manually.
co.PARlSON OF QUEUE TYPES ~
we have seen, the delay time of a queuing system is sensitive to several pmawete.rs, notable of which are the following:
most
• Load on the server(s). • Distribution of 'service times.
• Number of servers. • Size of population serviced by the system.
We generally assume that arrivals are Poisson-distrib. h is useful to obtain a graphic feel for the effect of these parameters. We first c:onsiderthe ctisIribution of service times. Figme 4-S shows the JlOJ'fD8Iimi IeSpODSe time, Tp, of single-server queues with a variety of response-time distributions: • ExpoiaentiaI (MIMJl). • Unifmm (M/U/l). • Constant (MID/I). As was ~y discussed, a server with constant service time perfOllDS beUer tban one with a uniformly distribu!ecl service time and even IDOIe so relative to a server with an exponentially distributed service time. Though the server performance curves of F1g1I!e 4-S appear to be close, this can be misJeading. It is true that cIifferenc:es at low
Chap. 4
Comparison of Queue Types
133
.4
.8
.6
1.0
loads may DOt be significant. However, consider heavily loaded serveIS, as shown in the fonowing table.
TABLE 44. NORMALIZED DElAYllME (Tdm Server load .7
.8
.9
3.3 2.6 2.2
5.0 3.7 3.0
10.0 7.0
5.s
134
Basic Performance Concepts
Chap. 4
At 90 percent server load, the response time, Td , for a server with exponential service time is nearly twice that of a server with constant service time. Another practical consideration for a queuing system can also be noted with reference to FJ.gUIe 4-5. Consider an MIMII system that is SO percent loaded. Its nonnaJized delay time, TP, is 1/(1 - L) = 2. If the load imposed on this server blaeases by 10 percent to .55, then its noma)jzed delay time becomes 2.22, or an 11 pexcent increase. At an 80 pexcent load, the normaJized delay time is 5. A 10 pexcent load increase to 88 percent causes the delay time to increase to 8.33-a 67 percent increase. At 90 percent load, a 10 percent load increase to 99 pezcent causes a normalized delay time increase of 1000 pezcent! This effect is called amplification. As the load on the system is increased, small changes in load cause ever greater amplification of delay-time changes. The range of response times as seen by a user fluctuates over a wider range, usually increasing levels of ~on. . For this reason, a common rule of thumb is to keep resource loading to a level less than 60 to 70 percent. At this level of loading, a system will be reasonably well-behaved in the presence of small load variations. The comparison of systems with multiple servers or with finite populations or with both is more complex. However, a feel for the impact oftbese parameters can be obtained by studying FIgUIe 4-6. This figure shows DOJD1aJjzed response time, TP, as a timction of individual server load, L, for four cases: 1. The simple MIMII case of a single server serving an infinite population of users. 2. 'I'1ne servers serving an infinite population (MIMI3). 3. A single server serving a population of ODly 10 users (MlMllII0!10). 4. Three servers serving a population of 10 users (MIMI3/10!10).
Note the following characteristics: • BaviDg n servers serving a CO""'W)D queue ofusers (MIMI.3 in Figure 4-6) is more efficient than baviDg n servers each serving. lin oftbe users (each being aD MIMIl server in Figure 4-6). We would:rather Wait in a oo""'iOIl tine for sevenl baDk tellers tban have to pick a teller and thea wait in a IiDe dedicated to that tellec. • Respouse time improves for a queuing system as the popn1arim served by that system becomes smaller. This is because queues C8IIDOt grow as large or as quickly. • The IeSpODSe time characteriStic of finite populations approximates that for infinite popuIaIioDs when the population is much greater than the avenge queue leDgdt would be for infinite popuJaticms. ie., for small loads. In Figure 4-6, a queue length of 1 occurs in the MIMII system for a load, L, of 0.5. At tbis point, the delay time for 1he finite population case MIMII/10/10 is within 10 percent of the iDfiDite population case MIMII. Though DOt shown, a queue length of 1
Chap. 4
Comparison of Queue Types
135
7 I
I
6
5
4
-T MIMI! M/M/3110110
I I I I
~
".
~QUEUE LENGTH
.,. .,
.,.
o
.2
FOR M/MII
.6
.4
.8
1.0
L Iipre U
Effect of IIIIlIipIe se:rvas ad iDiIe popuIaIioas.
occurs for the MIMI3 system at a load, L, of .67~ Again, the respcmse times tor the MIMI3 and MIMI3IIOIIOSysteWs _ wi1biD 10 pen:eat at this point. In both cases, so loag as the fiDite:'pOpUJaDon is an ·Older of m8gmtnde greater than the queue length, (10 > > I), the iDfiDite populaDon sohition is a reasoaably accm:ate appJOxjymltiOll.
An iDterestiDg rule of thumb follows from tbis observation and from the 60 to 70 percent load mle suggested earlier. At 2J31oad (66.7 pacem), the avenge queue length is U(l - L) = 2. Since in most cases we do DOt want to exc:eed this load, we will DOt DODDally expect avenge queue leDgtbs to be greater than 2. 1'herefOIe, a population of 20 will gea.enllysuffice to qualify as an iDfiDite popuJation. For popuJatiOD sizes less 1ban 20. or for those cases where loads will exceed 67 pen:ent, it may be advisable to CODsider the system as ODe with a fiDite
poPUlation.
Basic Performance Concepts
136
Chap. 4
Figure 4-6 fails to answer one other imponant question. Given the need for a specific capacity, is it better to use a single higb-capacity server or several lower capacity servers operating from the same queue? Let us say that we decide that a resource must have an ultimate capacity to service 10 items per second. Figure 4-7 shows two solutions to this need: • A JDOIe powerful siDgle server with a service time of 0.1 second (MIMIl). • ThIee less powerful servers operating in puallel serving the queue of items, each of these having a service time of 0.3 seconds (MIMI3).
In either case, a maximum of 10 items per second can be serviced. As shown by Figure 4-7, the single-server system is decidely better at all loads, since its service time is much smaller, resulting in shorter delay times. If JDOIe servers were used in the multiserver case, then each would only get slower and aggravate the ~.
.
Thus, from Figures 4-6 and 4-7, we can make the following observations!elative to the applicability of sing1e-server and multiserver systems: • If the choice to be made is the organization of n like servers, then it is better to feed them from a common queue rather than from individual queues (FIgUIe 46). • If the choice to be made is between one high-speed server and many lower speed. servers, choose the high-speed server (Figure 4-7). Typical examples of these situations follow.
• A replicated set of server processes should be driven from a commcm queue. • A multipoc:essor system with n processors sbarlDg a common memory and serving a commOD task queue will. outperfOIDl a multicomputer system. with n c0mputers if an puesson have the same power. • A single higJl-speed computer will omperfOIDl a umltipiocessor system or a multicomputer system with the same cumulative capacity. • A siDg1e higJl-speed disk UDit will ouqcfoJmmukiple loweI: speed disk UDits with the same c::ombiDed capacity.
SUllIlARY The queuing models described in dDs c:bapter lie summarized in Appendix 2, with n0tational symbology being summarized in Appendix 1. The queuing expiessions are grouped according to their KeadaIl classifiadions, where the author bas defined certain classificati9D types for the terms AlBlclklmlz to meet the needs of the ~ preseuted.
Chap. 4
Summary
137
2.1
1.8
1.5
1.2
Td (sec)
0.9
0.6
0.51----------;;
o
.2
.4
.6
.8
1.0
For miva! and service time distributioris ·WB). tbe following classes bave beeR presented:
D --c:cmsIaDt G-general
For tbe D.UIDber of servers (c). we have l-sUlgle server t;-fiDite servers
Basic Performance Concepts
138
Chap. 4
=
For the maximum queue length. k, we have always assumed an iDfiDite queue (k =). For finite queues, most models provide the probability distribution of queue lengths so that the probability of queue overflow can be considered. For m, the number of potential users in the system, we have m-a finite population of users =-an infinite population of users Finally, for type of queue discipline, we have A -any FIFO-first-in, first-out pp -preemptive priority NP --liOIlpreeDlptve priority
If any of the last three classificaDon terms Iklmlz are left off a queue system classification, it implies an infinite queue, infinite population, and a first-in, first-out service (Iklmlz = 1=1=1FfPO). Also note that if a finite population of size m is considered, the maximum queue length is also m. The queue systems we have studied include MlGllI=I=/A (Tbe Kbintcbjne-Pollaczek case) M1M1lI=I=/A MIU/I/=I=IA MlDllI=I=!A (These three are derivatives of MlGlI) MlGlI (for delay time variance) MIMII (for queue cIistdbuDoDs) MlG/I/=I=!PP (peemptive priorities) MIGII/=J=~ (nonpreemptive priorities)
MIMIc (mu1tjcha rmel server) MlGlc (an approximation) . M!MJcI=/=!PP (multiserver with p:eemp1ive priorities) MlMlcI=/=!NP (mulriserver with ncmpreemptive priorities) MIMII/mlm (siDg1e-chaDnellimited population) M!MJclm/m (muJticbannellimited population) .. Though many adler cases have been stDdied in the litetatwe, these are the ones that have useful solutions and which are of most intaest to us. The remainder of the book deals with the application of these concepts to the performance analysis of ttaDsadion ~ systems. The intent is to show how to use these tools to c::reate solutions to !eal-life problems. Though some application !eSUlts are general, their iDtent is not to form a cookbook. Radler, the goal is to be 8ble to look at a ~ and UDique problem aud determine an adequate, if approximate, solution.
5 Communications
The processiDg of a ttaDsaCtion begiDs with some pertinent event outside oftbe ttaDsaCCion processiDg system. 'Ibis event could be a customer making a IeqUeSt to a teller or ticket salesperson, a staIUS change in a power network, or an alarm generated by a patient. mcmitorlDg UDit in a hospital. The first tbiDg that must be done is to sead the data desczib.. iDg this event to the TP system. 1bis is the role of COI'II1tIlI1Iic1lS. The study of con .11lI1IIK:aIion facilities fins many volumes. The comllPmicaMD industty is a multibillion dollar industry with aearly a cemury of bistoly. It is highly regulated aad weU-undedtood tecImicaJly. It is a subject of iDteDse staDdaIdization by cqaniwioas such as the Amican Naticmal StaDdmds IDstitate (ANSI) and the EmopeaD IDrematioDal Telegmph aad Telephone Consultative Coillmittee (CClTI'). (Compue this state of affairs to tbat of the large but sti1l8edgJiDg c:ompater iDcIustty, where we CODtiDDe to dabble in seetiDg to 1IDders1aDd what we me doiDg.) Thaefore, we can only scrarch the surface of this massive body ofkDowledge. And we will do so only to the e:aem that we can uncIe:rsIand aad accomtt for the perfoImaDce issues involved in OC"DII'II'dcatiDg a traasactioD to the TP system and in ft!hD'!!ing a reply. The first half of this cbaprer consists· of sectioas tbat provide the cmmrqnjt:atj1)D background for the performance secUODS tbat foDow. For the c:owmuu1cadon novice, these iDiti.al secUODS range from the sDnpIe to the sublime, covering chancteristics of cormmmica1ion channels, methods of dam tnmsmission, protocol concepts, and modem open systems via layered potocols. Those well-versed in CQ!D.nnnjcatiODS may want to simply :skim these secUODS for the terminology used later. .
139
140
Communications
Chap. 5
The later sections use examples to develop performance-aualysis techniques for message ttansfer and establisbmentltermiDation procedures. These include half-duplex and full-duplex message ttaosfer, point-to-point and multipoint (LAN) contention net- . works, and multipoint polled networks. .
PERFORMANCE IMPACT OF CO• •UNICADONS
In a 1P system, a communication line is a server. It is a resouroe of finite capacity that . must pass data between the user and the 1P host. The average time that it takes to pass this data is its service time.
Ccnnmnnication lines are often sbared by many user tem1inals. Therefore, what may form are queues of user transactions awaiting access to the line. The role oftbe conumJDication facility in 1P system performance is shown generally in FJ.gUre 5-1. The data describing the transaction mives at the facility but must wait for access to it (1). The transac:tion data is tben transmitted over the cnmmuuication line (2) and entexs a queue (3) of work waiting to be processed by the host (4). When the host bas geneiated a reply (which is a performance study in ilself), it enters that reply iDlo a queue (5) to await access to the outgoing line. Fmally, the reply is returned to the user (6) via the communication facility. So far, simple. However, the analysis of waiting times and service times bas many complexities not evident in this simple description. COIDI"lmication queues. for example, are oftal not first-in, mst-out queues. Rather, access is granted to the line in an orderly fashion by polling terminaJs in a roDDd-robin fashion or in a diSOIderly fashion by lettiDg a terminal grab the line and see if it is successful in transmitting without colliding with anotheI' terminal's ttaDsmission.
TRANSACTION
-TIill---1
UNE
T1I1E.
UNE QUEUE (I)
TP HOST
REPLY
------1
....
UNE TillE
r---mrr--~ LINE QUEUE
(6)
(5)
Fipre 5-1 ComlllDoiadiCm JiDe ped'oaoaoce.
(4)
Chap. 5
Communication Channels
141
Line service times me complicated by the fact tbat considerable overhead may be required to pass one block of data. This overhead is created by the protocol procedures necessary for ensuring proper identification of CQ!D1Dnnication traffic and its protection agaiDst errors. Furtbe.nnoxe, line service time can be a function of the line etror rate, as blocks in etror may have to be retraDSmitted. In the following sections we will discuss the various communication techniques, message protocols, and network concepts which make up a lP communication facility and which can have an impact on lP peIformance.
The first level in the communication bienrchy is the physical communication cbaDnel itself, which can take many forms. Some forms are obtainable from public netwoIks, and others may be privately furnished.
DeGICIIted Unes The simplest of all communication channels is the dedicated line (Figure 5-2a). This ch!lDJlel is petmaneDt1y established and may support a single temliDal or multiple terminals. If the line supportS a single terminal (a point-to-point connection), then communiCation may be under control of the·host,· i.e., the host is the "master," and the terminal the "slave." As an alternative, the te.rminal and host may both act as master and contend for the line. If many teIminaJs are multitJropped on the line (a terminal cormection is referred to as a drop), thea. they lie usually conttol1ed by a polling pIOfOCOl in which the host master queries each terminal for incoming dara on a round-robin basis or according to some other poll schedu1e. The host also directs outgOing traffic to a specific tamiDal or group of terminals. Dedicated Jines lie highly ef1icieDt, as DO time is speDt in establishing the c0nnection; the CODIIeCtion is pennanent. However, dedicated lines are also quite costly and lie genemlly justified ODly if tbey can be higbly lJtiliud
is
Dialed LilIes For occasioDal use, dialed CODDeCtions via a public netwOlk are ofteu used (Figure 5-2b). WbeD a user wishes to commmricate with the TP host, he dials the host marmally or via an antoimtic terminal dialing fancticm and establishes a point-to-point connection for the dmation of his session with the host. Diak:d CODDeCtioDs can be quite economical for occasiODal use. However, connectioo-establislmmt time can be significant (many seconds); even worse, all host potts could be found busy. Oaly one terminal perctialed c:onnection can be su.ppotted, and data rates are sigDiicantly less 1ban those achievable on dedicated lines. Compounding the ~rate limitations are the higher etror rates found on dialed lines (typically an order
142
Communications
Chap. 5
o----~ HOST POINT- TO - POINT
MULTI DROPPED
DEDICATED
LINE
(0)
-1
o--z-
HOST
DIALED LINE
VIRTUAL CIRCUIT
(b)
(e)
I I
BUS
I
?t6
HOST
6 '
RING
i?'
IL. _ _ _ _ _ _ _ _ _ _ _ _ _ .J
SATEWTE CHANNEL
LOCAL AREA NETWORK
(d)
(e)
fi&are 5-2 CoaanmIicadaa cbamIe1s. "of mapitlJde dIaD those fouad OD dedicated liDes). These error Dtes slow the ~ even further because of r:ettaDsmissiOD ItqUileaDeuIs.
Peater
Virtual ClJ'Cllits the dedicated and dialed coaaecti.oas described above bave ODe cbaracteristic in COJDIDOIl: once, tbe CODDeCdOD is eslablisbed, equipllClDt is dedicar.ed to tbat conversation fiom source to demnation UDtil tbe circuit is b!oken. 1'heref0le. valuable co.... mmit;atioo equipment lies idle during pauses in the daD. convenarion. equipment !bat could in principal be used to support other convenatioDs. theIeby iDcreasiDg the capacity of the netwOlk and mduciDg the cost to the user. The emergenc:e of public and private packet switching networks bas addressecl this diJ.emma (Figme S-2c). Using tbis tecbDology, a user's data message is broken up into ~-leugth packets. Each packet is routed iDctependemly tbrougb'1he networlc to its
.
Chap. 5
Communication Channels
143
destination. Thus, each physical circuit coll1leCting switching nodes in the network actually carries traffic from several users on an as-needed basis. In fact, if a node getS very busy, many networlcs will route packets around that node using alternate routes. Thus, the packets comprising a message may, in fact, take different paths tbrough the network to their common destination. The different transmission times imposed by these various padls, compounded by random delays caused by queuing and line errors, mean that there is no guarantee that the packets will arrive at their destination in the same order in which they were sent. The proper disassembly of the message into packets at its source and the subsequent reassembly of the packets to recoDStitute the origjDal message at its destiDation is the function of a specialized piece of terminal equipment called a PAD (Packet Assembly and Disassembly). PADs may either be furnished by the customer and be located on the customer's premises or may be fumished by the network operator at the switch sites. In the latter case, customers CODUDUDicate with the PAD over a standard dedicated or dialed COTDID1JIIication line, as desc:r.ibed above. The host PAD is often implememed witbin the host via software, thus eliminating the need for special PAD hardware. The connection between customers through a packet-switched network is c:allecl a virtual circuit, since it is logically the.te (what I send, you receive) but not physically there. That is, one cannot point to specific equipment and say that equipment is devoted to a particular c:onnection. Just as in standard telephone technology, virtual circuits can be dedicated (permanent vU:tual circuits) or "ctia1ed" (switched virtual circuits). The dedication or sharing of these circuits !elates to the use of logical :cesourc:es in the switches that act to define the circuits, rather than relating to specific M1'D!D1mication lines. The use of packet-switcbed vinual circ:uits brings with it significant economics. The one disadvantage is a somewhat longer propagation time through the netwo!k (the "line time" ofFigule 5-1).
SIdeI'. Chan..... Another medium for uansmissioD of data is 1he stztellik cltDnn81 (F1gUIe 5-2d). Functi0nally, a sateDite ch8llJ1tl!1 is much like a ctialed or dedicaIed line. Channels can be dyDamically. allocaIed to users as 1hey need them (ljke a dialed CODDediOn) or can be decticared to
a pair of users. SareJ]ite ch8JJDt!ls are iDhe:raltly OJl&.way. In a 1P application, two chllJl1'lels are needed, one to sead the uaasacdcm and one to zeceive the zeply. -SatelJjre ch8llllftls can have very high baDdwidtbs aDd ccmseqaeatly can Support large data mtes. An interestiDg possibility is that the sigDa1s IeJayed by a satellite can be received by any receiver m1he "footprint" of the sarellite, thus opening 1he way to a variety of broadcast oppo.ttnnmes, such as disUibutiDg Sllmmary data of inIemt to all users of a TP syStem. There is, however, an impoItant performance issue that !elates to satellite clwmels. That issue is its "line time," or propagation delay. A typical geostationaly satellite is about 36,000 kilometers from its earth stations. At the speed oflight (300,000 kmlsec.),
Communications
144
Chap. 5
the propagation time between an.earth station and its satellite is 120 msec., or 240 msec. from the tnmsmitting earth station to the receiving earth station.
Satellite propagation delays coupled with comparable packet-switch delays can cause serious performance problems in a TP application using a packet-switched service, if satellite cbannels are used by the packet switch.
Local Area Networks A local area. network (Figure 5-2e) interconnects multiple users via a bigh-speed channel . to which an users connect. The netwodc medium usually comprises twisted-pair cable or coaxial cable configured as a bus or a ring. Contemporary local area networks typically support data mtes in the one- to ten-megabit/second range. More complex networks can support multiple channels of these capacities or greater. Usually, an users on a local area network are equals-that is, there is no master on the network that controls access to the network. Network access is either by contention (start tnmsmitting if DO one else is and hope DO one else does) or by a masterless fann of polling known as token passing. These protocols are discussed in a later section. Local area networks can provide very bigh speed communication channels between large numbers of users over limited geogmpbica1 range. Typical local area networks will span a building and perhaps even a college campus or industrial parle.
The efficiency of dedicated circuits often can be improved by combining the traffic from multiple users onto a single line in a mauner more efficient than the use of simple polling techniques. One way to accomplish this is tbrough the use of multiplexers. Multiplexing is the sharing of a chaDnel by several users in a way that is substantially traDspaIeDt to the users. In effect. the channel is broken up into subchamels. The subch8DIJels are then available as independent channels to individual users as shown in Figure 5-3a. Of course, the combined data IeqUiIements of the users must be somewhat less than the capacity of the channel. TbeIe are several established tecJmiques for mn1tip~
• Frequency Division Multiplexing (FDM) divides the cll2nmel into separate cbannels in the ftequeDcy domain, as shown in Figw:e 5-4&. A wide baDclwiddl is carved into sevaal subcbanneJs of smaJler bandwiddls. Since the data lite that is supportable over a channel is pIOpOItioDal to its bandwidth, tbeD the capacity of each subclJannelis only a fraction of the main cJmmeJ capacity. This very important characteristic of the data capacity of a bandwiddl-limited channel derives from the well-known Nyquist theorem (Nyquist [22]). FDM is very economic:a1 in tams of the equipment J:equirecl to support it. However, it is an inefficient use of the main channel because of the guard bands
required to prevent inI:aference between subchamels. 'Ibis is wasted bandwidth, not available for data transmission.
Chap. 5
145
Communication Channels MAIN CHANNEL
USERS
M P X
-------
a
USERS OR HOST POR1S
R SU8CHANNELS
MULTIPLEXER (a)
MAIN CHANNEL
USERS
c o
HOST
N
C SUBCHANNELS
CONCENTRATOR (b)
I'igare 5-3 Sbaled c:baaDel use•
• Tune-DivisionMuItiplexing (TDM) divides the channel in the time domain (Figure 5-4b) IIIher tbaD in the fIequency domain. The bigh-speed data stteam of the main channel is divided iDto time slots which are ptealloc:ated to snbclvmnels. The time sloes may be ODe bit wide, ODe byte (chal3cter) wide, or some other size.
Data1'eCeived from a useroWDblg ODe of the subt:hannels is mserted iDto that sqbc:bannel at the UIIIsnritting ead and is ex2:lacred aad IecoDSInJctecl iDto abe user's message at the sec:eiviDg cmd. Except for some syncIaoDiziDg ovedIead needed to gwaaId'ee that the mceiver can determine the beginning of a sabchannel sequeace (known is a m..a.e), the TOM teclmique is highlyefficieut in its use of maiD channel capacity. • Statistical Multiplexing fUdber inaeases the utilization of the maiD channel when the subcbannel users are casual. Casual users use a suhcbanne1 when they Deed to, but these users are idle a substaatiallDlOUDt of the time. Most 1P system users are casual. A problem with both the FDM and roM approaches just described when users are casual is that the subcbaane1s are often idle. Available capacity is not Peing used.
Communications
146
Chap. 5
MAIN CHANNEL
_r--------.A'------\ '-y-----J SUBCHANNEL - - - -.... FREQUENCY
FREQUENCY DIVISION MULTIPLEXING (a)
MAIN CHANNEL FRAME A
\
SUBCHANNEL ----~... TIME
TIME DIVISION MULTIPLEXING (b)
Stmsrical multiplexiDg is a roM variant thai solves this problem. Wltll statistical mul1iplexiDg. S1Jbcbannel time slots are not Ift8Ilocatecl to users. In fact, theIe me many IIlOle users tban time slots. WbeD ,data is n=ived at the trIIismiaet. it is placed 'in the next available time slot and is seat to the receiver with some clara idemifyiDg die sencHng user. In this way. maay casual users ,can be serviced by fewer subchann.e1s. Statistical multiplexiDg does have 1be problem of cmrload when data bursts arrive tbal CIDI10t be handJt4. In this case, die multiplexer IIlIISt buffer tbe excess data umiJ. it can c:atch up. If its baffas fill. then'the multiplexer must execute ftow CODtrol ~ to stall the users sending data or else it wiIllose data and will have to request leIDDsmission. In any event, statistical multiplexing will impose delays on transaction and Ieply uaffic dmiDg peak periods when it caDDOt baDdletbe peak traffic. However.,
Chap. 5
Communication Channels
147
properly engineered. the statistical multiplexer can significantly reduce channel costs while remajning substantially tIaDspareDt to the user. The above discussion has reviewed the primary techniques for multiplexing. In most cases, these tecImiques will have mjrrimal if any impact on perfonuance. The!e is another device. a concentrator. which is often used to perfonn a similar function (Figme 5-3b). Essentially just balf of a TOM multiplexer. the concentrator combines traffic fcom many users into a single data stream for the host to untangle. In the case of the concentrator. the time slot size often is large enough for an entire message or at least for a sigDificant packet of data from a message. In this case. there is a performance . penalty equal to the message (or packet) transmission time over the maUl channel. since the message must be completely IeCeived at the transmitting end before it is sent to the receiver.
So far we have discussed c:omnmnication channels without concemiDg ourselves very Though this is the topic for the next section."
~ with how data getS passed over them.
o USER
Ul.. o~__lJL. ___-(o)-_lJl._--t CHANNEL
DIGITAL CHANNEL (a)
ANALOG CHANNEL (b) Pipre 5-5 Modems.
HOST
~--...
148
Communications
Chap. 5
there is a very imponant device that is often needed to convey a data. stream over a physical communication chamlel. Though there are purely digital channels available today on which the user can directly impIess his binary data stteam (FiguJe 5-5a), most data channels are analog Channels that are derivatives of voice chaIinels (see Figure 5-Sb).They caD cari:y tones but cannot cmy binary levels as we know them inside the terminal or computer. Thelefore, a device is needed to convert the data signals into tones suitable for the channel. This can be done by sending a loud tone for a one and a soft tone for a zero (chaugiDg the amplitDde of the tone, or amplitutJe modulation). As aD alternative, the pitch oftbe tone can be alteIed to designate a one or a zero (frequency modulation). There is also a teclmique similar to ftequeDcy modulation kDowD as phase modulDtion. Because of various CODSideratiODS, such as error perfonnance and cost, frequency modulation is most often used, though phase modulation is also common. A device is needed at the user and host ends to modulate the data stream in order to create the analog tones and to demodulate received tones in Older to recreate the origjDal data stteam sent from tbe other end. The device that perfoIms this modulation and demodulation function is known as a modem. Usually, the modem is traDspa1'eIlt so far as performance is concerned. However, there is a very important case in which it may have a significant peEformance impact. This case is that of half-duplex conmnmication, especially for dialed cormections, and is discussed in DlO!e detaillarer.
One fiDal note is in Older about c:c,mmnmicatioo channel performance; it concerns propagation delay over'tbe ciIaJit. We have akeady discussed propagation delay over sate1lli:e chsmnels and through packet switched Detworks. But what about simple dedicated and c6aled telephooe chanDels? Can tbese delays be significant? We can begin by lookiDg at the simple propagation delay of the sigDal over a wile CODDectiDg tbe user to the host. Depending upon the type of wile (from loaded rmal c:iIa1its to c:ouja] cable), the speed of a signal over wile can wry between aboUt 0.5 to 0.9 of the speed of light. Even at the speed of light, a signal wID take 16 m&eC. to traverse the 3000 miles'across the USA or 65 msec. to go,halfway amund the world. Mcxe ~ times might be twice as laqe. Add to this the fact that every time the signal goes tbrou.gh an elecIronic repeater or fiber, another few miDjSf'!CX'Dlis are added. Long-haul circuit delays of several teas of nriDjSl'roDds are cJefiDirely to be expected Dialed lines typically will be WOlSe than leased lines, since 1hey may go tbrough more c:emral offices (and their associated eqaipwent) than conditioned leaseclliDes. Furthemlore, their lower bandwidth usually goes baDd-iD-band with longer propagation delays. Long propagation delays can be of special conc:em in large polled netwolks in which each transadion must be chaIged with some poll oveDlead. If each poD requiIa two propagation delays (one to seud the poll JeqUeSt and one to IeCeive the response), ~
Chap. 5
Data Transmission
148
can be quite slow, whether it is successful or not. This is discussed in more detail later.
DATA TRANSMISSION Once we have a data channel over which we can feed a data stream, we must then agree on how the data will be represented on the channel. It is not enough to just send a message comprising a string ofbinary bits. We must know where the message begins and ends and how to inteEpret the bit patterns contained in the message.
Character Codes Typically, messages in 'IP systems are made up of strings of cbaracters: alphabetic characters, numeric chamcters, and special characters, such as punctuation maries. This is not tnle in all applications. Scientific data might be sent as large binary numbers in sciemific notation (mantissa pius exponent). Satellite telemetry data might be long streams of binary data, as would a computer object progEaDl file being downloaded over a ciR:uit. We are interested in the alphanumeric data of TP system messages in which a character can be represeotecl by a specified set of bits. Early teletype systems used 5 bits per character (32 combiDations). This was the Baudot code. Two special characters (FIGS and LTRS) shifted between alphabetic meaning and numeric/special symbol meanings of the mnairring 28 combiDations (all Os and all Is had special meaning). In the 19505, 6 bits was a popular definition of a chaIacter. 'Ibis chaJ:acter size provided 64 combinations covering the alphanumeric character set (36 charactms) plus ample special , characters. However, it soon became apparent that this was not enough. Uppa--case and lowercase characters wae desired. Fwtheanole, IIlOdem protOCOls requhed a rich set of contto1 characters (this topic is discussed in more detail later). Sevea-bit charactas (128 combi1IIIions) wae IDOle appopriate to meet these needs. Thus was born the ANSI SIaDdaId ASCII code (American StaDdard Code for information 1Dtmcbange). The ASCD code set is a seven-bit codeplus an eaor...cfetec1ing parity 'bit, as shown in Figure >6a. The parity bit may be unused (always set to 0 or 1), or it may be set sudl that the total number of hils in the cbaracter is odd· (odd parity) or even (even parity). If either an even or odd parity bit is used, tbeD the resulting eight-bit code is error..defeding. This is because the changing of any one bit in the character because of noise will cause the parity mit to fail. A Competing code set is IBM's EBCDIC (Exrended BiDary CodedDecimal). 'Ibis is also an 8-bit code in which all 256 combinations are used (see Figure 5-6b). Thus, modem tec1mology bas settled on an 8-bit character code. This grouping of 8 bits.is called a byu. (Four bits is eaough to repesean a number and is used in some applications. Four bits is called a nibble). Note that today's compateIS typically use word sizes that are multiples of bytes-word sizes of 16 bits or 32 bits.
150 ~
Communications
EVEN, ODD, OR NO PARITY
Q-Z =61-7A A-Z
Chap. 5
0-9
= 60-69
= 41-SA ASCII CODE (Q)
Q-Z
= 81-89
0-9" F'O-F9
91-99 A2-A9 A-E
= CI-C9 01-09
E2-E9
EBCDIC CODE (b) NOTE: Character codes ore in hexadecimal
Having de1iDed an eight-bit byte as a basic UDit of iDfolmation in a 1F system, we must DOW be ~to seDda string of bytes over a COIIIIYI11Dic:atioD channel in an jnte1Jigmte fashion. Simply sending a loDg string of bits is DOt sadsfactory, siDce we would never kDow wIae the byte bouDdarles wae (see FigIue 5-7a). Cleady, additioDal iDforn1atioD must be embedded in the ~ of bUs so that the mceiver can cfetmnjne wIae a byte starts. ODe tedmique for doiDg this is caDed ~ ctIItIIrIlIIIic.· As shown in FigtD S-7b. a steady ""I" sigDal is traDsJ:DiUed betweeD c:baracters (a IlUD'ting signal). This interval is caDed the stop iDtetval. WheA it is desiIecl to seDd a byte. a stIlTt bit oomp.rising a single "0" bit (a.spacing sigDal) is &eDt, fonowed by the eight data-bits. Then the )iDe r:etams to marking for 1he next stop interval. The stop interval is guamnteed to be of a minimum leugth (typically 1, 1.5, or 2 bits in length). Thus, each byte is framed by a I-bit start sigDal at itS bElgiunjng and by a stop sigDa1 ~ its end (which is at least one bit in lengdl). To recogaize byte bouDdaries, the receiver
Chap. 5
Data Transmission
151 WHICH 8 BITS IS A BYTE '>
J .
1000110100000110
SIMPLE BIT STREAM (a)
BYTE
" STOP
MARK
I
LJl
SPACE
0
0
0
I
I
0
I
' l1Ja
t
STOP
-
START
ASYNCHRONOUS ENVELOPE (b)
S Y N
S Y N
S Y N
D
D
A T A
~ A
....
D A T A
S Y
.N
S Y
S Y
N
N
D A ••• T A
SYNCHRONOUS ENVELOPE (c)
1'ipre5-7 Byte 1IaDS1 lissioD
Simply looks for a stop-starttraDSition (a mark-space uaasitioD), discaIds ibe fiDt bit as ~ S1art bit, and stmes eight bits. The next bit should be a stop bit, aad the next start bit is awaited.
.
To achieve byte m:ogaition, the asynchronous COI""'Ilui<Son tecimique has cre.ated a 2-bit enVelope arouDd each byte (assuming a 1,-bit stop interval). A byte beiDg traD.Smitb:d over an asynchronous communjcatiOll c1wmel therefore requiJ:es 10 bits to pass 8 bits of iDformatioa-a 2S percent overhead. One iDterestiDg characteristic of 1:bis techaique is 1bat the traasmitter may transmit a char8cter at any time, since the stop iDterval between cbarac:ters can be axbittaI:ily long. This characterisUc is particularly useful for data tbat is raadoJDly generated (e.g. from a keyboatd) and gives rise to the term asyndvonous. applied to 1:bis tedmique for c0mmu-
nications.
152
Communications
Chap. 5
SpchI'OllOll$ COmmunication Syncbronous cornmunjcation takes advantage of blocks of data tbat can be 1raDSmitted as uninterrupted byte streams to achieve a reduction in enveloping overhead (and also to achieve an improvement in error pedormance, as described later). Basically. one or more special synchronization characters (SYN) are inserted periodically in the byte 5Ueam. The receiver can search for the syncbroDization sequence and then can count out 8-bit bytes thereafter. A typical syncbronous sequence is shown in FlgUre S-7c. The transmission is initiated with 3 SYN cbaracters (the ASCII SYN character is hexidecimall6). If the receiver is not in synchronization, it will look: for a SYN character by continuously evaluating the last 8 bits received. When it finds a SYN character, the receiver then starts accumulating 8 bits at a time as data bytes (the next two should also be SYN characters, a condition tbat can be used as a saDity check). Periodically, the transmitter will iDsert additional SYN characters to allow the receiver to ensure itselftbat it is still in synchronization. When the data has been sent, the traDstIritter can go idle, or it can send a steady stteam of SYN cbaracters to maintain synchroDization with the receiver. A typical interval between SYN cbaracters is 128 data bytes. If 3 SYN cbarac:teIs are sent after every 128 data bytes, then the envelope overhead IeqUired for byte recognition is 3/128 = 2.3%. This is an order of magnitude better than asynchronous communication.
Noise OIl the commnniraioD liDe as well as phase. frequency, and amplitude distortion caused by IiDe chmacterlstics will distort the data sigDal as it travels over the channel. This is evideDced in the demoduJated .sigDa! by a pheDomeaon known.as jitter. If successive received bits are viewed overlapped OIl an oscilloscope. the J'f!SIJJting pattem will appear as shown in Figure s-8a. Each bit ttaDsitioD. will geaen1ly not occ:ar exactly at the time tbat it should. Radler, it will be a little early or a little Jam. appeariDg to ujiuer" back and forth as successive bits are viewed. Though the relation betweenjiUer. liue distortion, aDd line noise is quite complex. the amount of jitter can be used as a measmre of the inteasity of the noise and distortion OIl the line. Figme 5-8b shows the effed: of jitter OIl an asynchronous sigDal. Let T. be the duration of a bit interval. The apploprlate strategy for asynchronous !reCeption is as f0llows:
.
1. Look for a srop-start traDsition. 2. Wait for a one-balf-of-a-bit interval (1',;2). 3. Sample the IreCeived sigDal. This should be the start bit (01herwise. declare a . false cbar.Icrer and JetDm to 1). 4. Sample eight more times at the bit interval. T•• to obtain the. eight data bits.
Chap. 5
Data Transmission /
153
OVERLAPPED RECEIVED BITS
L / .lITTER
"
EXPECTED TRANSJnON
JITTER (a)
ASYNCHRONOUS TOLERANCE (b)
• • • SAMPLE nilES
SYNCHRONOUS TOLERANCE (e)
s.
.
.
Sample ODe more time after an iDterva1 of Tb to easme that a proper stop iDterval bas been ~ved (otherwise, dec1ale a synchroDization error). 6. Repeat 1 tbrough S for each suc:cessive character.
From Figure S-8b, it is seen that a jitter of lnagnude T,)4 can move the start-stop traDsition 1/4 of a bit to the right and the traDsi1:ion of any data bit 114 of a bit to the left. At tis point, the sample of the bit !My be in error. Thus, the maximum jitter that can be ~ by an asyncbJOnous c:haDne1 is T,)4. Figure 5-Sc shows the equivalent: case with a synchronous stteam. For synchronous ~, the sample time is DOt detamined by a single trmsition as it is in the
Communications
154
Chap. 5
asynchronous case. Rather. the sampling time is based on the long-term averaging of many data transitions (or in some cases is derived from the modulated sigDal itself) and is therefore quite accurate relative to the expected transition times. Given accurate sampling times, it is evident from F"'1gure 5-8c: that it would take jitter of a magpitude equal to TJI2 to aeate an error. Therefore, a synchronous channel can tolerate twice the jitter that can be tolerated by an asynchronous channel. We have now seen that asynchronous teclmiques incur a byte-identification overhead Which is an order of maguitude more than synchronous channels and that they can only tolerate half the noise. The reduced noise tolerance means a higher incidence of rettansmissions and a fmther reduction in efficiency. So wby is asynchronous communication even used? ne reason is cost. The· requirements for more 8ccurate clocking and for a ~ complex byte boundary recogDition algorithm (recognizing a SYN character rather than a simple transition) make synchronous traDsmission IIlOIe expensive than async:bronous traDSlnission. Therefore. asynchronous techniques tend to be used for lower-speed applications (up to 2400 bits per second). and synchronous tec:bniques tend to be used for higher-speed applications, where getting as much out of a cbaDnel as possible is desirable.
Error Protection We have discussed the generation of errors because of line noise and distortion that cause jitter in the IeCeived signal. We have also seen one example of an error-detectiDg code to proteet against such eaors: the parity bit used in the ASCII cbaEac:ter set. This is commonly known as a vertical redundtmcy cMck (VRC). UDfortuDately, errors on CODIDlUIJication lines are not isolated. They tend to occur in short bursts. 1'helefore, it is qm possible that an even Il1IIIiber of errors will occur within a cbaractec. In this event, the c:baracter parity check, or VRC, will still be satisfied; and the error will go UDCIetecIed. For this reason, a SUODger amr-detedion scheme is often mqujred. This is twicaUy done by protediDg the message (or transmission block) with additional error-detection· codes placed at the end of the message. There are two in common use. 1. The loDgitudinal redundancy cbeck (LRC) adds ODe byte to the message that is itself a parity byte. Each bit is set so that the sam of an CODespondiDg bits in an by1es of the message is even (or odd. as. the case may be). 2. The cycIicalledlmdaDcy c:beck (CRC) isa much stroager error detecrion code. It is typically a 16-bit code added to the ead of the message, tbough lcmge.r codes give better protection. Though the theory behind CRC codes is quire exteDsive aDd complex (see lfamming [8]),· the CRC is essentially tbat sequence of bits that, if appended to the message, creates a biliary DUmber that is exactly divisible by some prede1mninetS DUmber.
CRC codes can be extended to provide forwatd error c:oaecting Sys1emS. In these systems, : . thele is so muchmmdancy provided by the error c:omction code (often up to SO
Chap. 5
155
Data Transmission
percent-see Stallings [25]) that Dot only can an error be detected but also the specific bit in error can be identified.. Therefore, that bit can be corrected. In this case, the code is a single-bit error-correcting code and may, in fact, couect many multiple-bit errors. Codes can be defined. that will cottect up to e eJl'OIS and detect up to d errors, where d > e (see Gallagher [6]). However, the price that is paid is efficiency. As error codes get more powerful, they impose a higher overllead on the system. In the cummt art, error-c:oIreding codes are used only in situations where retransmission is very expensive or impossible. Satellite c1wmels are a good example of the use of these techniques, since retransmission uses expensive chaJmel capacity. Broadcast systems are an example in which IetraDsmission may be impossible, since there may be no return path.
, HaH-Dupiax Channels : ..\., SO far, we have concemed ourselves with the encoding of data so as to be able to identify it (byte identification) and to proteCt it against eD'OIS. There is one other major pedOImallce . consideJ:ation at the data traJlSmjssion level, and that is whether the communication channel is simplex, balf-duplex, or full-duplex. A simplex channel can ttansmit information in only one ctirec:tion. A half-duplex channel can transmit in either direction but ODly in one dUection at a time. Afull-duplex channel can transmit in both directions simultaneously. We will ignoIe simplex channels, since they are not usually of use in TP systems. If they are used, they bebave, for performance purposes, as bali of a full-duplex clwmel. . A balf-duplex channel aeates several performance conside:mtions. Not only is traffic in one dUection affected by traffic in the other diIection, but there also can be significant delays in QmJiD.g a channel around. Channel turnaround time comprises two c:ompouents: 1. Chllnnel settling time. When a channel is relinquished by one ttansmitter and acqui1ed by another ttaDSDDtter, tbeIe is a paiocl during which the energy imparted to the channel by the fust nnsmiuer is decayiDg, and the energy imparted by die seccmd 1l'8IISIIUt:Ier is buildiDg. ODly when the new transmission eaergy is peatertban the old lIansmission energy by a significaat amount can the line be used for Ietiable oomnnmicalioa. This time is typically a few byte. intervals. . 2. Echo. suppressors. LoDg telephone tines 1r:nCl to develop echoes because of impedance mismatches alcmg tbeir leagth.To }RVent 1bis from becohdng a nuisaDce to telephone usea, echo suppteSS01S have been insIalled 1bIOugb.out the telephone DetWodc. These devices detamiDe die dimction ofprednnrinant transmitted eneIgy and suppms 1rIDSmissions in the IeVerse direction. When the direction of ttansnrission nMDeS, the echo supptessors IeVerSe di.tection. This can take a few tellS or even a few htmdreds of tmllisecmds. (Have you ever noticed. the firSt syDable of the from the other end being cut off?) : On a balf-cluplex clwmel, Ieliable communication cannot be achieved until the
conve:rsmon
156
Communications .
Chap. 5
.
echo suppressors on the.cbannel are all reversed. Though the telephone companies have undertaken a program to upgrade their equipment in order to eJjminate echo suppressors, many remain and probably will remain for the foreseeable future. This is primarily a problem for dialed lines, as d.edic:ated tines are conditioned in many ways which preclude the need for echo suppressors. The tumaround delays required by channel settling time and by echo suppressors may be established by timers in the terDrlDal or host equipment or may be compensated for in the modem itself. The modem provides two signals for this purpose: 1. A Request To Send (RTS) sigDal to the modem, requesting permission to send data. 2. A Oear To Send (crs) sigoal from the modem, indicating that data may DOW be sent.
crs
In actual fact, the 0Dly logic that links the sigoal to the RTS sigoal is a timer in the modem. The time-out is set by the user to the minimum safe time determined for channel tumarouDd. Typical timer values range from tens of nriUjseconds to hundreds of nrilJjseconds. Oearly. channel tumarouDd delays can have a sigDificant impact on communication channel perf"0l1D3IlCe. A lOO-msec. tamaround penalty for every 2OO-byte block over a 24OO-bitIsec. c:bam:ael requiring (8)(200)12400 667 msec. per block is not to be taken ligbtly. This becomes even worse for a polled channel, wherein a typical poll sequence might be 3 eharacters and a poll response 1 character (10 msec. and 3.3 DJsec., IeSpectively, at 2400 bitslsec.). Adding 100 JDsec. to each of these completely distorts the pmfomJance picture.
=
FulI-Dup1ex CIIann_
-.-
A full..duplex channel is conc:epcua1ly simple. Both sides of the conversation can transmit simu1taDeous1y. Full-duplex c:bannels can be derived in several ways from physical channels. For iDs1aDc:e. two separate cImmeIs may be provided. one to be used for each direction. This is usually the tedmiqueused for higher speed CQI1JID1IDicaDODS. For lower data rates. a single physical channel may be divided iDro two logical channels using FlequeDcy Division Multiplexing (FDM). as described earlier. The lower half of the frequeacy spectmm supported by 1be channel may be used to send data in one diIection, and the upper half of 1be.frequeacy specuum may be used to send data in the opposite ctiIection. This teclmique is used by. low speed (300 bit per second) modems. One significant advantage of this technique is that a fuD-duplex cmmec:tion can be established over a dialed line. Though fuD-duplex channels me c:onceptualIy simple. the message tmDsfer pr0tocols RqDired to take maximum advantage of the full-duplex capability can be quite a bit more complex dian those used with half-duplex chaUDels. These protocols are discussed in detail in the next section. .
Chap. 5
Protocols
157
PROTOCOLS The previous discussions can be perceived as descnbing the process of data transmission. The facilities and teclmiques described allow sequences of bytes to be delivered between
users. Let us now discuss data communications. In order to have a meaniDgful communication of data between a sender and a receiver, the data must be identifiable and transferred reliably. The procedure for accomplishing this is called a protocol. A protocol is an agIeemeDt between the sender and receiver of data as to exactly how that data will be transferred. Protocols provide tIuee primary functions:
1. Message identificotion-They identify the bounds of messages carrying the transaction and response data. 2. Do.tIl protection-They protect data agaiDst euor, ensuring its reliable delivery.
3. Chtmnel alloco.tion-They provide the mec:lwrism for allocating the channel in an O1derly mauner to the various competing users of the cbarmel.
Protocols typically have tIuee distinct parts:
1. The establishment procedure, which serves to establish a virtual CODDeCtion . between two users (or many users in the case of a bmadcast). . 2. The message transfer procedure, which describes the form of message ttaDsfer. 3. The termino.tion procedure, which bJeaks the virtual c:cmnection• .The establishment and tenxrination plocedutes saDsfy the ehaUDe! allocatiOD functiOD. The message-traDsfeprocedure is the message-ideDtifcaticm fImction. DaIa potectiOD spans and sigDUicandy complicates all pmcedm:es. Tbere me many standan:Iized protocols, and their SIUdy alcme would fill volumes. Our COIlCeED is to 1IIidea:staud1be basics ofpmtocols fmma paformaDce Viewpoint so tbat, given a protocol specific:atioD, we can evaluaIe i1s pedomumc:e wiI:biD a given ccm"'PIWc:atiODs eavimnment Therefole, we wiJl SIIIdy some classes of prorocols·and zeJate tbem geoen1ly to some of 1be IIlOIe popular protocoJs in USe today.
" ••8.,. ,.,.,,1icafiWJ .." I'Iotedioft Just as start-stop bias and sy&ebroDizatiOll bytes are USed to identify the boundaries of data bytes in a bit stream, so must there be a u:w:c:hawsm to iden1ify messages in a byte stream. This is typically accomplished with CODttol characters that are chosen to be UDiqae in the bytesttam. Quite simply, a 1IDique start-of-text byte may indicate the start of a message. and an end-of-text byte may iDcficate the end of a message. These are COIDDlODly desiguatwl STX and m;x, and are, for blstanc:e. fOUDd in the ASCII control set.
158
Communications
Chap. 5
Em>r-protec:tion (and in sOme instances, enor-correcUon) bytes follow the ETX' byte. As discussed earlier, these include LRC (longitudinal redundancy check) or CRe (cyclical redundancy check) codes. A typical message protected by a sixteen-bit CR.C code would be formatted as follows: STX (data) ETX CR.C CR.C
Let us next look at the major function of any protocol: the reliable transfer of a message from its source to its destination. Just as with communication cIwmels. we may dichotomize protocols into balf-duplex and full-duplex protocols. When using a half-duplex protocol, traDsmission occurs in ODly one direction at a time. A full-duplex protocol supports simultaneous communication in both directions. Half-duplex protocols may be implemented using either balf-duplex or full-duplex communication cbaDDels. However, full-duplex protocols require a full-duplex clwmel.
Half-Duplex Message Transfer. In a typical ba1f-duplex protocol, user A sends a message to user B and then awaits a respouse from user B, as shown in Figure S-9a. This IeSpODSe may be a positive acknowledgement that the message was received comctly (ACK) or a negative acknowledgement iDdic:atiDg that the message was received in enor (NAK). If user A receives an ACK, the next message is sent. However, if user A receives a NAK, the pIeVious message must be retransmitted and the above process repeated. This protocol causes the tnmsnrit:terto pause between messages in order to receive an acknowledgement from the receiver. 'Ibis pause is IeqUired because the half-duplex channel supports ODly ODe traDsmission at a time. Full-DupIex Message Transfer. A ftdl-dDplex clwmel allows the traDsmitfer to send ccmtin~, acknowledgements can be xebiIUed over the reverse channel while message traDsmission coadDDes. In fact, the. other ead can be tnmsmiairag its own series of messages at 1he same time. .The problem is tbe COO11IiDation of acknowledgements with messages, siDcethe traDSliiiaa may be able to seud many messages before it gees' an acknowledgement to a pmous message (especially over cbarmels with long propagation times, such as sateDite or pactet-switdlecl cJwnMls). The solution to this problem is to number messages and tb.en to acknowledge by message.number. In fact, to allowbotb. eDds·to tnwswit sjmqbaneo\1sly, the ACK'or NAK can be piggybacked into each message. Figure S-9b shows this proc:edure. User A is sending messages AI, A2, etc., wbile user B is sending messages BI, B2, and so on. Each piggybacks an acknowledgement of the last message received comctly or the first message RCeived iDcoDecd.y. . For iDstaDce, by the time user B is ready to send its fourth message, B4, it still bas ~y been able to process user A's message Al and so sends an ACKI (just as it had with
sa.ce
Chap. 5
Protocols
159
USER A
USER
B
HALF DUPLEX PROTOCOL (a)
USER A
~ ~
[ill
IB1
[ill
m:J IA41
[ill [ill
ACKO
ACK2
ACIG
ACK3
ACK3
ACK6 ACK8
ACKO
ACK5
XXXXXXXX
USER B
L!IJ
[!!] I!!] IE] [!!] [ill [!!] [!!J [!!]
ACKO
ACKO
ACKI
ACKI
ACK3
NAK4 ACK3
ACK4
ACK6
FULL DUPLEX PROTOCOL (b)
I'igare 5-9 Basic pmcocoIs.
its p!8Vious message. 83).
By the time user B is n:ady to traDsmit. its next:message. B5, it bas appoved two mme messages from user A and so sends an ACIO.
However. user B DOW fiDds user A's message A4 in Cl1VJ'. 1berefcxe, user B sends a NAK4 wiIh its aext message.. 86. When user A mcei:ves the NAK4, it resets i1se1fto start senctiDg at its message A4, aud the process comimJeS. This is obviously a mmecomplex pIOtOCOl1ban that mquhed for baJf4lplex dIaD. nels. It also zeqaires stcnge at the transmitter for seven! messages to support the poteDtial tettaMmission of1hese messages. This canbe.compared to a stonge JeqUhemeat of only one message for the simpler ~-cIupJex channel, However, in Iebml for this added cost, tile fuI1..dup1ex channel is utilized to a much greaaer.exteDt.. Note t:bat in tis example the u:ausmitter was xequired to back up to the message in eaor and to Jetnmsmit all d8ta from t:bat point OIl. We will xefer to tbis as the Go-Back-N protocol. An altemative stritegy that is also used is sel«tive retrtI1Ismissio, in which ODly the message in eaor is leO'ansmitted, In this case. mfen:iDg to Figme 5-9b, user A's message sequence would have been ( ••• A4. AS, A6. A4, A7, AS •.. ). However, this tecImique requires stoDge not only at tile ~ but also at the receiver, since later
Communications
160
Chap. 5
messages must be held until the message in error is received properly. Again, higher cost yields higher performance. .
Channel Allocation In 0Ider to send a message from a source to a destination, the sender must acquire the sole use of the chamlel and must then be able to address the receiver. This is the establishmeDt" procedme.
When the message (or perhaps a block of data repteSeDtiDg a partial long message) bas heeD sent, the sender must release the charmeL 'Ibis is the terminatiOD procedme. In order for a chamlel to be allocated to a sender, one of two procedures may be used: 1. Assignment of the chaDDeI by an orderly procedure. 2. Uncontrolled contention for the channel by all users. Ordedy procedures for channel assignment include polling of users by a master station or the passing of a token giving the current token.1lolder the right to use the cbaDDel. If a conteDtion protocol is used, then provision must be made to detect collisions and to recover the lost data. 'IbeIe is one further dichotomy to be m:ogDized wbeD consicIering establishmeDtl termiDation procedures, and that is the Il1.DDber of users of the channel. If tbe!e are ODly two users, then the cbaImel is a point-to-point channel. If there are more thaD two users, thea the clumue1 is a multipoint chanuel.
So far, the protocols we have disc:ussed are byte-orieDted. Control characters are takeD from a byte set, and error control characters are based on a byte structure. In fact, data is expected to be bytes• .Tbere are IDIIlY app1icatioas mwhich a byte SUUdUre is DOt to tbe data. A good example is public aetworks, which must be tDIISp8Ialt to any data structure. Data chanuels derived from telephoDe HDes meet 1bis crllerioD as the particaJar form of data ICpII*.....'ion and die protocol are up to die user. The situaDODbeoomes IIlO1'e complex for packet-switdJiDg netwoJb, siDce by tbeir very DatDre 1hey must bleak tbe users· daIa imo pacbIs. They use their OWD dam suuctares and p:otocoIs, aud these must be 1laIISp8leDt to any user da!a~. . The protocols used for such appIicatioDs are bit-oriellted, as they mast deal with data at the bit ratber dian the byte level. Since syncbronous HDes are typically used with these protocols for pedormance considerations, these protocols syuchroDjzation 'proceduRs. 'IberefoIe, we mfer to such p:otocoIs as bit synchronous protocols. Bit syncbroD01IS protocols use fall-duplex channels and are fall-duplex protocols. Message tlaDSfer procedures aud establisbmeDtIton procedures are entwiDed in these protocols. The two most commonly used bit synchtonous p:otocols are HDLC (a CCl'lT stan-
m
umve
J'rovide
Chap. 5
161
Protocols
dard) and SDLC, IBM's offering. HDLC stands for High-Level Data Link Control, and SDLC stands for Synchronous Data Link Control. The ANSI ADCCP and CCI'IT LAP-B
protocols are other examples. HOLC and SDLe are quite similar, especially c:oncemiDg our needs forpexformance . CODSideration. They are described in some detail in Stallings [25], Meijer [21], and Hammond [9]. We will take a bigb.-levellook at HOLC below as an example. Under HOLe, a message is broken up into packets, or frames. Each frame is enveloped with syncbroDization, control, and eIrOI' protection information as shown in Figure 5-10a. Frame elements include: • Leading and trailing flag tields that provide synchronization. Each flag is an eigbt-bit field CODtaining the bit sequence 01111110. FLAG
IADDR£"I~I 8 BITS OR MORE
8 BITS
8 BITS
OR MORE
I I
DATA
.'
FCS
16
VARIABLE
or 32
81TS
HDLC FRAME FORMAT (Q)
2!
45678
N(S)
I
0
S
I
0
U
I
I
'.
P/F
N(R)
S
PIF
N(R)
M
P/F
II
N (S)..
SEND SEQUENCE NUMBER
N (Rl .. PI F..
RECEIVE SEQUENCE NUMBER POLL I FINAL BIT
S
SUPERVISORY FUNCTION:
II
RR RNR RE.. SRE..'" M
..
RECEIVE READY RECEIVE Nar READY RE..ECT SELECTIVE RUECT
UNNUM8ERED FUNCTION
CONTROL FIELD (b) Fipre 5-18 HOLe.
F1.AG
8 81TS
Communications
162
Chap. 5
• An address field of eigbtor more bits to identify the IeCipient of the frame. • A control field of eight or more bits that defines the type of frame (information, supervisory, or uanumbered). • The data field, which may be of any length (in some implementations, it is constrained to be a multiple of eight bits). • A frame check sequence (FCS) field that contains a 16-bit or 32-bit eRe character for error detection. The flag fields provide synchronization by including a unique bit sequence in each frame. The UDiqneness of this flag sequence IDIlSt be preserved in that it must not appear in the RSt of the frame. Should a sequence of six Is that can be misinteIpreted as a flag field be found in the frame, the sequence is broken up by a technique caJled bit stuffing. The transmitter simply inserts a 0 after every sequence of five Is (except, of course, in the flag fields). The receiver, upon recei$.g five Is, checks the next bit. If it is a 0, the receiver deletes it. If it is a I, and the seventh bit is a 0, the receiver interpIets the sequence as a flag field (seven Is sigDal a special abort condition). The control field provides for three frame typeS:
1. I1Tjo17fl4titm frames (I-frames), which cmy the data. 2. Supervisory frames (S-frames), which provide flow conttol and error control. 3. U1I1UI1IIberedframes (U-frames), which are used for a variety of channel control functions. The information and supervisory frames are used for the estabJishment and eaor recovery functions in which we ate inteIested. Many of these functions ate implemented in the frame's control field, as shown in FIgUre S-IOb. The first ODe or two bits cIe.fiDe the type of frame to follow. If tbis is an information frame, a pair of tbree-bit sequence numbers is provided. One sequence number, N(S), specifies the sequence IlIIIIlber of the CUlmlt frame being sent. The 01her, N(R), specifies the sequence Il1IIIIber of the DeXt frame aDticipaIed In other wonts, N(R) tens the otber end that all previous frames bave·been mceived pmpedy and that the otber end may flush these messages from its baffas. TJms, message acknowledgement is piggybacked onto information pacbIs as they flow tbIOugh the system. If an acknowledgement is due to be sent to the otber end, but DO infomudion frame is available; then a supervisory flame may be seat U1sread. An RR frame (Receive Ready) is seat with the DeXt expected frame number in N(R) if tbis end is in a position to receive a frame. 0d1.erwise, an RNR. (hceive Not Ready) supervisory frame is sent. "Ibis also indicates the next frame to be expected but forees the 01her end to-delay its transmission UDtil a subsequent RR frame is sent. 11ms, RNRIRR. couples provide flow control over the link. If a frame is !eCeived in error, then a supervisory REJ (Reject) frame is seat, indicat,ing in N(R) the frame from which retransmission is to begin. This is the Go-Back-N:
Chap. 5
Bits, Bytes, and Baud
163
protocol. If selective retransmission is desired, the supervisory SREJ (Selective Reject) frame is sent instead. In this case, only the frame indicate4 by N(R) is retransmitted. Note that the sequence number fields are three bits in length and provide sequence numbers 0-7. This meaDS that the window size, W, on the c:lwmel is seven messages. That is. the receiver can get up to seven messages behind the sender before the sender must wait for an acknowledgement. A window size of7 is used instead of 8 to prevent confusion over message numbers. To understand the reason for this, let us consider the following example. Assume that the sender bas sent messages 0 through 7 and is waiting for an acknOWledgement while it is holding these eight messages. It then receives an acknowledgement with N(R) = 0, indicating the next message the receiver is expecting. Is it the meSsage 0 cunently being held by the transmitter. or is it the message 0 which the transmitter is due to send next? This CODfusion is avoided by always )jmiting the window size to one less than the sequence numberrange (the window size may. of course, be further limited by other factors, such as available buffering). Though the HDLC protocol described here limits the window size to 7, an extension to a window size of 127 is available through HDLC. This can be important for bigbspeed.long-delay channels such as satellite channels. Let US now look at the establishment functions built into HDLC. These are c0ntrolled via two fields in the f.nIme, tbe PlFbit in the control field and the address field. The PIF bit is tbe pon-fiDal bit. The host uses this bit to poll a termiDal, the te.nDinal uses this bit to indicate to the host that it bas nothing more to send. The specific temUnal is addressed by the host via the address field. In Older to select a termiDal for transmission, the host simply addresses a message to it via the addIess field. In Older to pon a temIiDal, the host sends a frame COD1aining that termiDal's address with the PIF bit set. If an information flame is due to be sent to the temIinal, the poll bit in that frame is set. Otherwise, a supervisory RR frame is sent. If the termiDal has no daIa to send, it wiB Ietuman RR. frame with the PlFbit set. If it bas daIa, it will mum informadon &ames to the host. ·An but the last I-frame wID have a zero PIF bit. -A onePIF bit in the last flame indicares to the host that the terminaJ bas finished its mmsmissioD.
BITS, Bna, AND BAUD A passing CQ111iiMM1t on somecmnnmricatioa terminology is appmpriate at this point. Let us take a 2400 bit-per-secoocl syacbloDous COIIIIDI.1nic:a 1iDe. .Some refer' to this as a 2400-bit-per-sec:oud line, others as a 24OO-baud line, and still otherS-- as a 300-byte-persecond line. Ale these all equivalent? I thiDk so, but I doubt it. And if that sounds coafusing, it's because we have let ourselves get sloppy with JlODK'Alclature. Let uS first take the term boIId. Baud is tecImica1ly a measme of the number of state transitions per second that ':
164
Communications
Chap. 5
.$X)mmUDication line is capable of acbieviDg while still having the receiver accurately detect the state sequences. In many cases, the line shifts between only two states: one and zero. In this case, baud is the measure of bits per second that the line can bindle.
However, many traDSmission schemes use more than 2 values per transition. A common modulation technique is 4-pbase modulation. With tIJis technique. each transition is to one of 4 states and thus represents 2 bits. In this case, a 2400-baud line supports 4800 bits per second. In general, if each state can be one of M values, tben baud and bits per second are related as fonows: Bits per second = baud x 10l2M The term bits per 8ec01ll1 is not all that clear either. From a purely information theory viewpoint, a bit is a unit of information. TheJ:e is no iDformation cmied in asynchronous start/stop bits. Since a l2OO-bauci asynchronous line (in the loose sense) transmits 120 100bit cbaracte.rs per second, with each cbaracter contaiDing only 8 information bits, is it a 1200-bit-per-second line or only a 960-bit-per-second line? Well then, we say, let us use bytes as our 1DeaSUle. But again, there is no information carried in synchronous SYN bytes nor in control bytes such as STX. ENQ. or eaor control characters. Do we eliminate these from our measure of line capacity? Enough said. Modem usage bas become somewhat sloppy, and all forms are accepted. Let's make sure that our use is clear, either by context or by explicit definition.
LA YEllED PROTOCOLS In the p:evious sections, as we discussed communicatioas from the physical dumnel1evel up to some fairly complex potocols, we described how each lower layer adds its own information to the data in order to coD"nmic:are. An appJication might SIart out with a message to sfzd. This gets passed to a p.tOtOCOl baDdler, which :flames the message with . SIart-of-textaDd end-of-text ideJltifiers and which adds some eaor control and channel COD1rOI information. FiDally, this expmded message is banded to a ttaDsmitter, which may add startlstop bits or SYN cbarac1as to get the data across the line. At the 0Iber eud, the zeceiver S1rips out the sync:broDizadoD bits aad hands _ • message in its protocol euvelope to the protocol handler. The protocol baDdler extracts the message and bands it to the application program. This pmcecIme is diapmmed in Figure 5-11. In effect; the applicaiion progDlI1 at the SOUR:e end is ~ as jf it is connmJl!k:ating dilecdy with the appJi<:ation pogram at the destilJarion eact Each does so by passiDg data between itself and a protocol handler. The details of the protocol haDdler are of DO concem to the application programs. The protOcol handler also ICfS as jf it is 1a1kiDg diIec:dy to its compaDion protoCOl handler. On the one band, each passes data between itself and a mysterious application program. On the 0Iber hand, each excbaDges data with a line handler. The protocol bandler ~ DO information about the imler workings of eithertbe application programs
Chap. 5
Layered Protocols
SOURCE
165
DESnNATION
APPUCATION CREATE MESSAGE
ro-------
APPLICATION PROCESS MESSAGE
PROTOCOL HANDLER
ro-------
PROTOCOL HANDLER
------
UNE RECEIVER
UNE TRANSMITTER
t
I
Iigare 5-11 Layered protoc:ol.
Or the line handlers. Only the interfaces need be defined. However, the protocol handler must have intimate knowledge of and cooperate with its companion protocol handler. The same description applies to the line handlers (the transmitter and IeCeiver). They talk to each other but have no knowledge of higher layers. Thus is born the concept of Iayer~ protocols. Each lower layer deals in a greater abstraction relative to the data being handled. It provides services to the next higher layer by using the primitives of the next lower layer. Each layer acts as if it is communicating with a copy of itself via its own layer protocol. It has no knowledge of higher or lower layers, except for the immediate interfaces. Major networks today are based on layered protocols. The two most common are the Intemational Standard Organization's Open System IntercoDnect (lSOIOS!) and IBM's Systems Network Atchitectwe (SNA). The popular X.2S protocol used widely in packetswitching networks implements the lower three levels of ISO/OS!.
ISO/OSI DlOIe of a model for layered atdlitectDres tban it is a specification. In fact, it is called the Os] Reference Model (ISO [12]). The OSI Reference Model establishes seven layers, as shown in Figure 5-12. Each layer has its own defined mponsibilities and provides services relative to these responsibilities, as shown in FIgUre 5-13, to the layer above it (except for the application layer, which is the highest). It can cx.mummiOite with its peer layer, which is typically in a separate system, via a oonnmmicaUon cJvmnel provided by the next lower layer. Each peer layer has a protocol it uses to inte:rcomnqrnicate; this protocol is of no concem to other layers (though protocols often are designM with other protocols in mind to simplify the system). The lowest layer, the physical layer, has clliect access to the physical c0mmunication cbaImel. We discuss DOW the seven OSI layers.
OS! is
The application layer is the layer at which the application System and network agement functions also reside at this level.
Application layer.
processes execute and at which users interact with the system.
man-
Communications
166 LAYER
Chap. 5
--
7
APPLICATION
6
PRESENTATION
5
SESSION
4
TRANSPORT
NETWORK
2
DATA UNK
PHYSICAL
Presentation layer. The lower layers under the application layer serve to allow diverse applications rumrlng on different equipment to converse wi1h each other. This IeqUites tbat the differences in the way these applicatiODS comrmmic:ate must be made traDspareDt to the applicatiODS. At the highest level, these diffeIences may include the code used, IlUIIlber Iepresemat:ion, compression of IqJetitive data, message structme, and termiDal fmmats. The resolution of these cti1Ia=.ces is the responsibility of the presentll.tion layers and may be CODSidered the syn1aX of the conversation between application layers. . TheIe are tine syntax venions to be considered for the data: tbat syntax used by ~ of the ~ application processes and tbat syntax provided to the presentation layer ~ LAYER N+I
TASK
LAYER N+I
RESPONSE
TASK
LAYER N
RESPONSE
LAYER N
COMMUNICATION
CHANNEL
~---------------------~
Chap. 5
Layered Protocols
167
the channel comprising the lower layers. It is the respoDSl"bility of the pesentation layer at
each encs to convert between the application layer syntax and the lower level channel syntax.
Session layer. The session layer is IeSpODSlble for establisbing a connection between application entities and for managing the session. Session establishment is done at the request of an application and may involve requesting an appropriate channel from the transport layer, establishing the availability of the destination application, identifying the SOUICe application for the destination, and obtamiDg permission from the destination application to communicate with the source application. At the end of the session, the session layer must temlinate the session. Session management includes the enforcing of the interaction between the applications, whether it be fuU-duplex, baJf-duplex, or simplex. It may be responsible for ensuring the proper sequence of messages if the lower levels might interchaoge, add, or delete messages. This may occur especially if the physical c:cmummjcati.on channels fail and are :recovered by lower layers. If system usage is to be accounted for. this is a valid function for the session layer. . . Transport layer. The session layer is aware of wbich application it is to talk to, but it has DO knowledge of where that application is. The establishment of a cbaDnel to that application begins with the tTtZ1lSp(Jrt layer. The primary function of the t.raDSport layer is to provide a uanspment meaDS of data traDsfer for a session. If messages from the session layer need to be broken up into packets. then the ttaDspOrt layer is RSpODSible for disassembling the message into 0utgoing packets and assemb1iDg incoming packets into messages (the PAD function). It interacts with the netwOlk layer to provide flow conuol if necessary. ensuring that DO more data is passed to the Detwcd: tbaD the network can baDdle. The traDspOrt layer also may be responsible for providing a certain class of service derNIDdM by the session layer. Classes of service include priority. delay. and security specifications. as wen as services such as multiple addressing.
Network layer. The network layer Idieves the traDsport layer of any concem over how the systems are ~. It is the uetwork layer that knows the topology of the system, and it is the uetwodc layer's respoasibility to see tbat pactecs submitted by the traDspOJt layer are routed p:opedy to their destiDarioDs. Tbere are a variety ofJOUtiDg medvmisms tbat may be used. Fi%etJ TOUting defiDes a specific path betweaa each pair of eadpoials. Altemate routing defiDes a PrimarY path for each endpoint and one or DlCR altemate paIbs in case of failure or coagestion on the primary path. Dynmnic rtnIdng allows packets to be routed through the netwodc along a path which makes best use of the·networlt facilities at that iDstaDt in time. The network layer is respoDSible for flow control in the network and interacts with
ne
the transport layer to restrict incoming data if it c:aanot bebandled by the networlc. networlc level may or may not be responsible for the proper OIdering of received packets, depeDding upon the characteristics of the cJmmel. It must be mponsive to c1ass-ofservice requestS from the traasport level.
Communications
168
Chap. 5
., Note that the routing of a packet may carry it through several nodes in a network. How does the reference model apply to a node in which an application for this session is not active? The answer is that the networlc layer is responsible for routing, as shown in Figure 5-14. A message originates at an application layer and is broken into packets by the traDsport layer in that system. The packets then flow through the networlc layers of all intervening nodes until they mive at the destination system. There the networlc layer passes the packets to the transport layer, where the message is reassembled and delivered to the application layer via the session and presentation layers.
Data link layer. The purpose of the data liDk layer is to provide virtually errorfree COIDDlUDication across a link connecting two nodes in the network. It must do so in a manner which imposes no restriction on the data (data transparency). As such, the data link layer provides the following primary functions: • Syncbronjzation, so that packets (frames) may be identified.
• Error detection. • Error correction, either through forwud error correction or via retransmission. Note that it must be assumed that a COIIIlection spans several nodes. The data link. layer guarantees error-free communication only across the separate links in the CODDeCtion-it C8DD0t guarantee euor-free operation across the channel. That is why higher' layers ate also involved in eaor control. The HDLC protocol which we have described earlier is a contemporary example of a data link layer.
Physical layer. The pbysicallayer is IeSpODSible for the IIUIDageIDeDt of the :physical communication liDk.. This includes the elec:ttical characteristics of the cba~ LAYER APPUCATION
A
A
PRESENTATION
P
P
SESSION
S
S
TRANSPORT
T
T
NETWORK
N
-r
-y-
N
DATA UNK
D
D
D
D
PHYSICAL
X
X
X
X
~
DATA PATH
. I'ipre 5-14 OSI muldnocIe ~.
MESSA.a
-+PACKETS
Chap. 5
Layered Protocols
'69
and the mechanical specification of the connectors to the clwmel. It also includes the modem interfaces and the functions and usage of modem signals.
SNA The mM Systems Networlc.Archi1ecture (SNA) is a specific implementation of a layered protocol. Though its layers are somewhat different from the ISO R~erence Model, there
is a strong similarity. Under SNA, there are five layeIs. They are shown in Figure 5-15 with their general con:espondences to the OSI model. , Under SNA, there is DO defined physical layer; the existence of a suitable physical connection is implied. The d4ttJ link controll4yer is very much like the OSI data link level. SDLe, which is more or less a subset of the HDLC protocol described earlier, is specified for this level. The path control layer provides services similar to the OS! network layer. It pr0vides logical chaDnels and flow control between entities known as Network Addressable Units (NAUs). Since dynamic routing is sapported at this level, pad1 control is IeSpODsible for the delivery of data units in proper order even if they are IeCeived :&om. the data link control layer in improper order. The transmissioIi control layer' is similar to the OS! traDSpOrt layer except that it haS some responsibilitie related to session management. A session under SNA is a l~
OSI
SNA
APPLICATION
PRESENTATION
FUNCTION MANA_MENT DATA SERVICES
SESSION
DATA FLOW COIITROL
TRANSPORT
TRANSMISSION CONTROL
NETWORK
PATH CONTROL
DATA UNK
DATA UNK CONTROL
PHYSICAL
170
Communications
Chap. 5
.connection between NAUs. Transmission control is responsible for establishing and terminating a session and for maintaining the CODDeCtions required during the session. The d/lIaftow control layer is the other part of session control and coaesponds to the OSI session layer. It is responsible for managing the session, including balf-duplex or full-duplex data low, for ensuriDg all-or-none data delivery, and for bracketing transactions SO that the range of a transaction comprising several messages can be identified. The ftmction mIlIIIlgement dIlta services layer provides services to the end user. It is sDnilar to OSI's presentation layer and spills into the application layer. It provides OSI presentation services such as data !efon:nat:ting and data compression. It also provides network management functions such as network reconfiguration, collection of network statistics, and fault identification and isolation. Refer:ence may be made to Meijer [21] for a more detailed discussion of SNA.
JC.2& X.2S is a protocol that is widely used internationally. his packet-oriented and is therefore used widely in packet-switching networks. It coaesponds roughly to the OSI Reference Model layers 1, 2, and 3 (physical, data liDk, and network). The data liDk layer is a subset of the HOLC protocol described earlier. The network layer is designed specifically to interface easily with the HDLC protocol. The following description of X.2S is a condeusation of material found in Stallings [25] and Meijer [21]. Under X.2S, a user always communicates with a rommunicaIion facility. The user's equipment is called Data Terminal Equipment (DTE), and the comJDUDication facility is called Data Circuit-Terminating Equipment (DCE). F1ple 5-16 shows the use ofX.2S in a packet-switched enviroameat Users connecttbeir DTEs to the packet-switched network via a DeE provided by the network and communicate with that DeE ovec an X.2S line. The DeE is IeSpODSlDle for establisbing a virtual CODDeCtion ~ the switch to the destiDation-user's OCE, wbich ta1lcs to the dNtiDation-user'~
DTE - DATA TERMINAL EQUIPMENT DCE - DATA CIRCUIT-TERMINATING EQUIPMENT P S N - PACKET sWITCHED NETWORK
Chap. 5
Layered Protocols
171
D~
yia an X.2S link. There is no requirement tbat X.2S be used within the packetswitched network itself. Data must be submitted to the X.2S link in packets. The maximum packet length is specitied-l28 bytes is typical. would be more accurate to use "octets" instead of bytes, as the X.2S protocol implies no structure of the data.) Packets of data are enveloped with a tbIee-byte X.2S beader shown in Figure 5-17. The fields are as fonows:
at
• The qualifier bit, Q, which allows the distinction between data flows-for instance, information or control. . • The delivery bit, D, which requests a delivery confirmation from the destina-
tion. • The 1fIDIlulo, a field that specifies whedle.r a window size of7 or 127 is to be used, i.e., a sequence number range of 8 or 128. It also specifies certain fcmnat extensions. • The group field and channel number field, which, when concatenated, allow the designation of up to 4095 logical channels. • The receive-N(R)-and the send-N(S)-sequence numbers, analogous in their use to the same HDLC fields described earlier. • The more-do:ttz bit 00. which signifies tbat the current packet is an inrermediare pactet in the message. The last packet sets this bit to zero.
.
87654
Q
.
D
S
GROUP NO.
MODULO
CHANNEL NO. COMMAND IFCONTROt.. PACKET
N(R)
-z.
2
··
·
N(S)
M
DIC
fA
-
QUAUFIER
D -
DEUVERY
Q
MODULO - SEQ. NO. RANSE = 8 OR 128 GROUP NO. - LOGICAL CHANN.EL GROUP NO. CHANNEL NO. - LOGICAL CHANNEL NO. N (R) - RECEIVE SEQUENCE NO. N (S) SEND SEQUENCE NO. M - MORE DATA . Figure 5-17 X.2S header. DIC - DATA (0) OR CONTROL (I)
172
Communications
Chap. 5.
• The d4uz1control bit (DIe), which speci1ies whetPer this is a. data (information) packet or a control packet.
Control packets have the same format in the first two bytes as a data packet. However, bit 1 of the third byte is set to 1; and bits 2-8 are used as a command. The commands are shown in FJ.gUIe 5-18. 'Their inteJ:pIetation is always in pairs. depending upon whether the control packet was sent by a D1E or a DCE. X.2S supports two types of c:onnectiODS. One is a permanent virtual circuit (pVC). This is a dedicated circuit that IeqWres DO call establishment or discoJmection. The other is a virtual call (YC). which establishes a virtual cireoit and then discoDnects it at the end of the call. The establishment and discoDnection procedures are shown in Figure 5-19. using some of the control packets listed in Figure 5-18. The originating OTE first sends a Call Request packet. which is passed tbrough the network and received by the destination DTE as an Inc:oming Call packet. If it can handle the call. the destination OTE IeSpODds with a Call Accepted packet, which is delivered to the originating OTE as a Call Connected packet.
At this point, the two O'IEs can communicate. The message transfer procedures of the HDLC protocol described earlier are used for data communication. When one D1E is ready to clear the call, it sends a Clear Request packet, which is received by the other DTE as a Oear Indication packet. It responds by remming a Clear Confirmation packet. It is interesting to note the multiple encapsulations of data in layeJed protocols. using X.2S as an example. In FJ.gUIe 5-20 a data packet is shown as it is received for traJ1smis.. sion over the physical channel. The data message (which itself may have bigher-level headers and 1rai1ers) is appended with an X.2S header at the uetwork layer. This packet is then traDSfeDed to the data-liDk layer, whae the HDLC envelope is added (as shown in Figure 5-20). At the receiver, the data-link layer strips the HDLC envelope from the packet. The X.2S envelope is stripped by the netwclIk layer before sending the packet up the layer structme.
.aMGE 1'JIAIISFSI1'EIIFOIIIIANCE Let us DOW look at some performance CODSideratioDs for message U'aDSfer. In the followiDg sections, we CODSider 1he tiDe efficiency and 1DDSit time for both balf-duplex and fall-duplex message traDsfer procedures.
Ilalf..DapIex •• ssage Transfer Efficiency The efticieDcy of message transfer is affected by two CODSideratioDs: 1. The overhead created by the protocol for message identifi.cation and error detection.
Chap.S
173
Message Transfer Performance FUNCTION
CALL SETUP
CALL CLEARING
DATA AND INTERRUPT
FLOW CONTROL
DeE TO DTE
DTE TO DCE
INCOMING CALL
CALL REQUEST
CALL CONNECTED
CALL ACCEPTED
CLEAR INDICATION
CLEAR REQUEST
DCE CLEAR CONFIRMATION
DTE CLEAR CONFIRMATION
DCE DATA
DTE DATA
DCE INTERRUPT
DTE INTERRUPT
DCE INTERRUPT CONFIRMATION
DTE INTERRUPT CONFIRMATION
DCE RR
DTE RR
DCE RNR
DTE RNR DTE RE"
RESET
RESTART
RESET INDICATION
RESET REQUEST
DCE RESET INDICATION
DTE RESET CONFIRMATION
RESTART INDICATION DCE RESTART CONFIRMATION
RESTART REQUEST DTE RESTART INDICATION
2. R.ettansmission of messages in emlI'. StalliDgs [25] presents a very UDdastandable description of message-ttaDsfe efficieDc:y, wbich we will follow in priDc:iple. Let as make the tonowiDg de1iDitions:
U = message tmDsfer efficieDcy. tift
= average time teqDiled to transmit a messaae (sec.).
tp,.o
= ovedIead time per messaae imposed by" the protocol '(sec.).
k,. = the average IlUIIlber Oftimes tbai a message must be seat (IetraDsmission factor) due to eaors and the successful traIISIDission.
Then 1he efJicieoc:y of message traDsfer, U, can be expressed as U-
- k,.(r,.
tift
+ tp,o)
(5-1)
Communications
174
Chap. 5
PSN DTE
DCE
~
DCE
DTE
--
-- --CLEAR CONFIRMATION
- - -
CLEAR CONFIRMATION
That is, it is the lido of the basic message time, tm, to the actIial1ime spent deliveriDg the message, 1;(tm + IJIPIt). Note 1bat errors mprotocol c:oatrol 'bloclr;s (ACK, NAK) are igaoJed, as these are typicaJly sma1l blocks less subject to error. Also DOte fiom equation 5-1 dial: 1he actual C:O""'nmicadoD c:banrte11iD:ie used to send a message is t,,/U. To 1IIIdersrand protocol ovedIead, let us trace 1he path of a balf-daplex message traDsfer using Figm: 5-21. The fust step is to ttaDsmit the message; this zequires a time, tm. as c1efiDed above. The message must pmpagate over the cormmmication etullmel. To do so mquiJ:es a time, tp",p. It must then be processed by the receiver, t,n:I, wbic:h will traasmit an ACK if 1he message is c:o.uect. The ACK will requD:e a traDsmission time of ,_. FmaIly, the ACK will propagate to the traDsmittcr (a time of ~) and wm be processed by the traDsmitter,~. At tbis point, message 1raDSfer is complete.
Chap. 5
Message Transfer Performance
HEADER
"
HOLe
175
TRAI LER
~
Iii
~D3
Figure 5-2t X.2S data eucapslIlation.
From this description, the protocol overhead time, t,-, can be expressed as tpro
= 21"". + tp1'C + ta
(5-2)
where tpl'OfJ
= one-way chaunel propagation time (sec.),
processing time for the message and for the ACK (t~l + tp1'Ci),
tp1'C .=
t_ = time to send the ACK. Let us define the following term,
a, such that
a = !f!! =
tpr'!f
+ (tIM" + t~
21",
tIll
(5-3)
That is, a is the avenge protocol delay per tmJsmjmon and is normaUzeci by the message time. There are two traDsmissiODS per message 1mDSfer: one for the message itself and one for the acknowledgement. Using equation 5-3, equation 5-1 can be rewritten as
U
1
=1;(1 + 2a)
(5-4)
The mtraasmimon :factor, k,., can be evaluated as follows: Let
Pb
= probability of a bit error in the cnnpmmic:ation charmel,
Pm = probability that a message c:ormrins M = message size, in byles, and H = message ovedIead, in bytes. PROPAGATION
~ ~
DELAY
Z
tm
~ PROCESS ACK . tprc2
..
PROCESS MSG
tprop
PROPAGATION _
DELAY
Z
XMIT
ACK tprop
tprc I
ODe
or more eaors,
Communications
176
Chap. 5
•Typical values for Pb are 10-5 for dialed lines and 10-6 for dedicated lines. We assume that bit errors are random. and independent. This is hardly the case in typical communication cluumels, wbeIe errors tend to occur in bursts. However, the assumption of independence is conservative and will lead to somewhat higher message error rates than will actually be observed. We also assume that our error detection algorithm is perfect. The message overbead of H bytes includes header and trailer data which frame a message. Examples of this overhead will be given later. For synchronous transmission, the to1al number of bits in a message is 8(M+H). (The multiplier will be 10 or more for asynchronous transmission.) The probability that a particular bit is euor-free is (I-Pb)' The probability that all bits are error-free is the probability that the message is error-flee and is (I-Pb)8()4+H). Thus, Pm
= 1 - (1 - Pb)8()4+H)
(5-5)
A message must always be transmitted at least once. It will be transmitted a second time with probability Pm. a tbiId time with probability Pm2 , etc. It will be transmitred on the average Ie, times, where Ie, = 1 + Pm
+ p",2 + ...
or Ie,=--L
(5-6)
I-Pm
Substituting this IeSUlt into equation 54 yields U
=1 -
Pm
1+2a
= (I-Pb)8~+H) 1+2a
(5-7a)
If the bit error probability is sufficiently smaIl, then (I-Pb~+B) - 1-8(M+H)Pb
and
u- 1-8(M+H)Pb 1+2a
.
(5-7b)
This approximaDon is valid if8(M+H)Pb «I. For values of Pb in thecxder of IO~6 and for reasonable message sizes, this appIOYimarion holds.
The IDIlysis of the efficiea.cy of a fu11.duplex protocol is compJicated by the fact that there are two cases to CODSide:r, both based on the extent of the protocol delay, a. To understand these cases, we must add one additioDal concept to our knowledge of protocols: the window, in particular, the window of size w. As we memioDed earlier. the ttansmitter must be able to buffer messages so that it
Chap. 5
Message Transfer Performance
177
error.
can Ie1IaDSmit from the point at which the receiver received a message in Furthermore~· the receiver must provide equivalent buffering if selective retransmission is used. But how much buffering is required? The number of messages that the transmitter can get ahead of its acknowledgements from the receiver is called the window. It, in fact, is specified by the protocol, as the window size !elates to the size of the message number that can be specified in an ACK or NAK message. (See Stallings [25] for a mOIe thorough discussion of this.) A typical window size, W, is 7, though larger windows are sometimes used for channels with very long propagation times. Since the diffexenc:e between the transmitter and receiver can be no greater than W messages, this is the amount of buffering requiIed. Messages are held in the buffer until they are acknowledged, at which time they are Bushed. (For selective Iettansmissic:m, messages must remain in the xeceive buffer until all pxevious messages have been properly received.) Now let us assume that the time IeqUUed for the ttansmitter to send W messages is greater than the total propagation time to receive an acknowledgement. That is, Wtm a=: (1 + 2a)tm
Then the transmitter will always receive an acknowledgement to a message before it fiUs up its buffers with unacknowledged messages. Tbel'efore, the triDsmitter can continually use the channel for transmission; efficiency, U, is affected only by Je1mJsmissions: U
= 11k,. , Wa=: 1 + 2a
However, if the transmission time for W messages is less than the total propagation time, then after W messages are sent, the transmitter must wait a time (1 +2a)tm - Wt". before it can start sencting again. 'Ibis cycle willlepe8t every W messages. Thus, ODly W messages will be sent every (1 +2a)tm secxmds, and W U = k,(1 +2a)' W < 1+2a Restating the above xesults, tile efficiency for a fall duplex protocol can be expressed as
U = lIAr W
= ";"'k,(":::1";':'+':"2a~)
Woe: 1+2a W< 1+2a
(5-8)
It remaiDs DOW to evaluate Ar. For the seled:iveleIDDSmission case, the pmcedme is simple, since only the message in error needs to be retransmitted. This is equiva1eut to the balf-duplex protocol case, and equation S-6 holds. Thus, for selective TetrfJl'lSl'l'lissi U=I-Pm
= W(I-p",) 1+2a
Wa=: 1+2a (5-9)
W< 1+2a
178
Communications
Chap. 5
.. The technique in which all blocks after the block in. eIIOI' are retransmitted is called the Go-Back-N teclmique; let us evaluate its efficiency. With this technique, N messages must be retraDsmitted. In a manner similar to the derivation of equation 5-6, we note that a message must always be transmitted once. With probability Pm. N messages will have to be retransmitted; with probability Pm2, N messages will have to be transmitted a second time, and so on. (We ignore here the probability of eIIOI' in other than the first message being retransmitted, as this will simply initiate a new retransmission sequence). Thus, the retransmission factor, k,., is
or (5-10) For the case in which the trausmission time of W messages is greater than the total propagation time to JeCeive a message (W ~ 1+2a), the number of messages that the transmitter leads the receiver by is N = (l+2a). This is because it will take (1+2a)tm secouds for au acknowledgement to be returned to the transmitter for a message that is just begiDDing its ttaDSmission. DuriDg this time, N = (1 +2a)t,,/tm messages will be seut, including the message for which the acknowledgement is being retumed. If that acknowledgement is a NAK., then (1 +2a) messages must be rettaDsm.itted. Likewise, if W messages take less time than the round-trip time to receive the acknowledgement (W < 1+2a), then the traDsmitter must pause after W messages aDd must wait for the acknowledgement. If it is a NAK, then N = W messages must be retransmitted. SubstitutiDg these values for N iDto equaDon 5-10 to obIain kn and then combiDing that J:eSUlt with equation 5-8 yields, for Go-&cIc-N retTtl1IS1fIiss
U=
I-Pm 1+2Dpm
=
W(l-p"') (I-Pm+Wp")(l+2a)
Wi!: 1 + 2a
(5-11) W< 1 +2a
Me•• age TIWISit n.... From a system peJ.'formana: viewpoint, we ale as iDIaested in the tnmsit time of a message as we are in line efticieDcy. Let us evaluate message ttaDsit time for the tIDe cases discussed above. Let
r:.. =message traDsit time (sec.), or the average time reqWrecl to send a message aDd to have it successfully received. For baJf-duplex protocols, we note that the first copy of a message passes through the receiver in a time t".(1 +a). Subsequent retraDsmissions reqWre a time t".(l +2a).
Chap. 5
Message Transfer Performance
179
Since. the average number of retnmsmissions per message is p"/(l-p"J, then
t;" = t",(1+a)
+ t",(l+2a)p"/(1-p,,,)
This can be rearranged as follows:
~ = (1+2a) - a(l-p"J t I-Pm
",
= (1+2a I-Pm
a)t m
(5-12)
Using equation 5-7a, this becomes the holf-duplex message transit time,
(5-13)
t;,,=(i,-a)tm
This intuitively obvious result is that the message transit time is the total round-trip message time, t"/U, minus one propagation time (since the successfully received message does not have to await an acknowledgement). For the case of selective retransmission with a full-duplex protocol, the same argument holds. An euor-free message will xequire t",(1 +a) time to pass tbrough the receiver. A retransmitted message occurs with probability p"/(1-p",) and requires a time of t",(1 +2a). Thus, equation 5-12 holds. However, this cannot be written in the form of equation 5-13. since the efficiency, U, of a selective retransmission line is a ctifferent expression, one given by equation 5-9. We therefore use equation 5-12 forthejull-duplex selective retransmission message transit time:
~ = ( 11-p", +2a _ a)tm
(5-14)
For the balf~ex and selective retransmission cases, we have DOt been concerned with the effect of eJ'!OIS on other messages. When a ccmnmmjcation line accepts a message for uansmission, it is sent fonbwith and is received by the receiver in a time indepeDdent of other errors. Ettor zetraDSmission time affects the average service time t,,/U, and thus is a factor in the communication line load. This will affect the queue delay but not the ttaDsit time for a message. However, for the Go-Back-N protocol, the situation is not so simple. If a message must be retraDsmi1ted, aBN-1 messages foDOwiDg it must also be retransmitted. 1l:ms, N messages axe delayed by a time r".(1 + 2a)p"/(l-p"J, and the average message traDsit time is
t;. = t,..(1 Using the
+ a) + Nr".(1 + 2a)p"/(1-p"J
maDipul8tion tbatled to equation 5-12, this can be written as t;' = [
(!:;:)(1 +
(N - l)p"J - a
]rift
As was done intuitively for the half-cluplex case of equation 5-13, equation 5-14 for selective retnmsmission can be inte.tp!eted as the l'OUDd-trip delay of an average message minus one propagation time, a, since the message receipt does not have to wait for its acknow~. For the Go-Back-N protQCOl, the average round-trip message time,
Communications
180
Chap. 5
..~luding error retransmissioDS, is inaeased by N - 1. retransmissions for p", of the time.
As noted earlier for this case:
N= 1+2a
We: 1+2a
N=W
W< 1+2a
Thus, for full-duplex Go-Back-N message transit time: t;" = t;" =
[(! ~!:)
(1
+ 'lap",) - a }",
We:I +2a
[(:~!:)(1 + (W-l)p",) - a Jr",
(5-15) W< 1+2a
h is important to note that if a COIDIIlUDication line is to be coDSidered a server serving a queue of waitiDg messages, then its service time is its effective cbamlel utilization, t"JU, not the message transfer time, t;".
Let us look at a typical line and evaluate line utilization and message transit time for the above t:bree protocols. Our example line will be a long-haul dedicated full-duplex synchronous line with the following parameters:
= bit eaor rate = 10-s M+H = message 1eDgth = 300 bytes tprop = propagation time = SO JDSeC. tpre = pmcessing time = 20 D1SeC. S = line speed = 19,200 bitslsee. = 2,400 bytesIsee. '- = ackaoWledF, packet time = 6 bytes @ 2,400 bytesIsec = 2.5 1DSeC.
Pb
. 1m
W
= message time of 300 bytes @ 2,400 bytesIsec. = 125 D1SeC.
= window size for fuIl-duplex protocols = 7
From equation 5-5, the p:obabDity of a letrallswission, Pm, is
Pm = 1 - (1 - 10-5)
8(300)
= ,(11.37
From equation 5-3, the avemge nonnaJi'ed pmtocol delay, a, is
a
= SO + (20+2.5)12 = 125
An .~7
Thus, W > 1+2a; that is, a full-duplex acknowledgement will be tetumed before the window is exbaustect
Chap. 5
EstablishmentlTermination Performance
181
message;
. U~ these values, Table 5-1 is CODStrUcted. It shows the line efficiencies, transit times, and communication line service times for all three protocol classes as well as for an ideal line, i.e.,
DO
delay and no errors.
TABLE 5-1. MESSAGE PROTOCOL EXAMPLE t;' Message traDSit time (msec.)
1.0 0.493 0.976 0.954
125 192 . 192 198
t"lU LiDescnice lime (mscc.)
125 2S4 128
131
Note the relationship between the message transit time and the line service time. For a baJf-duplex line, the service time is much grear.er than the transit time, since the line must be held until an acla10wledgement is received, though the message has akeady been delivered. For full-duplex Jines, messages are DOt delayed for prior acknowledgements. The next message is sent before the prior message may have propagated to the receiver. Thus, service time is less than transit time. The superiority oftbe full-duplex protocols is appment. Though the message transit time is S1Jbstantjally the same for all cases, full-duplex protocols make use of the line twice as efficieDdy. Thaefore, 1bey can bandle twice as IIIIlCh traffic as the balf-duplex p:otocols underthe conditioDs assumed in tbis example. The efficiency of the balf-duplex line is further c:ompmmised by the fact that traDsDDssion can occm: in only one direction at a time. TheIefore, if the traffic in both diIec:tioDs is equivaleDt, the capacity of a balf-duplex line is mduced by a further factor of two !elative to a fall-duplex line.
The 8Mignment of a channel to a data communication session adds acktiti.cmal ovedIead to message transmissioa. If the session is long, 1bis overhead may be minor. However, for short sessioDs. channel esrabJisJunent aad tennjnatjcm overhead may be quite sigDifi.cant. Two COIDIDOIl cases ate CODSide:Ied by example in the foIlowiDg sectioDs. iii the first m:ample, a dedicated line ~ two users who are peers. EithUmay tty to use the line, aad collisioDs must be teSOived. This is an example of a C01l1mltion pIOtoC01. The other example is for a multipoint line in which a master staIion polls the other users (slaves) in an onk:dy JDaIIDer'. CcmnnmricatiQll is allowed ODly between the master and a se1ec:tecl (polled) slave. 1'beIe are many other types of protocols in use. The role of master station may be tEansferred between users in some netwod:s. The line may be held for a Ieply in some
Communications
182
Chap. 5
·ases but released after the tnlDsmjsUon of a traDSaction in other cases. The techniques used in the foDowing examples are extendable to these and other protocols. A more complex class of protocols are those that provide multipoint contention. These are discussed in the subsequent section about local area networks.
Poinf-To-Poinf Contention . ..
, .
The first procedure to be discussed is a balf-duplex contention protocol on a point-to-point chaDoel. This is a typical case in TP systems in wbich a temIinal is connected to the TP host via a dedicated line. Usually, the temJiDal will iDitiate the traDSJDission of a request to the host and wiD await a reply. However, on occasion the host may want to send an unsolicited message to the temJinal. An important cbarac.terlstic of this example is that the line is not heavily utilized. Therefore, if either the temJiDal or the host simply seizes the line whenever it wants to send, the probability of colJjcting is quite low. An example of this sort of protocol is povided by the ANSI X3.28 protocol described in ANSI [2]. Subcategoty 2.3 (Two-Way Alternate NODSWitched Point-ToPoint) describes the conteDtion protocol, which typically is used with the X3.28 subcategory B1 message transfer protocol (Message .Associated Blocking with Longitudjnal Checking and Single Acknowledgement). A simplified repaemation of this protocol is shown in F1g1Jre 5-22 via the "railroad" diagram used so successfully in the ANSI specification. This diagram shows the low of control iDfo.rmad.on and data in an 1III8IIlbiguous and compact foml. Actions taken by one user are shown in blocks; actioDs taken by the other user are UDblocked. The maiD low is indicated by heavy lines. Let U$ denote our two users as user A aDd user B. When user A wams to acqujre the channel, it sends an ENQ character (1) (one oftbe control chaIacters). If user B is
AScn
ESTABLISHMENT
I
MESSAGE TRANSFER
I
I
TERMINATION
NEXT MESSAR
S
'---- =- ..III
18
~--i~
."~TOATATC X
Q (I)
XC
(4)
..
(7)
I
NEYlOUS
I·
MESSA8E (3)
(6)
I
1
Chap. 5
EstablishmentlTermination Performance
183
ready.to receive the message, it responds with an ACK (2). If it is not ready, it responds
with a NAK ( 3 ) . . . When user A receives the ACK, the cbannel belongs to it (this is the establisbment procedure). User A then can send one or more messages (4), pansing at the end of each to await an acknowledgement from the receiver (5). If a NAK is received (6), the message must be retraDsmitted. When user A is finished, it releases the channel by sending an BOT chaIacter (7). This termination procedure completes the sequence. At this point, either user may seize the channel. . Of comse, if both users attempt to seize the channel at the same time by sending simultaneous ENQ charaders, a collision occurs. The contention is IeSOlved by having bOth users Wait ctiffeIalt times for iesponse befOre giving up and trying again. The usei with the shorter time-out wins. The probability of a collision can be determined as follows. As defined earlier, let
a
tFOP
= chaDneI propagation time between the two users
Also let
Ra = message rate of user A (messageslsec.). Rb = message rate of user B (messageslsec.).
t. = delay time for user A after a collision. tb
= delay time for user B after a collision.
to. = chameJ aaplisition time for user A (sec.). feb = c:baImel acquisition time for user B (sec.).
One user (say user A) aII'eIIIpG to seize the dmmel by sending an ENQ. On either side of its seizm:e atImDpt time, tbeI:e is a cdtical time slot, ~, durlDg wbich user B may seize the channel and cause a coIlisioIl. If user B seads an ENQ befo1e user A )JUt.witbiD ODe pmpagaDOJl time, user A wm not see it befo1e it sends its ENQ. Likewise, user B wm DOt see user A's ENQ for ODe pmpagatinD time and may sead its ENQdurlng that
time.
The probability that user B wm coDide with a user A seizme attempt is 2tFOP~' LiDwise, the probability that user A will c:oDide widl a user B seizme attempt is
2tp,.Ra.
In die m:at of a coIlisioD, one user wm wiD the time-out race. Let us say that it is always user A. That is, t,.
We DOW can calcuJate the cbannel aaplisition times for user A and user B. In either .case, a wait time of2t,-p always occ:m;s as the ENQ pmpa~ to the other end and as ~ .
184
Communications
Chap. 5
.~CK is returned. Wrth probability 2tpropRz (x = a, b), a .collision Occurs. User A must wait a time td before trying again and is guaranteed success. Thus, the user A channel
acquisition time, leG, is
or tca
= 2tprop[1 + RlAta + 2tprop)], ta < t#l
(5-16)
t_ can be added to the message time, which is calculared according to the pro<:edmes in the previous section. This gives total message time from a performance viewpoint. Total line milization is obtained by also adding in the teDDiDation time required to send an BOT, which is simply another tP"'P time. Following a collision, user B must wait user A's delay time plus its message time. It then must reacquire the channel. Let
'- = message traDSit time for user A Also let
kcz = number of messages sent by user A Then user B's chanuel acquisition time, tcI" is
teb = 2tprop + (2tpropR.)(ta+kot~ + (2tpropR.)(2tP"'P) The first teml is user B's fiISt attempt at channel seizure. The sec:oncl teml Iepresents the delay of user B's mpJeSt while user A seizes the channel (ta + k.t:u>, which occurs with probability It,-,R.. The thb:d term IepIeSeDtS the second acquisition attempt after user A has 1'eleased the channel. This OCCUIS with probability 2t",.R. and takes a time 2tprop.
teb can be written as
Icb = 2t",.[1
+ RJ.ta + 2tp,. + V,.)l, ta < tb .As ali example, COD.Sider the foJlowiD& case:
(5-17)
tP"'P = cfunmel propagaIio.D time = SO m&eC.
Rt, = 1 message per 30 secooc1s (traDsaCtion).
R" = 1 message per 30 seconds (lq)ly). ta
= user A delay time =200 IDSeC.
kcz
= DUJDber of user A messages = 1.
t:-
= user A message time = 1 second (say, a 300-byte message over a 24OO-bitIsec. line).
Then,
Chap. 5
EstablishmentlTerrnination Performance
185
= .101 seconds. teb
= user B cbannel acquisition time
= .104 seconds. Since the ideal time is a round-trip propagation time of 0.1 seconds, the contention protocol behaves well in 1bis case.
Multipoint Poll/Select Another rommcm configuration in TP systems is a multipoint clwmel in wbicb te.rminals are polled by the 11> host. Typical of these protocols are the ANSI X3.28 Subcategory 2.4 and 2.S and IBM's bisync (binary syncbronous, or BSC) protocol. Both are quite sbDilar, 8nd a simplified leplesenta1ion of the bisync piotocOl is shown in Figure 5-23. This protocol suppol1S two functioas. One is polling, in which the host invites one of the ~ to send a message. To do so, the host seuds a poD sequence (1) comprising the tenninal's identification in upper-case characters followed by an ENQ character. If the terminal bas DO message to send, it returns a NAK (2). If the tenninal bas a message to send, it sends that message (3) (ac:cording to some message-transfer protocol DOt shown). Following either event, the host temrinates the process by sending an BOT cbaracter (4). At this point, the host is flee to initiate its next function. The other protocol functiOn is that of selection, wbich is used to send a message to a ~. To do so,. the host sends a selection sequence (S) comprising the temrinal iden~ MESSAGE TRANSFER
ESTABLISHMENT POLL A
A D D R E S S
E N
MESSAGE . TRANSFER
Q
(3)
(I)
SELECT
a
d d r
e s
E
N
Q
(7)
MESSAGE TRANSFER (8)
5(5)
!'ipre5-23 MultipoiDt poDIselect.
TERMINATION
g-0
T
(4)
Communications
186
Chap. 5
.Pfi.cation in lower-case letters followed bY an ENQ. If tbe terminal is not ready to receive the message, itretums aNAl{ (6). If it is ready, itretums an ACK rl), and tbehost then sends the termmal its message (8). Agabl, the message transfer protocol is not shOWD. An alternate selection sequence is called/ast select. With fast select, the terminal is assumed to be ready at any time. The ~n sequence (S) and ~ (~) are sent together, with no lntervening ACK (j) or NAK (6) being returned by die tei1DinaJ.: ..After either event, the host sends an EOT (4) to terminate the process. From a performance viewpoint, the interesting question is the frequency of negative . polls (a poll with DO response) and the consequent poll cycle time. Martin [19] presents a solution to this as follows. Let Q
= number of terminals which have a message to send and which are awaiting a poll.
= total number of terminals on the line. Pen) = probability tbat the nth terminal being polled is the first to IeSpoDd with a mesV
sage. The probability tbat ~ first terminal to be polled bas a message is
P(1) = QIV
(5-18)
The probability that the second temUnal is the one to respond is the probability tbat the first terminal bas DO message and that the second terminaJ does. Thus,
P(2)
= (1 - Q) .JL = Q V-Q V V-I V V-I = P(l) V-Q V-I
Similarly.
. P(3)
= (1- ~)(l- V~l) V~2 _ QV-QV-Q-I V-2
. - vV-r _
V-Q-l
-P(2) V-2
P(JI) = P(}i-l) V-Q-N+2
V-N+I
(5-19)
. Let
M = the number of termiDals 1bat must be polled in Older to get a message.
Chap. 5
EstablishmentlTermination Performance
187
.We know that in the worst case, thefust V-Q terminals will be idle and that the first terminal with a message will be V-Q+ 1. Therefore, the average number of terminals that must be polled to get the first message, M, is V-Q+l
M=
I
NP(N)
(5-20)
N-l
where P(I) is given by equation 5-18 and P(N) is given by equation 5-19. h can also be shown thit
as would be expected. Equation 5-20 gives the es1ablisbmeDt proc:edme ovedlead for polling, since (M-l) negative polls me associated with each message. h is based on the assumption that at least one ten:niDa1 is always active. What about a lightly loaded line which only occasionally sees a message from a terminal-and when tbere is a message. thele is only one terminal on the line with a message? In this case we can expect~ DUIIlber ofnegative polls to be about half the total number of terminals. More prec:isely, the probability that any one tenninal will bave the message is IIV. The average IlUIDber: of terminals that will be polled to get the message once the ~ is ready is, from equation 5-20,
Ai =
f N(!) =.! V(V+ 1) ~ !:!:! V V 2 2
N-I
(5-21)
In order to detennine wbedler to use equatioa 5-20 of 5-21, and in order to evaluate equation 5-20. welleeCl to know the JlDIIIbeio ofwaitiDg terminals, Q. Q will depend upon the line ~ wbich dependS 11pOIl the DepIive poD load, which depends on Q. 'Ibis circ1l1ar relationship leQuhl?$ 1hat we guesS iDi1ially at Q, make a ca1cn1ation, and perhaps iterate to obtain DIOIe accmare msalIs. ODe way to approach this is to use theM/GIl queuing IDOdel as an initial guess. If this malts in a value of Q lesS -than I, use the simpler equation 5-21. Odlerwise, use equation 5-20. Let us illUSllate tbis with an example. Coasider a line :for which V = totalllUlDber of 1mDiDalS = 8.'
1iansadion rate = 1 traDsactiODI15 sec:ondsI1ermiD. TDDSadion message time ~ 1 second, a constant. sent in response to a pon. o
Response message time = 0.5 seconds, a constant, sent via fast select. ,. . ..... tpoll = time to seDd a pollar select seqaeace = 30 DJSeC. tp,..,p = channel propagation time = 15 JDSeC.
188
Communications
Chap. 5
Temlinals send a transaction message of fixed length (say, 300. bytes at 2400 bits! sec., requiring one second to transmit as assumed). The reply is also a fixed-length message (say, 150 bytes at 2400 bits/sec., requiring 0.5 seconds) .. A terminal sends its transaction to the bost in response to a poll. The host responds later with a reply via a select sequence. Every traDsaction is followed by one and ODly one reply. As an initial cut, the load on the line is calculated using just the transaction and reply load:
L
= 8(1+.5) = .8 15
The distribution coefficient for the MlGll model is given by equation 4-18: 1 E(1'2)
K =2
1 (.5)(1)'-
+ (.5)(.5)'-
-:rr = 2 [(.5)(1) + (.5)(.5)]2 = .56
The average number of termmals, Q, waiting for the line (including the one being ser-
viced, if any) is from equation 4-6:
·L Q = l-L [1- (l-k)L] = 2.6
Since Q must be an integer, let us use Q = 3. Equation 5-20 applies, which yields
M
= (.375) + = 2.25
2(.268) + 3(.179) + 4(.107) + 5(.054)
+ 6(.018)
Thus, each successful poll requRs a total of 2.25 poDs (1.25 negative polls plus one successful poll), each xequiriDg(2t.... + tpoa) 60 msec. Since the traDsacIiOD tate is 8115 uausactions per second, poll load is (8115)(2.25)(.06) = .07. When added to the .8 load imposed by the messages. total line load is .87. This load can be used to repeat the above caJcnJariOll UDti1 the iDtegral number of busy termiDaJs IeIDaias CODStaDL For iDsamce. successive itaatioDs for 1bis case give
=
TAllLEN. POWNG EXAMPLE
LoadU.> IunI:ioD
Q
lIIiIial
FiDal
1 2 3 4
3 S
.80
Z1
,g']
4
,as
.as
4
.86
.86 .86
II 2.2S 1.50 . ·1.80 1.80
The iteration has converged Dicely, n:sultiDg in I.S· polls on the avenge being requiIed to receive a message (.8 negative .polls plus.ODe successful poll).
Chap. 5
Local Area Network Performance
189
.Establishment and termination times now directly follow for this example. To establish the channel for an incoming (polled) transaction requjIes 1.8 poll times (which inclUde the message' propagation times by our definition). This is 1.8 x 60 ,;.. 108 msec.Thus, to receive our 1 second message actually requjIes 1.1 seconds due to polling. Termination is effectively transparent, as the EOT termination character can be simply piggybacked onto the next poll 'or select sequence.
LOCAL AREA NETWORK PERFORJlANCE The connection of user terminals to a TP host via a local area network (LAN) is rapidly gaining popularity, especially if the terminal population is fairly close geograpbic:ally to the host. Also, remote termiDal clusters may be interconnected by a LAN and then CODnecred to the host by a LAN gateway. A gateway is another device on the LAN which passes LAN data to the host and which distributes host data to the LAN. We now consider techniques for managing data on a LAN. A local area network is a higb-speed data link generally c:omprisiDg twisted pair wires, coaxial cable, or a fiber optic link (Figure 5-24&). k may span a building, a college campus, or an industrial complex. Users attach directly to the LAN and can COIDIIlDDicate with any other user of the LAN according to a specific protocol. In fact, when one user traDsmits, all other receivers IeCeive that transmission. Tbe tnmsmitted data contams addressiDg information identifying the intended m:eiver(s). In this section we discuss cerIain protocols that are popularly used for LANs. 1bese protocols tend to combine the establishment/termination proc:edures and message transfer procedures of classical protocols and for tbis le8SOD are trealed separately. 11 is noted that the family of IEEE 802.x standaIds describe CODteII1pODIIy LAN
protocols.
_uItipoint ConteIItioa: CSV...lCD .
. '
In a pmrious section we CODSidered poiDt-to-poiDt COD1eDtion pmtocols. in which either one of the two terminals Could attempt to seize the tiDe and traDsmit. In the eveat of a collision, proc:edures were establisbed so that each_ ii1;na) would back off and try again. at diffcmlt times to !eSOlve the cOmamon. Tbere is 8n iwpoI1aDt class of Uaultiuser local area netwodts in wbich an users conumd for· tbe netwOJk. Various procedmes are used. to zesolve CODtlids One of the popular LAN content:i.oD protocols is a potoco1 known as CSMAICD, which SIaDds for Carrier-Seus