Advances
in COMPUTERS VOLUME 45
This Page Intentionally Left Blank
Advances in
COMPUTERS Emphasizing Parallel Programming Techniques EDITED BY
MARVIN V. ZELKOWITZ Department of Computer Science and Institute for Advanced Computer Studies University of Maryland College Park, Maryland
VOLUME 45
ACADEMIC PRESS San Diego London Boston New York Sydney Tokyo Toronto
This book is printed on acid-free paper. Copyright 0 1997 by ACADEMIC PRESS All Rights Reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Academic Press 525 B Street, Suite 1900,San Diego, California 92101-4495,USA http:llw ww.apnet.com Academic Press Limited 24-28 Oval Road, London N W 1 7 D X ,UK http:llwww.hbuk.co.uWapl ISBN 0-12-012145-X A catalogue for this book is available from the British Library
Typeset by Mathematical Composition Setters Ltd, Salisbury, UK Printed in Great Britain by Hartnolls Ltd, Bodmin, Cornwall
97 98 99 00 01 02 EB 9 8 7 6 5 4 3 2 1
Contents CONTRIBUTORS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . PREFACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ix xv
Control in Multi-threaded Information Systems Pablo A . Straub and Carlos A . Hurtado
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Process Model Control Specification . . . . . . . . . . . . . . . . PetriNets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Simple Control Property . . . . . . . . . . . . . . . . . . . . 5 . A Theory of Threads of Control . . . . . . . . . . . . . . . . . . 6 . Applications of Thread Theory . . . . . . . . . . . . . . . . . . . 7 . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix: Proofs of Theorems . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1. 2. 3. 4.
2 7 13 17 25 33 46 47 50
Parallelization of DOALL and DOACROSS Loops . a Survey
.
A . R. Hurson. Joford T . Lim. Krishna M Kavi and Ben Lee
1. 2. 3. 4. 5. 6. 7.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Loop-scheduling Algorithms for DOALL Loops . . . . . . . . . . Comparative Analysis of DOALL Loop-scheduling Schemes . . .
DOALLLoopSchedulingonNUMAMultiprocessors. . . . . . . Comparison of Affinity-scheduling Schemes . . . . . . . . . . . .
DOACROSS Loop Scheduling . . . . . . . . . . . . . . . . . . . Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
54 55 59 63 67 71 90 101
Programming Irregular Applications: Runtime Support. Compilation and Tools Joel Saltz. Gagan Agrawal. Chialin Chang. Raja Das. Guy Edjlali. Paul Havlak. Yuan-Shin Hwang. Bongki Moon. Ravi Ponnusamy Shamik Sharrna Alan Sussman and Mustafa Uysal
.
1. 2. 3. 4.
.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CHAOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compilation Methods . . . . . . . . . . . . . . . . . . . . . . . . Runtime Support for Pointer-based Codes: CHAOS+ + . . . . . . V
106 108 117 124
vi
CONTENTS
5. Interoperability Issues: Meta-Chaos . . . . . . . . . . . . . . . 6 . Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
135 143 148 149
Optimization Via Evolutionary Processes Srilata Rarnan and L. M. Patnaik
1. 2. 3. 4.
5. 6. 7. 8. 9.
Lntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Evolutionary Strategies (ESs) and Evolutionary Programming (EP) Genetic Algorithms (GAS) . . . . . . . . . . . . . . . . . . . . . Extensions to Genetic Algorithms . . . . . . . . . . . . . . . . . Other Popular Search Techniques . . . . . . . . . . . . . . . . . . Some Optimization Problems . . . . . . . . . . . . . . . . . . . . Comparison of Search Algorithms . . . . . . . . . . . . . . . . . Techniques to Speed up the Genetic Algorithm . . . . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
156 160 162 168 177 184 192 193 193 194
Software Reliability and Readiness Assessment Based on the Non-homogeneous Poisson Process Arnrit L. Goel and Kune-Zang Yang
1. Introduction and Background . . . . . . . . . . . . . . . . . . . 2 . Software Reliability and Readiness Assessment . . . . . . . . . . 3. NHPP and its Properties . . . . . . . . . . . . . . . . . . . . . . 4. Trend Testing for Software Failure Data . . . . . . . . . . . . . . 5 . Parameter Estimation for NHPP Models Using Laplace Trend Statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6. Software Reliability Evaluation . . . . . . . . . . . . . . . . . . 7 . Readiness Assessment . . . . . . . . . . . . . . . . . . . . . . . 8. Readiness Analysis of a Commercial System to . . . . . . . . . . 9. Readiness Analysis for an Air Force System . . . . . . . . . . . 10. Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
198 202 214 220
225 234 241 244 254 263 264
Computer-Supported Cooperative Work and Groupware Jonathan Grudin and Steven
E. Poltrock
1. TheCSCWForum . . . . . . . . . . . . . . . . . . . . . . . . . 2. Research and Development Contexts . . . . . . . . . . . . . . . 3. From Small-Group Applications to Organizational Systems . . . 4. CSCW in North America, Europe and Asia . . . . . . . . . . . . 5. Groupware Typologies . . . . . . . . . . . . . . . . . . . . . . .
270 272 276 278 282
CONTENTS
vi i
6. Communication Technologies . . . . . . . . . . . . . . . . . . . 285 7 . Shared-information-space Technologies . . . . . . . . . . . . . . 291 304 8 . Coordination Technologies . . . . . . . . . . . . . . . . . . . . 9 . Challenges to Groupware Development and Use . . . . . . . . . 309 10. New Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . 311 11. Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . 313 314 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Technology and Schools Glen L . Bull
1. 2. 3. 4.
Technology and Schools . . . . . . . . . . . . . . . . . . . . . . 322 Trends in Educational Computing . . . . . . . . . . . . . . . . . 323 Diffusion of Innovation . . . . . . . . . . . . . . . . . . . . . . . 335 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352 354 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
AUTHORINDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
357
SUBJECTINDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
365
CONTENTS OF VOLUMES IN THISSERIES
. . . . . . . . . . . . . . . . 377
This Page Intentionally Left Blank
Contributors Gagan Agrawal is an assistant professor of Computer and Information Sciences at the University of Delaware. He received a Bachelor degree in Computer Science and Engineering from the Indian Institute of Technology, Kanpur, in 1991, and M.S. and Ph.D. degrees from the University of Maryland, College Park, in 1994 and 1996, respectively. His research interests are in compiler optimization techniques, distributed memory compilation, parallel and distributed systems and runtime support design. Glen Bull is an associate professor of Instructional Technology in the Curry School of Education at the University of Virginia. He served as director of Teacher-LINK, a regional K-12 Internet system, and designed (with Tim Sigmon) one of the nation’s first statewide telecomputing networks, Virginia’s Public Education Network (PEN). He is a founding member and past president of the Virginia Society for Technology in Education, and is currently president of the Society for Information Technology and Teacher Education. Chialin Chang received his B.S. in Computer Science from National Taiwan University, Taiwan, in 1987 and his M.S. in Computer Science from UCLA in 1991. He is pursuing his Ph.D. in Computer Science at the University of Maryland, College Park, where he is currently a research graduate assistant in the High Performance Systems Software Laboratory. His research interests include parallel and distributed computing, compiler and operating system runtime supports, and high-performance databases. Raja Das is an assistant professor in the College of Computing at Georgia Institute of Technology, Atlanta, GA. He has worked as a staff scientist at the Institute of Computer Applications in Science and Engineering, NASA LaRC, and as a Research Associate in the Department of Computer Science at the University of Maryland, College Park. His research interests are in the areas of compilers for high performance workstations and parallel architectures, interactive environments and out of core compilation techniques. Dr. Das received a B.S. in Mechanical Engineering from Jadavpur University in 1984, an M.S. in Mechanical Engineering from Clemson University in 1987, and a Ph.D. in Computer Science from the College of William and Mary in May 1994.
IX
X
CONTRIBUTORS
Guy Edjlali is a post-doctoral Research Associate with the Department of Computer Science and the Institute for Advanced Computer Studies at the University of Maryland, College Park. He is working on development of runtime support for heterogeneous environments. Dr Edjlali received a B.S. in Computer Science from the Burgundy University in 1989, an M.S. in Computer Science from the Pierre and Marie Curie (Paris 6) University in 1991, and a Ph.D. in Computer Science from the Pierre and Marie Curie (Paris 6) University, Paris, France in 1994. Amrit Goel received the B.S. degree from Agra University B.Eng. from the University of Roorkee and the M.S. and Ph.D. from the University of Wisconsin, Madison. His Ph.D. was in Engineering with a minor in Statistics. He is a professor of Electrical and Computer Engineering and a member of the Computer and Information Science Faculty at Syracuse University. He also taught at the University of Wisconsin, Madison and was a visiting professor at the University of Maryland, College Park, and the Technical University of Vienna. He has served on the editorial board of Computer. In 1979 and 1980 he received the P.K. McElroy Award from IEEE/RAMS. His current interests are in software reliability and testing, fault-tolerant software engineering, machine learning algorithms, and software metrics. He was a distinguished visitor of the JEEE Computer Society and was elected a Fellow of IEEE for his contributions to the reliability of computer software.
Jonathan Grudin is an associate professor of Information and Computer Science at the University of California, Irvine. He works in the Computers, Organizations, Policy and Society (CORPS) group. He earned degrees in Mathematics at Reed College and Purdue University, followed by a Ph.D. in Cognitive Psychology from the University of California, San Diego. He has been active in the ACM Computer and Human Interaction (SIGCHI) and Computer-Supported Cooperative Work (CSCW) organizations from their inceptions. He is currently interested in identifying factors that contribute to successful use of groupware applications, and in the indirect effects of these technologies on individuals, organizations, and societies. Paul Havlak is a Lecturer in the Department of Computer Science at Rice University in Houston, Texas. As a Research Associate at the University of Maryland from 1993 through April 1996, he collaborated on compiler research at the High Performance Systems Software Laboratory. His research interests include improved compiler understanding and restructuring of programs through language extensions and symbolic and interprocedural analysis. Dr Havlak completed his Ph.D. degree in Computer Science at Rice University, Houston, Texas, in 1994. He designed global
CONTRIBUTORS
xi
and interprocedural methods for symbolic analysis and dependence testing in the PFC vectorizer and the Parascope programming environment, predecessors to the D System. A. R. Hurson is a professor of Computer Science and Engineering at the Pennsylvania State University. His research for the past 14 years has been directed toward the design and analysis of general as well as special purpose computer architectures. He has published over 1.50 technical papers in areas including database systems and database machines, multidatabases, object oriented databases, computer architecture, parallel processing, dataflow architectures, and VLSI algorithms. Dr. Hurson served as the Guest CoEditor of special issues of the IEEE Proceedings on Supercomputing Technology, the Journal of Parallel and Distributed Computing on Load Balancing and Scheduling, and the Journal of Integrated Computer-aided Engineering on Multidatabase and Interoperable Systems. He is also the cofounder of the IEEE Symposium on Parallel and Distributed Processing. Professor Hurson has been active in various IEEElACM conferences. He served as a member of the IEEE Computer Society Press Editorial Board and an IEEE distinguished speaker. Currently, he is serving on the IEEE/ ACM Computer Sciences Accreditation Board.
Carlos A. Hurtado received a degree in Industrial Engineering and a Master in Engineering Science (Computer Science) from the Catholic University of Chile in 1995. He is currently at the Department of Computer Science at the Catholic University of Chile, where he has taught courses on computer languages and formal methods. His research interests include formal methods for the modeling and analysis of information systems, coordination issues, and collaborative work. Yuan-Shin Hwang received the B.S. and M.S. in electrical engineering from the National Tsing Hua University, Hsinchu, Taiwan, in 1987 and 1989, respectively. He is a Ph.D. candidate in computer science at the University of Maryland, College Park, where he is currently a research assistant in the High Performance Systems Software Laboratory. His research interests include parallel and distributed computing, parallel architectures and compilers, and runtime support for sparse and unstructured scientific computations targeted to massively parallel supercomputers. Krishna M. Kavi is a professor of Computer Science and Engineering at the University of Texas at Arlington. For two years he was a Program Manager at the National Science Foundation, managing operating systems, and programming languages and compilers programs in the CCR division. He was an IEEE Computer Society Distinguished Visitor and is currently on the editorial board of the IEEE Transactions on Computers. His research
xi i
CONTRIBUTORS
interests span computer systems architecture (dataflow systems, cache memories, multithreading, microkernels), formal specification of concurrent processing systems, performance modeling and evaluation, load balancing and scheduling of parallel programs. He has published over 100 technical papers on these topics. He received his B.E. (Electrical) from the Indian Institute of Science, and M.S. and Ph.D. (Computer Science) from the Southern Methodist University.
Ben Lee received the B.E. in Electrical Engineering in 1984 from the Department of Electrical Engineering at the State University of New York at Stony Brook, and the Ph.D. in Computer Engineering in 1991 from the Department of Electrical and Computer Engineering, the Pennsylvania State University. Since joining the ECE Department at Oregon State University in 1991, he has taught a number of courses in computer engineering. In 1994 he was awarded the Loyd Carter Award for Outstanding and Inspirational Teaching from the College of Engineering at Oregon State University. His research for the past 10 years has been directed towards the design and analysis of parallel architectures, including numerous technical papers in parallel processing, computer architecture, program partitioning and scheduling, and multithreaded systems. Joford T. Lim is a Ph.D. candidate in the Computer Science and Engineering Department at the Pennsylvania State University. His research interests are in the area of loop scheduling, program allocation, parallel processing, and computer architecture. He has published several papers on DOACROSS loop scheduling. Joford Lim received his B.S. in Electronics and Communications Engineering from De La Salle University, Manila, in 1985, and his M.S. in Computer Engineering from the Pennsylvania State University in 1993.
Bongki Moon received his B.S. and M.S. degrees in Computer Engineering from Seoul National University, Korea, in 1983 and 1985, respectively. He is pursuing his Ph.D. degree in Computer Science at the University of Maryland, College Park, where he is currently a research graduate assistant in the High Performance Systems Software Laboratory. From 1985 to 1990, he worked for Samsung Electronics Corp. in Korea in the Communication Systems division. His current research interests include high performance spatial databases, data mining, and parallel and distributed processing. L.M. Patnaik obtained the Ph.D. in 1978 in the area of real-time systems, and D.Sc. in 1989 in the areas of computer systems and architectures, both from the Indian Institute of Science, Bangalore, India. Currently, he is a professor with the Electrical Sciences Division of the same Institute and directs the Microprocessor Applications Laboratory. His research interests are in the areas of computer architecture, parallel and distributed computing,
CONTRIBUTORS
...
Xlll
real-time systems, neural networks, genetic algorithms, CAD for VLSI circuits, and mobile computing. In these areas he has published over 300 papers in refereed international journals and conference proceedings and has co-edited seven books. He is a Fellow of the IEEE and serves on the editorial boards of ten international journals. He has served as the Program/ General Chair and Member, Steering Committee, of many IEEE sponsored international conferences.
Steven E. Poltrock is a Senior Principal Scientist in the Research and Technology organization of Boeing Information and Support Services. He manages the Workgroup Integration Technology program, leading projects that introduce and evaluate groupware and workflow technologies. He earned degrees in Engineering from the California Institute of Technology, Mathematics from UCLA, and Cognitive Psychology from the University of Washington. He has conducted research in perception, cognition, mathematical psychology, and human-computer interaction. He has researched and written about collaborative user interface design and development practices and about deployment of groupware systems. Ravi Ponnusamy received the Ph.D. in Computer Science from Syracuse University in 1994, and his B.E. in Computer Science and Engineering from Anna University, Madras, India, in 1987. His research interests include parallel I/O, parallelizing compilers, supercomputer applications and performance evaluation. He has been designing and developing toolkits and techniques for High Performance Fortran compilers to produce efficient parallel code for large-scale scientific applications. Srilata Raman is a Senior Staff Engineer in the Unified Design Systems Laboratory of Motorola in Austin, Texas. She holds a Ph.D. degree in Electrical Engineerihg from the University of Illinois at Urbana-Champaign. Her research interests include optimization algorithms, computer-aided design of VLSI circuits, and parallel algorithms for VLSI CAD. She serves in the Technical Program Committee of IEEE conferences and the editorial board of an international journal. She is a member of the IEEE, IEEE Computer Society and ACM. Joel H. Saltz is an Associate Professor with the Department of Computer Science and the Institute for Advanced Computer Studies (UMIACS), and the Director of the High Performance Systems Software Laboratory at the University of Maryland at College Park. He leads a research group whose goal is to develop tools and compilers to produce efficient multiprocessor code for irregular scientific problems, i.e., problems that are unstructured, sparse, adaptive or block structured. He collaborates with a wide variety of applications researchers from areas such as computational fluid dynamics,
xiv
CONTRIBUTORS
computational chemistry, computational biology, environmental sciences, structural mechanics, and electrical power grid calculations.
Shamik D. Sharma received his B.Tech. in Computer Science at the Indian Institute of Technology, Kharagpur, in 1991. He is pursuing his Ph.D. in Computer Science at University of Maryland, College Park, where he is currently a research graduate assistant in the High Performance Systems Software Laboratory. His research interests include compiler and runtime support for parallel scientific applications, operating system support for parallel architectures and distributed computing over the Internet. Pablo A. Straub received a degree in Industrial Engineering from the Catholic University of Chile in 1985. He received his Ph.D. from the University of Maryland at College Park in 1992. He is currently an Assistant Professor of Computer Science at the Catholic University of Chile, where he has been since 1985. His research interests include software engineering, formal methods, information systems, and business process models. Alan Sussman is a Research Associate with the Department of Computer Science and the Institute for Advanced Computer Studies at the University of Maryland, College Park. He is working on developing effective compiler and runtime techniques for parallelizing various types of irregular and adaptive applications, and on various methods for supporting distributed applications with large 1 / 0 and communication requirements. He received a B.S.E. in Electrical Engineering and Computer Science from Princeton University in 1982 and a Ph.D. in Computer Science from Camegie Mellon University in 1991. Mustafa Uysal is a Ph.D. candidate in the Computer Science Department of the University of Maryland, College Park. His research interests include parallel and distributed computing, high performance and scalable 1 / 0 architectures and systems for workstation clusters, operating systems. He received a B.S. in Computer Science from Bilkent University, Turkey, in 1992 and an M.S. in Computer Science from the University of Maryland in 1995. Kune-Zang Yang received the B.S. in Electrical Engineering from TsingHua University, Taiwan, in 1982 and the M.S. and Ph.D. in Computer Engineering from Syracuse University, NY, in 1991 and 1996, respectively. He was a software system engineer at Chung Shan Institute of Science and Technology, Taiwan, during 1984 to 1988. His current research interests are software reliability and metrics, artificial neural networks and pattern recognition.
Preface Advances in Computers, first published in 1960, is the longest running anthology in the computer industry. The goal is to highlight those computerrelated technologies that have the most impact on our lives today. Topics range from software, to the underlying hardware, to how computers affect the social fabric of our society today. This volume in the series is no exception. We present a variety of topics that are affecting the information technology industry today and will continue to have an impact in the years to come. The first three chapters all look at the problem of multiple computer systems. They discuss how to divide a program across several machines in order to allow this parallelism to speed up overall program execution, by simultaneously executing different parts of the program on different processors. In the first chapter, Pablo A. Strdub and Carlos Hurtado discuss “Control in Multi-Threaded Information Systems.” For simple program designs, such as on a personal computer, a computer executes from the first statement of a program until the last statement, and then the program terminates. At any time, only one statement is executing. However, with more powerful machines, in order to solve complex problems more quickly, several processors may be executing at the same time, each processing a different part of the program. Understanding which sections are in control at any time and coordinating the execution behavior across multiple machines is a major design problem in large-scale applications. Straub and Hurtado discuss these control issues and present their theory on parallel control flow. The second chapter, “Parallelization of DOALL and DOACROSS Loops - A Survey” by A. R. Hurson, Joford T. Lim, Krishna M. Kavi and Ben Lee, continues the discussion of parallel program execution that began in the preceding chapter. Most program execution time is spent in executing loops, so mechanisms to allow for multiple processors to simultaneously execute different paths through a loop at the same time would greatly speed up program execution. They discuss the concept of static and dynamic processor allocation via concepts they call the DOALL and DOACROSS loops. In chapter 3, “Programming Irregular Applications: Runtime Support, Compilation, and Tools” by Professor Joel Saltz and his colleagues at the University of Maryland, concepts similar to the preceding chapters are explored. Given data stored as large irregular arrays, what dynamic
xv
xv i
PREFACE
techniques can be developed to allow for efficient processing of this data across networks of such machines? For data that cannot be represented as regular arrays, dynamic programming techniques are more efficient than statically developed optimization algorithms. Using the CHAOS system as a model, they describe algorithms for processing such arrays. In chapter 4, Srilata Raman and L. M. Patnaik discuss genetic algorithms in “Optimization Via Evolutionary Processes.” Their basic problem is optimizing a function for a solution. Typically, a “cost function” is developed, and then a solution with the best cost functional value is chosen. A genetic algorithm is a stochastic search algorithm based upon the principles of biological evolution. They discuss genetic algorithms and then survey several variations of such algorithms and discuss their search properties. In chapter 5 , Amrit Goel and Kune-Zang Yang present “Software Reliability and Readiness Assessment Based on the Non-homogeneous Poisson process.” A major problem in software design is to understand the reliability of the system that is produced. Simply stated, how long will the system execute before it fails due to an error in the software? Various techniques, developed from hardware reliability theory, have been applied to this software problem. Goel and Yang survey many of the common reliability models and then discuss their own extensions to these models using non-homogeneous Poisson processes. Increasingly, with the spread of the Internet, worldwide networks, and intranets within a corporation, software development for a given product may be spread over a large geographical area. Coordinating the activities of this group of individuals to produce a single well-designed product has become known as Computer-Supported Cooperative Work (CSCW). In “Computer-Supported Cooperative Work and Groupware”, Jonathan Grudin and Steven E. Poltrock discuss these concepts and provide a broad overview of the current trends in CSCW developments. In the final chapter, Glen Bull discusses “Technology and Schools.” There is a constant stream of writing decrying the lack of computers in the primary and secondary grades, from kindergarten through twelfth grade (the socalled K-12 grades). However, what would a school do with a computer if it had one or more? In this chapter, Bull discusses many of the options available to the K-12 computer specialist and discusses many of the problems faced in trying to use such technology effectively. Having the hardware is not the main problem; understanding how computers can aid in education is the major effort. I would like to thank the authors for contributing their time and expertise in writing their chapters. This book has taken about 18 months to complete, from the time the authors wrote their manuscripts, had them reviewed, then revised, and then had this book produced. I have enjoyed working with them
PREFACE
xvii
in creating this volume. If you have any suggestions for future topics to discuss, you can reach me at
[email protected]. I hope you find this volume of interest.
MARVINV. ZELKOWITZ
This Page Intentionally Left Blank
Control in Multi-threaded Information Systems* PABLO A. STRAUB AND CARLOS A. HURTADO Depto. de Ciencia de la Cornputacion Universidad Catolica de Chile Santiago, Chile
Abstract Information systems design has traditionally dealt with both data modeling and process modeling. Regarding process modeling, most design techniques based on structured design or object-oriented design specify possible data flows and interactions among components, but are not very precise in specifying system control flow. On the other hand, the introduction of computer networks has enabled technologies like work flow automation and ED1 to coordinate collaborative work in and across organizations, allowing process re-engineering to shorten process times by introducing parallelism. The introduction of these technologies requires a precise specification of control for a parallel process. Moreover, process specifiers are not necessarily computer scientists, nor are they willing or able to learn complex languages. Several languages have been developed to specify control in worwlow systems. Most languages specify control using diagrams similar both to traditional single-threaded control flow diagrams and CPM charts. These languages can represent both choice and parallelism. But this combination of language constructs, required for modeling processes, can introduce control anomalies, like useless work or even worse, deadlock. This paper is a first treatment of multi-threaded control flow in information processes. It presents common language constructs and some extensions, semantics, analysis methods, and a theory of threads of control which is used to analyze process models and to define the semantics for some of the language extensions. The formalization is given using Petri nets.
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Automated Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Process Model Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Control Anomalies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Contents of this Article . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Process Model Control Specification . . . . . . . . . . . . . . . . . . . . . . . . . .
2 3 4 6 7 7
*This work is funded in part by COMCYT through project FONDECYT 1940677.
1 ADVANCES IN COMPUTERS, VOL 45
Copynght 0 1997 by Academic R e v Ltd All nghts of reproduction in any form reserved
2
PABLO A. STRAUB AND CARLOS A . HURTADO
2.1 Basic Control Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 CICN: A Standard Control Language . . . . . . . . . . . . . . . . . . . . . . . 2.3 Advanced Control Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . 3. PetriNets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Place/Transition Nets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Free-choice Nets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Partial Order Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Behavioral Properties and Invariants . . . . . . . . . . . . . . . . . . . . . . . 4 . The Simple Control Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 The Control Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Petri Net Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Behavioral Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 . A Theory of Threads of Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Thread Labels and Threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Threads and Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 . Applications of Thread Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 BaseModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Alternatives within a Thread . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Alternatives between Multiple Threads . . . . . . . . . . . . . . . . . . . . . . 6.5 Unbalanced Connectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Summary: Incremental Case Composition . . . . . . . . . . . . . . . . . . . . 6.7 Dealing with Unspecified Situations . . . . . . . . . . . . . . . . . . . . . . . 6.8 General Unbalanced Connectors . . . . . . . . . . . . . . . . . . . . . . . . . 7. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix: Proofs of Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8 10 11 13 13 14 15 16 17 17 18 21 25 25 27 31 33 34 36 37 38 40 43 44 44 46 47 50
1. Introduction Organizations are distributed. interactive. parallel systems. that handle incomplete and inconsistent information . From the perspective of this definition. it is possible to realize that computer science can help to understand and support work within organizations. for parallelism. interaction. data handling. etc., are clearly in its realm . In fact. in the last decade or so the coordination of collaborative work has been automated by so-called collaborative systems. which are based on techniques loosely identified by the term Computer-Supported Collaborative Work (CSCW) [21] . The idea of process automation traces its origins back to the invention of the assembly line in the beginnings of this century . Taylor’s theories on rationalization of functions within organizations led to the definition of organizational processes. defined as sets of interrelated functions performed by several individuals. Only recently. with the development of inexpensive networked computers. has the possibility of automating coordination of work by many people been realized .
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
3
1.1 Automated Processes Automated processes are usually more complex than similar manual processes because of added cases and parallelism. Automated process models usually have more cases (Le., choices of possible execution paths) than manual processes, because manual processes-followed by intelligent people-can be changed during process execution if need arises. In fact, most procedures from organizational procedure manuals are just a sequence of unconditionally performed steps. While most organizational processes have intrinsic parallelism due to the relative independence of some activities, manual procedures usually cannot take advantage of it, because coordination becomes too complex. Automatically coordinated processes, on the other hand, can safely handle complex parallel procedures. Another dimension of complexity that can be handled in automatically coordinated process models is the structure into models and submodels, with complex call relations, even including recursion. Many organizational processes are fairly rigid or at least there is a preestablished set of possible choices. They are called routine processes. There are also many organizational activities that cannot easily be structured in terms of fixed procedures. They are called non-routine processes. Most processes fall between these two categories, so they are usually executed as routine processes, but may sometimes include non-routine parts. It is not possible to create a meaningful process model for a completely non-routine process. Of course, automation of routine processes is simpler than that of non-routine processes (see Table I). Support for nonroutine processes has been an active line of research and development. Some systems use messaging to handle exceptions by supporting informal communication, which is initiated once the process is off its normal course of action [S].Another approach is to explicitly model the interactions among actors involved in the process [ 141. However, those aspects pertaining to the control flow of a process during and after exception handling have not been treated in depth. In particular, TABLEI ATTRIB~JTES OF AUTOMATEDPROCESSES
Kind of process Attnbute Process definition Tool support
Routine
_
Eimple good
Semi-routine ~ hard fau
_
Non-routine impractical poor
4
PABLO A. STRAUB AND CARLOS A. HURTADO
when processes have parallelism not every state is acceptable because of the possibility of control anomalies, like deadlock.
1.2 Process Model Components To reliably automate a process, the process has to be formally defined, that is, there must be a formal procedure or process model, usually written using a graphical language. The process model is enacted (i.e., executed) creating particular processes from the same model. Process models comprise function, data, behavior, organization, and other process features [lo]. These features can be dealt with separately by a family of related process modeling languages (e.g. Kellner [lo] used the Statemate family of languages [8] to model processes). A complete process model must describe not only the activities performed, but also their sequencing, parallelism, resources, organizational context, etc. Kellner [ 101 identifies four main perspectives of process models: function, behavior, data, and organization; these perspectives cover most process aspects. Kellner’s process perspectives are present in the representation of information processes under new automation technologies, because information processes can be modeled at different levels. In this work we describe these process models using a so-called generic model, which is based on several specific process modeling languages and techniques [7]. In the generic process language, there are four related submodels for a complete process model: control model, context-independent model, organizational model, and context-dependent model. The relationship among these models is shown in Fig. 1. 0
Control model. The control model is basically a graph whose nodes are
Context-dependent model Context-independent model
7,
Grganizationd model scripts role assignment object references
FIG.1. The four submodels of the generic process language.
-
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
5
activities and connectors. Activities represent basic units of work: they are either atomic activities, calls to other models, or special. The control model is a transition diagram with constructs resembling control flow diagrams (CFD) and critical path method (CPM) charts. Control is a subset of process behavior. Process behavior is defined as “when and how they are performed through feedback loops, iteration, complex decision-making conditions, entry and exit criteria, and so forth” [lo]. From the point of view of coordination theory [ 131, control models are the specification of causal interdependencies between activities. That is, the control model represents partial orders between the activities of the different processes for a given process model (as opposed to total orders or sequences, due to parallelism). The control model does not represent functionality, and other aspects like resource sharing, timing, etc., even though all these aspect do determine actual process behavior. Like programming languages, process modeling languages include both basic control constructs-like selection, iteration, and synchronization-and more advanced constructs-like recursive calls to submodels and activity replication-some of which will be described in this chapter. 0
Context-independent model. The context-independent model is an extension to the control model, which adds local data and adds a functional description. That is, this model adds the description of what data is passed from activity to activity, and how data is changed in the activities. This model is independent of the organizational context in which it is executed, not unlike the way in which a program written in a high-level language can be executed in different computers and operating systems.
0
Organizational model. The organizational model includes classes of objects, types of objects, types of object relationships, actual objects and actual relations between objects. Each class has a defined set of operations or methods for objects of the class. There are two distinguished object classes called actors and roles and also a relationship between actors and roles. This model represents resources-people, machines, data-and resource organization-organizational structures, data structures-.
0
Context-dependent model. The context-dependent process model comprises the context-independent process model, the organizational model, and their relationships. This model assigns roles to activities, references to organizational objects, and scripts which call organizational object methods. Executors are related to activities. Object
6
PABLO A. STRAUB AND CARLOS A. HURTADO
references are treated like data. Scripts are mainly used to fetch data from organizational objects before executing the activity and to store data in them after the activity executes; scripts are thus related to activities.
1.3 Control Anomalies As was previously mentioned, the capacity to handle complex processes with parallelism and choice is the main difference between process execution under automatic coordination and process execution under manual coordination. However, even though parallelizing a business process might ‘‘dramaticallyreduce cycle times and the resultant costs” [17], the existence of both parallelism and choices might lead to anomalous behavior, like deadlock or the execution of useless activities. Thus, some process modeling languages constrain the forms of parallelism (e.g., parbegin and parend) [2] that can only describe correct behavior. Alas, languages that do allow general forms of parallelism do not test for incorrect behavior, a notion that is not even well defined. Sequential processes cannot have behavioral anomalies (except infinite loops). On the other hand, parallel processes without choice cannot have control anomalies. Thus, it is not surprising that the usual way to avoid these anomalies is by defining simple-minded process models that inhibit the natural parallelism of activities and abstract away the handling of exceptional cases. Oversimplification is especially relevant when dealing with complex processes. While there are many real-world simple business process models, a study cited in [ 3 ]on the modeling of 23 processes of the US Department of Defense included 17 process models with more than 50 activities, and 3 of them had more than 200. There are three main approaches to find control anomalies: (1) Build a model and then verify its properties. One way to verify is building the space state and exhaustively checking the properties (e.g., a deadlock is an inappropriate final state). Another way is finding net invariants; so-called place invariants and transition invariants can be computed by solving systems of linear equations derived from the net [12]. (2) Build a model that is correct by construction, because all grammatical rules used guarantee correct behavior. Abbati et al. [ 1] presents a grammar that introduces parallelism using a construction similar to parbegin-parend pairs. DeMichelis and Grasso [2] annotates the activities in a defined manner to the same effect. (3) A third approach is using only small models by abstracting models
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
7
and submodels; the intent is that it is more likely to have correct behavior if the models are simpler. For example, [9] suggests models with less than 10 activities. The first approach does not explain why a particular model has control anomalies nor which specific part of the model is the culprit. The second approach works by limiting the forms of parallelism; in particular, it is impossible to model all the parallelism in a PERT chart. The third approach is just a rule of thumb, that may or may not work and inhibits parallelism. Besides, in addition to the need of having correct control in a process model, exception handling poses the additional problem of ensuring correctness after an exception is handled, even if due to the exception the model needs to be changed at run time.
1.4
Contents of this Article
In this article we are concerned with the modeling and analysis of control, the definition of a notion of control correctness, a theoretical framework for these purposes that we call the theory of threads of control, and applications of the theory. Section 2 describes constructs for control specification languages in general and the particular language used in this article. Section 3 introduces those aspects of the theory of Petri nets that will be used in latter sections. The main contributions of this work are in Sections 4 to 6. Section 4 formally defines CICN and its mapping into Petri nets, and then describes a series of control anomalies and control properties that characterize control correctness. Section 5 introduces an algebraic theory of threads of control that explains control anomalies. Two applications of the theory of threads are presented in Section 6: an incremental development method and a new advanced language construct called the unbalanced connector, which is used in the development method. Finally, Section 7 puts these results into perspective.
2. Process Model Control Specification Most graphical languages to specify control flow have similar constructs. Thus, instead of doing a detailed language survey we will analyze language constructs on a small sample of representative languages. Language constructs are classified as either basic or advanced. While this classification might seem arbitrary at first, it is based on whether or not the semantics of a construct can be expressed using place/transition nets (as in Section 4.2) using a bounded number of nodes.
8
PABLO A. STRAUB AND CARLOS A. HURTADO
In most process modeling languages, control is expressed by a directed graph with several kinds of nodes. One such kind of node is an activity node that must be present in one form or another in all languages. For example, control flow diagrams (CFD) have three kinds of nodes: statements, twoway conditionals, and two-way joins' (a case-like statement needs also n-way conditionals and joins). Even though edges are not always drawn with arrow heads, they are directed and we will call them arrows; if the arrow head is not present we will assume the edge is directed from left to right.
2.1
Basic Control Constructs
In this section we present basic control constructs like sequencing, choice, introduction of parallelism, and synchronization. We also present abstraction and simple procedure calls. These constructs constitute a meaningful set in which to write control process models. Models written using only basic constructs are called basic models-although they might be very complex. 0
0
Sequencing. In almost all languages, sequencing between activities is expressed using an arrow. Thus, an arrow from an activity A to an activity B means that B can start execution only after A has finished execution; if A is the only predecessor of B , usually finishing A executions is the only precondition that enables B.2 In fact, this is true of both CFD and the critical path method (CPM), as well as most other modeling languages. Arrows do not express sequencing in data flow diagrams or similar languages like SADT or IDEFO [16]. This is a common source of confusion on the semantics of this kind of languages, which do not specify control flow, but merely constrain it. Choice. There are two common ways to represent choice: by special choice nodes or implicit in activities. In CFDs choice is represented by a diamond-shaped node to split control flow and the joining of two arrows in one (sometimes the arrows join in a dot). Languages like CICN use or-nodes, which might be either or-split nodes or or-join nodes, both of them drawn with clear circles. Informally, the behavior of an or-split is that control reaching the node flows to just one of the outputs, and the behavior of an or-join is that control reaching an input flows through the output.
'Joins in CFDs are implicitly represented as the joining of two arrows, or explicitly represented as a small filled circle. 'As far as control is concerned, i.e., abstracting away from resource utilization and other behavioral conditions.
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
0
0
0
9
A combined or-join-split node is present in several languages. In fact, in several languages in which activity nodes allow multiple inputs and multiple outputs, the activity itself behaves as an or-node, with implicit choice. That is, the activity is enabled by just one input and at the end it produces just one output. Iteration. Iteration is not a primitive construct in graphical languages. It is defined using choice constructs, so it is not discussed here, except to note that the possibility of infinite loops is not a concern in control flow: knowing which branch of an or-node is selected is a behavioral problem, but not a control flow problem. If a loop is forced to be infinite because there is no path that leaves the loop, then this is a control flow problem. Parallelism. Like choice, parallelism is represented either by special nodes or implicit in the activities. CICN uses and-splits to introduce parallelism (a thread of control is divided in two or more parallel threads) and and-joins to synchronize threads (two or more threads are replaced by a single thread). A combined and-join-split node is also usual. In the critical path method (CPM), for instance, all activities behave like an and-node, that is, the activity starts when all its predecessors finish, and upon finalization control flows to all outputs. This semantics is very convenient for CPM charts that express parallelism, but cannot express choice, for there is only one kind of node: the activity itself. Simple abstraction. When processes are complex, it should be possible to abstract parts of them in subprocesses. Conversely, given a process, it should be possible to refine it adding more detail. For example, an activity might be replaced by another process. If a process P has an activity A that is refined by a process P' we say that P calls P'. Thus, complex processes are represented by a set of processes and a calls relation, with one root process that directly or indirectly calls all others3 If the culls relation does not allow cycles it represents simple abstraction. In other words, the abstraction is simple if there are no recursive calls. Simple abstraction has a semantics known as the copy rule [19, page 2881, which is basically the semantics of macro expansion. Most commercial and experimental work-flow systems that support abstraction have this semantics, and thus disallow recursion.
'Lamb [ l l ] recognizes several relations between program elements in addition to the culls relation, like the uses and defines relations. In process models, usually the defines relation is empty (i.e. there are no local definitions) and the uses relation is equal to culls (i.e. all uses are calls and vice versa).
10
PABLO A. STRAUB AND CARLOS A. HURTADO
This is not surprising for two reasons: the semantics is easily understood, and it is not always apparent why a process modeler might need recursion. Even simple abstraction is not very simple when abstracting from a multithreaded process. If an activity A is refined by a process P' it must be the case that there is a one-to-one mapping from inputs and outputs of A to those of P ' , or else there will be dangling arcs once the copy rule is applied. In the particular case that both A and P' have a single input and a single output, semantics is simple (e.g., CICN). When more than one input and output is allowed, abstracting control is powerful but might be misleading. There are several possible ways in which abstraction takes place: there is a fixed or semantics (e.g., Rasp/VPL, Copa), that is, activities are single-input-singleoutput, there is a fixed and semantics (e.g. CPM), there are both or and and semantics, there is a general propositional semantics (e.g. P I ) .
2.2
CICN: A Standard Control Language
Information Control Nets is a family of models developed at the University of Colorado for information system analysis, simulation, and implementation 151. The ICN is a simple but mathematically rigorous formalism that has similarity to Petri nets. ICNs are actually a family of models which have evolved to incorporate control flow, data flow, goals, actors, roles, information, repositories, and other resources. This integrated model is well adapted to the traits of a generic process model as described in the introduction. ICNs have been studied in academia and applied in industrial workflow products. The control ICN (CICN) is a simple, known, and representative language for the specification of control in ICNs models. A CICN is a graph in which nodes are labeled as an activity, as an or-node, or as an and-node. There are two distinguished activities start and exit, without a predecessor and without a successor, respectively. Other activities have one predecessor and one successor. Usually or-nodes and and-nodes have either one predecessor and more than one successor (a split) or more than one predecessor and one successor (a join). An or-split represents a decision point. Graphically, activities are depicted as labeled ovals, and-nodes are depicted as dark circles, and or-nodes as clear circles. As un example, consider the credit application process in Fig. 2, whose activities are explained in Table II. The execution of this process begins with the start node, and then executes activities A, B, C and D in parallel. After both C and D are completed, activity G is executed in parallel with the
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
11
n
FIG. 2. A mortgage loan process model in CICN notation. Activities are represented by ovals and and-nodes are represented by black circles. TABLEI1 ACW?ITIES OF THE MORTGAGELOANPROCESS Activity
Description
Start
Fill out application form Verify creditworthiness of customer Register life insurance Set up expense account Identify property to buy Get legal data for property Verify legal status of property Appraise property and verify value is okay Notify customer of outcome
A B C D E F G exit
sequence E, F. When A, B, G and F are done, the process executes the exit node and stops.
2.3
Advanced Control Constructs
Like some programming languages, some process modeling languages have an array of relatively sophisticated control constructs. Among these advanced constructs there are exception handling constructs, which define complex state transitions, but whose semantics is expressible using P/T nets. We will show in Section 6 how the theory of threads provides a framework to define a very general state transition, called an unbalanced connector. In this section we briefly mention two other kinds of constructs: recursive calls between process models and replication of activities.
12
PABLO A. STRAUB AND CARLOS A. HURTADO
0
Recursion. Consider negotiation processes between a provider and a requester. Using software such as the Coordinator system [14], the state of a negotiation between a person A acting as supplier and another person B acting as consumer can be maintained and transitions can be recorded by the computer. It is usual that as a result of a negotiation process the supplier requests something from another supplier C, that is, the negotiation process is intrinsically recursive. If process models are not recursive, the subordinated negotiation process will not have any formal link to the original process. Changes in the state of one of these processes might affect the other. For example, a renegotiation between B and A might trigger a renegotiation between A and C as a subprocess. With recursion, this trigger is automatic. Implementing recursion requires separate states for the execution of called models; the state of the computation of a model must be linked to the state of execution of the calling model. This is simpler than in a language such as Pascal where separate dynamic and static links are needed, because process models do not have local submodels (much as C functions cannot define local functions). On the other hand, recursion in the presence of parallelism cannot be implemented simply with a stack and a single current instruction pointer, as in most sequential programming languages. The semantics of recursion cannot be expressed using P/T nets, because the copy-rule as semantics of procedure calls cannot be applied [19]. High-level nets can be used to express recursion.
0
Replication. Replication of activities occurs when an activity must be performed several times in parallel, as opposed to several times in sequence. If “several” means a fixed number n of times, it can be modeled by an and-split followed by n copies of the activity followed by an and-join. If n is not fixed, a new construct is needed. In languages like Rasp/VPL replication is denoted by an icon that looks like a stack of regular activity icons. But does replication occur in practice? Yes, for instance, consider a software development process, where a set of modules needs to be implemented by possibly different people. This can be modeled by a replicated activity. Another use for replicated activities is defining a single activity performed by an unspecified number of people, like a meeting. Replication can be implemented in a workflow system by a loop that creates all required parallel processes; to synchronize these processes, an integer semaphore initialized to n might be used. Again, the semantics of replication cannot be expressed with a fixed P/T net, but can be expressed using high-level nets.
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
13
3. Petri Nets Petri nets are a mathematical formalism developed in the early 1960s from the work of C.A. Petri in 1962, in which he formulated a basic theory on the communication of asynchronous components of systems. Petri nets are a generalization of transition-state diagrams from automata theory. Unlike ST diagrams, the global state is defined as the union of local states which enable transitions in the net by themselves. Having a distributed state allows the expression of not only choices but also parallelism within a system. Petri nets have been widely used to describe the semantics of many process modeling languages and workflow systems (see, e.g., [ l ,5,18, 25,28,29]). Its success is due to the possibility of formally analyzing and simulating dynamic aspects of systems. There are many kmds of Petri nets. They are classified in two groups: low-level nets and high-level nets. In both groups, a system model is presented by a graph with two kinds of nodes (places and transitions) and a set of tokens. In the first group, tokens represent boolean conditions that enable the firing of transitions. Upon a firing of a transition, the tokens that enable the transition are deleted and new tokens are created, as if tokens were moving through the net. Important kmds of low-level nets are elementary nets and place/transition nets. In the second group, tokens represent not just boolean conditions, but complex data items or predicates. Important kinds of high-level nets are colored Petri nets, predicate/transition nets, and environment/transition nets.
3.1 Place/Transition Nets Place/transition nets are adequate to model basic control constructs. However, they are not useful to model more complex forms of control like recursion and replication of activities, let alone other process model issues like functionality, timing, resource utilization, etc. In this section we will show some basic aspects of P/T nets, which will be used to base the semantics and analysis of basic control models. A P/T net is a directed graph with two kinds of nodes, called places and transitions. A net is a triple N = ( P , T , F ) , where P is a set of places, T is a set of transitions, and F is a set of edges from places to transitions and from transitions to places, that is, F C ( P x T ) U (T x P ) . Places and transitions are called nodes. A node cannot be both a place and a transition, i.e., P n T = 0. The flow relation F defines for each node x E P U T a set of successors, denoted x', and a set of predecessors, denoted 'x. Nets are drawn as follows: places are represented by circles, transitions
14
PABLO A. STRAUB AND CARLOS A. HURTADO
p2
t2
d
P4
b2
(a)
(b) FIG.3. (a) A Petri net; (b) one of its processes.
are represented by rectangles, and the flow relation is represented by a set of directed arcs (arrows). For example, for the net in Fig. 3(a), p = { P I , P ~ ~ P ~ , P T~ =, P{ f i~? fIz ~t f 3 1 , and F = { ( P i , f i ) , ( f i , P 3 ) 9 ( P 3 r f 3 ) , ( f 3 , P ~ ) (, P 2 9 f 3 ) v ( P 2 r f 2 ) r ( f 2 9 P 4 ) I .
A path in a net N is a non-empty sequence n = x o x l ... x, such that ( x i - l ,x i ) E F , for 1 C is k. A net is said to be connected if every pair of nodes ( x , y) belongs to the symmetric, transitive, reflexive closure of F , i.e. (x, y) E ( F U F - I ) " . A net is strongly connected if for every pair of nodes ( x , y) there is a path from x to y. A P/T system is defined by a net and an initial marking or state, where a marking is a mapping from places to numbers (i.e., the count of tokens in where N = (P.T , F ) is a net and each place). A P/T system is a pair ( N , Mi) M i:P + N, is the initial marking. For example, the initial marking in Fig. 3(a) is M i =( p i + + 1, p2++1,~ ~ - ~0 ~, - 0~ ~, - 0 ) . If in a marking M all input places of a transition t have tokens, the marking enables t. If a transition is enabled at a marking M , t can fire and produce a marking M' by removing one token from each input place and producing one token in each output place. A marlung M is reachable from a marking M' if there is a sequence of firings enabled at M' whose final marking is M.A marking M is reachable if it is reachable from Mi.
3.2
Free-choice Nets
In a P/T net, synchronization is represented by a transition having more than one input place, and choice is represented by a place having more than one output transition. In general, choice and synchronization could be mixed. For example, in the net in Fig. 3(a), choice at place p 2 is interfered with by synchronization at transition f3, for choice depends on the existence of a token in place p 3 . Free-choice Petri nets [4] form a subclass of the P/T-nets, in which choice in a place is not interfered with by other parts of the system. This means that choice and synchronization do not mix, or that choices are in a sense free. A sufficient condition for free choice is that places with more
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
15
than one output transition (where choices are made) are not connected to transitions with more than one input place (where synchronization occurs). While this condition is not necessary for free choice as defined by [4],it is satisfied by the systems considered in this paper. The importance of a net being free-choice is that many interesting properties can be decided in polynomial time.
3.3 Partial Order Behavior A sequence of enabled transitions in a system describes executions of the net. These sequences, however, do not determine causal dependencies between transition occurrences. To describe causal dependencies, an alternative representation of executions is needed, in which an execution is a partial order of firings (as opposed to a sequence, which is a total ordering). This partial order is also described by a net called a causal net. A causal net has no choice nor cycles. A causal net is a place/transition net N' = ( B , E , F ' ) , where B is a set of places, E is a set of transitions, F' is the flow relation, each place has at most one predecessor and at most one successor, i.e. V b E B : # ' b s 1 A # b * G 1, and finally the net is acyclic, that is, there are no paths with two occurrences of the same node. For example, the causal net of Fig. 3(b) defines a process for the system in Fig. 3 ( 4 , where q(b,)= p i . q ( e i ) = t l , d b z ) = p 2 , 4 ( b 3 )= p 3 , q ( e d = t 3 , q ( b J = Ps. A process is represented by a causal net which is related to the execution of a P/T system.
Definition 3.1 (Process) A process is a tuple n = ( B , E , F ' , q ) where ( B , E , F ' ) is an acyclic place/transition-net without choices, q is a total function from elements of n to elements of a P/T net N = ( P , T , F ) , and the following holds: q ( B )c p A q ( E ) , V e € E : q ( ' e ) = ' q ( e ) A q ( e ' ) =q(e)'. The first condition relates places and transitions of the process with those in the system. The second condition implicitly defines F' in terms of F. The initial state in N before the execution of the process is defined by the number of places without predecessors in the process that corresponds to each place in N :
M,(p ) = # { b E B l p = q ( b ) ,'b= 0 } . Likewise, the state reached in N after the execution of the process is defined
16
PABLO A. STRAUB AND CARLOS A. HURTADO
by the number of places without successors in the process that correspond to each place in N .
M,( p ) = #( b E B 1 p = q ( b ) , b' = 0 ) .
3.4 Behavioral Properties and Invariants There are several important concepts and properties related to the behavior of a P/T system. Some of them are defined here. 0
0
0
0
A deadlock in a system is a reachable state in which no transition is enabled. A system is deadlock-free if for any reachable state M there is a transition t , enabled at M . A system is live if for any reachable state M and for any transition t , there is a state M', reachable from M that enables t. A system is n-bounded if for any reachable state M and for any place p , M ( p ) s n. A system is bounded if there is such an n. A system is safe if it is 1-bounded.
A comprehensive study of these and other properties is in [4]. The dynamic behavior of a P/T system depends upon the net structure and its initial marking. The influence of the structure is very important, because it holds under every initial markmg. Moreover, the structure can be studied independently from the initial marking. Invariants are properties that hold in all reachable states of a system. Different kinds of nets have different kinds of invariant predicates over net states. In particular, functions that map places to rational numbers which depend only on the net structure have been studied extensively. These functions determine an invariant property of a system: the weighted sum of the tokens in the net is constant under all reachable states. By a slight abuse of notation, these functions are known as place invariants or S-invariants. Given a net, a place invariant is a rational solution to the homogeneous set of linear equations
c,,.,,I ( P ) = c,,;, I ( P ) where the unknown variable I is a vector whose dimension is the number of places in the net. The fundamental property of a state invariant I is given by the following equation, which defines the conservation property described in the preceding paragraph:
C P EI(p p ) x Mi(p ) = &, I( p ) x M ( p ) = constant where M iis the initial marking and M is any reachable marking.
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
17
S-invariants are strongly related to behavioral properties of Petri nets. Among other things, the existence of an invariant in which all components are positive ensures that the net is bounded. We will see (Section 5) that a special kind of non-numeric invariant is related to notions of correctness in control models.
4. The Simple Control Property Figure 4 shows a simple process model to approve a loan for a home. The first activity, a , is the customer’s application, then and-node x splits execution in two parallel activities ( b , credit approval and c, mortgage approval). After each activity, a choice is made (at or-nodes u and v). If both activities are successful, they synchronize in and-node y and the process proceeds to the exit node, so that the credit might be issued. Of course, not all applications are approved; if the credit is not approved the mortgage approval activity c becomes useless; in a realistic situation this activity is composed of several other (now useless) activities. Moreover, upon finishing the mortgage approval activity and-node y will attempt (and fail) to synchronize with the credit approval activity b. On the other hand, if both the credit and the mortgage are rejected, the process will produce two tokens in the w or-node, both of which will reach the exit node (the process exits twice!).
4.1
The Control Model
A CICN is a directed graph with three lunds of nodes: activity nodes, ornodes, and and-nodes. There are two special activities: the start node and the exit node. A node that is not an activity node is a control node.
Definition 4.1 (CICN) The control model is a directed graph (A, 0, N , start, exit, R ) , where A is a set of activities, 0 is a set of or-nodes, N is a set of and-nodes, start is the start node, exit is the exit node, and R is the flow relation. The set of all nodes is V = A U 0 U N .
FIG. 4. Example of a CICN net with a potentially anomalous loan approval process model:
Or-nodes are represented by white circles.
18
PABLO A. STRAUB AND CARLOS A. HURTADO
The following conditions hold: 0
0 0 0 0
Start and exit are activities, i.e., (start, exit) A . Activities and control nodes disjoint, i.e., A r l 0 = A n N = 0 f~ N = 0. R is a relation on nodes, i.e., R C V x V. Start has no predecessors; exit has no successors. For every node x there is a path from start to exit that includes x , i.e. Vx E V: start R*x A x R* exit
where R * is the reflexive and transitive closure of R. The semantics of CICN can be expressed directly, without the use of Petri nets [ 5 ] . A marked CICN is a CICN along with a function m from nodes and edges to the naturals, i.e., unlike P/T nets, all nodes and arcs might be marked. The initial marking has a single token in the start node. In general, marked edges enable the start of a node and marked nodes represent activities in execution and thus enable the termination of the node. Thus, the initial marking enables the termination of the start node, which will mark the edge following the start node. Or-nodes require one token in one of the incoming edges to start; upon termination they add a token to one of its outgoing edges. And-nodes require one token from each incoming edge to start, and produce one token in each outgoing edge upon finishing. Activities require one token in their incoming edge to start, and upon termhation produce one token in their outgoing edge. While not part of CICN, the most usual semantics for activities with several inputs and outputs is that of an or-node. 4.2
Petri Net Semantics
The behavior of CICN can be modeled by a P/T net by translating each of the elements in the net into part of a P/T net as in Fig. 5 and then connecting these parts. In CICN edges can be marked, hence we translate each CICN edge into two P/T net edges with a place in the middle. CICN nodes can also be marked and have two changes of marking: start and termination. Hence each node will be translated into a sort of sequence: transitionplace-transition. An and-node is translated as a sequence of one transition connected to the place connected to one transition. An or-node with n incoming edges and m outgoing edges is translated into n transitions connected to a place connected to m transitions. A regular activity may be regarded as an or-node; if it has one incoming edge and one outgoing edge it translates into the same as an and-node. The sfurt (respectively, exit) activity has a special translation, as a sort of incomplete activity, because it does not have a predecessor (respectively, successor).
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
CICN
PIT-net
CICN
19
P/T-net
FIG.5. Translation of CICN components into P/T net components.
It is not hard to realize that the resulting P/T net has the same behavior as the CICN, because the correspondence between CICN marlungs and P/T system markings is trivial.
Definition 4.2 (Place/transition net of a CICN model) The place/transition net corresponding to a control model At is the place/ transition net N(&) = ( P . T , F ) where P=V UR T = ( a , I a E A - (start]] U (a,( a E A - {exit]]
[transitions of activities 1
u ( x0 1 0 E 0 , ( x , 0)E R ] u ( ox 1 0 E 0, (0,x) E R 1 u{n,InEN)U{nfInEN) F = { as-u I a E A - {start]1 U u-afI
[transitions of or-nodes I transitions of and-nodes I
a E A - ( e x i t )}
[within activities]
u (xoHOIOE 0, (x,o)ER)U (o-ox(oEO,
(0,X)ER)
[within or-nodes1 U(n,Hn(nEN]U(nHnf(nENJ [within and-nodes1 u ( x p(x,y) 1 X E A u N ) [from activities and and-nodes1 [from or-nodes] u b Y H ( 0 , Y ) I 0E 0 ) U I (x,Y ) + + x , 1 x E A U N ) [to activities and and-nodes] [to or-nodes1 u ((x, o ) + x 0 1 o E 0 ) An example is in Fig. 6(a) which shows the place/transition system corresponding to the model in Fig. 4. This translation creates only free-choice nets [ 2 5 ] , because the only places with more than one successor are those of or-nodes, but their
20
PABLO A. STRAUB AND CARLOS A. HURTADO
FIG.6. (a) Place/transition net corresponding to the loan approval in FIG. 4. (b) One of the processes corresponding to the net, representing the case in which the mortgage is found acceptable but the credit is rejected.
successors have only the place as predecessor. That is, when there is choice there is no synchronization. The P/T net for a given model JU becomes a P/T system if it has a defined initial marking.
Definition 4.3 (Placeltransition system of a ClCN model)
A control model At has one place/transition system defined over the P/T net N(JU). The initial marking is
Call semantics. The translation above does not include the possibility of assigning a whole process model to an activity, i.e. having an activity whose execution comprises the execution of another process. This implies that there is a hierarchy of models. The semantics for simple calls can be expressed by the copy rule as in programming languages [19], that is the process model hierarchy is flattened before its conversion into Petri nets. Another possible translation for a call can be developed as a refinement of Petri nets. In that case, the structure of calls at the process model level is kept and there is a mapping between Petri nets. Figure 7 represents how an activity u in a model At is mapped to another model A ’ , in terms of Petri nets. The basic idea is that the place a in N ( A ) is refined into the whole net N ( A ’ ) .
21
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
...
Caller start
-0
a
exit
Caller’s translation
Called’s translation
0-43-0
Called
exit
FIG. 7. A caller model ht with an activity u that calls a called model A’.The figure shows the translations in terms of Petri nets and the refinement mapping of a into a whole net.
4.3
Behavioral Properties
One basic property of a good model is that it does not deadlock, that is, each process of the model reaches an output socket to produce a termination response to the environment. Inappropriate control structures can create deadlocks as in Fig. 8.
Definition 4.4 (Deadlock) A final marking M, is a deudlock in a process model if Mf(exit) = 0. Property 4.1 (Deadlock freedom) A model is deadlock-free if none of its final markings is a deadlock.
b
b
(4
(b)
FIG. 8. Models that deadlock; (a) guaranteed deadlock; (b) deadlocks when activity u chooses one path and b chooses the other. This is a distributed decision, i.e. a single decision must be taken by two independent executors.
22
PABLO A. STRAUB AND CARLOS A. HURTADO
Looking at Fig. 8(b) it seems that distributed decision is a structural property of a net. A relevant question is whether there is a class of nets that do not have distributed decision, i.e. local-choice nets. The property of being local-choice is not a structural property that can be checked by a simple predicate on the flow relation (as the free-choice property). Figure 9 shows a model that is not local-choice, but the and-nodes that may deadlock are far away from the activities that made the distributed decision. Process models can suffer from prescribing too much work, so that some activities are unnecessarily performed, because in some particular execution the exit of the model does not depend on them, i.e., the activity is useless. Useless activities are those that are unnecessarily performed in some particular execution, because there is no causal relation from the activity to the exit node. In other words, the process could produce the same output without executing the activity. For example, activity c is useless in the process pictured Fig. 6(b). If tokens are regarded as information placeholders, useless activities represent unused information. Useless activities are the consequence of choices that involve parallel activities. Given a process, a place is useless if there is no path from the place to the exit place. To define useless activities we need a notion of behavioral semantics of Petri nets that can represent causality, i.e. true parallelism as opposed to interleaving semantics. A parallel semantics of Petri nets represents executions of a net by a process, in which precedence represents causality (Section 3). In a process, a place represents a token, a transition represents a firing occurrence, and a path between two nodes represents a causal dependency between them. Useless activities are defined in terms of processes instead of final markings, because from a final marking it is impossible to know whether an activity was even executed. However, the existence of useless activities can be characterized in krms of final markmgs. A process model has no useless activities if and only if in all final markmgs all tokens are in the exit node [22].
FIG.9. This process model has distributed decision between activities A and B : executors of A and B are both to decide on the future of the process (decisions are representedby the ornodes immediately after the activities). If they take different decisions, the process deadlocks because the and-join will have two tokens on the same arc, instead of one token on each arc. The blobs represent submodels without control anomalies.
23
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
Definition 4.5 Given a process, a place is useless if there is no path from the place to the exit place. In other words, a place e within a process is useless if ( e ,exit) E F where F' is the transitive closure of F. An activity a is useless if there is a place e in a process for the model which is related to the activity, i.e., q ( e ) = a , and e is useless in the process. +,
Property 4.2 (Usefulness) A process model is useful if no activity is useless in any of its processes. For example, consider the process model from Fig. 10, whose translation into a Petri net is shown in Fig. 1 la. In one possible process of that net (Fig. 1lb), activity B is useless, hence the process model is not useful. Tokens in interior places (i.e. places that does not correspond to sockets) in a final marking that is not a deadlock define an overloaded markmg.
Definition 4.6 (Overloaded state) A final marking M f is overloaded if it is not a deadlock and there is an interior place p such that Mf(PI > 0.
C FIG. 10. A process model in CICN notation: activities are represented by ovals, and-nodes by black circles and or-nodes by white circles.
-0-DC)
exit
FIG. 1 1 . (a) The translation into a Petri net of the model in FIG. 10. (b) One of the processes of the Petri net, in which activity 8 is useless.
24
PABLO A. STRAUB AND CARLOS A. HURTADO
Useless activities are defined in terms of processes instead of final states, because from a final state it is impossible to know whether an activity was even executed. However, the existence of useless activities, can be characterized in terms of final states, as is stated in the following theorem. Theorem 4.1 A process model is useful overloaded markings and is deadlock-free.
if and only if it has no
Single-input-single-output is the most commonly accepted form of model abstraction, as used in ICN, VPL/Rasp, Action Workflow, and other languages. If a process is viewed as a black box, then if there is one token in the process, eventually there will be one token in the output. Given the semantics of activities in these languages, abstraction of a process as a compound activity is only possible if the model has single-input-single-output. Some languages [6,9] define other lunds of behavior, where and and or outputs are mixed, defining an unnecessarily complex control logic within activities. Hence, another property of a good process model is single-response, that is, each enaction of the process produces exactly one output (a singleresponse process model is by definition deadlock-free). If a process can produce more than one output in the same enaction we call the process multiple-response (Fig. 4).
Property 4.3 (Single response) A process model is singleresponse if all final markings M , satisfy M, (exit) = 1. It is multiple-response if there is a final marking M, such that M,(exit) > 1. The simple control property summarizes or-behavior.
Property 4.4 (Simple control) A model has simple control if the only final marking M , is
Mf@) =
1 i f p =exit 0 otherwise
Simple control implies that if a model begins with a single token in one of its input sockets, it ends with a token in one of its output sockets and there are no other tokens. Theorem 4.2 provides an alternative definition of simple control in terms of other properties. Theorem 4.2 A process model has simple control single-response and useful.
if and only if it is
A model with simple control is said to be behaviorally correct. There are two reasons to adopt this notion of correctness. First, from the above theorems there are no control anomalies, like deadlock, useless activities,
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
25
and multiple response. Second, a simple control model behaves like an activity; this allows consistent or-abstraction in which a process model can be used safely within a larger model without assumptions of its behavior (except for simple control).
4.4
Discussion
Basic constructs are all free-choice, i.e. they lead to free-choice Petri nets [25].This can be observed from the semantics of most languages to model process behavior. This allows simpler model analysis of control properties and the development of thread theory in the following section. Should choice and synchronization mix in BP models? In most situations no, because synchronization occurs within the model and choices are the result of the functionality of the activities which are executed without reference to the model’s state, that is, the state is not an input to the execution of an activity. We call this the principle of individual decisions; following this principle necessarily leads to free-choice nets. This principle is desirable from a language perspective, as it allows a sort of referential transparency with respect to the meaning of an activity. That is, the functionality of an activity is independent of the model in which the activity is embedded. This is needed if a system supports abstraction so that a model can be used as an activity in several other models. For example, in a banking application, there can be a process model for credit approval which is used as an activity in several process models for different kinds of loans. However, sometimes we want to model the situation in which an actor decides for another actor working in parallel, taking away his authority to choose. If we regard this authority as an order to make a decision, then this situation is a counterorder. Traditional modeling constructs do not allow one to specify this situation. The unbalanced connector and its supporting theory of threads is a simple, high-level and succinct construct to model this phenomenon. Simple control is related to some behavioral properties of free-choice Petri nets. In fact we prove in [22] that a model has simple control if and only if a connected free-choice net derived from the model is live and safe, as defined in Section 3. Because these properties can be determined in polynomial time for free-choice nets [4], this implies in turn that simple control in free-choice nets can be decided in polynomial time.
5. A Theory of Threads of Control We use a thread metaphor in which a thread is a set of strands for a single rope that goes from an input to an output in a process model. The metaphor
26
PABLO A. STRAUB AND CARLOS A. HURTADO
is best understood by considering a model with parallelism but no choice (e.g., a PERT chart). A single rope in which all strands are as long as the rope is passed from the input to the output by dividing the groups of strands into splits and uniting them in joins. In the metaphor, a thread is a set of strands. This metaphor can be extended to models that do have choice; in that case, whatever choices are made, all ropes that get to a choice point (i.e., an or-node) are part of the same thread. In other words, making a choice does not change the thread: only and-nodes change threads. The theory defines the concept of the thread of control ?#(n)of a node n, the subthread relation C, and thread addition Q. Threads are algebraic expressions whose operators are nodes in the net. The intuition behind the theory of threads is that threads are divided by and-splits and added by andjoins, in such a way that for every and-node the sum of the threads of its predecessors equals the sum of the threads of its successors. Each activity and or-nodes has one and only one thread. We have shown elsewhere that a model has no control anomalies like deadlock (i.e., no output), multiple response (i.e., more than one output), or useless activities, if and only if threads are well defined: the thread of the start node equals the thread of the exit node. The thread of a node x is denoted +(x), thread addition is denoted 8 , and the subthread relation is denoted For example, in Figure 12, the following relationships among the threads are true: v ( A ) = v ( o r l )= v(B)= v ( o 4 c v ( s t a r t ) = v ( e x i t > , v ( C >Q v ( o Q ?#(A)@w(C>= ?#(start), ?#(A)is not comparable to ?#(C). In Fig. 10 threads are not well defined. The reasoning is as follows: (a) It should be the case that ?#(A)8 v ( B ) = ?#(or2) so that the and-node that joins A and B is balanced. (b) It should be the case that v ( A ) = v ( o r l )because they are connected, and also that ?#(or,)= ?#(orz).But these equations are unsatisfiable because there is no equivalent of zero or negative threads. Thus, activity A and or-node or, belong to two different threads. This in turn implies that the model has behavioral anomalies, in this case the (unresolved) useless activity B which may lead to an extra token upon reaching the exit node.
c.
FIG. 12. A CICN model without choice and well-defined threads
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
27
The following goals are derived from the thread metaphor to characterize good models.
(1) An and-node joins input threads into a superthread and splits the superthread into output subthreads. ( 2 ) Two activities connected in sequence have the same thread. ( 3 ) Threads are executed sequentially, i.e., if two activities can be active at the same time, they belong to different threads. (4) If a thread is active (i.e. one of its activities is active), then its subthreads are inactive. If a subthread is active then the thread is inactive. (5) The start node and the exit node belong to the same thread, called p . (6) Every thread is a subthread of p . There is a strong relationship between threads and behavior. First, threads are a lund of non-numeric place invariant (i.e., the weighted sum of the tokens is constant). Second, a model has simple control if and only if every node belongs to one thread and the thread of the start node equals the thread of the exit node. Thus, the model in Figure 12 has simple control, while the model in Figure 10 does not. Moreover, if this condition does not hold, an explanation on the origin of the problem can be derived (e.g. distributed decision between A and B would be diagnosed for Figure 9, and a connection from a subthread to a superthread between the or-nodes would be diagnosed in Figure 10, page 23). Section 5.1 defines an algebra in which terms can be interpreted as threads satisfying all these goals. Section 5.2 shows that we can assign a unique thread to each place in a model if and only if the model has simple control. Moreover, if the model has no simple control, analysis of threads sheds light into the reasons for not having simple control.
5.1
Thread Labels and Threads
The definition of threads is done indirectly via thread labels. A thread label for a place represents one possible destiny or future history for a token in that place (i.e. a set of paths that is part of a process that begins with a token in p ) . Ldcewise, a thread label for a transition represents one possible destiny for a set of tokens which enable the transition. Because there are choices, hence several possible destinies, each node in the net is assigned a set of thread labels. Only those destinies that end with one token in the exit node of the model are considered. Thus, thread labels capture successful executions.
Definition 5.1 (Thread labels) The set of thread labels of a
28
PABLO A. STRAUB AND CARLOS A. HURTADO
process model At whose net is ( P , T , F ) , denoted L,, includes the following and nothing else: 0
0 0
every place or transition x in P U T ; thesymbolp; the label multiplication a @p, where a and /3 are labels; the label addition a o/?,where a and /? are labels; ( a ) ,where a is a label.
Label equality is defined by the following axioms: (1) (2) (3) (4)
addition is associative, a @ (/?8 y ) A ( a S B ) o y; addition is commutative, a Q p = /? Q a ; multiplication is associative, a QI (B @ y ) ( a @/?) y; multiplication distributes over addition, a @ (B o y ) a @/? Q a QI y , and ( a @ / ? ) @Gya @ y s / ? @ y .
As usual, multiplication has higher precedence than addition, and juxtaposition means multiplication, e.g. a o/?8 y = a 0 (/?@ y ) = a Q B ~ . The meaning of a label is a set of paths in the net. For places and transitions, these paths have length one. Label multiplication @ denotes a sort of relational cross product of paths, i.e. an element of the product is the catenation of an element from the multiplier with an element from the multiplicand. Label addition 8 denotes union of sets of paths. It is easy to check that whenever two labels are equal then they denote the same sets of paths, because cross product and union satisfy axioms 1 to 4. A label for a place represents one future history of a token in that place. If there are two or more tokens in a process, the set of futures of the process does not include all possible combinations of futures of these tokens, because their futures might eventually interact (e.g., by synchronization). Labels are said to be consistent if whenever they refer to the same place, the future history of a token at that place is the same, i.e. decisions taken at common places are the same. The definition of label consistency is syntactic if labels are expressed in a normal form.
Definition 5.2 (Label normal form) A label is in normal form if it is written as a sum of factors, without parentheses nor multiplication symbols. Any label can be normalized, by distributing multiplication successively and then dropping all parentheses and multiplication symbols.
Definition 5.3 (Consistent labels) A set X of normalized labels is consistent if for each place p that occurs in the labels, either all occurrences
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
29
of p are the rightmost symbol of some label, or there is a transition t such that all occurrences of p are right-multiplied by t. In other words, a given place has one successor in all the labels, or no successor in all the labels.
Definition 5.4 (Model labeling) The labeling of a model A is a function z : P u T+2: from nodes of the net to sets of labels, defined by 0
the only label for the place y that corresponds to the exit node is the place itself
d P ) = {PI. 0
Labels of a transition t are computed by adding a consistent set of labels, one from each successor of t , and pre-multiplying it by t : z ( t ) = ( t m ( a , CB. .. 0 a,) 1 t'
= ( p , , .. ., p , ) A
(A:=, 0
a , € t ( y ! ) ) A ( a ,,..., a , ) isconsistent).
Labels of a place p # exit are computed by pre-multiplying labels from its successors by p . z( Y ) = U r e p . { p & a I a E z(t)I
For example, part of the labeling of the model in Fig. 6(a) is shown in Table E is a shorthand for exit. There is an equivalence relation for thread labels, which is the base of the definition of a thread. 111, where
Definition 5.5 (Label equivalence) Label equivalence, denoted G , is the least equivalence relation that satisfies the axioms for equality and the following axioms: (6) OPE.,p m c 8 a a ( 7 ) p + p , if p is the place of the exit node. TABLE111 LABELING FOR
FIG. 6(a)
30
PABLO A. STRAUE AND CARLOS A. HURTADO
Intuitively, axiom ( 5 ) represents the firing of a transition. Applying this axiom changes the interpretation of labels, though: they are not always paths in the original net (they are paths in a net that may be collapsed in parts). Finally, axiom (6) says that should the model be associated with an activity a in another calling model, p denotes q(4)in the other model. A thread is an equivalence class of thread labels. A threading is a total function from the places of a model to threads, such that the start node is mapped to the same thread as the exit node, i.e. p.
Definition 5.6 (Thread and threading) A thread is a non-empty equivalence class of labels. A labeling t such that all labels assigned to a given place p are equivalent and the start node belongs to the thread p , i.e., (b'a,B € z( p ) : a
defines a threading $J: P + 2 ;
B ) A (3E
start) :I& p )
such that W ( p ) = [ a [ 38 E t( p ) :a t P}.
Notation. We usually denote a thread by a label (one of its members), e.g., q(p ) = a means a E q(p ) . Likewise, operations on labels are extended to operations on threads. Thread equality is simply set equality; hence, with a little abuse of notation, we write a = B to mean that both labels belong to the same thread, i.e., a B. For example, the threading of the model in Fig. 12 is as shown in Table IV, in which fy is the name of the start transition of y, p o is the name of the place between the final transition of or2 and t,, and p c is the name of the place between the final transition of c and ty. Now, we cannot derive a threading for Fig. 6(a). Consider the partial labeling shown in Table llI. Node Y has two labels that must be equivalent. The first label (the one going to translation t9) can be simplified to p, because all transitions in the label have only one predecessor. The second cannot be simplified to p; in fact its simplification is vt7yp,which is different from p because transition t7 has more than one predecessor and no addition is present in the label.
Definition 5.7 (Subthread, superthread) A thread a is a subTABLEIV THREADING OF MODELOF FIG. 12
Place p exit, start, and,, andz A , or,, B , or2
C
Thread v( P )
P PJ,P P
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
thread of a thread B, denoted a p, if there is thread d such thata or a is a subthread of one of the addends of B, i.e.
31 =B@S,
a C B W (36, y : a = 6 8 B ) V ( 3 y , E : B = y @E A a c y ) . The inverse relation is called superthread. Our definition of threads meets the six goals at the beginning of this section. This is because threadings can be regarded as positive non-numeric invariants, hence properties of invariants apply to threads. Theorem 5 . I Given a model whose connected net is ( P , T , F ) , its connected net has an extra transition t, such that {exit}= ' t , and t; = { start}. Then, i f the model has a threading I),for every transition t E T U 1 f , 1 , Q P E . ,I ) ( P ) = Q , , ; + ( P I .
If the function I)is rational-valued, as opposed to thread-valued, and the symbol Q is replaced by a summation symbol, the above equation is the standard definition for so-called S-invariants in Petri nets. The characteristic property of S-invariants is the fact that the weighted sum of tokens remains constant. This property also holds for threadings. Thus, if a model has a threading, the summation of the active threads (those that have a token) is ,a, in any reachable marking M.
5.2 Threads and Behavior Given a set of compatible labels whose summation is equivalent to p , these labels represent a (partial) process. The construction of the process from the labels is rather simple. The importance of this construction is that the final state implied by the process has a single token, in place exit, that is the set of labels represents a correct process. Furthermore, it can be proven that a process model has a threading if and only if it has simple control.
5.2.1 labels and processes Different labels are related to different processes. In fact a set of labels satisfying certain constraints defines a process. Given a consistent set [ a , ,..., a,,} of normalized labels taken from a labeling z, such that QY,i a, Ap, then it is possible to build a causal net ( B , E , F ' ) with only one place without successors. The construction has the following four steps:
S1. Let a , = 6,,@ @ 6 , ,where , 6,)has no operators. Each 6, is an odd-length string of alternating places and transitions, whose last symbol is the place of the exit node. - . a
32
PABLO A. STRAUB AND CARLOS A. HURTADO
S2. The set B of places in the causal net is obtained from the places of the Q,. S3. The set E of transitions is obtained from the transitions of the 6,. S4. A node x is a successor of a node y if there is a 6, of the form axyp, where a and p are possibly empty strings. The following lemmas claim that the net so constructed is a causal net and also a process of the original net N ( A ) , where the initial state is the one that marks all states related to the labels in the set. This process ends in a state whose only token is in the exit node (i.e. a state that satisfies the simple control condition). Lemma 5. I lfall labels have no cycles (hence arefinite), the net defined by the four-step construction is a causal net, i.e. ( a ) it has no cycles; ( b ) each place has at most one successor and ( c ) each place has at most one predecessor. Lemma 5.2 I f all labels have no cycles, the causal net of the four-step construction is a process of N(A)that begins in a state with one token in each place related to the labels (and no more tokens) and ends in a state with one token in an output socket (and no more tokens).
5.2.2 Threads and simple control The purpose of this section is to relate simple control and threadings. Theorem 5.2 uses the results of the previous Section to prove that a model that has a threading has simple control. Then a series of lemmas are introduced to prove in Theorem 5.3 that a model with simple control has a threading. Theorem 5.2 (Threading implies simple control) .A has a threading 11, then A has simple control.
If a process
model
We want to prove that simple control and the existence of a threading are equivalent. To prove that all models with simple control have a threading we use some properties of connected free-choice place/transition nets. Lemma 5.3 I f a model has simple control all places have at least one label. Lemma 5.4 If a model has simple control, all labels for a given place are equivalent, i.e.,
vp E P :v 1I , I ,
E z( p ) : 1 1 A
12
Theorem 5.3 (Simple control implies threading) A model .A that has simple control has a threading.
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
33
There are three possible causes for not having a balanced threading, which can be interpreted in terms of behavioral properties. First, there might be a place with no labels. If a place in the model has no label, this implies there is a proper trap in the net (a proper trap is a set of places that once they receive a token they always have some tokens [4]) and the model has an overloaded state, unless a deadlock upstream impedes reaching the trap. Second, there might be two unequivalent labels for a place. If there is a place with unequivalent labels, this means that this place is an activity whose output sockets are connected to different threads: this implies either deadlock or overloaded state. Third it might be the case that the thread of the start node is not p; in that case it is not possible to reach the proper final state, because the threading is an invariant.
6. Applications of Thread Theory The main two applications of the theory of threads are the development of consistent models and the definition of a new modeling construct called the unbalanced connector. These two applications are related, because the unbalanced connector is basically an and-node with a special semantics that guarantees consistency. There are basically two approaches to build consistent models: (1) build a model, then deduce that the model is consistent (otherwise modify the model); or (2) inductively develop the model using a set of primitives that guarantee consistency.
Proving consistency of a model. The usual method to deduce properties like simple control is based on a standard Petri net analysis, the reachability graph, which shows all reachable states for the net. There are many useful properties derivable from the reachability graph. Unfortunately the size of the graph might in general be exponential on the size of the net, so some methods to reduce the graph are needed.4 Another method to prove simple control is by computing a labeling and showing that it is (or it is not) a balanced threading. An algorithm to compute a threading is derivable from the definition. The idea is to assign labels to the outputs and compute labels of predecessors. Whenever a place has more that one label, they are proven equivalent by term r e d ~ c t i o n .If ~ they are no1 equivalent, then there is no 4The rank theorem [4] shows that the well formedness property of free-choice nets can be checked in polynomial time. This property is intimately related to simple control, hence it can be proven that simple control is also polynomial in free choice nets [22,25,29]. ‘Because thread addition commutes, unless care is taken, attempts at automatic proofs can lead to infinite iteration.
34
0
PABLO A. STRAUB AND CARLOS A. HURTADO
threading and the model has no simple control. If they are, one label is kept as a representative and others are deleted, thus the algorithm does not compute full labels but it does compute a threading. Of course, full label sets can be computed if desired; they give details about all possible futures of the computation. The best method is of course to have a small net: this can be accomplished if complex processes are described in terms of simpler processes, analyzing first subprocesses and then the composition of these subprocesses. Building consistent modeZs. It is possible to use a context-free grammar to model the primitives in the second approach. If a model can be generated using a grammar with a set of constructions, it is possible to prove by structural induction that the model has simple control, provided that each rule preserves simple control. One possible set of rules is sequential composition, alternative composition, and parallel composition. In addition, we have as an axiom the fact that each atomic activity has simple control. These rules are sound, although they are not complete (there are process models with simple control that cannot be parsed).
In what follows we present a variant of the grammar approach called the incremental case composition method, where instead of using a contextfree grammar the model is built incrementally by adding alternatives to a socalled base model. Each alternative is added by performing an operation on the model and creating a new model. The base model has simple control and each operation preserves this property, hence no control anomalies will occur. This method is not complete in the sense that there are models with simple control that cannot be obtained using the method. Creating a complete method in this sense is an open problem. 6.1
Base model
A base model is a model that describes one execution scenario, i.e., there are no alternatives. Most corporate procedure manuals describe processes without alternatives, i.e., they describe base models. Because behavioral inconsistency is the consequence of inappropriate use of parallelism and choice and the base model has no choices, it has no control anomalies. Figure 2 shows the base model for the loan example; it assumes there are no problems whatsoever with the credit so every activity is performed successfully and once. A process model is built beginning with a base model that represents the
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
35
"normal" case (whatever that means for the process developer). Then the model is enhanced by adding alternatives. Each alternative or case has a condition (specified as an or-split), a controlling activity or set of activities, and a next state (specified as a connection). At each step, the method checks that no behavioral anomalies are added.
Definition 6.1 (Base model ) it has no or-nodes.
A behavioral model is a base model if
An interesting property of a base model is that the only process denoted by the model is the same net as N(JU), because the model has no choices. This implies that those activities not related by F are concurrent. In P/Tnet parlance, a base model is a process of itself [20]. Moreover, the base model is the onfy process of JU (defining q as the identity function). +
Theorem 6. I
Every base model hus simple control.
Because every base model has simple control, it has a balanced threading
q.To compute the threading, the net is executed backwards as a P/T-system
( P , T , F - I ) , initially defining q(exit) = p and upon firing a transition t defining ;~ ( q )for p q @ ) : = p8 t 8eqG The following algorithm gives details.
E 't.
Algorithm: Threading of a base model Input Output Local variable
The place/transition net ( P , T , F ) of a base model At The threading i+ defined for each node of the net T ' , the set of processed transitions, and P ' , the set of processed places Loop invariant b'x E P' U T', y E P U T : x F*y w( y ) is well defined.
*
(1) Let q(exit):=,u. (2) Let P' := [ exit 1, and let T' := M. (3) While T' # T do (a) Let t be an element of T - T' such that t' 2 P'. (b) ForeachpE't, let q ( ~ ) : = p 8 f t @ ~q. (, q. ) (c) Let P' := P' U ' t , and let T' := T' U ( t ) Some comments on the algorithm: 0
This algorithm computes a labeling that has one label for each net element. Because every place has one predecessor, only one label is defined, so the labeling is a threading.
36
PABLO A. STRAUB AND CARLOS A. HURTADO
0
It is easy to show that the loop invariant is established before the loop, and that the loop invariant, together with the negated loop condition and the connectivity of the net establish the desired output. The fact that the invariant is indeed invariant is also simple, provided that there is always one transition t that satisfies the condition of statement 3(a). But that is the case, because the set of places p in P' such that ' p is not in T' represent a state of the system (i.e. it is a maximal cut in the process) and there must be a t whose firing leads to that state.
0
The threading computed by executing the net as above is an invariant. The only initial state has just one token in start, because the model has simple control. Hence, it must be the case that ly(sturt)= p , that is, ly is a threading.
6.2 Exceptions The base model describes a fixed procedure to handle a process. An exception is defined as any situation in which the base model does not apply. Exceptions are an intrinsic part of process modeling. This is specially true when modeling non-routine processes, but most routine processes do have exceptions, e.g., due to different cases or incorrect data. As in programs, exception handling in processes is done with regular branching and iteration, and also with special mechanisms that perform global control state changes, e.g., by manipulating the stack. In general, an exception comprises three parts:
(1) a control state and a condition in which the exception is raised; (2) a new control state resumed after the exception is handled; (3) an optional process that handles the exception. For example in an Ada program, the exceptional condition might be an attempt to divide by zero, the controlling activity is the exception handler, and the new control state is derived from the actions of the exception handler. We have shown that without exceptions, models have no behavioral anomalies. It is easy to prove that without parallelism there are no behavioral anomalies. Adding alternatives in the presence of parallelism can create behavioral anomalies. Consider a connection from a thread y to a thread B. If B < y the connection leads to deadlock. On the other hand, if y c B the connection may lead to an overloaded state or to multiple response. In the following subsections we show how exceptions can be added to a base model preserving the simple control property. The method adds
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
37
alternatives to a base model in such a way the the weighted sum of the tokens is always p , hence the process ends correctly and preserves the simple control property.
6.3
Alternatives within a Thread
Let .A be a model that is being developed and 1/1 its threading. In this section we describe how to extend JM by adding an alternative. The resulting model At.' becomes the new version of the model. A new alternative is specified by giving: (1) an or-node where the exception is raised; (2) An optional exception handler given by either a new activity a ; ( 3 ) an or-node where control returns after exception handling.
The addition of an alternative can be constructed with three more basic operations on the model: adding an or-node; adding a connection between two or-nodes; adding an activity.
Operation 6.1 (Adding an or-node) The new model resulting from the addition of an or-node o within a connection c = SH d is defined by Table V. Adding an or-node always preserves the simple control property. Lemma 6. I Adding an or-node o within a connection c = s + + dalways preserves the threading of existing nodes. The threading is extended by definingpsi (0)= q ( c ) .
Operation 6.2 (Adding a connection) The new model resulting from the addition of a connection c = s ~ between d or-nodes is defined by Table VI. TABLEV ADDITION OF AN OR-NODE Set
New value
A' := A 0' := 0 u I 0 ) N' := N R':= R U {s ++o,o++dJ - { s ~
d
)
38
PABLO A. STRAUB AND CARLOS A. HURTADO
TABLEVI ADDINGA CONNECTION BETWEEN OR-NODES Set New value A' := A 0' := 0
N' := N
R':=RU[s-d]
This operation preserves the simple control property and the threading if and only if the source and destination belong to the same thread. Lemma 6.2 Adding a simple connection c = SH d preserves the threading of existing nodes if and only if s and d belong to the same thread. The threading is extended by defining V ( c ):= q ( d ) .
Operation 6.3 (Adding an activity) The new model resulting from the addition of an activity a within a connection c = s~ d is defined by Table VII. Like adding an or-node, adding an activity always preserves simple control. Lemma 6.3 Adding an activity a in a connection c = s o d always preserves the threading of existing nodes. The threading is extended by defining * ( a ) = ~ ( c ) .
6.4 Alternatives between Multiple Threads Sometimes adding a simple connection is not enough. In a multiple connection a new and-node n is added and instead of returning control to a single or-node, control is returned to a set D = { d , , ...,d k ]of or-nodes. The addition of a multiple connection can be constructed with three more TABLEVII ADDINGAN ACTIVITY Set New value
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
39
basic operations on the model: 0 0
0
adding an or-node; adding a multiple connection from an or-node to several or-nodes, through an and-node; adding an activity.
Of these, the first and third are already analyzed.
Operation 6.4 (Adding a multiple connection) The new model resulting from the addition of a multiple connection from an or-node s to a set D = { d , , ..., d k ] of or-nodes through an and-node n is defined by Table VIII. This operation preserves the simple control property if and only if the thread of control of the source equals the summation of the threads of control of the destinations. Lemma 6.4 Adding a multiple connection preserves the threading of existing nodes, if and only if q ( s ) =@;=I
@(dJ
The threading is extended by defining ly(sn):= @ ( S > v ( n > : =v(s) q(ndi):=q(di)
for 1 G i c k .
The operation of adding a multiple connection was defined by adding an and-node that has exactly one predecessor. If an and-node with more than one predecessor is added the resulting model is necessarily not deadlockfree. In fact it leads to the phenomenon that we call distributed decision (Figure 8(b), Figure 9), in which the execution path to follow must be decided by two or more executors, but they all must make the same decision. TABLEVIII ADDITION OF A MULTPLS CONNECTION Set
New value
40
PABLO A. STRAUB AND CARLOS A. HURTADO
Theorem 6.2 (Distributed decision) if an and-node with more than one input is added to a model with simple control, the resulting model is not deadlock-free.
6.5
Unbalanced Connectors
Not all exceptional conditions can be handled by adding alternatives with balanced connections. While exceptions are raised locally in a single thread, other parallel threads of control might be involved, e.g. an exception may need to abort activities in other threads of control. An unbalanced connector is used to abort activities; it has always one source and one or more destinations. Information processes need exception handling facilities to succinctly describe many possible outcomes of a process. In principle, all exception handling can be done using conditionals and iteration, by adding tests for all possible events in every relevant point in the model. The idea of special exception handling constructs is factoring out those tests. For example, if it is known that because of some event a whole range of activities become irrelevant, these activities should be canceled. Furthermore, if an exception cannot be handled at the level where it is detected, some form of exception propagation must occur. To enrich information process control languages we can draw from ideas prevalent in the programming languages community. However, because of multi-threading, it is not obvious that ideas from languages such as Ada, Lisp or C can be mapped to information processes. Thus, it is not surprising that few languages have exception-handling constructs. In Rasp/VPL and in WAM [15] when a token reaches the end of a model, all activities within the model are deleted. This semantics ensures that there cannot be multiple response, i.e., more than one output (defined in Section 4). An explicit construct to abort the execution of unneeded activities is the unbalanced connector [25]. The unbalanced connector is a sort of and-node in which one predecessor is explicitly shown, and other predecessors are implicit (all successors are explicitly shown). The semantics can be informally expressed as “take a token from the input and other tokens from the model so that the number of tokens is just right”. The meaning of ‘‘just right” is such that it ensures that at the end the model will produce one token on the exit node and there will be no other tokens left. The semantics of the unbalanced connector is based on the theory of threads. Unbalanced connectors are different from regular and-nodes and thus its translation into P/T-nets is different. The semantics of unbalanced connectors can be expressed using Petri nets. In the example from Figure 13, the unbalanced connector is represented in Figure 14 by three transitions, which
41
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
CFIG.
13. A process model with an unbalanced connector corresponding to the model in
FIG. 9.
unbalanced
DDO-
Uu-0
exit
start
FIG. 14. Semantics of the unbalanced connector in FIG. 13
produce a token in the place of the following or-node, and consume a token from the connector’s incoming edge and a token from the places of either B, its incoming edge, or its outgoing edge. The translation ensures that extra tokens are consumed by the one of the transitions corresponding to the connector. An unbalanced connector produces dangling activities, i.e., those whose output is not needed. Dangling activities are those whose thread is less than the threads of the destinations. In Figure 13 B is dangling. Dangling activities violate the invariant, so they must be aborted.6 In fact, dangling activities are very much like useless activities, so if they were not aborted they would lead to overloaded states. In principle, Petri nets are useful to model unbalanced connectors, but they are not practical because the number of transitions needed to represent an unbalanced connector can grow exponentially with the number of nodes in the net due to the state explosion that may occur. In a realistic loan approval example [ 2 5 ] , the CICN net had 24 nodes, 3 of which were unbalanced connectors, and the corresponding Petri net had 1266 nodes; for comparison, the CICN net without unbalanced connectors has 20 nodes and ‘This requires a user interface where users are advised that their pending activities have been interrupted. Here we d o not delve in these matters.
42
PABLO A. STRAUB AND CARLOS A. HURTADO
the corresponding Petri net has 35 nodes. On the other hand, the intuitive semantics of the unbalanced connector is relatively simple: “abort now all activities that are not required to produce output,” or more technically “delete all tokens in places whose thread is a subthread of the destination of the unbalanced connector and put one token in the destination”. An implementation can use the second intuitive semantics, with the subthread relation computed off-line when the process model is compiled into an internal form. Formally, the semantics of a deterministic unbalanced connector c is given in terms of its translation into a P/T-net, i.e., by extending the definition of N ( & ) (Definition 4.2) as follows.
Definition 6.2 (Extension of Definition 4.2) The connection from an or node s to a set of or nodes D = ( d l ,...,dk} through an unbalanced connector c is translated by adding nodes to the P/T net, once regular nodes are translated. Let PI,...,P,, be all possible sets of dangling places in the connection at any state M such that M(’c) > 0. The translation will add 1 place p and n + 1 transitions { f, t , , ...,t , , ) , connected as follows: ‘P= (tl ‘t=s ‘ t i= Piu { p )
p ’ = i f , , . ..,f”) t’ = p tl. = D.
An important property of the semantics for an unbalanced connector is that the transitions of the connector are all balanced. Thus, even if the model seems unbalanced, the P/T-net that defines its semantics is balanced. However, an important property is lost: the nets are no longer free-choice. For example, in Figure 14 the representation for the unbalanced connector shows a place with three successors, each of them with a different set of predecessors, hence the net is not free-choice. That is, unbalanced connectors mix choice and synchronization. This means that choices are constrained by the global state, i.e. they are no longer local. However, this is precisely the intended meaning: in the presence of an exception an executor might unexpectedly lose control of its thread. Lemma 6.5 Adding a multiple connection with an unbalanced connector preserves the threading, if and only if
W )G @;=
1
V(4).
The threading is extended by defrning
1v( P > := 1v(s).
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
43
6.6 Summary: Incremental Case Composition The method to build a model by incremental case composition is as follows.
(1) Identify one possible case of the model. This case might be the most likely or the one that is considered normal. (2) Create a base model by enumerating all activities of this case and identifying the precedence relation. This process is the same as creating a critical path method (CPM) chart. (3) Repeat, until all possible cases are covered: (a) Identify a condition and place of evaluation of the condition. (b) If a handling process is needed, develop it using this method or use one that is already developed. (c) Determine the new state of the process after the exception is handled. (d) Check that the connection is feasible. If not, this exception cannot be added to the model. Feasibility of a connection can be checked automatically, if the system keeps the threading as the model is being developed. For example, to develop the loan approval process, the base model shown in Figure 2 is the case in which all activities succeed (this might not be the most frequent case). Figure 15 is the final model which was built by adding several cases to the base model. The first case was the possibility of not approving the credit: this case added an or node ( u l ) after activity A, an unbalanced connector (a,), and an or connector immediately before exit. The second case was the possibility of errors in the legal data of the property: this case added a loop in activities E and F , with two new or
exit
FIG. 15. The complete loan approval process, with four exceptional cases.
44
PABLO A. STRAUB AND CARLOS A. HURTADO
nodes. The third case had to do with the possibility of the property not being appraised as valuable enough to cover the credit: this case added an activity H (“to notify the customer”) and an unbalanced connector ( a s ) .The fourth case involved letting the customer decide upon this notification whether or not to use the credit for another property: this case added or node o4 after activity H and an unbalanced connector (a6). Because all steps in the process keep the threading, it can be proven that the model has simple control. Theorem 6.3
Models built with the described method have simple
control.
6.7
Dealing with Unspecified Situations
In most business processes it is impossible to know in advance how to handle every possible situation. It is likely that regardless of how complete the model might be, at some point none of the pre-programmed alternatives apply. Thus, if a process modeling system insists on the model being complete before enaction, once an unforeseen situation arises either the problem will not be solved (“sorry, madam, the computer doesn’t allow that”) or it will be very hard to keep the computer up to date; that is, at this point the workflow system would be more hindrance than help. A system should allow addition of unspecified exceptions at run time. When such an unforeseen exception arises, the executor of the activity manually solves the problem and creates a connection to bring the process to a desired state. The system checks that the connection is feasible; if the connector is infeasible it is rejected, else execution continues as if the connection existed before enaction. Because the connection is feasible no behavioral anomalies will ensue.
6.8 General Unbalanced Connectors A generalization of the unbalanced connector includes the possibility of producing extra tokens instead of deleting them [26]. This happens if the sum of the threads of the incoming arcs is greater than the thread of the outgoing arc. A further generalization is allowing incomparable threads, so that some tokens are deleted while others are produced. Producing tokens in a thread poses the problem that there are several places in the same thread: which place should be chosen? If the choice is random, then we have a nondeterministic system with the power to decide on the work performed by people. We rule that out. Fortunately, there is a better choice: produce the token in the place right before the connector that will be
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
45
waiting for it. This connector can be determined by looking at the thread expression. Figure 16 shows the use of such a kind of unbalanced connector, depicted with dangling arcs for both predecessors and successors. The extra token is produced in such a way that it minimizes work along its thread, i.e. no work is implicitly assigned due to the unbalanced connector. In the example, when or-node or2 chooses the lower path to iterate, activity D is aborted, activities C , D , and E are re-executed, but B is executed just once. While the semantics of general unbalanced connectors imply the deletion and addition of tokens to ensure that the weighted sum of tokens is equal to p , this is not always feasible. That is, general unbalanced connectors cannot always be added from any or node to any other set of or nodes. The constraint is that the destinations must add up to something less than or equal to p. This implies in turn that no thread will be active twice, and no thread will be active if a subthread is active.
Definition 6.3 (Feasible connector) A connection from an or node s to a set of or nodes D = { d ,,..., dk] using a generalized unbalanced
FIG.16. A generalized unbalanced connector and its semantics in terms of Petri nets.
46
PABLO A. STRAUB AND CARLOS A. HURTADO
connector n is feasible if and only if Note that because C is a partial order it might be the case that these threads are uncomparable.
7.
Conclusion
The modeling of process control is a non-trivial task, especially when processes are complex. These problems are not conveniently handled in all tools we are aware of, even though it does not make much sense to analyze efficiency, timing, data flow, data availability, and resource utilization, if the model behaves incorrectly. Traditional solutions to the problem of ensuring correct behavior do not seem adequate. The use of context-free grammars to define sets of allowable models ensures correctness, but overly constrain the range of expressible models, inhibiting the natural parallelism in processes. The verification of models based on reachability graphs (i.e. finding all reachable states) or on Petri net invariants are computationally tractable methods for free-choice nets,' but they do not give clues on the causes of behavioral anomalies and possible corrections. In this article we have identified a series of relevant control properties, including deadlock freedom, useless activities, consistent abstraction, etc. These properties are related and form the basis for a notion of behavioral correctness in a model. Process semantics of Petri nets helps to determine the causal relations between activities in a process, i.e., whether two activities are independent and are executed in parallel, or they are executed in sequence. This paper defines the concept of useless activity using process semantics. Surprisingly, there are situations in which the possibility of having a useless activity cannot be avoided [26]. In this case, once an activity is determined to be useless it can be aborted. An algebraic formalization of the rather elusive concept of thread of control was given. The thread algebra is a suitable framework to understand behavioral properties; in fact, there are strong relationships between threads and behavior: behavior is correct if and only if threads of controls are properly mixed in the model, a notion that has been formally defined. We have recognized several applications of thread theory, applicable to several other languages. One application of thread theory is the incremental composition method to develop process models by iteratively adding exceptions to a so-called base 'All basic control constructs lead to free-choice nets 1251.
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
47
model. At each step, an appropriate mix of threads is kept in the control model, hence preserving the simple control property. This method can also be applied to the problem of handling an unforeseen exceptional condition at run time. A thorough understanding of control allows the modification of an executing process due to unanticipated exceptions, guaranteeing that the modified model will not have behavioral anomalies: it suffices to (incrementally) recompute the threading for the new model, using the techniques of Section 6. Using the theory of threads we have extended basic control models adding so-called unbalanced connectors that mix threads in a controlled way [ 2 3 ] . Thread theory is used to identify which activities should be canceled. This is a generalization of the rule in VPL/Rasp that once a token is put into an output socket’ of the model all pending activities within the model are canceled. While the semantics in terms of Petri nets is complex, the informal semantics can be simply stated as “abort now all activities that are not required to produce output”. The theory of threads identifies those places based on the threads of predecessors and the successor of the unbalanced connector. A further generalization of the unbalanced connector not only deletes extra tokens, but also add those that are now needed. The generalized unbalanced connector has an even more complex semantics in terms of Petri nets, but its informal semantics is still simple “abort now all activities that are not required to produce output and create tokens so as to avoid deadlock”. Again, the theory of threads identifies those places that will be affected. ACKNOWLEDGMENTS This paper has been improved by the comments of one anonymous referee.
Appendix: Proofs of Theorems This appendix includes most proofs. In some cases where the full proofs are lengthy and do not give much insight, only a sketch of the proof is provided and the reader is referred to the original proofs.
Proof of Theorem 4.1 A process model is useful if and only if it has no overloaded markings and is deadlock-free. (If.) Let n be a complete process that corresponds to one execution of the ‘An output socket of a model is the equivalent of an exit node.
48
PABLO A. STRAUB AND CARLOS A. HURTADO
P/T-system of a CICN model. Assume x has a useless place 4 ( b ) . Then there is no path from b to a place corresponding to exit, that is, if q ( x ) = exit, (b,x ) E F'. Then for every successor b' of b there is no path that leads to exit. Because n is a complete process (hence finite), there must be at least one successor b" of b that has no successors. The state denoted by x is such that M ( q ( b " ) )> 0, and a ( b " )+ exit. That is, the final state of n is either overloaded or a deadlock. (Only if.) Assume the model has an overloaded state or a deadlock. In any case there must be a place p # exit and a process x whose final state satisfies M,( p ) > 0. From the definition of reachable state of a process, there must be a place b E B such that q ( b ) = p and b' = 0.Then p is a useless place in n.
0 Proof of Theorem 4.2 A process model has simple control if and only if it is single-response and useful. (If.) From Theorem 4.1 the model has no overloaded states nor deadlocks, hence for every place p # exit, M,( p ) = 0. Because it has single response, then M,(exit) = 1, so the model has simple control. (Only if.) Immediate from the definition of simple control.
0 Proof of Theorem 5.1 Given a model whose connected net is ( P , T , F ) , its connected net has an extra transition t, such that { e x i t ] = ' t , and ti = { start }. Then, if the model has a threading V , for every transition t E T U k l , O p E . ,V ( P ) = e P € iV ( P > . We first prove that equality for t, and then for other transitions. For t,, we have ' t , = { exit] and t: = ( s t a r t ) . From the definition of label equivalence, 1/,(exit) = ,u and from the definition of labeling, V (start)= p. Let c E T be a transition different from I,. Because all labels in a given place are equivalent, the proof can be done by choosing any label from each predecessor and successor of t. Let a,, ..., a , be a compatible set of labels each for one of the successors of t. Let p be a predecessor of t. Then a label for p is p
@ t @ (a1 @
... @ a,)
and a summation of labels from predecessors of t is
[Because of rule 41 [Because of rule 71
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
49
Proof of Lemma 5.1 If all labels have no cycles (hence are finite), the net defined by the four-step construction is a causal net, i.e. (a) it has no cycles; (b) each place has at most one successor; and (c) each place has at most one predecessor. For a proof of this lemma, please refer to [7]. The proof hinges on the fact that if the net had a place with more than one successor, then the labels would not be consistent. Now, if no place has more than one successor, a cycle would be an infinite cycle only obtainable from an infinite label or from an infinite set of finite labels, but all labels are finite. Finally, if there were a place with more than one predecessor, the labels would be irreducible to p by the given rules. 0 Proof of Lemma 5.2 If all labels have no cycles, the causal net of the four-step construction is a process of A'(&) that begins in a state with one token in each place related to the labels (and no more tokens) and ends in a state with one token in an output socket (and no more tokens). For a proof of this lemma, please refer to [ 7 ] . The proof hinges on the fact that all labels end in the exit node and that if the labels add up to p they must be reducible to p. 0 Proof of Theorem 5.2 If a process model A, has a threading q then A, has simple control.
Because the threading is a place invariant (proved in Theorem 5.1) every reachable state M must satisfy v ( M ) = p . Let ( a l ,..., a,,] be a set of consistent labels, one from each place marked in M (these labels exist, just take for each successor transition the same label). Because the labeling is a threading, this set of labels adds up to p , hence it is possible to build a process that ends with a single token in the exit node. Hence, given any reachable state it is possible to reach an adequate final state (i.e. no reachable state is deadlock or overloaded or multiple response). Thus the model has simple control. 0
Proof of Lemma 5.3 If a model has simple control all places have at least one label. For a proof of this lemma, please refer to [7]. The proof is based on 2 other lemmas, which state: (1) If the connected net has a trap' that does not include start, then the model has no simple control. (2) If there is a labeling 'In Petri net theory, a trap is a set X of places such that x'
'X.
50
PABLO A. STRAUB AND CARLOS A. HURTADO
that includes a place without labels, then the net has a trap that does not include start.
0 Proof of Lemma 5.4 If a model has simple control, all labels for a given place are equivalent, i.e. t l p E P :V l , ,I, E t(p ) : I, 2 I, Assume there is a place p with two non-equivalent labels I, it I,, such that all successors of p only have equivalent labels (i.e. p is the first offending place discovered by a labeling algorithm working backwards). Let M be a reachable state that marks p . This state must exist because if the net has simple control, the net is live [22,29]; because the net is safe (or 1-bounded) then M ( p ) = 1. Then, let M' be a state obtained from M by firing transitions that do not consume the token in p until no more transitions can be fired without consuming that token. Because the model has simple control, from M' all processes end in a state with a single token in the exit place. Thus, it is possible to choose one label from each place marked by M' such that the summation of all labels is p. In particular, we might choose I, to be in that set. From the set of labels we can construct a process. In this process all places but those marked by M' are successors (under F ' ) of p . Now, because the election of I, was arbitrary ...
0
Proof of Theorem 5.3 A model At that has simple control has a threading. From Lemma 5.3 all places have a label. From Lemma 5.4 all labels in a given place are equivalent. To prove that the labeling defines a threading we only need to prove that a label a for the start node is equivalent top. But that must be the case, because from the second part of the proof of Theorem 5.1 if all labels are equivalent the threads are an invariant. Now, because the model has simple control the threads of all final states add up to p , hence the thread of the initial state must also be p. (This can be seen more clearly by executing the net backwards, the invariant property holds both ways.) 0 REFERENCES
1. Abbati, D., Caselli, S . , Conte, G., and Zanichelli, F. (1993). Synthesis of GSPN Models for Workload mapping on Concurrent Architectures. Proceedings of the International Workshop on Petri Nets and Performance models. 2. De Michelis, G., and Grasso, M. A. (1993). How to put cooperative work in context:
CONTROL IN MULTI-THREADED INFORMATION SYSTEMS
51
Analysis and design requirements. In Issues of Supporting Organizational Context in CSCW Systems (L. Banon and K. Schmidt, Eds). 31 August. 3. Dennis, A. R., Hayes, G. S., and Daniels, R. M. (1994). Re-engineering business process modeling. Proceedings of the Twenty-Seventh Annual Hawaii International Conference on System Sciences. 4. Desel, J., and Esparza, J. (1995). Free-choice Petri Nets. Tracts in Theoretical Computer Science 40, Cambridge University Press, Cambridge. 5. Ellis, C. A,, and Keddara, K. (1995). Dynamic Change within Workflow Systems. University of Colorado Technical Report, July. 6. Gula, J. A,, and Lindland, 0. A. (1994). Modeling cooperative work for workflow management. 6th International Conference on Advances Information Systems Engineering, CAISE, June. 7. Hurtado, C. A. (1995). Modelaci6n y Analisis de Procesos de Informaci6n. Tesis de Magister, Pontificia Universidad Cat6lica de Chile, Santiago (in Spanish). 8. Harel, D., Lachover, H., Naamad, A., Pnueli, A., Politi, M., Sherman, R., ShtullTrauring, A., Trakhtenbrot, M. R. (1990). STATEMATE: A Working Environment for the Development of Complex Reactive Systems. IEEE Trans. on Software Engineering, 16(4). 9. Integration Dejnition for Functional Modeling (IDEFO). National Instihte of Standards and Technology, USA 1992. 10. Curtis, W., Kellner, M. I., and Over, J. (1992). Process modeling. Communications ofthe ACM, 35(9). 11. Lamb, D. A. (1988). Software Engineering: Planning for Change. Prentice-Hall, Englewood Cliffs, N.J. 12. Lauterbach, K. (1987). Linear algebraic techniques for place/transition nets. Lecture Notes in Computer Science, 255. 13. Malone, T. W., and Crowston, K. (1994). The Interdisciplinary Study of Coordination. ACM Computing Surveys, 26(26). 14. Medina-Mora, R., Winograd, T., Flores, R., Flores, F. (1992). The Action Workflow approach to workflow management technology. Proceedings of CSCW, November. 15. Messer, B., and Faustmann, G., (1995). Efficient Video Conference via Workflow Management systems. Workshop “Synergie durch Netze”, Universitat Magdeburg, October. (English translation by the authors.) 16. National Institute of Standards and Technology (NIST) (1993). Integration Definition for Function Modeling (IDEFO). FIPS Pub 183, NIST, December. 17. Parry, M. (1994). Reengineering the Business Process. The Workflow Paradigm, Future Strategies Inc., 1994. 18. Peters, L., and Schultz, R. (1993). The application of petri-nets in object-oriented enterprise simulations. Proceedings of the 27th Annual Hawaii International Conference on System Sciences, 1993. 19. Pratt, T. W., and Zelkowitz, M. V. (1996) Programming Languages Design and Implementation. Prentice-Hall, Englewood Cliffs, N.J. 20. Reisig, W. (1985). Petri Nets: An Introduction. Springer-Verlag. Berlin. 21. Robinson, M. (Ed.) (1991). Computer Supported Cooperative Work. Cases and Concepts. Proceedings of Groupware ’91. Software Engineering Research Center. 22. Straub, P., and Hurtado, Carlos A. (1995). The simple control property of business process models. XV International Conference of the Chilean Computer Science Society, M c a , Chile, 30 October-3 November. 23. Suaub, P., and Hurtado, Carlos A. (1995). A theory of parallel threads in process models.
52
24.
25.
26.
27. 28.
29.
30.
PABLO A. STRAUB AND CARLOS A. HURTADO
Techical Report RT-PUC-DCC-95-05. Computer Science Department, Catholic University of Chile, August (In URL ftp://ftp.ing.puc.cl/puWescuelddcc/techReportslrt95-05.ps). Straub, P., and Hurtado, Carlos A. (1996). Understanding behavior of business process models. In Coordination Languages and Models, First International Conference, Coordination’96, LNCS 1061, Springer, Cesena, Italy, April 15-17. Straub, P., and Hurtado, Carlos A. (1996). Business process behavior is (almost) freechoice. In Computational Engineering in Systems Applications, Session on Petri Nets for Multi-agent Systems and Groupware, Lille, France, July 9-12. Straub, P. and Hurtado, Carlos A. (1996). Avoiding useless work in workflow systems. International Conference on Information Systems Analysis and Synthesis, ISAS’96, International Institute of Informatics and Systemics, 14269 Lord Barclay Dr., Orlando, USA, 22-26 July. Swenson, K. D. (1993). Visual support for reengineering work processes. Proceedings of the Conference on Organizational Computing Systems, November. Touzeau, P. (1996). Workflow procedures as cooperative objects. In Computational Engineering in Systems Applications, Session on Petri Nets for Multi-agent Systems and Groupware, Lille, France, 9-12 July. van der Aalst, W. M. P. (1995). A class of Petri nets for modeling and analyzing business processes. Computing Science Report 95/26, Dept.’ of Computing Science, Eindhoven University of Technology, August. Workflow Management Coalition (1994). Glossary. Document no. TC00-0011, 12 August.
Parallelization of DOALL and DOACROSS Loopsa Survey A. R. HURSON, JOFORD T.LIM The Pennsylvania State University Department of Computer Science and Engineering University Park, PA
KRISHNA M. KAVl The University of Texas at Arlington Department of Computer Science and Engineering Arlington, TX
BEN LEE Oregon State University Department of Electrical and Computer Engineering Corvallis, OR
Abstract Since loops in programs are the major source of parallelism, considerable research has been focused on strategies for parallelizing loops. For DOALL loops, loops can be allocated to processors either statically or dynamically.When the execution times of individual iterations vary, dynamic schemes can achieve better load balance, albeit at a higher runtime scheduling cost. The inter-iteration dependencies of DOACROSS loops can be constant (regular DOACROSS loops) or variable (irregular DOACROSS loops). In OUI research, we have proposed and tested two loop allocation techniques for regular DOACROSS loops, known as Staggered distribution (SD) and Cyclic Staggered (CSD) distribution.This article analyzes several classes of loop allocation algorithms for parallelizing DOALL, regular, and irregular DOACROSS loops.
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Loop-scheduling Algorithms for DOALL Loops . . . . . . . . . . . . . . . . . . 2.1 Self-scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Fixed-size Chunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVANCES IN COMPUTERS. VOL. 45
53
54 55 56 57
Copyright0 1997 by Academic Ress Ltd. All rights of reproduction in any form reserved.
54
3. 4.
5. 6.
7.
A. R. HURSON ETAL.
2.3 Guided Self-scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Factoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Trapezoid Self-scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparative Analysis of DOALL Loop-scheduling Schemes . . . . . . . . . . . DOALL Loop Scheduling on NUMA Multiprocessors . . . . . . . . . . . . . . . 4.1 Affinity Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Partitioned Affinity Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Locality-based Dynamic Scheduling . . . . . . . . . . . . . . . . . . . . . . Comparison of Affinity-scheduling Schemes . . . . . . . . . . . . . . . . . . . . DOACROSS Loop Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 The Regular DOACROSS Model . . . . . . . . . . . . . . . . . . . . . . . 6.2 Comparison of DOACROSS Scheduling Schemes . . . . . . . . . . . . . . 6.3 Irregular DOACROSS Loop Scheduling . . . . . . . . . . . . . . . . . . . . 6.4 Comparison of Irregular DOACROSS Scheduling Schemes . . . . . . . . . 6.5 Other Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.
57 58 59 59 63 64 64 66 67 71 71 76 79 87 89 YO 101
Introduction
Since loops are the largest source of parallelism (Polychronopoulos et al., 1986) considerable attention has been paid to the partitioning and allocation of loop iterations among processors in a multiprocessor environment. The key goal is to maximize parallelism while minimizing the processor load imbalances and network communication. The literature abounds with scheduling algorithms for loops. These algorithms can be categorized as static and dynamic (Krothapalli and Sadayappan, 1990). In static scheduling, the division of iterations among the processors is determined prior to the execution time. This results in a low runtime scheduling overhead. On the other hand, static scheduling can cause unbalanced distribution of load among the processors if the execution times of individual iterations vary. The variance in execution can result from conditional statements (Hummel et al., 1992), or because of interference from the environment (the operating system, switching between iterations or time-sharing with other programs). Dynamic scheduling determines the division of iterations among processors at runtime. Some algorithms may dynamically reassign iterations to different processors based on the progress made by processors on previously assigned iterations. Thus, dynamic schemes can achieve better lead balance, but this comes at the expense of runtime scheduling overhead. Loops can be categorized as sequential loops, vector loops (DOALL), and loops of intermediate parallelism (DOACROSS) (Cytron, 1986). For a DOALL loop, all N iterations of the bop can be executed simultaneously.
DOALL AND DOACROSS LOOPS
55
When there is a sufficient number of processors, all iterations can be executed in parallel. But with a finite number of processors, iterations are divided among the processors. When iterations of a loop must be executed completely sequentially (sequential loops), no improvement can be gained by using multiple processors. However, some loops may exhibit intermediate levels of parallelism permitting some overlapped execution among iterations. The DOACROSS loops model proposed by Cytron (1986) can mimic sequential loops, vector loops and loops with intermediate levels of parallelism. Iterations may be either data- or control-dependent on other iterations. Control dependencies are caused by conditional statements. Data dependence appears in the form of sharing results computed by other iterations. Data dependence can be either lexically forward (data from higher indices used by iterations with lower indices) or lexically backward (data from lower indices used by iteration with higher indices). Normally, lexically forward dependencies (LFD) do not contribute to delays in executing loop iterations. Sometimes a lexically backward dependence (LBD) can be transformed into a lexically forward dependence by reordering the statements of the loop, provided the statements do not form a dependence cycle (Cytron, 1986). DOACROSS loops where the LBD cannot be transformed into LFD lead to delays in executing successive iterations. Such loops are the subject of most research. This chapter presents an introduction to several loop allocation techniques and analyses these techniques for their complexity, scheduling overhead, communication cost, processor utilization and expected speedup. Section 2 surveys DOALL loop scheduling algorithms and Section 3 compares these algorithms. Section 4 presents affinity scheduling schemes for DOALL loops, while Section 5 compares these techniques. Regular and irregular DOACROSS loop scheduling algorithms are presented and analysed in Section 6.
2.
Loop-scheduling Algorithms for DOALL Loops
Static scheduling schemes assign a fixed number of loop iterations to each processor. For a loop with N iterations executed on P processors, each processor will receive a total of [ N I P 1 iterations. Variations on how these iterations are distributed among the available processors lead to different algorithms. Block scheduling or static chunking (SC) assigns iterations 1 through [ N I P ] to the first processor, iteration [ N I P 1 + 1 through 2 * r N / P 1 to the second processor, and so on. Cyclic scheduling allocates iterations i, i + P , i + 2 P , ..., to processor i (1 d is P ) .
56
A. R. HURSON ETAL.
When the execution times of individual iterations vary, static chunking leads to different processors performing different amounts of work, and finishing their computations at different times. For example, when the execution times of iterations monotonically decrease (i.e., triangular iteration space), the chunks containing smaller iteration indices consume more time than chunks containing iterations of higher indices. In such a case, the execution time of the DOALL loop is bounded by the completion times of the earlier chunks. Thus static chunking could perform suboptimally, and cause under-utilization of processor resources (Hummel et al., 1992). Since cyclic scheduling assigns consecutive iterations to different processors, a better load balance across processors is achieved. The main advantage of static scheduling methods is their simplicity (hence small scheduling overhead). No runtime overhead is incurred by such methods since all scheduling decisions are made at compile time. This implies the availability of information on loop bounds and number of processors. When such information is not known statically (or changes dynamically), static scheduling methods lead to unbalanced workload among processors. Dynamic scheduling schemes are proposed to address the limitations of static methods. Typically, shared variables and critical sections are used to control the distribution of iterations to idle processors. Thus, an idle processor locks the shared variable and obtains an iteration (or a set of iterations). This leads to runtime overhead, both in terms of the time required to access the shared variable (including communication cost and synchronization cost), and the time needed to compute the next schedule. Complex dynamic schemes with high scheduling costs and large communications costs may negate any performance gained. In order to simplify the analysis, we will assume that the scheduling cost per scheduling step, for all dynamic schemes will be the same, and is given by TsCw= 2C + twhed,where C is the communication cost for accessing a shared variable as well as the communication cost for returning an updated value to the shared variable, is the time required to calculate the chunk size. Some of the and t,, dynamic scheduling algorithms are discussed below.
2.1
Self-scheduling
Self-scheduling (SS) (Tang and Yew, 1986) is a dynamic scheme that schedules iterations of a loop, one at a time. An idle processor obtains a new iteration and executes it. Hence, processors finish at nearly the same time and the workload is balanced. However, since this method requires N scheduling steps (one for each iteration), the overall scheduling cost may be unacceptable. In addition, processors may have to contend with synchronization delays
DOALL AND DOACROSS LOOPS
57
in accessing shared variables. For example, with P processors attempting to obtain a loop iteration, one processor must wait ( P - 1)T,h& waiting for all the other processors to access and update the shared variable. The average wait time for P processors is given by P ( P - 1)Tsched/2.With N iterations, the average wait time per processor is given by N ( P - 1)cched/2.
2.2
Fixed-size Chunking
In an attempt to reduce the number of scheduling steps needed, Fixed-size chunking (FS) schedules a fixed number of iterations to each idle processor (as opposed to one iteration in SS) (Kruskal and Weiss, 1985). This reduces scheduling overhead, but the trade-off is increased load imbalance due to coarser task granularity. It is often difficult to determine the optimal number of iterations to schedule at each step. Small chunks increase the number of scheduling steps (hence scheduling overhead), while large chunks may cause imbalanced load across processors. Kruskal and Weiss (1985) have proposed a scheme to calculate an optimal chunk size based on the number of iterations, the number of processors, the standard deviation of the execution times of individual iterations, and the scheduling overhead. Since it is often difficult to determine the variance among the iteration execution times before executing them and because the variance may depend on the environment of the processor to which they are assigned, this method is not practical for real applications. Several schemes have been proposed to minimize the limitations suffered by both self-scheduling and fixed-size chunking (Hummel et al., 1992; Polychronopoulos and Kuck, 1987; Tzen and Ni, 1991). These schemes are based on scheduling chunks with decreasing number of iterations. Typically, larger chunks are initially scheduled, reducing the scheduling overhead, while smaller chunks are subsequently scheduled to smooth any load imbalances resulting from previous assignments.
2.3
Guided Self-scheduling
In guided self-schedulin (GSS), the size of the chunk scheduled to the next idle processor is [ R I P , where R is the number of remaining iterations (Polychronopoulos and Kuck, 1987). Thus, the chunk size varies from r N / P1 iterations down to one iteration. This algorithm allocates large chunks at the beginning of a loop’s execution to reduce the scheduling overhead. Smaller chunks are then allocated as the number of remaining iterations to be executed decreases. The last P - 1 chunks consist of one iteration that can be used to balance the load, thus increasing the likelihood that all processors finish at the same time. A feature of GSS is that approximately two thirds of
4
58
A. R. HURSON ETAL.
the remaining iterations are allocated over every P chunks (Hummel ef al., 1992). For example, if there are N = 100 iterations to be executed on a fourprocessor system, the sizes of the chunks are: 25, 19, 14, 1 1 , 8 , 6 , 5 , 3 , 3 , 2 , 1, 1, 1, 1. It should be noted that GSS addresses the problem of uneven starting times of processors resulting from the delays in acquiring the chunks. Simulations involving constant-length iterations and uneven processor starting times, as well as iterations with variable-length running times, were conducted and found that GSS performs better than the SS method (Polychronopoulosand Kuck, 1987). The number of scheduling steps required for GSS, in the best case, is P , when the number of iterations N is approximately equal to the number of processors P. Otherwise, the maximum number of scheduling steps is l'Hr,,,,pl, where H,=ln(n)+ y 1/(2n) is the nth harmonic number and y = 0.5772157 is the Euler's constant (Polychronopoulos and Kuck, 1987). For large N this approximates to P l d N / P 1 (Yue and Lilja, 1994a). The number of scheduling steps required for GSS is more than that for SS, but less than that for FS. Although GSS often achieves a balanced load when iteration execution times vary widely, it is still possible that some initial chunks (due to their large sizes) do not complete by the time all other chunks have completed.
2.4
Factoring
Factoring was specifically designed to handle iterations with widely varying execution times (Hummel et al., 1992). Similar to GSS, this scheduling strategy uses variable and decreasing chunk sizes. At each round, factoring schedules half of the remaining iterations into P equal sized chunks. In other words, each chunk contains rR/(2P)1 iterations, where R is the number of unscheduled iterations. Factoring allocates smaller initial chunks than GSS, hence alleviating one of the main problems of GSS. The chunk sizes for N = 100 iterations to be executed on a four-processor system are: 4 chunks with 13 iterations each, 4 chunks with 6 iterations each, 4 chunks with 3 iterations each, 4 chunks with 2 iterations each, and finally 4 single-iteration chunks. The chunk size for factoring is determined by: I$=
I(;)"
j d 0,
where Kj is the chunk size for factoring step j , N is the total number of iterations, and P is the number of processors. The number of scheduling steps can be determined by setting Kjto one and solving equation (1) for j-the number of factoring steps. However, since factoring schedules P
DOALL AND DOACROSS LOOPS
59
equal size chunks per batch (factoring step), the total number of scheduling steps is approximately equal to d 1 . 4 4 I n ( N / P ) 1 (Yue and Lilja, 1994a). As can be seen, the number of scheduling steps for factoring is 1.44 times that for GSS. However, it has been shown that factoring performs better than GSS when the iteration execution times vary significantly (Hummel et al., 1992).
2.5
Trapezoid Self-scheduling
Trapezoid self-scheduling (TSS) is another scheme that is developed for loops with varying iteration execution times (Tzen and Ni, 1991), by using variable and decreasing chunk sizes. TSS attempts to reduce the synchronization cost of obtaining work by individual processors by simplifying the scheduling computations in the critical section. TSS uses a simple linear function to determine the size of the chunk allocated at each step. This method will rely on a programmable number for the size of the first and final chunks, f and 1. The sizes of the chunks between successive scheduling steps are decreased by s = ( f - Z)/(C - I), where C = r 2 N / ( f + f)l is the number of chunks to be scheduled. Thus, the first chunk size is c1= f, and the second is c2 = cl - s. In general, c,,, = c, - s. The typical values for the first and last chunks are f = ( N / 2 P ) and 1 = 1 (Tzen and Ni, 1991). The number of scheduling steps for trapezoid selfscheduling is equal to the total number of chunks C , which ranges from 2P to 4P. For large N , the total number of scheduling steps is approximately equal to 4 P (Yue and Lilja, 1994a). TSS allocates smaller initial chunks than GSS, and requires fewer scheduling steps than factoring.
3. Comparative Analysis of DOALL Loop-scheduling Schemes The advantages and disadvantages of the various scheduling algorithms are summarized in Table I. As can be seen, fixed-size chunking requires the smallest number of scheduling steps while self-scheduling requires the most. Fixed chunking is more efficient since the chunk sizes can be determined at compile time. Unldce fixed chunking, self-scheduling balances the load on processors more evenly; however, the N scheduling steps needed may offset any performance gains. Since the processor must access a shared variable to obtain work, SS also adds delays due to network and memory contention. Factoring requires more scheduling steps than GSS, but the chunk size is computed less frequently (every P steps instead of every step in GSS). Factoring allocates more smaller chunks than GSS in order to balance the
TABLEI COMPARATIVE ANALYSIS OF DOALL SCHEDULLNG ALGORITHMS ~~
Algorithm
Scheduling steps
Chunk size
Selfscheduling (SS)
N”
1
Fixed size chunking (FS 1
Ph
Guided Selfscheduling (GSS)
~~~
~~
Advantages
Disadvantages
Can balance the workload well.
Requires N scheduling steps. Should only be used in systems in which the overhead for accessing shared variables is small. Chances of network and memory contention are very high. Contention for network and memory becomes a major problem.
El
Requires the minimum number of scheduling steps. Chunk size can be determined at compiletime or during run-time before the loop is executed.
May not balance the workload very well, especially if the variance in iteration execution times is large.
El
Trade off between load balancing and scheduling overhead. Number of scheduling steps between S S and FS, and tries to handle variations in iteration times by balancing the workload.
Early chunk could be so large, it does not complete by the time all other chunks have completed. The current chunk size must be calculated at every step.
Factoring
Trapezoid Selfscheduling
P 1.44 In11
4P
c,,
, = c,
- s"
(TSS)
Allocates more smaller chunks than GSS in order to balance the workload. Chunk size only needs to be calculated every P steps.
Requires more scheduling steps than GSS.
The chunk size decreases linearly, hence the difference between the current chunk and the next chunk is constant. The calculation of the current chunk size can be performed in parallel, eliminating the need for a critical section. Fewer scheduling steps than GSS and factoring when the iteration-to-processor ratio is larger than 55 and 16, respectively (Yue and Lilja, 1994a).
The algorithm for computing the chunk size is fairly complex. Allocates larger portions of remaining work to later chunks, which may generate large load imbalances for the last few scheduling steps.
Number of iterations. Number of processors. Number of remaining iterations. s = (f- I ) / ( C - I), where C = r 2 N / ( f + and c , = f. The typical values for the first and last chunks are f = N / 2 P and I = 1 (Yue and Lilja, 1994a). a
/)I,
62
A. R. HURSON ETAL.
load, accounting for the increased number of scheduling steps. The earlier chunks in GSS may take longer to execute than all other chunks, leading to unbalanced load, particularly when the execution time of iterations decreases with increasing indices. It has been shown that when the ratio of the number of iterations to the number of processors is larger than 55 TSS requires fewer scheduling steps TABLEI1
NWER OF ITERATIONS ASSIGNED TO A PROCESSOR AT EACHSCHEDULING STEP W m n = 1o00, P = 4 TSS
f = 125, Step
GSS
1 2 3 4 5 6 7 8 9
250 188 141 106 79 59 45 33 25 19 14 11 8 6 4 3 3 2 1 1 1 1
10 I1 12 13 14 15 16 17
18 19 20 21 22 23 24 25 26 21 28 29 30 31 32
-
FS
Factoring 125 125 125 125 63 63 63 63 31 31 31 31 16 16 16 16 8 8 8 8
4 4 4
-
4
-
1
2 2 2 2 1 1 1
1=1
DOALL AND DOACROSS LOOPS
63
(4P steps) than that required by GSS, and when the ratio is 16, TSS requires fewer scheduling steps than Factoring (Yue and Lilja, 1994a). This is true, because the next chunk size differs from the current chunk size by a constant, and thus the scheduling computation is simpler. In TSS, even later chunks may remain large, potentially causing load imbalance. GSS and factoring, on the other hand, guarantee that the last P chunks contain only one iteration per chunk. These small chunks can be used to balance the finishing times of all processors. Performance of GSS, factoring, self-scheduling, and static chunking have been simulated on the RP3 multiprocessor platform for several benchmark loops (Hummel et al., 1992). This study shows that factoring is scalable, and unlike GSS, its peformance is resistant to variance in iteration execution time. In another study, it was shown that GSS did not perform well when the variance in iteration execution times is large (e.g., adjoint convolution programs) (Yue and Lilja, 1994a). GSS assigns too much work at the beginning of the execution and does not save enough work at the end for balancing the load. Factoring and TSS balance the workload better than the other methods. These studies also suggest that none of the algorithms perform well when N is small since there is insufficient work to offset the overhead of scheduling. Since the scheduling overhead is minimal for staticchunking and fixed-size chunking, they perform better when the variance among iteration execution times is small. Table I1 shows the number of iterations assigned to a processor at each scheduling step for GSS, fixed-size chunking (FS), factoring, and TSS.
4.
DOALL Loop Scheduling on NUMA Multiprocessors
The loop-scheduling algorithms discussed in the previous sections assumed a shared memory with uniform memory access costs, and hence our discussion did not take data locality into consideration. In this section we will introduce scheduling schemes designed for shared memory systems with non-uniform memory access (NUMA) where the memory access cost increases with the distance between the processor and the memory. Such scheduling methods should consider the location of the data to improve the performance of parallel loops. Loop iterations can be viewed as having an afinity to the processor which contains the required data (Markatos and LeBlanc, 1992). To exploit processor affinity, loop iterations are normally scheduled on processors that contain the required data either in their local memories or cache memories. Such an assignment can significantly reduce execution times-by as much as 30-60% (Subramaniam and Eager 1994).
64
A. R. HURSON ETAL.
4.1 Affinity Scheduling Afiniiy scheduling (AFS) is an algorithm which attempts to balance the workload, minimize the number of synchronization operations, and exploit processor affinity (Markatos and LeBlanc, 1992). The affinity of a loop iteration to a particular process is due to: (i) the same data is repeatedly used by successive executions of a loop iteration (e.g., a parallel inner loop within an outer sequential loop), and (ii) the data is not removed from the local memory or cache before it is reused. In AFS, the iterations of a loop are divided into chunks of size rN/P1 iterations and each chunk is statically assigned to a different rocessor. When a processor becomes idle, it takes the next chunk of rl/P iterations from its local work queue and executes them. When a processor com letes its assigned iterations, it finds a heavily loaded processor, and takes a 1/P1 fraction of that processor’s unexecuted iterations and executes them. The initial assignment of chunks to processors in AFS is deterministic. That is, the ih chunk of iterations is always assigned to the i* processor. Normally, this ensures that repeated executions of the loop will access data that is local to the processor. The AFS assumes a balanced load at initial assignment and assigns an equal number of iterations to all processors. Each processor can access its local work queue independent of other processors. As load imbalances occur due to variances in iteration execution times, iterations are migrated from heavily loaded processors to lightly loaded processors. Such migrations can cause the data to be migrated twice; from the heavily loaded processor to the lightly loaded processor to balance the work load, and back to the heavily loaded processor for the purpose of maintaining the original affinities. This in turn could lead to penalties due to cache reload and negate any performance gained from processor affinities. It should be remembered that the overhead is incurred only when load imbalances occur. The synchronization costs associated with accesses to the local and remote work queues are the same and equal to O(Plog(N/P2)). Hence, AFS incurs at most a cost of O ( P log(N/P2) + P log(N/P*)) in synchronization operations or scheduling steps on each work queue (Markatos and LeBlanc, 1992). AFS offers higher performance than the dynamic scheduling schemes previously discussed, since synchronization operations on local work queues are usually less expensive than global synchronization operations. Moreover, network traffic is reduced since processors independently schedule iterations from their local work queues.
f
f
4.2 Partitioned Affinity Scheduling For loops with widely varying execution times, two affinity scheduling algorithms have been proposed (Subramaniam and Eager, 1994). These
DOALL AND DOACROSS LOOPS
65
algorithms are based on the assumption that iteration times vary in a correlated fashion, i.e., the execution time for the ’i iteration is a function of i (for example, a linear function gives a “triangular” iteration space). In this case, a uniform initial allocation of iterations to all processors may result in an unbalanced load. These algorithms are discussed below.
4.2.1
Dynamic Partitioned Affinity Scheduling
The dynamic partitioned aflnity scheduling (DPAS) algorithm is very similar to AFS, except that it balances the load by readjusting the sizes of the allocated partitions on subsequent executions of a loop. This, in turn, reduces the cache reload due to migration of work that occurs in AFS. The algorithm keeps track of the number of iterations that were actually executed by each processor and computes the distribution of iterations for subsequent scheduling steps. The algorithm consists of three phases. (1) Loop initialization phase: As in AFS, this phase partitions the loop iterations into chunks of N I P iterations. This chunk size is not used for further execution steps. (2) Loop execution phase: A processor removes 1/P of the iterations from its local work queue and executes them. If a processor’s work queue is empty, it finds a heavily loaded processor, removes 1/P of the iterations from this processor and executes them. An array called executed is used for keeping track of the actual number of iterations executed by each processor. (3) Re-initialization phase: This phase performs the adjustment to the size of the initial chunks by calculating a new chunk size to be assigned to each processor. Assuming that processors are numbered from 0 to P - 1 , the new chunk size is computed as:
when i = 0 partition-start[i] = 0; partition-end[i] = executed[i] - 1; when i > 0 partition-start[i] = partition-end[i - 1] + 1; partition-end[i] = partition-start[i] + executed[i] - 1; By dynamically changing chunk sizes, the DPAS is cap,-,: of hanc ing imbalances in workloads resulting from varying iteration execution times. The scheduling overhead for DPAS is less than that for AFS, since the synchronization costs associated with remote work queues will decrease on each subsequent execution of the loop. Eventually, the only synchronization operations needed are those associated with local work queues.
66
A. R. HURSON ETAL.
4.2.2 Wrapped Partitioned Affinity Scheduling The wrapped partitioned a@nity scheduling (WPAS) aims to rectify the load imbalances of GSS. Iterations are allocated in a wrapped-around fashion whereby a processor is assigned iterations that are at a distance P (the number of processors in the system) from each other. The implementation of WPAS is very similar to that of AFS, except for the wrapped allocation of iterations to a processor. An example of a wrapped allocation of a loop with 18 iterations indexed from 0 to 17, and for 4 processors is shown below (Subramaniam and Eager, 1994). processor 1: processor 2: processor 3: processor 4:
2 7 4 5
6 11 8 9
10
14
3
15 12 13
0 16
1
17
The wrapped assignment of iterations results in assigning consecutive iterations to distinct processors, thus violating spatial locality. Since cache often misses load blocks of data that may belong to multiple iterations, processors may not be able to take advantage of the data localities resulting from large cache blocks. It is difficult to partition the data to fit the wrapped allocation and yet take advantage of large cache blocks. When the number of processors is small, it is possible that a large block will cache data belonging to successive iterations assigned to the same processor, thus exploiting cache localities.
4.3
Locality-based Dynamic Scheduling
AFS and DPAS assume that the data locality can be exploited only when the data is partitioned and distributed in blocks. The locality-based dynamic scheduling (LDS) algorithm (Li et al., 1993), on the other hand, takes data placement into account, by requiring processors to first execute those iterations for which data is locally available. Thus, the LDS can adapt to any data partitioning methods, including cyclic or block-cyclic. In LDS, the data space is assumed to be partitioned to reside on P processors. When a processor is ready to execute the next chunk, it computes the chunk size as rR/(2P)1. This creates chunks about half as large as those in GSS. The processor must then decide which iterations of the chunk to execute. Unlike other dynamic scheduling algorithms, processors in LDS do not execute iterations of a chunk in the order of the indices. For example, if the data distribution is cyclic, a processor may execute iterations in the following order: p + P , p + 2P, ...,p f S,*P, where S,, is the chunk size assigned to the processor. If data distribution is
DOALL AND DOACROSS LOOPS
67
block-cyclic, the processor will execute iteration in the following order p * B + 1, p * B + 2, ..., p * B + S , , where B is the block size. As with other affinity scheduling methods, in LDS, an idle processor acquires work from a heavily loaded processor. The number of synchronization operations for LDS is O ( P log N ) . Unlike AFS, DPAS, and W A S , in LDS each processor must access a central work queue or location to obtain the size of the next chunk. This can lead to network traffic and synchronization delays.
5. Comparison of Affinity-scheduling Schemes Table I11 summarizes the advantages and disadvantages of the various affinity-scheduling schemes discussed in the previous section. In AFS, DPAS, and W A S , each processor independently schedules iterations from its local chunk. Thus, AFS, DPAS and WPAS do not require a central queue and they reduce the overhead due to network congestion and synchronization delays. Furthermore, since the chunk size to be scheduled on each processor is fixed, there is no need for each processor to calculate a chunk size, leading to a low scheduling overhead. Each processor needs only to remember which iterations are unexecuted. However, in LDS each processor must coordinate with a central queue to schedule iterations. A processor cannot compute the size of the next chunk since chunk sizes are computed dynamically. Scheduling can thus create a potential bottleneck in terms of network congestion or access to a central queue. The synchronization delays from accessing a central queue could force processors to remain idle between scheduling steps. The performance can be somewhat improved by pre-calculating the chunk sizes for all scheduling steps. Scheduling steps of AFS can be expressed as (Markatos and LeBlanc, 1992):
O ( P log(N/P2) + P log(N/P2))
(2)
The first part of this equation is associated with accesses to the local work queue, and the second part shows the accesses to remote work queues. DPAS incurs less scheduling overhead than AFS, since the chunk sizes are dynamically adjusted after the initial allocation. This reduces the contribution of the second part of the above equation for subsequent executions of the loop. However, each processor must keep track of the actual number of iterations already executed. The dynamic computation of the chunk size may be amortized across a large number of iterations executed by each processor. The overhead due to LDS was found to be insignificant compared to AFS (Subramaniam and Eager, 1994). WPAS incurs similar scheduling overhead to AFS with respect to the first part of the overhead equation shown above,
TABLEI11 COMPARISON OF AFFINITY SCHEDULING ALGORITHMS
Algorithm
Scheduling steps
Chunk size
Advantages
Disadvantages
o
d
,b,c Cache-reload overhead incurred only when load imbalance arises.
Affinity scheduling (AFS) Dynamic partitioned affinity scheduling (DPAS)
0
Wrapped partitioned affinity scheduling (WPAS)
Same as DPAS
n b r ,
1
A cache reload may occur for each execution of the loop since different iterations may migrate on different executions of the loop. d
f
Incurs less scheduling overhead than AFS. Improved initial load balance compared to AFS . Performs well for loops with triangular workload.
Requires several executions of the sequential outer loop in order for the partition to converge (iterations > = 4).
u b c
The data has to be partitioned in the same manner as the iterations in order to get the best performance.
9
,
Incurs the lowest scheduling overhead. It avoids assigning all the time consuming iterations to a single processor, minimizing load imbalance. Total number of migrations is significantly less than both AFS and DPAS. Very effective for loops with rectangular workloads, and performs well for triangular workloads (Subramaniam and Eager, 1994).
Localitybased dynamic scheduling (LDS 1
Chunk sizes can be determined before execution to reduce overhead. Data placement is taken into account by always having the processor first execute those iterations for which the data is local to the processor.
O ( P log N )
~~
a
Each processor has to dynamically obtain the next chunk size from a central work queue (scheduling is serialized). Requires more scheduling steps than the other three schemes. Scheduling steps cost more, hence more overheads incurred compared to the other three schemes.
~
Each processor independently schedules iterations from its local partition (scheduling done in parallel). Fixed chunk size, hence no calculation is needed, resulting in low scheduling overhead. Majority of the scheduling is inexpensive, since it accesses the local work queue. Memory locality can only be exploited when data is also partitioned and distributed in blocks.
70
A. R. HURSON H A L .
since both schemes assign the same number of initial iterations to processors. The second part of the equation, which is associated with the migration of iterations from other processors, would be less for W A S . Since WPAS assigns consecutive iterations to distinct processors, it avoids assigning all the time-consuming iterations to a single processor and minimizes the chances of load imbalance. The number of iterations to be migrated due to load imbalance would be less than those for AFS. Even though consecutive iterations are not scheduled on the same processor, scheduling in WPAS is similar to scheduling a loop with a stride of P. It was shown in Subramaniam and Eager (1994) that this additional overhead is negligible, and that the total number of migrations for WPAS is significantly less than those in either AFS or DPAS, both when the load is balanced and when the load is unbalanced. LDS incurs O ( P log N ) scheduling steps (Li et al., 1993). In addition to more scheduling steps, each step of LDS is more expensive since each processor must access a central work queue to obtain the size of the next chunk to be executed. The other three affinity schemes described can perform the majority of the scheduling actions in parallel since they need only to access a local work queue. AFS and DPAS assume that the data is partitioned into blocks. These schemes partition the iteration space into blocks and assign a block of consecutive iterations to each processor. This implies that the memory locality can only be exploited when the data is also partitioned and distributed in blocks, which is normally the case. However, for W A S , it is more difficult to manage the data locality with a wrapped assignment of iterations. LDS takes data placement into account, by requiring processors to first execute those iterations for which data is locally available. The data placement must be known prior to scheduling of iterations to obtain good performance with LDS. Load balancing is inherent in all the schemes discussed since idle processors steal work from heavily loaded processors. However, such migration of iterations may defeat the data locality advantages of affinity scheduling (Lilja, 1994a). Performance results have shown that WPAS is very effective for loops with rectangular workloads (Subramaniam and Eager, 1994), where the execution time of a set of iterations remains the same, while the next set of iterations have a smaller execution time. This is because W A S avoids assigning all the time-consuming iterations to a single processor. Results also show that both W A S and DPAS perform well for loops in which execution times of iterations decrease linearly (triangular workload). This is due to the fact that these two algorithms start with a good initial load balance and minimize migration of work, leading to a reduced cache reload cost. Although DPAS appears to perform best among the four schemes, it does have some limitations. DPAS takes several executions of the outer loop
DOALL AND DOACROSS LOOPS
71
before the sizes of the partitions converge for the inner DOALL loop. It was found that convergence is not possible unless the outer sequential loop is executed at least four times (Subramaniam and Eager, 1994). When the number of inner loop iteration changes with each outer loop execution, DPAS must compute new adjustments for chunk size (since previous adjustment would be based on a different number of inner loop iterations). All four affinity scheduling schemes rely on processors snooping on other processors for finding additional work. This implies that these affinity schemes are suitable only for bus-based systems.
6. DOACROSS Loop Scheduling Chen and Yew (1991) have used an event-driven simulator to measure the parallelism inherent in application programs. Six real application programs from the PERFECT benchmark suite were used in their study. They observed that the loss of parallelism after serializing DOACROSS loops was very significant. This supports the need for good schemes for the parallel execution of DOACROSS loops. DOACROSS loops can be classified as regular and irregular loops. In a regular DOACROSS loop, dependence distances are constant while the dependence distance varies from iteration to iteration in irregular DOACROSS loops. Regular DOACROSS loops are easier to parallelize than irregular loops.
6.1 The Regular
DOACROSS Model
Cytron (1986) developed a DOACROSS model for the execution of loops with some degree of parallelism among various iterations. Consider a single loop L with s statements (S,, Sz, .. ., S,) and N iterations. If T ( S , ,S,) is the execution time of statements Si through S, (iGj),then the DOACROSS model has d = 0 for vector loops, d = T ( S , , S,) for sequential loops, and 0 < d < T ( S , ,S , ) for loops with intermediate parallelism. In this model, each iteration is assigned to a virtual processor and execution of two successive virtual processors is delayed by d time units. This is similar to cyclic scheduling discussed earlier. In general, the delay d can range from zero (the vector loop case) to T (the sequential loop case), where T is the execution time of one iteration of the loop. The total execution time for a DOACROSS loop L with N iterations for an unbounded number of processors is:
T E ( L )= ( N - l)d + T
(3)
72
A. R. HURSON ETAL.
When there are only P processors, the execution time is (Polychronopoulos and Banerjee, 1987):
T E ~ ( L ) (=r N / P l - l ) m =
( ~ , ~ d ) + ( ( ~ - i ) m o d ~ )(4) d + ~
In Section 1, it was stated that data dependence can be either lexically forward (data from higher indices is used by iterations with lower indices) or lexically backward (data from lower indices is used by iterations with higher indices). Normally, lexically forward dependencies (LFD) do not contribute to delays in executing loop iterations. Sometimes a lexically backward dependence (LBD) can be transformed into a lexically forward dependence by reordering the statements of the loop if the statements do not form a dependence cycle (Cytron, 1986). DOACROSS loops where the LBD cannot be transformed into LFD lead to delays in executing successive iterations. Hence, we focus our attention on regular DOACROSS loops with LBD. Consider two statements of a loop, S4 and S8, where S, lexically precedes &. Statement S, of iteration Z, computes the data used by statement S, of iteration I,. The semantics of sequential programs guarantee that iteration Z2 is executed before iteration I,. If these two iterations were assigned to different processors, a delay must be introduced for executing iteration I , , such that statement S, of iteration I , executes before statement S, of iteration I,, in order to satisfy the dependence. Hence, a delay d equal to 5 statements must be introduced to iteration I , . This loop example exhibits a lexically backward dependence. The DOACROSS loop of the example shown in Figure 1 has N = 8 iterations, a delay d = 4, and a loop execution time T = 10. The parallel execution of the loop on three processors takes 38 time units, resulting in a speed-up of 2.1. Communication cost among processors is not included in this model. The overall execution time depends on the communication cost due to interiteration dependencies (Su and Yew, 1991). For a shared memory system, the delay d should include not only the delay due to the lexically backward dependence (LBD), but also the delays in accessing shared variables. For distributed memory systems data must be shared using messages which take several orders of magnitude longer than a processor execution cycle. It has been reported that the Intel iPSC/l, iPSC/2, and iPSC/860 hypercubes have communication/executionratios of 26, 59, and 1000 respectively, while the ratio for the nCube 3200 and 6400 hypercubes are 30 and 107, respectively (Dunigan, 1991). Performance studies on CM-5 also show that some classes of problems are communication limited on that machine (Kwan et al., 1993). Using a balance factor b = tcom/tcomp, t,,,,,,,, = fmd - fcomp, a system is communication limited if b a 1. The Laplace solver on a 256 node partition has resulted in balance factors ranging from 2.1 1 for an 8192 x 8192 mesh size to 14.38 for a 64 x 64 mesh size. If we let d be the delay due to the
DOALL AND DOACROSS LOOPS
73
LBD, and C be the total communication and synchronization cost (communication delay in the sequel) incurred, then the execution time of a DOACROSS loop with N iterations on P processors is
T E ~ ( L=) (rN/Pl- 1) max { T , ~ ( + dc)1 + ( ( N - 1) mod ~ ) ( +d c)+ T (5) Here for every delay d due to LBD, a comrnunication/synchronization cost C is added, increasing the dependency delay from d to d i C. For the example of Fig. 1 , if we assume a communication delay C = 6, the total parallel execution time becomes 80 time units, leading to a speed-up of 1. Increasing or decreasing the number of processors will not change the execution time. Larger values for C will make a parallel execution of the DOACROSS loop ineffective and lead to under-utilization of processors as they remain idle between the termination of an iteration and the initiation of the next assigned iteration. Dynamic scheduling schemes such as GSS and factoring are not effective in scheduling DOACROSS loops. When chunks (a number of consecutive iterations) are assigned to processors, iterations in successive chunks must wait for the completion of all iterations in the preceding chunks. Since chunk sizes are greater than one, the delay among processors assigned successive DOACROSS I = 1 , 8 {d= 4)
:1 1 0
END P1
P2
P3
:I 8
12
32 36
40L FIG 1. Allocation of a DOACROSS loop.
74
A. R. HURSON ETAL.
iterations is now equal to ( n - l)T + d + C , where n is the size of the chunk assigned to the previous processor. The total execution time of the DOACROSS loop shown in Fig. 1 using either GSS or Factoring is 56 when C = 0, and 80 when C = 6. Both schemes reduce the amount of communication overhead when compared to cyclic scheduling, at the expense of reduced parallelism. They perform worse than the cyclic method when the communication cost is zero; but with non-zero communication cost, they perform no worse than cyclic scheduling. The execution time for the same example using static chunking is 68 when C = 0, and 80 when C = 6. Thus, static chunking performs better when the communication cost is significant, since it only incurs ( P - 1) communication delays.
6.7.7
Pre-synchronized Scheduling (PSS)
Krothapalli and Sadayappan (1990) proposed a dynamic scheduling scheme called pre-synchronized scheduling (PSS) for eliminating processor idle cycles that result from scheduling schemes such as GSS and Factoring. Here, iterations are scheduled only when their data dependencies and synchronization requirements are met. Loop iterations are uniquely identified using indices, and a ready queue of enabled iterations is maintained by a global control unit (GCU). An idle processor gets an id from the ready queue and executes it. When the execution is complete, successor loop iterations that are enabled are added to the ready queue. A DOACROSS loop is divided into two separate loops and scheduled separately. The two loops correspond to a T - d portion of the loop that can be executed in parallel, and a d portion that must wait for synchronization from previous iterations. This method introduces a scheduling/synchronization overhead equal to 2N, resulting from the fact that we now have 2N loop iterations to schedule. The performance of the PSS is largely dependent on two factors: (1) how the ready queue is implemented, e.g., FIFO, priority; and (2)the scheduling cost. Even though the T - d portion is a parallel loop (akin to DOALL), it is necessary to schedule the iterations in proper to facilitate an interleaved execution of iterations from the T - d portion and the d portion of a DOACROSS loop. For the example of Fig. 1, the best performance achievable with PSS is 38, when the scheduling cost is ignored. This is comparable with that achieved by the Cyclic scheduling scheme. However, the PSS scheme incurs significant scheduling costs. We need to include a communication (C) each time an idle processor obtains the id of a loop iteration from the ready queue and a communication (C) to update the ready list when a processor completes a loop iteration. Since PSS schedules 2N loop iterations, we have a 4CN communication cost. For example, with C = 6, the execution time for the loop shown in Fig. 1 will become 146.The
DOALL AND DOACROSS LOOPS
75
cost can be reduced by assigning several loop ids (i.e. a chunk) each time an idle processor accesses the ready queue. However, it is difficult to arrive at an optimal chunk size.
6.1.2 Staggered Distribution Scheme A staggered distribution scheme (SD) was originally developed for multithreaded dataflow multiprocessors (Hurson er al., 1994a; Lim et al., 1992). Here loop iterations are unevenly distributed among processors in order to mask the delay caused by data dependencies and communication. Performance studies have indicated that this scheme is effective for loops containing large degrees of parallelism among iterations. It has been observed that near optimal speed-up can be attained even in the presence of communication delays. In order to use SD for the DOACROSS loop, the loop is separated into two loops, in a manner similar to that of the PSS method. The first loop (the T - d portion) is scheduled according to the following policy: the iterations assigned to PE, succeed the iterations assigned to PE,-, and PE, is assigned m more iterations than PE,_,. This results in a more iterations assigned to higher numbered processors. For example, with six iterations and three processors we may assign 1, 2 and 3 iterations respectively to the three processors. The main objective of the SD scheme is to mask the delay due to data dependencies and communication in executing the second (d-portion) loop iterations by assigning more T - d loop iterations. The number of additional iterations assigned to PELis given by
where nz-,is the number of iterations allocated to PE,_,, T is the execution time of one iteration, d is the delay and C is the inter-processor communication cost. The total number of iterations n, allocated to PE, would be
Thus, the staggered distribution masks delays resulting from the lexically backward among loop iterations and the communication delays involved in transmitting the dependent data among processors since later processors execute more ( T - d) loop iterations. The performance of this scheme can be fine tuned (to accommodate different communication costs) by selecting an appropriate number of iterations n , , assigned to the first processor. Note that equation (7) also determines the maximum number of processors needed to
76
A. R. HURSON ETAL.
execute the DOACROSS loop with N iterations. The synchronization overhead is only ( P - l)*C, which is smaller than that incurred by cyclic scheduling and PSS methods. For the example shown in Fig. 1, we use a distribution of 2-3-3 iterations when C = 0, giving 44 units of execution t h e and a speed-up of 1.82. When C = 6 we use a distribution of 1-2-5 iterations, giving 50 units of execution and a speed-up of 1.6. Staggered distribution accounts for different communication costs by selecting an appropriate n,. The Staggered scheme however, distributes an uneven load among processors; heavily loading later processors.
6.1.3 Cyclic Staggered Distribution (CSD)
A modified version of the staggered scheme called cyclic staggered distribution (CSD) was proposed to address the uneven load distribution (Hurson et al., 1994b). CSD will also handle loops with varying iteration execution times. The CSD has been found to be effective when the number of processors is less than those needed by the staggered scheme (rnaxpe). Unlike using n , iterations, CSD will start with one iteration assigned to the first processor, and n, iterations to the remaining P - 1 processors based on equation (7). The remaining iterations are redistributed to all P processors based on the staggered allocation. Note that the delay that must be masked by higher numbered processors now is smaller than that in the original SD approach, since some loop iterations would already have completed due to prior staggered allocation. Thus a smaller number of additional iterations are assigned to PE,, as compared to equation (7). The number of iterations n, for this new scheme will be (8)
where n, is the number of iterations previously allocated to processor PE,. This approach results in a more balanced load and improved speed-up than the original staggered scheme on P processors. When the execution times of loop iterations vary, CSD can use estimated worst case iteration execution time (possibly augmented by runtime support to adjust these estimates with actual execution times) in determining the distribution for the second and subsequent passes.
6.2 Comparison of DOACROSS Scheduling Schemes Table IV compares the characteristics of the four DOACROSS loop allocation schemes discussed. Cytron’s cyclic scheduling scheme for
DOALL AND DOACROSS LOOPS
77
TABLEIV COMPARISON OF DOACROSS SCHEDULING ALGOrUTHMS Algorithm
Advantages
Disadvantages
Cyclic scheduling (cyclic)
Exploits the parallelism present in a DOACROSS loop.
Does not take into consideration the effect of inter-processor communication cost. Overhead increases linearly as a function of ( n - 1)(C + d). Offers low hardware utilization.
Pre-sy nchronized scheduling (PSS)
Balances the load and eliminates busy waiting period. Iterations are scheduled when their synchronization requirements are met.
Introduces scheduling overhead equal to 4CN. No implementation details on the ready queue management were presented. The performance of this scheme is unacceptable if the scheduling cost is significant.
Staggered distribution scheme (SD)
Considers the effect of both delay Produces an unbalanced load among processors, with the (d) and communication cost (C). higher numbered processors Automatically controls and receiving the larger amount of determines the maximum number of processors required for work. efficient execution of the loop based on the physical characteristicsof the loop and the underlying machine architecture -higher resource utilization. Lowest scheduling overhead.
Cyclic staggered distribution (CSD)
Balances the load by cyclically Advantages are only possible if the number of PEs available is less assigning the remaining iterations to processors, and at the same time than rnnrpe (Hurson et ol., masking out the effect of both 1994b). delays due to LBD and communication. Increases the amount of communication delay, relative to the SD, but simulation results have shown that it still improves the performance and offers a higher speed-up (Hurson et al., 1994b).
78
A. R. HURSON ETAL.
TABLEV N W E R OF ITERATIONS ASSIGNED TO A PROCESSOR AT EACH SCHEDULING STEP WITH T = 10, d = 2, n = 500, = 5 , P =4.
c
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47..500
Total execution time
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3..
1
1 1 1 1 1 1 1 1 1 1
1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 3503
1 2 4 6 7 9 10 11 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 6 -
1298
DOALL AND DOACROSS LOOPS
79
DOACROSS loops does not take into consideration the effect of interprocessor communication cost. When the communication delays are significant, the overhead of this scheme increases as a function of ( n - 1)(C + d ) , and is independent of the number of PEs. This scheme offers low hardware utilization as a result of the processor idle cycles between the termination and initiation of successive iterations assigned to the same PE. Re-synchronized scheduling (PSS) while attempting to balance the load and eliminate idle cycles, introduced scheduling overhead proportional to 2 N and a communication cost of 2C per iteration. The Staggered distribution scheme (SD) accounts both for the processor delays due to LBD and communication. This is achieved by assigning a varying number of loop iterations to processors. This scheme achieves better results than the previous algorithms, and utilizes an optimal number of processors. The major weakness of the staggered scheme is the uneven load assignment to processors. The cyclic staggered distribution (CSD) answers this load imbalance of SD. It should be noted that CSD results in larger communication delays for a loop than that with SD; however, the more balanced load of CSD leads to a better performance, particularly when the number of processors is less than the optimal number required for SD (Hurson ef al., 1994b). We have conducted simulation studies for determining the number of iterations assigned to processors at each scheduling step using the various schemes described. The results are shown in Table V. Static chunking was included for the sake of completeness and because it performed better than the cyclic scheme when the communication cost is significant. The total execution time for SC shown in Table IV was obtained by separating the loop into two separate loops as done in SD and CSD. Re-synchronized scheduling (PSS) was not included in Table V, since the authors have not suggested any strategies for managing the ready list, malung an accurate analysis difficult. As discussed earlier, PSS in general performs poorly when the communication costs are significant. Table V shows that the cyclic scheme has the worst performance, followed by static chunking. The cyclic staggered scheme (CSD) produced the best performance.
6.3 Irregular DOACROSS Loop Scheduling Regular DOACROSS loops have constant distance dependence patterns which can be determined during compile-time. For irregular DOACROSS loops, the dependence patterns are complicated and usually are not predictable at compile-time. An example of an irregular DOACROSS loop is shown in Fig. 2. Here the dependency between loop iterations is based on the content of arrays B and C. Hence, the dependence relation cannot be
80
A. R. HURSON ETAL.
DO I = l , N s p : A ( B ( I ) 1 :=. . . :=A(C( I )I+. sq : END
. .
.. .
FIG 2. Irregular DOACROSS loop.
determined until runtime. We need to consider approaches that are different from those used for regular DOACROSS loops to achieve good performance for irregular loops.
6.3.1 Pre-synchronized Scheduling The pre-synchronized scheduling (PSS) for scheduling presented earlier can also be used for irregular DOACROSS loops (Krothapalli and Sadayappan, 1990). In PSS, iterations are scheduled only when their synchronization requirements are met. Each iteration of a loop is uniquely identified by its index value. The dependence relations among iterations of a loop are represented as a directed graph called iteration space graph (ISG). The nodes in ISG represent the iterations while the edges show dependence relationships between iterations (Fig. 3). Edges are not differentiated for flow dependence, anti-dependence, and output-dependence. The number of predecessors of each iteration is computed from the ISG and stored in a trig-count array. Figure 3(b) shows six nodes in the ISG format, each corresponding to an iteration of the loop in Fig. 3(a). The edges in Fig. 3(b) represent inter-iteration dependencies corresponding to the values of array B shown in Fig. 3 (c). For instance, there is a flow-dependence from iteration 1 to iteration 3, because iteration 1 writes to location A(1) and iteration 3 reads from the same location. Similarly, there is an anti-dependence from iteration 2 to iteration 3 through the element A(3). Thus, the trig-count for iteration 3 is 2, and iteration 3 cannot start until both its predecessors, iterations 1 and 2 have completed. Initially, iterations without any predecessors (i.e., trig count = 0 , iterations 1, 2 and 4 in Fig. 3(b) are placed on the ready queue managed by a global control unit (GCU). An idle processor obtains an iteration id from the queue and executes the loop body for that index. Upon completion of the execution, the processor updates the trig-counts for all successor iterations. The GCU is informed of the updates by transmitting an instruction packet to the GCU. The GCU decrements the appropriate trig-count by one. The iterations with a zero trig-count is placed on the ready queue. The algorithm for generating the ISG can be found in Krothapalli and Sadayappan (1990). The algorithm executes a skeleton of the loop in two passes and generates a trace of the memory
DOALL AND DOACROSS LOOPS
81
(c) FIG 3. Illustration of irregular DOACROSS loop execution scheme: (a) an irregular loop;
(b) iteration space graph of Fig. 3(a); (c) values of B .
references. These memory traces are used for identifying data dependencies. In the first pass, flow and output-dependencies are identified, while antidependencies are identified in the second pass (by executing the skeleton to generate the reverse trace of references to memory locations). Obviously, the construction of the ISG at runtime introduces overhead. However, for irregular loops, some runtime analysis is necessary regardless of the actual scheduling strategy used for loop allocation. It is observed that in some scientific applications using iterative techniques, or applications that model the behavior of a structurally invariant physical system through time, the same dependencies among iterations exist for repeated executions. The overhead in computing the ISG for such applications can be amortized over the repeated executions of irregular loops. PSS incurs scheduling overhead proportional to N , and a communication cost of 2C for each iteration, leading to an overall cost of 2CN. An analysis of the trade-off between the overhead and the increased parallelism in executing irregular loops depends on the application itself. The PSS scheme requires a GCU to manage the ready queue and means of communicating updates of the trigcount array.
82
A. R. HURSON ETAL.
6.3.2 Run time Paralleliza tion Schemes Runtime parallelization schemes perform dependence analysis at runtime and, depending on the dependencies, executes the loop in parallel (Chen et al., 1994). For example, consider the case where the arrays B and C in Fig. 2 are not available until runtime. All three types of dependencies (flow, anti and output) between instances of statements S, and S, are possible. When B ( l ) = C(3) = B(4) = B ( 5 ) = x the following dependencies result:
Sp (1)flow
-+
Sq(3)anti -+Sp (4)output-+ Sp(5)
It is normally assumed that the values of B and C do not change during the execution of the loop. In general, runtime parallelization schemes require an inspector and an executor. The inspector determines the dependence relations among the data accesses, while the executor uses this information to execute iterations in parallel. If both the inspector and the executor are parallel algorithms, the scheme can take full advantage of parallel machines. The key to the success of these schemes is to reduce the communication overhead between the inspector and the executor.
The Zhu- Yew Runtime Parallelization Scheme (ZYRPS). Zhu and Yew proposed a runtime parallelization scheme that is general enough to handle any dependence pattern (Zhu and Yew, 1987). We will call this ZYRPS method. Using ZYRPS, the loop in Fig. 2 will be transformed into the form shown in Fig. 5. Figure 4 outlines the transformation. Two fields are associated with each element of array A : the data field stores the data value while the key field is used to order accesses to the array elements. Here, an iteration i is allowed to proceed only if all accesses to the array elements A ( B ( i ) ) and A ( C ( i ) )by all iterations j < i have been completed. The inspector determines the set of iterations that can proceed, by having all unexecuted iterations visit the array elements they need to access and store their own iteration number in the key field of these elements if it is less than the value already stored. After doing so, the numbers that are now remaining in the key fields are the numbers of the iterations that can proceed. In the executor phase, iterations check to see if the key field of the elements they need to access have values that are equal to their iteration indices. If so, no unexecuted predecessor exists and the loop iteration is allowed to proceed. Once iteration i completes, Done(i) is set to TRUE and the process continues until all iterations are executed. This approach has two limitations (Chen et al., 1994). First, the inspector cannot be reused across different invocations of the same loop, even if there is no change in dependencies, since the inspector and the executor are
03
DOALL AND DOACROSS LOOPS
R e o e a t u n t i l a l l i t e r a t i o n s h a v e been e x e c u t e d I N S P E C T O R PHASE
I n i t i a l i z e a l l key f i e l d s t o i n f i n i t y . f o r a l l u n e x e c u t e d i t e r a t i o n s ( 7 = i t e r a t i o n number) I f i t e r a t i o n number < key f i e l d o f A ( B ( i ) ) t h e n R e p l a c e t h e key f i e l d o f A ( B ( i ) ) w i t h i t e r a t i o n number I f i t e r a t i o n number < key f i e l d o f A ( C ( I ) ) then R e p l a c e t h e key f i e l d o f A ( C ( I ’ ) ) w i t h i t e r a t i o n number. “The key f i e l d s now c o n t a i n t h e ( l o w e s t ) i t e r a t i o n numbers t h a t a r e now a l l o w e d t o a c c e s s t h e s e a r r a y e l e m e n t s . A1 1 p r e d e c e s s o r i t e r a t i o n s h a v e already accessed t h e s e array elements.” EXECUTOR PHASE F o r a l l unexecuted i t e r a t i o n s ( i
I f i t e r a t i o n number
=
= i t e r a t i o n number) key f i e l d o f b o t h A ( B ( I ) ) and
A ( C ( 7 ) ) then
Execute loop body. “ T h e key f i e l d s o f b o t h a r r a y s must m a t c h t h e i t e r a t i o n number i n o r d e r f o r i t t o p r o c e e d . I f t h e y b o t h match then a l l p r e d e c e s s o r i t e r a t i o n s t h a t a c c e s s t h e s e a r r a y e l e m e n t s h a v e a l r e a d y been e x e c u t e d . ”
FIG 4. Pseudocode of transformed of loop Fig. 2 using the Zhu-Yew scheme.
tightly coupled. Second, the execution of iterations with dependencies cannot be overlapped. The executor checks the key fields of all the accesses needed by an iteration and executes an iteration only if all key fields contain a value that is equal to the iteration index. This limitation not only reduces the amount of parallelism present, but also causes unnecessary traffic since all key fields have to be inspected. For the example of Fig. 2, 3r memory accesses are required for each iteration, where r is the number of references to array A per iteration.
Chen‘s Runtime Parallelization Scheme (CRPS). In order to address the limitations of the previous scheme, Chen et al. (1994) proposed a new algorithm that reuses the inspector results across loop invocations and permits the overlap of dependent iterations. This is done by separating the
a4
A. R. HURSON ETAL.
Done ( 1: N ) = . FALSE. REPEAT UNTIL ( ( D o n e ( i ) . E Q . . T R U E )
for a l l i )
Inspe c t or Phase DOALL i = l , N A ( B ( i )1 . key=A ( C( i ) ) . k e y = END DOALL DOALL i = l , N I F (Done ( i 1 . E Q . . FALSE 1 “ t h e next t w o i n s t r u c t i o n s a r e atomic” c o r n p a r e & s t o r e {i f ( A ( B ( i 1 ) . k e y > ? ) { A ( B ( i 11. k e y c i ; I I c o r n p a r e & s t o r e l i f ( A ( C ( i) ) . k e y > i ) 1A ( C ( i 1 1 . k e y c i : I t END I F END DOALL
Executor Phase DOALL i = l . N I F ( D o n e ( i ) . E Q . .FALSE) I F ( ( A ( 5 ( i ) ) . k e y . E Q . i ) . A N D . ( ( A ( C ( i ) ) . k e y . E Q . 7 ) THEN
... ... ... . . . =A ( C ( i ) ) . d a t a + . . .
A(B(i)).data=
Done( ? ) = . T R U E END I F END I F EN0 DOALL END R E P E A T FIG 5. Transformed loop of Fig. 2 using the Zhu-Yew scheme.
inspector and executor phases. All the dependence information is gathered and stored in a table called Ticket by the Inspector. This information is then used in one or more executor phases. To reduce the time in building the Ticket, each processor builds the table in parallel. This method, however, is very expensive, since it requires some interprocessor communication. The algorithm tries to minimize interprocessor communication by constructing
DOALL AND DOACROSS LOOPS
85
the table first locally (focal inspector phase) and then combining the local tables during a global inspector phase. In the inspector phase, the references accessing the same location are ordered (i.e., serial execution order) while maintaining the original dependencies. Processors share the Ticket table, whose rows (i) correspond to iteration while the columns ( j ) make the order of the references correspond to a shared location. An example is shown in Fig. 6, which shows the Ticket table for the loop of Fig. 2, for an array reference A ( x ) , with the following dependence relationships:
B(l)= C(3) = C(4) = B(7)= B(9)= C(9) = C(11)= x The first column of the Ticket table represents the accesses to B ( i ) and the second column represents C ( i ) . The first access to B(l)corresponds to the Ticket(1,l)and an initial value of 0 is stored there. Ldcewise the second
1
0
2
3
1
4
2
5
i
6 7
3
8
9
4
5
10 11
6
FIG 6. Example Ticker table for the loop in Fig. 2.
86
A. R. HURSON H A L .
access C ( 3 ) corresponds Ticket(3,2) and a value of 1 is stored there. Similarly, a value of 2 in Ticket(4,2) corresponds to the third access for C(4), and so on. The objective is to store the order of accesses involved in this chain of dependencies and this order is enforced by the executor. In the executor phase, an extra field key is associated with each shared element to enforce the order of accesses. This field is initialized to 0 and after each access the value is updated to permit next access in the dependence chain. At any time, the key value indicates the permitted access as indicated by the Ticket table entries. When a processor is executing the i" S e t a l l key f i e l d s t o 0 . For e a c h i t e r a t i o n ( 7 ) Busy-wait u n t i l f i r s t a c c e s s key=Ticket t a b l e e n t r y ( T i c k e t ( i . 1 ) ) . A c c e s s t h e d a t a and e x e c u t e f i r s t p a r t o f l o o p b o d y . A ( B ( i ) ). d a t a = . . . I n c r e m e n t t h e key by o n e . Busy-wait u n t i l second a c c e s s key=Ticket t a b l e e n t r y ( T i c k e t ( 7 , Z ) ) . A c c e s s t h e d a t a and e x e c u t e s e c o n d p a r t o f l o o p b o d y . . . . = A ( C( i ) 1 . d a t a + . . . I n c r e m e n t t h e key by o n e . FIG 7. Pseudocode of the executor algorithm for the loop in Fig 2.
A ( : ).key=O DO i = l . N busy-wait i n g DO W H I L E ( A ( B ( 7 ) . k e y ! = T i c k e t ( 7.1) ) access t h e d a t a A ( B ( 7 ) ).data=. .. increment t h e key A ( B ( i 1 1. k e y + +
.... busy-waiting DO WHILE ( A ( C ( i ) ) . k e y ! = T i c k e t ( i . 2 ) access t h e data ... = A ( C ( i ) ) . d a t a + . . . i n c r e m e n t t h e key A ( C ( 7 ' ) ).key++
.... ENDDO
FIG 8. Executor algorithm for the loop in Fig.2.
1
DOALL AND DOACROSS LOOPS
87
iteration of a DOACROSS loop, for each access j in the iteration, Ticke t ( i , j ) gives the sequence number S. The processor must wait to access the shared structure A ( x ) until the key for this element becomes equal to S. For the example in Fig. 6, a processor executing iteration 4 will wait until the key value of A(C(4)) becomes 2 before accessing A(C(4)). The key is incremented to 3, permitting the access by the processor executing iteration 7. A pseudo algorithm and FORTRAN code for the executor algorithm are shown in Figs. 7 and 8. A SPMD form of the same algorithm can be found in (Chen et al., 1994). Static cyclic scheduling was chosen for scheduling iterations in the study.
6.4 Comparison of Irregular DOACROSS Scheduling
Schemes The characteristics of the three approaches for scheduling irregular DOACROSS loops are summarized in Table VI. Any approach for irregular DOACROSS loops requires some runtime analysis, which adds to the overhead of scheduling and synchronization delays. However, this overhead may be amortized over the repeated execution of the DOACROSS loop in some scientific applications that use iterative techniques and in applications that model the behavior of a structurally invariant physical system through time. In Re-synchronized scheduling (PSS), the runtime overhead is in the construction of the 1%. For runtime parallelization schemes, an inspector phase is necessary to determine the dependencies between iterations. PSS uses generated traces of memory references for determining data dependencies. Flow and output dependencies are first resolved (phase 1) and anti dependencies are resolved in the second phase. The two-phased approach for determining dependencies can lead to more complex implementation along with higher overhead. The two runtime parallelization schemes, on the other hand, use a single-phase algorithm and records accesses made by all iterations to capture all types of dependencies (viz., flow, anti and output). This reduces the complexity of the algorithm and overhead. Unlike the other approaches, the runtime overhead for ZYRPS cannot be amortized across multiple executions of an inner DOACROSS loop because the inspector and executor phases of the algorithm are tightly coupled, making it impossible to reuse dependence information across multiple executions of the loop. Chen’s scheme (CRPS) permits the overlapped execution of operations in dependent iterations, since the algorithm analyses dependencies based on accesses to shared structures. This ability to overlap dependent iterations may increase the amount of parallelism in the inspector and executor phases. In addition, it removes redundant operations in the inspector (unlike the
88
A. R. HURSON ETAL.
TABLE VI COMPARISON OF IRREGULAR DOACROSS SCHEDULING A L G O W S ~
Disadvantages Pre-synchronzied scheduling
(PSS 1
Zhu-Yew runtime parallelization scheme (ZYRPS 1
Runtime analysis phase is independent from their execution phase. If computations with the same dependenciesare repeatedly iterated over, the enhanced parallelism realized can offset the overhead of performing runtime analysis. No busy-waiting is introduced as well as unnecessary memory accesses.
The algorithm for generating the ISG introduces complexity and could increase overhead. No overlap of dependent operations.
The inspector phase is tightly Utilizes a single algorithm that coupled to its executor phase, simply checks the accesses making independent execution made by all the iterations in of both phases impossible. order to detect all three types of dependencies. This reduces Causes redundant traffic and requires the complexity in the algorithm. several memory accesses, since the inspector will inspect the iteration more times than is required. No overlap of dependent operations.
Similar to PSS, the parallelism Chen’s runtime can offset the overhead of parallelization scheme (CRPS) performing runtime analysis if the same computations are repeatedly iterated over, since the runtime analysis phase is independent from their execution phase. Similar to ZYRPS,utilizes a simple algorithm that checks the accesses made by all iterations in order to detect all types of dependencies. Only scheme that allows the overlap of dependent operations. Removes redundant operations of ZYRPS in the inspector phase.
Increased spin locking during execution. Deadlock is possible. Increased accesses to memory locations. Utilizes static cyclic scheduling, which might not be able to balance the load very well if there is a variance in iteration execution times.
DOALL AND DOACROSS LOOPS
89
ZYRPS). The main weakness of the CRPS algorithm is the delays that can result in waiting for the key field to match the access order. Deadlocks could occur in cases where the iterations are randomly assigned to processors, since all iterations could be waiting for their turn (possibly on different shared elements). The simplicity of PSS may outperform CRPS, even though PSS does not overlap the execution of dependent iterations. Unlike CRPS, a processor in the PSS approach obtains only an iteration that is ready for execution, thus eliminating the need for further synchronization on key fields (and avoid deadlocks). In Z Y R P S , the executor checks the key fields of all the accesses needed by an iteration and only executes an iteration if all key fields are equal to the iteration number. As mentioned earlier, this limitation not only reduces the amount of parallelism but also causes more repeated memory accesses than are really needed to inspect dependencies. The studies made by Chen utilized the static cyclic method for scheduling iterations, which may lead to load imbalances across processors. One could have used self-scheduling as done in PSS. The self-scheduling would have incurred a scheduling overhead proportional to 2CN (refer to Section 6.1.1). It was suggested (Chen et al., 1994) that the loops can be executed in a single-program-multiple-data form by distributing the iterations equally among the processors. However, a naive distribution of iterations could lead to load imbalances across processors, since the order of accesses to shared structures affects the order of execution of iterations. A study to compare CRPS with ZYRPS was performed using a set of parameterized loops running on a 32-processor Cedar shared-memory multiprocessor (Chen et al., 1994). Loops with varying number of iterations and references were used. The results show that CRPS yields speed-ups as high as 14 when the inspector is not reused and as high as 27 when the inspector is reused. CRPS consistently outperformed ZYRPS.
6.5 Other Research Since DOALL loops are easy to parallelize, several heuristics were proposed and studied. In addition to the approaches presented in Section 2, researchers have explored other dynamic scheduling of DOALL loops. It was believed that despite the runtime overhead incurred by dynamic scheduling approaches, dynamic scheduling of DOALL loops could lead to better execution times than those using static schemes. Exploiting parallelism among DOACROSS iterations is much more difficult because of the inter-iteration dependencies. Scheduling such loops must overcome communication and synchronization costs when dependent iterations are scheduled on different processors. Developing better synchronization
90
A. R. HURSON ETAL.
schemes for efficient execution of DOACROSS loops must be discovered. Su and Yew proposed a DOACROSS execution scheme which utilizes direct communication and static message passing (Su and Yew, 1991). This scheme exploits the nearest shared memory feature of distributed shared memory multiprocessors. In this method, either the producer writes (or sends) the data in the nearest shared memory module or the data is bound to a buffer location at compile time. The compiler can generate the necessary instructions for utilizing these features and execute DOACROSS loops in parallel with reduced communication costs. The researchers have also investigated conditions under which the message buffer size can be greatly reduced. Researchers are also investigating techniques for reordering statements in a DOACROSS loop to maximize the parallelism and to minimize interiteration dependencies. Since the optimal reordering is NP-complete, heuristics have been proposed (Chen and Yew, 1994b). Statement reordering may also reduce the amount of synchronization needed for accessing shared data items (Chen and Yew, 1994a; Krothapalli and Sadayappan, 1991).
7.
Summary and Conclusions
There has been considerable interest in parallelizing loops since they are the major source of program parallelism. In this chapter, we examined how loops with inter-iteration dependencies (DOACROSS), and without dependencies (DOALL) can .be executed in parallel. Both static and dynamic scheduling approaches were studied. The various approaches presented in this article were also compared for their complexity, scheduling overhead, communication cost, processor utilization, and expected speed-up. Yue and Lilja (1994a) measured performance of the different DOALL scheduling algorithms on two different types of loops. The first loop is a matrix multiplication program which is parallelized on the outer loop. The size of the parallel tasks is large and all the iterations have the same number of operations so that the variance in iteration execution times is small. The second loop is based on the adjoint-convolutionprocess. It is parallelized on the outer loop and, in contrast to the first loop, each parallel iteration has a different number of operations so that it has a large variance in iteration execution times. The results are shown in Figs. 9 and 10. The figures do not include performance of the self-scheduling scheme because it performs poorly on their system. The results from the first experiment (Fig. 9) show that all the algorithms performed similarly when N is large and the variance
91
DOALL AND DOACROSS LOOPS
2o I
---
FS
_ _ - _GSS _
a
l5
i
....... Factoring
-TSS .
-. ..
?
U
8
a
10
Linear
~
(0
5
0
1 5
0
I
I
10
15
20
Number of PEs
FIG 9. Performance of DOALL scheduling algorithms on matrix multiplication ( N = 300).
_______
01 0
FS GS5
I
5
10
15
I
20
Number of PEs
FIG 10. Performance of DOALL scheduling algorithms on adjoint convolution ( N = 100).
is small. Hence, the effect of load imbalance is not significant. They also found that fixed-sized chunking (FS) performed better than the others when N is small. On the other hand, the results of the second experiment (Fig. 10) show that if the variance is large, fixed-size chunking (FS) attains only half of the possible speed-up. Guided self-scheduling (GSS) also does not perform well as it assigns too much work at the beginning of the execution and does not save enough work at the end for balancing the load. Factoring and trapezoid self-scheduling (TSS) balance the workload better than the
92
A.
R. HURSON ETAL.
other schemes and attains significantly better speed-up. It should be noted that when the number of iterations is small, none of the scheduling approaches perform well, since there is insufficient work to offset the overhead due to scheduling and distribution of work. Based on these results, we can conclude that among the techniques investigated in the study of parallelizing iterations with varying execution times, fixed-size chunlung performs well when the variations in execution times and the number of iterations is small. On the other hand, factoring and TSS perform better when the variance is large. When loop iterations are scheduled across multiple processors, one must account for the distribution of the data needed by the iterations. Loop iterations frequently demonstrate an aflnity for a particular processor containing the needed data. By exploiting processor affinity better performance can be obtained since communication overhead in accessing needed data is reduced. Affinity scheduling methods also achieve a better workload by permitting idle processors to steal work from busy processors. However, this limits the scalability since processors must snoop (on a bus) to steal work. Performance measurements of affinity scheduling (AFS), dynamic partitioned affinity scheduling (DPAS), wrapped partitioned affinity scheduling (WPAS), and GSS using a synthetic application program and a real application (Jacobi iterative algorithm) were conducted by Subramaniam and Eager (1994). Three different cases were used for the synthetic application. The first case had a triangular workload, in which the iteration size decreases linearly. The second case had a rectangular workload, in which a fraction of the iterations are of a constant large size, while the remaining fraction has a constant smaller size. The third case has constant iteration sizes. The Jacobi iterative algorithm was used since it offers a significant amount of data locality that can be exploited and at the same time it also exhibits a significant amount of load imbalance. The results for the rectangular workload and Jacobi algorithm are shown in Figs. 11 and 12, respectively. From Fig. 11, one can conclude that WPAS offers the best performance. This is because WPAS avoids assigning all the timeconsuming iterations to a single processor. The same is also true for the Jacobi algorithm (Fig. 12)-Even though the performance of GSS and AFS has improved, W A S and DPAS still performed better. Furthermore, both WPAS and DPAS perform well when execution time for iterations decreases with the increasing index (triangular workload), and the three affinity scheduling algorithms (AFS, DPAS, and WPAS) exhibited the same performance for a balanced workload. Based on these results, we can conclude that of the three affinity scheduling schemes tested, W A S performs well for a rectangular workload, both W A S and DPAS equally
93
DOALL AND DOACROSS LOOPS
--- GSS _ - _ _AFS _
6-
1-
01
1
I
I
2
3
1
I
4 5 Number of PEs
I
I
1
6
7
8
FIG 11. Performance of affinity scheduling algorithms on rectangular workload
4.0
r
_--
(N= 128).
GSS
----- AFS ....... DPAS
-WPAS
0.5 0.0 I
1
1
I
I
2
3
4
I
I
I
5
6
7
8
Number of PEs
FIG 12. Performance of affinity scheduling algorithms on Jacobi algorithm (matrix size = 128 x 128).
perform better than AFS for triangular workloads, and all three schemes perform equally on balanced workloads. Unlike DOALL, iterations of DOACROSS loops must be executed in a predetermined order to maintain inter-iteration dependencies. As can be expected, the serialization of iterations leads to a significant loss of parallelism (Chen and Yew, 1991). DOACROSS loops can be either regular or irregular. In a regular DOACROSS loop, inter-iteration dependence distances are constant.
94
A. R. HURSON ETAL.
The staggered distribution (SD) scheme and the cyclic staggered distribution (CSD) attempt to mask the communication delays resulting from inter-iteration dependencies. This is achieved by assigning a monotonically increasing number of iterations to higher numbered processors. CSD is a modified version of SD that overcomes the load imbalance caused by SD. These schemes perform better than other scheduling methods for regular DOACROSS loops. Effectiveness of the staggered schemes has been simulated and compared against those of static chunking and cyclic scheduling (Hurson et al., 1994a, 1994b; Lim et al., 1992). The test-bed includes a representative loop with the execution time of T = 50 and loops 3, 5, 11, 13, and 19 of the Livermore loops, which have cross-iteration dependencies (Feo, 1988): Loop 3 is the standard inner product function of linear algebra, Loop 5 is taken from a tridiagonal elimination routine, Loop 11 is a first sum, Loop 13 is a fragment from a two-dimensional particle-in-cell code, and Loop 19 is a general Linear Recurrence Equation. In their simulation: (1) The inter-PE communication delays are varied based on the ratio of communication time to iteration execution time (C/ T ) . (2) Delays due to LBD are computed for various k values, where k is the fraction of delay d to the execution time of an iteration T , k = d / T .
Pre-synchronized scheduling was not considered, since the best-case performance of this scheme would be equivalent to cyclic scheduling.
FIG 13.
Maximum speed-up (MS), n = 2000, C / T = 0.2.
95
DOALL AND DOACROSS LOOPS
Figure 13 shows the maximum speed-up attained by SD and CYC schemes for n = 2000 and C / T = 0.2. The speed-up for SD is significantly better than CYC for all cases. The average parallelism (AP) of the loop (can also be considered the maximum speed-up of a loop) when k = 0.1 is equal to 9.9, which is very close to the speed-up attained by SD even with communication overhead. The speed-up for CYC is less than two and about one when k = 0.7. Other results show that the maximum speed-ups attained by CYC for C / T = 1.0 and up are all less than one. This means that the loops can obtain better performance if they were executed serially in one PE. The number of PEs required to realize maximum speedup for CYC is shown in Fig. 14. This number drops to two independent of k for (C/TaO.S. This is due to the fact that for C / T = 0.5, after two iterations, the communication delay would be equivalent to the execution time of one iteration T . Therefore, the third and fourth iterations can be executed in the same two processors without any additional delay. The cycle will be repeated for every pair of iterations-using more processors does not affect the performance. Table VII shows the speed-up of SD over SC and CYC when the Livermore loops were simulated. Timing values and inter-processor communication used in the simulation were based upon instruction and communication times for the nCUBE 3200 (Dunigan, 1991). The ratio of communication to instruction execution ( C / E ) for the 3200 is 30. Loop 19 consists of two loops. Hence, each loop was tested separately (19(1) and 19(2)). The number of iterations for each loop were based on the specification of each loop. Loops 3, 5 , and 13 were simulated for n = 1000,
5
4 v)
w
a
\
\
I
0
$ 3 n
‘\,C/T=O.2
~
\
s z
\
\ \
2
CiT= 0.5
1 (
1
0.2
0.3
0.4
0.5 k
0.6
0.7
0.8 0.9
FIG 14. Number of PEs to attain maximum speed-up for cyclic scheduling.
96
A. R. HURSON H A L .
TABLEVII
su(sc)
SPEED-UP OF STAGGERED DISTRmmON RELATIVE TO STATIC C H U ” G AND CYCLIC SCHEDULING s u ( c Y c ) FOR THE LIVERMORE LOOPS WlTH C / E = 30. ACTUAL NWER OF PES USED BY STAGGERED DISTRIBUTION IN PARENTHESES.
PE=4
PE=8
LOOP #
k
C/T
su (SC)
su (CYC)
s u (SC)
s u (CYC)
3 5 11 13 19 (1) 19 (2)
0.25 0.30 0.25 0.05 0.33 0.27
3.75 3.00 3.75 0.71 3.33 2.73
1.20 1.21 1.21 1.07 1.24 1.23
10.72 8.22 10.50 2.82 7.53 6.86
1.21 (7) 1.16 (6) 1.21 (7) 1.14 1.34 (4) 1.28 (5)
13.10 (7) 9.35 (6) 12.18 (7) 5.05 7.53 (4) 6.93 (5)
Loop 11 with n = 500, and Loops 19(1 & 2) with n = 100. Although the number of iterations for Loops 11 can reach a maximum of 1000, Hurson et al. felt that 500 iterations would give a different perspective from Loop 3, since they both have the same value of k. The speed-up for SD increases compared to SC and CYC as the C / T ratio increases and decreases as the value of k increases. There was not much speed-up for Loop 13, since it had a negligible delay. For Loops 3, 5 , 11 and 19(1 & 2) when PE = 8, the SD scheme utilized fewer PEs than the available number of PEs. These results show that SD offers better resource utilization. Furthermore, the number of PEs required also decreases as the communication cost increases. Effectiveness of the cyclic staggered scheme (CSD) was also simulated and compared against the original Staggered scheme (SD) using the same test-bed. As can be seen in Figs. 15 and 16, CSD performed better than SD regardless of the values of n, C / T , and k , especially when the number of PEs was halfway between 2 and maxpe - 1. Finally, CSD attained an almost linear speed-up for smaller number of PEs, even with delays due to LBD and communication cost. Since CSD outperforms SD, we can conclude that CSD comes even closer to the maximum speed-up possible for a particular loop. However, these advantages are made possible if the number of PEs available is less than maxpe. The performance of the Staggered schemes (SD & CSD) has also been evaluated by running loop 13 of the Livermore loops on an C U B E 2 multiprocessor. These schemes have been compared to static chunking (SC) and cyclic scheduling (CYC). Loop 13 was chosen due to its size and the fact that it has a large amount of exploitable parallelism (AP=4.29). Furthermore, it possesses a reasonable amount of delay that hinders the
97
DOALL AND DOACROSS LOOPS
4r
k = 0.3
FIG 15.
1
I
I
Comparative analysis of the staggered schemes, C / T = 3.0.
k = 0.3
I
2
I
3
I
I
4
Number of PEs FIG 16. Comparative analysis of the staggered schemes, C / T = 5.0.
ability of easily executing the loop in parallel. Fig. 17 shows that the SD scheme again attained better speed-up. Furthermore, the SD scheme utilizes less than 8 processors, since it controls the number of processors that are used effectively. The peak speed-up for SD was 2.723 utilizing 7 PEs which is a 36.5% speed-up reduction from the average parallelism of 4.29, and SC had a 46.85% speed-up reduction utilizing 8 PEs. Furthermore, as expected, cyclic scheduling is ineffective if the communication cost is significant.
98
A.
'F
R. HURSON ETAL.
,
,
4
6
,
,
,
12
14
16
,CYC, .....................................................................
0
2
8
10
Number of PEs
FIG 17. Speed-up for loop 13.
Figure 18 shows the speed-up of the two staggered schemes. In was seen in Fig. 17 that the number of PEs needed by SD to achieve maximum speedup (rnaxpe) was seven. Hence, in Fig. 18, the number of PEs utilized for CSD is maxpe - 1. Interestingly, unllke the previous results, CSD performed better than SD only when the number of processors are between 3 and 5 . This was due to the additional overhead incurred to implement the cyclic
2
3
4 Number of PEs
5
6
FIG 18. Speed-up of the staggered schemes for loop 13.
DOALL AND DOACROSS LOOPS
99
staggered scheme-each PE has to continuously check for more iterations to execute after execution of each chunk. This overhead is also the reason for the small performance gain of the cyclic staggered schemes over SD compared to their previous results. With these results in mind, we can conclude that the staggered schemes are very effective in the execution of DOACROSS loops. In irregular DOACROSS loops the inter-iteration dependencies cannot be resolved at compile time. Runtime analysis is needed to identify the dependence patterns. In some applications, the overhead due to the runtime analysis can be amortized across repeated execution of the DOACROSS loops since the dependencies computed once can be reused. Such cases are common in scientific applications and applications that model the behavior of structurally invariant physical systems. Re-synchronized scheduling (PSS) schedules only those iterations for which all synchronization requirements are met. This eliminates processor idle cycles when processors with assigned iterations are waiting for dependent data. Chen’s runtime parallelization scheme (CRPS) requires two phases for scheduling loop iterations. The inspector phase determines the dependence relationships among data accesses, while the executor phase uses this information to execute the iterations in parallel. CRPS allows the overlapped execution of operations among dependent iterations, thus permitting better overall execution times. This, however, may cause more delays due to spin-locks used by iterations waiting their turn to access shared data. CRPS uses static cyclic scheduling for distributing loop iterations among processors which may lead to load imbalances when iterations take varying amounts of execution times. A study to compare CRPS with the Zhu-Yew runtime parallelization scheme (ZYRPS) was performed using a set of parameterized loops running on a 32-processor Cedar shared-memory multiprocessor (Chen and Yew, 1994). Loops with varying number of iterations, iteration grain size ( W ) , and varying number of references ( r ) per iteration with different dependence patterns were simulated. Table VIII shows the speed-up of CRPS using 32 processors when both the inspector and executor are performed (the inspector is not reused)-a loop with long dependence chain and therefore low parallelism is referred to as a “mostly serial” loop. On the other hand, a loop with a short dependence chain has a large amount of parallelism and is referred to as a “mostly parallel” loop. The results show that CRPS yields speed-ups as high as 14 when the inspector is not reused. The best results were attained when the size of the loop body ( W ) is large and the number of accesses ( r ) is low. Also, as expected, performance is better if the dependence chains are short (mostly parallel). They have also shown that a speedup as high as 27 can be attained (Chen and Yew, 1994) if the results of the
100
A. R. HURSON ETAL.
TABLEVIII SPEED-UPOF Cms USING 32 PROCESSORS
Mostly serial loop
W
N = 1600
N = 3200
N = 1600
N = 3200
8
1.10 1.50 2.35 3.66
1.13 1.56 2.65 4.76
1.36 2.26 3.49 4.67
1.42 2.26 3.76 6.18
8
2.41 3.93 6.23 9.55
2.48 3.86 6.94 11.95
2.96 5.67 9.23 11.65
2.97 5.40 9.57 13.60
T
160 ,us (941 cycles)
Mostly parallel loop
14
(3765 cycles)
TABLEIX RATIO BETWEEN THE EXECUTION TME OF ZYRPS AND CRPS USING 32 PROCESSORS
Mostly serial loop
W
r
160 ,us
(941 cycles)
8 (3765 cycles)
Mostly parallel loop
N = 1600
N = 3200
N = 1600
N = 3200
31.70 12.93 4.57 1.58
37.55 13.96 5.05 1.66
7.22 2.77 1.04 0.72
7.92 2.86 1.12 0.91
3 1.69 13.04 4.49 1.71
37.35 13.13 4.76 1.85
7.25 2.88 1.27 0.95
7.61 2.79 1.27 1.oo
inspector analysis is reused across loop invocations. Table IX shows the ratio between the execution time of ZYRPS and CRPS. As can be concluded, CRF’S is nearly always faster than ZYRPS. Moreover, it is also relatively faster in the mostly serial loops-this is due to its ability to overlap execution of dependent iterations. In summary, it is clear that scheduling of DOALL loops is well understood, while efficient solutions for DOACROSS loops require further research.
DOALL AND DOACROSS LOOPS
101
ACKNOWLEDGMENT This work in part has been supported by the National Science Foundation under Grants MIP9622836 and MIP-9622593.
REFERENCES
Chen, D. K., and Yew, P. C. (1991). An empirical study on DOACROSS loops. Proceedings Supercomputing, pp. 620-632. Chen, D. K., and Yew, P. C. (1994a). Redundant synchronization elimination for DOACROSS loops. Proceedings 8th International Parallel Processing Symposium, pp. 477-48 1. Chen, D. K., and Yew, P. C. (1994b). Statement re-ordering for DOACROSS loops. Proceedings International Conference on Parallel Processing. Cytron, R. (1986). DOACROSS: beyond vectorization for multiprocessors. Proceedings International Conference on Parallel Processing, pp. 836 -844. Dunigan, T. H. (1991). Performance of the Intel iPSC/860 and Ncube 6400 hypercubes. Parallel Computing, 17, 1285-1302. Feo, J. T. (1988). An analysis of the computational and parallel complexity of the Livermore loops. Parallel Computing, 7,163- 185. Hummel, S . F., Schonberg, E., and Flynn, L. E. (1992). Factoring: a method for scheduling parallel loops. Communications ofthe ACM, 35(8), 90-101. Hurson, A. R., Lim, J. T., Kavi, K., and Shirazi, B. (1994a). Loop allocation scheme for multithreaded dataflow computers. Proceedings 8th International Parallel Processing Symposium, 3 16-322. Hurson, A. R., Lim,J. T., and Lee, B. (1994b). Extended staggered scheme: a loop allocation policy. Invited Paper, World IMACS Conference, pp. 1321-1325. Krothapalli, V. P. and Sadayappan, P. (1990). Dynamic scheduling of DOACROSS loops for multiprocessors. Proceedings Parbase-90: International Conference on Databases and Parallel Architectures, pp. 66-75. Kruskal, C., and Weiss, A. (1985). Allocating independent subtasks on parallel processors. IEEE Transactions on Software Engineering, SE-11(10), 1001- 1016. Kwan, T. T, Totty, B. K., and Read, D. A. (1993). Communication and computation performance of the CM-5. Proceedings International Conference on Supercomputing, 192-201. Li, H., Tandri, S . , Stumm, M., and Sevcik, K. C . (1993). Locality and loop scheduling on NUMA multiprocessors.Proceedings International Conference on Parallel Processing, 11, 140- 147. Lilja, D. J. (1994a). Exploiting the parallelism available in loops. IEEE Computer, 27(2), 13-26. Lim, J. T., Hurson, A. R., Lee, B., and Shirazi, B. (1992). Staggered distribution: a loop allocation scheme for dataflow multiprocessor systems. The Fourth Symposium on the Frontiers of Massively Parallel Computation, pp. 310-3 17. Markatos, E. P. and LeBlanc, T. J. (1992). Using processor affinity in loop scheduling on shared-memory multiprocessors. Proceedings Supercomputing, pp. 104- 113. Polychronopoulos, C. D., and Banerjee, U. (1987). Processor allocation for horizontal and vertical parallelism and related speedup bounds. IEEE Transactions on Computers, C-36 (4), 4 10-420. Polychronopoulos, C. D., and Kuck, D. J. (1987). Guided self-scheduling: a practical scheduling scheme for parallel supercomputers. IEEE Transactions on Compurers, C-36(12), 1425- 1439.
102
A. R. HURSON ETAL.
Polychronopoulos, C. D., Kuck, D. J., and Padua, D. A. (1986). Execution of Parallel Loops on Parallel Processor Systems. Proceedings International Conference on Parallel Processing, pp. 5 19-527. Su, H. M., and Yew, P. C. (1991). Efficient doacross execution on distributed shared-memory multiprocessors. Proceedings Supercornputing, 842-853. Subramaniam, S., and Eager, D. L. (1994). Affinity scheduling of unbalanced workloads. Proceedings Supercornputing, pp. 214-226. Tang, P. and Yew, P. C. (1986). Processor self-scheduling for multiple-nested parallel loops. Proceedings International Conference on Parallel Processing, pp. 528 -535. Tzen, T. H., and Ni, L. M. (1991). Dynamic loop scheduling for shared-memory multiprocessors. Proceedings International Conference on Parallel Processing, 11, 247-250. Yue, K. K., and Lilja, D. J. (1994a). Parallel Loop Scheduling for High-Performance Computers. Technical Report No. HPPC-94- 12, Department of Computer Science, University of Minnesota. Zhu, C. Q., and Yew, P. C. (1987). A scheme to enforce data dependence on large multiprocessor systems. IEEE Transactions on Software Engineering, SE-13,726-739.
FURTHER READING
Abraham, S. G., and Hudak, D. E. (1991). Compile-time partitioning of iterative parallel loops to reduce cache coherency traffic. IEEE Transactions on Parallel and Distributed Systems, 2(3), 318-328. Chen, D. K., and Yew, P. C. (1992). A scheme for effective execution of irregular DOACROSS loops. Proceedings International Conference on Parallel Processing, 11, 285-292. Cytron, R. (1987). Limited processor scheduling of doacross loops. Proceedings Infernational Conference on Parallel Processing, pp. 226-234. Hamidzadeh, B., and Lilja, D. J. (1994). Self-adjusting scheduling: an on-line optimization technique for locality management and load balancing. Proceedings International Conference on Parallel Processing, 11, 39-46. Hudak, D. E., and Abraham, S. G. (1992). Compile-time optimization of near-neighbor communication for scalable shared-memory multiprocessors. Journal of Parallel and Distributed Computing, 15, 368-381. Krothapalli, V. P., and Sadayappan, P. (1991). Removal of redundant dependencies in DOACROSS loops with constant dependencies. IEEE Transactions on Parallel and Distributed Systems, 2(3), 281 -289. Lilja, D. J. (1994). The impact of parallel loop scheduling strategies on prefetching in a shared-memory multiprocessor. IEEE Transactions on Parallel and Distributed Systems, 5(6), 573-584. Polychronopoulos, C. D. (1987a). Advanced loop optimizations for parallel computers. In Lecture Notes in Computer Science No. 297: Proceedings International Conference on Supercornputing, pp. 255-277. Polychronopoulos, C. D. (1987b). Automatic restructuring of Fortran programs for parallel execution. Proceedings 4th International DFVLR Seminar on Parallel Computing in Science and Engineering, pp. 107- 130. Rudolph, D. C., and Polychronopoulos, C. D. (1989). An efficient message-passing scheduler based on guided self scheduling. Proceedings International Conference on Supercomputing, pp. 50-61.
DOALL AND DOACROSS LOOPS
103
Saltz, J. H., and Mirchandaney, R. (1991).The preprocessed DOACROSS Loop. Proceedings International Conference on Parallel Processing, 11, 174- 179. Saltz, J. H.,Mirchandaney, R., and Crowley, K. (1989).The Doconsider loop. Proceedings, International Conference on Supercomputing, pp. 29-40. Saltz, J. H., Crowley, K, Mirchandaney, R., and Berryman, H. (1990).Runtime scheduling and execution of loops on message passing machines. Journal of Parallel and Distributed Computing, 8,303-312. Saltz, J. H.,Mirchandaney, R., and Crowley, K. (1991). Runtime parallelization and scheduling of loops. IEEE Transactions on Computers, 40(5), pp. 603-612. Tzen, T. H., and Ni, L. M. (1992).Data dependence analysis and uniformization for doubly nested loops. Proceedings International Conference on Parallel Processing, 11.91 -99. Yue, K. K.,and Lilja, D. J. (1994b).Parameter Estimation for a Generalized Parallel Loop Scheduling Algorithm. Technical Report No. HPPC-94-18,Department of Computer Science, University of Minnesota.
This Page Intentionally Left Blank
Programming Irregular Applications: Runtime Support, Compilation and Tools JOEL SALTZ CHlALlN CHANG GUY EDJLALI YUAN-SHIN HWANG BONGKI MOON RAVl PONNUSAMY SHAMIK SHARMA ALAN SUSSMAN AND MUSTAFA UYSAL UMIACS and Department of Computer Science University of Maryland College Park, M D
GAGAN AGRAWAL Department of Computer and Information Sciences University of Delaware Newark, DE
RAJA DAS College of Computing Georgia Institute of Technology Atlanta, GA
PAUL HAVLAK Department of Computer Science Rice University Houston, 7X
Abstract In this chapter, we present a summary of the runtime support, compiler and tools development efforts in the CHAOS group at the University of Maryland. The principal focus of the CHAOS group's research has been to develop tools,
105 ADVANCES IN COMPUTERS, VOL. 45
Copyright Q 1997 by Academic Press Ltd. All rights of reproduction in any fonn reserved.
106
JOEL S A L E ETAL .
compiler runtime support and compilation techniques to help scientists and engineers develop high-speed parallel implementations of codes for irregular scientific problems (i.e. problems that are unstructured. sparse. adaptive or block structured). We have developed a series of runtime support libraries (CHAOS. CHAOS++) that carry out the preprocessing and data movement needed to efficiently implement irregular and block structured scientific algorithms on distributed memory machines and networks of workstations . Our compilation research has played a major role in demonstrating that it is possible to develop data parallel compilers able to make effective use of a wide variety of runtime optimizations . We have also been exploring ways to support interoperability between sequential and parallel programs written using different languages and programming paradigms .
1. Introduction
.....................................
2. CHAOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Runtime Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Summary of the Runtime System . . . . . . . . . . . . . . . . . . . . . . . 3 . Compilation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Program Slicing-based Loop Transformations . . . . . . . . . . . . . . . . . 3.2 Interprocedural Partial Redundancy Elimination . . . . . . . . . . . . . . . . 4 . Runtime Support for Pointer-based Codes: CHAOS+ + . . . . . . . . . . . . . . 4.1 Mobile Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Globally Addressable Objects . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Building Distributed Pointer-based Data Structures . . . . . . . . . . . . . . 4.4 Data Movement Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 CHAOS+ + : Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5 . Interoperability Issues: Meta-Chaos . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Meta-Chaos Mechanism Overview . . . . . . . . . . . . . . . . . . . . . . . 5.2 Data Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Communication Schedule Computation . . . . . . . . . . . . . . . . . . . . 5.4 Meta-Chaos Applications Programmer Interface (API) . . . . . . . . . . . . 5.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 . Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Runtime Support for Irregular Problems . . . . . . . . . . . . . . . . . . . . 6.2 Runtime Support for Irregularly Coupled Regular Mesh Applications . . . . 6.3 Compiler Methods for Irregular Problems . . . . . . . . . . . . . . . . . . . 7 . summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
106 108 108 111 116
117 118 121 124 126 127 129 132 133 134 135 136 138 139 141 142 143 143 144 145 148 149
1. Introduction The last decade has seen the emergence of a variety of parallel computer architectures. A number of research groups and vendors have successfully
PROGRAMMING IRREGULAR APPLICATIONS
107
demonstrated that powerful parallel machines can be built from commodity hardware. While building these machines has become increasingly easy and inexpensive, programming such machines has remained a major challenge. The important tasks in providing better support for programming parallel machines can be summarized as follows: 0
0
0
0
carefully studying the features of important classes of parallel applications; developing runtime support systems for parallelizing these classes of applications; developing compilation techniques for incorporating such runtime support into compilers for parallel machines; developing techniques for interoperability of different classes of runtime support and programming paradigms.
The principal focus of the CHAOS group’s research has been to develop tools, compiler runtime support and compilation techniques to help scientists and engineers develop high-speed parallel implementations of codes for irregular scientific problems (i.e. problems that are unstructured, sparse, adaptive or block structured). We have developed a series of runtime support libraries (CHAOS, CHAOS++) that carry out the preprocessing and data movement needed to efficiently implement irregular and block structured scientific algorithms on distributed memory machines and networks of workstations [l, 34,531. Our compilation research has played a major role in demonstrating that it is possible to develop data parallel compilers able to make effective use of a wide variety of runtime optimizations. We have also been exploring ways to support interoperability between sequential and parallel programs written using different languages and programming paradigms. Successful techniques would facilitate the design of complex scientific applications that are composed of separately developed components, and provide the infrastructure required to make use of highly distributed data and computational resources. While our research on this topic is still at an early stage, we have already demonstrated an ability to compose parallel programs that have been written using different programming paradigms (e.g. High Performance Fortran [36] and message passing with MPI [61]). In this chapter, we give an overview of these activities in the CHAOS group at the University of Maryland. Our purpose is to give a flavor for the range of activities; detailed technical descriptions of each of the components of our research have been reported elsewhere. The central focus of our group for several years has been to develop a highly optimized runtime support
108
JOEL SALTZ ETAL.
library for irregular applications executed on distributed memory parallel machines. This runtime system is called CHAOS. In Section 2 we give an overview of this runtime library. We particularly focus on the major challenges associated with efficiently executing irregular applications on parallel machines, and techniques for optimizing their performance. We show how these techniques are incorporated into the widely distributed CHAOS runtime library. In Section 3, we describe effective techniques for incorporating the CHAOS runtime support into compilers for parallel machines. Over the last several years, our group has developed a number of successful prototype compilation systems demonstrating that irregular applications can be compiled for efficient execution on distributed memory parallel machines. We specifically describe two techniques: program slicing and interprocedural partial redundancy elimination. In Section 4, we describe an extension of CHAOS: CHAOS + +. The main limitation of CHAOS is that it primarily targets may-based applications. In CHAOS+ +, we have demonstrated how the essential ideas behind the CHAOS runtime system can be used even for distributed pointerbased codes with complex user-defined data structures. With the increasing availability of a variety of runtime systems and the emergence of a number of programming paradigms, it is becoming increasingly important to be able to interoperate between these systems and a tool that facilitates such paradigms. We have developed META-CHAOS, interoperability. This is described in Section 5. In Section 6 we present a review of related literature and in Section 7 we present a summary.
2.
CHAOS
In this section, we briefly describe the class of applications that CHAOS targets and outline the important features of the CHAOS runtime support library that we have developed. The compilation techniques for incorporating this runtime library as part of compilers for distributed memory parallel machines are presented in Section 3. 2.1
Overview
We have addressed a particular class of scientific application programs, called irregular programs, which require special runtime and compiler support for parallelization. Examples of irregular applications are found in unstructured computational fluid dynamic solvers, molecular dynamics codes (CHARMM, AMBER, GROMOS, etc.), diagonal or polynomial
PROGRAMMING IRREGULAR APPLICATIONS
109
preconditioned iterative linear solvers, and particle-in-cell (PIC) codes. Each of these applications simulates the interactions among objects in a physical system. The class of irregular applications can be further divided into two subclasses: static and adaptive. Static irregular applications are those in which each object in the system interacts with only a fixed, predetermined set of objects. Although the properties of the objects in the system (e.g. positions in space or velocities) will change during the course of computation, the set of objects that every object interacts with remains the same. For example, some computational fluid dynamics applications model systems, such as wings or fuselages of aircraft, using unstructured meshes. Each mesh point represents a location within the physical space and the values at each mesh point correspond to properties of the location (e.g. tension or pressure). Every mesh point communicates values with only neighboring mesh points. Although the values at mesh points change, the set of neighboring mesh points for every mesh point remains constant. In adaptive irregular programs, such as molecular dynamics codes, interactions between entities (e.g. atoms, molecules, etc.) change during computation (due to movement of the entities). However, in many adaptive applications the set of interacting objects for each object changes slowly over time. For instance, the molecules in a molecular dynamic simulation move very little in a single time step, and consequently, the set of molecules that interact with a given molecule remains static for many time steps in the simulation. On the other hand, some applications exhibit a high degree of adaptivity. One such example is a direct simulation Monte Carlo (DSMC) code for simulating the movements and collisions of molecules in a flow of gas on a spatial flow field domain overlaid by a Cartesian mesh. Only molecules in the same cell can possibly collide with each other during any time step. However, as molecules collide they move between mesh cells, and consequently, the set of molecules in a particular cell changes at almost every time step. A common characteristic of all these irregular applications is the use of indirect indexing of data arrays to represent the relationships among elements. This means that the data arrays are indexed through the values in other arrays, which are called indirecfion arrays. In other words, the data access patterns are determined by the values of elements in the indirection arrays. Figure 1 illustrates a typical irregular loop. The data access pattern is determined by indirection arrays ia and ib. The use of indirect indexing leads to difficulties in allowing a compiler to determine the data access patterns in a program (i.e. the indices of the data arrays being accessed), since the values of elements in the indirection arrays are not available until runtime. The analysis and optimization techniques
110
L1: L2:
JOEL SALTZ H A L .
real x (max-nodes ) , y (max-nodes ) i n t e g e r ia(max-edges). ib(max-edges)
! data arrays ! indirection arrays
d o n = 1. n-step d o i = 1 . sizeof-indirection-arrays x(ia(i)) = x(ia(i)) + y(ib(i)) end d o end do
! outer loop ! inner loop
FIG.1. An example with an Irregular Loop.
exploited by current compilers require that the indices to data arrays must be symbolically analyzable at compile time. Consequently, most current compilers can only recognize indices that are either constants or affine functions of loop induction variables. However, the use of indirection arrays prohibits existing compilers from identifying array data access patterns at compile time. The inability to determine the array access patterns prevents existing compilers from generating efficient code on distributed memory parallel systems. On distributed memory machines, the memory is partitioned into separately addressable spaces. Each processor can only access its local memory, and all data residing in the memory of other processors must be transferred to local memory before being referenced. Furthermore, data arrays of applications running on distributed memory systems are usually partitioned over local processor memories so that portions of the arrays can be accessed by multiple processors simultaneously. As a result, processors sometimes must reference data elements that reside in the memory of a remote processor. Therefore, compilers must either be able to determine the array access patterns at compile time and transfer remote data elements to local memory before referencing them, or generate code to efficiently identify remote array references at runtime and transfer the values when they are needed. Since the array access patterns for irregular applications cannot be determined at compile time, a two-phase inspectorlexecutor runtime strategy [45,59] has been developed to parallelize irregular applications. The inspectorlexecutor strategy works as follows. During program execution the inspector examines the data references made by a processor and calculates what off-processor data needs to be fetched and where that data will be stored once it is received. The executor then uses the information from the inspector to fetch remote data elements and perform the actual computation. This approach can be directly applied to static irregular applications to achieve high performance. However, additional optimizations are required to achieve good performance when adaptive irregular applications are parallelized using the inspector/executor approach.
PROGRAMMING IRREGULAR APPLICATIONS
L1:
L2:
do n = 1, n s t e p s d o i = 1 . sireof_indirection_arrays x(ia(i)) = x(ia(i)) end do
S:
.t
! outer loop ! inner loop
y(ia(i)) *y(ib(i))
! under c e r t a i n c o n d i t i o n s ! i n d i r e c t i o n a r r a y may Chdflile
if ( r e q u i r e d ) then regenerate i c ( : )
L3: do i = 1 . s i z e o f - i c x(ic(i)) = x(ic(i)) e n d do end do
111
! inner loop
+
y(ic(i))
FIG. 2. A code that adapts occasionally.
Since interactions are specified by indirection arrays, the adaptivity of irregular programs is represented by the frequency of modifications to indirection arrays. Figure 2 illustrates the properties of loops found in molecular dynamics codes and unstructured fluid dynamics codes. In the example, multiple loops access the same data arrays, but with different access patterns. In loop L2 the data arrays x and y are indirectly accessed using arrays i a and i b. In loop L3 the same data arrays are indirectly accessed using indirection array i c . The data access pattern in loop L2 remains static, whereas the data access pattern in loop L3 changes whenever the indirection array i c is modified. The adaptivity of the loop is controlled by the conditional statement S.
2.2 Runtime Preprocessing The library is designed to ease the implementation of computational problems on parallel architecture machines by relieving users of low-level machine specific issues. The CHAOS runtime library has been developed to efficiently handle irregular programs. Solving such irregular problems on distributed memory machines using CHAOS runtime support involves six major phases (Fig. 3). The first four phases concern mapping data and computations onto processors. The next two steps concern analyzing data access patterns in loops and generating optimized communication calls. Detailed descriptions of these phases can be found in [54]. In static irregular programs, Phase F is typically executed many times, while phases A through E are executed only once. In some adaptive programs where data access patterns change periodically but reasonable load balance is maintained, phase E must be repeated whenever the data access patterns change. In highly adaptive programs, the data arrays may need to be repartitioned in order to maintain load balance. In such applications, all the phases are repeated.
112
JOEL S A L E ETAL.
Phase A:
Data partitioning
Phase B: Phase C :
Data remapping Iteration partitioning
Phase D:
Iteration remapping
Redistribute indirection array elements
Phase E:
Inspector
Translate indices; generate schedules
Phase F:
Executor
Use schedules for data transportation Perform computation
Assign elements of data arrays to processors Redistribute data array elements Allocate iterations to processors
FIG.3. Solving irregular problems.
2.2. I
Schedule Generation
A communication schedule is used to fetch off-processor elements into a local buffer before the computation phase, and to scatter these elements back to their home processors after the computational phase is completed. Communication schedules determine the number of communication startups and the volume of communication. Therefore, it is inlportant to optimize the schedule generation. The basic idea of the inspector/executor concept is to hoist preprocessing outside the loop as much as possible so that it need not be repeated unnecessarily. In adaptive codes where the data access pattern occasionally changes, the inspector is not a one-time preprocessing cost. Every time an indirection array changes, the schedules associated with it must be regenerated. For example, in Fig. 4, if the indirection array i c is modified, the schedules inc sched c and sched-c must be regenerated. Generating inc-sched-c involves inspecting sched-ab to determine which off-processor elements are duplicated in that schedule. Thus, it must be certain that communication schedule generators are efficient while maintaining the necessary flexibility. In CHAOS, the schedule-generation process is carried out in two distinct phases. The index analysis phase examines the data access patterns to determine which references are off-processor, removes duplicate offprocessor references by only keeping distinct references in hash tables, assigns local buffers for off-processor references, and translates global indices to local indices. o The schedule generation phase generates communication schedules based on the information stored in hash tables.
0
The communication schedule for processor p stores the following information:
(1) send list-a list of arrays that specifies the local elements of a processor p required by all processors;
PROGRAMMING IRREGULAR APPLICATIONS L1: do n = 1 . risteps c d I I g a t h e r i y ( b e g i n - b u f f i , y. h ~ h e d - ~ i t ) ) cal 1 zero-out bufferix(begiri-biiffJ. o f t p - x ) 12: d o 1 = 1 . local_sireof_indir-array:
x(local-ia(i))
=
113
! outer loop ! f e t c h o f t - p r o c data ! i n i t i a l i z e buffer ! inner loop
x(local-ia(i ) ) + y(local-ia(i 1 ) * y(local-ib(i))
end d o
S:
L3:
i f ( r e q u i r e d ) then
m o d i f y part-ic(: CHAOS-clear-mask ( h a s h t a b l e . itamp-c 1 local-ic(:) = p a r t - i c ( : 1 stamp-c = CHAOS-enter-hash (local-ic 1 i nc-sched-c = CHAOS-i ncrernprit a l - s c h e d u l e ( s t a m p - c ) stamp-c) sched-ac = CHAOS-schedule(stamp-a. endif
! i c i s modified ! clear i c
c a l l gather(y(begin-buffPi. y . inc_sched-c) c a l l r e r o - o u t - b u f f e r ( x ( b e g i n - b u f f Z i , otfp-xP) d o i = 1, l o c a l - s i z e o f - i c x ( 1 ocal-i c ( i 1 ) = x ( 1 ocal-i c ( i 1 ) + y ( 1 ocal-ic e n d do
! incremental gather ! i n i t i a l i z e buffer ! inner loop
ca17 s c d t t e r - a d d i x ( b e g i n - h i i f f
i . x . sched-ac)
(
i
! e n t e r new i c
! i n c r e i n e n t a l sched ! sched f o r i a , ic
)
! scatter addition
end d o
FIG.4. Schedule generation for an adaptive program.
(2) permutation list-an array that specifies the data placement order of off-processor elements in the local buffer of processor p ; ( 3 ) send size-an array that specifies the sizes of out-going messages of processor p to all processors; (4) fetch size-an array that specifies the sizes of in-coming messages to processor p from all processors. The principal advantage of such a two-step process is that some of the index analysis can be reused in adaptive applications. In the index analysis phase, hash tables are used to store global to local translation and to remove duplicate off-processorreferences. Each entry keeps the following information: (1) global index-the global index hashed in; (2) translated address-the processor and offset where the element is stored; this information is accessed from the translation table; (3) local index-the local buffer address assigned to hold a copy of the element, if it is off-processor; (4) stamp-an integer used to identify which indirection array entered the element into the hash table. The same global index entry might be hashed in by many different indirection arrays; a bit in the stamp is marked for each such entry.
114
JOEL SALTZ ETAL.
Stamps are very useful when implementing adaptive irregular programs, especially for those programs with several index arrays in which most of them are static. In the index analysis phase, each index array hashed into the hash table is assigned a unique stamp that marks all its entries in the table. Communication schedules are generated based on the combination of stamps. If any one of the index arrays changes, only the entries pertaining to the index array, i.e. those entries with the stamp assigned for the index array, have to be removed from hash table. Once the new index array is hashed into the hash table, a new schedule can be generated without rehashing other index arrays. Figure 4 illustrates how CHAOS primitives (in pseudocode) are used to parallelize the adaptive problem. The conditional statement S may modify the indirection array ic. Whenever this occurs, the communication schedules that involve prefetching references of ic must be modified. Since the values of ic in the hash table are no longer valid, the entries with stamp stamp-c are cleared by calling CHAOS-clear-mask( ). New values of i c are then entered into the hash table by CHAOS-enter-hash( ). After all indirection arrays have been hashed in, communication schedules can be built for any combination of indirection arrays by calling CHAOS schedule( ) or CHAOS-incremental schedule ( ) with an appropriate combination of stamps. An example of schedule generation for two processors with sample values of indirection arrays i a , i b , and i c is shown in Fig. 5. The global references due to indirection array ia are stored in hash table H with stamp a,i b with stamp b and ic with stamp c. The indirection arrays might have some common references. Hence, a hashed global reference might have more than one stamp. The gather schedule sched-ab for the loop L2 in Fig. 4 is built using the union of references with time stamps a or b. The scatter operation for loop L2 can be combined with the scatter operation for the loop L3. The gather schedule inc-sched-c for loop L3 is built with those references that have time stamp c alone because references with time stamps a or b as well as with c can be fetched by using the schedule sched-ab. The scatter schedule for loops L2 and L3 is built using the union of references with time stamps a and c. PARTI, the runtime library that preceded CHAOS, also had support for building incremental and merged schedules [ 191. However, in PARTI, such schedules were built using specialized functions for these purposes. The CHAOS library restructures the schedule generation process and by using a global hash table provides a uniform interface for building all types of schedules. Such a uniform interface is easier to use for both users and compilers that automatically embed CHAOS schedule generation calls.
115
PROGRAMMING IRREGULAR APPLICATIONS
Initial
dlstrlbutlon o f data a r r a y s Processor 0
ProceLsor
112131415
1
161 7 1 H j Y I
10
Inserting indirection a r r a y s i n t o t h e hash t a b l e < P r o c e s s o r 0 ) i n d i r e c t i o n a r r a y la = 7 . 2 . 9 . 1 . . 3
addr
~
irldir-cxtion a r r a y
ib
=
-
4
addr
4
1.5.7.8.2
h
/
indirection a r r a y
ic
-/ It).
?.
13.
Q .
3
FIG. 5 . Schedule generation with hash
table.
2.2.2 Light- weigh t Schedules We have developed a variety of techniques that have been incorporated into CHAOS and efficiently support a class of applications (e.g. particle codes such as Direct Simulation Monte Carlo) which manifest access patterns that change from iteration to iteration [34]. When data access patterns are not repeated every iteration, performance becomes very sensitive to scheduling overheads. The important observation here is to recognize that in many such codes, the crucial communication intensive loops are actually implementing a generalized reduction in which it is not necessary to control the order in whch data elements are stored. In such applications a significant optimization in schedule generation can be achieved by recognizing that the semantics of set operations imply that
116
JOEL S A L E ETAL.
elements can be stored in sets in any order. This information can be used to build much cheaper light-weight communication schedules. During schedule-generation, processors do not have to exchange the addresses of all the elements they will be accessing with other processors; they only need to exchange information about the number of elements they will be appending to each set. This greatly reduces the communication costs in schedule generation. A light-weight schedule for processor p stores the following information: list of arrays that specifies the local elements of (1) send list-a processor p required by all processors; (2) send size-an array that specifies the outgoing message size of processor p to all processors; ( 3 ) fetch size-an array that specifies the incoming message size of processor p from all processors. Thus, lightweight schedules are similar to the previously described schedules except that they do not carry information of data placement order in the receiving processor. While the cost of building a light-weight schedule is less than that of regular schedules, a light-weight schedule still provides the same communication optimizations of aggregating and vectorizing messages [19].
2.3 Summary of the Runtime System The CHAOS system can be used 0 0 0
to provide irregular compiler runtime support; directly by programmers who wish to port irregular codes; as benchmarks used in comparisons with new irregular runtime support efforts.
In the course of developing and optimizing CHAOS, we have collaborated with applications scientists to produce CHAOS-based parallel implementations of a number of full application codes, including the molecular dynamics code CHARMM [33], an unstructured multigrid Euler solver [17,44], a Direct Simulation Monte Carlo code [47], a flame simulation code [46], and the PARKA high-performance knowledge-based system. The applications demonstrate that CHAOS can be applied on various types of programs. They range from programs with static access patterns (Euler solver), to adaptive applications (CHARMM) and highly adaptive codes (DSMC). Furthermore, CHAOS can efficiently handle programs with time-varying computational costs (flame simulation code), and even codes with loop carried dependencies (PARKA and sparse triangular solvers).
PROGRAMMING IRREGULAR APPLICATIONS
117
The high performance achieved by these applications is facilitated by a set of data and computation decomposition primitives in CHAOS. These primitives provide users with a convenient tool to partition data and computations over processors appropriately based on the properties of applications such that communication overheads can be minimized and good load balance can be achieved. The parallelized CHAOS-based application codes have frequently been among the best performing codes m their application area. For instance, we collaborated with Bernard Brook,; at the National Institutes of Health to develop a highly optimized parallel version of the molecular dynamics code CHARMM [33]. While Brooks w,xs at Harvard, he was one of CHARMM's original developers. Before he began to collaborate with our group, Brooks had developed his own parallel CHARMM implementation. In the course of our collaboration with Brooks, we were able to make use of the flexibility inherent in CHAOS to jointly develop and implement a new scheme for parallelizing molecular dynamics codes. The new scheme is a hierarchical decomposition method for which CHAOS proved to be well suited. The performance obtained depended on the target architecture-on 128 processors of the Intel iPSC/860, we achieved speed-ups of roughly 100 compared to Brooks' original sequential code using a standard benchmark case (myoglobin with water) that is frequently used to characterize the performance of molecular dynamics codes. These performance improvements are of considerable practical importance because molecular dynamics codes can require hours to days to run on workstations.
3. Compilation Methods In the previous section, we had mentioned how our runtime library can be used either by application programmers or by compilers. In this section, we give an overview of our research in developing techniques by which the compilers can automatically insert calls to runtime libraries. In [MI, we demonstrated a simple compiler that was able to make effective use of the inspector/e:tecutor framework. A serious limitation of this work was that the methods could only be used when programmers limited themselves to a constrained set of programming idioms. Over the past few years, one of the principal goals of our research on irregular problems has been to develop compilation methods for irregular problems that are able to make effective u'se of irregular problem runtime support. Our close collaboration with the Rice Fortran D [32]group has played a central role in the development of sophisticated prototype compilers that are now able to deal effectively with challenging irregular codes. We will highlight
118
JOEL SALTZ H A L .
two major compilation problems that we have addressed: A Program Slicing-based loop transformation technique and lnterprocedural Partial Redundancy Elimination.
3.1
Program Slicing-based Loop Transformations
Standard methods for compiling irregular accesses to distributed arrays generate a single inspector-executor pair [66]. The inspector analyzes the subscripts, a gathering communication occurs if off-processor data is being accessed, a single executor carries out the original computation, and finally a scattering communication occurs if off-processor data has been written. Many application codes contain computations with more complex access functions. Subscripted subscripts and subscripted guards can make the indexing of one distributed array depend on the values in another, so that a partial order is established on the distributed accesses. Loops with such multiple levels of indirection commonly appear in unstructured and adaptive applications codes associated with particle methods, molecular dynamics, sparse linear solvers, and in some unstructured mesh computational fluid dynamics solvers. We have presented various optimizations which are part of our runtime support system and also methods for handling loops with complex indirection patterns by transforming them into multiple loops each with a single level of indirection. We have implemented this method in the Fortran D compiler developed at Rice University. Our experiments demonstrate substantial speed improvements through message aggregation [ 18 1. Figure 6 shows a loop with a single level of indirection. In this example, assume that all the arrays are aligned and distributed together in blocks among the processors, and that the iterations of the i loop are likewise block partitioned. The resulting computation mapping is equivalent to that produced by the owner computes rule, a compiler heuristic that maps the computation of an assignment statement to the processor that owns the lefthand side reference. Data array y is indexed using the array i a , causing a single level of indirection. The compiler produces a single executable; a copy of this executable runs on each processor. Each processor’s copy of the program determines what processor it is running on, and uses this information to determine where its
do i = 1. n x(i) = y(ia(i)) end do
=
z(i)
FIG.6. Simple irregular loop.
119
PROGRAMMING IRREGULAR APPLICATIONS
data and iterations fit into the global computation. Let my$eZems represent the number of iterations of the loop in Fig. 6 assigned to a processor, and the number of elements from arrays x , y , z and ia mapped to a processor. We obtain the following code (omitting some details of communication and translation for clarity): d o i = 1 , myBelems index$y(i ) = i a ( i 1 enddo
// i n s p e c t o r
. . . f e t c h y e l e m e n t s t o l o c a l memory. modify indexBy so t h a t i t r e f e r s t o l o c a l l y stored copies o f t h e e l e m e n t s o f y accesstad i n t h e l o o p . . . d o i = l , myBelems x ( i )=y(index$y(i) enddo
/I executor
1+z(i)
Many application codes contain computations with more complex access functions. Subscripted subscripts and subscripted guards can make the indexing of one distributed array depend on the values in another, so that a partial order is established on the distributed accesses. Loops with such multiple levels of indirection commonly appear in unstructured and adaptive applications codes associated with particle methods, molecular dynamics, sparse linear solvers, rind in some unstructured mesh computational fluid dynamics solvers. Figure 7 depicts examples of these kinds of loops. Consider the loop in Fig. 6, but assume now that while all the other arrays and the loop iterations are block-distributed so that x ( i 1, ia ( i 1 , and y ( i 1 are all on the same processor as iteration i, array z has been cyclicdistributed so that z ( i ) usually lies on a different processor. In addition to the irregular potential off-processor references to y , we now have regular off-processor references to z. Let my$elems represent the number of elements from each array and iterations from the original loop locally mapped, let n$procs represent the number of processors, and let rny$id represent the processor identification = 1. n i f (im:(i) ) t h e n x ( i d ( i ) ) = ... endif enddo
do i
do i
=
1, n
x(ia(ib(i))) enddo
A
=
_..
B
do i = 1, n do j = i a ( i ) , i a ( i + 1 ) x ( i a ( j ) ) = ... enddo enddo
C
FIG.7. Dependence between distributed array accesses
120
JOEL SALT2 ETAL.
(ranging from 0 to n$procs- 1). We obtain the following code (omitting some details of communication and translation for clarity): d o i = l , my8elems lI inspector indexdz ( i 1 =my$ id + i*n $ p rocs index$y(i) = i a ( i )
enddo
. . . f e t c h y a n d z elements t o l o c a l memory, modify indexdy a n d indexdy t o r e f e r t o l o c a l l y s t o r e d co p i es of t h e elements o f y accessed i n t h e l o o p . . . d o i = l , my$elems x(i )
=
y(index$y(i 1)
// e xe c utor + z(index$z(i ) )
enddo
Because the subscripting for y accesses only local elements of the distributed array ia , its inspector requires no communication and can be combined with the inspector for Z. However, if an inspector needs to make a potentially non-local reference (either because of a misaligned reference or multiple levels of indirection), the single inspector-executor scheme breaks down. The inspector must itself be split into an inspector-executor pair. Given a chain of n distributed array references, each in order depending on the previous, we must produce n + 1 loops: one initial inspector, n - 1 inspectors that also serve as the executor for the previous inspector, and one executor to produce the final result(s). The transformation required to eliminate non-local references in loops can be divided into two distinct parts. e The first part of the transformation process breaks up a loop whose references have multiple inspection levels into multiple loops whose references have no more than one inspection level. Each nonlocal reference is then a distributed array indexed by a local index array. e The second part of the transformation completes the inspectorexecutor pairs for each of the loops or code segments generated. For the completion of the inspector-executor pairs we insert runtime library calls for collective communication and for global to local address translation.
In [ 181, we present the algorithms required to perform the first part of the transformation; insertion of collective communication is presented in [30 1. The essence of the technique for the first part of the transformation can be described as follows: we take complicated subscript computations, replicate
PROGRAMMING IRREGULAR APPLICATIONS
121
them outside their original loops, and save the sequence of subscripts in a local index array. The subscript computations must be copied, transplanted back into the original program and the values saved without disturbing other computations. The transplant descriptor is a data structure which contains information to replicate the value of an expression at another point in the program. The replicated computati.on is built by program slicing. In the literature, a slice is a subset of a program’s statements determined to affect the value of a variable at a particular program point [65].Our method builds each slice as a derivative program fragment, tailored to compute the values of interest and ready to be transplanted elsewhere in the program. We have implemented our loop transformation algorithm as a part of the Fortran D compiler. We have successfully parallelized a number of kernels derived from various irregular applications. The structure of these applications are such that they cannot be parallelized using existing compilation techniques without a severe degradation in performance. The automatic method that can be used to pafallelxze these kernels, other than the technique described in this paper, is runtime resolution [ 5 8 ] , but that causes poor performance as each off-processor reference is communicated separately.
3.2
Inte rprocedura I Pa rtia I Redundancy EIi minatio n
An important optimization required for irregular applications is placement of communication preprocessing and communication statements. The key idea underlying these schemes is to do the placement so that redundancies are reduced or eliminated. These schemes are based upon a classical data flow framework called partial redundancy elimination (PRE). PRE encompasses traditional optimizalions like loop invariant code motion and redundant computation elimination. Our interest is in applying the PRE framework for optimizing placement of communication preprocessing statements and collective communication statements. The first step in this direction was to extend the existing PRE framework interprocedurally. For applying this transformation across procedure boundaries, we need a full program representation. We have chosen a concise full program representation, which will allow efficient data flow analysis, while maintaining sufficient precision to allow useful transformations and to ensure safety and correctness of transformations. In Fig. 8, we show an example program (which involves irregular accesses to data). The program representation FPR for this program is shown in Fig. 9. We now briefly show how partial redundancy elimination is used for optimizing placement of communication preprocessing calls and collective communication routines. The details of our techniques are available elsewhere [2,3,4]. We use the example presented in Fig. 8 to show the
122
JOEL SALTZ ETAL
Program Example Real X (nnodes 1 , Y (nnodes 1 Real Z ( n e d g e s 1 . W ( n e d g e s 1 Integer IA(nedges). IB(nedges)
C
Input data ... do 10 i = 1. 20 C a l l Proc-A(X.Y.Z.IA.IB) i f ( n t . g t . 0) then C a l l Proc-B(X.W. I A ) endif do 5 0 j = 1. n e d g e s
50 10
continue continue end
IB(j)
20
30
35
40
45
=
.. I B ( j ) . . Main
S u b r o u t i n e Proc-A(A.B.C.D.E) d o 20 i = 1. n e d g e s C(i) = C(i) + A(D(1)) continue d o 30 i = 1 . n e d g e s C(i) = C(i) + B(E(i)) continue do 3 5 i = 1 . n n o d e s B ( i 1 = ... continue end
Subroutine Proc-B(X.W.IA) do 4 0 i = 1 , n e d g e s W(i) = W ( i ) + X(IA(i)) continue do 4 5 i = 1, n n o d e s X ( i ) = ... continue end
FIG. 8. An irregular code.
0
Procedure entry node
I--
{ :' Procedure return node
.-/'
FIG.9. FPR for the example program.
communication preprocessing inserted by initial intraprocedural analysis, and the interprocedural optimizations that can be done. Initial intraprocedural analysis inserts one communication preprocessing call and one gather (collective communication routine) for each of the three data parallel loops in the program shown in Fig. 10. We have omitted several parameters from both the communication preprocessing routines and collective communication routines to keep the examples simple. The communication preprocessing routine IrregSched takes in the indirection array and information about distribution of the data arrays.
PROGRAMMING IRREGULAR APPLICATIONS
P r o g r a m Example Real X ( n n o d e s 1 , Y ( n n o d e s ) Real Z ( n e d g e s ) , W ( n e d g e s ) I n t e g e r IA(nedges), IB(nedges 1 C Input data . . . do 10 i = 1. 20 C a l l Proc-A(X, Y , Z , I A . IB 1 i f ( n t . g t . 0) t h e n C a l l Proc-B(X,W.IA) endi f do 50 j = 1 , n e d g e s - l o c a l I B ( j ) = .. I B ( j ) .. 50 c o n t i n u e 10 continue end
S u b r o u t i n e Proc-A(A.B.C.D.E)
Schedl = Irreg_Sched(D) Ca 1 1 G a t h e r ( A . S c h e d l ) do 20 i = 1 , n e d g e s - l o c a l C ( i ) = C(i) + A(D(i)) 20 c o n t i n u e SchedP = I r r e g - S c h e d l E l Ca 1 1 G a t h e r f B , S c h e d P ) do 30 i = 1 , n e d g e s - l o c a l C(i) = C(i) + B(E(i)) 30 [ c o n t i n u e do 3 5 i = 1 . n n o d e s - l o c a l B ( i ) = ... 35 c o n t i n u e end S u b r o u t i n e Proc-B(X.W.IA) Sched3 = Irreg-Sched ( I A 1
C a l l Gather(X.Sched3) do 40 i = 1 . n e d g e s - l o c a l W ( i ) = W ( i ) + X(IA(1’)) 40 c o n t i n u e do 4 5 i = 1 . n n o d e s - l o c a l X ( i ) = ... 45 c o n t i n u e end
C
123
P r o g r a m Exarnpl e Real X ( n n o d e s ) . Y ( n n o d e s ) Real Z ( n e d g e s ) , W ( f l e d g e s ) I n t e g e r IA(nedges), IBcnedges) Input data
Schedl
=
lrreg_SchediIA)
do 1 0 i = 1. 20 [ C a l l Proc-A(X.Y.Z.IA.IB) i f ( n t . g t . 0) t h e n C a l l Proc-B(X.W.IA) endi f do 50 j = 1. n e d g e s - l o c a l IB(j) = .. I B ( j ) . . 50 c o n t i n u e 10 c o n t i n u e end S u b r o u t i n e Proc-A(A.B.C.D.E) Ca 1 1 G a t h e r ( A , S c h e d l ) do 20 i = 1 . n e d g e s - l o c a l C ( i ) = C ( i ) + A(D(i)) 20 c o n t i n u e SchedP = I r r e g - S c h e d ( E )
Call G a t h e r ( B . S c h e d 2 ) do 30 i = 1 , n e d g e s - l o c a l C(i) = C(i) + B(E(i)) 30 c o n t i n u e do 35 i = 1 , nnodes l o c a l B ( i ) = ... 35 c o n t i n u e end S u b r o u t i n e Proc-B(X.W,IA) do 4 0 i = 1 , n e d g e s - l o c a l W(i) = W ( i ) + X(IA(1)) 40 c o n t i n u e do 45 i = 1 , n n o d e s - l o c a l X ( i ) = ... 45 c o n t i n u e end
FIG. 10. Result of intraprocedural compilation (left), and code after interprocedural optimizations (right).
124
JOEL S A L E H A L .
In Fig. 10, we also show the program after interprocedural optimization of communication preprocessing routines and gather routines. We refer to the loop in the main of the program (which encloses the calls to the routines Roc-A and R o c B ) as the time step loop. Initially, interprocedural partial redundancy elimination is applied for communication preprocessing statements. Since the array IA is never modified inside the time step loop in the main procedure, the schedules Schedl and Sched3 are loop invariant and can be hoisted outside the loop. Further, it can be deduced that the computations of Schedl and Sched3 are equivalent. So, only Schedl needs to be computed, and the gather routine in P r o c B can use Schedl instead of Sched3. For simplicity, Schedl is declared to be a global variable, so that it does not need to be passed as a parameter at different call sites. After placement of communication preprocessing statements is determined, we apply the IPRE analysis for communication routines. The gather for array I A in routine Proc-B is redundant because of the gather of array D in routine Roc-A. Note that performing LPRE on communication preprocessing statements before applying IPRE on communication statements is critical, since it is important to know that Sched3, which affects the gather for array I B, can be replaced by Schedl . We developed an initial prototype implementation of our schemes as an extension to the existing Fortran D compilation system. We experimented with two codes: an Euler solver on an unstructured grid and a molecular dynamics template. Our experimental results show that optimized interprocedural placement of communication and communication preprocessing can improve performance by 30-50% in the codes which require runtime preprocessing. 4.
Runtime Support for Pointer-based Codes: CHAOS+ +
Unfortunately, many existing runtime systems for parallelizing applications with complex data access patterns on distributed memory parallel machines fail to handle pointers. Pointers are frequently utilized by many applications, including image processing, geographic information systems, and data mining, to synthesize complex composite data types and build dynamic complex data structures. CHAOS+ + is a runtime library for object-oriented applications with dynamic communication patterns. It subsumes CHAOS, which was developed to efficiently support applications with irregular patterns of access to distributed arrays. In addition to providing support for distributed arrays through the features of the underlying CHAOS library, CHAOS++ also provides support for distributed pointer-based data structures, and allows flexible and efficient data exchange
PROGRAMMING IRREGULAR APPLICATIONS
125
of complex data objects among processors [16]. CHAOS + + is motivated by the way pointers are often used in many real parallel applications. In these applications, hierarchies of data types are defined, such that ones at higher levels serve as containers for those at lower levels. Pointers are often used by container objects to point ro the objects they contain. Objects that are solely contained within a container object are referred to as sub-objecfs. A sub-object is effectively part of its container object, although it does not necessarily occupy memory locations within that of its container object. Objects of data types at the top of the hierarchy (i.e., objects of the outermost container class) can further be connected through pointers, forming complex pointer-based data structures. Such data structures are dynamic: their elements are often created and/or deleted during program execution, and accessed through pointer dereferences. Access patterns to such data structures cannot be determined until runtime, so runtime optimization techniques are required. As an example, Fig. 11 shows the declaration of a set of C + + classes, which can be used to describe how pixels of an image are clustered into regions, and how regions containing pointers to adjacent regions form a map. The R e g i on class is implemented as a container class for the P i x e l class, so that a P i x e l is a sub-object of a R e g i on. Since different regions may consist of different numbers of pixels, the Reg i on class uses a pointer to an array of its constituent pixels. A set of regions interconnected with pointers then form a graph, defined by the class R e g i on-Map. Figure 12 gives an example of such a graph. When a graph is partitioned among multiple processors, the runtime system must be able to traverse pointers to support remott: data accesses. CHAOS + + is implemented a!; a C+ + class library. The design of the library is architecture-independent and assumes no special support from C+ + compilers. CHAOS + + currently uses message passing as its transport
class Pixel { int x. y:
//
d
s i n g l e p i x e l o f an image
// x . y c o o r d i n a t e s
t ; c l a s s Region I i n t num-pi x e l s ; Pixel * p i x e l s : i n t num-neighbors: Region * * n e i g h b o r s :
// a r e g i o n c o n s i s t i n g o f p i x e l s / / rrimber o f p i x e l s / / (in a r r a y o f p i x e l s / / number o f a d j a c e n t r e g i o n s // l i s t o f p o i n t e r s t o a d j a c e n t r e g i o n s
I: c l a s s Region-Map
I
//
I Region * r e g i o n :
//
d
p o i n t e r t o some R e g i o n i n t h e g r a p h
t; FIG. 11. Pointer-based data structures containing complex objects.
126
JOEL S A L E ETAL.
i
-
p T J
array of Pixels
Region pointer to Region
-----
pointer to Pixel
FIG. 12. A graph of Region objects.
layer and is implemented on several distributed memory machines and on networks of workstations, 4.1
Mobile Objects
CHAOS++ defines an abstract data type, called M o b j e c t , for mobile objects. These are objects that may be transferred from one processor to another, so they must know how to marshal and unmarshal their contents. In general, the object model that CHAOS++ supports is one in which an object is owned by one processor, but other processors may possess shadow copies of an object, as will be discussed in Section 4.2. This implies that a distributed array of objects is treated by CHAOS + + as multiple objects, so that it can be distributed across multiple processors. The M o b j e c t class is designed as a base class for all objects that may migrate between processors, and/or will be accessed by processors other than the ones they are currently assigned to. M o b j e c t contains two pure virtual member functions, p a c k and u n p a c k , which must be supplied by any derived objects so that CHAOS++ can move or copy a M o b j e c t between processors. An implication of requiring the user to provide p a c k and u n p a c k functions for all M o b j e c t s is that CHAOS++ does not allow distributed arrays of C+ + base types (e.g. d o u b l e , i n t , etc.), because C + + does not allow a user to define member functions for base types. One way for an application user to implement such a distributed array using CHAOS+ + is
PROGRAMMING IRREGULAR APPLICATIONS
127
to define a class derived from Mo b.j ec t consisting solely of a member with the base type, and then provide the p a c k and u n p a c k functions for that class. In the applications we have investigated so far, this is not a major problem, because all the distributed arrays have been arrays of complex structures. For an object that occupies contiguous memory, the p a c k and u n p a c k functions consist of a simple memory copy between the object data and the message buffer. For a more complex object that contains pointers to subobjects, the p a c k and u n p a c k provided by the user must support deep copying. p a c k can be implemented by deriving the classes for all subobjects from Mobject, and having the p a c k function for an object recursively call the p a c k function of each of its sub-objects. On the receiving processor side, the u n p a c k function must perform the inverse operation (i.e., recursively unpack. all sub-objects, and set pointer members to sub-objects properly).
4.2
Globally Addressable Objects
In pointer-based data structures, elements (objects) may be added and removed dynamically. No static global names or indices are associated with the elements, and accesses to those elements are done via pointer dereferences. It is therefore not feasible for the runtime system to rely on the existence of global indices, as would be the case for distributed arrays. Furthermore, partitioning a pointer-based data structure may assign two elements connected via pointers to two different processors. This raises the need for global pointers. As supported by such languages as Split-C [41], C C + + [ 2 3 , 3 5 ] , and p C + + [11,67], a global pointer may point to an object owned by another processor, and effectively consists of a processor identifier and a local pointer that is only valid on the named processor. In CHAOS + +, these problems are addressed by introducing an abstract data type, called globally addressable objects, which we now discuss in detail. One obvious mechanism for managing global pointers is to define a C + + class for global pointers and ovmload the dereference operator (*), so that whenever a global pointer is ciereferenced, the necessary interprocessor communication is automatically generated. This approach, however, does not allow collective communication, which is an important technique for achieving high performance in a loosely synchronous execution model. Furthermore, dereferencing a global pointer requires a conversion between a reference to a remote object and a reference to a local buffer. This imposes additional overhead with every dereference of a global pointer. It is more desirable to perform the conversion only when the binding between the global pointer and the local buffer changes.
128
JOEL S A L E ETAL.
Instead of defining a class for global pointers, CHAOS++ defines an abstract C + + base class for globally addressable objects, or Gobjects. A Gobject is an object with ownership assigned to one processor, but with copies allowed to reside on other processors. These copies are referred to as ghost objects; each processor other than the one assigned ownership of the Gob j e c t may have a local copy of the Gob j e c t as a ghost object. Figure 13 shows a graph which is partitioned between two processors. The dashed circles represent ghost Gobjects. Each Gobject has a member function that determines whether it is the real object, or a ghost object. A ghost object caches the contents of its remote counterpart, but the decision about when to update a ghost object from a real object is determined by the application. The contents of ghost objects are updated by explicit calls to CHAOS++ data exchange routines. This descriptio
