Page i
ComponentBased Network System Engineering
Page ii
For a listing of recent titles in the Artech House Telecommunications Library, turn to the back of this book.
Page iii
ComponentBased Network System Engineering Mark Norris Rob Davis Alan Pengelly
Page iv
Library of Congress CataloginginPublication Data Norris, Mark. Componentbased network system engineering / Mark Norris, Rob Davis, Alan Pengelly. p. cm.— (Artech House telecommunications library) Includes bibliographical references and index. ISBN 1580530087 (alk. paper) 1. Component software. 2. Systems engineering. 3. Computer networks. I. Davis, Robert E., 1950 II. Pengelly, Alan. III. Title. IV. Series QA76.76.C66 N67 2000 99052405 004.6—dc21 CIP British Library Cataloguing in Publication Data Norris, Mark Componentbased network system engineering.— (Artech House telecommunications library) 1. Computer network architectures 2. Systems engineering I. Title II. Davis, Rob III. Pengelly, Alan 004.6'5 ISBN 1580530087 Cover design by Igor Valdman © 2000 ARTECH HOUSE, INC. 685 Canton Street Norwood, MA 02062 All rights reserved. Printed and bound in the United States of America. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Artech House cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark. International Standard Book Number: 1580530087 Library of Congress Catalog Card Number: 99052405 10 9 8 7 6 5 4 3 2 1
Page v
To Kate, Amy, and Adam for not deleting my text files, when it would have been just so easy. (MTN)
Page vii
Contents Preface
xvii
A User's Guide to This Book Acknowledgments 1 At the Edge of Communication
xix 1
A Global Issue
2
Some Basic Definitions
3
An Interface
xviii
3
Some Types of Interface
4
Why Interfaces Matter
5
Interfaces Enable Integration
6
Plug and Play?
7
The Real World
8
Tools for Integration
9
About This Book
10
Summary
11
Selected Bibliography
11
2 Interfaces and Complexity
13
Types of Interface
14
Protocols
15
Page viii
Application Program Interface
16
Middleware
18
Remote Procedure Call
19
Messaging
20
The Right Interface
20
The Design Problem
21
The Interface Equation
22
Summary
24
Selected Bibliography
24
3 Components Components—So What Is New?
26
Why Use Components?
28
What Is a Component?
29
An Example: Components in Car Assembly What Do We Design First—System or Components?
30 32
Car Assembly Revisited
33
Personal Computer Assembly
36
Bits and Pieces: What Have We Learned so Far
37
Views of Components
39
Software Components: At Last!
39
Generic Types of Software Components
44
Black Box Components
44
White Box Components
45
Glass Box Components
45
Gray Box Components
46
Examples of Specific Types of Software Components
25
47
Procedures, Subroutines, Objects, and Class Libraries
47
Objects and Class Libraries
47
Operating Systems
48
Databases and Spreadsheets
48
Page ix
Plugable Components Logical and Physical Components
52
Why Aren't Software Components the Same As Hardware Components?
52
A Layered Model of Components
54
Component Technologies
55
Component Object Model (COM)
56
Common Object Request Broker Architecture (CORBA)
56
Enterprise Java Beans
57
Software Components
58
MidTier Components
58
SystemLevel Components
60
BusinessLevel Components
63
Enterprise Components
63
Commercial Off the Shelf (COTS)
64
Summary
65
Reference
67
4 Integration What Is Integration?
50
69
69
Dealing with Existing Installations
70
Stovepipe Designs
70
Multiple Access Requirements
71
Closed Interfaces
73
Diverse Data
73
Nonstandard Users
74
Loose and Tight Integration
74
Key Concepts
75
Scrap
76
Trap
76
Map
76
Wrap
76
Unwrap
77
Page x
The System Integration Process Requirements
78
Design
79
Development
79
Acceptance
83
Build
83
Test
83
Release
83
Installation
88
Operation
88
Cease
88
Practical Integration
88
Publish a Strategy
92
Define an Architecture
92
Define an Evolution Plan
93
Prescribe Technology
93
Summary
94
References
95
5 Architecture and Structure
78
97
Would You Buy a Car from This Man?
97
Sum of the Parts
98
Structured Flexibility
101
Architecture or Design?
105
Architectural Styles
108
Views of Views
109
Architectural Layers
110
Other Architectural Models
112
Layers of Generality
112
Butler Model
113
The Rational 4+ 1 View
114
Architectural Structures
117
The Open Group Architectural Framework (TOGAF)
118
Page xi
DomainSpecific Architecture
119
Architectural Description Languages
119
Structure in Architectures
120
Patterns
120
AntiPatterns
123
Pattern Systems and Languages
123
Frameworks
125
Product Lines and Product Families
128
Stovepipe Architectures
130
Business Component Architectures Pipes, APIs, XML, and XMI
133
ERP Systems
134
The Open Applications Group (OAG)
135
Business Frameworks
136
Business Objects
138
Choosing an Architecture
139
Evaluating an Architecture
140
Using an Architecture
141
The Role of Architects
143
The Shape of Things to Come
146
Summary
147
References
148
6 By Threes and Fours—Bringing It All Together
131
151
Motivation
153
The Scenario
164
Interfaces
165
Networks
165
Server Technology
166
Desktop Technology
167
Processes
167
Information Systems Integration
168
Project Management
169
Page xii
Phase I—Project Preparation
169
Phase II—Solution Design
169
Phase III—Detailed Design
170
Phase IV—Integration and Testing
170
Phase V—Installation and Training
170
Summary
171
References
176
7 Engineering the Vision
179
The Network
180
LAN
181
WAN
184
Connection Control Switching
186
Message Switching
186
Packet Switching
187
Network Architectures
187
PeertoPeer Networks
187
ServerBased Networks
188
Network Operating Systems
189
Network Applications
190
Interface Software and Standards
190
Application Programming Interfaces
193
CORBA
195
ThreeTier Architectures
198
DCE
198
COM/DCOM
199
What Should I Use?
200
Summary
201
References
202
8 Towards ComponentBased Engineering
203
ComponentBased Development
204
System Specification, Analysis, and Design
184
205
Page xiii
Production Engineering
222
Component Design, Building, and Procurement
229
Systems Assembly and Delivery
237
Component Maintenance
242
Rapid Application Development and CBD
247
Methods for CBD
248
Structured Methods
250
OO Methods
252
CBD Methods
253
A Generic Approach to CBD The Need for an Overall Approach
257
An Experiment in Component Approaches
258
Case Study Results
259
A Process Framework for CBD
260
Butler Forum Universal Component Concepts Initiative
261
Describing Components
262
Repositories, Indexing, and Retrieval
264
Funding Component Approaches
265
Successful Use of Components
266
Business Issues
266
Cultural Issues
267
Technical Issues
269
Summary
270
Final Checklist
271
References
272
9 Interfaces and Integration—Standards and Formality The Nature of Languages
257
273
276
VDM Example—Creating and Removing Bank Accounts
276
CSP Example—A Simple Specification of the Process Diagram in Figure 9.2
277
Page xiv
Inherent Complexity
278
Formality and Protocols
279
Formal Methods
284
Summary
287
References
288
10 From Here to Eternity
289
Lost Horizon
291
Treasure Island
294
Summary
294
Selected Bibliography
296
Appendix A The Interface Equation—A Formal Theory of Interfaces Introduction
298
Formal Methods
298
Solving the Interface Equation
300
The Discarding and Constructive Algorithms
301
A Small Example
307
Conclusion
313
References
315
Appendix B Standards, Organizations, and Initiatives
297
317
Architectural Tradeoff Analysis (ATA) Initiative
317
Business Application Programming Interfaces (BAPI)
318
Business Object Component Architecture (BOCA)
318
Business Object Domain Taskforce (BODTF)
319
Component Definition Interchange Format (CDIF)
319
Component Definition Language (CDL)
320
COM/DCOM
320
CORBA
321
Page xv
CORBA Component Model
322
Distributed Computing Environment (DCE)
322
Department of Defense (DoD) Software Technology Programs
322
ESPRIT Software Evolution and Reuse (SER)
323
Interface Definition Language (IDL)
324
IEEE P1471: Recommended Practice for Architectural Description
324
INCOSE Systems Architecture Working Group
325
Microsoft Repository
325
Meta Object Facility (MOF)
325
NIST Advanced Technology Program—CBD
326
Open Applications Group Integration Specification (OAGIS)
326
Open Applications Group Middleware API Specification (OAMAS)
327
Object Management Architecture (OMA)
327
Object Management Group (OMG)
328
Open Applications Group (OAG)
328
Open Distributed Processing (ODP)
329
Open Group
329
The Open Group Architectural Framework (TOGAF)
330
Rational Unified Process
330
SEI Software Architecture Technology Initiative
331
Software Architecture Analysis Method (SAAM)
331
Software Architecture Technology Guide
331
Telecommunications Information Networking Architecture (TINA)
332
Unified Modeling Language (UML)
333
Workflow Management Coalition
333
WorldWide Institute of Software Architects
334
World Wide Web Consortium
334
XMI
335
Extensible Markup Language (XML)
335
Page xvi
Glossary
337
About the Authors
365
Index
367
Page xvii
Preface The pace of change in networks and systems seems to get ever faster. The speed with which new and innovative solutions can be assembled is increasingly what differentiates the best from the rest. Hence the interest in componentbased engineering. The ability to combine hardware, software, and network elements in a predictable way is no easy task, but the rewards are considerable. It is important not only to know what the techniques are, but also what you can do with them and what is on offer to meet your needs. Where do pragmatism, standards, and theory help and how can you make them work in your environment? This book has several unique features: • It puts a wide range of issues into useful context. Rather than explaining the details of one area, it explains interfaces, integration, components, and architecture and shows how they relate to one another and what you need to know to capitalize on them. The aim is to provide a howto guide to building systems that is based on firm theory and broad experience. • It takes the pragmatic view of a complex area that has come to be dominated by technology, not always in the user's best interests. Care has been taken here to abstract from this complexity and make the topics covered accessible and relevant to real needs. The focus is on practical application, rather than technology per se. The underlying theory is not ignored—it is simply taken as a support to the endgame, rather than an end in itself.
Page xviii
This book is: • Essential reading for those engaged in the construction, design, and implementation of networked systems; this book provides the broad understanding required to avoid expensive mistakes. • A valuable professional updating guide for network designers, systems integrators, technical architects, telecommunications engineers, system analysts, and software designers, as well as business and information planners. • A useful text for final year and postgraduate students in computer science, electrical engineering, and telecommunications courses. It seems likely that there will be few speed limits on the information superhighways—and no turning back. Those who choose to stay in the slow lane will be left behind very quickly. Those who choose to compete in the new age need to be aware of what lies ahead. Informed choices, made now, will pay handsome dividends as complexity and choice (inevitably) rise. The end of the twentieth century is likely to be seen, in retrospect, as the adapt or atrophy period for many organizations—this book can inform an exciting but perilous journey. A User's Guide to This Book This book was really inspired by our wish to give a straightforward account of a fragmented topic. We spent a long time ourselves making sense of systems, components, interfaces, and integration and would like to spare others from this subtle form of torture. So we have tried our best to cater to a wide range of tastes by explaining both the basic ideas and how they fit together. Different parts of the book will, no doubt, be more or less relevant to different people. Some parts have been written to outline general principles, others to recount a specific technique. To help you select a suitable path through the book, here is our summary of the joys that we think each chapter contains.
Technical Content
General Interest
Specialist Detail
Chapter 1
*
****
*
Chapter 2
**
***
**
Chapter 3
***
***
**
Chapter 4
***
***
**
Chapter 5
***
***
**
Chapter 6
***
***
***
Page xix
Chapter 7
****
***
****
Chapter 8
***
***
****
Chapter 9
***
***
***
Chapter 10
*
****
*
Appendix A
*****
*
*****
Appendix B
***
**
****
Glossary
*
*
****
To help those who prefer an occasional dip into a technical book, rather than a concerted attack, we have appended a fairly large glossary that should get you through the more challenging sections. Acknowledgments The authors would like to thank a number of people whose help and cooperation have been invaluable, including those kind individuals who contributed ideas, advice, words, and pictures and even volunteered (we use this word in its loosest sense) to review early drafts: Professor Darrel Ince, Professor Martyn Sheppard, Dr. Alan O'Callaghan, and Ray Lewis and his colleagues in the BT Systems Integration team. Their observations, illustrative stories, guidance, and constructive criticism have always been valuable and have done much to add authority, interest, and balance to the final product. We would also like to particularly acknowledge the contribution of the BT Component Systems Engineering Team whose ideas and experience in this exciting and challenging area underpin much of what is presented in this book. In particular we thank: Richard Shortland, Mike Scott, Andy Kelly, Alec Edwards, Greg Howett, Jim Hutton, and Philip Williams. Finally, thanks are due to our many friends and colleagues in the telecommunication and computing industries, standards bodies, and professional organizations whose experience, advice, and inside knowledge have been invaluable.
Page 1
1— At the Edge of Communication Federations work better than monolithic organizations because, along with strength, they offer the degree of flexibility we need to deal with these turbulent times. Warren Bennis
Just over a hundred years ago Alexander Graham Bell invented the telephone—and thus began true global communications. From that point on, technology has come to play an ever more important part in all our lives. Our reliance on networks, computers, and software has reached a level of addiction: most people are painfully aware of what happens when a supermarket checkout or flight booking system goes wrong. Less visible are the legion systems that we rely on for everyday necessities such as payment of bills, monitoring of air traffic, and delivery of information. If these go wrong, it is not simply inconvenient—the consequences of misoperation can be altogether more serious. This book is all about building systems that work in a world that wants results in a hurry. The premise is very simple—that effective and manageable systems should be built from components that are put together using a sound interfacing and integration strategy. In effect, this is no more than what already happens in many of the well established engineering disciplines. For instance, a civil engineer knows how to place a structure on firm foundations and has building regulations as a measure of the fit between component parts of the building.
Page 2
Here we show how the components of a communication system should be assembled. Just as with civil engineering, this entails the definition of components and how they fit. With so much established infrastructure (also known as cherished, legacy, or old systems, depending on your viewpoint), as much care has to be taken when introducing new system elements as when introducing new parts in a medieval cathedral. Our focus is very much on how to systematically tackle this integration challenge. Our contention is that this will become ever more important in a world that relies on interconnected systems for so much of its daytoday dealings. A Global Issue Around about the same time that Bell put telecommunications on the world map, Babbage invented his inference engine, the forerunner of today's computer. Thus began the age of the electronic computer (though some would argue that it was not until Von Neumann and Turing came along that the forerunner of today's computer really began to emerge). The computer provides something quite special—flexibility. By using one or more from the great range of programming languages, we can get a computer to perform an almost infinite number of functions—we are constrained only by memory and storage (and, of course, our own imagination). It is only in hindsight that we can appreciate just how momentous Bell's and Babbage's ideas were—they have shaped the way we live, work, and communicate on a truly global scale. The world would be a very different place without the computer and the telephone. Indeed, if they disappeared overnight, there is no way that we would be able to carry on with our modern lives as they are. So we live in a world that is dominated by computers, software systems, and the means by which we and computers communicate with each other—networks. Virtually every office in virtually every country on the planet is now littered with PCs, which are almost certainly going to be networked locally, enabling employees to exchange information with their colleagues on a different floor or in a different building. It is becoming more likely they will, in turn, be part of a businesswide intranet that not only connects the various offices in a building, but connects different buildings in a city, different cities, and ultimately different countries. This level of connectivity is immensely powerful and has altered beyond all recognition the productivity and effectiveness of our daily work. And with the expansion of the networked computer to the home, why bother going to the office—why not work from home?
Page 3 1
Perhaps the most sobering thought here is that this is just the beginning—with technology improving at an exponential rate, who knows what lies around the corner? Exciting as all this is, in this book we do not really talk about computers and communication systems per se. There are already many excellent texts. What we are interested in is something more fundamental, something which makes all of what we hope for actually feasible. Our concern is the problem that has arisen from the universal reliance on ''software inside"2—and that is how to take a set of components (in this instance software that sits on networked computers) and to integrate them (both with each other and the status quo) through welldesigned interfaces. Some Basic Definitions So far, we have talked rather loosely about communication systems. Given that we want to forge a systematic basis for their construction, we should be clear about what they are! In this text, we understand a system to be comprised of a number of components, some hardware and some software. Because systems of any real size or significance need to send and receive information, they have to be part of a network and so have a number of external interfaces. The task of integration is that of assembling the most appropriate components in such a way that all of the interfaces work correctly and that the system functions as desired. We will illuminate these definitions in subsequent chapters. For now, a little more detail on one of our key notions—an interface. An Interface An interface arises whenever two or more distinct entities or components need to communicate in some way. The interface itself defines the allowable transactions that the two components can undertake. It defines the mapping between data sets and between protocols. This typically involves the exchange of information and is performed to carry out some function which neither component, in isolation, can itself perform. It may be that the sequence of communication 1.
There are a number of laws of technology that support this. Moore's and Metcalfe's laws state, respectively, that computer processing power and network usefulness increase over time (and have done so for over ten years). 2.
Even simple devices like shavers have thousands of lines of code in them. TVs and cars contain significant amounts of software, and networks rely on many millions of lines of code (and the printout for each million lines of code runs to some 3 miles).
Page 4
between components is important; it may not. At the same time, it may be that all components need to work concurrently or sequentially. The key point is that there are many types of interface, some simple, some not. The concurrent system is usually the more complex and dynamic, with numerous transactions occurring simultaneously. Hence, nondeterminism can occur, and this makes prediction of the system's future behavior problematic. The sequential case on the other hand follows a more predictable pattern of behavior. In both cases, though, the interface between each component is critical; without suitable interfaces the system as a whole will not function as required. Hence the success of system integration depends on the correctness of the interfaces. At a more general level, interfaces pervade all corners of life. As humans, we spend our days interfacing with each other, with vending machines, ticket dispensers, and computers. We integrate ourselves into cars, trains, and aircraft where we have to interface with displays of key parameters and control systems. For commercial aircraft, getting the interface right is absolutely essential. Accidents involving the A320 Eurobus have been firmly attributed to poor interface design. Not only do humans and machines interface, but machines interface with each other. It is with this type of interface that this book is primarily interested. Why? Because when performing the integration of a large software system it is invariably the case that if a problem occurs it will be with the system interfaces. Why this should be so is attributed to various reasons. The move toward outsourcing has resulted in a multitude of suppliers developing sets of components in isolation. The moment of truth is when the system is integrated. All too often a poor requirements specification or illconsidered design reveals itself—one design team wasn't quite sure what some of the system components were really doing. The integration process is fraught with problems. Ask software engineers—if they want to find a problem, they just look for an interface! Some Types of Interface There are different types of interface, and the type depends, to a large extent, on the information being transferred. For example, a humantocomputer interface will not generally involve the physical transfer of data. Information is either transferred visually or by the human pressing the keys. An interface, say, in a database system, however, will typically involve the movement of large amounts of data from one system to another. Protocol interfaces are yet another variant that usually entail message passing (though data is also transferred).
Page 5
More recently, interactive speech interfaces have been developed. In the case of human processes, information is transferred in the form of paper memorandums or e mail messages. For flybywire aircraft, the data is transferred from the pilot, via arm and leg movements and electrical pulses, eventually to the control surfaces. The computer will also send signals back to the control column to simulate feedback. Different though all these are in terms of implementation, they all entail the same underlying principle—that of a welldefined mode of interaction (sequence of steps) that constitutes the interface between two systems. The next chapter will provide a deeper examination of the key issues involved: how we can categorize interfaces, what the key criteria for their effectiveness are, and what practical difficulties in building often arise. It will also propose solutions to some of these difficulties that rest on the development of mathematically based design and development paradigms. For now, some headlines. Why Interfaces Matter We have said that an interface is an element of software that either passively (the data passes through the interface unchanged) or actively (where the data is transformed from one form into another) enables one system to exchange information with another. By analogy consider the following situation. Mr. Lee is Chinese but speaks no English; Mrs. Smith speaks English but no Chinese. They are put into separate rooms, but can speak to each other via microphone and headsets. Thus, they cannot use body language, gestures, or pictures, only the spoken word. Clearly no exchange of information can take place here that is meaningful to either party. Mr. Lee can make no sense of Mrs. Smith, and vice versa, until an interpreter arrives who can understand and translate between both languages. The interpreter is an interface. In this case, the interface is interactive since it must convert from one language to another (a passive interpreter would merely relay the information in the same language). In this example the presence of the interpreter is essential—nothing can happen without him or her being there—and so it is with computer networks and software systems. Without appropriate interfaces the various components which make up a software system would not be able to engage in transactions between each other. As with the natural languages in the example above, there is a wide diversification of network systems (for example FDDI, Ethernet, and ATM), protocols (such as TCP/IP, RS232, CCSS7, and GSM), programming languages (the de facto standards C, C++, Java, COBOL, and Fortran), database technologies
Page 6
(the widely used Relational, Object, and Network databases), and hardware platforms (Mainframe, PC, Workstation, and Mac). Each of these examples has its place in the grand scheme of things, and when viewed at a local level (on a small scale such as an office), they certainly each perform the task for which they were intended very well indeed. However, the ever increasing need to work across technological as well as geographic boundaries means that the problem of interfacing these various platforms and networks has become a key issue. Just how do we communicate if I use an Ethernet network and you use a FDDI network? How do we share information if I use a Sun workstation and a colleague uses a PC? How does a business bring together two "stovepipe" systems, one an IBM/CICS/COBOL system and the other a client/server system implemented using Unix Workstations, DCE, and C++? With the diversity and distribution of systems and information, the answer cannot be "I don't know!" Interfaces Enable Integration There are, in fact, solutions to each of the problems posed. An FDDI and Ethernet network can be connected via a network switch known as a router, usually a proprietary device that supports many protocols. A Sun workstation can emulate a PC using a product such as Exceed, which provides a PClike window from which PC applications such as Microsoft Word can be run. So, there are products that help to join one island with another. An increasingly popular general approach to bringing together two stovepipe systems is to use middleware, which as the name suggests, provides a backbone onto which the systems can be accessed by a user who does not need to know anything about where the data they require is located. Each solution relies on an interface—whether it be an interface between hardware and software, or between hardware and a user (as in the case of Exceed). Interfaces exist in countless numbers around the globe. They are the solution to the diverse range of systems and products that is available. Ask experienced systems engineers the first place they will look if they are experiencing problems with a system and they will point to the interfaces. Why that is we will discuss later. Interfaces are all important; they are the glue that binds the various components together. Get them wrong and the system will either fail to work or will quickly disintegrate. So given that, where should the project manager allocate the primary resource? What are the real showstoppers that are going to bring such projects down? When integrating such systems what are the key issues regarding project management, testing, and delivery?
Page 7
We will be addressing these issues (and more) as we unfold our story. Later on we will explore the technology available to assist the engineer to develop systems, as well as some tips on how to manage them. We will also look ahead and examine what the future holds, both in terms of standards and technology. Plug and Play? Although standards bodies have attempted to bring some degree of order, and certain de facto standards such as Windows and Unix have emerged (more by chance than design it must be added), the overall picture is still one of bewildering confusion. It would be tempting to hope that eventually standardization will occur and that integration would be a simple "plug and play" task. To this end, it clearly would be ideal to adopt a single technology for a specific set of requirements, for example, Lotus Notes or Microsoft Office for office automation or Unix plus C++ for software development. To a large extent these technologies are rightfully regarded as standards, but they are the exception rather than the rule. In the case of hardware, be it computer or network, the degree of diversity is far greater and a comprehensive set of standards far from view. Although standardization alone is not going to remove the need for interfaces, those interfaces would be a lot simpler to develop and manage if they were specified to a common standard. There are clear commercial incentives for companies to adopt standard technologies, but the truth is that there is already too large an installed base, and too large an investment, for companies to strip out their existing systems and capital assets simply to replace them with more uniform technologies. Even though many of these systems, such as mainframes, are largely antiquated, the cost of replacing these (often missioncritical) systems can run into tens of millions of dollars. So while few would argue that standardization is the holy grail, in truth it is something of a utopian dream and at best many years off. Because convergence is unlikely, we are forced to look at methods and tools whereby heterogeneous systems can, in fact, talk to one another. There is little choice but to try and integrate, aim for peaceful coexistence, and give the impression of a single logical system. A good example of this is the Internet. It actually consists of a hugely diverse variety of networks patched together to give the semblance of a global network. Transfers from one subnetwork to another are achieved through what is referred to as a gateway, or protocol converter. This software translates from one protocol to another, changing the syntax but preserving the semantics, very much like our interpreter helping Mr. Lee and Mrs. Smith to communicate.
Page 8
They can be expensive to produce, but once in place they provide an important means of connecting various networks together. The Real World It is often the case that more than one interface unit is required because the process of interfacing two large systems can be more than a little complex. Consider the following example. A bank has two main IT sites. One is in London, where most of the inhouse development takes place, while the other is in Paris, which looks after the customer accounts and batch processing. Both of the systems are large mainframes using ECL technology (rather than the more uptodate CMOS). Both are dedicated systems, designed for the particular aspect of the business for which they were constructed. In other words, they are stovepipe systems. The bank in question has been fortunate enough to have been steadily growing over the last decade or so. Being forward looking, they are preparing themselves for the single European currency, the Euro. Up until now they have been able to cope with extra demand by buying new hardware, but ECL technology is no longer supported, so the company is forced to consider alternative strategies. Wholesale replacement of the existing hardware is not economically viable. These are legacy systems in the truest sense of the word, vital to the business and too deeply embedded within the business to surgically remove and replace with a new system. The usage patterns of the systems (separately) are that the development site is very busy during the online day, but very quiet at night, whereas the customer support site is busy dealing with transaction processing applications during the online day and batch programs in the evening. It is clear from this that the development site (which we will call site A from now on, with the customer support site being site B) has spare capacity (in terms of the CPU) during the offline day. It could certainly support the batch programming requirements of the company if the two sites could be linked, as it makes sense for an ISDN network link to be established between site A and site B. The first requirement is for a gateway that will enable the two systems to communicate. The details of the protocols are now important here. Suffice it to say that without the gateway the systems will still be logically separated even though there is a physical link. So that is interface number one. Once data is flowing freely between the two sites the problem of multiple updates emerges and data integrity becomes very important. This is interface
Page 9
number two. To illustrate, site B sends site A a batch program request which it duly undertakes. This program requires data from site B, so site A has to operate remotely on the data. But B could also be accessing the same data, so there is the danger of deadlock (and worse) unless some sort of data integrity device is added to the system. In addition, the extra processing requirements need to be dealt with by the addition of new CPUs to each site. The situation is further complicated by the fact that the data volume has grown so large that the 32bit CPU (which will provide access to 2 gigabytes of data) is no longer sufficient (resulting in too much paging and consequently poor performance), and that 64bit machines are required. Site A is also designated the disaster recovery site so it will now need to mirror the contents of site B. Something like IBM Sysplex technology would be utilized to deal with these requirements. Also required is some means of synchronizing the two systems (via the coupling facilities that are a key component of the Sysplex approach). Although the complete replacement of the IT infrastructure is generally not viable, the use of existing components is by no means simple. The fact remains that interfacing two (or more) legacy systems is a complex and expensive undertaking which requires very careful planning and implementation. It is a bit like performing surgery on a patient who is not only awake, but performing his/her usual daily activities as well. Tools for Integration As stated, there are various methods, tools, and technologies available to help an integration team. In practice, their application may seem somewhat haphazard and unsupported with no clear underlying method, notation, or strategy. They do, however, provide a valuable service and without them things would be a lot more difficult. So where are the support tools going? What new techniques are being developed to provide a more generic and structured method? Later on in the book, we will examine a number of approaches which range from the almost hereandnow to ideas that may be a few years away in terms of widespread application. A good example of the latter is a concept referred to as the interface equation. This is a systematic method of developing and generating system interfaces which is underpinned by formal mathematics and thus presents the integrator with a precision tool: there is hope that the craft of integration may one day become a science. The essential idea of the interface equation is that any two components (which we assume to be different) can be described in a formal and unambiguous notation. The interface equation operates on the specification for each component to produce, and generate the software for, a suitable interface
Page 10
which, when combined with the original two components, provides transparent communication. This theory represents the current state of the art in interface design. What the interface equation is, how it works, and what its relevance is to practical system integration will be explained in subsequent chapters and with technical detail in an appendix. The theory itself arose from the need to interface ISDN services to existing networks. It soon became clear that the approach developed worked well and could be generalized to include not only other protocolrelated problems, but also more abstract system interface problems (from hardware to human processes). About This Book The next chapter will examine in a little more detail what we mean by an interface and will illustrate its importance (for instance, how a service provider is reliant on electronic interfaces to do business). Chapter 3 will look at the constituent parts of the system being interfaced as well as its components and how they are defined and used. Chapter 4 takes a practical view of the demands and potential pitfalls of system integration, with particular focus on how a topdown approach that specifies and tests the interfaces between key systems components can forestall many of these pitfalls. To complete the review part of the book, Chapter 5 will explain some of the logical structures, or architectures, used to build real systems. The way in which the ideas and theory apply in practice is illustrated in Chapters 6 through 8. Chapter 6 brings together concepts introduced in the first part of the book and shows how they relate to the real world. Practical application is further illustrated in Chapters 7 and 8, which use examples across a range of applications, from telephony to network computing. Concluding chapters give an assessment of the current stateoftheart in interface development and systems integration. The key points that will be reinforced are that two disciplines are intimately connected and that a systematic, topdown approach needs to be enforced to assure fitness for purpose. Interfaces that are correct and error free will greatly ease the problems associated with system integration. Likewise, a more formal approach to system integration could provide valuable and concise information for the interface designer. The growth in distributed working, componentization, and electronic trading means that the issues associated with interface design and system integration are likely to become more and more pressing. To close we will summarize the key issues faced by system integrators and interface designers in industry. The formal basis for interface design is appended along with descriptive material on the standards and concepts used throughout the book.
Page 11
Summary We now live in a world dominated by technology. It is all around us, and we are completely reliant on it for the smooth running of our lives. Over the years, people have gone to some lengths to ensure that information is shared between systems and that resources are pooled so that the available technology can work together. This means that changes have to be carefully managed so that they fit in with the status quo. Integration has become a demanding activity and interfaces a vital part of successful integration. In this opening chapter, we have explained why interfaces are so important and hinted at a framework in which they can be systematically produced. Most experienced systems engineers already know how tricky it is to get the various parts of a complex system to work together as required and how quickly things disintegrate when the job is not done right. Our aim from here on is to put a key aspect of system design onto firmer footing. Selected Bibliography Davies, D., C. Hilsum, and A. Rudge, Telecommunications after AD 2000, London: Chapman and Hall, 1993. Gore, A., "Infrastructure for the Global Village," Scientific American (Special Issue), 1995, p. 156. Lipnack, J., and J. Stamps. The Age of the Network—Organizing Principles for the 21st Century, New York: Oliver Wight Publications, 1994. Norris, M. Understanding Network Technology: Concepts, Terms, and Trends, Norwood, MA: Artech House, 1999.
Page 13
2— Interfaces and Complexity Between the idea and the reality, between the motion and the act, falls the shadow. T.S. Eliot
If we lived in a simple world, there would be no need for this book. The fundamental reason that we worry about the systems that underpin just about everything we do is that they are so complex. It is not that each part is beyond comprehension, rather that the assembled set needed to cooperate on a task is so diverse and subtly interrelated. Most real world transactions call on the services of many pieces of technology. The simple and intuitive act of making a phone call, for instance, involves transmission systems, routing algorithms, storeandforward switches, intelligent peripherals, and many other pieces of equipment. More often than not these are separate pieces of software developed by different people at different times that have to work in harmony to complete a specific task. There is inherent complexity in this multithreading, and this flows from the fact that each contributing part has many degrees of freedom. The most basic handle we have on that complexity is the way in which each part of the whole is connected: its interface. Interfaces, as we have already hinted, come in many guises—protocols, plugs, graphical user interfaces (GUIs), application programming interfaces (APIs), special purpose languages (such as IDL, the Interface Definition Language), and basic human interaction. One generalization that we can make about all of these is that if you conform to the interface presented, you stand a good chance of getting what you want. Cash, information, electricity, and so
Page 14
forth can all be yours if you offer the right sort of plug or the right command(s) in the right sequence. Smart as they are, machines are not as adaptive as humans when it comes to interaction with a third party, so they have a bit more difficulty with interfaces. Careful design is needed to ensure reliable interworking between one system and another. And with a huge number of sequences of interaction, it is important to ensure that all of the undesirable ones (those, for instance, that lead to deadlock1 between two systems) are avoided. So much for the generalities of interfaces. In the next section, we look in a little more detail at the role of the interface in computer interactions and the assembly of telecommunication networks. The remainder of the chapter shows where interfaces and the tools to help with their design fit into a systematic scheme for building networked and interactive systems. Types of Interface Although not backed up with firsthand experience, we have been told that there is more than one way to skin a cat. The same is true of connecting system components. There are several ways to build an interface, each with its own characteristics. These have been referred to already, but now we explain just what each type of interface is, where it is generally used, and what its strengths and limitations are. One general point that we need to make is that there are two distinct aspects to interfacing. The first is the physical means that enables separate systems to link together. The second is the more abstract notion of information exchange—just what is it that the systems are passing between themselves? Hence, there are (at least) two things to consider when we look at types of interface: the underlying infrastructure (by analogy we might think of these as the roads and railway lines connecting cities, or as doorways between rooms) and the actual information (the trains and trucks traveling along the railway and road system). When it comes to computer systems, there are essentially four mechanisms by which these interactions can be achieved: 1. Protocols and communication networks; 2. Application program interfaces (commonly known as APIs); 1.
This occurs when two processes are waiting for the other to do something. With both unwilling to proceed without input, the system stalls.
Page 15
3. Shared memory, shared data, and middleware; 4. Remote procedure calls and messaging. In practice, real life systems may well include aspects of all of these. Quite often it will depend on the geographical nature of the systems involved. Do they reside on the same computer, or are they dispersed throughout an openplan office or across several sites separated by some distance? In the text that follows, we will explore what each mode of communication actually involves. By way of example, consider a computer, a PC in fact, which has installed some proprietary operating system (e.g., Microsoft Windows). Also installed on this PC is Microsoft's groupware product, Outlook. Now there are quite a few things you can do with Outlook. You can construct your own programs (written, for example, in Visual Basic) which can interface with Outlook. Why would you want to do this? A groupware product is designed to do a particular job and that is to enable groups of people to work together. Indeed, Outlook includes a range of inbuilt functions such as email. However, different people or companies work in different ways so it is important that a product such as Outlook has some builtin flexibility without compromising the integrity of the product. This may involve the ability to communicate with other Microsoft products (or applications), or it may involve the facility whereby a programmer can add new functions. To achieve this, the product will incorporate a number of APIs. These are like recognized access points into the system. For a proper interface, you also need a means of communicating and the necessary hardware. The rest of this section considers the main types of interface in a little more detail. Protocols When two parts of a system are separated, they need some means of communication that both parties recognize. A protocol provides just that—an agreement between components (or people) about which of them can do what, to whom, and when. In practice, this is realized by a set of messages and a mechanism by which they are exchanged. In the world of telecommunications, communication protocols have been an important element for many years. Signaling between network switches relies on the CCITT Signaling System Number 7. XModem, Kermit, and X.21 all provide network interfaces to computers, and communication between clients and servers typically requires the Internet Protocol (IP). The basic elements of any communications protocol are a set of messages (usually taking the form of a data packet with predefined fields, as shown in
Page 16
Figure 2.1 and a set of rules governing the exchange of messages (who can send what, and when). We can illustrate this with reference to IP. This specifies a format for the packets that are used to interchange information. This has designated fields that wrap the data carried within the packets. These fields identify the sender's address, the intended recipient's address, the length of the packet, its time to live, and a whole host of other data that helps to control information interchange. If a packet is lost or misdirected, then this can be detected and remedial action can be taken (e.g., a replacement packet sent). The ideas of using a checksum to detect corrupt transmission, sequence numbers that allow information streams to be reassembled in the correct order, and flags to delineate control data from the information carried are common protocol features. A good protocol allows efficient and reliable communication between two parties. Achieving this is by no means straightforward, and protocol designers have to ensure not only for absence of deadlock and race condition but also that resilience against transmission errors is not secured at the cost of throughput. Application Program Interface Application program interface (API) is a term which is widely encountered in software engineering circles but which is often shrouded in mystery when time comes to explain what it actually means. An API consists of the calls, subroutines, or software interrupts that together comprise a formally specified inter
Figure 2.1 A typical frame format.
Page 17
face. The API allows a (typically) higher level program, such as an application program, to make use of the (usually) lower level services and functions of another application (the host), operating system, network operating system, driver, or other lower level software program. The resulting library of functions provides a new capability, such as writing a file in an application program's proprietary format, communicating over a TCP/IP network, or accessing a SQL database. A good example of an API, and one that is both popular and visible to the user, is the Winsock API, which gives a Microsoft Windows PC the capability to talk TCP/IP. This capability is required if you want to gain access to the Internet, for example, or to a TCP/IPbased enterprisewide Intranet. The lower level software may be integrated with the application program at compile/link time (using an include file or object library) or loaded (before the application program) as a driver, DOS TSR, Novell NLM, or Windows DLL. Figure 2.2 illustrates the idea of the API. This particular one is located between an application program that needs the services of a protocol stack (such as a TCP/IP protocol stack) and the TCP/IP protocol stack (which itself has an interface to an Ethernet adapter, for example). APIs are becoming increasingly prevalent now that open systems are gaining a wider user base and customers insist that systems and products interwork with those of other vendors. Going back to our PC example, we now have included a number of APIs which greatly enhance the functionality of the proprietary application software already loaded onto the system. The inbuilt APIs have enabled us to customize, to some extent, the software package. Otherwise, we would have had to bend the process to fit the software, or constructed some bespoke software. We do not really want to do either of those if at all possible.
Figure 2.2 Example of an API.
Page 18
However, in today's organizations, there is an increasing need to work laterally as well as up and down the command chain. This results in the need for the PC to communicate with a number of other systems, most of which were never designed to interface with a PC running DOS. How do we achieve this? The answer is middleware. Middleware Many of today's large companies have IT systems which are both complex and heterogeneous. The complexity stems from many causes, but mostly from the move to greater levels of process automation. They are typically heterogeneous because not only are the systems designed with a particular purpose in mind, and the most appropriate technology chosen on the basis of which will do the job most effectively, but also technology is progressing at such a rapid rate that this year's model is next year's legacy system. Some would call it variety; some would call it the ''best of breed." The bottom line is that most businesses have a huge range of systems to integrate. What is appropriate from a technical perspective is not always the same from a business perspective, and this tendency toward IT systems based on numerous different technologies and vendors soon becomes a very difficult beast to manage. Many companies, if they could afford to, would strip out their existing systems and start again and base the systems on standard technologies. But few, if any, are in such a position (though examples do exist). The only alternative is to find a way to make these different systems (which were typically designed as standalone, bespoke systems) appear to be one single logical entity. It is with this aim in mind that middleware was developed. In the computer industry, middleware is a general term for any program that serves to glue together, mediate between, or enhance two separate and usually already existing programs. In somewhat glib terms, it is the "/" in client/server. A common application of middleware is to allow programs written for access to a particular database to access other databases. Some users may also use the term to describe programming between a software program and hardware logic. Some example application models for middleware are: 1. Workflow; 2. Distributed transactions; 3. Remote file or database access; 4. Distributed objects; 5. Distributed database access.
Page 19
Middleware is software that facilitates crossplatform, distributed computing (that is, getting computers from different vendors and running different operating systems to communicate). It can also be viewed as an API which shields developers from underlying transport protocols and operating systems, along with a mechanism—for example, message passing or remote procedure calls (RPCs)—for cooperating applications to communicate (over a LAN or WAN or within the same machine). The effect is that from a user's point of view an Oracle database looks the same as an IDMS database, and DOS looks the same as UNIX. It is this middle layer of software that effectively does all the translation necessary. Hence, it acts as an interface agent. Clearly, this apparent transparency between systems does not come for free. Middleware can be very complex. What might have been little more than the act of pressing a button on a keyboard to invoke some batch process at a local database, for example, will typically involve a series of programs in the middleware layer just to get to the pressing of the button stage. The advantage is that all this interface complexity is bundled into a single layer where it can be properly managed and controlled. Later on we'll examine the candidate technologies for building middleware (e.g., CORBA and EJB). For now, suffice it to say that we have APIs and middleware working together to open up the corporation's operational support systems. The former can be quite simple but is localized, the latter more widely usable but complex. Between them, APIs and middleware only provide a platform. What is the actual communication mechanism? This is where RPCs and messaging come in. Remote Procedure Call Remote procedure call (RPC) is a method of programtoprogram communications, usually used when there is a need for multiplatform distributed computing. Generically, an RPC is an interprocess communication API whose strength is communication between different computing platforms, using different protocol stacks. It is often used with TCP/IP, in which there are (unfortunately) two incompatible standard RPC implementations: 1. Open Software Foundation's Distributed Computing Environment (DCE); 2. Sun's ONC+ (probably the more widely implemented of the two). To an application program, RPCs are local procedure calls that just happen to start processes on remote machines. Communication is synchronous
Page 20
(sometimes called blocking), in that the requesting process must wait for a response before continuing. From the programmer's point of view, there are local resources (that can be called directly) and remote resources (that are provided by stub code and invoked by the RPC). The idea of stubs—code placed on a server for general access—is central to the RPC approach. An alternative to this concept is called messaging. Messaging Simply put, messaging is a loosely coupled method of communication between platforms which uses mailboxes rather than RPCs. Mailboxes permit asynchronous interprocess communication. A good example is Microsoft's MAPI, which stands for Messaging API. This is a standard way of providing communication services to applications (making them mailenabled) so that they can send and receive mail (blocks of data, documents, files, and so forth), directly from within applications. The APIs are independent of platform, mail system, and transport protocol. MAPI is the most popular messaging API for the PC environment as it is backed by Microsoft, Novell/WordPerfect, and Lotus. Two levels are defined: 1. Simple MAPI—this will perform basic mail functions (such as sending and receiving messages). 2. Extended MAPI—this has a service provider interface (SPI), which will interface to software, which in turn provides an interface to other mail services, such as CompuServe. In this way, any application using the extended MAPI will be able to use CompuServe as a message carrier. MAPI's main competitor is VIM (which is used mostly for Lotus applications running under Windows or OS/2) and XAPIA's CMC (which is used more for cross platforms). As you can see, what started out as a basic standalone PC running DOS can, by acquiring a range of interface packages and tools, become a very powerful business tool. The configuration of these facilities, however, is by no means straightforward. The Right Interface With power comes responsibility, and in this instance there is the issue of choosing an appropriate interface. Poor interface design is fairly evident when, for instance, a human uses a badly designed information system. When the interfaces
Page 21
are more deeply embedded and deal with computertocomputer interaction, problems tend to be less obvious. Sometimes they manifest themselves in poor performance, sometimes in the system hanging as two sides of an interface wait for the other to act. Occasionally, the interfaces that hold a system together can harbor quite dramatic latent traits. The U.S. telecommunications brownout in the 1980s was traced to a protocol problem. It was not that the protocol did not work, rather that it permitted a slim but real chance of deadlock—both parties waiting for the other to act. Given that it is a distributed network of computers that has altered beyond all recognition the productivity and effectiveness of our daily work, we should be aware of the frailties of interfaces as well as their capabilities. The variety of ways in which computer systems can be plugged together through interfaces provides something quite special—flexibility. With each of the components behind the interface created using programming languages, we can get computerbased systems to perform a virtually infinite range of functions. This ranges from the control of a washing machine to the flybywire system of the space shuttle or the automatic control of a nuclear power station. We are constrained only by available memory, processor power, and storage requirements. Given such diversity and potential complexity, it is not enough to describe what can be used. We need to consider how the interfaces are developed, and this means looking at design. The Design Problem The design of an interface needs to specify the allowable transactions that two communicating systems can undertake. To achieve this, a mapping must be defined between the datasets of each system and between the protocols used. At a trivial level, we can think of this in terms of the social protocols we engage in when we meet friends or other people—particularly in a formal setting. With computers, the designer needs to be more precise and take a more formal or rigorous approach. Computers do not have the inbuilt ability that humans have to deal with ambiguity and redundancy, and so the design has to be assured for all states, sequences, and configurations. We have already mentioned the potential for deadlock and the complexities of concurrent systems—the moment of truth is when the components that make up the whole system are integrated. All too often, poor requirements, specification, and design, with one design team being not quite sure what remote components are doing, mean that the integration process is fraught with problems.
Page 22
The issue is exemplified by the scientist in Carl Sagan's science fiction novel Contact, in which an alien intelligence sends a message to Earth explaining how to build a machine to travel and meet them. The message is meaningless until a primer is discovered which acts as a metalanguage (more on this later). Eventually an interpreter arrives who can understand and translate between both languages. So the designer's job is to establish how the meaningful and accurate dialogue between components is achieved. In the absence of any standard approach, this is no mean feat. With interfaces being developed on an ad hoc basis, it is perhaps not surprising that the global network that has evolved is a hugely complex beast. And it is getting more and more complex every day. Interesting as complexity is from a general systems theory point of view, from the viewpoint of the engineer it creates many problems. A systematic approach to interface design would help with at least a part of the problem. The Interface Equation A good example of a tool that supports a systematic and generic approach to interfacing is the interface equation. This is a mathematically formal means of developing and generating system interfaces. The essential idea is to describe each system (which we assume to be different) in a formal and unambiguous notation and to operate on these two specifications via an appropriate algorithm. This algorithm produces, and subsequently generates, the software for a suitable interface that (when combined with the original two components) provides transparent communication. The initial development of the interface equation was motivated by the need to interface newly developed telecommunications services to the existing switched telephone network. Clearly, the approach developed could be generalized to include not only other protocolrelated problems, but also more abstract system interface problems (in fact, any interaction between components that operate with a constrained sequence of states, inputs, and outputs). Essentially, it is just a formula, but one where the variables are not numbers but systems or machines as they are usually called. The problem is, given two machines that wish to talk to each other, is it possible to derive an interface machine automatically from the descriptions of the given machines? The answer is yes, and while the basic idea is simple enough, the mathematics are actually quite complex. (More details will be in Chapter 9 and in Appendix A.) Nevertheless, computer programs have been developed which completely automate the process.
Page 23
The interface equation (below) works by first specifying the input machines, P and Q, using a special language called CCS (Calculus for Communicating Systems). This language provides all the necessary syntax and semantics to capture the information to describe a general purpose systems component (in terms of its states and transitions). A number of algorithms have been developed which take as their input these CCS specifications and produce, as output, a CCS specification of the interface—in this case X—required to make a whole set of components work as an integrated system.
The set A refers to the set of actions via which the two machines (P and X) will communicate. The effect of all this is that the righthand side of the equation, (P|X) \{A}, represents a machine that from an observer's point of view is indistinguishable from Q. Thus Q is duped into thinking that P speaks the same language. Note that P and X are composed in parallel. The simplest way to understand this is to return to Mr. Lee and Mrs. Smith, where Chinese can be considered as protocol P and English as protocol Q. We'll assume that the set {A} contains Chinese words for which there is no English translation (hence we want to hide them). The machine X acts as a translator. But while a human translator will tend to operate in batch mode, in this case the translator is spontaneous and operating in real time—Mr. Lee and the machine X are operating concurrently so that externally they appear to speak the perfect English that Mrs. Smith can understand in real time. Protocol conversion is precisely the same, except that here the languages in question are designed for computer communication. There are several algorithms for solving the interface equation. The simplest algorithm, called the discarding algorithm, is rather slow as it carries out exhaustive checks. More complex constructive algorithms based on a graphtheoretic approach are more efficient. All of them work by combining the given machines in a certain way and by generating a candidate solution from this composite machine. Later on, we will give an overview of all the main methods of solution and provide a glimpse as to what might happen in the future. The interface equation has been applied to a number of protocol conversion problems. The case studies in subsequent chapters will give some details of these and suggest how the work may be used to address largescale industrial problems. Certainly in the field of protocol conversion, there are numerous possibilities. Efforts to standardize protocols have largely failed; the tendency is for more conversion as opposed to convergence, but there are many other application domains.
Page 24
We also consider how techniques such as the interface equation can be applied to the general problem of system integration, possible future developments whereby system integration becomes much more formal and largely automated. Summary As the technology wave gathers strength, system complexity rises, and our dependency on networked computers increases, issues of integration of separate parts are becoming more and more important. In this chapter we have examined these issues with specific reference to the choice of appropriate interfaces and the way in which they are designed. The languages, or protocols, used by computer networks are the fundamental means by which computers talk to each other. If we take these as the components that we are given to design with, then it is their interfacing that poses the key part of the designer's task. Given the criticality of sound interface design, we outline one possible approach, known as the interface equation, to formalizing (and hence securing) the process. In practice, the diversity of interfaces and the problem of integrating subsystems is vast. It is not possible to put an end to all the system engineer's ills within the confines of one book, nor with a few simple examples. But we have started to provide some insight and potential solutions to everyday system interfacing and have developed an outline of a more systematic approach to building computerbased networks. We go into this in more detail in the next chapter. Selected Bibliography Held, G., Understanding Data Communications, New York: SAMS, 1996. Freeman, R., Fundamentals of Telecommunications, New York: John Wiley & Sons, 1999. Norris, M., and N. Winton, Energise the Network: Distributed Computing Explained, London: Addison Wesley Longman, 1996.
Page 25
3— Components Today we have naming of parts. Henry Reed
Once you have a reliable interfacing mechanism, few would doubt the sense in using it to build things from a recognizable set of components. In some areas, it is accepted practice. Your car, for instance, can take wheels from many different manufacturers, its bodyshell can accommodate one of a number of engines, and you have your choice of car radios. This level of standardization is not confined to the car industry—many others are just as mature in the choice they offer the consumer and the opportunity they offer the supplier. This level of maturity did not come about overnight and the history of componentbased manufacturing is a long one. As the western frontier of the United States was being pushed back in the late 1800s, settlers and the military created a heavy demand for guns with interchangeable parts. This called for precision mass production and led to a process known as the American system of manufacture. Before this, every gun was handmade, so every replacement part was handmade. Hence, the highly skilled craftsmen who supplied guns perpetuated their status. The rising demand for guns overstretched the ability of craft supply and drove the development of the American system. The car industry followed the same path some fifty years later. Information technology, which relies heavily on software, is in much the same state as the early gun industry. The reliance of many organizations on key individual programmers is testimony to this. Change is nigh, however, and
Page 26
there is a strong push for the software business to adopt a more regular way of working. Components play in important part in this. They constitute the essential building block for manufacture. In this chapter, we look at what is new about a component based approach, what components are, how they are used, how they should be used, and what the benefits will be. We suggest that components are not just object orientation, object request brokers, or Java Beans. These play their part (in much the same way that standard backplanes and plugs do in the world of computers), but there is more. Higher level functions can also be described, and in this chapter, we show the different types of components, their characteristics, and how they might be used. Components—So What Is New? Software engineering is probably unique in that every few years a new method or technique comes along which everyone hails as the "silver bullet" [1]—that thing that is suddenly going to change everything, solve all the problems, and make software engineering as easy as everyone thinks it should be. At best, the technique turns out to be a useful step forward; at worst it is forgotten within months. Is componentbased development (CBD) the latest silver bullet or will it genuinely transform the way software is constructed? Ideally, the answer to both questions should probably be no. Once in a while something radical does come along that creates a paradigm shift, but for the most part, new techniques tend to build on and strengthen what has gone before. Software engineers have a great tendency each time a new technique or method comes along to forget everything that was learned in previous years. For instance, the World Wide Web revolutionized the way people access information, but programming in HTML and CGI was actually a major step backwards in terms of programming technique. This is in stark contrast to other engineering disciplines, which build on the experience of previous generations. After all, this is what engineering is all about! At this point you may be tempted to ask: don't we already use components in software engineering? Of course we do—subroutines, class libraries, operating systems, databases, and DLLs are all examples of components used throughout software development. Don't we already use the concepts of components in software engineering methods? Again the answer is yes—functional decomposition, encapsulation, modular programming, and so forth are all important elements in current software engineering practice. Don't we already achieve software reuse? Well, up to a point. There are good examples of software reuse, particularly in product lines or in small teams,
Page 27
but reuse is still not as widespread as either hardpressed software development managers or impatient customers would like. So there is nothing radically new in a componentbased approach; however, despite the widespread use of these types of components and methods, there is still a vast amount of code written each time from scratch. Moreover, much of this code is very similar to that being written by other people for different applications, either in the same domain or for completely different domains. Various studies have been undertaken over time which suggest that as much as 40–60% of code could be reused between applications and that typically only 15% is probably unique to that application. In practice, however, few organizations achieve these levels. Code is probably not the most appropriate thing to reuse. Designs, specifications, and requirements can all be reused and may have a bigger impact, but even if this is correct, we shall see later that there are many other potentially good reasons for using components. There are good examples of the availability of commercial components in some areas, for instance Visual Basic Microsoft® libraries, but few examples of largescale components off the shelf (COTS) that can be bought in and assembled into software developments. Furthermore most of these techniques assume systems are being built from scratch. Little understanding exists in the industry about how to systematically break up large legacy systems into useful components that can be reassembled and then used in future developments. Client/serverbased legacy wrapping has gone some way towards this, but it is not a complete solution. The vision for a componentbased approach that enables one to assemble a new application or system almost entirely from preexisting components (inhouse, COTS, or both) has yet to be achieved. If we go further and consider systems development as opposed to software development, then the drive to a component approach is even stronger. Large companies may have many hundreds of complex interacting systems. The U.K. telecommunications operator, BT, for example, has about a thousand different operational support systems to look after product billing, intercompany accounts, and workforce management. When a new product or service is to be launched, it is necessary to put together a support solution from an appropriate combination of these systems and possibly new developments. Each of the systems in the corporation's legacy of systems is potentially a component. They are certainly reused many times over, but it is rarely the case that the available components can simply be assembled into the new solution. More often it is necessary to integrate these systems. What this normally comes down to is making changes to all of the systems to allow them to be stitched together, possibly with additional "software glue."
Page 28
A true componentbased approach to systems engineering would offer considerable benefit in reducing cost and improving time to market. In addition, most major companies are realizing that it will not be possible to implement the systems necessary to support complex global products and services without using a component approach. It simply will not be possible to build everything from scratch every time and achieve the complex levels of system interaction required. A particular difficulty is that the concept of a component (and CBD) means different things to different people. To some people they are objects (as in objectoriented programming), to others they are things produced using the ''TI Method" (a method produced by Texas Instruments Software, now part of Sterling Software). More recently there is a tendency to say, "CBD, that's Microsoft DCOM or CORBA or Enterprise Java Beans." All of these things play a part in CBD, but none of them, by themselves, will fully achieve the vision of componentbased assembly. The authors strongly believe that a more systematic approach to componentbased engineering is required that considers how variable components fit together across the enterprise. It is also vital to identify different approaches to using components because it is naive to believe that one approach will suit all types of applications. The goal is not just componentbased software development, but componentbased systems development. Why Use Components? We have already hinted at some of the benefits of using components. Many people believe that the only rationale for a component approach is to achieve software reuse. While this is an important consideration, it is not the only one. Typically the benefits of using components might include: • Time to market: Reuse of existing and boughtin components reduces the amount of new development. Partitioning of work into components allows effective parallel development. • Quality: Use of component approaches encourages good design practice: decomposition, encapsulation, abstraction, and better documentation. • Reliability: Good quality components will be more reliable. Reuse accumulates more extensive test coverage across a wider domain. • Cost: Reduced development effort, testing, and purchase of COTS can all be expected to reduce cost.
Page 29
• Maintenance: Reuse results in fewer defects, since encapsulation and decomposition aid understanding. Components can be readily replaced by functionally equivalent ones that are high quality, perform well, and are reliable. • Flexibility: Adding new functionality may be easier; components can be reassembled in different ways. Much the same list could have been drawn up to support the American system of gun manufacture. We can see that some of these benefits are related to reuse and others are not. Many of the most successful examples of using CBD have not had reuse as their primary goal, but have found reuse benefits at a later stage. Often projects which focus too hard on reuse are unsuccessful; it's almost as though you have to creep up on reuse without it seeing you. There are several different flavors of reuse. We have already mentioned the reuse of designs and so forth, but components can also be reused in different ways. The use of a component approach to design can enable components to be easily generated from highlevel system models using readily available code generators. It may, therefore, be quite attractive and effective to generate a new set of components for each new application. This is often termed generative reuse. Some of these components may be identical to those used before, but it will be the design or component model that has been reused rather than the physical component or its code. Of course, the use of components may have a downside as well. While reuse of components can improve quality and reliability, there is no guarantee that this will be so. It is perfectly possible to promote the reuse of rubbish. This is most likely to happen in complex domains where people may be pushed into reusing low quality, but available, components rather than commissioning a new, better design. The attraction of assembling a new system entirely out of existing components may lead to unacceptable compromises being made regarding the design or specification. As with any engineering domain, careful management is needed to get the right balance between meeting the functional requirements and producing a wellengineered design that meets the nonfunctional requirements such as cost, reliability, and timely delivery. A componentbased approach potentially offers the framework to undertake this engineering management. What Is a Component? We all have a concept of components from everyday life so it is somewhat surprising that there is so much debate within the software industry about the na
Page 30
ture of components. This has led to a proliferation of definitions. We believe that the main problem is that people are trying to come up with a single definition for what are in fact a wide variety of different sorts of components that will be used in different ways. We shall offer a definition of our own later on, but we believe what is more important is to identify the characteristics of components in general and, for any particular type of component, to understand which of those characteristics are important for it to exhibit. For the moment, let us think of a component as a selfcontained, recognizable thing that does something useful and well understood. It does it largely by itself, but can be combined with other components using known interfaces to build something more complex. Traditionally, when discussing the nature of components, we start by giving an example from another more familiar domain. Not wishing to break with tradition, we shall do the same and look at automobile assembly. Before you yawn and turn to the next chapter, we ask you to stick with us for we hope to take this further than the simplistic approach often used and to identify a range of different types of components and concepts that will be useful later when translating to examples in software and systems engineering. An Example: Components in Car Assembly The automotive industry is a fine example of component engineering. Ever since Henry Ford set up his first production line, the industry has embraced the idea of interchangeable parts. In Japan, customers choose the style of panels, fixtures, and fittings that they want to be assembled into their Toyota Cedric. Some typical components from which cars are built include: Engine
Gearbox
Seats
Dashboard
Alternator
Tires
Exhaust
Spark plugs
Fasteners
Fuel tank
Fuel pump
Chassis
Perhaps the most important component is the engine. Most of the main characteristics of the car are determined to a large extent by the engine, but not exclusively so, because we know that engines can be reused in different models and sometimes even in cars from different manufacturers. Most of us have a concept of what an engine is and if someone were to say, "I am going to buy a reconditioned engine," then we would know roughly what he or she meant. This, however, highlights our first problem in using components. Knowing roughly what someone means is not good enough. The amateur mechanic putting the reconditioned engine in his car will probably be
Page 31
prepared to go out and buy extra bits, file things down, and generally fiddle about to make it fit. This is in many ways similar to the current concept of systems integration that we mentioned earlier. To achieve the vision of being able to assemble our system (the car) from components without any extra work, we need a much more accurate definition of what constitutes the component and how it fits with other components (the interface). So what are the boundaries of what we call the engine? We said that a component must be selfcontained, but what does that mean? It is normally taken to mean that it must fill some recognizable function, by itself, without the need to add other components. That would suggest the engine should be capable of operating and turning the crankshaft under power. If we take that to the logical extreme, we would require a battery, electrics, starter motor, alternator, spark plugs, fuel tank, fuel pump, radiator, exhaust, water pump, and so forth. This has gone beyond the boundary of what we think of as the engine. We could have a debate over whether the alternator or a spark plug is part of the engine, but most people would agree that the fuel tank is not. Going to the other extreme, we probably would not think of the engine block as a component. It certainly is a key part, but it has no recognizable function by itself so clearly it needs the addition of other parts before it can become a component. Somewhere in between there must be a sensible definition of an engine component. To find that definition, we need to look at all the various parts and find out if there are meaningful interfaces between them so we can draw a boundary around a sensible subset. We can see that there are some key interfaces: fuel, electrical, water, exhaust, transmission, and mechanical. We can use these to redefine our engine component such that it provides a recognizable function, producing defined outputs provided that it is fed defined inputs over defined interfaces. This allows us to now eliminate water and fuel pumps, fuel tanks, and exhaust as being clearly the other side of the interface. We still have some work to do as we need to make a decision whether the electrical interface is at the battery voltage level or the spark level so we can decide if spark plugs and alternators are part of the engine or not. However, things are now much easier and largely a matter of an agreed definition. What we have in fact done is to conduct rudimentary componentbased design process. What it has shown is that there are no exact answers to what a component is. It is a tradeoff between having the component do something useful by itself (given the interface conditions are met) and keeping it to a sensible size so that it offers the prospect of reuse or replacement. By looking at this example further, we can see that it highlights other aspects of using components. We have decided that the fuel pump and tank are
Page 32
not parts of the engine, but are they components themselves? And what about the fuel system as a whole? The fuel tank does not really look like a component. It is selfcontained and has defined interface, but it does not really do very much so it would be more accurate to describe it as a part or maybe subcomponent. The fuel pump on the other hand does have a welldefined function and interfaces, so it is a good candidate for being a component. The combination of pump, tank, fuel lines, and so forth has a welldefined interface and a wellunderstood function so it seems to meet the definition of a component. However, intuitively it does not seem right to describe it as a component. The main reason for doubting its component status is that the fuel system is not assembled into a complete unit and then mounted into the car, rather each of the parts are assembled individually. It is a logical component rather than a physical one (we shall discuss logical and physical components again later). Similarly alternators and water pumps look like components while batteries and seats are more like parts. We have to be a bit careful here of introducing a prejudice for components that have some dynamic function (turning gears for instance). But it does highlight that defining a component is mainly about the degree of function it performs. The dashboard is a different sort of thing again. It is made up of bits; it will have some physical and possibly electrical interfaces, but it does not really have a selfcontained function. It differs from the fuel system in that it probably is assembled separately and then fitted into the car, yet it does not really meet the criteria for a component and would be better described as an assembly. So we have already identified several different elements that make up a car, some of which meet our definition of components and some of which do not. We have also seen that the notion that our components should have a selfcontained function without the need for other components is correct provided that we define a clear set of interfaces, the conditions for which are met. However, we can also see that for this model to work properly we need some conceptual framework which shows what components and parts we need to build a car and how they fit together. Some aspects of this are well understood, and frameworks such as U.S. Military Standard 498 explain how to configure components and control changes. There still remains, though, the issue of design. What Do We Design First—System or Components? We now have an idea that we could put together a car from appropriate parts, assemblies, and components, but how do we ensure that we can obtain (or make) these elements, that they will fit together when we do, and that the re
Page 33
sult will be the car we want (or even a car at all). The glib answer is that they will have been designed to fit together, but by whom? What do we design first—the car or the components? Some people have a vision of software engineering where systems are built purely by searching the market (internal or external to the company) for existing components and then assembling them into the system that is wanted, or even that this might be done at runtime using distributed components. Is this vision feasible? Let us look at two examples: our existing car assembly example and the assembly of a personal computer. Car Assembly Revisited The car industry is probably the most advanced at component assembly, but car manufacturers today do not design new cars based purely around existing components. Let us revisit the various components of the car and see where they come from and who has designed them. We have already said that the engine is the key component for the car, and it will be rare for a manufacturer just to go out and buy an engine off the shelf. Normally a new engine will be designed as part of a range of engines that will be used in several models within a particular range of cars or across several ranges. The engine will be designed with particular characteristics (performance, economy, size, and cost) that match the concept of the car being produced. Part of that concept will define whether the engine will work with standard commercially available components (alternators, fuel pumps, and so forth) or whether, say in the case of a high performance car, special components need to be designed and manufactured. This concept of a product line built from components designed and chosen to match the concept of the product is the key to success. In the software engineering domain, this is still in its very early stages (see later). So we can see that the concept of the car determines the specification of the key components, and the cars in the range are then designed based around those key components. If we then look at components like alternators, carburetors, or batteries, we can see that while they are bought in (offtheshelf), their performance is important in meeting the car's concept. When that concept is straightforward, then an offtheshelf component will be used. When the concept is more specialized, the component will have to be designed specifically for that car. More often, the car manufacturer will collaborate (or contract) with the component manufacturer to produce the appropriate component that (depending on the contract) may then also be offered to other purchasers. The specification of the component will have some effect on the design of the car as it will be necessary for the car to be designed so the component will fit. Moving further down the chain to components like spark plugs and tires, it is even more likely that commercial offtheshelf components will be used.
Page 34
Parts and assemblies such as seats, dashboards, and so forth will be designed specifically for the product line but are unlikely to be used elsewhere or offered offthe shelf. This introduces the concept of local reuse in contrast to global reuse. Finally, when we come down to basic parts such as fasteners (nuts and bolts, for instance), then these will almost certainly be offtheshelf and globally reused, and they will not affect the design of the car. We can drill down even further and look at sheet steel, plastic, glass, and so forth. These are examples of material that will be formed into parts and components. We can summarize the elements of the car and their relationship to its design and reuse value in Table 3.1. Technology, Environments, and Patterns There is a vast range of parts (simple components) available from manufacturers. They are usually sold in ranges, either manufactured using a particular technology or designed to work in particular environments (e.g., stainless steel for the food industry, special rugged parts for the military, and so forth) so that users do not have to search to find what they need. Often the technology used will be aimed at allowing components to work in a specific type of environment or, if chosen for other reasons, will determine the environment in which the components can work. The technology will usually determine how components will interface (e.g.,Whitworth or metric for fasteners, CMOS or TTL for electronics, DCOM or CORBA for software). While technology is usually applicable to lowlevel components, the concept of "environment" can also be applied to higher level components. The commercial vehicle market is a harsher environment than the domestic vehicle market, and components such as alternators will be chosen from a range designed for this environment. Military components will have a higher specification again. Thus we can see that we may have components that are functionally identical but operate in different environments with different nonfunctional characteristics. We can also take this a step further. A dieselengine car will be very similar to a gasolineengine car in the same product line. Many of the components will be the same, but obviously some key components will be different. The concept of a diesel engine will determine which components are different. If we look at gas and diesel engine cars in a product range from a different manufacturer, although the actual components chosen may not be the same, the general differences between gas and diesel will be the same. In fact, if we just compare the gasdriven cars in the ranges, even though the designs and components may be different, we will still be able to see a great deal of common structure. What we are seeing is a pattern in the way components are generally assembled into gas or diesel engine cars. We discuss patterns in more detail in Chapter 5.
Page 35 Table 3.1 The Elements of a Car, Their Relationships, and Reuse Potential
Type
Value
Source
Interface
Designed for Car
Effect on Car Design
Reuse Value
Item
Global Reuse
Engine
component
high
specialist
special
yes
large
high
low
Alternator
component
medium
supplier
understood
maybe
medium
medium
medium
Spark plug
part
low
off shelf
standard
no
small
low
high
Dashboard
assembly
medium
supplier
special
yes
medium
low
none
Seat
assembly
medium
supplier
special
probably
small
medium
none
Fuel system
logical
medium
—
conceptual
pattern
conceptual
none
Tire
part
low
off shelf
standard
no
none
low
high
Air filter
subcomponent
medium
off shelf
standard
no
small
high
high
Fasteners
part
low
off shelf
standard
no
none
low
very high
Sheet metal
material
none
raw mat.
none
—
none
none
high (raw)
Page 36
In our car example, we would probably consider the chassis to be an assembly of all the components and parts needed to make the engine run and drive the wheels in a controlled manner. We would not think of it as a component, probably would not reuse it (in its assembled form), and it certainly is not the completed car. In fact, many modern cars do not have a chassis. However, in the commercial vehicle market, the chassis has a very different status altogether. It is very common for vehicle manufacturers and commercial body makers to reuse a basic chassis to produce a wide range of different utility vehicles. For instance, an ambulance, open bed truck, and box van may all have the same basic chassis. The chassis in this case does look like a component, but it is much more than that. It is a partial product assembly that can be completed, extended, and customized to meet a customer's exact requirements but is based on a common core. This is an example of a framework, which will be discussed further in Chapter 5. Personal Computer Assembly Taking another example, the assembly of personal computers, we see a very different story. The industry is now very mature and has clearly defined production layers. The chip manufacturing industry is very specialized, producing sophisticated very large scale integrated (VLSI) circuit components (e.g., microprocessors, largescale memory devices). Their designs are very complex; the plant needed to produce them is very specialized and the production quantities needed to be commercially viable is very high. In the early days of chip production, the designed function of the devices was fairly generic, and the components were used for a wide range of functions. Increasingly, however, the intended use is becoming more and more specialized. The second layer of component production corresponds to the physically larger things that make up computers (motherboards, display controllers, CDROMS, for example). Interestingly, these larger components, from whichever manufacturer, use much the same lower level components. The difference between them is often a very subtle difference in performance, price, and functionality. Items like motherboards and display controllers are mostly produced by a few specialist companies who supply the rest of the industry. Items like CDROMS are more often produced by very large multinational household names in the electronics industry. The most interesting area is the assembly of the PCs themselves. PCs are generally marketed by two types of companies: the household names in the computer industry (for instance, Compaq, Dell, or Dan) and a wide range of smaller companies. The specification of a PC is basically the same for all. There are a variety of options, including size of memory, number of hard disks, and multimedia facilities, but the underlying specification is the same.
Page 37
We have already seen that the components from which they are built are also much the same, so what is the differentiator? It is actually surprising given the commonality of the system that there is still differentiation (albeit diminishing) on price, function, and cost. It is in fact purely the choice of which midrange components to assemble together, the quality with which this assembly is done and tested, and finally, the aftersales support provided. Although apparently similar, some sets of these components worked better together than others simply because the interface standards were not that exact. So in the early days of PCs, it was usually safer to buy from the big name suppliers as they often commissioned their own midrange components and had better assembly facilities. However, increasingly, good quality PCs can be bought from almost any supplier. This reflects that suppliers are generally assembling from the same range of components and the quality of these components is improving, the specifications becoming tighter, and the interfaces more standard. This just leaves commercial considerations to be the true differentiator. So we can see that almost anybody can set himself or herself up as a PC manufacturer. They can produce a very basic design and simply look to the market to provide all the components needed. A very different story from car assembly, which is driven more by style and marketing, PCs are chosen more on the basis of function and price/performance. Will the PC assembly model work for software components and achieve the vision that we set out earlier? We will discuss this in more detail later, but it is worth noting at this stage that the main difference between software assembly and PC assembly is that PC assembly is a very tight, wellunderstood domain. There is very little difference in specification, and items are churned out in large numbers. Software, however, is a very wide domain, specifications are complex and rich, and frequently the products are oneoff. Bits and Pieces: What Have We Learned so Far Let us summarize what we have learned about components so far. Although we will try to give some specific definitions, we ask readers to look at the general concepts, rather than concentrate too much on the precise definitions. The differences between components, subcomponents, and assemblies is often indistinct, and it always possible to think up exceptions. The different entities we have discovered are: Bits • Material: The raw material from which parts and components are made. Closely related to the chosen component technology.
Page 38
• Parts: Items (usually boughtin) that are assembled into components, subcomponent assemblies, or systems. They have limited function and low added value. They may be considered as very simple, lowlevel components. • Subcomponents: Parts that have significant function and added value. They meet the broad definition of a component, but in practice are always used tightly coupled with other subcomponents and parts (following a pattern) to make a component. • Components: Selfcontained, recognizable entities that perform a wellunderstood function and can be assembled via known interfaces with other components to build something more complex. Are often reused and can be replaced with an identical functioning component without affecting overall operation. • Assemblies: Parts and components fitted together to make a convenient unit that can be assembled into a larger entity. An assembly differs from a component in that it does not have a welldefined, selfcontained function, although it may be reused elsewhere. Structures • Logical components: Conceptual decompositions of a design or architecture that may or may not exist as physical components. Where they do not exist as physical components, they can normally be decomposed into subcomponents that do exist. • Patterns: Recurring structures in the use of components. Patterns may be specific to a domain or more widespread. • Framework: A partial product assembly that can be completed, extended, and customized to meet a customer's exact requirements but is based on a common core. • Design: A detailed description of a system showing how it should be assembled from parts, assemblies, and components. • Product line: A series of designs showing how a family of strongly related products are to be built from common components. • Architecture: A highlevel conceptual representation showing how systems and components in a domain relate to one another. Any given domain may have a number of different architectures representing different viewpoints. • Domain: A wellunderstood area of common interest.
Page 39
• Environment: The set of conditions in which a system or component is designed to operate. Selecting an environment will often determine the range of available components that can be used. • Technology: A particular method or material used to manufacture parts and components which determines the detail of how they are assembled. Technology may be determined by, or determine, the environment in which components are to be used. Usage • Local reuse: Reuse of components within a product, product line, or by a small team in several products. People working closely together will have implicit knowledge of how to use the component. • Domain reuse: Systematic reuse of wellunderstood common components across a specific area of interest, often in specific environments. People working in the domain will have implicit knowledge of how to use the component. • Global reuse: Widespread reuse across domains, organizations, environments, and geography. All the knowledge needed to use the component has to be made explicit. Views of Components We can see from the preceding sections that we have different ways of viewing component organization. Figure 3.1 shows a hierarchical layered model of components, subcomponents, parts, assemblies, and so forth. We also have structured groupings: product lines, environments, frameworks, and patterns. Whatever view is taken, it is the consistency of approach that matters. A large part of adopting a practical componentbased engineering approach is commonality of approach (just as it is with any software method). Hence the key point is to decide which view suits best, socialize it, and stick with it. Software Components: At Last! The big difference between software and the other domains we have discussed is the richness of the function provided. This is the very attraction of designing software, but it is the thing that makes it different and conceptually more difficult than other engineering disciplines. We have spent quite a lot of time exploring the use of components in other domains and trying to tease out their
Page 40
Figure 3.1 Hierarchy and structure of components.
Page 41
nature and how they are used. We believe this was worthwhile because it has enabled us to describe in a straightforward way most of the key concepts we will need in discussing software components. At this point, let us revisit our earlier definition of a component and break down its essential characteristics in a bit more detail. Putting it a bit more formally, we can say that a component is a selfcontained, clearly identifiable, physically realizable, predefined entity that provides a wellunderstood function and is intended to be assembled, using a welldefined architecture and interfaces, with other components to provide more complex functions. A component is not so much a specific thing (i.e., something with a specific size and shape) but more a label that can be applied to types of things that meet certain criteria about how they can be used. Thus, to qualify as a component an entity must have the following attributes: 1. Selfcontained: The component must deliver its function in its entirety without the need to access other components or parts. That is, of course, provided that the right interface conditions exist. It will almost certainly be necessary to use other components to provide those interface conditions, but the component should be indifferent to this. If it is a condition that a component cannot operate without an association with another specific component then it is the combination of the two things that is really the true component. The use of a framework will provide a set of components that are intended to work together, but nevertheless the individual components should still meet the criteria. Allowing components to access global data without the use of a welldefined interface tends to compromise their encapsulation. 2. Clearly identifiable: The component's function should be clearly identifiable and well understood within the domain for which it is intended. Its specification should be as abstract as possible (to ensure the widest applicability), yet precise. The temptation to pack too many functions into the component (to make it even more useful) should be avoided. Nevertheless the component should do something significant and useful. Small software objects meet many of these listed requirements, but cannot be considered to do something significant by themselves. 3. Encapsulated: The component's function should be completely independent from its implementation, both in terms of design and technology. Its function is accessed only through its predefined interface. It may be comprised of lower level subcomponents or modules, provided they are encapsulated within the component and are delivered
Page 42
with it. In actual practice, a particular component may not be technology independent because of internal reliance on services delivered by an underlying technology (e.g., CORBA, or DCOM). However, provided that the component is used in the environment for which it was intended, then the component can be used without regard to its technical implementation. Usually when obtaining a commercial component, the user will chose the actual component based on its functional specification and then when placing the final order will choose the technology in which it is to be delivered (e.g., ActiveX or Java). 4. Physically realizable: The component must be capable of being used ''in its own right" and being assembled with other components. This is distinct from a logical component. While logical components are useful for design purposes, we believe that some architects place too much emphasis on logical entities and their mapping to systems. We believe that, to be useful, a logical component must be mapped, not just to physical systems, but to one or more physical components before it can be used in an implementation. Thus a component must be one of the following: • Physically in existence: It must be physically in existence and be available for assembly into a larger entity (e.g., a software module or class library in a repository). • Automatically generated: It can be automatically generated, on demand, from a preexisting definition or design and assembled into a larger assembly. • Preinstantiated: It has been preinstantiated within an operating system or network and can be accessed at runtime; thus, it is capable of assembly into a network of communicating components (e.g., a predefined server function in a 3tier clientserver network). • Instantiated ondemand: It can be instantiated ondemand and accessed within an operating system or network; thus, it is capable of assembly into a network of communicating components (e.g., a predefined distributed component instantiated by a component broker or possibly a software agent). • Embedded function: It is a function embedded within a larger system that can be accessed via a predefined specific interface (a service) without knowledge of or knowingly interacting within the rest of the system in which it is embedded.
Page 43
• Logical representation: It is a logical representation (model) of a physical component that when enacted will cause the actual component to be activated (e.g., a process component enacted by a workflow system). 5. Predefined: The function and interface of the component have been defined prior to use and fit within the architectural and functional definitions of the domain. Either the specific component must be defined or a generic component within whose definition a range of variants can be created. 6. Assembly: The component can be assembled into a larger structure of components either creating a single larger entity (a software application built of components) or a network of communication components (e.g., DCOM, or CORBA). 7. Replaceable: The component can be replaced by an alternative component with an equivalent specification (function and interface) without any effect on entities making use of the component. Depending on exactly how the component is used, the alternative component may, or may not, need to be of the same technology. 8. Architecture: The component must have been defined to be assembled within a known architecture or structure. It is not good enough to think up useful components and then try and fit them together to make something. This can sometimes work and be valuable, but, as we have seen from the car industry example, effective and systematic reuse comes out of careful planning and design. We shall discuss what is meant by architecture and structure in more detail in Chapter 5. 9. Interface: The component must have been defined to make use of a specific interface. The term interface is used in different ways when talking of components. In our definition, the interface is the mechanism and protocols by which two components communicate with each other within the defined architecture (equivalent to layers 1x of the 7layer ISO model). However, many people talk of the component interface as being the complete definition of what a component does (layers 1 to y) to a level of detail necessary to be able to use the component. An extension of this is the idea of a component being selfdescribing and being able to communicate a description of what it does to other components. While this makes sense, particularly when using a notation like UML to describe the component, we prefer the former usage as it sounds more natural. Moreover, in practice there will also be nonfunctional things, performance, quality, tech
Page 44
nology, and so forth, which are key to using the component and will not be described by the interface (see Glass Box components). 10. Status: The component must have an owner and a defined status. This is the only way to guarantee quality and reliability. This is of major importance when buying commercial COTS and leads to ideas about the certification of components to known and agreed standards. 11. Reusable: Although we have said that reuse is not the only reason for using components, generally the component should have the potential of reuse in at least one of the following ways: • Compositional reuse: Reuse of unmodified components (Black Box). • Configuration: Reuse of an unmodified component, but whose exact function is specified by data (Black Box). • Generative reuse: Reuse of generation process rather than particular components (White Box). • Adaptive reuse: Reuse by modifying existing component (White Box). • Integrated: Existing components used as a starting point for extensive modification and integration using newly developed interfaces (White Box). Generally, examples of reuse higher in the list are preferred over those lower down. Generic Types of Software Components We now have a set of criteria which enable us to identify types of things that look like components. Figure 3.2 shows broad classes of generic components. Black Box Components A Black Box component is specified entirely in terms of its function and interface. The user has no knowledge of its internal structure or method of implementation, and hence all the knowledge needed to use it must be made explicit by the definition of the component itself and the environment in which it is to be used. The component is used exactly in the form in which it is provided. This is thought of as being the purest form of component and the most likely form in which commercial components will be provided. Most software applications are components of this type, although the degree to which many ap
Page 45
Figure 3.2 Component transparency.
plications are really components is debatable. The provider is responsible for the maintenance and quality of the component. White Box Components White Box components are provided with all the source codes so that all details of the structure and implementation of the component can be seen by the user. The component can be modified and adapted to suit the exact needs of the user. This is probably the most widespread type of reuse which operates in an unstructured and ad hoc way. If the user does make modifications, he or she is entirely responsible for the quality and maintenance of the component. Using this type of component can yield significant benefits in reducing the time to market, but may not deliver benefits in terms of quality, cost, reliability, and maintenance. In particular, maintenance problems may be made worse if there is no traceability to a reliable source or documentation. Variants can proliferate and become unmanageable. Glass Box Components A Glass Box is a White Box component which is used unmodified. Using a component this way can bring most of the benefits of using Black Box components without the disadvantages. One of the most significant barriers to component reuse, particularly global reuse, is that all of the knowledge needed to
Page 46
make successful use of the component is not actually made explicit in the interface definition. In reality, there is a whole host of hidden assumptions (particularly nonfunctional issues such as performance) that the designer will not have documented. This may be due to lack of time (or will), lack of an appropriate language or format to express them in, or simply lack of awareness due to overfamiliarity with the domain. By examining the internal structure and implementation the user can glean additional knowledge that was not made explicit in the formal definition. This can be a very time consuming way of obtaining this extra information, and there may still be much information that is inaccessible from implementation level detail. There is a danger that the user will make too much use of implementation knowledge and use the component in such a way that prevents replacement of the component by another that is functionally isomorphic at the interface level. There is also no guarantee from the supplier that future versions of the component will be implemented in exactly the same way. Gray Box Components A Gray Box component is a White Box component with only minor modifications. This still, of course, has many of the disadvantages of White Box reuse. In particular, it can lead to many variants of a component with the inherent configuration management problems. A more attractive form of reuse with modification is one where the component's function can be configured or datadriven. Parameters that determine functional aspects of the component are loaded into database tables or passed to the component through an API. This allows a large degree of flexibility, but maintains the integrity of the component (provided the parameters are kept within given bounds). Many large applications can be used as components in this way. Of course, if the component allows too great a degree of customization, say by complex scripting languages, then its integrity as a Black Box component becomes compromised and it starts to behave more like a White Box component. We will discuss these types of components further when considering spreadsheets and databases (see below). In objectoriented (OO) environments, the function of a component may be extended or changed by overloading. It is important to note, however, that techniques such as this do not change the actual operation of the component, but wrap the component with an object layer that hides or modifies its function. Effectively, the component has become a subcomponent in a larger entity which
Page 47
is now the true component. This can be a useful technique, but needs care to ensure a clear understanding of where the true component boundaries lie. Examples of Specific Types of Software Components Let us identify the more familiar artifacts from software engineering, pick a few, and see if we can start to make some sense of them. Operating systems
Compilers
Languages
Algorithms
Subroutines
Functions
Class libraries
Objects
Methods
Spreadsheets
Databases
CASE tools
Procedure calls
Applications
Class frameworks
DLLs
Systems
Plugin drivers
Procedures, Subroutines, Objects, and Class Libraries Structured decomposition techniques have been a key part of good software engineering discipline for some time. Good programmers naturally structure their code into functional blocks that are easily understood and frequently reused. The extent to which these lowlevel code entities can be considered components depends on the extent to which they meet the criteria we have introduced. Perhaps the first good example of reusable software components was the libraries of FORTRAN mathematical subroutines that were widely used in the early days of scientific computing. Generally, procedures and subroutines tend not to be good examples of components. The functions they deliver are typically low in added value and often exhibit a high degree of interconnection (i.e., low encapsulation) with others of their type. Objects and Class Libraries On the face of it, the use of OO techniques would suggest a move to a more componentoriented approach. Strong concepts of encapsulation and interfaces (i.e., methods) seem ready made for components. However, while objects may well be good candidates for components, by no means are they automatically components. Some people become very concerned about whether it is the class or the object that is the component. Others freely interchange the terms component and object. Our definition above favors the object being a component, because it is the physical instantiation (must physically exist), but admits the possibility of the class being componentlike in that the object can be generated from the class.
Page 48
In practice, we do not believe that many smallscale software objects are in fact components at all. Typically objects provide very lowlevel functions that are not clearly recognizable; they have a high degree of interconnection with other components and hence are not truly selfcontained. The use of key OO techniques such as inheritance and polymorphism can also compromise a component approach. On one hand the ability to be able to subclass and produce a new, but related piece of function is very useful for promoting reuse, but on the other hand can lead to complex interconnected structure. In practice what tends to happen in OO is not that individual objects are reused, but that whole class hierarchies are reused. This led to a strong commercial market in class libraries (particularly GUI libraries), which while good for software reuse could not be considered as components. The use of OO techniques is continuing to mature and the use of object frameworks (OF) and business components (see below) is higher value OO component structures. Also interesting is the way that the OO technology architectures (CORBA, DCOM, and Java Beans) are repositioning themselves as component architectures. We shall discuss this in more detail later in this chapter. Operating Systems Operating systems are probably the most highly reused elements in software engineering. Although there has been a proliferation of operating systems over the years, particularly coming out of the research arena, there are no more than a handful in widespread use (for instance, UNIX, DOS, VMS, Windows, and VME). They meet all the basic requirements for a component (selfcontained, wellunderstood function, very clearly defined interface). However, operating systems are not very interesting when just used by themselves; they only come to life when applications are run on top of them. In this respect they are much more like frameworks (the commercial vehicle chassis). They are in fact complete, but are extended by the applications run on them. Databases and Spreadsheets These are familiar examples of a wider class of software components that are intended to be configured and extended by the users. Larger scale commercial examples include payroll, trouble ticketing, and callcenter systems. They are clearly components, but "freshoutofthebox" they will not support specific business needs. They are generally not frameworks because they are complete, and although they may be extended with the addition of other components, they will normally work entirely by themselves.
Page 49
They are made to do something useful by the user configuring or programming them to work in a specific way that meets their requirements. The programming is not done at a fundamental code level, but by using macros and scripting languages. In some examples, the tools will provide a wide range of option menus that virtually hide all of the macros and scripting, while other tools may provide a very sophisticated language that requires more conventional programming skills. Tools will often provide a variety of levels of sophistication that can be matched to the user's abilities and complexity of requirements. Vendors and thirdparty suppliers will often provide preconfigured components aimed at a particular domain, or provide consultancy and take on contract work to develop specific customer applications. The interesting aspect of these components, particularly for system integration, is that the nature of the component changes during its life cycle. Initially they are COTS Black Box components with medium added value and global reuse. It could easily be replaced by a functional equivalent component from another vendor and its quality is entirely dependent on the supplier. Let us assume the component is a database and that users, vendors, or third parties configure or customize the component for a particular application, let us say a fault reporting system. It is now a domainspecific component. It may be made available as an offtheshelf component and could be provided as a Black, White, or more likely Glass Box component. It will no longer have global reuse, but it would certainly be capable of significant reuse in a large corporation and, if made available, may have domain reuse potential. It can still be replaced by a functional equivalent component, but the range of available components is likely to be much more limited. Quality and reliability is now dependent on the vendor and the people who customized it. Let us take things a step further to the point where the system is now operational and holds customer and product fault data. The system is now a corporate asset. It will almost certainly be a component in a much larger operational support system (OSS) architecture. It may exchange information with other corporate systems and access may be available to users over a corporate intranet. It is now a unique online corporate component. It has very high added value, high reuse within the corporation, but zero reuse outside, although increasingly we are seeing corporations opening up these types of assets to collaborators, suppliers, and customers. It would now require major effort to replace, and its quality and reliability is dependent on the vendors, those who customized it, and those who operate it (maintain the data). If the component has been kept true to form as a component, then although it will need work to replace it with a functional equivalent, it will be straightforward. If the component concept has not been kept clean, then it will
Page 50
be very difficult to replace and will have become a "legacy system." What makes a legacy system is the addition of functionality that is not clearly bounded and not readily available from other components. Plugable Components In the car industry example given earlier, we discussed whether items such as spark plugs were components, subcomponents, or just parts. Things like this meet most of the criteria for being a component but generally have low added value and are not really very interesting. We can find similar examples in software. Peripheral drivers (e.g., printers, screen, audio, and so forth) are important elements in personal component systems—self contained, replaceable, and frequently supplied by equipment vendors and third parties. Although selfcontained, they are never intended to be assembled with other components, but rather plugged into an existing operating system or component framework. Slightly higher in the value chain are plugins for Internet browsers, which can add significant extra functions to the basic browser. So we seem to have a new type of subcomponent—the plugin. If we look where software has been developed to support very specific domains, we can find plugin components adding much higher value. Computer graphics is a good example where a strong and specialized plugin market has grown up around products such as Autocad , Adobe Photoshop and Adobe Premier . The originally supplied products are complete and fully operational in themselves, but by developing against the supplied open interface, third party developers can provide plugin software that significantly enhances the products or provides support for niche markets. Thus vendors have guaranteed a wider penetration for their product than they could achieve with their own development capability. At the technology level, the interfaces will be to a known standard, typically Microsoft DLLs, VBX, or more recently ActiveX, Java, or CORBA. At the application level, the interfaces are to a propriety standard either specified by the product vendor or developed as a de facto standard by interested parties in the domain. The user of the plugin needs to know nothing of how to assemble the plugin with other components, but simply follows installation instructions and then uses the new functions provided. Another interesting example is Microsoft Visual Basic . Initially released as a rapid application development environment, a whole range of plugin components were quickly developed by third party suppliers using DLL and VBX interfaces. Again these components were not intended to be assembled with other components into an application, but plugged into the Visual Basic
Page 51
(VB) framework (possibly with other components) to be then delivered as complete working applications. What is interesting is the speed with which these libraries of components became available and the extent to which they were successfully used. Many of the barriers to reuse previously encountered were swept away. Developers were no longer expected to assemble complex components (typically C or C++ class libraries) to do the most simple tasks. Simply installing these plugin components to the VB framework and writing relatively straightforward code gave access to a whole range of useful screen and data input widgets, graph drawing, database access, and even video and audio manipulation. The components were used and reused because they were simple to use (much more simple than writing your own) and did really useful things. This had a very significant effect on many large organization's data processing departments. Suddenly they found they had lost their traditional control of development activities. Virtually anyone in an office with a PC could buy a relatively inexpensive package and suddenly start writing applications. Moreover, the applications they wrote were often more visually appealing and appeared to be more oriented to what the users wanted. Worse was to come when terminal emulation plugin components with scripting interfaces allowed PC applications to pretend they were normal users accessing the corporate mainframes. These screenscraping applications (for a time at least) broke the back of the DP's power by allowing anyone access to previously closely guarded corporate data. Things have progressed since those days, but the example shows the significant effect that a component type approach can have. The rise of the Internet and in particular intranets (closed user group Internets for corporations) and extranets (a set of linked intranets, forming a closed user community) has carried on these trends and opens up new horizons for the way complex applications are assembled. There are no hard and fast rules as to what defines a domain. Typically they grow up in an ad hoc manner, often around very specialized but functionally rich areas. There is usually a wellunderstood problem being addressed, and everyone talks the same language. The functional richness provides the potential to enhance the base products. Typically the products are at the more expensive end of the market. There is a good deal of academic research interest in domainspecific programming, aimed toward a goal of providing languages and architectures which will allow users (the experts in the domain) to write their own applications rather than having to provide specifications to developers with the inevitable loss in the translation. We believe the whole area of domains, plugin components, and frameworks (which we will discuss in Chapter 5) offers significant potential for practical component reuse.
Page 52
Logical and Physical Components Several times so far we have mentioned the concept of logical components. By this we mean the conceptual decompositions of a design or architecture into components that may or may not exist as physical components. This concept often crops up when dealing with legacy systems as an attempt to try and visualize the legacy decomposed into useful components that fit into a more general architecture. For instance, it is common to split an architecture into presentation layer, business rules, and data storage. A legacy system can also be viewed this way especially if the system has a middleware layer that can be independently programmed to give different methods of access to underlying transactions and data. However, to implement the architecture it is necessary to map the logical view of the world onto a physical realization. This is often done using a data model to map the physical systems that manipulate data entities to the logical components that ''own" that data. This is a very useful technique and can be used effectively to produce system evolution plans that cluster related data manipulations into common systems and thus reduce the overall number of systems in a large organization. Unfortunately, although such techniques recognize components at the logical level, they do not necessarily tend to lead to useful components at the physical level. In fact, they can easily create more monolithic systems. What is needed is evolution plans that look for common function and seek, over time, to break large systems down into functional blocks that can be reused. Thus, the goal is to reduce the number of systems not by having fewer larger ones, but by having true systems that are comprised of different assemblies of a number of common components. Data is still a vital element which must be accessed through data services, collections of which are delivered by key components. While logical component modeling has its uses, we believe it is vital for the implementation of successful component techniques to understand early in the design process how logical components will be physically implemented and that longterm evolution planning should concentrate on the physical view of the world. Why Aren't Software Components the Same As Hardware Components? Many of the more mature software engineers came into the discipline from an electronic engineering background. We fondly remember the early days of Texas Instruments' 7400 series of TTL basic logic functions. This was the archetypal set of electronic components and started the electronics revolution
Page 53
which led to the microprocessor and the advanced state of electronics we see today. Every young aspiring engineer longed for the day when he or she would have his or her own personal copy of the TTL data book and eagerly devoured it looking for components that would enable some new and exciting piece of electronic wizardry to be created. It is therefore understandable to ask, can components be used in software, just like they were in hardware? The answer is yes and no and, although there has been much debate over this question, with many feeling it is a somewhat pointless comparison, we feel it can be quite illuminating to consider it further. We have already discussed Microsoft Visual Basic DLL and VBX components. These are analogies to the TTL logic components: a basic set of building blocks that share a common technology and a common set of basic interfaces, and operate in a fairly wellunderstood domain. TTL was slightly different in that there was not the equivalent of the VB framework into which the components fitted, but nevertheless the level of complexity is roughly comparable. TTL components could be assembled in a hierarchical manner. Many of the more complex TTL components could be built up from the more basic ones, but the use of large scale integration (LSI) just made it more convenient (from a size and performance point of view) to use the provided components rather than build them up from scratch. Going beyond this complexity, it was up to the designer to decide how to combine the components into still more complex assemblies. At this level there was no real sense of an architecture, but what is interesting is that more complex assemblies were often repeating sets of basic functions. For instance, look inside a microprocessor, large memory, or even video processing chip, and you will see blocks of repeated elements (typically basic registers and logic gates). Microprocessors appear complex because of the vast array of transistors integrated into a small space and the need to handle wide address and databases, but the actual function of the device is not all that complex. We can see a similar effect if we look at a large modern passenger aircraft. A Boeing 757 will contain millions of parts and thousands of subassemblies and components. Despite this, the function of the various components (if we exclude the avionics software) will be fairly straightforward to understand. While no one person will know the detail of every part, it is not too difficult for anyone to conceptually understand how all the parts fit together to provide the function of an aircraft. We can now start to see the difference with software engineering. While we have the same sort of basic building blocks, they do not tend to get used in the replicated way of hardware, but are assembled into increasingly more functionally rich subsystems. These subsystems are frequently highly interconnected and much of the function of software systems comes from the interaction between the subsystems rather than their individual operation.
Page 54
There is a longstanding joke in the aviation industry which describes any particular aircraft being denigrated at the time as being "50,000 parts flying in loose formation." There is a slight element of truth in this that clearly does not apply to software. For many large software systems, there is often no one who understands the concept of the complete system—knows all of the parts and how they fit together. We can simply say that the difference between hardware and software is that complex hardware is based on numerical (physical) complexity. It is functionally relatively straightforward and breaks down to understandable domains. Software on the other hand is functionally rich, has complex interactions, and often no one understands the whole thing. The challenge of using a component approach is to create components that operate in known domains and are loosely coupled so that the function of the systems is more in the components themselves and less in the interaction. A Layered Model of Components So far we have seen the different types and granularity of components and how they fit together in a hierarchical way (parts, subcomponents, components, and so forth). We have seen the different generic types (Black Box, White Box, Glass Box) and how they may be used. We have looked at familiar software entities and how they measure up to being components, and we have had an initial look at the structures used to bind components together (assemblies, patterns, and frameworks). The final piece of the jigsaw is to look at a business enterprise and what types of components we may find there. Just as trying to define a component reveals a whole raft of opinions, the same is true of layered models of components. Figure 3.3 shows one possible model of components used in an enterprise and those which could potentially be bought as COTS. The ascending scale roughly represents the perceived importance of those things to the enterprise. We say perceived because on one hand it is vital that the structure is underpinned by a solid component technology architecture, midtier components, and so forth. But, on the other hand, the business does not care about these things. It is the processes and business rules that matter. Most people are pretty clear what goes at the bottom, but once we start to move up to the business layers, it becomes a lot more difficult. On one hand, business process models can be seen as the very top layer because they drive the whole business. On the other hand, enactment of processes through workflow and business rules can be seen as the logic that binds midtier system components together. Of course, both views are correct and, as we shall see in Chapter 5, architecture is about viewpoints that relate to the context and area of interest at the time.
Page 55
Figure 3.3 Component model of a business enterprise.
We urge readers to make what use of layered models seems sensible, but not to spend too much time debating definitions. Let us start at the bottom and work up. Component Technologies The concept of using component approaches in software development has rapidly gathered pace leading to techniques originally intended for objectoriented development being expanded to become component techniques. Thus Microsoft's COM is now a Component Object Model, CORBA is taking on board business components and, more recently, Sun's Java Beans has spawned Enterprise Java Beans (EJB). Essentially these three are examples of component technologies or component architectures at the technology level. They represent a vital step in providing a technology infrastructure that will ensure that code or objectlevel components can reliably interface with one another, particularly across networks. In many ways they provide a level of infrastructure equivalent to the electronic interfaces of the TTL logic components discussed earlier. In much the same way, while they guarantee compatible interconnection, they do not say much about how business components should be designed to interoperate in a sensible way. We shall be discussing technology architectures in more detail in a later chapter and looking at
Page 56
how vendors and standards bodies are working to expand these architectures to realize larger scale business components. Component Object Model (COM) COM was introduced by Microsoft in 1993 as a natural evolution of the OLE (object linking and embedding) paradigm that they had been using in their suite of Microsoft Office products and the VBX components for Visual Basic. Initially intended for application on a single machine using the Windows operating system, it was expanded to allow communication with components on other Windows NTbased systems, and thus Distributed COM (DCOM) was introduced in 1996. COM objects can in fact be any one of a variety of things: a regular C++ object, an OLE component, a piece of code, or a complete application such as one of the Microsoft Office products. The main characteristic of a COM object is its interfaces. There are some basic interfaces which all objects must implement, but beyond that the user can have any number of interfaces. Somewhat unusually, the objects themselves do not fit the normal objectedoriented model of supporting class inheritance, but it is the interfaces that are inherited. Thus to create a new interface, an existing interface is subclassed. Once a COM or DCOM component is complete, it is compiled and supplied as a binary executable for the particular environment for which it is intended. Microsoft also introduced ActiveX controls as components intended for use in Internet applications. Initially these had to be written in C++, but since the release of Visual Basic 5.0, it has been possible to write them in VB. As a result, they have in fact been primarily used for Windows desktop applications. These two facts have led to a considerable expansion of commercially available components. Microsoft is now expanding the model with COM+, which will include transaction processing services, making the components more suitable as a standard for serverside components. COM and DCOM architectures can be used only on Microsoft platforms, so inevitably Microsoft is largely going it alone while most of the rest of the software industry is lining up against them with a converging CORBA and Enterprise Java Beans offering. Common Object Request Broker Architecture (CORBA) The Common Object Request Broker Architecture (CORBA) was introduced in 1991 by Object Management Group (OMG—see http://www.omg.org). It defined the Interface Definition Language (IDL) and the application programming interfaces (API) that allow objects to interact with each other via an object request broker (ORB). Unlike COM, which is vendor specific, the whole
Page 57
rationale of CORBA is to allow interoperability between components from any vendor, on any type of machine, or across a network. Client objects make requests to the ORB which calls the method of an object on a server. The client object has no need to know where in the network or on what type of machines the server object resides; this is all handled transparently by the ORB. Moreover, each time a client calls a particular method, it may in fact be serviced by a completely different server object that is routed via the ORB. This allows considerable flexibility in managing a truly networkedbased environment that can be dynamically configured for availability and performance. While this would normally be used with the network environment of a particular enterprise, such an approach would work across the Internet. Many people have been excited by the idea that systems could be dynamically assembled at runtime from the vast array of components available across the globe. However, it is doubtful any company would risk its mission critical operations on a system built in this way. CORBA is not so much a component model as middleware (as introduced in the last chapter) that sets a standard for interconnectivity. The IDL is the interface through which objects communicate with the ORB to guarantee that objects written in different languages running on different platforms will interoperate provided their interfaces are written in IDL. Objects have to be compiled for the particular platform they are to run on and, in practice, for the particular vendor implementation of ORB that is to be used. Interfaces do not exhibit inheritances like COM interfaces, but CORBA classes can be subclassed. The lack of a vendorindependent ORB implementation has slowed down the introduction of systems built around CORBA (especially as performance of ORBs varies considerably); however, the introduction of an Internetbased protocol IIOP (Internet InterORB Protocol) is likely to see progress gathering pace. Most people describe the CORBA object and its IDL interface as a CORBA component. The component can vary in size from a single code object to a framework of collaborating objects or even a complete wrapped legacy application. As time has progressed, COM and CORBA have become much more similar in their approaches, with standards and services becoming available that link the two architectures. Enterprise Java Beans One of the key attributes of the introduction of the Java environment is the "write once—run anywhere" approach. Components in Java are called Java Beans and were originally intended for delivery over the Internet to run on the client PC. By making use of visual design tools and design patterns, the aim is to be able to rapidly generate new components without writing code.
Page 58
Most Java Beans were initially produced as GUI components and did not lend themselves to serverside operation. The rapid increase in the use of WWW frontends to business systems created an approach based on a thin client requiring the development of a multitier server architecture and nonvisual server components. Enterprise Java Beans (EJB) was therefore launched to extend the component model to support server components. Visual Java client components operate with an execution environment provided by an application or Web browser. Java server components, on the other hand, operate in an environment called a container. This is an operating system thread, with supporting services, that is provided by a major serverside application such as a database, Web server, or transaction processing (TP) system. A class of EJB is only assigned to one container which in turn only supports that EJB class. The container has control over the class of EJB and intercepts all calls to it. Effectively it provides a wrapper for the EJB exposing only a limited applicationrelated set of interfaces to the outside world. The appearance of the EJB to the outside world is called an Enterprise Java Beans Object. The container is not an ORB and the client identifies the location of the EJB via the Java Naming and Directory Interface (JNDI). EJB will support CORBA protocols and IIOP. EJB interfaces can be mapped to IDL and can access COM servers. EJB and CORBA are rapidly converging, and it is likely that future versions of CORBA will define the standards for EJB. Software Components This is currently the world of object orientation, C++, and Java, but it is also the world of subroutines, function libraries, and Visual Basic. We must not be tempted to think that component implies OO as we have discussed previously. This is the domain where CBD and the drive for reuse was first introduced. It remains the area where CBD has been most successful, and the developments in component technology have served to underpin this success. Nevertheless, this success is still mostly confined to small teams and projects. Enterprisewide component approaches at this level of granularity remain elusive for all but the most mature of software development organizations. The idea of sharing smallscale components across multiple developments probably does not add as much value as many believe, and the goal should be to share business components. MidTier Components When the software industry started to move away from the central mainframe approach to providing information technology services, they moved from a 2
Page 59
tier model to a 3tier model. The 2tier system comprised a very thick mainframe tier containing all the enterprise data and business rules and a very thin presentation layer: a simple terminal. The 3tier model shown in Figure 3.4 was supposed to keep the presentation layer thin and separate the business rules from the data into the other two tiers. This offered an early componenttype approach which would allow systems in any of the tiers to be replaced without affecting the others. But there were several problems: • It was not clear how to structure the midtier to avoid just building another monolithic structure in a different place. • There were worries about performance issues in the midtier, particularly as they typically made use of UNIX platforms rather than the better understood transaction processing systems. • Increasingly PCs were used at the presentation layer and people were tempted to move more of the business rule processing onto them. • Most enterprises had significant business value locked up in their mainframes, which were difficult to migrate and separate. The result was that the corporate mainframes were retained as legacy systems and many 3tier models ended up looking more like Figure 3.5. In order to try and take a component approach to the continued use of the legacy systems, many people used the midtier to wrap embedded functions within the legacy system as we shall describe in the next section. The growth of the World Wide Web (WWW) initially seemed to threaten the whole concept of the 3tier approach. The use of an Internet browser to connect directly to a WWW server seemed to be a move back to a 2tier approach.
Figure 3.4 3tier architecture.
Page 60
Figure 3.5 A distorted 3tier architecture.
However, the limitations of having to store all the required data on the WWW server itself quickly became apparent. People wanted to access data from the corporate mainframes, and soon the WWW server was seen as a midtier server accessing the corporate legacy systems. Moreover, display of static data was only of limited use. People wanted to be able to do complex queries, place orders, and manipulate data from the WWW browser. Initially the concept of Java was that it would enable applets to be downloaded to the browser. The runanywhere concept would allow these clientside components to operate in any environment within a Java virtual machine. Significant worries about security, quality, and performance of downloaded applets has limited use in this way. What was really needed were serverside components that would increase the thickness of the midtier and keep the presentation layer thin. Enterprise Java Beans (described above) is seen as one possible answer and offers the promise of seeing a truly componentbased midtier. So it may well be that the rise of the WWW, far from destroying the 3tier concept, will actually give it the structure and momentum that was lacking before. SystemLevel Components It would seem ideal to build systems out of components and then treat the systems themselves as components, assembling them together to meet the needs of the business. Current systems and applications are probably too large and monolithic to be useful in this way, and business objects and frameworks (described next) offer a more practical way forward. In any case, building systems from newly created components is all very well, but very few large companies have the luxury to start from scratch. Most are burdened with existing systems, often very old systems, which have vital corporate data, business rules, and process locked within them. Such systems have been given a variety of labels
Page 61
over the years, but we believe legacy system is an apt name (although cherished system has a certain charm). Whatever the euphemism used, the image conjured up is one of a rather old and dusty relic which has real value, but may have some awkward strings or conditions attached. Finding a way of incorporating such systems into a componentbased approach is a major challenge for CBD. A common approach is to provide an interface to such a system through a middleware layer or client server system. By providing an appropriate stub on the client to call the interface, the legacy system can be hidden or wrapped to look like a component or object. We prefer to call this setup, shown in Figure 3.6, an embedded component as we believe it more accurately describes its nature. The function is embedded within the legacy system and is accessed through the interface via the client stub. It is tempting to say that the client stub is the component because to anyone writing code for the client that it is all they see. The reality (to meet our criteria above) is that the stub, the interface, and the legacy function are collectively the component. In the purest sense, the user does not need to be aware of legacy system, but of course in practice the system must be there and operating, and there may be legacy system issues (performance, cost, and so forth) of which the user needs to be aware. Another way of looking at this is that the stub and the interface provide a service for accessing the legacy function. This is a good way to
Figure 3.6 Wrapped legacy system.
Page 62
view it, as the idea of a service conjures up the idea of some sort of contractual obligation to provide something to a known standard and quality for which a charge will be made. This is often the case when accessing legacy systems as there will almost certainly be a cost to the business even if it is not passed on to the component user. A further approach is to provide a middle tier to this setup as shown in Figure 3.7. The term 3tier client/server architecture is often used to describe a presentation layer (e.g., a client PC) accessing corporate data in a legacy database via a midtier server (typically a UNIX server). This setup will work equally well when access is to an embedded function on a legacy system rather than just data. Just as before, the whole setup is the component. However, it starts to get more complex if we now start to do some processing in the midtier server. For instance, we might code some business rules that, following a request from the client, access data and functions on several legacy systems, make some calculations and decisions, and package this up to send back to the client. Now the question is: Where are the components? As before, to meet our criteria, we really should consider the whole lot as a component with the embedded legacy functions and midtier business rules looking like subcomponents. This starts to look very messy. Many would consider the midtier business rules to be the components in themselves, but in the purest sense they cannot be. They are tightly coupled to the legacy systems and should the legacy system
Figure 3.7 Client/server legacy system.
Page 63
be replaced, then the midtier component would probably also need to be replaced. If it is simply a data access to the legacy system, then it is likely that any new system might be functionally equivalent, but when legacy functions are being called, it is much less likely. It is certainly true that such an approach isolates the client from changes to the legacy systems, but the danger is creating a new legacy in the midtier. An effective component approach to architecting the midtier is essential to avoid recreating the very problems a 3tier approach was trying to avoid. BusinessLevel Components Although the term is often used, there is currently no standard for business components or agreement on what the term means. Typically a business component is a deployable component that represents a wellunderstood business function. That is, something that would be recognized by business analysts, as opposed to something understood only by software engineers. Thus, it represents a level of component granularity where assembly of these components into a system, if it could be achieved, would represent the greatest added value in terms of simplicity and reuse. Typically business components could be generic, representing common entities such as ''customer" or "domain specific," aimed at particular business domains. They are likely to be made up of smaller grained software components or objects. At present, the majority of things at this scale of granularity are business software packages rather than components. They are complete software applications that, while they can be made to interface with other applications, are not intended to be assembled together with other applications to make something larger. However, there are developments in the software industry that are moving things in this direction. Vendors of large enterprise resource planning (ERP) systems are moving to make their offerings more componentlike. Vendors of many business applications are working in the Open Applications Group to standardize interface definitions and work is going on in the OO world to define higher value business objects. We will look at this in more detail in Chapter 5. Enterprise Components Enterpriselevel components tend not to be physical systems, code, or even services, but highlevel business models, process definitions, or architectures, the highest level of which (e.g., a model of BT) may be unique and, hence, not reused.
Page 64
The individual steps in a business process model would be the ultimate component. Being able to put together a new process to meet a particular business need simply by assembling together component process steps, confident in the knowledge that the underlying infrastructure is in place, must be the supreme goal of CBD. This would be even more attractive if, when defining a process step not encountered before, the underlying system components required to support it could be generated automatically. This may all seem farfetched, but the coming of age of workflow systems and their integration with process modeling tools is starting to make this practical, at least at the lower levels of process description. It suggests an architecture with the workflow engine at the center, surrounded by midtier type components which are called by the individual process steps. This may be too simplistic for large applications and workflow can often be too inflexible to meet the daytoday challenges faced by real business. Research is looking at supplementing workflow with software agents (intelligent components) to undertake exception handling, load balancing, scheduling, and forecasting. The challenge for the future is to understand the role of workflow in a componentbased approach. Another challenge is the handling of business rules. Some business rules will be embodied in the process model, and hence in a workflow enactment, but there will certainly be other rules that will fit elsewhere. Many applications and most legacy systems have hardcoded rules in the software, making maintenance difficult and time consuming. As we have seen, some largescale components can be datadriven, and the business rules configured in this way. The use of components offers the possibility of placing the rules in the appropriate component, but they may still be hardcoded and difficult to maintain. One possibility is to have a rulesengine as a component serving other applications. There is still work to do in finding a pragmatic approach to rules encoding. Commercial Off the Shelf (COTS) Building systems from components readily available from commercial sources (commercial off the shelf, COTS) is gaining in importance in many areas of the software industry. Offering the potential to reduce time scales and costs, components may be general purpose or aimed at a specific user domain (e.g., network management). The main area where COTS has established a serious base to date has been in the types of lowlevel code components we have discussed already (e.g., Windows GUI, CORBA, and some Java Beans). COTS domains have also started to become established around particular CASE tools. COOL:GEN from Sterling Software (formerly TI Composer) has a strong following with a
Page 65
number of thirdparty vendors offering components that fit into COOL:GEN models. Obviously there are a wide variety of business software packages available, but for the most part these are applications rather than components. To try and move COTS to larger scale components, standards bodies, such as the Object Management Group (OMG), are attempting to define a common set of objects that can be used to build common business applications (see below). There are a number of risks and issues with COTS that must be considered before large scale commitment to this approach: • Commercial viability of suppliers; • Implications of different standards bodies; • Feature interaction; • Quality, reliability, certification; • Tailorability, configurability; • Maintenance, enforced upgrades, obsolescence; • Whole lifecycle costs; • Design methods. In order to meet the criteria we laid down earlier, components should be capable of being replaced by a component with an identical specification from another vendor. However, given the current level of maturity of the COTS market, it is rare that truly identical components will be available. This makes the choice of vendor and their commercial viability of crucial importance when implementing COTS. Summary This chapter has explained in some detail what we mean by a component. A key point is that a properly defined and managed set of components, along with well designed interfaces between them, is the basis of sound software systems. The specific aspects of this covered in the chapter are: • What is a component? A component is a selfcontained, clearly identifiable, physically realizable, predefined entity that provides a wellunderstood function and is intended to be assembled, using a welldefined architecture and interfaces with other components to provide more complex functions. A component is not so much a specific thing (i.e., something
Page 66
with a specific size and shape), but more a label that can be applied to types of things that meet certain criteria about how they can be used. • What is CBD? Componentbased development is not a technique (e.g., OO); it is not a product; it is not a technology (e.g., DCOM, CORBA); it is not an implementation standard (e.g., 3tier clientserver). At this stage in its maturity, it is not even a method (although we hope it will become one) but rather a vision of how we can assemble software and systems in a way that minimizes the amount of new code that must be written, the amount of systems integration, and the amount of testing needed. It is a culture of reuse at all levels that must pervade the whole of the company's operation. We will be looking at this in more detail in a later chapter. • Why do it? Without this capability, products and systems cannot evolve rapidly enough to meet the changing needs of the market. There is not time to create new products or systems from scratch each time. Reuse is not just about assembling systems but covers the whole question of how systems may be built to respond to the needs of change. • What won't work? We don't think that successful componentbased reuse will come from creating a load of components and then trying to build something from them. Components do not remove the need for careful and planned design. We do not believe that serious mission critical applications can be built by assembling, at runtime, a vast collection of distributed components from a variety of changing sources and vendors. Less critical applications may work this way, but would you risk your paycheck to such a system? • What will work? We argue that the key is structure, structure, and more structure. CBD should be a multilayer approach. It should provide a method (or range of methods) and architecture (or structure) that is appropriate for the layer, the environment, and for the type of assembly being done. Not only must components fit together and work as intended, but they must be capable of being effectively configured, enhanced, removed, or replaced. The way forward, at least for the moment, lies with structured collections of components, frameworks, plugin components, and domain specific components. We will look more at structure in Chapter 5. • The future: Intelligent components (agents); self configuring; self testing. And now that we have established how things are connected via interfaces, and that things in the software sense are components, we move on to look at the way in which software systems can be integrated.
Page 67
Reference [1] Brooks, F., "No Silver Bullet," IEEE Software, April 1990.
Page 69
4— Integration If the map and the terrain disagree . . . believe the terrain. Swedish Army Handbook
A good set of components and clean interfaces between them does not solve all problems. Far from it—even if our friends, Mrs. Smith and Mr. Lee, from Chapter 2 can communicate, they will not necessarily agree. They will probably know different things, have different aims, and will interpret a particular message in a different way. So it is with systems. The data held within systems is frequently as much of an issue as the interfaces between systems. This is especially true when extensions are being built onto an existing configuration. This chapter completes the overall picture of componentbased engineering by explaining some of the issues and options for integrating components both new and cherished. In addition to this, we describe a systematic approach to integration that imposes quality checks between each stage, and thereby assures the end result. To start with, we explain the term integration. What Is Integration? It would be nice to give a precise definition of integration; one that inspires confidence that a soundly based set of techniques to deal with it will follow. In truth, it would be wrong to even aspire to this at the moment. Integration is often seen as a necessary evil rather than a soundly based aspect of engineering. That is not to say that it is a lost cause. The realistic and practical expectation
Page 70
is that integration will be messy, and the reasons for this will be explained in some detail, but we will progress towards a systematic approach. First, then, our working definition. In the context of networked computer systems, integration covers all aspects of getting the myriad components of a real system to work together such that it is fit for that purpose. In practical terms, this is just as difficult as a complex mathematical integration and considerably less welldefined. To some people, integration is achieved as soon as a number of components have been connected, rather like welding together two halves of a car. The result is about as satisfactory in both cases. To others, the job is not complete until there is seamless interworking between all of the various parts. Perhaps the best way to characterize the term, though, is to explain some of the challenges that ensure that the life of a systems integrator is never dull. Dealing with Existing Installations This is usually called something like "coping with legacy," "overlaying backend systems," or "extending the systems estate." The issue is that any form of technical evolution implies that a collection of established and new hardware, software, data, and networks all work together. The legacy portion of the planned whole has been accumulating for long enough that risk, cost, and complexity are rising month to month in most softwarerich organizations. It is rarely welldocumented and the skills to support it are often at a premium. Yet the legacy systems hold key data and have to be designed in as part of the overall componentry. Research indicates that about half of the companies in the high technology sector have over 15 years worth of legacy. The good news (although this is arguable) is that the nadir of legacy is past. Awareness of the issue and the development of the concepts explained in the last two chapters have done a lot to help. Stovepipe Designs There are a myriad of choices to be made in designing a system. Some of them are internal and, although they may affect performance or efficiency, they tend not to have a greater impact. Other choices are critical to the way in which the system can work in the wider community. In particular, the general structure of the system— where the data, the application logic, and the presentation logic reside—is vital to its openness. In the past, each computer vendor has tended to build systems his or her own way, a fact celebrated in a variety of proprietary architectures. There are a number of monolithic applications and architectures that work well, up to a
Page 71
point. As soon as you try to (or, more likely, have to) work with applications that do not fit this structure, there are potential problems. By way of illustration, the use of a PC as an Xterminal working into a UNIX application leads to the use of a very lowlevel protocol (XWindows) between client and server. This requires high bandwidth to work and usually provides poor response times for the user. It does work, but it is far from ideal. There are other examples of mismatched structures that cause implementation difficulties. For instance, SQLsquirter products, such as Oracle Forms, move a lot of application functionality onto the PC client. This means that an SQL database query protocol has to be provided between the client and the server. This factor, if compounded with other similar requirements, can quickly sink the system in complexity. In both cases, extra effort is required to get two ends of the same, supposedly cooperating, system to work together. The general problem is illustrated below in Figure 4.1. Two systems that need to interwork are shown as autonomous stovepipes that have to reduce interactions to an unnecessarily low level. The figure represents one aspect of a situation illustrated further in the next chapter—that of trying to get differently archirected systems to cooperate on a single, end toend task. In an ideal world, this would be achieved by accessing the necessary resources from each system via common processes. In practice, though, this can demand a lot of specialist crafting. Hence the key problem of stovepipes: the shortcut of building system stacks that can not interwork because it is too difficult to make ones that do. Multiple Access Requirements Despite all of the changes that have happened in the computer industry over the years, one of the few things to remains constant is the need to know the identity of the user who is requesting service. And for this information to be gathered, most systems require the user to provide a name plus a password before access is allowed. This process of authentication is usually referred to as logging on and normally takes place when the user first connects to the computer. The problem is that further logons are then required if the user wishes to access other computers or services. In a distributed environment, with information and computers all over the place, multiple logons are required nearly all of the time. The base problem is compounded when user codes and passwords are not coordinated across computers leaving the poor beleaguered user trying to remember dozens of name/password combinations in a variety of formats. There are ways of providing an authentication scheme that operates in a multiprocessor, client/server system. The trouble is that each legacy system has
Page 72
Figure 4.1 Stovepipe systems—communication, the long way round.
Page 73
to accommodate this scheme and there is always unwillingness to change from the status quo. Closed Interfaces Most established computing equipment is set up to do things in a particular way. This includes the way in which it talks to peripherals, controls user access, and interconnects to other systems. Yet most real world computer environments are comprised of a variety of diverse elements of different vintages from different manufacturers. Given the desire to organize a computing environment so that it looks to the user like a single resource, there has to be some provision of routing and translation services so that a user can see what is out there and can negotiate services, wherever they originate. People do not want to change the way they do things, though, which is not surprising when you consider how much time and money they have invested in their system. So interfaces tend to remain closed with each requirement being handled with a different tactical work around. The accumulation of these make maintenance increasingly difficult, thus fueling the desire to rework. Diverse Data Information that once resided on one machine is often scattered to the four winds these days. Typically, a customer's identification will reside on one system, his or her current orders will be in a separate physical and logical location, and his or her market behavior will be somewhere else again. Yet all of these pieces of data may need to be assembled to conduct a particular session and often need conversion functions before they can be brought together. The problem is akin to a network composed of many subnetworks, all connected with a plethora of protocol converters. What the system user wants and views as a single logical entity may have to be gathered from many diverse sources over a variety of subnetworks. This point is worth discussing as the data binds a system together as much as its electrical or logical interfaces do. Explicit plans for data integration are an essential part of coping with legacy and are frequently overlooked. A corollary on this point is that any two systems with similar data (variants borne of a common ancestor, perhaps) may well disagree. With information becoming the currency of many a modern business, consistency of data becomes increasingly important. Getting data formats to agree and cleansing the existing data held can be a huge task. There are few tools to help and the task is not cheap either; even at 10 cents a record, the cost to many companies would be very significant.
Page 74
Nonstandard Users Just as there is no such thing as a green field for a new computer system, neither is there a typical system user. It is unrealistic to assume that anyone will be sitting at a terminal at any particular time or for any assumed duration, nor that he or she will react in a deterministic way. Applications that take a long time to assemble even the most complex record will not be perceived as helpful. Neither would a U.K.based user query that has Eastern Standard Time embedded in the response. Complex systems with many components bring with them all manner of access patterns. Add to this the host of (sometimes long forgotten) configuration files, macros, and the like that cause an application to execute in a particular way and there is considerable potential for unwitting error. One particular aspect of the assumed user that is becoming more prevalent is that more and more nonhuman users are accessing systems. The use of agents, robots, knowbots, and data miners has shot up over the last few years. They tend to hit systems and networks very hard, with high volumes and rates of transactions. The need to extract information from vast pools of data will, inevitably, increase the use of these intelligent agents. Loose and Tight Integration The extent to which two systems or applications have to work together should determine the way in which they are integrated. Some applications need to appear as a natural extension of the systems to which they have been added. Others can be added as a separate facility. The former (for instance, an address driven switch) requires tight coupling, which, for all its attractions usually requires a considerable amount of lowlevel work and, potentially, reduces future flexibility. The latter (such as filebased interactions) may entail extra user configuration and may be seen by those users as a collection of piecemeal elements, but there is at least some scope to change a part of the system without resorting to wholesale redesign. The choice of integration style needs to be carefully considered. The bottom line that follows from all of the above is that the integrator just has to cope with all of these difficulties. Some of them may turn out not be a problem, either by chance or by design. Others, almost inevitably, will be [1]. With around 50 percent of all operational computer systems between 10 and 15 years old and COBOL still the most common highlevel language, legacy is something that the integrator has to live with. This section has highlighted some of the areas where integration difficulties are likely to be and what issues need attention. Later in the chapter, we explain some of the strategies that
Page 75
can be adopted to deal with the problems raised here. To put all of these issues into context, it is worth identifying three key roles. These are: 1. The architect. This role is all about the setting of rules, philosophy, preferences, and structure that cover all systems. An overall strategy needs to be defined and the technologies that should be used need to be identified. The key objective is to reduce diversity (and, sometimes, to police conformity to the strategy). 2. The designer. This role is concerned with specifying a system that works together as well and as speedily as needs dictate. He or she needs to take all of the above issues into account but should be more driven by the concepts introduced in the previous chapters. The key objective of this role is to design with integration in mind. 3. The integrator. This role is more focused on delivering the specified system so that it works as intended once deployed. He or she needs to validate and verify the designer's work by showing that it operates within actual environmental constraints. The key objective of this role is to deliver the system so that it operates as desired. Usually, these roles are filled by different people (or groups of people). In practice, they need to work closely together. In any case, there are some specific guidelines that can be used to guide progress and these are laid out later in this chapter. Key Concepts Given that you have to live with history, there are choices in how you deal with this. Here we explain some of the strategies for integrating systems, from the basic fallback of screenscraping to more sophisticated options. Each strategy has its pros and cons. Which one is adopted needs to be considered on the basis of overall business benefit—time and cost against value added. After all, there is little sense in elegantly integrating a system just before it is removed from service! Likewise, a cheap and cheerful patch to access important information may well prove to be a false economy. Here are five basic strategies that might be considered for coping with basic system integration. There may be others but those below illustrate most of the cases that occur in practice:
Page 76
Scrap Very simply: throw the old stuff away. Buying or building new systems will be cheaper and easier in the long run. This strategy has to be driven by commercial judgment and based on the longevity and importance of the legacy system balanced against cost of replacement or alteration. The judgment of replace or reuse is a complex one, well covered in the literature on business planning [2]. Even so, the operational cost of an inappropriate or ineffective system is rarely counted. Trap Keep the legacy system but avoid perpetuating its use. In this case, integration all takes place within the client that uses the legacy system. This could be a mix of screen scraping,1 file transfer, and so forth. It traps the client with the legacy system, making it expensive to build and difficult to change. The likely next step is to scrap both legacy system and the client that allowed it to remain operational a little longer. The downside of this option is that it may result in a bigger problem later on. Map A front end is provided to the legacy system. This is the first level of real integration (as opposed to the two strategies described above—they simply avoid, delay, or work around the problem). In this strategy, a proxy interface is provided (usually hard coded in a server) to allow clients to communicate with the legacy system. Again, techniques such as screen scraping are often required. Wrap This option is the same as above except that the legacy system is also reengineered to some extent. The aim here is to pull the legacy system apart and reassemble it into more maintainable chunks. Many of the ideas introduced in the previous chapter apply here. This strategy is, in essence, a componentbased one with the aim of integrating reengineered legacy system pieces (usually by reworking the more problematical areas or, at least, the easiest to change areas). 1.
As the name suggests, this entails the retrieval of data from the point at which it is presented to the user. Even a small change in the screen layout means that the screen scraping has to be revised.
Page 77
Unwrap This is another componentbased strategy, where the legacy system is fully integrated. It follows on from the above, this time with the legacy system being completely reworked. In effect, the old system is rebuilt to modern architectural guidelines. There are methods and techniques that could be explained for each of these: business analysis methods to place value and strategic importance on systems, tools that can be used for screen scraping (e.g., DesqviewX), and software suites for reengineering (e.g., the Bachmann toolset). The main thrust of this text lies elsewhere, though, so these aspects will not be dealt with further. In any case, they merit lengthy coverage in their own right, mostly because they are complex and not readily prescribed. Figure 4.2 is a general view of the various types of legacy systems that surround most new developments. The key point to be made here is that whatever strategy is adopted for interfacing, there are four main types of legacy functions that need to be considered. These are: 1. User PCs, where each person tends to look after his or her own work and attend to their own security. In terms of technology, this group uses OLE, ODCB, and the members like to share information. 2. Systems administration, where specialist units are required to look after service and network management. The technology used here is guided by the Telecommunications Management Forum and the Internet Engineering Task Force.
Figure 4.2 A general template for systems integration.
Page 78
3. Information services, where a huge amount of data is collected in warehouses and these are managed to adhere to service routine reporting requirements (e.g., monthly sales returns). 4. Data, which is similar to the above, with the exception that requests are ad hoc and need to be serviced quickly. Direct database queries (e.g., via SQL) and transaction processing are relevant here. The aim from here on in is to look at ways of building systems that can readily integrate without resort to delicate (and risky) surgery. And so we now move on to give a few of the tried and tested tips for avoiding future legacy separation of concerns, standard interfaces, parameterization, and so forth—before giving some guidelines for planning the integration activity. The System Integration Process Just as complex systems can be built from welldefined components, so the way in which integration is managed can be tackled as a set of predefined processes. This does not imply that there is a simple or mechanistic way of carrying out integration, rather that there is some order that can be exerted and this brings a measure of control. Given that the main challenge of integration is to bring together different pieces of software, our integration process has a lot in common with established software development processes [3]. We now define ten stages that link together to give a systematic approach to integration. It does not guarantee success, but it does provide a level of assurance and reproducibility. In each stage there are a number of activities and checks to be performed as well as some necessary inputs and desired outputs. In view of this, a common format is used for the ten stages: the required inputs, the desired outputs, the activities during the stage, and the checks being made. The size of each stage will vary from one project to the next; sometimes each one will be a major piece of work, sometimes the stages will be almost trivial. Likewise, the techniques deployed to carry out a particular check will vary; there are plenty to choose from in the literature [4]. Perhaps the key point of note, though, is that each stage is valid, even if its execution is handled informally. Requirements This is the point at which the customer's needs, expectations, and wishes are sought and analyzed. In some circumstance there may be no readily identifiable customer. If this is the case, someone who will use or can pay for the work to
Page 79
be done should be found—absence of customer brings into question the need for the work! Assuming that an interested party is prepared to articulate what is needed and there are people set on satisfying that need, the vital aspects of this stage can proceed as shown in Figure 4.3. There are a number of methods for capturing and analyzing requirements [5]. Whichever is chosen, the outputs listed above need to be produced before the next stage can begin. Design It is a little invidious to bundle this intellectually demanding and creative activity under one heading in the integration process. Having said that, there are some clear objectives that a designer needs to meet. Again, we express these in terms of checks that need to be carried out and outputs that need to be produced (see Figure 4.4). A great amount has been written on design, most of it explaining good practice and providing expert guidance. The essential aim of the design stage is to develop a conceptual, endtoend view of how requirements could be realized and to ensure it can then be built from available or developable components. Again, our concern is to formalize the result of this process step rather than advise on its execution. One part of the design phase that we should say a little more about is the influence of standards. Earlier in this chapter, we mentioned the role of architect. The strategic guidelines and technical preferences provided by the architect are a key part of the designer's input and should be taken as a given. This may rule out some technical options but often provides a readymade blueprint for the systems specification and hardware/software specifications. Development This stage is the production part of the overall process. As with most of the other stages, there are some judgments that have to be made (e.g., on the suitability of various supply options) within the framework given in Figure 4.5. These are sometimes preempted by user requirements or by mandated architecture. This stage, more than any of the others, is likely to involve thirdparty products and services. As well as suitability for purpose, an aspect that should be given priority is the extent to which the third party is trusted. This is especially true when the component or contribution is central to the success of the whole venture.
Page 80
Figure 4.3 Inputs, outputs, checks, and actions for the requirement stage.
Page 81
Figure 4.4 Inputs, outputs, checks, and actions for the design stage.
Page 82
Figure 4.5 Inputs, outputs, checks, and actions for the development stage.
Page 83
Acceptance This is the first point at which the customer verifies that the system does what was intended. That does not mean that all is complete; there is still a long way between having a system operating in a controlled environment and the real thing working in the field (see Figure 4.6). By the end of this phase, there should be confidence that the right set of components have been put together. From here on we need to address some of those issues often perceived less glamorous (but still absolutely vital)—ensuring reliability, flexibility, robustness, and performance. Build This is the first of the product engineering stages (see Figure 4.7). The main aim is to ensure that the chosen system components can be assembled in the right way to meet the compliance statement generated in the last stage—a validation that the design can be assembled. One of the key aspects of this stage is configuration management. The discipline of tracking change and recording compatibility is one that has received little attention in the publications, partly because it is something that requires methodical attention to detail rather than the application of theory. Test The final stage before release is to test the whole system (see Figure 4.8). The functions it needs to carry out are but a part of this. By and large, these will already have been attended to during the acceptance stage. The main thrust now is to make sure that the system can cope with volumes anticipated in operation. A wide variety of test tools and techniques have been built up. These cope with flood, stress, and performance testing. Release At this stage, a product is handed to the customer along with all of the associated documentation with the intent that the customer accepts the systems previously seen in the lab as one that can be released into the field (see Figure 4.9). A large part of this stage has to do with customer engineering—the design of people process that matches the new technology being introduced (which, in practice is the art of getting a best match between the two, in the knowledge that neither can provide total flexibility).
Page 84
Figure 4.6 Inputs, outputs, checks, and actions for the acceptance stage.
Page 85
Figure 4.7 Inputs, outputs, checks, and actions for the build stage.
Page 86
Figure 4.8 Inputs, outputs, checks, and actions for the test stage.
Page 87
Figure 4.9 Inputs, outputs, checks, and actions for the release stage.
Page 88
Installation This is the stage that sees the product being rolled out to its end users (see Figure 4.10). Details of where data is kept, how it will be maintained, how communications links will be realized, and so forth all come together here. Operation Once the product is in situ and working it needs to be kept in step with an everchanging world. No matter how good the first release is, there will always be changes, problems, and issues. An effective mechanism for supporting these changes needs to be put in place (see Figure 4.11). The discipline required to support operations is similar to that needed in the build stage as an uptodate inventory of system configuration is central to assessing where problems lie and what changes are possible. Many of the procedures used in software maintenance apply here [6]. Cease This is more than just switching a system off. It is, at least, making sure that components are recovered (if need be) and that formal agreements are terminated. And with everincreasing interdependence between systems, it is becoming an exercise in safely decoupling a system and making sure that doing so does not leave problems for others (see Figure 4.12). Finally, it should be noted that the life cycle outlined here, like other life cycles, is for guidance only. It can help in bringing a level of uniformity into the delivery of componentbased systems but it does not supplant the technical knowledge, contextual understanding, or practical appreciation that contributes so much to real quality. Practical Integration The process detailed in the previous section goes a long way to ensuring that things come together as desired. One thing to say though is that, to some extent, integration is always going to be a problem. Today's new system will be tomorrow's legacy. Even with the best planning in the world, systems from the 1960s are unlikely to fit readily with modern equipment. The variable that can be controlled, though, is the time it takes for a particular system to become legacy. This should very much be conscious choice, taken in the light of local circumstance. It may be that freedom of action is so important that any attempt to
Page 89
Figure 4.10 Inputs, outputs, checks, and actions for the installation stage.
Page 90
Figure 4.11 Inputs, outputs, checks, and actions for the operation stage.
Page 91
Figure 4.12 Inputs, outputs, checks, and actions for the cease stage.
Page 92
constrain diversity is unacceptable. In this instance, systems can be purchased and developed to no set plan, the cost of subsequent integration (if, indeed, this is relevant) being accepted. The only observation to make here is that the anarchic approach should be selected and not happen by default. For most enterprises, some form of control is seen as desirable. The rest of this section outlines the main areas where control can be exerted. A few of the choices in each instance will be mentioned. Publish a Strategy This sounds rather grand but can be very straightforward. For instance, ''only buy IBM equipment" was a strategy adopted by many computer operations managers in the 1970s.2 The key point about establishing a strategy is that it lays down some ground rules—usually in the form of a limited set of options for the purchaser, designer, and planner. An effective strategy can be driven by a number of considerations. It may be that a cross company alliance motivates a particular choice, or a technical suitability may give rise to a preferred choice, or bulk discount from a particular supplier may win the day. Either way, the aim of a strategy should be to reduce overall (that is, whole life) costs. And this can be achieved by defining, among other things: • Key design options that allow for future flexibility; • Technology choice that eases integration; • Purchasing considerations that minimize cost. In practice, many of the organizations that have a strategy are driven either by purchasing requirements or by technical preference. Explicit integration strategies are few and far between. Indeed, in many instances, the cost of integration is considered as no more than a minor feature of stated strategy. The huge investment in computing infrastructure and the ever accelerating rate of change is causing this balance to shift, though. Define an Architecture This builds on the above in that it covers the how as well as the what. An enterprise architecture should go beyond stating intent—it should map out how systems are put together and which components are to be used. The level of de 2.
For the very good reason that at that time no one ever got the sack for buying IBM.
Page 93
tail tends to vary from one instance to the next. For example, the distributed systems architecture contained in the ANSA reference manual [7] gives a host of formal concept definitions and object specifications. In contrast, the more general threetier model of computing (usually attributed to the Gartner Group)3 provides a simple but elegant structure for system planning. Some organizations, especially large multinationals and information intensive businesses, need to have their own, inhouse, enterprise architectures. Typically, these will provide a definition of key processes, data used, and functions supported [8]. More than anything else, these architectures provide a common language for describing what software components should do. In addition to cataloging existing systems and showing how they are likely to evolve, they often provide design guidelines for developers. Many of the issues covered earlier in this chapter (such as the need for separation of presentation, application, and data) are included, either as recommendations or as mandates. Define an Evolution Plan Many of the problems of integration can be traced to the diversity of established systems. The move to componentbased engineering will only work if there is a plan that shows how legacy systems will be migrated towards a component structure. The wrap strategy introduced earlier will work anyway, but the unwrap strategy is probably unachievable without a plan. A practical evolution plan should cover more than just the legacy systems. It should also contain forward plans for componentoriented developments. Some organizations publish evolution plans as a guide to designers, who can intercept the appropriate set of components based on what will be available rather than what is here now. To some extent, they can also allow for knockon effects from other developments. Prescribe Technology There are many technology choices that can be made in constructing highly interconnected information systems. At some point, it is prudent to decide what the preferred set should be. For example, there are two basic distributed processing technologies: remote procedure calls, RPC, and transactional remote procedure calls (TRPC). The former is well suited to realtime applications and 3.
But difficult to say for sure as there does not appear to be a readily available reference to prove the matter.
Page 94
office automation, the latter is usually associated with data processing and batch applications. Within each of the two basic categories, there is a further raft of technologies, for example, for RPC, there are Sun's ONC and Microsoft's NT/RPC, and for TRPC, there are Encina, Tuxedo, and IBM's CICS. Time spent devising what you actually need and what constitutes a coherent set can be very well spent. The choice, in this instance, of a particular distributed middleware is key in building communication applications and in the decoupling of legacy. Diversity is nice but can be costly and (with the devil invariably residing in the detail) is a good way of storing up problems for the future. Quite apart from being able to cope more easily with change, there is significant financial motivation in constraining choice (e.g., less evaluation, retraining, support, and integration cost). With a little informed planning, it is possible to avoid many of the integration blues and prolong the active life of a system. The authors know of several systems, introduced in the mid to late 1970s, that still provide a worthwhile part of an organization's service. Many of the functions carried out by these legacy systems— customer handling, billing, and so forth—will not go away and will have to migrate from the old to the new. Hence, as new technologies are introduced they will inevitably have to work with what is already in place, and so integration is a now and forever task that componentization will make easier but will never eradicate. Summary Change is inevitable. Computer systems are far from immune, and there is always going to be a mix of the old and the new in any organization, just like the contents of most people's wardrobe. The consequence of this is that you either have to replace everything (which is usually prohibitively expensive) or blend the new items in with the old (which is tricky but manageable). The only trouble is that the style challenge of matching clothes becomes a lot more complex when the components that make up complex, networked computer systems are the object of attention. The challenge is not one that can be shunned, though. There is little doubting the central role of computers in business these days. They are an essential element in supporting many of an organization's basic operations. Further to this, the way in which they are deployed is becoming a key differentiator as we enter the information age—a source of competitive advantage. So it is desirable to be faster and better than the rest. The extent to which this can be realized, though, is constrained by history. There are now some thirty years worth of computer systems in place. To
Page 95
a greater or lesser extent, they do the job for which they were built. In some cases, investment has been recouped. In many others, it has not. When newer ideas— such as moves to mobile code, distributed systems, Web technology, and so forth come along, the installed base has to be taken into account and this means that the new has to be integrated with the old. In practice, this is a major problem for many organizations. This chapter explains some of the reasons why integration has become such a major issue. The challenges that have to be faced when designing in a constrained environment are explained here. The main message that emerges is that integration is a complex and often messy exercise for which there are precious few prescriptions. That said, there are some useful guidelines and strategies, borne of experience, that can minimize the challenge: • Architecture—akin to the building regulations that allow progress while preserving the beauty of an old town; • Constraints—paring down the range of options to an approved set, one that gives enough scope but not too much rope; • Strategy—the broad set of alternative paths to follow and a blueprint for the designer to follow; • Process—a systematic approach to integration with inbuilt checks and balances. In support of this, the larger part of the chapter deals with ten process steps that can, and should, be taken to help integrate new systems into an established environment. References [1] "Software's Chronic Crisis," Scientific American, September 1994. [2] Whittaker, et al., Strategic Systems Planning, Wiley, 1993. [3] Norris, M., Survival in the Software Jungle, Artech House, 1995. [4] Rakitin, S., Software Verification and Validation: A Practitioners Guide, Artech House, 1997. [5] Jackson, M., Software Requirements and Specification, Addison Wesley Longman, 1995. [6] Norris, M., and P. Rigby, Software Engineering Explained, Wiley, 1992. [7] Advanced Network Systems Architecture Handbook, ANSA Consortium, Cambridge, MA, (http://www.ansa.co.uk). [8] West, et al. Computer Systems for Global Telecommunications, Chapman and Hall, 1997.
Page 97
5— Architecture and Structure Good practice comes from experience, and experience comes from bad practice. Fred Brooks
Up to this point, we have considered the essential elements of componentbased systems engineering. First we looked at interfaces, then at components themselves, and then at the processes by which systems are integrated. Before moving on to practical illustration, there is one more element that we need to consider—the master plan against which systems are built. In this chapter we discuss how structure is essential for the successful use of components. We look at different techniques for structuring software and whether these allow us to retain the flexibility that is such a strong attraction in software development. To put some meat on these fairly abstract bones, we then look at some real product line architectures and the challenges presented by complex product delivery organizations. To close the chapter we provide some pragmatic guidelines on how to choose and use an architecture. The overall aim of this chapter is to show that architecture is about making key decisions on structure, implementation, standards, constraints, and tradeoffs. We illustrate how architecture impacts existing products and guide the reader to the application of architecture in their own environment. But first, we make the case for being interested in the first place. Would You Buy a Car from This Man? Imagine someone wanting to buy a new car. He or she visits the local car dealer, but there is no showroom full of bright shiny cars, no glossy brochures, just a
Page 98
small office with a table and two chairs. The customer sits down with the dealer who painstakingly writes down all of the customer's requirements and promises to provide a quote. The dealer gives the requirements to another person who turns the requirements into a detailed specification of the car and works out the price and how long it will take to build. The dealer gives this to the customer and asks them to sign and pay for it. The dealer now gives the specification to a car designer who produces detailed plans of how the car is to be built. Finally the designer gives the design to the car assembler. The assembler takes the design plans and enters a huge warehouse, full of every conceivable part that might be used in making a car. The assembler walks up and down the rows of stacked shelves looking for parts that could be used to meet the design. Eventually, the assembler has collected together most of the parts needed and sets about putting them together. Some of the parts fit together well, others need filing down or adapters made. Eventually about 80 percent of the car is complete. It is not exactly what the customer wanted and there are some bits still missing which will have to be specially made. The assembler persuades the dealer to go back to the customer and tell them the car will take another month to complete and will cost 10 percent more than the quote. Would you buy a car from this dealer? I am sure you would not, yet this is exactly what is happening in some parts of the software industry at the moment. Of course the vision is that when the assembler wanders around the warehouse it will be possible to find all the necessary components and there will be no need for the delay and the price hike. All that is needed, people claim, is for the right set of components to be available and for an effective system for identifying and locating the components; then everything will work as planned. Moreover, by taking the same set of components and assembling them in a different way, it will be possible to create a completely different product. Not just a different car, but a completely different type of vehicle or even something that is not a vehicle at all. When we look at this software engineering scenario and translate it into the motor manufacturing domain, it appears completely ludicrous. Why is that? Part of this book is about trying to understand this and looking for a way that the scenario can be made to work or finding an alternative scenario. Sum of the Parts In the earlier chapter on components, we explored in some detail the example of assembling motor cars. We saw that the use of components was a vital element, but that the use of components was not an ad hoc searching for things
Page 99
that might work, but a product line carefully designed to reuse common components—very different from the tongueincheek example above. It is easy to claim that car assembly is mass production and very different from software engineering. That is certainly true, but gone are the days when cars were produced in batches of different colors and with limited options. Today cars are made to order with assembly lines capable of taking the basic model and characterizing it to meet each customer's requirements. This mass customization model is what we need to achieve in software engineering. What allows mass customization to work is that the range of options and variation in the product line, Figure 5.1, is very carefully controlled. The flexibility points are carefully chosen so that they can be delivered from the set of available components and within the production capability of the assembly line. Moreover, a great deal of the variation is based on adding or substituting options. That is to say options add function to the basic car rather than drastically changing it. Thus the model that is top of the range can be expected to have most of the options while other models have fewer of them fitted or have lower cost alternatives. This is very different from both the practice and the vision of the software industry which thrives on the prospect of a very rich (almost infinite) set of flexibility points. Can we think of another component assembly example that is more like the vision of a software assembler? The second example we looked at earlier was assembly of personal computers. This seems much more like our car dealer example with PC assemblers literally taking useful components from the warehouse and putting them together to deliver what the customer wants. What makes this work? There is some sense of a product line, but it is not nearly as advanced as the motor car industry. At first sight it seems there is a great deal of flexibility in how PCs can be assembled, but in practice there are very few flexibility points. The basic functional components of a multimedia PC are the
Figure 5.1 A simple product line.
Page 100
same in all cases: processor card, RAM, hard disk, video processor, audio card, CDROM, and so forth. What enables PCs to be assembled in this way is the very strong architectural definition of a PC. Some elements of this architecture have been carefully designed and agreed on while other parts have arisen through widespread de facto use of particular items. Whatever the route to the architecture, there is now a very clear functional definition of each of the components and an even stronger definition of the interface. The flexibility comes more in terms of nonfunctional requirements: performance, price, and reliability. The successful assembly of these components is very domain specific. The components are always assembled to make PCs. Assembling the components in a different way does not produce any other sort of product (it does not produce anything at all), nor are the largescale components used in any other sort of product. We now have two examples of successful componentbased products, one using product lines and the other a strong architecture. Neither seems to closely match our needs for extensive flexibility so let us look at the example so often held up as the vision for software engineering, that of integrated circuits. In the earlier chapter we introduced the TTL family of logic gates and showed how they represented a range of basic building blocks that can be assembled in different ways to produce different products. This seems to be exactly the sort of thing we want for software, the software IC. What is it that makes the concept work for hardware? There are three key factors that make TTL work: • Structure; • Design patterns; • Replication. First there is a very strong sense of structure for the family. Each element and its interface is precisely defined. With the very basic blocks, there is no ambiguity at all in how they fit together and even with the more complex devices, the interconnection is well understood. Secondly, there are very well understood design patterns which define how the building blocks can be assembled together to make high order functions (e.g., making a shift register out of basic logic gates). In fact many of these patterns are mathematically underpinned by Boolean logic. Thirdly, as we discussed earlier, much of the complexity of items assembled from these components comes from the replication of the basic elements (e.g., memory chips, microprocessors). It is arguable whether these views constitute an architecture. Certainly at the interconnect level there is a technology architecture, similar to DCOM or CORBA. But at the higher functional level it is less distinct. What is interesting is that many of the more complex devices in the family, which were added
Page 101
later, are more self contained and perform significant functions in their own right. Based on the definitions we introduced in Chapter 3, they seem to be true components while the smaller elements are subcomponents or parts. The thing that tempts us into thinking that we can assemble software in a similar way is that we see lots of examples of common software elements frequently recurring in different developments. The granularity of these software elements (e.g., loop structure) seems very similar to the TTL logic components. However, the key difference is that software components are not combined in replicated structures. They are reused in much more complex interlinked structures that do not build in such a clearcut hierarchical manner. From what we have said so far the reader might think electronic circuits always combine in a hierarchical way. This is not always the case. In fact if we examine digital electronic circuits we can see two types of pattern. Circuits made out of very basic building blocks, and which by their nature don't have a high degree of replication, instead tend to have a high degree of interconnection (see Figure 5.2). Circuits which are built of much larger elements, themselves built of replicating structures, show much clearer interfaces and simpler structure. If the circuit diagrams show every connection they still appear very complex and highly interconnected, but if the individual connections, say 32bit or 64bit communication bus lines, are shown as one interface, then an elegant simplicity is revealed (see Figure 5.3). The point of this discussion is that the basic building blocks are analogous to software objects, highly interconnected and not really components. The large structures are much more like true components; they have a significant level of function. They are loosely coupled with simple interfaces, albeit with very complex information passing over the interface. So although hardware is different from software there are commonalities and the hardware experience confirms the view that more value will be gained from components that are large grained and loosely coupled. Of course, we must not take the analogy too far, because someone is sure to point out that these complex hardware components provide their function by running software and that really complicates the picture. Structured Flexibility Why do we want to use components in software engineering? We have suggested that we want to capture that set of software elements that we see cropping up time and time again and encapsulate them in such a way that we can assemble them together in different ways to produce different products. In that way we can get the key benefits of reuse: better quality and reduced cost, while retaining the flexibility to produce new and better products that will give our
Page 102
Figure 5.2 A highly interconnected digital electronic circuit.
Page 103
Figure 5.3 A componentoriented digital electronic circuit.
Page 104
business the competitive edge. We have looked at three examples of successful componentbased assembly and seen that they work for different reasons. Those reasons are inextricably connected with how the components are structured to fit together and also how the degree of flexibility is constrained. The choice of components is not ad hoc and from this we can draw these key conclusions: • Successful use of components requires planned structure for those components. • Assembling components together in an ad hoc way just doesn't work. • If we want to be able to plug in components we must have something to plug them into. • Flexibility, while desirable, must be constrained. We have seen that the structuring can come from architecture, design, patterns, planned flexibility points, and replication. When talking about components in the earlier chapter, we also introduced another concept, that of frameworks. Frameworks are the core of an application built from a set of components designed to work together to deliver function in a particular domain of interest. The framework is partially complete and has plug points in which to fit other components that complete and extend the product. Frameworks provide flexibility, but provide order and constrain the degree of that flexibility. Frameworks are attracting a lot of interest in the software world, because they move away from the ad hoc assembly of components to a much more structured approach. We will look at them a bit more closely in a moment. However, the use of frameworks raises the issues of whether it also moves away from the vision, that is, being able to assemble components together in a different way to produce something new. It seems a contradiction to say that a framework of components designed to work together in a particular way to deliver a particular type of function could actually be assembled a different way to produce something different. The attraction of frameworks is that they deliver a solid, reliable, reusable core, but provide the flexibility we seek around the edges. We can also see that corporations will be prepared to buy into such frameworks, if built by wellrespected software houses, in a way that they will not purchase components downloaded over the Internet. So maybe the vision of software components can be achieved, it is just that it is phrased in the wrong way. We hope it is now clear that structure is vital for the successful assembly of components and that, pragmatically at least, the use of a solid core is a desirable starting point. The Butler CBD Forum [1] states:
Page 105 Components do not exist in isolation but within an architecture that provides context across a business or technical domain. For components to work together they need to have agreement on common concepts. Architecture is important not just to comprehend what the final application will look like, but also what the individual components are and how they will connect to one another.
It is now time to look at methods of structuring software in a bit more detail to see what they are, how they differ, and whether individually or in combination they can deliver both the structure and flexibility needed. Architecture or Design? We have used the terms architecture and design quite freely so far without explaining what they mean. Just as the term component is open to wide interpretation, there is also very little agreement about what architecture means. There is also a lot of misunderstanding about what is architecture and what is design. The concept of software architecture has been around for a number of years and can be traced back at least to the 1980s and beyond. Much of the early work on architectures has been in the academic world and, despite some focus in the military market, formal use of architecture has found little acceptance in industry. Interest in largescale architectures had mostly fallen out of favor; however, the surge of interest in components has given it a new lease on life. Unfortunately, there has been a tendency to label DCOM and CORBA as architectures and assume that is all there is to it. Even though people call these detailed technical standards component architecture, the same people, if asked, would consider architecture to mean some high level conceptual (perhaps logical) representation of how systems and software fit together. The Carnegie Mellon Software Engineering Institute [2] has an excellent Web page that surveys a whole range of different definitions of architecture and provides an extensive bibliography. Once again, we do not believe a protracted debate over the nature of architectures is of great value. It is more important to identify some key characteristics. But the fact that we can already see different perspectives starts to give a clue to what architectures really are. Our working definition is: An architecture is a set of views within a specific context showing how elements (components) can be arranged to fit together to form a functioning system.
Page 106
We mean system in the true and wide sense of the word: ''a complex whole; a set of connected things or parts; a set of devices functioning together" [3] rather than the more limited sense (i.e., much the same as an application) used in software engineering. The characteristics of an architecture we think are important are: 1. It describes the functional interrelation between the components and thus implies that a set of components will not work together (other than by trial and error) unless an architecture exists. This is a key point which we think is so often missed in discussions of componentbased approaches. 2. It describes, in detail, component interfaces, performance, constraints, and limitations. 3. There is not a single architecture for a system but a whole set of architectures taking different viewpoints (e.g., electrical, mechanical, functional, implementation, and so forth). These are many and varied. They may be hierarchical or overlap. There is no sense of the complete set of architectures for a system. 4. It shows how elements can and should be fitted together. It gives policies and rules and also includes options and alternatives. 5. It is not a highlevel overview, but an abstraction that is sufficiently detailed to provide flexibility without ambiguity. Each of the architectural views may be very detailed. 6. It is not a complete description of a specific implementation—that is the design. 7. Architectures will vary in size and shape depending on the project, organization, and purpose to which they are put. Essentially an architecture is a set of design patterns that show how things can be put together to build certain sorts of systems. A design, on the other hand, is a complete set of plans showing exactly how a particular system will be manufactured. It is that set of information that is necessary and sufficient to ensure that the correct thing is built to the specification. Think of our car example earlier. The architecture ensures that it is a car that is being built and in fact a certain sort of car. The design ensures that a specific car will be built. The design coupled with the order ensures that the car for a specific customer is built. We can see that there is an intellectual step, the design process, that takes us from the architecture to the actual design which is capable of being manufactured. A design may consist of many parts and in itself be structured; however, it must be complete and selfconsistent.
Page 107
In theory there is no intellectual step between design and manufacture. We expect that craftsmen can follow plans to manufacture and assemble using known skills and techniques. In many cases this could be automated. In mechanical and electrical engineering this would be familiar, but in software we are much less disciplined. We tend to give the manufacturers, that is, the people writing code, far more freedom in adding intellectual content. The benefit is that we can get much more innovative and rich functionality, but the disadvantage is that we get far more variation, errors, divergence from specification, and lack of control. It can be argued, and most software developers frequently do, that software is not the same as mechanics or electronics; it is more of an art than engineering. While this may have been true in the past, it is becoming less so. Advances in the power of desktop PCs are starting to see a resurgence of the CASE (computeraided software engineering) tools. These tools use a variety of modeling techniques that support the software engineer in producing the design within a given architectural style. Many tools will then automatically generate code, and some even components, to implement the design. Such tools hold the vision of allowing software engineers to concentrate their intellectual efforts in the design process rather than in code manufacture. A designer may not be aware of explicitly referring to or using an architecture while doing a design, but it can be argued that such an architecture must always exist. A survey of early implementations of componentbased techniques in British Telecommunications plc showed that none of them had formal architectures in place, but all knew how their components fitted together. The problem with not having an explicit architecture is that it will not be easy, or even possible, to fit together components produced by different teams. This is where many developers will claim adherence to technical architectures such as DCOM, CORBA, or Java Beans to solve this problem. So far these developers have been concentrating on the nuts and bolts of just getting the components to fit together. Adherence to these technical architecture standards is starting to guarantee that the components will fit together at a plumbing level and this is certainly a major step forward. However, it still leaves the problem of understanding how the components will functionally interwork. Thus we currently have the syntax but not the semantics of interoperation. We saw in Chapter 3 that one of the main reasons that software reuse has not achieved the success hoped for was that, in all but the simplest of components, the explicit information defined in the interfaces is insufficient to get components to work together. There is an architectural mismatch [4] between assumptions made by the designers of one group of components about how they will fit together with those from another group who have different assumptions. We believe this is one of the main problems that prevents the
Page 108
assembly of systems from a large number of independent components and currently limits the scale of commercial components. Architectural Styles In the process of creating the design, various decisions have to be made about the way the design will be implemented. For instance, will object orientation be used, will it be based on a CORBA or Java Bean distributed model, will client/server or transaction processing be used, is real time used, will a state machine approach be used? While these decisions are implementation decisions, they are also architectural decisions and will constrain the choice of architectural views that can be taken. Often referred to as architectural styles they are very useful in guiding the decision maker through the vast range of design options that are available. Mary Shaw describes styles [5] as those features and rules that ensure architectural integrity is preserved. In the same book, Bass et al. define architectural styles as: A description of component types and a pattern of their runtime control and/or data transfer.
We might prefer to be a bit more general and define styles as sets of patterns that determine how the architecture will be implemented. These may be design patterns, process patterns, or patterns defining technological infrastructure. In achieving this, we would want integrity to be preserved, but this is not the fundamental aim in any rigorous academic sense. The styles make use of previous experience, well understood techniques, best practice, as well as current standards and fashions. Researchers at Carnegie Mellon [6] define an architectural style as typically comprising: • A vocabulary of design elements—components, connectors, interfaces, databases, and so forth; • Design rules and constraints; • Semantic interpretation of component composition; • Analyses that can be done on systems built to the style. As well as promoting good practice and reuse, they also aid understanding and agreement, providing a common language for discussion and comparison. Part of the design process is also concerned with production engineering, that is ensuring that not only does the design meet the specification but that
Page 109
the design can be implemented and, moreover, with the tools available. This is a familiar concept in mechanical and electronic engineering where often a separate production design team will be responsible for addressing all such issues. To some extent production design issues may be addressed by architectural styles and often there will be specialist teams that consider testing techniques, performance analysis, and software reliability. However, it is rare for a specialist team to be assigned responsibility for ensuring that all nonfunctional issues are explicitly addressed and resolved. More likely, the development teams will be asked to look after these issues and as a result they are often neglected or performed only perfunctorily. We believe this is a shortcoming of current software engineering practice and it is essential to consider such issues when taking a componentbased approach. It would address many of the nonfunctional issues concerned with choosing components: reliability, availability, cost, reputation of supplier, and how easy is it to understand what the component does. These issues have nothing to do with the architecture or functional design, but are vital to ensuring successful implementation. Views of Views We have suggested that architectures are views. They should be detailed and there can be any number of them. We also suggested that there is no concept of a complete set of views. That doesn't seem very promising if we want to use architecture to impose some sort of order on software development. Can we therefore identify a useful set of views that, while not complete, would be adequate for most applications? Looking through the literature there is no shortage of suggestions. Here are just a few of the architectural views that various authors have proposed: Business
Logical
Process
Functional
Application
System
Physical
Technology
Conceptual
Software
Operational
Data
Standards
Nonfunctional
Communications
Component
Development
Client/server
Infrastructure
Security
Network
This is by no means a complete set, but in Figure 5.4 we have tried to show the relationships between some of them.
Page 110
Figure 5.4 Architectural views.
Architectural Layers We can see that some views of the architectural views shown above form an elementary hierarchy (see Figure 5.5). • Business architecture—business functions, organization, processes described at a conceptual level. • Application architecture—mapping of the business architecture onto the software and systems that will implement them. This layer provides the basic infrastructure of the organization. • Technology architecture—the very detailed level of component and hardware infrastructure that underpins the applications infrastructure. The component execution environment. Ideally the layers should be as loosely coupled as possible. There should be no coupling between the top and bottom layers while adjacent layers will have a higher degree of interdependability. Decisions made at the higher business layer should be supported by the lower layers, but not unduly change them. For instance, a change to a business process should be implemented by the applications architecture, but not require a change to the technology infra
Page 111
Figure 5.5 Simple hierarchical model.
structure. Similarly a decision to change the technology infrastructure should not make any difference to the highlevel business model. Other architectural views pervade the whole model (data, nonfunctionals, and so forth—we didn't even try to show security maps!) while some views seem to be different sorts of things altogether (operational, development). The complex relationship between these views presents a significant challenge for componentbased engineering and the methods used. Some of the architectural views clearly underpin the model and function as foundations, for instance the component technology used (DCOM, CORBA). Provided these standards are agreed upon and adhered to, the development teams can build their components relatively independently. Yet other views, for instance development environments (tools, methods), are specific and internal to certain communities. Other communities could, in principle, choose a different set of standards without affecting the rest of the architecture. Although this is true in principle, in practice we often see the problem of architectural mismatch, mentioned earlier. The methods and tools used to build components tend to build implicit assumptions into the component design. These assumptions are not explicit in the component interface definitions which lead to interworking problems later. Although it does not eliminate architectural mismatch completely, it is much easier to do componentbased engineering if a common development architectural style is adopted. Thus when looking for commercial components, as well as seeing domains of interest around certain types of components, we also see component domains centered around a development method or tool. There is a thriving market in components built using Sterling Software's Cool: Gen CASE tool because of the componentbased focus of the tool itself and the modelbased approach that underpins it.
Page 112
Architectural views, such as security, error handling, and data, pervade the whole of the architectural model. This requires that component builders must adhere closely to the model to ensure that functions like security interwork properly throughout the model. Such functions are clearly common to many systems and it is very tempting to try to build security components. Some people claim to have done this, but while it is certainly possible to build logon components and components for other specialized security functions, it is not easy to partition system security neatly into separate components. Component decomposition of error handling is even more of a challenge. A more effective approach seems to be to build such infrastructure type functions into a framework into which components plug. The framework effectively wraps up the component with a surrounding layer that handles secure access and traps error conditions. We will come back to this later. Other Architectural Models Many authors and organizations have tried to simplify the picture we presented in Figure 5.4 by grouping the architectural viewpoints together under various headings. Layers of Generality Jacobson [7] and others often draw the model in Figure 5.5 a different way with the business layer in the middle and the application layer at the top (see Figure 5.6). Drawn this way the model represents generic components at the bottom with increasing specialization at the higher levels. The system utility components would be widely used and represent global reuse as we described in
Figure 5.6 Layers of generality.
Page 113
Chapter 3. In this model the business layer represents common business components that frequently occur within particular business domains or even across domains and examples of local reuse. The application layer represents components that are specific to the application in question and are not used anywhere else. The application may represent a specific commercial package that is to be supplied or a purpose designed system for a business enterprise. There are many varieties of this sort of model, often with many layers. Such a model can also be drawn for specific domains showing the type of entities that are meaningful for that domain. Butler Model The Butler Group's CBD Forum [1] also subscribes to a similar threelayer model. Butler takes this basic model and considers how components in these layers might be purchased from vendors, any necessary customization and configuration implemented and then reused within the business. These lead them to an interesting pyramid model as shown in Figure 5.7.
Figure 5.7 Butler approach to build and buy. (Source: Butler Group CBD Forum.)
Page 114
We can see that the architecture of the core infrastructure is largely built from smallscale general purpose commercial components with a limited amount of special purpose and inhouse built components. Nearer to the top of the pyramid, in the specific business domain of the enterprise, there is far more use of custom built components while the more specialized applicationspecific areas are almost all custom built. At the current state of maturity of componentbased approaches, the middle layers of this architecture are much the same as the very top layer, that is almost all custom built. It is only at the very bottom layer where commercial components are used in great numbers, mostly GUI widgets and components of a similar granularity. Some largerscale components are starting to find their way into the business operational layers, but wellarchitectured business domain components are waiting for largescale integrated business frameworks to become established, as Butler shows in Figure 5.8. Butler's view is that many of the key infrastructure components will become commodity items. Things that were specialized, custombuilt items only a few years ago have now become part of the basic infrastructure of most systems. Items that were once business differentiators and gave their builders competitive advantage become, over time, business as usual. Vendors see the opportunities to produce such items in quantity and thus they in turn become commodity items with the corresponding reduction in cost and improvement in quality (see Figure 5.9). Thus over time the march of commoditization spreads further and further up the pyramid. Of course, we might think that we can extrapolate the curve to the limit where all software is produced as components and simply assembled into the required system. That is certainly the vision of CBD, but probably we have all come across those infamous extrapolating predictions of fledgling technologies that have never come true. What happens is that technology does not march endlessly up a steady growth curve, but moves in fits and starts as new ideas and inventions bring in radical changes. The Rational 4+1 View The 4+1 View from Rational Software Corporation [8] describes a software architecture as consisting of five views: 1. Logical—typically an object model or entity relationship diagram; 2. Process—threads, tasks, synchronization; 3. Physical—mapping of software to hardware including distribution;
Page 115
Figure 5.8 Butler model of market for components. (Source: Butler Group CBD Forum.)
Page 116
Figure 5.9 Component commoditization.
4. Development—static organization in development environment plus; 5. Use case scenarios embodying the four views above. Each view in Figure 5.10 can have its own particular architectural style which allows several styles, although presumably only four, to be present in the architecture. Each of the views addresses a particular area of interest and the fifth view, the use case scenarios, is used to show how the other views fit together and to validate them against the scenarios. • The logical view describes the design's object model when an objectoriented design method is used. Alternative approaches from 00 can be used where appropriate. • The process view describes the design's concurrency and synchronization aspects. • The physical view describes the mapping of the software onto the hardware and reflects its distributed aspect. • The development view describes the software's static organization in its development environment.
Page 117
Figure 5.10 Rational 4+1 model.
• Scenarios capture the system's critical functionality—functions that are the most important, are used most frequently, or present significant technical risk. Rational describes their view as being rather generic because it is possible to use other methods and notations. Also it is not always necessary to use all four views and views can be combined if they are similar. Architectural Structures Walter Kriha, Daniel Kesch, and Stephan Pluess [9] claim that a large system can be decomposed into several categories of software structures. The most important structures of a system's architecture: 1. Analytic structure: decomposition of the system into a set of key abstractions and the consideration and weighing of all other architectural structures and the consequences of their impacts upon each other. 2. Logical structure: decomposition of the problem without regard for physical implementation. Defines the generic constructs within the system. 3. Physical structure: the physical things in the system and their relationships with each other. 4. Social structure: defines how the development team is to be organized and how the roles within the team are to be allocated.
Page 118
5. Reflective structure: captures the selfdescribing aspects of the system in terms of metadata, team knowhow, or an interface and implementation repository. 6. Extension structure: encompasses everything that is needed for the system's customization and extension. 7. Source code structure: governs how the development team's entire code base is managed and administered. 8. Generation structure: defines the automatic construction of the system, its parts, and its configuration. 9. Usage structure: determines how the developer's use what and when, and how reuse is to be implemented. 10. Runtime structure: defines how the system is to be set running and how it is to behave over a period of time. This is a very interesting set of structures. Some of them are clearly architectural structures (1, 2, 3, and 10). Some of them look much more like architectural styles (8 and 9). Some of them are not really anything to do with architecture or implementation as we have viewed it here, but are very localized issues (4 and 7). Nevertheless they are all important issues and the example serves to show how closely architecture, implementation, and organization are closely intertwined. The Open Group Architectural Framework (TOGAF) The Open Group Architectural Framework (TOGAF) recommends a set of views, some or all of which can be used to aid an organization in a particular architecture development. The views [10] can be used to crosscheck that the architecture will meet the needs of the system and, in particular, can be used to compare the architecture of an existing system with that of a proposed enhancement to see what needs to be added or changed. • Function views: what the system is intended to do: • Functional: operational aspects of the system. • Implementation views: how the system is implemented: • Management: how the resulting system can be managed. • Security: security aspects and protection of information. • Builder's: aspects of software development. • Data management: storage, processing, archiving, and security.
Page 119
• User: usability aspects. • Physical: location, type, and power of the equipment and software: • Computing: assembly of software and hardware components. • Communications: structuring to simplify network planning and design. DomainSpecific Architecture We have mentioned several times the idea that certain aspects of architectural models are specific to particular domains of interest. It is therefore not surprising that complete domainspecific software architectures (DSSAs) have been developed to support well understood business or industrial areas of interest. There are many examples of very specific domain architectures (e.g., military systems, logistics systems, global information systems, DBMS systems, workflow, and knowledgebased systems). The United States Department of Defense (USDOD) has a strong interest in DSSAs and related architectural description languages as they see them as important technical foundations for systematic reuse [11]. Perhaps one of the most advanced areas is the Telecommunications Information Networking Architecture (TINA). TINA is a telecommunications domainspecific open software architecture developed by a consortium (TINAC) of over forty of the world's leading network operators, telecommunications equipment, and computer equipment manufacturers. It works to integrate the World Wide Web, multimedia, and current computer technologies with the more traditional and mature telecommunications technologies. We describe it briefly in Appendix B. Architectural Description Languages Much of the academic interest in architectures has centered around producing architecture description languages (ADLs) for defining and analyzing architectures. However, these have had very little impact on the software industry. A brief summary and references to some key ADLs can be found in [6]. The United States Department of Defense has a number of important initiatives that are starting to address architecture representation issues. They see ADLs as the basis for being able to share and assess architecture descriptions across DOD projects. They are currently undertaking research to look at the extent that these techniques are ready for use and what further work is needed [11].
Page 120
Structure in Architectures We have defined architectures as being viewpoints showing how elements can fit together to build a system and we have discussed the sorts of viewpoints that would be useful. How do we describe those viewpoints? We have introduced the idea of architectural description languages as one way of describing architectures, but these are yet to find mainstream use. The methods in most common use for describing architectural structures are patterns, frameworks, and, more recently, product lines. We have mentioned them a number of times already and now is the time to look at them in a bit more detail. Patterns The concept of design patterns was invented by Christopher Alexander to simplify the problem of designing buildings [12]. Although on the face of it there is a great variety in buildings, in practice the same common use of space occurs time and time again. For instance, meeting rooms, entrance lobbies, restaurants, and so forth. As with all good ideas, its seems obvious now that there should be standard templates that could be used to design such structures. Alexander defines a pattern as: A relationship between a certain context, a certain system of forces that occurs repeatedly in that context, and a certain spatial configuration which allows those forces to resolve themselves.
Thus a pattern consists of a solution to a particular problem in a certain situation. Somewhat later the same ideas were applied to software engineering in the now seminal book by Gamma et al. [13]. Essentially, patterns capture the solution to common and recurring problems—not necessarily the specific implementation, but more the way of going about it. This can be a technical way of implementing a software solution or a pattern for the process of solving a problem. Patterns are therefore reusable templates which build up a body of engineering knowledge in a way that is readily assessable, and in some cases patterns, if described in an appropriate way, may be formally provable. Patterns can also be applied to the business domain, for instance, a pattern for doing business or providing a service. Unlike many flavorofthemonth new software engineering techniques that come and go, the strength of design patterns is that they capture wellproven experience and best practice. Most work on software design patterns has been focused on OO, but there is nothing inherent in the concept that limits patterns to OO. One of the
Page 121
main benefits for use in OO is that patterns allow complex interactions between classes to be specified in a way that normal class models do not facilitate. For a componentbased approach, this can be a mixed blessing because we do not necessarily want to encourage complex interactions between our components. It may well be that patterns will find most effective use when handling smallscale components, where there is likely to be a lot of interaction rather than largescale components. Patterns may be discovered in a number of different ways. Norman Kerth and Ward Cunningham [14] classify three basic approaches: • Introspective: analysis of patterns from a single system; • Artifactual: analysis of systems built by different teams in a similar domain; • Sociological: analysis of recurring problems and experience of developers building similar systems. In addition to providing an outline solution to a recurring problem, the use of patterns provides a common way of describing the problem and the design techniques used in the solution. This starts to overcome one of the major barriers to successful component reuse, that of having a common language in which to explain what a component does and the specific context in which it is intended to be used. The way of going about things is key to our earlier description of architectures so it can be very useful to think of an architecture as a set (or system) of related design patterns. Thus patterns also provide a common way of describing a software architecture. We also said earlier that while an architecture should contain detail, it is not a specific implementation. This idea also fits well with the idea of patterns. As their name suggests, they are patterns or templates for a solution, not an actual implementation. There are now a number of books [13, 15, and so forth] that describe many of the common patterns that recur in software engineering, although, surprisingly there are no actual electronic libraries of patterns that we are aware of. One way of thinking of patterns is as a predefined set of components and the interactions that provide a solution to some particular problem. A very common pattern is the modelviewcontroller pattern [16] used for simple systems with a user interface (see Figure 5.11). The model is the core of the application including the main function and data handling. The view is the part of the system that displays the data, and the controller is the part of the system that responds to user input, signaling to the model to undertake some function or to the view to change the display. Each of the three elements (model, view, and controller) can be thought of as components
Page 122
Figure 5.11 Modelviewcontroller pattern.
and thus allow investigation of the interactions between them. At this point one might ask ''Is this not the same as a framework?" At first sight it seems that it is, as we also described frameworks as a set of related components. This difference, however, is in the degree of abstraction. A pattern is the conceptual set of components while the framework is an actual set of components based upon a particular technical solution or architectural style. Thus the model viewcontroller pattern might well be implemented using the Microsoft Visual Basic framework, making use of a wide range of VB custom controls or GUI components. The components defined at the architectural level could well be thought of as logical components that are mapped onto real framework components during the design stage and implemented in a particular technology using the chosen architectural style: • Pattern: conceptual level, logical components; • Framework: design level, actual components; • Architectural style: implementation level, low level components, and parts. One of the key attractions of a framework is that it provides the bulk of the implementation for a wellunderstood domainspecific pattern or set of patterns. However, things are not that simple as patterns can themselves be used at different levels of abstraction. Buschmann et al. [15] describe three categories of pattern:
Page 123
• Architectural patterns: An organizational structure for a software system. A set of predefined subsystems with rules and guidelines for their interrelationships. They work at the architectural level to help the decomposition of a system and avoid the sea of components problems. • Design patterns: The structure and relationships of components (or further refined subsystems) in a system. They do not affect overall architecture, but define architecture of subsystems and components, and continue the structural decomposition. • Idioms: A pattern for implementing a solution in a specific language with a certain style. Effectively, it is an implementation pattern, with a similar level of abstraction to framelets. Patterns can be taken beyond the idea of solving the software problems themselves to problems of how to produce the software itself. Thus process or organizational patterns embody methods of software development. For an excellent summary of the historical development of patterns and an extensive bibliography, see [17]. AntiPatterns Perhaps it is part of human nature that we like to dwell on things that go wrong. After all, holidays would not be enjoyable if we did not have something to complain about! Nevertheless, as the quote from Brooks at the start of the chapter indicates, it is always possible to learn something from previous failure, and antipatterns are templates of those things that should be avoided. Antipatterns were initially proposed by Andrew Koenig from Bell Labs, who described two types of antipattern: (1) a bad solution to a problem which resulted in a bad situation, and (2) how to get out of a bad situation and how to proceed from there. The philosopher Karl Popper argued that true objectivity in science is only attainable if scientific theories are capable of being falsified. In much the same way, it is possible to test the design of a system by showing that it does not contain antipatterns. Pattern Systems and Languages Design patterns are not just used by themselves, but together with other patterns and according to certain rules. Such a system of patterns (or patterns of patterns) can be described by pattern languages [12]. Buschmann et al. [15] prefer the term pattern systems as they suggest the term language implies a mathematical
Page 124
completeness which rarely exists. They suggest that the term pattern systems implies collection of patterns and rules for implementation in software architecture. We similarly suggested earlier that an architecture is essentially a set of patterns and, perhaps more strongly, that a pattern system is not "for" an architecture, but perhaps "is" the architecture. Alexander [12] rather elegantly says: A pattern language gives each person who uses it the power to create an infinite variety of new and unique buildings, just as his ordinary language gives him the power to create an infinite variety of sentences.
Norman Kerth and Ward Cunningham [14] describe pattern languages in a slightly different way as embodying both the architecture and the philosophy of a design. Kerth and Cunningham discuss the relationship between the size of a project and the architectural structure needed to support it. In single project teams, the architecture is in the designer's head, and the mapping between architecture and design is an unconscious process. As teams become larger, the architecture has to become more and more explicit and more rigidly enforced. Once the teams become large enough to require people to undertake specific architectural roles, then it becomes increasingly important to communicate the philosophy behind the architecture. Kerth and Cunningham suggest that patterns start to play a valuable role in communicating solutions to common design problems. Once designs have to be communicated across unrelated teams, then communicating the philosophy must become more explicit. Here Kerth and Cunningham suggest that pattern languages can communicate both the architecture and philosophy and thus "minimize the chaos that results when many people work independently on a single system." Kerth and Cunningham believe that much of the problem with patterns as currently used in software engineering is that they are used in isolation. To build large systems, collections of patterns must be used that reflect the philosophy of the overall design intention and the chosen architecture. Buschman et al. [15] give detailed guidelines on how to compare and implement pattern systems. An interesting concept from Michael Beedle, author of "Reengineering the Application Development Process," in [17] is of the "effects of using pattern languages to be the generation of emergent behaviours: spontaneously recurring patterns of dense local interaction between entities, resulting in dynamic, selforganizing systems that are adaptive, open, and capable of multiscale effects." The concept of behavior emerging in a deterministic, but unpredictable, way from the complex interactions of biological and social systems is well known. For instance, there is a growing, albeit controversial, body of thought that human consciousness arises from emergent behavior of the extremely complex interactions of the billions of neurons in the brain [18]. One might expect,
Page 125
as Beedle suggests, that we could see behavior emerging in complex software systems. However, although it can be argued that neural networks and genetic algorithms exhibit emergent behavior, the authors cannot think of a single example of a large software system with positive emergent behavior. Of course many of the problems and bugs that occur in large systems may be thought of as examples of negative emergent behavior. Beedle then goes on to say: The patterns and rules in a pattern language combine to form an architectural style. In this manner, pattern languages guide system analysts, architects, designers, and implementors to produce workable systems that solve common organizational and development problems at all levels of scale and diversity.
We think that is cheating a bit. By bringing the social interaction of people into the model, he cannot really claim the behavior is emerging from the pattern language, but it is a nice idea. Some authors get a bit carried away with the concept of pattern languages, describing them as artful tapestries, gestalts, ecosystems, and so forth. Even Alexander can't resist the temptation to say "the kind of order that is achieved when there is a perfect balance between the needs of the parts and the needs of the whole." However, there is no doubt that pattern languages will gain in importance for providing a vocabulary for sharing the concepts and complexity that are necessary in putting together large systems. Frameworks The concept of frameworks arose from the objectoriented world where the term depicts a group of cooperating classes that form a reusable infrastructure. The framework can be customized for a particular use by generating subclasses of the abstract classes in the framework. An obvious example is the GUI frameworks, which are the most common elements currently reused by software developers. Frameworks were intended to be domain independent, describing those functions, like GUI, that are common to many applications. Often designers will have a set of classes that implements something in their application that they use time and time again. This might be an assembly of GUI controls that does something specific, but which is unlikely to be of use in any other domain. These are a type of framework, but are often given different names: Framelets [19] or Kits [20]. The concept of frameworks has developed over time, moving away from necessarily being OObased and referring to large collections of entities. In
Page 126
Chapter 3, we described a framework more generally as "A partial product assembly that can be completed, extended, and customized to meet a customer's exact requirements but is based on a common core." Perhaps the best known example of a framework described in this more general way would be Visual Basic. This larger view of frameworks tends to target specific domains or types of problems. The extensibility of frameworks is one of its key benefits. Because of the strong structure embedded in the framework, it is much easier for users of the framework to understand how to extend it and add their own functionality. Particularly attractive is the ability of some frameworks to allow other software modules and components to be easily pluggedin in a welldefined way. In some very specific markets, products have evolved (e.g., Autocad, or Adobe Photoshop) specifically to allow a thirdparty plugin market to develop. This symbiotic relationship both supports the thirdparty plugin vendors and provides enhanced sales of the core tool. Frameworks are also very attractive in implementing realworld applications. Theoretical views of components (and objects) require a strong adherence to the concepts of encapsulation and data hiding. In practice most business applications are strongly driven by corporate data, which is often contained in large legacy systems. While it is possible to wrap legacy functionality and provide data services to encapsulate data, putting together a largescale architecture that rigorously maintains this separation is often impractical. The end result is that shortcuts are taken which compromise the architecture and often make component replacement very difficult in practice. The use of frameworks allows components to be defined that cooperate to provide a set of functions that would be much harder to achieve using truly isolated individual components. Another attraction of thinking of a framework as an infrastructure into which components plug is that it offers a sensible way to deal with issues such as security, transaction processing, and error handling. We saw earlier that these issues pervade all layers of the architecture, and it is difficult to deal with these issues in a truly independent and componentlike way. By building these facilities into the framework, it is possible to have a systemwide approach with a clearly defined set of standards to which the components have to adhere. In many cases the components are wrapped by the framework, which alleviates component designers from having to worry about these detailed issues (see Figure 5.12). In many ways, the framework is now acting in the same way that distributed component execution environments (e.g., CORBA and DCOM) do in providing the lifesupport mechanisms that components need to operate. To some extent this allows us to achieve what Szyperski calls nonstrict layering while retaining clearly defined components boundaries [21]. The way we have drawn the component interfaces in Figure 5.12 could be interpreted to mean that a component has more than one interface, a func
Page 127
Figure 5.12 Framework with infrastructure layers.
tional interface defines the highlevel operation, and one or more other interfaces handle the infrastructure aspects of connecting to the framework. The multiple interfaces do not necessarily have to be individual physical interfaces, but could be logical interfaces over the same physical interface. However, we do find some attraction in the idea of separate interfaces. This could allow a range of components to be produced that have the same functional interface, but different infrastructure interfaces targeted at the different frameworks they were intended to plug into. Brown and Wallnau [22] report that the role of functional and extrafunctional interfaces was discussed in a panel session at the 20th International Conference on Software Engineering. The panelists decided that the concept of two interfaces unduly emphasized the role of frameworks in software architecture. While from an academic point of view, this may well be correct, we believe in practice it will be the use of frameworks that drives architecture. Vendors will have to provide different variants of their components in order to support very different framework standards. We already see this to some extent with widgetsized components. The use of multiple interfaces could well allow this to be managed more cleanly. We have discussed several times already the problems of architectural mismatch and why we do not believe that system builders are likely to assemble systems from a large range of individual, isolated components from many vendors. The level of understanding required would be much too great. However, by providing frameworks of cooperating components, the level of understanding required about how each individual component works is much less. That
Page 128
knowledge is effectively built into the framework. Users now only need the highlevel knowledge about how the business functions operate and knowledge about how the flexibility points in the framework allow other components to be added. If a developer should wish to know in detail how components within the framework operate, then the framework itself provides contextual knowledge that will aid understanding. We were keen to point out earlier that a framework is actually a set of components, based upon a particular technical solution or architectural style, that implements a system of design patterns. Some people describe a framework as a miniarchitecture or even as a fullblown architecture. We said earlier that an architecture was a set of abstract views, not the detailed implementation, so on this definition a framework is not an architecture itself, but the implementation of one. Nevertheless, because a framework is openended and allows extensibility through the flexibility points or plug points, it still retains some of its abstract nature. Thus a framework sits somewhere between an architecture and a fully detailed design and provides a practical way of providing architectural reuse. This, of course, is its attraction. Many developers who would vigorously deny ever being near an architecture are perfectly happy to use a framework. By choosing to use a particular framework, the architecture and architectural style of a design is dictated. This is, again, a good example of design reuse, relieving the designer of the need to go step by step through a long and detailed decisionmaking process. The framework has already captured the results of design decisions and constraints already determined by people who are experts in the domain. A system is typically not just one framework, but a collection of domainspecific frameworks, with an overarching framework into which they fit. Thus we can conceive of a hierarchy of frameworks, maybe system frameworks and subsystem frameworks. This is still a very immature area, and it is difficult to decide if we mean a hierarchy of frameworks, or a pattern of frameworks, or possibly just a single framework with a hierarchical structure. Just to complicate things further, Szyperski describes how frameworks can be described as White Box or Black Box depending on the way in which they are reused. Product Lines and Product Families We have previously introduced the concept of product lines to describe a set of related products that are built from a set of common components. Work from the SEI Reuse and Product Lines Working Group at WISR98 [23] has produced a slightly more specific definition that distinguishes between a product line as the set of related products that addresses a common market segment and a product family as the set of related products based on common components.
Page 129
Clearly they do not necessarily imply each other, but in practice it makes the most sense when the product line is delivered by a product family. The power of the product family emerges when a company can use knowledge of its market to structure its products into components that can be mixed and matched to deliver a range of products and variants that exactly match the market's needs. The architecture is strongly driven by commercial considerations and enables real reuse to be achieved. The benefits come not just from the cost savings from reuse, but from the ability to put together a costeffective range of products that can be sold into the market at the right price and at the right time. This is clearly not the ad hoc use of available components, but the use of very carefully designed components in a domainspecific way. The SEI workshop presented a number of case studies with impressive results: a product family with 55 variations, 100–300 separate programs with over a million lines of code each, and 80 percent reuse. A Swedish company, Axis Communications AB [24], has produced an interesting range of printer, storage, and scanner server products using objectoriented software techniques. They use a hierarchical product line architecture where the reusable assets are frameworks of varying sizes. It is also an interesting example because each product is built, not just based on a single framework, but on a set of frameworks. Typically the components in a product line will reflect only the variation required across the products. However, it may be considered more useful to expand the architecture to include components that will implement all the functions within the products. Another Swedish company, Securitas Larm AB has successfully implemented this approach [24]. The case study of the Swedish product lines experience gives a detailed review of many of the problems experienced and some of the potential solutions. The WISR98 [23] workshop also considered what discriminating factors might determine whether any particular organization might successfully implement a product line. Organizational factors included issues about domain knowledge, legacy systems, and funding. Environmental factors included scale of components, stability of domain, customer base, and stability of market. However, there is one significant factor they did not consider, which is the relationship between the product line and the organization's business systems. The successful proponents of the product line approach seem to be those where the product concept is distinct from the operation of the business that produces the product. For instance, a motor car manufacturer will have billing systems, payroll systems, computerized manufacturing systems, and so forth. The organization of these systems and the software that runs the business has no connection at all with the architecture of the products that it produces. Even in a company that produces software, the software that is the product will be distinct from the software used to run the business.
Page 130
Consider now the business the authors are involved in: telecommunications. In the past, telecommunications was all about hardware: copper wire, switches (telephone exchanges), transmission links, and so forth. Those things all still exist, but these days the competitive edge is delivered by software: networkbased services, Internet services, intelligent call routing, itemized billing, and so on. The products are delivered by the operational support systems (OSS) that operate the business. It is true that some of the systems: payroll, finance, and human resources are distinct from the product, but most other systems are not. Each time a new networkbased product is launched, changes are potentially required to billing, fault tracking, customer handling, network management, and so forth. It is not unknown for products to have been released before a mechanism is in place to bill for them! Is it possible to operate a product line architecture in this environment? There are two key challenges: First, to get senior management, particularly the marketing department, to see that support systems are a key part of the product rather than an unnecessary and costly overhead which is best contracted out, and secondly, to find a way of componentizing the product line in such a way that it maps onto the system components. To use the SEI Workshop definitions, the difficulty is mapping the product family onto the product line. In the more conventional examples given, that mapping is relatively straightforward. Choosing the right components needs care, but is conceptually straightforward. In the more complex telecommunications example, the mapping is not at all obvious. There is little point in producing a very wellstructured and componentized product line if they have to be supported by a monolithic network of legacy systems that are difficult to change. It would seem that other utilities and financial institutions would have similar problems and would be useful places to look for potential solutions. In particular, the financial institutions to a large extent have led the way in pioneering the use of CBD. In practice the products seem to be a lot simpler and have been around for many hundreds of years. The concept of an insurance policy is in no way connected with software systems. It is true there have been many advances in financial products over recent years, but they do not seem to be as complex and fast moving as those in telecommunications. The Software Engineering Institute (SEI) is promoting a Product Line Practice Initiative [25] to enables widespread product line practice through architecturebased development particularly the reengineering softwareintensive systems from a product line perspective. Stovepipe Architectures It is not yet clear if product line architecture will scale into complex organizations. If it does not, then the ability to rapidly create and deliver radically new
Page 131
products will be stifled. A typical response to this is to build a completely separate set of systems to support each new product. This is known as a "stovepipe," to represent the vertical stacking of separate functionality in an inconsistent way rather than the reuse of existing systems and components in a layered architecture (see Figure 5.13). The problem with a stovepipe, apart from the cost and time taken to develop it, is that in a complex domain like telecommunications, it is frequently impossible to operate an independent systems stack. Many of the emerging Internetbased products, linked as they are to networks, multimedia sources, and service delivery platforms, may require interfaces with almost every system the organization has. It is not just telecommunications that suffer from stovepipes. It is also very common in military systems. The US DOD uses the informal term stovepipe system to denote: closed systems that embody idiosyncratic compositions of thin and thick layers of various functions/services and that are inflexible to evolve, expensive to maintain, and do not interoperate very well and make sharing data hard.
War is a global business just like telecommunications! Business Component Architectures Although the term business component is often used, there is currently no standard for business components or widespread agreement about what the term means. Often confused with business objects, they have the potential to be the most important contribution to achieving largescale benefit from a component approach. Two definitions from wellknown vendors are: SAP A Business Component supplies a dedicated, encapsulated business functionality via stable interfaces. A component of this type has its own development, implementation and maintenance cycles. A number of these components can run on their own dedicated databases. Rational Software Corporation A business component represents the software implementation of an "autonomous" business concept or business process. It consists of all the software artifacts necessary to express, implement, and deploy the concept as a reusable element of a larger business system.
Page 132
Figure 5.13 A stovepipe architecture.
Page 133
Typically a business component is a deployable component that represents a well understood business function, that is, something that would be recognized by a business analyst as opposed to something understood only by software engineers. Thus it represents a level of component granularity where assembly of these components into a system, if it could be achieved, would represent the greatest added value in terms of simplicity and reuse. Business components might be generic, representing common entities such as customer, or domainspecific and aimed at particular business domains. They are likely to be made up of smaller grained software components or objects. We might imagine that large scale applications might be comprised of perhaps ten to twenty of such business components. At this scale of component, the interfaces will be extremely complex and the problems of architectural mismatch become great. Down at the widget level, it is possible to be extremely precise about what a component does. It may well describe using an interface description language (IDL) or a modeling notation such as UML. Clearly, describing the function of a largescale billing component in IDL would be a near impossible task. The use of UML is more tractable, but it remains to be seen if UML will both handle the scale and richness of describing the function of such systems, particularly systems that are very process oriented. Pipes, APIs, XML, and XMI One way of simplifying the interface between largescale business components is to define them simply in terms of a data transmission pipe through which a wide variety of messages can pass. Thus the interface defines the syntax of component communication, but says nothing about semantics of what the component does. Most largescale software applications define application programming interfaces (APIs) in this way. The technical problem of linking systems together is then relatively straightforward, and the issues of understanding operation and data format are left to software integrators. Often glue logic or even translation systems will have to be inserted between the systems to undertake the necessary translation of data. One approach to moving from a simple message which can in principle contain anything is to add semantic content to the message itself. For instance, instead of just sending a piece of data that holds a customer's name, the message will also contain ''metadata" that describes the nature of the data. The most simple form of this is just the addition of some information about the name of the data, its format (numeric, text, and so forth) and its length or value range. Arguably that is not really semantic information, but just a definition of the syntax, however, at least it separates the actual content of the data from the labels.
Page 134
The problem of separating data content from information about how to present the data is prevalent in the current use of HTML for WWW pages. The data content of a WWW page is buried in the markup language used for formatting the information on the screen. There is no label for the data and the only way to search for data on a WWW page is to hope that the author has put some meaningful text labels on the page. To try and overcome this problem, The World Wide Web Consortium (W3C) have defined an extensible markup language (XML). XML [26] describes a class of data objects called XML documents. It uses the HTTP for data exchange, but has a higher level metadata representation separating the definition of data from the presentation. Names can be assigned to various types of data, and relationships can be established between the data. An interesting aspect of XML is that to make it work, different domains of interest will define their own metallevel data models. At the very least they will need to agree on common naming standards. If done properly this will allow the easy interchange of data between domainspecific systems and possibly the automatic translation of data traveling between domains. A major step forward is that the OAG has adopted XML as the interchange format for its business object architecture. Microsoft is also supporting XML and is creating XML vocabularies, including one for software installation and one for personal finance applications. In the enterprise resource planning (ERP) world, SAP, Peoplesoft, and Oracle have already announced support for XML. Many people are hailing XML as the answer to everything to do with data description. It remains to be seen if it will really meet the challenge of such an important task or whether it is just a hypedup extension of HTML. More directly related to software components, the object management group (OMG) is defining the XMI Metadata Interchange format [27] as a proposed standard for interchange of object design and programming artifacts. It integrates XML, UML, and other interchange formats with the aim of enabling an easy interchange of metadata between tools and metadata repositories. It is intended to encourage collaborative software development of applications by distributed teams. XMI and XML may well be a useful step forward in setting the standards for exchanging data between largescale business components. However, it does not do away with the need to define a structure which determines if components connected together in this way will do anything meaningful. ERP Systems The current approach we described above is very much about software integration and not really about component assembly. It occurs frequently in the ERP
Page 135
domain. ERP applications (from vendors such as SAP and Baan) will literally provide a complete set of systems to run an enterprise out of the box. They handle key activities such as payroll, billing, human resource, customer handling, and so forth. Many companies make use of ERP systems, but also have to integrate them with existing legacy systems or specialized systems support areas not covered by the ERP system. The integration of systems with ERP is no trivial task and a host of thirdparty companies has grown up to support this market. SAP has openly published their business application programming interfaces (BAPI) to enable thirdparty vendors and integrators to link into their SAP/R3 product. Many vendors of other large scale business applications have also realized that the days of locking customers into proprietary systems and interfaces are coming to a end. Large global companies cannot build a credible systems strategy based on the products of just one vendor. Takeovers, mergers, and global joint ventures are the flavor of the month. Even if a company should successfully implement such a single vendor approach, it is almost certain that a change in the organizational structure of the company will force on it the need to integrate with applications and systems. However, even ERP vendors are seeing the potential benefits of using a component approach. Not every customer wants to buy the whole system so it is advantageous to the vendors to be able to just provide the parts the customers want. Both SAP and Baan are committed to using a component approach to building their systems. Customers also have the ability to buy an ERP core and then plug and play the key components of their choice. Of course, it remains to be seen if the ERP vendors will want or be able to provide customers with as much commercial and technical flexibility as this. Baan is known to be developing a component approach to its product. SAP is a founding member of the OAG and has committed itself to making all relevant BAPIs compliant with the OAG's integration specification (OAGIS). The largescale use of SAP/R3 and strong links with the OAG may well lead to the BAPIs becoming de facto standards in the ERP business framework market. The Open Applications Group (OAG) The solution to application integration problems will only come through partnership and open standards. The OAG has been set up as a nonprofit making consortium of application software vendors to create common standards to enable the integration of enterprise business applications. They are currently tackling applications integration at two levels. • OAGIS—Open Applications Group Integration Specification; • OAMAS—Open Applications Group Middleware API Specification.
Page 136
The middleware API specification (OAMAS) is a proposal [28] for a common way of connecting business applications to each other at the technical level. It is not built on a specific architecture or middleware, but the OAG describes it as "technology sensitive, but not technology specific." It will form a layer on top of distributed component environments such as CORBA and DCOM. The integration specification (OAGIS) [29] is targeted at a higher level at integrating the business objects which contain the main business functions that occur within an enterprise. The OAG has created an architecture termed the business object document (BOD) to provide business object integration across heterogeneous environments, including multiple hardware platforms, operating systems, databases, middleware, and so forth. The OAG says that although their architecture is very objectoriented, it "does not require classical objectoriented technologies to be implemented." The OAG does not suggest, at least at this stage, that OAGIS will enable plugandplay interoperability. They say that business objects can be more difficult to integrate than other types of objects because of the various data architectures required for differing business objects. Each business object is not discrete, but part of a larger, implied business data model and process model. Their technical architecture is intended to be technologysensitive, but not technologyspecific. OAGIS defines business processes that are used for communication between OAG compliant software so that users can be assured that information sent from one business application has been received and successfully processed by the receiving application. The OAG also aims to standardize the core functionality of business processes. Processes unique to a particular implementation are mapped individually, yet will use the OAG architecture to communicate. A metadata architecture is used for building the BODs based on the OAGIS Specification as well as XML. Business Frameworks The concept of frameworks we discussed earlier is particularly attractive for the delivery of business components. Relatively highlevel business functions are mostly well understood, but will be difficult to implement as isolated large grained individual components. Breaking them down into much smaller commercial components then implies a much higher degree of understanding of the operation of each component and the context in which it is to be used. As we have already seen, frameworks embody within them much of the architectural knowledge needed to assemble components that have been designed to fit within the framework. What the software industry is looking for is business frameworks which will set the industry standards. Vendors who build com
Page 137
ponents for these frameworks will be assured the components will fit together, that the function will interwork with other components, and that key infrastructure services (for instance, security, error handling, transaction processing, and so forth) will be handled. Currently there are no independent business frameworks. A number of application vendors are in the process of establishing their own frameworks including: • SAP BAPI; • IBM San Francisco; • Microsoft Windows DNA Industry Frameworks; • Baan Series; • Andersen Consulting; • iFRAME. The frameworks from these vendors are very different sorts of things. At the top level, the SAP Business Framework is a componentbased product architecture that is the basis for the SAP R/3 enterprise applications. Widely used by thirdparty vendors and integrators, this framework represents the largest of business components. IBM's San Francisco office (see Figure 5.14) is trying to move objected orientedbased components up the value chain to build complete applications. Based on Java, it comprises a foundation layer to provide platform independence upon which sits a layer of common business objects and the core business processes. Applications can be built on top of any of these layers. IBM has already developed some of its own applications, including Warehouse Management, General Ledger, and Accounts, but intends that endusers and thirdparty vendors will design applications using the framework. IBM claims that over 700 companies have already entered into licensing agreements to use San Francisco. IBM also supplies development tools and is currently migrating to San Francisco to support a Java Beans base. The development of business frameworks can be seen as a race between the ERP vendors trying to componentize their large applications and move down into the middle market verses the OO community trying to move up from lowlevel widgets to decent sized business components. The ERP vendors have the edge because they operate at a business level that is understood in the boardroom. Business object frameworks require a big investment, but they still seem a bit technical for businessoriented people. Will they meet in the middle? Is there room for both? Or will something new come along that changes
Page 138
Figure 5.14 IBM San Francisco architecture.
the picture altogether? Only time will tell, but this is certainly the area to watch to see if componentbased approaches make it to the "big time." Business Objects Are business objects the same as business components? Many people use the terms interchangeably, while others use the terms assuming people know exactly what they mean. We have discussed components in enough detail to make it clear that components are not necessarily objects, but that objects can be components. At the level of meaningful business function it is much more likely that objects of this granularity will be more like components, and we have seen that IBM's San Francisco framework makes extensive use of business objects. There is no formal standard for business objects. The OMG made a request for proposals for business objects in 1996 [30]. In it they refer to common business objects (CBOs) and to a business object facility (BOF) as the technical infrastructure into which business components would plug and play. The OMG defines a business object as: a representation of a thing active in a business domain.
It goes on to say: Business objects can be viewed as Modelling Objects used in the design process (or BPR) and as (run time) objects in the information system.
Page 139
Business objects may represent common entities such as customer or be domainspecific. The OMG has also produced the business object component architecture (BOCA). This is an architectural template that specifies, but separates, application domain semantics from specific technology implementations. BOCA includes [31]: • Domain semantic model: metamodel describing interaction of domain objects, based on the OMG object model; • Component definition language (CDL): textual representation of the contract among cooperating domain objects or components; • Contract: configures and structures domain concepts—a model instantiating the domain semantic metamodel; • Framework: an execution environment for implementing domain components. The proposals were evaluated by the business object domain task force (BODTF). Despite a lot of effort in trying to use IDL and extensions to IDL to define business components, nothing substantial has yet to emerge. As a result vendors and implementors are generating their own definitions of common business objects (e.g., IBM San Francisco) and business object frameworks (e.g., SSA's BOF [32]). Choosing an Architecture An architecture might be chosen for a particular project, for a product line, or for a complete enterprise. So before choosing an architecture, it is important to be clear on the purpose the architecture is to serve. We have also seen that an architecture is not so much a specific thing as a set of views and those views can operate at various levels of abstraction. Couple this with other architectural entities such as architectural styles, patterns, and frameworks and it can be seen that making such decisions in anything but the most trivial cases is a major undertaking. Neither is choosing an architecture just a technical issue; the nature of the organization in which it is to be used will have a significant effect on the success of its uptake. Some of the key issues to be considered when making a choice are: • Role of the architecture: Is it for a small team, a large organization, a consortium, or for the whole industry?
Page 140
• Nonfunctional requirements: Are quality, performance, size, and other nonfunctional requirements well understood? • Nature of the organization: Is it large or small, centralized or distributed, newly formed or longstanding? • Culture of organization: Is it liberal or bureaucratic, centrally controlled or anarchic, conservative or innovative? • Software capability: Are there mature development methods and best practice in place or just a load of hackers? • Skills: Are there people with the skills to develop and use architectures or will extensive training be needed? • Tools: Is a common set of tools and languages in place or do people just use whatever takes their fancy? • Type of industry: Is it productbased (architecture will structure the product line) or servicebased (architecture will structure the support and delivery systems)? • Constraints and tradeoffs: What decisions have already been made? What financial constraints are there? Is there a limited window of opportunity? By carefully considering such questions an organization will be able to better understand how valuable a chosen architecture might be, what needs to be put in place to make it work, and what the chances of success are. Table 5.1 provides some of the questions and issues more specifically related to the role the architecture might play. We have drawn these from our own experience and from many of the references given in this book. Evaluating an Architecture It is a wellknown fact that the later in the software development life cycle problems are found, the more expensive it is to rectify them. It therefore seems obvious that particular attention should be paid to evaluating the suitability of a chosen architecture to avoid costly mistakes later. Many of the nonfunctional aspects of a system design (e.g., quality, reliability, performance, and security) are embodied in the decisions that are made while choosing and developing the architecture and architectural styles. It is also well known that developers are rarely enthusiastic when it comes to considering nonfunctional aspects. However, there is now a growing body of opinion that an effective way of addressing these issues is to assess how well
Page 141
the chosen architecture supports these desired qualities. Such an assessment would both validate the architecture and the nonfunctional requirements at the same time. Evaluating architectures is not easy. We have already seen there is a wide variety of opinions on what constitutes an architecture, before we even consider the meaning of the specialist terms used within it. Styles and fashions also change rapidly, making it difficult to compare architectures separated by any significant period of time. The Software Engineering Institute (SEI) developed a software architecture analysis method (SAAM) specifically to evaluate software architectures. More recently they have established an architecture tradeoff analysis initiative [33] based on case studies, promoting, best practice, and analysis of successful product lines. The initiative has developed an Architecture Tradeoff Analysis (ATA) Method, (see Figure 5.15). The method [34] can be used to assess architecture against a range of different quality attributes, but of particular interest is that a method is being developed to provide analysis of patterns and architectural styles. SEI says: A designer will choose one pattern because it is "good for portability" and another because it is "easily modifiable." But the analysis of patterns doesn't go any deeper than that. A user of these patterns does not know how portable, or modifiable, or robust an architecture is until it has been built. Moreover, one does not appreciate the impact of choosing combinations of patterns.
Architectural evaluation is definitely an area to watch for the future. Using an Architecture We have already said that choosing an architecture is not an undertaking to be made lightly, but the value of the effort will only be realized if the architecture is used effectively and continues to be used. Here are some more questions (Table 5.2) and guidelines that may be useful. As we have already suggested, it is vital to look at the skills the organization has, the degree of software development maturity attained, and the organizational culture. Successful use of largescale architectures to interconnect widely disparate systems needs an organization with strong discipline, central control, and an understanding of the need to fund infrastructure development. On the other hand, more fluid organizations built around small teams can develop highly innovative product lines provided there is a high degree of software development maturity and a good understanding of architectural issues.
Page 142 Table 5.1 Why Do You Think You Need an Architecture? Reason
Guidelines
To support a product line with a number of Consider product family architectures [25] variants To allow integration with products from other vendors or use of boughtin components
Use commercialstrength technical architecture style (e.g., CORBA or COM) Check if the vendors are part of the OAG or have formed alliances Look for domainbased architectures or frameworks. Look for existing commercial components Consider using patterns and pattern languages
To allow the reuse of inhouse developed components
Appoint a chief architect to enforce companywide standards Use patterns and frameworks to promulgate best practice Do not initially worry about reuse— establish widespread use of good methods and reuse will follow
To allow the interconnection of many systems and applications
This is the hardest to achieve! Set a strategy and define evolution plans Enforce standards and plans Use a commercialstrength technical architectural style Look for domain architectures Make alliances with key vendors Only build that which gives competitive advantage Avoid stove pipes Do not worry above lowlevel reuse
To conform to international or domain standards To avoid starting from scratch
Use agreed architectural standards Use commercialstrength technical architectural style
Buyin business frameworks
As an aid to understanding of complex systems
Use patterns and pattern languages
(table continued on next page)
Page 143 Table 5.1 (continued) To test nonfunctional requirements Use standard architectures (performance, maintainability, and so forth) Use evaluation methods before design
It is always vital to establish clear ownership and authority and to have someone with strong control and a clear vision to lead it, thus avoiding architectural drift and stovepipes. Most important of all, there must be a clear commitment from management to establish and fund architectural development, evolution, and adherence. Configuration management and version control, both of the architecture and the products and components produced, is always essential. The Role of Architects The Institute of Software Architects has developed an 8stage plan [35] which defines the role an architect should play in systems design, implementation, and delivery. Phase 1: PreDesign The architect listens to understand the scope of the project, the client's key design points, requirements, and expectations. The architect also studies the context of the project, the entire enterprise of which the project is a part. Client resources are assessed, such as the financial and intellectual capital available, and the problems the client needs to resolve. The architect identifies possible solutions available through technology, and organizational, management, and product changes. A design direction begins to take shape, with the architect and client collaborating and refining their understanding until a shared vision emerges. Broad budget and schedule objectives are set. Phase 2: Domain Analysis Phase The architect undertakes to thoroughly understand and document the areas (domains) for which the system will be built and to learn the client requirements in detail. The desired behaviors of the system are outlined. The architect assesses the client's business and technology environment and the interplay with scope. The domain terms and concepts are accurately defined. Phase 3: Schematic Design Phase Architecturallevel designs depicting the domain characteristics and technology structure are prepared. The look and feel
Page 144
Figure 5.15 SEI Architecture Tradeoff Analysis (ATA) method.
Page 145 Table 5.2 What Sort of Organization Will Use the Architecture? Reason
Guidelines
Small teams
An informal approach is best if built on a commercialstrength technical architectural style
Several cooperating teams
Common tools are essential to avoid architectural mismatch and implicit assumptions Use a common approach to configuration management Appoint architectural representatives in each team Hold frequent design reviews Use patterns to achieve reuse
By the whole organization
A chief architect and a formal architecture is essential A funding model that will support infrastructure and architectural development must be in place Limit the number of tools and vendors through strategic alliances Establish a portfolio of key components, but do not be too ambitious to achieve reuse Buyin business frameworks where possible Reward good practice and cooperation rather than clever new code
By cooperating organizations
Establish alliances Play an active part in domain standards bodies Use domain architectures
of the system—user interface style—is designed. Prototypes are built at this point, if they are needed. Migration and risk assessments are performed. Phase 4: Design Development Phase The architect continues the expansion of detail and design refinement to converge on the final design. All the domain and technology design drawings—that is, what the client needs to validate that the requirements are met—are finalized at this stage.
Page 146
Phase 5: Project Documents Phase The architect focuses on the requirements of those who will construct the system. The construction process, the roles of the team members, and the construction sequences are documented. The construction guide, user interface style guide, and test guide are written. The architect specifies tools and methodologies, as needed. All the details needed by those who will build the system are completed. Phase 6: Staff or Contracting Phase The architect assists in identifying the actual builders of the system. For projects that are outsourced, bids are submitted to outside contractors and potential participants are evaluated. The architect assists with contract details and in assessing cost. Sequences are arranged and contracts signed. Phase 7: Construction Phase The architect's supervisory role during construction ensures that the client's vision is understood and executed. The architect reviews constructionlevel designs to the degree dictated by the complexity and vicissitudes of the construction process. The architect conducts design reviews and analyzes problems and change requests. The architect designs the accepted changes, assesses the impact on overall design and cost, and sequences the changes. The architect participates in testing and acceptance reviews to the extent the client desires. Phase 8: PostConstruction Phase The architect assists the client with the project rollout and the migration to the new system. The architect can be involved with the training of system operators and users, as needed. The architect assists in warranty issues and ongoing maintenance procedures. The architect and client meet when it is all over and reminisce about the trials and triumphs. They hold a big party at a Mexican restaurant, complete with mariachi band, for all the builders, employees, and customers involved with the project. Those naysayers who whined incessantly and said it could not be done now stand mute, sipping their margaritas. The Shape of Things to Come Some clear trends are apparent for the future direction of architectures: • Components are forcing a resurgence of interest in architectures. • Internet has not killed client/server; it is giving it new life. • Component execution environments are almost business as usual. • Monolithic architectures are dead; frameworks and patterns are the thing.
Page 147
• XML and UML will take over the world (well maybe). • Architectural evaluation could have a strong role to play. • Business frameworks will set the pace for largescale components. But, there remain some important questions: • Will components scale up to the business level? • Will there be a market in independent frameworks? • Will we get true multivendor plug and play? • Will serverside components just push complexity somewhere else? • Will we ever stop writing code? • Where does workflow fit? • Will standards lead or will vendors lead? • Will intelligence and emergent behavior play a part? We expect the main thrust in componentbased development over the next few years to be centered around the business component frameworks described above. It is only here that the vendors involved have the resource and business credibility to establish products that large corporations will use. The technology wars will sort themselves out through middleware layers and adapters. Of growing interest is the relationship of business processes to components and the role of process modeling and workflow in the business component arena. The Unified Modeling Language (UML) is becoming the de facto standard for objectbased modeling, and there is little doubt that it will be expanded to specifically embrace components and process flows. Research is likely to focus on creating intelligent components by building on work already done on intelligent software agents. Also attractive is the idea of self adapting components. They would negotiate with other components to establish an agreed set of interface mechanisms, rather like fax machines and modems do at present. As with all research it is not clear if, and how soon, such technology might become practical. As we have seen, the hard problems of component assembly are to do with the semantic connection between components while these adaptive technologies may work best at the syntactic connections. Summary We have seen that an architecture provides a common language for describing the concepts of system and its implementation in an abstract, but detailed way.
Page 148
It defines the components and how they will interact, but moreover the various viewpoints of an architecture describe all the key decisions that have been made, the options and flexibility that exist, and the nonfunctional parameters that are essential to achieve the desired aim. Architectural styles document the constraints that have been imposed on the design to allow implementation in a defined way. Structuring techniques such as patterns, frameworks, and product lines provide pragmatic ways of reusing good practice. They provide partlymade solutions while still allowing the ability to flex and extend the system. Business frameworks offer the potential for implementing componentbased approach at a meaningful business level. Thus, if done correctly, the whole architectural package will provide the managed or constrained flexibility that we saw at the start of the chapter. But architectures are not easy to use. They require the right decisions to be made. They require that the organization has the right culture, maturity, and skills. Most important there must be the will to make architecture the focus for delivering the product. References [1] ComponentBased Development: Strategies and Technologies, 1998 (http://www.butlergroup.com). [2] SEI Architecture Definitions. (http://www.sei.cmu.edu/architecture/definitions.html) [3] Swannell, J. (ed), The Oxford Modern English Dictionary, Clarendon Press: Oxford, 1992. [4] Garlan, D., R. Allen, and J. Ockernbloom, ''Architectural Mismatch or Why It's So Hard to Build Systems Out of Existing Parts." Proceedings of the 17th International Conference on Software Engineering, Seattle, 1995. [5] Bass, L., P. Clemens, and R. Kazman, Software Architecture in Practice, Reading, MA: Addison Wesley Longman, Inc., 1998. [6] Monroe, R. T., A. Kompanek, R. Melton, and D. Garlan, "Architectural Styles, Design Patterns, and Objects." IEEE Software, January/February 1997. [7] Jacobson, G., and Jonsson, Software Reuse. Reading, MA: AddisonWesley. 1998. [8] Kruchten, P. B., "The 4 + 1 view Model of Architecture," IEEE Software, November 1995, 12 (6), pp 42–50. (http://www.rational.com/sitewide/support/whitepapers/dynamic.jtmpl?doc_key=350) [9] Kriha, W., D. Kesch, and S. Pluess. Architectural Structures For Large Systems Design. OOPSLA '97 Workshop; Exploring Large System Issues. [10] The Open Group Architectural Framework (TOGAF). (http://www.opengroup.org/architecture/)
Page 149
[11] United States Department of Defense Software Technology Programs. (http://wwwast.tdsgn.lmco.com/arch/arch008.html). [12] Alexander, C., The Timeless Way of Building, Oxford University Press, 1979. [13] Gamma, et al., Design Patterns: Elements of Reusable ObjectOriented Software, Reading, MA: AddisonWesley, 1995. [14] Kerth, N. and W. Cunningham. Using Pattern to Improve Our Architectural Vision, IEEE Software, January/February 1997. [15] Buschmann, F., et al., PatternOriented Software Architecture: A System of Patterns. John Wiley & Sons, 1996. [16] Krasner, G. E. and S. T. Pope. "A cookbook for using the ModelViewController user interface paradigm in Smalltalk80," Journal of Object Oriented Programming, 1(3), 1988. [17] Appleton, B. Patterns and Software: Essential Concepts and Terminology (http://www.enteract.com/~bradapp/docs/patternsintro.html). [18] Alwyn, S., Stairway to the Mind, Copernicus, 1995. [19] Pree, W., Components for Applications: Concepts and Case Study, Architectural Approaches to Software Engineering Seminar, The Open University, Milton Keynes. [20] Tepfenhart, W. M., and J. J. Cusick, "A Unified Object Topology," IEEE Software, January/February 1997. [21] Szyperski, C., Component Software: Beyond Object Oriented Programming, Reading, MA: AddisonWesley. [22] Brown, A. W., and K. C. Wallnau, "The Current State of CBSE," IEEE Software, September/October 1998. [23] Clements, P., Report of the Reuse and Product Lines Working Group of WISR8, Software Engineering Institute. [24] Bosch, J., Product Line Architectures in Industry: A Case Study. (http://www.ide.hkr/~bosch). [25] SEI Architecture Product Line Practice Initiative. (http://www.sei.cmu.edu/activities/plp/plp_init.html) [26] XML Specification. (http://www.w3.org/TR/RECxml) [27] XMI Features IBM. (http://www.software.ibm.com/ad/features/xmi.html) [28] Open Applications Group Middleware API Specification. (http://www.openapplications.org/oamas/loadform.htm) [29] OAGIS—Open Applications Group Integration Specification. (http://www.openapplications.org/oagis/loadform.htm) [30] OMG, Business Objects: Request for Proposals. Document cf/960104. [31] Digre, T., "Business Object Component Architecture," IEEE Software, September/October 1998. [32] Kozaczynski, W., "Architecture Frameworks for Business Components." IEEE Software. 10859098/98, pp 300–307. [33] SEI Architecture Tradeoff Analysis Initiative. (http://www.sei.cmu.edu/activities/ata/ata_init.html)
Page 150
[34] Steps in an Architecture Tradeoff Analysis Method: Quality Attribute Models and Analysis. CMU/SEI97TR029, ESCTR97029. 1998 [35] The Role of the Software Architect, The Institute of Architects, http://www.wwisa.org
Page 151
6— By Threes and Fours—Bringing It All Together Things that think, want to link. Nicholas Negroponte
In any book there is a point in the story where all the various subplots, threads, and characters come together and the raison d'être for the whole enterprise is drawn into focus with the author's intentions coming clear. So far we have not had many subplots as such, or any subversive characters, but what we have introduced are four distinct branches of systems technology which we are now going to bring together. The assertion is that these four technologies will form the cornerstone of systems development as we move into the next century. Back in Chapter 2 the concept of a system interface was explored. These come in all shapes and sizes, but there is usually a common kernel, something they all have in common. This pointed to the possibility of a general theory of interfaces. Indeed, it can be demonstrated by a reasonably indepth study of the interface equation (a rigorous exposition of which can be found in Appendix A) that it is possible to be very formal when designing an interface between two (or more) systems. The focus in Chapter 3 was on components, the items between which interfaces exist. In general, components are not simple entities. They can be described at various levels of abstraction and in various ways. However specified, a collection of components can only cooperate effectively if they interact with one another in a predictable way. The right set of components, put together in the right way, doing the right thing gives you the system you need. First, though, you have to establish what that right way is.
Page 152
Hence, in Chapter 4, the spotlight was on the subject of systems integration. With time scales for delivery getting ever shorter and the systems themselves ever more complex, systems integration has become a key discipline within the systems engineering community. Like cities, a company's support system will be a mixture of old and new. As with a city, a company cannot exist for very long without certain basic functions, for instance, payroll, billing, and manufacture control. These are equivalent to transport, sewage, water, and energy supplies. These services are always needed. Thus to completely renew a large city's sewage system would probably mean going without suitable waste disposal for too long a period. Correspondingly, a company cannot do without its payroll system, for example. Thus there is this need to adopt an evolutionary approach, gradually replacing old with new over a period of time. Thus the primary job of a systems integrator is somewhat akin to city planning, getting the old and the new to work together in harmony. Whether you are interfacing systems or integrating them, you need a framework or an architecture that provides the general structure of the system in question. In Chapter 5, it was shown that there needs to be a close relationship between business processes and the system that supports them. Achieving such a close match is not easy, but there are tools and techniques that can help, such as workflow, data modeling, and componentbased development. The use of components is seen as an important step, but developing a suitable repository is no mean feat in itself. So we have the four elements shown in Figure 6.1. Each is a discipline in its own right and has numerous research activities associated with it. But we have now had enough of the various ideas that contribute to componentbased system engineering. In this chapter we want to bring them all together to show how each is relevant when it comes to today's systems.
Figure 6.1 The four technologies.
Page 153
IT systems have rapidly assumed a role of major importance to companies, both as a strategic tool for competitive edge and to improve efficiency [1]. Retailers wishing to address a multichannel market strategy have to rapidly become adept in a wideranging set of skills. They must become more agile in the way that they support their core business processes, being ready to change those processes that support the business as the business changes and evolves. And of course, there is ebusiness, which is set to grow substantially over the next decade. Most people would agree that the future lies in the ability to integrate and support new channels with the existing ones. New technologies like Interactive TV, call centers, Internet shopping, and Mobile Commerce (mCommerce) combined with competition and regulation are forcing many organizations to rethink their strategies [2]. Although markets are increasing in complexity (i.e., through multichannels and globalization), many companies are under increasing pressure to reduce costs. The need for a costeffective, easily managed, yet complete infrastructure becomes paramount. So, when developing an IT system for a large multinational, there are a number of things to consider. For example, it should be truly global (not just multilocal); it needs to be flexible and adaptive; it needs to be responsive and provide firstclass customer support. There is also the trend towards increasing mergers and joint ventures. When two companies merge or participate in a joint venture, what happens to the existing IT systems, and what happens to the individual companies' intranets? Such scenarios form the basis for the case study presented later on in this chapter. Although names have been changed, it is in fact close to a real system. Before we discuss this in any greater detail, let us take a step back and look at the drivers for a change. The reasons why a particular organization wants to revise its systems are manyfold, but it is usually to meet some urgent busy need, such as offering a new service, without starting from scratch and building yet another bespoke, standalone system. Motivation When launching a new product or service an organization has to think about more than just product development and sales. It also needs to consider handling direct customer interactions (helpdesk and sales for example), and managing the support and related processes. The quality of these support processes and innovations to improve their effectiveness are increasingly key differentiators for many companies. This is particularly true when there is a significant dependence on suppliers and other companies in the value chain, such as a supermarket chain or a large retail store. For many large and established companies, where a
Page 154
significant investment in IT and support processes has already been made, there is the option of developing new systems from scratch or utilizing existing systems in some form [3]. It could be that the system is used "as is," or as was discussed in Chapter 5, it might be componentized into smaller, more manageable systems. The need to react to market changes and new value potential means that systems have to be delivered in far shorter time scales than what was previously considered acceptable. On top of that, support systems are becoming ever more complex. As a consequence, companies are looking at their existing systems as a resource to be exploited and reused. In Figure 6.2 we can see the four essential layers of a typical information system. Firstly, their whole reason for existence is to support the business in generating revenue, either directly or indirectly. Currently, the levels of process automation in most organizations is not that high. As a consequence, a typical company office will be something of a sea of desktop machines and people. The primary data they are using will be stored and accessed via the operational support systems. The medium by which the queries and data travel from one node to another is the network. Things are in fact getting much more interesting. Companies are more often than not merging to form joint ventures or conglomerates, targeting new value potential, or establishing themselves in new markets. In doing so the processes used to develop, deliver, and support a product or service must straddle any number of companies. All companies, however, have the same problem: they have existing systems that they want to reuse and few (if any) can afford the luxury of stripping out the old systems and starting from a green field.
Figure 6.2 The essential building blocks for an information system.
Page 155
One approach is to develop a community of interest network (CoIN). This physically connects each company and allows endtoend processes and communication with each company's backend systems (or support systems if you prefer). The level of information sharing can be controlled, and there is scope to increase or decrease the scope of the CoIN with minimal effort. Thus, we have a prospective solution that has the agility needed to cope with the information demands of a fast moving world. A typical architecture is shown in Figure 6.3. The main issue with a CoIN is that the resulting beast can be extremely complex and a significant challenge to the computer systems engineer. It poses many difficult problems when it comes to interfacing and integration, but as we shall see, there are solutions, so we will carry on to explore the CoIN route in some depth. With the ever quickening pace of new product launches, and particularly with the advent of online multimedia services, more imaginative ways of providing customer support are required. In this chapter we will explore the many practical issues that systems engineers and managers encounter when interfacing different systems together. A word that is becoming more prevalent in systems circles is dispersal. The purest form of dispersed support systems and data offers a radically new approach with increased flexibility and is based on two concepts:
Figure 6.3 How an extranet works. Company A and Company B have their own intranetsthat are protected from the outside world by firewalls. An extranet,which may be a dedicated, private network or a segmented part of theInternet, is a network that links the two intranets together, enabling the companies to work collaboratively. Thus they can share selected data, systems, and processes [4].
Page 156
1. Dispersal: The decentralization of responsibility, business processes, information, and systems. The basic idea is that organizations are responsible for elements of a process which are most pertinent to themselves. 2. Mass customization: Empowering customers to configure and tailor the complete service offered, including the way it does business with the service provider. In other words, giving the customers precisely what they want. One of the prime motivators behind this is to build closer business relationships with customers and suppliers [5]. This is in answer to the long held belief by many organizational scientists that locating the processes and data in only one organization leads to extra work for everyone. Allocating responsibilities, to those best able to undertake them (and this could be to both supplier and customer) removes unnecessary process steps and so reduces costs for everyone. Such an approach cannot be achieved with monolithic stovepipe systems. It requires an architecture that is more flexible and dynamic, where each function is, for example, a distinct business object or component. The functions could include provisioning, fault reporting, pricing, and billing. These are all potential examples of where a dispersed IT infrastructure which exploits distributed systems technology could enable more flexible and dynamic support processes. The relevant components can be regarded as interacting entities which cooperate to provide the complete process required for a service. A provisioning component, for example, can communicate with a billing component to ensure that the service can be billed. The mode of business supported by this dispersed approach, together with information sharing and collaborative applications, is a marked change from the customersupplier value chains that are commonplace today [6]. For example, it requires direct access to each other's IT systems—realtime and ondemand. It is here that the ideas presented in Chapter 5 are particularly powerful. Starting with the business goals of the organization and working through the methodology step by step, we derive an IT systems architecture that is fit for purpose. When dealing with large multinational organizations, it is natural to consider CoINs. Many companies have established intranets, internal IP networks upon which many of their processes are based. The CoIN concept expands on this idea by using an extranet to link various intranets together. This enables collaborative workflow, or endtoend processes which straddle many organizations. This in turn operates over a series of component repositories which are distributed across the participating companies.
Page 157
Real business benefits are obtained when systems are built to support business processes in a flexible and costeffective manner. The prime enabling technology to achieve this is workflow [7]. This addresses the previously untouched areas of automating manual processes. Typical objectives are to improve customer service levels, improve productivity, and reduce process times and costs, while improving quality of service. Very large sums of money have been spent on mainframe or client/server applications that support key functions such as accounting and billing. Equal sums have been spent on desktop tools such as PCbased word processing, spreadsheets, and the like. Unfortunately the actual processes, carried out by people who tend to work in a procedural way, have not benefited to anything like the same degree. Implementing a workflow system is a strategic decision since it will affect the dynamics of the processes and, ultimately, the way a company does business [7]. From the network point of view we have a potential problem. Are these intranets compatible? Do they use the same protocol or carrier technology? Extending access to systems (by networking techniques) is nothing new. However, a number of desirable aspects of the Internet and World Wide Web's success are preserved by the specific use of extranets, which provide the infrastructure needed to support electronic commerce (ecommerce or ebusiness) and the electronic society at large [8]. Figure 6.4 summarizes what typically happens when a group of companies join forces over a CoIN and move from intranets to something more like the Internet [9]. This is, in fact, no trivial matter. Each organization will have developed its media largely in isolation. It is likely that the only aspect that is likely to be common to both will be email. This is in contrast to Web publishing, for example, which is somewhat less established, although many have applied Web publishing to internal domains, combining it with groupware or workflow. Taking a bird's eye view of the Figure 6.4 provides further insights. The level of electronic messaging within an organization can range from simple, sequential email (where someone sends a message and then waits for a reply and so on) to realtime collaboration using tools such as ICQ or NetMeeting where audiovisual transactions can take place. Likewise the level of management will vary from none at all (a typical seatofthepants type organization) to highly structured and managed. This is summarized in Figure 6.5. The four quadrants in Figure 6.5 describe four different kinds of interaction, each placing its own demands on the IT infrastructure. In many cases companies have developed pointsolutions for each, but it is becoming increasingly the case that these need to be fully integrated to form a harmonious whole. As such, they are converging, greatly facilitated by the wide acceptance
Page 158
Figure 6.4 Moving from intranets to internets via extranets.
of Internet standards and the rapid development of numerous applications based on them. As a result, the technologies used in different domains is now crossing over. The growth of the Internet has certainly been accelerated, to some degree, by the success of Web publishing. Groupware, along with workflow, has been developing a greater degree of maturity and integration within many organizations [6]. Now that the World Wide Web and groupware (which we will use as a generic term to encompass workflow) have begun to form a more coherent alliance, the exploitation of published media is being realized. This is particularly clear in an intranet where groupware is typically understood in terms of the three Cs (Figure 6.6).
Figure 6.5 Management and communication.
Page 159
By communications we mean basic, sequential messaging, a good example of which is basic email. Collaboration or conferencing (notably forums or bulletin board systems such as ICQ or NetMeeting, which organize messages into topical threads of group discussion maintained in a shared database) provides a more concurrent mode of operation. A good example of this is the chat room. This mode of collaboration becomes particularly powerful when used in conjunction with groupware (such as Lotus Notes or Microsoft Exchange) or workflow technology (such as Oracle Workflow) by applying predefined rules to automatically process and route messages. In the case of extranets (a network of intranets) there are a number of complications, particularly when considering business transactions from one organization to another. Such transactions may require access by one party to the backend information systems of another. This would certainly be true in the distributed case. However, in order to overcome issues such as commercial sensitivity and security, many extranets are being formed which utilize a ''shared server," which acts as a hub for the community of interest supplying the communication, collaboration, and coordination for the respective organizations. It is not unusual for these servers to be provided by a third party. Internetbased transaction systems are gaining maturity, typically with Java interfaces to the backend systems. The primary driver here being the rapid growth of Internet access and ecommerce. Telecommunications customers can, for example, access their bills and check the status of their orders via a Web browser, but the information is coming from a traditional system. It will take industry many years to develop new, fully IPenabled systems, though many are aggressively pursuing this path [10].
Figure 6.6 The three Cs.
Page 160
Adding a Web frontend to an existing system can often reap new and unseen rewards by presenting to the customer or user a richer range of functional options and possibilities. This is particularly true as the desire for more complex and customized services grows. But it also provides the supplier with a unique platform from which to influence the customer. The customer will have, within the confines of a page on a Web browser, access to an enormous wealth of information and services. Interorganizational collaboration is typically achieved via bulletin boards or shared folders within a groupware application such as Lotus Notes, Netscape's Collabra, or Microsoft Exchange. However, achieving seamless support for collaborative working across organizations is not common. This is due in part to the surprising diversity of desktop systems and groupware applications. Quite often, achieving any but basic email message transactions becomes a major triumph! It is here that the Web becomes a particularly powerful tool as it is largely independent of technology (i.e., desktop computers) and browsers. We can now see why the convergence of groupware and the Web is so advantageous to companies and organizations. With this, convergence and extranet will be achievable even in the most heterogeneous of environments. It is a key step, but one which has yet to really happen. But this convergence is coming, make no mistake. And it is not only the Internet, Web, and e commerce that are acting as the catalysts. There is also the belief by many captains of industry that organizations need to find a radically different way of operating in readiness for the challenges of the next millennium. This is highlighted by initiatives such as "Inventing the 21st Century Organization," a collaborative venture headed by the Sloane School of Management at MIT. Terms such as virtual corporation have become common as the language of a new organizational science evolves [11]. The trend, it appears, is toward more seamless organizations where the barriers between suppliers and customers will disappear to accommodate a more dynamic and diverse market. Hence, in answer to the question "intranet or extranet?"—it doesn't matter. They will, in time, become the same entity.? The term intranet has already become too limiting as emerging applications are beginning to serve these ever more amorphous networks of customers and suppliers. The Gartner Group describes the firewall between Internet and intranet as "a semipermeable membrane that will become more porous with time." Whether or not that is true, it is clear that the convergence of the Web and groupware will open many new doors by lowering entry barriers to new and enriching existing markets. As companies exploit this convergence they are already beginning to see advantages in terms of economies of scale, greater flexibility, and greater opportunity [12]. But it is not all a pot of gold. There are many questions and challenges which may seem mundane, but which in fact will be key issues. Who
Page 161
manages the network? What of security? What happens when the alliance dissolves? These will no doubt be solved—somehow. But they are unlikely to stop the tide—it has already gained momentum and will continue to do so. In the lower lefthand quadrant we have an unsophisticated company in terms of infrastructure, but which could well be very complex from an organizational point of view. Being in the lower left is not a bad thing. Indeed, there are many examples of companies which deliberately operate in this way. Some choose not even to have email! IT systems may be bespoke, stovepipe solutions with little coupling with other business systems. In short, we could say that the degree of coupling within such a company is low. That is, there will be little interfacing and integration between systems, and systems and processes. What links there are, are likely to be static. The top right represents a company which has fully integrated processes and workflow. The workflow tools interact with component repositories to invoke the necessary components to enable one step of a process to progress to the next step. There will be formally defined architectures and standards. The degree of coupling between systems and processes is likely to be high in such a company. There will be numerous interfaces between systems and processes. These links are likely to be very dynamic. The question is when moving from the intranet domain to the internet domain, what is the likely level of work for the IT teams? In a typical intranet we would expect to find email, groupware, and simple collaboration via bulletin boards, for example. Web publishing, as already stated, is still relatively rare. Establishing an extranet between many intranets does not necessarily mean that all these domains are transported from the intranets. Indeed, Web publishing, for example, has long been a common activity on the Internet, as has email, but there is little evidence of groupware, workflow, or collaboration between members of a CoIN. This represents the righthand column of Figure 6.6 and demonstrates that most of the effort would need to be targeted at establishing endtoend workflow with the relevant links to backend systems. If we look at the complexity/coupling table in Figure 6.7, we see that in an ideal world it would be best to have as few new functions as possible along with the lowest amount of coupling. Of course, you can reduce the amount of functionality either by reusing existing systems or borrowing someone else's! And a CoIN provides an ideal vehicle for one company to borrow another's IT systems. It is referred to as linking backend systems, and it is high on the agenda of many large multinational companies. It is here that workflow technology is particularly useful, since a workflow engine will automatically seek out and utilize functional components wherever they are, even if one component lies on a mainframe in London and another on a PC in Sydney! From a system management perspective, we would need to establish where the CoIN project as a whole stands with respect to Figure 6.4. Now it may be
Page 162
Figure 6.7 System complexity.
that few new functions need to be developed, and if all companies were of the structured and managed type then integration could, in principle, involve little further development. But if they are all of the ad hoc unstructured type then although the relevant functionality may exist, it may be buried deep within legacy systems. Thus, a program of componentization, as discussed in Chapter 5, would need to be initiated. So it is very likely that for each member of the CoIN a separate evolution strategy would need to be established, which in turn would feed into the overall strategy. But we can take one thing as a given, that the extranet itself will be an IP network and completely nonproprietary. The nonproprietary nature of modern networks makes the interconnection of various organizations easier. In building a true partnership, different organizations involved can use different technologies that can be readily interfaced, including the core network and client/server applications. The open standards of the Internet provide an ideal basis for interconnection. The extranet approach also means that existing intranet investments can be used. This reuse of corporate infrastructure in each organization is very desirable from a cost perspective. The same extranet IPbased infrastructure used to build the CoIN can also be used to run distributed applications which support managed information sharing and associated responsibility for ownership as well as shared processes and workflow support. This can be extended to include applications for conferencing and collaborative working (see Figure 6.8). As we hope will be apparent by now, this is a complex beast and many a project manager's worst nightmare! However, it is not a fantasy dreamed up for the purposes of the case study. This is for real. Many companies have already built such networks, and many more will follow.
Page 163
Figure 6.8 The threelayered strategy.
Page 164
The Scenario A large multinational company has embarked on a joint venture with four other companies. Each is successful in certain niche parts of the target market, but by working collaboratively together they hope to address the market more fully and more efficiently. The establishment of a CoIN is seen as the best way forward. The companies are not too dissimilar in terms of IT, but differences do exist, particularly with client desktop systems. Interestingly, these companies will continue to compete in certain segments of the market, so security is a key issue. This is compounded by the fact that it has been agreed that direct access to backend systems will be encouraged. Indeed, it is seen as the only costeffective way to go forward. Hence, there will be a need for endtoend workflow which straddles the various companies. The joint venture sees itself as a world beater, but it requires highquality support systems, and the market window is tight. Company A is the most structured and managed and has been chosen by the group to lead the development of the CoIN. Web publishing is not important, but collaboration is seen as key. A possible architecture is given in Figure 6.9. Hypothetically, the situation for the CoIN could look something like those presented in Table 6.1. Some of the examples may seem a little unrealistic, but they do exist. From the system integrators point of view, this is a challenge. Just
Figure 6.9 The CoIN architecture with the five companies sharing a central server, protected by a firewall. Note that the shared server is optional. Another method is to use the existing databases within each company, thus forming a distributed, federated database [9].
Page 165 Table 6.1 A Heterogeneous Mix of Information Technology
Network
Database
Desktop
Communication
Internet Access
Company A
Ethernet LAN
IDMS
Microsoft, PC
MSMail
No
Company B
Ethernet LAN
Oracle
Lotus Notes, Mac
Email
Yes
Company C
No coordinated LAN
None
Windows 3.11, PC
No email
No
Company D
LAN, WAN
Oracle
Microsoft, PC
MSMail, ICQ, NetMeeting
Yes
Company E
Ethernet LAN
Sequent
Various
Email
Yes
how do you bring this diverse group of technology together to act as a federated system? Interfaces The complication with this scenario is that it looks to be a twodimensional problem. Not only is there the task of integrating networks, platforms, and systems vertically (see Figure 6.8), but there is the additional dimension of horizontal integration across organizational boundaries and the various technology domains within the CoIN. And this occurs at all three levels. Networks Today's technology can provide quick and accurate data access for decisionmaking at many organizational levels. Business people and decisionmakers need fast data access through interfaces whose use does not require extensive training. There has been a growing trend toward PCbased information systems driven by advances in microprocessor technology, data communications, and local area networks (LAN) systems. Advances in data communication and LANs have allowed the integration of PCs with the existing networks of mainframes. Networks make it possible to let end user PCs share files via a central network file server. Networks have increased productivity by reengineering the basic business processes within many organizations. A LAN's or WAN's structure will determine controllability and shared resources. For example, if the CoIN was to incorporate a large, centralshared server, then a star topology
Page 166
might be the best solution. A physical star, logical ring topology will enable both types of companies to configure and manage the LAN from a central location. It is important to note that there are some guidelines that should be followed when building networks. First, an overall architecture should be developed. This simply means that a set of company policies and rules should be created that, when followed, will lead to the network environment that is desired. Secondly, find a way to interface many dissimilar networks. The goal is not a single, coherent network, but rather finding a way to link other networks together. Lastly, use standards. Complications in networking are caused by incompatibilities that can be reduced by using standards. Standards should be the foundation used to develop the overall architecture. The first practical step is to put in place a secure network. The key question here is whether to install a dedicated network consisting of installed fiber links between each partner, or whether to use the Internet. There are security issues with the Internet, but they can be resolved via firewalls, encryption, and digital certificates, but it remains a public network, so vigilance is always required. The consortium, therefore, may wish to install the fiber, thus creating a private network along with controlled links to the external Internet if need be (protected by firewalls). In both cases, an extranet is created, linking the various intranets together. It is at this level that the access mode (Frame Relay, SMDS, or Dial), protocol conversion, routers, path restoration technology, and security need to be addressed. Most likely, the default protocol will be TCP/IP, but there are other choices. Thus, you create the essential backbone of the CoIN. Once this network is in place the more tortuous process of integrating the software systems and other applications can begin. Figure 6.10 depicts a typical network architecture. As well as applications integration, it is important to ensure that the endtoend processes are coordinated across the CoIN. It is often easier to get software from different suppliers to work together (after all, we have seen plenty of integration technology) than it is to get people in different organizations to work together effectively. In practice, it is important to design the processes at the same time as the technology; use cases [13] provide a useful and straightforward method of working out what to do before embedding it into a workflow tool. Server Technology Would it be best to use a shared, central server managed by a third party, or would it be better to have no shared spaces at all? The answer depends on a number of issues, such as the nature of the interaction between the various parties, security [9], and the volumes of data involved. It will also depend on
Page 167
Figure 6.10 A typical network architecture.
whether access will be via a WWW browser such as Netscape or to a dedicated server such as Exchange. If the server is to be distributed, then one needs to consider whether to integrate using CORBA or DCE. Desktop Technology Such is the level of dominance that Microsoft has achieved that it is tempting to think that a suitable platform for a CoIN would be a pure Microsoft solution, an Exchange server with Outlook client software. But the degree of heterogeneity in desktop technology is surprisingly high, and proprietary solutions are often not advisable. Again, the primary driver ought to be to make use of the technology that already exists. After all, this joint venture may dissolve quite rapidly if things do not work out as planned. It is important that not much has changed from the viewpoint of an employee in any organization. The essential look and feel is as it was before, but now with increased power and flexibility. If access is via a Web browser, then Java applications will need to be developed. If Outlook and Exchange are used these applications will most likely be written using Visual Basic. Processes It is inevitable that the construction of a CoIN is going to trigger a spate of process reengineering initiatives. This will be especially true if a workflow
Page 168
engine is to be used [14]. The challenge with this example is interfacing processes from different organizations. We are almost back to the Mrs. Smith and Mr. Lee example in Chapter 2. In fact, language is often an issue, though not as starkly as our example. Sequential processes, concurrent processes, bowwaves, and critical paths all will have to be considered here. Information Systems Integration The responsibility for system development or integration still largely resides in central IS functions. In the evolving organization, a significant amount of emphasis was placed on training the IS professional, particularly the system integrator, as a business manager rather than as a technical expert and on the need to recruit people proficient in business functions. This was seen as necessary to be effective in the integrator and advisor roles. The system integrator's function will be increasingly responsible for managing the infrastructure for decentralized or dispersed computing that includes networks, platforms, and desktop applications. At the strategic level, systems are predominantly either decision support systems (DSS) [3] or executive information systems (EIS) [15]. The EIS is datadriven with graphical representation of the data created ondemand being a key feature. The DSS generally deals with a specific problem domain and contains a modeling component as well as a data management capability. At the operational control or middle level of an organization, the systems are for managerial control (budgeting, manufacturing) or for specific decision support in a problem domain (financial management, logistics management). Many operational control systems are embedded in the production technology of the firm and are varianceoriented. The transactional level contains the more traditional IS applications, such as personnel systems, accounting systems (general ledger, chart of accounts, billing, ordering), and integrated logistics systems (inventory management, transportation). Such distinctions are useful when embarking on largescale integration on the magnitude envisioned in this CoIN example. It will become even more important to tie IS planning to the normal business planning of the organization. This will be an increasingly important topic as the modern organizational form continues to evolve. Although there were several different methods used to accomplish this, most involve the formal integration of users into the planning and management process of systems. A prevalent method is to use a steering committee for the IS function comprised of the senior executives of the major functional areas of the organization.
Page 169
Project Management As with all projects, the fundamental driver for project management is the optimization of cost, time, and quality. Everything that a project manager does can be traced back to these basic parameters [16]. A community of interest network is only as useful as it is perceived to be by its users. As such, it is imperative that the project is comprehensively managed; by its very nature, with companies from different backgrounds and cultures, diplomacy and politics will form a large part of the job, as will communication, education, and training. These aspects need to be planned and integrated with the technical development and deliveries. Getting back to basics, let us assume a simple waterfall development lifecycle [17]. Broadly, there are five phases, with some distinct activities in each. These are listed below. Phase I—Project Preparation During Phase I, the setup and planning for the project organization begins. An initial assessment of the informationflow and the extent requirements must be made during this first phase. This entails a thorough analysis of the objectives and needs of all those involved in the venture. Many questions will be addressed such as the following: • Should the existing system be continued? • Should the existing system be modified? • Should the existing system be replaced? • What are the hardware requirements (PC, mini, mainframe)? Some questions may be difficult to assess early on; however, one of the most important questions is that of the implementation cost [14]. Companies will question the costs involved, especially if they have a milliondollar solution to a thousanddollar problem. A systems development plan, a technological plan, an implementation plan, and so forth will be defined during the preparation phase. Phase II—Solution Design During Phase II, problems defined during the project preparation phase are examined in greater detail during the solution design. An analysis of both the individual needs and the organizational needs will be thoroughly examined, addressing questions such as:
Page 170
• What are the requirements of the current system users? • Do those requirements fit into the overall information requirements? End users and the project team must work together to formalize processes and to identify potential problem areas. Such cooperation is vital to define the appropriate metrics by which the new system can be measured. Also during this phase, it is advisable to develop a technical scope definition that would consist of defining the system interfaces and data conversion (if required). Phase III—Detailed Design During Phase III, the design of the system's processes are completed. The objectives of the customization phase are to configure the system to match the CoIN organizational structure. The system will be configured and documented. This includes all the necessary technical specifications for the networks, platforms, screens, menus, reports, and other devices that might be used. The necessary systems interfaces, which were defined during the solution design, can be built. During customization, enduser training materials can be designed and built. Once customization is completed, detailed integration and unit testing can begin. Phase IV—Integration and Testing During Phase IV, the testing of the system will take place. Each defined business transaction will be tested to make sure the system is consistent with the business goals. The systems interfaces and data conversion programs will be tested along with the training materials and courses. Once the testing is completed, the company should approve the implementation plan. User authorization and system administration will be established during Phase IV. This will identify what information each user should have access to. The integration and testing phase is usually the longest of the five phases, as companies could have anywhere from fifty to several hundred business transactions that need to be tested. Phase V—Installation and Training Phase V is where the system is readied for use—for real. The system is implemented within the subject organization. This phase covers the final system tests, enduser training, and migration of data to the new system. During the initial stages, there will be a period of coding, testing, and debugging to verify that the system is operational. Any necessary system changes will be made during this
Page 171
phase. Training goals will be to develop the expertise needed for daytoday business and to promote enduser acceptance. Adopting such a methodology will certainly help, but it will only get you so far. In addition, the project team will need personnel and commitment from all levels. Table 6.2 gives a brief checklist for the key roles within a project team. As we said in the section on information systems integration, you cannot separate IS from the business as a whole, and therefore you need to work closely with senior management to ensure that the system that is developed is what the business needs. Table 6.3 gives examples of tasks for senior managers. Summary The move to increased collaboration is real. However, the motivation from the business community is more than just some altruistic gesture to the New Age. It is a trend worth hundreds of billions of dollars in online transactions and potentially millions in cost savings to the companies involved. In addition to generating new revenue streams, interbusiness World Wide Web networks may be permanently changing the relationship between suppliers and customers. There is already a sea change, as a large number of companies are moving from the intranet environment to extranets. To many observers, this is a natural evolution for the corporate intranet as the World Wide Web becomes more stable and secure enough to handle businesscritical transactions. The message is clear: to stand still is to fall behind, and consequently to fail. Forrester Research, for example, predicts that online businesstobusiness transactions will reach an annual $327 billion by 2002, as opposed to $17 billion in online retail sales. But making it work is largely a technology problem. There are a whole range of issues, some of which are only just beginning to be suitably addressed, such as, electronic commerce, Internet business models, network computers, Java, push technology, intranets and extranets, Internet security, firewalls, secure electronic payment systems, Internet privacy issues, online analytical processing (OLAP), multidimensional data analysis, data warehouses, data mining, and knowledge discovery. The rapid emergence of new technology and ideas places everchanging demands on information systems management. These days a company must focus on the interconnections between technology, the organization, and the information system. There is only one way to deal with the complexity of these interconnections, and that is to develop a sound and rigorous approach to systems interfaces and integration. Many companies, including IBM, BT, Federal Express, and Cisco, have based, or are currently basing their whole IS strategies on such notions as shared data, dispersed system, and endtoend processes with the World Wide Web as
Page 172 Table 6.2 Key Roles and Phases Within the Project Role/Phase
Initiation
Implementation
Delivery
Project management
Initiate the project Tailor life cycle process Develop contingency plans Plan and initiate feasibility studies Plan resource allocation Plan for training of resources Plan timeboxes Plan iterations and increments
Implement plans Monitor progress Adjust resources to meet needs
Implement plans Monitor progress Adjust resources to meet needs
Testing
Verify testability of requirements Assist with acceptance test plan Identify test environment needs
Develop system, integration test plans Participate in design reviews
Execute unit integration, system test plans
Systems analysis
Identify context of system Identify source of requirements Engineer software requirements
Refine requirements Participate in design reviews
Refine requirements Evaluate builds for compliance with requirements
Development
Participate in relevant training Familiarize with the business environment Assist with requirements engineering
Identify and refine classes and roles Identify persistence class requirements
Optimize design Build classes
(table continued on next page)
Page 173 Table 6.2 (continued) Initiation
Implementation
Delivery
Role/Phase
Construct object model
Map roles to classes Design user interface Prototype
Implementation
Develop implementation plan Assist with developing support plan
Deliver product to customer
Technical documentation
Identify technical documentation needs Write online help
Write user manuals Finalize other documentation Assist with other documentation
Deliver and revise user documentation
Training
Identify training needs
Specify training environment Develop training materials
Deliver training
Business partners
Provide mission and objectives of work Establish priorities Establish criteria for success Identify progress measures Help identify user requirements Help with conops document
Clarify user requirements Assist with user preparation Act as liaison with business organization
Clarify user requirements Assist with user preparation
(table continued on next page)
Page 174 Table 6.2 (continued) Role/Phase
Initiation
Implementation
Delivery
User
Help identify user requirements Help prepare test data and scenarios Help with conops document
Evaluate user interfaces Evaluate functional prototypes Familiarize collaborators with user environment
Assist with acceptance testing Attend and evaluate training Evaluate user documentation
Business sponsor
Approve initial expenditures
Approve continued expenditure Participate in formal reviews
Sign off deliverables Approve production changes
Page 175 Table 6.3 Key Management Roles and Responsibilities Executive management
All of the executives and staff use the capability to advantage Enterprise wellness Customer success scorecards Merger, alliance analysis Competitor success scorecards
Marketplace management
All marketing and sales use the capability to advantage Competitive advantages and edges Customer/prospect competitiveness Product/service demographics
Business process management
All product and manufacturing and all quality group personnel use the capability to advantage High/low value processes Process value benchmarking Value delivery analysis Process reengineering
Systems integration management
All the systems people use the capability to advantage Systems integration state/status Information systems Systems reengineering Generating systems
Knowledge management
All the human resource staff, instructors, and training groups use the capability to advantage Knowledge value benchmarking Organizational learning Competencies Value skill building
Financial management
All the financial control personnel use the capability to advantage Operating cost control Market capture control Investment effectiveness Profit management
Value management
All the value delivery personnel use the capability to advantage Portfolio value analysis Value ledgers
(table continued on next page)
Page 176 Table 6.3 (continued)
Value delivery benchmarking Value maximization
Supply chain management
All the procurement personnel use the capability to advantage Supply source competitiveness Sourcing effectiveness
Global expansion management
All the global expansion personnel use the capability to advantage Opportunity prospecting Supply sourcing Alliance partners
a key feature [18]. In conjunction, organizational theorists are proposing radically different structures, which places further demands on the corresponding information system [19, 20]. These demands manifest themselves in many ways: more interfaces, heterogeneity, automation, reuse, and complex business applications. Within the systems engineering community, it is clear that new approaches need to be developed to deal with such systems. For the whole project team, the development of a CoIN is a significant challenge. In defining interfaces and integrating systems, it is probably the pinnacle in terms of complexity. But for the JV concerned, the business benefits could be huge. If the system is designed right, integrated right, and used right it will provide a strategic, agile, and adaptable platform for any twentyfirst century company. References [1] Galliers, R., and W. Baets, Information Technology and Organisational Transformation—Innovation for the 21st Century Organization, Wiley, 1998. [2] Butler, J., Information Technology—Converging Strategies and Trends for the 21st Century, Computer Technology Research Corporation, 1997. [3] Hogbin, G., and D. Thomas, Investing in Information Technology—Managing the Decision Making Process, McGrawHill, 1994. [4] Loshin, P., Extranet Design and Implementation, Sybex, 1997. [5] Bort, J., and B. Felix, Building an Extranet—Linking your Extranet with Vendors and Customers, Wiley, 1997. [6] Chaffey, D., Groupware, Workflow and Intranets—Reengineering the Enterprise with Collaborative Software, Digital Press, 1998.
Page 177
[7] Kobielus, J., Workflow Strategies, IDG Worldwide Books, 1997. [8] Frost, A., and M. Norris, Exploiting the Internet, Wiley, 1997. [9] Pfaffenberger, B., Building a Strategic Extranet, IDG Worldwide Books, 1998. [10] Ward, J., and P. Griffiths, Strategic Planning for Information System, Wiley, 1996. [11] Toffler, A., The Third Wave, Bantam Books, 1991. [12] Fairchild, A., Reengineering and Restructuring the Enterprise—A Management Guide for the 21st Century, Computer Technology Research Corporation, 1998. [13] Jacobson, I., M. Christerson, P. Jonsson, and G. Overgaard, Object Oriented Software; A Use Case Driven Approach, Reading, MA: Addison Wesley, 1993. [14] Jackson, M., and G. Twaddle, Business Process Implementation: Building Workflow Systems, Reading, MA: AddisonWesley, 1997. [15] Cassidy, A., A Practical Guide to Information Systems Strategic Planning, Saint Lucie Press, 1998. [16] Norris, M., P. Rigby, and M. Payne, The Healthy Software Project, John Wiley & Sons,1993. [17] Norris, M., Survival in the Software Jungle, Norwood, MA: Artech House, 1995. [18] Rosenfeld, L., and P. Morville, Information Architecture for the World Wide Web, O'Reilly and Associates, 1998. [19] Beer, S., Diagnosing the System for Organizations, Wiley, 1988. [20] Korolainen, S., On the Conceptual and Logical Foundations of the General Theory of Human Organizations, Helsinki School of Economics, 1980.
Page 179
7— Engineering the Vision Opportunity is missed by most people because it is dressed in overalls and looks like work. Thomas Edison
The vision presented in Chapter 6 is fast becoming the norm, at least in spirit if not in detail. In this chapter we examine some of the underlying technology and standards that have made such scenarios possible. Starting with the network layer itself, the chapter will then look at the various standards and applications that reside on that network and the interface that enable these applications to interact. What will be apparent as we work through is the sheer diversity of networks, protocols, standards, and applications. Making the right choice for any one organization or circumstance is a nontrivial task and requires very careful consideration and planning. In Chapter 6 we started at very highlevel business objectives and gradually refined these into a concept based largely on flexibility, communication, and ebusiness. In terms of the implementation, we got only about halfway through the development life cycle, up to the design phase. There are now some very hard choices to be made. What sort of network, LAN or WAN? What architecture, peer topeer or client/server, twotier or threetier? Should it be an open Webbased solution or proprietary? Should it be objectbased? What about the overall architecture, DCOM or CORBA? What kind of interfacing do we want: APIs or signaling interfaces? Any IT team will need answers to all of these questions and many, many more along with a clear rationale as to why a particular technology is chosen.
Page 180
The aim of this chapter is to provide an awareness and an overview of the various technologies and standards available out there (see Figure 7.1). Each could easily be the subject for a book in its own right (and indeed, many are—see the References and Selected Bibliographies at the ends of chapters). The Network The idea of a community of interest network is to share resources. In this respect, the network is the computer. Distributed computing architectures of this type are ideally suited for growth and have the ability to recover from failures. They are built on a backbone network that connects the hardware components such as servers, computers, and databases [1]. Such networks come in many shapes and sizes, and in this section we will highlight some of the key concepts involved. The basic features of a network are: • Transmission medium; • Adapters; • Servers; • Clients; • Topology; • Network software.
Figure 7.1 A road map for the chapter.
Page 181
LAN LAN stands for local area network. As the name suggests, this type of network covers a small geographical region (a few kilometers) and is typically used by small to medium businesses (or departments of a large concern). The transmission medium will be one of twisted pair, coaxial cable, optical fiber, or wireless (radio or microwave). A related network is a MAN (metropolitan area network) which uses fiber circuits and covers a wider area (up to a few tens on kilometers). For the clients one will normally find PCs with network interface cards (NICs). These PCs can be connected together in a variety of ways; there are a number of distinct topologies, each with its own characteristics (Figures 7.2–7.5). Mesh • Every node is connected to every other node, that is, fully connected; • Pointtopoint link with all other nodes; • Typically used in older networks with few computers; • Not really practical for today's applications and scale. Bus • Terminated ends; • Passive with each station reading an address; • The basic topology is a tree; • Widely used by Ethernet LANs and MANs.
Figure 7.2 Mesh topology.
Page 182
Figure 7.3 Bus architecture.
Ring • Closed loop of pointtopoint repeaters; • Data is passed around the ring; • Widely used by tokenbased rings (Token Ring and FDDI). Star • Hubs, concentrators, multiport repeaters; • The hubs in turn are classified as: • Active hubs; • Passive hubs; • Intelligent hubs. The protocols used for access control are themselves every bit as varied.
Figure 7.4 Ring topology.
Page 183
Figure 7.5 Star topology.
Polling • Ideal for host to terminal, hierarchical, centralized systems; • The primary station polls secondary station. Token Passing • Uses tokens; • Transmission rates are 1, 4, 16 Mbps; • Typically up to 250 nodes (computers) per ring; • Deterministic method of access control; • Allows node priority; • Commonly used for controlling equipment. The basic mechanism is as follows: 1. Sending node retrieves a token from the network (called a busy token). 2. The station attaches its data and sends the token back to the receiver (still busy token). 3. The token travels to a receiver via a MAU (multistation access unit) or another device to which the nodes (PCs) are connected. 4. Receiving station accepts token, copies data, and sends back data via token. 5. Sending station finishes process by removing data and placing free token back on the network.
Page 184
Ethernet • CSMA/CD—carrier sense multiple access with collision detection; • Developed by DEC, Intel, and Xerox; • Probabilistic access control; • Media access control (MAC). The mechanism is as follows: 1. Checks network for other transmissions. 2. Transmits if clear. 3. If a collision is detected, both stop and send a jam signal. They then wait for a random time interval before trying to retransmit. WAN WAN stands for wide area network, and as the name suggests, it encompasses a much larger area than a LAN (see Figure 7.6). The network will usually consist of a multitude of transmission types: • Private wire/leased circuit; • Switched network (PSTN/ISDN); • Satellite; • Mobile; • Internet. Each node of the WAN may in fact be a LAN. Hence, the network management and control requirements for a WAN are somewhat more sophisticated. The Internet is an example of a WAN, as is an intranet. Connection Control The routing requirements in a LAN are fairly straightforward. However, in the case of a WAN, where the signal may need to cross several networks, some form of switching will be required to route the signal to the desired node (which may be the other side of the world). There are three basic technologies: switching, message switching, and packet switching.
Page 185
Figure 7.6 Typical WAN network.
Page 186
Switching • Uses the telephone network; • A circuit is established by creating an endtoend connection, which must exist before data can be transferred; • After data is transmitted the connection is terminated. There are a number of disadvantages here: 1. Generally uneconomical from the point of view of bandwidth due to idle time (particularly for data). 2. Requires same protocol and transmission speed. 3. Relatively inefficient for data transmission. 4. Entire channel capacity for both sending and receiving stations is dedicated to the call for the duration of the connection. Message Switching • Examples are email or fax; • Store and forward using shared resources; • Transmits complete message as a packet from node to node; • Stored until transmission route is free; • Reaches destination through relays; • Good use of bandwidth; • Due to the storeandforward approach, this is not good for interactive exchanges such as videoconferencing. There are a number of advantages to message switching: 1. It is able to send more messages over a single line for any given time. 2. Less total transmission capacity is needed. 3. Sender and receiver do not have to be available at the same time. 4. A message can be sent to several places. 5. A message is guaranteed to be sent due to rerouting and storing features.
Page 187
Packet Switching Packet switching is becoming the most common transmission mode due to the increasing popularity of TCP/IP and the Internet. Its distinguishing features are: • Message is broken into small packets; • Allows interactive exchanges; • Combines message and circuit switching; • Uses messages that are all the same length; • Packet lengths are 128, 256, 512, or 1,024 bits; • Connectionless service: • Uses datagram packets; • Treats each packet individually; • Packets may be sent in different paths and may be received in the wrong sequence so the receiver must be able to reorder them; • Connectionoriented service; • Uses virtual circuits which are logical connections between two nodes; • Virtual circuit is established before packets are transmitted; • Each packet has a code attached that identifies the virtual circuit to follow; • Since the route is set up before transmission, no routing decisions need to be made; • Circuits exist only for the duration of the call. Network Architectures Which basic architecture do you choose? Peer to peer, client/server, or some other [2]? The answer will depend on the nature of your business, whether it is large or small, likely to grow rapidly or remain static, whether it will use email extensively or publish on the Web, and so on. Such decisions need to be carefully thought through and a sound longterm strategy devised. PeertoPeer Networks In peertopeer networks, each computer is considered a server and holds its own accounts database. Each computer can share resources that it owns, for
Page 188
example, files, CDROM drives, printers, modems, and fax machines. The advantages of peertopeer networks are: • Each workstation is available to others; • No centralized server required; • Security is the responsibility of each workstation; • Each station runs the same software; • Each computer has its own database; • Costeffective to set up for small groups. When the number of workstations in the network increases (as it inevitably will), problems will arise due to administration overheads. For instance, the maintenance of security across many workstations with their own accounts makes it relatively easy for loopholes to develop in which unauthorized users could gain access. The most common example of peertopeer networks is workgroups. A workgroup is a collection of computers grouped together for a common purpose. In any organization, logical workgroups exist, like sales, marketing, accounts, salaries, and support. By allowing the people in each department to share their files and resources, it facilitates the interaction between people and leads to increased productivity. Advanced PeertoPeer Networking (APPN) is a term that has appeared recently. This is seen as a critical enhancement to IBM's System Network Architecture (SNA) and key to future networking for users of IBM networks. Unlike peertopeer networks, APPNs do typically include a central directory server. Peertopeer networks are suitable for less than 10 computers, low security requirements, and slow growth. ServerBased Networks In this case there is a specific computer (or computers) that acts as a server for the other computers in the network. As the name suggests, client/server systems are an example of a serverbased network (see Figure 7.7). The term client/server was first used in the 1980s in reference to personal computers (PCs) on a network. The actual client/server model started gaining acceptance in the late 1980s and is now widely deployed for internet application (most notably, for Web serving). The client/server software architecture is a versatile, messagebased, and modular infrastructure that is intended to improve usability, flexibility, interoperability, and scalability as compared to centralized, mainframe, timesharing
Page 189
Figure 7.7 Serverbased networks.
computing. A client is usually defined as a requester of services and a server as the provider of services. In reality, a single machine can be both a client and a server depending on the software configuration. The servers in a serverbased network or client/server system, may be used for several purposes: • File and print servers; • Application servers; • Communication servers: • Directory; • Mail; • Fax; • Gateways/routers; • Database servers; • Internet/intranet servers. The principal advantage of serverbased networks is first in the ease of backing up (as it can be done from the server directly rather than having to locate several remote machines). Secondly, security is much easier to administer. Clearly a disadvantage is that if the server goes down, it affects several users. Network Operating Systems This refers to the concept of modifying the client operating system to provide a network operating system. A good example of a software package that does just this is Novell's Netware. The client operating system is not greatly extended
Page 190
but does provide a network awareness. Microsoft NT and Citrix are examples of this technology and both are widely used, particularly for intranets. Network Applications There are a host of applications to choose from once the network and the software to allow communication over it are in place. Choosing which particular applications to install will depend on local IT strategy and specific business needs. Quite often there will need to be a compromise between the desire of the IT staff to standardize (and hence make their job a lot easier, especially for support and the purchase of enterprisewide licenses) and the (often diverse) business needs of the department. Here is an example of the types of applications readily available offtheshelf: • Electronic mail: • Attachments; • MIME; • SMTP, POP, IMAP, CCITT X.400, MHS; • Protocol gateways; • Scheduling; • Groupware: • Shared documents; • Routing and tracking; • Bulletin boards and chat; • Workflow; • Middleware; • Usenet news; • Videoconferencing. Interface Software and Standards Of the above examples, middleware (see Figure 7.8) is likely to be the key application in a typical business scenario. Middleware is software that consists of a collection of services or functions that allow multiple processes, which may be running on one or more machines, to interact across a network. In short, it enables distribution [3].
Page 191
Figure 7.8 Middleware architecture.
Middleware can play an essential role in migrating mainframe applications (typically legacy systems) to client/server applications, and for enabling communication across heterogeneous platforms. It is in fact a set of APIs that provides a more functional and applicationspecific interfacing toolkit than an operating system. The key requirements for middleware are: • Locate transparently across the network, providing interaction with another application or service; • Be independent from network services; • Be reliable and available; • Scale up in capacity without losing function. Middleware can take on the following different forms: • Transaction processing (TP) monitor, which provides tools and an environment for developing and deploying distributed applications across a network. • Remote procedure call (RPC), which enables an application to be distributed across the network. Programs on remote systems can be executed as simply as calling a local routine. • Messageoriented middleware (MOM), which provides programtoprogram data exchange and interaction, enabling the creation of a diverse range of distributed applications that can be configured for a particular business need. MOM is somewhat analogous to email in the sense it is asynchronous and requires a degree of intelligence at the
Page 192
receiving end to interpret the meaning of the message and to take appropriate action. • Object request broker (ORB), which enables the objects that comprise an application to be distributed and shared across heterogeneous networks. The main purpose of middleware is to address the application interoperability problems prevalent in most large and established organizations. Regrettably, it is not the answer to everything. 1. There is a gap between theory and practice. Many offtheshelf middleware services use proprietary software, thus making applications dependent on a single vendor's product, which given the reasons for adopting middleware, is not the way to go. 2. The sheer number of middleware services and functions is seen by many to be a barrier to using them. To keep their information technology environment manageable, developers have to select a small number of services that meet their business needs. 3. While middleware services raise the level of abstraction and hence avoid unnecessary detail when programming distributed applications, the application developer still has some hard design decisions to make. For example, the developer must still decide what functionality to put on the client and server sides of a distributed application. This is not a trivial issue and is a subject of much debate. Given the drive to distribute applications, the key to overcoming these problems is to understand the application domain and the business value of middleware services. To determine the types required, the developer must identify the functions needed by the business and these can be placed in three classes: 1. Distributed system services, which include communications, programtoprogram, and data processing services. This type of middleware service includes RPC, MOM, and ORB. 2. Applicationenabling services, which give applications access to distributed services via the underlying network. This type of service includes transaction and database services such as IDMS and Structured Query Language (SQL). 3. Middleware management services, which enable applications and system functions to be monitored to ensure optimum performance of the environment.
Page 193
A significant number of middleware options and suppliers exist to service the demand from organizations for greater interconnectivity, and applications will continue to grow with the installation of more heterogeneous networks. A commonly cited example of middleware in action is the Delta Airlines Cargo Handling System, which uses the technology to link over 40,000 terminals connected to UNIX services and IBM mainframes across 32 countries. By 1999, middleware sales are expected to exceed $6 billion. Hence, it is a significant market. The costs of using middleware technology in systems development are largely dependent on the operating systems and the types of platforms as implementations are unique to the vendor. Not surprisingly, perhaps, this results in a dependence on the vendor for maintenance support and future upgrades. This reliance is good news for the vendor, but could have a negative effect on a system's flexibility and maintainability. Indeed, many observers argue that we are merely replacing one legacy system with another. However, when evaluated against the cost of developing a unique middleware solution, many a business would consider the cost (or risk) against benefits acceptable. Application Programming Interfaces As we saw in Chapter 2, an API is a set of rules, or a protocol, for writing subroutine calls that access functions in a program, library, or component repository. Programs that use these rules in their API interfaces can communicate with any others that use the same API. APIs work with a wide spectrum of applications (i.e., program interface schemes) to facilitate information exchange. These include database access, client/server, peertopeer, realtime, eventdriven, storeandforward, and transaction processing. APIs combine error recovery, data translation, security, queuing, and naming with an interface that comprises a simple but powerful set of actions and commands. To invoke an API, the program calls a sendtype function, which includes parameters for destination name, pointers to the data, and return confirmation options. The API uses the data and performs the communication work for the application. As such, much of the complexity is hidden from the user. However, it should be remembered that although this complexity is hidden from the user, it most certainly is not hidden from the API developer. Consequently, APIs can be difficult to develop. There are four types of APIs that enable data sharing between different software applications on single or distributed platforms: 1. Remote procedure calls (RPC); 2. Standard Query Language (SQL);
Page 194
3. File transfer; 4. Message delivery. Using RPCs, programs communicate via procedures that act on shared data stores. SQL is a nonprocedural data access language that allows data sharing between applications by access to a common database. File transfer allows for data sharing by sending formatted files between applications, via FTP (File Transfer Protocol) for example. Message delivery provides data sharing by direct interface communications of a small message set between applications. Current standards that apply to APIs include the ANSI standard SQL API. There are ongoing efforts to define standards for other types. APIs can be developed for all computing platforms and operating systems and can also be purchased for most platforms and operating systems. All four API types can be used both on homogeneous and heterogeneous applications. However, because of the added complexity required to share data across multiple platforms (e.g., multiple updates and the potential for deadlock), RPC, SQL, or file transfer APIs are best used to facilitate communication between different applications on homogenous platform systems. These three APIs communicate data in different formats (e.g., shared data buffers, database structures, and file constructs). Each data format requires specific network commands to communicate the data properly, which can cause many different types of errors. Therefore, in addition to performing the datasharing activities, these APIs must deal with very many network parameters and hundreds of possible error conditions. Each application must be able to deal with these errors if it is to deliver robust interfaces. A message delivery API, in contrast, has a smaller set of commands, network parameters, and error conditions because this API deals only with messages. Because of this reduced complexity, message delivery APIs are generally preferred when applications require data sharing across multiple platforms. APIs may also exist in many forms and the potential user or developer should make sure they understand the implications of each. They may be bundled as part of commercial software packages or as separately licensed COTS software packages, or uniquely developed by a project using the internal capabilities and features of the applications that need to communicate. In the last case (which should generally be the exception) the development team will incur analysis and development costs since they will need to spend a significant amount of time understanding the internal features of the software application concerned. In addition the cost to develop and maintain the bespoke API will also need to be considered. There are also training costs associated with learning how to use the APIs as part of the development and maintenance activity. Then there are the costs associated with developing and
Page 195
using APIs to communicate across multiple platforms. As already described, network communications add complexity to the development or use of the APIs. In short, APIs are not a cheap option by any means, though the potential payback can be high. Recently there has been a resurgence of interest in APIs, and the view that they are both simpler to specify and develop than signaling interfaces. In truth there is no evidence for this; indeed, the many flavors of APIs and vendor dependencies mean that in fact APIs are no simpler to develop than signaling interface, and perhaps less general. However, there is evidence to suggest that APIs may represent the next wave of computer technology, even though APIs are far from new. The reason is the aforementioned need for greater systems integration (and not just within one company, but between many different organizations) along with mass customization. Nowhere is this more apparent than the telecommunications industry, and in particular in intelligent networks. There seems to be a trend toward pushing the intelligence to the edge of the network so that thirdparty software suppliers can use the network to generate revenue from valueadded services. The API would be into the network intelligence systems, an area of a network that has traditionally been very much offbounds to all but the network operator, with software companies exploiting this interface. Hence, we can imagine the scenario where a network becomes a host to a virtual community of thousands of software suppliers and packages. There are strong parallels with the open interface to DOS, and how that too acted as a catalyst for many niche software products. The weight of support for APIs as opposed to signaling interfaces rests on the programmability of APIs, with the desire, eventually, for a programmable world. This is a compelling vision for the future, but it should be remembered that APIs are not a panacea, merely one way of achieving the goal. CORBA The Common Object Request Broker Architecture (CORBA) is often seen as a somewhat mysterious entity. Ask anyone what CORBA is, and you will typically get a variety of fairly vague responses. CORBA is, in fact, no more than a specification of a standard architecture for ORBs. An ORB (see Figure 7.9) is a variety of middleware technology that manages communication and data exchange between two or more objects. ORBs promote interoperability of distributed object systems because they enable users to build systems by interfacing objects, which could be from different vendors that communicate with each other via the ORB. The implementation details of the ORB are generally not important to developers building distributed systems. The developers are only
Page 196
Figure 7.9 An ORB.
concerned with the object interface details. Hence, each object is effectively a Black Box. The developer only has to know where the door is and how to open it. They need know no more. This form of information hiding enhances system maintainability since the object communication details are hidden from the developers and isolated in the ORB. A standard architecture allows vendors to develop ORB products that support application portability and interoperability across different programming languages, hardware platforms, operating systems, and ORB implementations: Using a CORBAcompliant ORB, a client can transparently invoke a method on a server object, which can be on the same machine or across a network. The ORB intercepts the call, and is responsible for finding an object that can implement the request, passing it the parameters, invoking its method, and returning the results of the invocation. The client does not have to be aware of where the object is located, its programming language, its operating system or any other aspects that are not part of an object's interface. OMG
The vision behind CORBA (see Figure 7.10) is that distributed systems are conceived and implemented as distributed objects. The interfaces to these objects are described in a highlevel, architectureneutral specification language that also supports objectoriented design abstraction. When combined with the object management architecture, CORBA can result in distributed systems that can be rapidly developed and can reap the benefits that result from using highlevel building blocks provided by CORBA, such as maintainability and adaptability.
Page 197
Figure 7.10 CORBA.
The CORBA specification was developed by the Object Management Group (OMG), an industry group with over six hundred member companies representing computer manufacturers, independent software vendors, and a variety of government and academic organizations. Thus, CORBA is an industry (or de facto) standard, not a formal (de jure) standard in the ANSI or ISO sense of the term. The OMG was established in 1988, and the initial CORBA specification emerged in 1992. Since then, the CORBA specification has undergone significant enhancement. The distributed/collaborative enterprise architecture emerged in 1993. This software architecture is based on ORB technology, but goes further than CORBA by using shared, reusable business models (not just objects) on an enterprisewide scale. The benefit of this architectural approach is that standardized business object models and distributed object computing are combined to give an organization flexibility, scalability, and reliability and improve organizational, operational, and technological effectiveness for the entire enterprise. This approach has proven more cost effective than treating the individual parts of the enterprise. Clearly CORBA does offer the potential for very flexible and resilient systems, but the Black Box approach to systems development, with the focus very much on interface development, does have limitations. The most glaring is the lack of understanding what the box does! The answer has been to develop a formal specification language that describes, in detail, what the particular object does in terms of functionality. However, as we will see in Chapter 9, this approach is in itself subject to a number of pitfalls. One final caveat before moving on is to mention the problem of developing spaghetti systems. Systems based on CORBA may consist of hundreds if not thousands of objects, all strung together via interfaces. In reality, maintaining
Page 198
such systems may be just as troublesome as maintaining more traditional and monolithic systems. Time will tell! ThreeTier Architectures An alternative or, perhaps, adjunct to the CORBA distributed/collaborative enterprise architecture is the threetier architecture. The threetier software architecture (see Figure 7.11) emerged in the 1990s to overcome the limitations of the twotier architecture. The latter was comprised of just servers and clients, but as the diversity, complexity, and heterogeneity of systems grew the need for a middle interfacing layer became increasingly apparent. The third tier (middletier server) is between the userinterface (client) and the datamanagement (server) components. This middle tier provides process management and control where the business logic and rules are executed by providing functions such as queuing, application execution, and database staging. Middleware technology is a good example of the middle layer. The threetier architecture is used when an effective distributed client/server design is needed that provides (when compared to the twotier) increased performance, flexibility, maintainability, reusability, and scalability, while hiding the complexity of distributed processing from the user. DCE Developed and maintained by the Open Systems Foundation (OSF), the distributed computing environment (DCE) (see Figure 7.12) is an integrated distributed environment which incorporates technology from industry. The DCE is a set of integrated system services that provide an interoperable and flexible dis
Figure 7.11 The threetier architecture.
Page 199
Figure 7.12 Distributed computing environment.
tributed environment with the primary goal of solving interoperability problems in heterogeneous, networked environments—a problem which, by now, is probably sounding very familiar. The OSF provides a reference model on which all DCE products are based. The DCE is both portable and flexible, for example the reference model is independent of networks and operating systems. It provides an architecture in which new technologies can be included relatively painlessly, thus allowing for controllable systems evolution. The basic idea of the DCE is that it will include tried and tested technology that can be used in part, or as a complete integrated solution. The DCE supports the construction of client/server applications while hiding the complexity of the distributed processing from the user, much as we encountered with middleware. COM/DCOM COM and DCOM (see Figure 7.13) can be viewed as a direct competitor to CORBA. COM refers to both a specification and implementation developed by Microsoft which provides a framework for integrating various components. This framework supports interoperability and reusability of distributed objects by allowing developers to build systems by assembling reusable components from different vendors which communicate via COM (see componentbased development). In this way it is very similar to CORBA, but with the emphasis on reusability. Thus, it is an important concept regarding componentbased systems engineering.
Page 200
Figure 7.13 COM and DCOM.
COM defines an API which allows the creation of components for integrated custom applications. It also allows diverse components to communicate with each other. However, in order to interact, components must adhere to a specification defined by Microsoft. As long as components use this binary structure, components written in different languages can interoperate. Distributed COM (DCOM) is an extension to COM that allows networkbased component interaction. Hence, it will facilitate the construction of componentbased systems where the components span a number of repositories across a network as long as DCOM is available within the environment. While COM processes can run on the same machine, the DCOM extension allows processes to be spread across a network. COM and DCOM are usually viewed as a single technology that provides a range of services for components. This includes services promoting component integration on a single platform, and component interaction across heterogeneous networks. In fact, COM and its DCOM extensions are merged into a single runtime. COM and DCOM specify a lowlevel technology that allows components to interact. For most applications it would be useful to have higher level services. OLE and ActiveX represent such higher level application services that are built on top of COM and DCOM. OLE builds on COM to provide functionality that enables object linking and embedding used in the creation of compound documents (documents generated from multiple tool sources). ActiveX extended these basic capabilities to allow components to be embedded in Web pages, and more recent developments have incorporated all of the elements into Microsoft's Distributed Network Architecture. What Should I Use? While there are numerous permutations of all of the technology covered here, there are essentially two alternative paths to chose: proprietary or open, and by open we mean Webbased. Hence, an organization can attempt to standardize on the basis of a single supplier such as Microsoft or Lotus, or they can accept that heterogeneity is a fact of life, is likely to be so for the foreseeable future,
Page 201
and decide that the easiest approach therefore is to develop a common interface, a Web browser such as Netscape, Internet Explorer, or Mosaic. Which to choose is the subject of much debate within the industry as a whole, and there are convincing advocates for both approaches. By opting for a single supplier, the hope is that the range of applications developed by them will interwork seamlessly and that interfacing and integration will be quite painless. Such an outcome is possible, but in fact reality has shown on a number of occasions that this is by no means guaranteed. Slight variations of products often exhibit a reluctance to converse with one another across any network. In addition, and this is perhaps the crux of the matter, if your organization does not already have such a strategy, then replacing your existing systems (which no doubt come from a spectrum of suppliers) with a proprietary solution is potentially a massive task. Much will depend on the size of the organization, of course. Many have looked at this option and decided to go down an alternative path. In today's business world, there is only one other real option, and that is to use a Web frontend to access your systems. With this approach each system is accessed through a Web browser over HTTP links. The link is in fact little more than a remote procedure call, but it achieves this in a very flexible and dynamic manner. The point here is that TCP/IP, HTTP, HTML, and so forth are the nearest things to universal standards in the software industry. As such, they provide an ideal vehicle for addressing the inherent variations in platforms and desktop technology and not only within a single organization. With ebusiness set to boom, and joint ventures and mergers becoming the norm, the pressure to integrate different information systems from different organizations is growing. Of course, there is no such thing as a free lunch. The price of achieving such open systems is the need to introduce a middle layer which allows Web access to the relevant information systems, usually via the copious use of Java, lots of Perl and CGI scripts, or Web application server technology [4]. While the lead times may be longer than those required for a proprietary approach, the payoff is a system that can be accessed from virtually anywhere. And, with that level of availability, it is likely that a business will not want its systems to be accessed from anywhere and so security will become an important issue [5]. Summary The chapter began with a very brief overview of the most basic element of any distributed system—the network. There are a variety of basic types to choose from, each well suited for particular needs. The desktop hardware and software applications that rely on the network to provide a communication path need to
Page 202
be largely independent of it. Hence it must support applications without intruding on their operation. Clearly there needs to be some control here, otherwise anarchy will quickly ensue. Developing a sensible and effective framework for all elements of the information system is the job of the systems architect. It is he or she who must decide whether to choose CORBA or DCOM, whether to use middleware, whether to use signaling interfaces or APIs. This is not a trivial task and the job of the systems architect is often underrated. It will no doubt be apparent by now that definitive standards are difficult to come by in systems engineering (we will have more to say about this in Chapter 9). There are always options, and there are always vendors who would be delighted to lock you in to their particular way of doing things. For many organizations, the proprietary approach is the right one; for others who want a more flexible, open system, it most certainly is not the way to go. In this latter case the better solution is to implement a Webbased system. From the point of view of interfaces and integration, each has its merits. We would hopefully expect all the components of a proprietary solution to interface seamlessly and painlessly. Likewise, we know from daytoday experience that a Web browser enables access to a whole host of other platforms. This chapter has laid out many of the options available to the systems builder. The pros and cons are explained in each case, so that sensible choices can be made. References [1] Norris, M., and S. Pretty, Designing the Total Area Network, Wiley, 1999. [2] Muller, N., Desktop Encyclopedia of the Internet, Norwood, MA: Artech House, 1999. [3] Norris, M., and N. Winton, Energize the Network, Reading, MA: Addison Wesley, 1996. [4] Norris, M., P. Muschamp and S. Sim, The BT Intranet—Information by Design, IEEE Computer, March 1999. [5] Oppliger, R., Internet and Intranet Security, Norwood, MA: Artech House, 1998.
Page 203
8— Towards ComponentBased Engineering All truths are easy to understand once they are discovered; the point is to discover them. Galileo Galilei
Only a couple of years ago, there were two types of software engineers: those who had heard of componentbased development and those who had not but did it anyway. Today it would be difficult to find any software engineer who has not heard of CBD, but how hard is it to find people who are doing it? As we have seen throughout this book there are many differing viewpoints on what constitutes a component, an architecture, an interface, and CBD. If by CBD we mean the use of smallgrained GUI type components, then most of the software industry is doing CBD. If we mean larger entities based on CORBA, DCOM, or EJB, then there is a growing community. If we mean wrapping legacy systems, we will find a different group. If we mean the use of largescale business components, then there is hardly anybody. Nearly all software tool vendors will tell you their product is componentbased. The popular software journals will tell you the whole software industry is using components. But ask them about methods for componentbased approaches, componentoriented design tools, ways of describing components, configuration management, or architectures for business components, and you will get very little response. Many vendors look totally blank at these questions and wonder why you are even asking. They will say, ''We support CORBA (or DCOM or Java)," as if this should explain everything. The whole componentbased engineering domain is very immature. People have only recently started to address these issues, and few have answers.
Page 204
Vendors are rushing to label their products as CBD with little idea of what that means. A few of the more componentliterate vendors have coherent component approaches, but generally they want to sell you their method or tool or consultancy. Most of the professional bodies still treat components as though they were just better objects and are failing to address the practical issues of using components. The Butler Group CBD Forum [1] is one of the few organizations that is trying to cut through the hype and establish some practical approaches that could be used by all. Is CBD different from componentbased engineering? In our view it is, at least today. CBD is about building software applications from software components. It is about making software, buying software, assembling software, and delivering software all from a component perspective. CBE is doing the same, but at a systems level. It is about building those systems that support the enterprise from large components. It is about understanding the goals and the processes that drive the business and assembling a set of components that will deliver them. It is about supporting the business without writing any code. This may be unattainable in its purest sense, but it is a goal to which most large companies aspire. The amount of code that is currently being written is becoming unsustainable. It is too costly and too slow to develop. This whole life cycle view is critical if we are to address the real problems facing large organizations, and not push the consequences of illadvised, local economies to other parts of the business. It goes without saying that economy, in the widest sense, is the key, and that current system costs and time to market need to be radically reduced if companies are to achieve the level of response that is so desired. So in answer to the question in the title of this section, we can say that using components is easy, and doing methodical CBD is possible, doing componentbased systems engineering is very hard. In the rest of this chapter we will look at some practical approaches, mostly at the CBD level, that can be used to at least make a start. ComponentBased Development What we want to do is build, not buy—compose, not code. How does that affect the typical software life cycle? Well, it is not radically different. Companies using good software practices can easily migrate to using CBD. A component approach is not a completely new way of working, but brings different viewpoints and some additional steps. In Figure 8.1 is a simple waterfall model modified to account for CBD. We only use a waterfall model for the sake of simplicity. We take it as read that,
Page 205
Figure 8.1 A simple CBD software life cycle.
as with all waterfall models, we can in practice iterate around various parts of the process. Of course, iterating around adjacent phases is to be preferred to larger iterative loops which becomes progressively more expensive. We shall also see later that there is not such a hard and fast demarcation between the stages. However, such a simple model provides a useful starting point. Let us look at each of these stages in a bit more detail. System Specification, Analysis, and Design This stage typically consists of steps shown in Figure 8.2. Business Analysis Where does CBD start? With some requirements you might say. But many would argue that the business analysis phase comes before the requirements capture phase. This already points to the fact that waterfall models, such as the one we are presenting here, are a drastic oversimplification. In practice there may be iteration around the requirements capture and business analysis phases from different viewpoints. It also very much depends on the scope of the task being undertaken.
Page 206
Figure 8.2 A simple CBD software life cycle: system design and specification.
Page 207
If the task is to reengineer the whole or a major part of the business into components, then the major first stage will be business analysis. In effect we already have a simple requirement to "reengineer the business to make it more profitable." The business analysis phase is responsible for determining what this means and producing a highlevel model of the business showing the drivers, the inputs, and outputs. When done at this very high level, we are effectively involved in business process reengineering (BPR) and such a highlevel model in many BPR systems would be known as a value chain. The value chain simply shows what the business does, how inputs are transformed into outputs, where the business adds value to this process, and, of course, where it makes its profits. In an ambitious vision of componentbased engineering, this value chain would be decomposed into a more detailed process model that would be implemented by infrastructure components. This may be system components or component processes. At the same time the company's product ranges would be analyzed against these value chains with the aim of producing a product family that could be delivered using a product line component architecture. At the end of this business phase the major systems and product components will have been identified and be ready for further detailed specification. With the current state of maturity of componentbased engineering, there are probably few companies that go through such a process. Of course lots of companies will undertake BPR and business analysis, but not from a component viewpoint. Thus, many componentbased projects may not really start until the requirements specification stage (or even later). This may well mean that there has to be a pseudobusiness analysis phase after requirements capture to understand if components to be produced in the project form part of a larger picture. The vision for attaining a truly effective componentbased approach is that it must be a fundamental part of the highest level of business planning. It must also be a natural part of the decomposition down through specification, design, and development stages rather than being added in at the coding stage as an afterthought. Some of the more advanced process modeling tools will support highlevel value chain modeling right down through process modeling, data modeling to OO analysis and design, and even code generation. Vendors of such products will admit, if pushed, that hardly anybody actually does this, but it is clear that tools are developing, and we can expect many more people taking this holistic approach to systems design over the coming years. Requirements Capture Once we understand how our system or product fits into the bigger picture, we can start to capture detailed requirements. One might ask whether the use of a
Page 208
componentbased approach makes any difference at this stage. Many purists will argue that requirements capture should be done without any consideration of the method of implementation. While this is true to an extent, in practice we always have to work within a specific context. For instance, if the intention is from the outset to build the product or system from components the company has already invested time and money in producing, then we would expect the requirements to at least reflect that. Perhaps of more importance, it is vital to ask specific questions about the nonfunctional aspects. Typically: • Is this an isolated system or one of a large number that the company makes? • Is the product part of a product family? • Is this company building the system by itself or in collaboration with others? • What performance requirements must be met? • What will be the scale of deployment? None of these questions require answers that delve into implementation issues, but they will have a significant impact on future decisions about the use of components. If requirements capture has been preceded by a detailed business analysis phase, then it may be that many of these questions will have been answered. By the way, we are freely interchanging the term product and system when describing the thing that is being built, as most of the issues discussed are common to both. If the system being built is an isolated one (and we can argue whether in a company of any size there is such a thing), then the decision about using components can be largely left to the development team. They may choose to use components to improve their productivity, either by using components they already have or by creating components that might be reused in the future. On the other hand, if the system is one of a range the company produces, then decisions about component reuse should be made earlier, at the architecture and design stages. Ideally, they should be part of business analysis. Taking this a step further, if the product is part of a tightly coupled product family, then the requirements capture should be done for the whole family. The specification should be written directly in terms of products built up of components and options and with clearly defined flexibility points. Only then will it be possible to map this onto a product line architecture that can implement the product family. Specifying a single product with the intention of creating a product family around it at a later stage should be avoided if at all possible. Splitting an im
Page 209
plemented product into components is a difficult and costly business. It can be done and may be worthwhile if it is suddenly discovered that a product has much wider marketability than first thought. We will give an example of such a project much later. This is a good argument for using a componentbased approach even if a product family is not envisioned. It is much easier to generate a range of variants for a product that has been built out of sensible components in the first place. It is also important to understand at the specification stage any organizational or cultural aspects that will affect the product or system. Is it a collaborative venture with another company or a consortium? Is it to be produced by a particular division of the company (a decision often made for political or commercial reasons rather than technical)? Is it targeted at a part of the market that has a wellrecognized architectural style (e.g., Microsoft Windows or the plugin market for graphic design tools)? As we saw when discussing architectures in Chapter 5, these types of requirements will have a dramatic impact on decisions about the use of components and choice of architecture. Sound technical decisions can come unstuck in a big way if there is no understanding of the context in which the product or system will be developed and deployed. Perhaps most important of all, and possibly the most often neglected, is consideration of performance criteria and scale of deployment. So many large software projects fail or are delayed because software technologies used to build prototypes will not scale up to meet operation requirements. Products built on any type of WWW or distributed component technology are very prone to this problem. It can be very easy to build an impressive prototype in the laboratory where two or three users are connected to a server in the room next door. When the system goes live with 100,000 users using servers connected across a network clogged with Intranet traffic, the systems can fall apart very rapidly. For example, British Telecommunications plc (BT) has one of the largest corporate intranets in the United Kingdom [2]. The most popular site on their intranet provides realtime access to the current share price and company news. This has transformed employee awareness of the impact of the commercial world on the company. So much so that when there was a dramatic change in the share price as a result of the proposed merger with MCI being called off in 1998, the network ground to a halt under the weight of people trying to read the news. If there had been operational systems making use of the same network, then there could have been a significant performance impact on customer service. Another experience from BT was a survey that found that nearly every major system development was planning to make its system Webenabled in the coming year. Hardly any of these projects had discussed the potential network loading with the unit that managed the corporate network. Everyone just assumed the corporate intranet had infinite bandwidth.
Page 210
Very few system designs undertake detailed performance modeling of the systems and the networks that connect them. Tools and techniques for doing such modeling and simulation are still in their infancy. It is therefore essential to ask key questions at the requirements capture stage; the answers will be used to make key design decisions later on or to sanity check those decisions. Make sure you ask the marketing department: "How many of these do you want to make?" and "If it is a real success how many will you want then?" Make sure you understand the issues of scale so you can challenge the design and developers later on. When they recommend the use of the latest distributed technology, be prepared to ask them: "Will it scale to 100,000 users?'', "How many users can one server support?", and "What happens when the network times out halfway through the transaction?" If you do not capture these issues at the requirements capture stage, it is very likely no one will ask them later. Finally, a key aspect to consider is requirements traceability. This is an important aspect of any software development, but of particular importance when considering a component approach. The whole point of using components is being able to encapsulate functions into reusable and replaceable blocks. Thus it is essential when specifying components (at a later stage in the life cycle) to be able to map the original requirements onto the components that deliver that function. Choice of Component Approach We now have a requirements specification and will have asked many of the key questions that will enable us to make decisions about the use of components. The first and most important question is: "Why use components?" Instinctively we may feel that using components is a good idea, but we must not go blindly on without understanding if it is really appropriate and, if so, then for what reason. Just because a component approach could be used does not mean it should. It might be too costly or involve too much risk, especially if the potential users have little experience using such techniques. We feel that too often software developers rush into using the latest approach or technology without any real consideration of the benefit. The use of a rapid applications development (RAD) approach is a case in point. It can be a very valuable method of building systems, but it is not appropriate to use it in every case. The dynamic systems development method (DSDM), produced by the DSDM Consortium [3] introduces the concept of a suitability filter which encourages prospective users to ask a set of key questions that will determine if using the DSDM approach to RAD is appropriate. It is useful to use a similar idea for CBD. Table 8.1 shows some key questions to ask and considerations that may help in deciding if a component approach will be useful or appropriate.
Page 211
We believe it is best to take a component approach to aid good design and engineering practice, particularly to create a product family. If you do this well then the benefits of reducing costs and time to market will follow. If you start out purely with cost cutting as your aim, you are likely to be disappointed. If you have specific benefits you wish to achieve then it is important to make sure they are properly identified and quantified. Table 8.2 shows some typical aspects to consider. It is important to understand at the outset what type of thing it is that you are trying to componentize and whether it will be feasible and beneficial within the context of your organization. Table 8.3 gives some pertinent questions. It is a common mistake to think that once you embark on a component approach then you must turn everything into components. As with most things, just do what makes sense. It may be that the benefits of componentization we introduced above do not apply to the whole system. Some parts may be worth componentizing for one reason while there is a different reason for other parts. We remember an enthusiastic university researcher approaching the authors with a method for reengineering legacy systems into Java components. He could not understand our lack of enthusiasm. "Surely you want to turn all your legacy systems into Java," he said. "If it's not broke, don't fix it," we responded. Table 8.4 will help you decide how much to componentize. You have now determined the benefits of componentizing your system. You know how much of it to do and why. So let us leap and get started. Hang on a moment, it may be good to pause and consider our approach in Table 8.5. By now you may well be asking is it all worth it? If you are not sure, then it probably is not. But we have deliberately provided a lot of questions to challenge whether a component approach is the right one. Most of these questions, would in fact apply to any development approach, and the only reason they seem onerous is that we are not normally in the habit of asking them. Perhaps we should be. Many projects are less than successful because they leap straight in without proper consideration. As we have said before, CBD is not in itself new, it is about good engineering practice with some additional perspective. Specify the System In a conventional development life cycle, the information from the requirements capture phase and business analysis would be documented in a formal written specification. Increasingly RAD and OO techniques are making this phase more informal and often replacing the specification with a succession of demonstrators and prototypes. The use of techniques such as use case and object modeling can put back that formality in a form that can be directly used in software development. The increasing availability of usable CASE tools and the rise of notations such as UML continue to make these approaches more and more effective.
Page 212 Table 8.1 What Is the Perceived Benefit of CBD? What Is the Perceived Benefit? Cut costs
• Component reuse may achieve this. But: • Reuse is hard; • May initially cost more (typically 2–5 times more); • Typically a component needs to be used at least 3 times before it is mature enough for general reuse; • Buying commercial components may be cheaper initially, but beware of hidden costs: integration problems, maintenance upgrades, and charges for developing variants. • Cost cutting through reuse is a longterm aim. You need to have some components before you see the benefit. • Cost cutting is often stated as the prime reason for using components, but is rarely achieved. Do not build a business case that relies on achieving this in the first project.
Deliver faster
• Using a component approach to encourage good design practice will benefit the life cycle time. • Design phase may be longer, especially if there are few people with analysis skills.
Improve quality
• Good design practice with structured and encapsulated functionality will inevitably improve software quality. • Testing at component level can ease overall testing. • Reuse of components will improve quality through repeated use and testing in new contexts. • Check the component's credentials. Reuse of useful, but badly constructed components may proliferate poor quality.
Ease maintainability
• Building components that can be replaced can significantly ease maintenance, particularly regression testing. • Good design practice with wellstructured and encapsulated functionality will aid understanding and ease maintenance. • Recognizing the parts of systems that change rapidly due to technology change or business change and building them as components will allow easy replacement and will allow planned product evolution.
Build more complex things
• The most significant driver at the systems level: • It is just not possible to build everything from scratch.
(table continued on next page)
Page 213 Table 8.1 (continued)
Create a coherent family
• Complex systems get too highly coupled unless a component approach is used. • Very difficult to achieve. • Perhaps one of the most attractive reasons for using product components. • Product family looks sensible, well planned, and attractive to customers. • Delivers flexibility in a constrained and managed way. • Benefits of cost cutting and faster delivery more easy to achieve across a range of products.
In all but the most critical applications, the best results usually come from a carefully managed combination of the two. Weighty written specifications are best avoided except where there are very onerous commercial, military, or legal requirements. However, there are things that must be written down, particularly the nonfunctional aspects we discussed earlier. Do not believe developers when they say the specification is in the object model. If the use of components is a formal part of the specification, maybe components are being provided by vendors or provided to other members of a consortium, this then puts additional requirements on the specification. Smallgrained components can be described using a notation such as IDL or UML, but large components go beyond what such notations can support. This means component interfaces must be described in words. This leads to misunderstanding and flexible interpretations. Specifying large components is a very immature area, and those few components that are available are usually supplied based on a particular vendor's CASE tool and will probably come with a vendorspecific model. With the rise of business frameworks, we will expect to see the framework have a detailed specification. Components supplied to fit such a framework will then inherit that specification. It still remains to be seen how the business and functional aspects of those components will be specified. Preliminary Search for Components It may seem a bit premature to search for components at this stage. After all we have not even chosen our architecture yet. However, it is worth having a look around the commercial market and inhouse to see what is available. If there is not much, then we carry on as planned. However, if we find a rich seam of components that may do the sort of things we want, then it is useful to have
Page 214 Table 8.2 Quantifiable Benefits of CBD Quantifiable Benefits Costs
• What is the likely reduction or impact on: • Overall finances; • Cost of skilled people; • External expenditure (e.g., buying in components); • Tools acquisition and software liceneses; • Specialist resource (e.g., architects); • Training; • Adherence to standards (e.g., attendance at standards bodies, developing inhouse standards, and so forth)? • Can you quantify how much this would cost anyway and what would result from using a component approach? • Is the benefit worthwhile? • How certain can be you be of achieving the cost benefits?
Time scales
• Can you identity the reduction or impact on the various stages of the life cycle? • Is the overall life cycle reduced? If not, is there benefit in reducing one part of the expenses of another (maybe where there are bottlenecks)? • What effects does using components have? • How dependent are you on component suppliers or collaborating companies? What effect could this have on your time scales? • Have you got time to develop the components yourself if your supplier fails to deliver? • What are the risks? If time scales are extended will you: • Incur punitive damages; • Miss a window of opportunity; • Create a breakin service; • Let down a collaborator?
Quality
• What sort of quality profile does your product or systems have: • A high dependency system; • A market leading product; • A speculative leading edge product? • Do you understand the effect introducing components will have? • Are you trying to specifically improve quality?
(table continued on next page)
Page 215 Table 8.2 (continued)
• Do your customers want better quality or are they happy with a cheap and cheerful product? • Will your customers recognize and value an improvement? • What are the risks if quality deteriorates?
Maintenance
• Do you understand your current maintenance costs? • What is your main maintenance activity: • Bug fixing: components may help if they improve engineering practice. Components will not fix poor development practice. • New requirement: if new requirements can be expressed as part of an evolving product line then components can have a significant benefit. If requirements are diverse and unpredictable then components are not so much help. A component system can be easier to evolve, but if new requirements require complex interactions then partitioning into components can often be difficult. An 00 approach may be easier. • Changing requirements: if this is due to poor planning or requirements capture, then it is better to fix the problem rather than regard CBD as a panacea. • Technology changes: component layering can be substantial help in providing isolation against technology change. • Business changes: replaceable components handling business aspects such as tax calculations, currency conversion, or legal rules can make keeping the product uptodate very easy.
these options in mind at the later stages. In particular, if we find a business framework that seems to support components with the type of function we require, then this would make it a strong candidate when choosing our architecture. But keep them as options. Do not jump to a decision yet. Choose an Architecture We have talked extensively about the need for architecture and the factors that influence the choice of an appropriate architecture or at least a method of structuring software. Chapter 5 provides information about the techniques available and the choices that can be made. Consideration of architecture is important for any software development, but it is imperative for a componentbased approach. Table 8.6 gives a reminder of some of the key issues. As we described in Chapter 5, the choice of an architecture, and in particular an architectural style, embodies many of the decisions and constraints about producing the system. This is why we believe it is so important to have
Page 216 Table 8.3 Why Use Components? What Do You Propose to Componentize? An isolated system
• Relatively easy to achieve: • Architectural issues are not too complex; • Good approach if skills are limited; • Does not require complex configuration management. • Benefits are mainly good engineering practice, quality, and ease of maintenance. • Unlikely to cut costs.
A product family
• Very attractive approach. • Needs careful planning. • Must be done in collaboration with marketing department, but it is a concept they can understand and relate to. • May require more cost upfront, but benefits can be substantial. • But the product family must map well onto a productline architecture. Products that are delivered by complex interacting systems (e.g., telecommunications) are more difficult to apply this to.
Corporate infrastructure
• Makes good sense, but is difficult to do. • Requires very careful architectural design. • Can be very costly (it is costly not to do it, but these costs are hidden). • Business case is very difficult to write. It is difficult to show direct revenue generation from this sort of infrastructure expenditure. • Needs a senior management champion. Pretty much an act of faith. • Needs the right corporate culture. Only works in organizations that have strong, centralized control of IT or who have a strong culture of cooperation. Pointless to try this in very disparate, distributed, or anarchic organizations.
ascertained nonfunctional requirements, the real drivers for using components, and the availability of existing components before we get to this stage. It is now that we start to make the real decisions that affect the whole concept of the design and implementation. It is here that we define the system components and their relationships.
Page 217
Unfortunately there are few guidelines about how to go about this process, and we cannot lay down a method to follow. Many of the initiatives that we reported on in Chapter 5 are working towards ways to help people do this. We recommend that readers look at some of them and choose parts from each to suit their own particular circumstances. There is a wellknown answer to the question: "What is an architecture?" The answer is, "It's what architects do!" This is never truer than at this stage in the life cycle. Good architects are worth their weight in gold. It is not that they are necessarily highly technically qualified in architecture, but that they have a wealth of knowledge and experience that enables them to weigh up the various options and constraints that enable them to make those key decisions that are needed at this stage. System Analysis and Design We now come to the real meat of this stage of the life cycle. It is here that we need the most help in order to identify and design the components. Ideally, this phase should be driven from the business model and process models previously developed. Depending on the particular nature of the project, then a certain amount of component modeling may have already been done in these earlier stages. In fact, if we could really achieve the vision of a top to bottom componentbased approach, then it is the business analysts who would assemble wellunderstood, and previously defined business or process components to deliver the business goals. The role of systems development would be to ensure that the systems components could enact the process steps required. Thus, if there is any real difference between componentbased engineering and other software engineering techniques, it is this focus on business process as the driver, rather than purely functional requirements. There is still much debate within componentbased circles about where component specification and design takes place. We have already seen that it can occur in several places, and it seems likely that a process of iteration and refinement is realistic with more and more detail being added as one gets closer to the actual coding stage. This also makes sense when we consider the wide range of component granularity that we have discussed throughout this book. Even with conventional software engineering, this step is not all that easy, but at least we now have some wellestablished software design methods that can help us. There are many methods around, but some of the key ones that will be familiar to many readers include: • Information engineering; • Yourdon; • Dynamic systems development (DSDM);
Page 218 Table 8.4 How Much to Componentize? How Much Do You Propose to Componentize? Whole system
• Do the reasons for componentizing apply to the whole system? If not, would you be better off just doing bits of it? • Are you starting from scratch? It may be hard work to do the whole thing as components. • Are you trying to componentize an existing system? Ask yourself why and choose the components to match your needs. Do not componentize the whole just because it is there. • Big or small components? This is partly determined at the system analysis stage, but make sure the granularity of components correctly matches the benefits you expect to gain. • Will your components have the potential for reuse? Some parts of the system may be very specialized. If it is worth componentizing for ease of maintenance, then choose the scale components for this purpose.
To isolate pieces of function
• Why do you want to isolate them? • Good engineering practice. A good reason—it will aid understanding if done correctly. • To allow ease of maintenance. Another good reason provided that you have to do frequent maintenance on this part.
To allow replacement
• Yet another good reason, but make sure in practice the new part will fit. Where is the part going to come from (another team, bought in)? How soon will you want to replace it? How much will technology have moved on in the meantime? If the architecture, architectural style, or underlying component execution environment is not compatible then it will not work. Think of all the PC manufacturers who will sell you upgradeable motherboards, yet whenever there is a significant increase in processor speed your old mother board is useless!
To allow others to • Have you defined architecture and interfaces standards? use your • Are your prepared to properly test and document your components in their offering? systems • Will you support it and produce variants? • How context dependent is it?
(table continued on next page)
Page 219 Table 8.4 (continued)
• Can they use it as a Black Box or do you need to tell them all your assumptions? • Have you thought about configuration management, repositions, or corporate data standards? If you are not prepared to do this properly, the other team will end up either writing their own anyway or modifying yours beyond recognition. In either case you wasted your time!
To provide services to others
• This commonly applies to the wrapping of functions within legacy systems or clientserverbased systems. It is a very attractive way of providing infrastructure services. At one level it can be very easy to do, but it can rapidly get out of hand if not managed properly. Some things to consider: • Is the function clearly defined and understood? • Have you defined interface standards? • Do you understand any interaction problems with other parts of your system? • How will people get access? • Have you considered performance effects on your system? • Have you established agreed quality of service levels? • Can you achieve them? • Are you going to treat them as serious customers of your system or are they at the end of the food chain? • How will you communicate changes in your components to dependent users? • Have you got a configuration management process in place? • Are you prepared to Make changes if they request them and can you deliver within reasonable time scales?
• Booch; • Objectory; • Structured development method (SDM). Given the current state of maturity of CBD, it is not surprising that none of these methods specifically address componentbased approaches. Some of them are object oriented which, depending on your view of how objects relate to components, may be of help. However, more recently a number of vendors have been producing methods that claim varying levels of component support including:
Page 220 Table 8.5 Approach to CBD Your Approach to Componentizing All in one go
• Do you have to do it all in one go? If you are componentizing and existing system, can you do it in bits? • If you are starting from scratch, do you need to build all the components before you can assemble something useful or can you have a prioritized and phased approach?
Resources
• Do you have the skills necessary to do this? If not: • Can you buy them? If so, can you afford it? What is the risk of depending on contract labor? Have you got a program to transfer skills to your own people? • Can you train people? Have you allowed time for this? • What sort of team do you have? If you are starting out in CBD, it is best to do it in a small team. Working with large or distributed teams requires a large degree of architectural understanding and definition. • Do you understand what roles you will need? Are you going to have separate teams for build and assembly? • Are there commercial components available? Will these be of benefit to or impact your time scales and costs?
Tools and methods
• Have you got appropriate design and code generation tools? • Do tools exist? • Can you buy them? Have you the time and budget for acquisition and training? • Are you using standard software development methods? Do they support CBD? Do you know of any that do? • How dependent on tools are you? • For using a particular design method; • For using commercial components; • For sharing components with other teams. If you are trying to do CBD within large or distributed teams, we believe that common tools are essential to cut down on errors due to false assumptions and architectural mismatch.
Component management
• Have you thought about who is going to own and support components? • Have you got a funding model to support this?
(table continued on next page)
Page 221 Table 8.5 (continued)
• Have your internal customers (marketing department, systems specifiers, people who wish to use components) agreed to the funding model? • Have you got common standards in place? • Have you got a configuration management process in place? • Have you got configuration management tools?
Testing
• Have you got a testing team in place? • Have you got independent testers? It is very important to have independent testing of components, as the developer will only test within the context in which the component is expected to be used. Other users of the component will try and use it in all sorts of unexpected ways. Testing and documentation of boundary conditions is essential.
Boughtin components
• Are you going to test boughtin components yourself? • Have you bought components to known standards (quality or technical)? • Does the use of these components require specific development methods, tools, or architectural styles? • Have you done any vendor assessment or qualification? • Have you taken any steps to reduce the dependency on commercial components? • Source code ESCROW arrangements; • Punitive damage clauses; • Established shared risk partnerships; • Identified alternative sources; • Have you got agreements with the vendor about fixing bugs or making modifications? These are major issues in what is a very immature market. The hardware component industry went through similar problems. Over time the quality standards became such that people were confident to use components without separate testing.
• Select Perspective (Select Software Tools); • COOL: method framework (Sterling Software); • Catalysis (ICON); • Unified Process (Rational Software Corp.); • Iframe (CMG).
Page 222 Table 8.6 Issue for Choosing an Architecture Architecture Corporate architecture
• Does a corporate architecture exist? Are you using it? If not, why not? It is not absolutely essential for projectbased reuse, but if any enterprisewide reuse is planned a largescale architecture is essential. Building systems without an architecture leads to largescale system integration issues. Just placing components in a repository and hoping people will use them is largely a waste of time.
Are there domain architectural standards?
• If there are domain standards, then customers will expect you to adhere to them. Domains with a plugin culture can provide many opportunities for original products and for third party plugins.
Product families
• Are you building a product to fit into a product range? • Is there an existing productline architecture? If not, maybe you should consider establishing one.
Architectural styles
• Is there an established style in the organization or domain (distributed object technology, clientserver, and so forth)? • Is this likely to persist or may it be changed by mergers, collaborations, or impending new technology?
As yet none of these is a complete approach to CBD, and we will discuss some of these methods in more detail later in the chapter. Of course many of these approaches, and certainly the ones specifically aimed at CBD, will help us define the components of which the system will be comprised. But is the definition of components part of the architecture phase or the design phase? Or are these methods really methods for architecture and design? It does not really matter. What matters is having a sensible approach for identifying components and their relationships at a conceptual level and then being able to go through a structured decomposition to put in the detail. Anything that helps us do that is to be welcomed whatever we choose to call it. Production Engineering We have called this phase of the life cycle production engineering because it is at this stage that we must ensure that not only can the product or system be de
Page 223
signed, but that it can be practically built from the resources that the organization has available. Of course as with any life cycle model, we can debate whether it really is a distinct phase. In practice, such aspects will be considered during specification, architectural analysis, and design. However, we think it useful to put in a distinct stage as a check that all the necessary issues really have been addressed. So often key issues such as performance, quality, and reliability are given perfunctory analysis or left to the developers. Typical production engineering steps might include those shown in Figure 8.3. When we take a component approach we are more apt to think of software development as two distinct phases: development and assembly (often called the software factory approach). But we also add a new dimension because we first need to make sure that the components can be built, found, or bought. If we can not do this in a satisfactory way, the specification or design may have to be refined or a compromise sought. It is this stage before development that we call production engineering. It bears more resemblance to production engineering in a mechanical or electronic environment than in conventional software engineering where there is more temptation to dive straight in and start writing code. Classify Components We probably raised some confusion earlier by questioning whether identifying components was part of the architecture phase or the design phase. Now we seem to have added to the confusion with this classification step. However, what we really mean is not the process of identifying components from scratch, but deciding how we need to handle the production and acquisition of these components. In any system design, not all components will be equally important. Of course they may all need to be present before the system will work, but their effect on performance, the overall system and the success of the project will vary. Some components we may wish to buy in, others we know will have to be written inhouse. Some will be some smallscale GUI components while others, we hope, will be largescale business components. Some components will be unique and critical to the system, others may be widely available from a number of vendors. We also saw earlier that we might use components for different reasons (reuse, ease of maintenance, functional isolation, and so forth). These different reasons for using the components will affect the specification of their nonfunctional attributes. Therefore we need to classify all the components in our system, and specify their nonfunctional attributes and their production requirements. We may also identify some of those components as critical to the system such that we will do much more detail production engineering and risk analysis for them. We can then take our shopping basket in search of some of these components.
Page 224
Figure 8.3 A simple CBD software life cycle: production engineering.
Page 225
Readers who can remember as far back as Chapter 5 will recall that we were very dismissive about the idea of searching a warehouse for useful components. However, in that example the searching step was almost at the start of the process. There was no concept of how the components would fit together, just a vague hope that useful things would be found. In this life cycle, we have already put in a lot of work to determine how the components will fit. We had a detailed business analysis phase. After we had done requirements capture and specification, we had a preliminary look for commercial (and inhouse) components and frameworks that might be suitable. We then chose our architecture, in particular taking account of nonfunctional requirements. We then did detailed analysis and design and, finally, in this stage we classified our components to understand the key production issues that would affect them. Thus we are not going out into the world in the hope of finding something (finding anything!), but we know exactly want we want, why we want it, and probably even where to look. Identify Available InHouse Components There is a view that we would identify inhouse components by searching a company database or repository to identify components that match (or nearly match) our requirements. Many companies (including BT) have tried setting up such repositories and putting useful things in them in the hope of promoting reuse. Most people can point to some examples of success, mostly involving GUI components, but these rarely have any significant impact on the business. To a large extent this phase is actually redundant. It should be obvious that if the business analysis, architectural, and design phases have been done correctly then we will already know about, and have taken account of, the components that already exist. Alternatively, such work will have determined that such a component should be bought in, for corporate use, in which case there is no point searching for it. Small components that might only be used at development phase should still have been identified at the analysis and design stage, especially if a software factory type approach (e.g., separate development and assemble teams) is being used. Of course, developers will always swap code fragments and good ideas between themselves. That is to be encouraged, but those are not the scale of components that we are concerned with in this phase. It may be more fruitful to embody such ideas and techniques as design patterns rather than components. A repository of useful design patterns may in practice be much more practical than repositories of small components. Identify Available Commercial Components We argued in the last section that the step, identifying inhouse components, was redundant. So why should we not argue the same for identifying commercial
Page 226
components? In an ideal world this might be true. Our original analysis and design phases would have taken full account of available commercial components so there would be no need to search for them later. In reality, this is such an immature and developing market that it will be difficult to take full account of available components at the early stages. We also need to recognize that while it is reasonable to expect that systems analysts and architects would be fully conversant with components in their own organization, being aware of all available commercial components would be a much more onerous task. Thus it is probably impractical to do a comprehensive search at the analysis phase. The components that are most likely to be searched for and found are the GUItype components in which there is already a flourishing market. There are a number of companies that broker such components, and most provide WWW sites with classification and searching facilities. Such components will be selected to fit in with the chosen architectural style. When looking for slightly larger components, it is necessary to have some common theme that is likely to ensure that the components will be relevant to the system being built. At present there are few independent architectural standards or frameworks that might be used to ensure such a fit. Instead, we find components tend to group into certain user domains. Most often this is based on the tool set or modeling method that has been used to define the component. A good example is components that have been modeled and produced using the Sterling Software tool set. Searching for even larger business components would be futile at this stage in the process. There are two types of entities that might be found: largescale applications and business frameworks. Largescale applications are not attractive because of the degree of integration required. Choice of such applications would not fit an acquire and assemble model. If such applications are to be used, then these decisions should have been at the very early requirement and analysis phases. Making such a decision now would be very costly. The use of business frameworks (e.g., IBM San Francisco, SAP/R3, and so forth) is a much more attractive, albeit still immature, way of ensuring that large components can be successfully acquired and integrated. But once again, the decision to use such frameworks should be made at an earlier stage in the process. The more the market moves towards a highvalue frameworkbased approach, then the less relevant this step will be. Refine/Compromise Design and Specification to Match Available Components This is the pragmatic stage of the process where the designer and analyst have to persuade the customer to accept a revised specification in order to provide effective delivery time scales and costs. This is a good example of where the sim
Page 227
ple waterfall model we have presented needs in reality to be much more iterative. Of course, such negotiation, refinement, and compromise goes on all the time in software engineering. Here, the only difference is that it is being driven by the aim of reusing components already available and limiting the amount of new code to be written or largescale integration work required. Where an organization has immature component production or where the commercial market is still underdeveloped, there may be so little offered that there is little scope for compromise. As an organization builds its base of infrastructure components, then more can be accomplished by reusing existing components. Customers will be prepared to compromise specifications because of the benefits in cost and delivery times that they may gain. Over time, customers, requirement specialists, and analysts will become more informed about the components available. Then we would expect to see the initial requirements and business models starting to be written in terms of available components, thus simplifying later design stages. Sophisticated companies with highlevel componentbased business models would be able to build new products and services by combining existing components. This could all be done at the analysis stage, making later compromise almost unnecessary. Specify Unavailable Components Inevitably the system will not be able to be implemented entirely using existing components. Those that are not available will have to be specified for development in house or by commercial vendors. The design of the component will already have been completed and there may well be component models already developed (say in UML). The problem now is how to provide a specification that is unambiguous, defines all the nonfunctional aspects, has made explicit all assumptions, and could form the basis of a legally defined contract. Currently, this is unexplored territory and hardly anyone (perhaps nobody) knows how to do this effectively. It has been a major problem in software engineering for years, and while the use of components simplifies some aspects (hiding detailed technical implementation), it tends to emphasize the nonfunctional aspects and problems with implicit assumptions. We may look to military and aviation programs for solutions here as they have put a great deal of effort into formalizing software and requirements standards. However, that there are still many delayed and canceled projects in these areas indicates there is still a long way to go. Check for Potential Other Uses and Abstract Design This phases highlights what is perhaps one of the hardest aspects of software reuse and the one that in practice most often prevents success. In order to get successful component reuse it is necessary to get the right level of component
Page 228
abstraction so that it is genuinely useful to many different projects. But how do you go about this? One way is for a project to build the component it wants and then look for other potential users. Conventional wisdom has it that a component must be used at least three times before it can be considered to be reusable. However it is very unlikely that this first cut of the component will be as useful as it is to other users. Inevitably they will want a slightly different version of the component and may well pay to have it modified. A third project may also find the component useful and again will want it modified. If things have gone well then the component will have become more generic each time and now will represent a truly useful component. However, life is not that simple. Firstly there is no guarantee that the modifications will make the component more generic. They may add other bits of more specialist function and effectively bolt bits on the side. After a while things can get out of hand and instead of having a sensible core component we end up with a mess that really should be two or three components. When designing a system from scratch we might hope that our software development method would help in making the right decisions about how to bound the components. However, such methods provide no help in bounding a component that may be used across several systems. Secondly, do the modifications add incrementally to the component or do they keep changing its nature and redefining the interface? If we add functions incrementally, perhaps by adding additional interfaces (a model used by COM and soon to be adopted by CORBA), then at least the original users can use the new versions of the component. However, what happens if the component changes substantially such that those original users would have to modify their systems to take the new version of the component? Will they make the change? Why should they? What is in it for them? Unless they are prepared to take a strategic view they will want to keep using the original version. What tends to happen is that we do not get true reuse of the same component but a sort of adaptive reuse where we end up with three versions of the same core component. There may still be benefit in this provided new users always use the most recent version. Things get very messy if they choose to use an intermediate version. This introduces significant configuration management problems, an issue to which we will return later. The obvious answer may seem to be to ensure that the component is sufficiently generic in the first place so that it can be reused without modification. This seems an attractive proposition and many have tried this approach. It works reasonably well with smallscale GUI components where it is relatively easy to think up all the potential uses and it is not too expensive to code the extra function. However, with larger components where do you draw the
Page 229
boundary—when is enough, enough? It becomes very difficult to abstract an appropriate specification and decide what should be in and what should be out. It is particularly difficult if, initially, there is only one set of potential users and they have specific requirements. Even if it is possible to come up with a generic specification, who will pay for development? Funding is a major issue for componentbased engineering, and we will return to this issue later. One pragmatic way of addressing this problem is to develop a generic core component framework and provide the modifications and enhancements through the use of plugin components. Most effort would be focused on ensuring that the function of the core is abstracted over time to become more and more generic. All users would be encouraged to use the most uptodate version of the core, but would be allowed more flexibility in choosing what versions of plugins to work with. This produces a good compromise between limiting the effort required to undertake configuration management while giving users the freedom and flexibility to develop the overall solution they require. Component Design, Building, and Procurement Typical component design, building, and procurement phases might be as shown in Figure 8.4. Design Components We are now at the detailed design stage. Depending on the methods and tools being used, this may be a separate stage, done by a separate team, or it may just be an ongoing decomposition of earlier analysis and design phases. In small projects, most of what we have described may be carried out by one or two people using a single tool. In large projects, there will be different teams and different tool sets. Develop and Test Components This should be a fairly straightforward stage as the complex work of architectural analysis, deciding what is a component, where its boundary is, and what interfaces it has should all have been done. As with component design, this may be a distinct phase or just a continuation of higherlevel decomposition. We would expect that previous phases would provide component models and specifications, and this step would be a relatively straightforward coding exercise using good software engineering practice. As well as the developer testing the code, it is essential that the development organization formally test components and verify and validate them against the specification. Completely separate teams who are as free as possible
Page 230
Figure 8.4 A simple CBD software life cycle: component design, build, and procurement.
Page 231
from making explicit assumptions about the operation and use of the component should do this. Buy or Tender for Components This should be a relatively straightforward purchasing exercise to obtain components that meet the required specification and conform to specified quality standards. However, as we discussed before, the problems of providing complete and sufficient specifications is a major one which means a lot more intelligent purchasing is required than for other types of nonsoftware components. Formal quality standards for purchasing software components do not yet exist. It is likely that the military and aviation industries may lead the way here, but in the meantime, extensive vendor assessment will be required to ensure some level of quality. Buying software from invalidated WWW sites may be fine for the enthusiast, but it is a total nonstarter for large organizations. This is a shame in many respects because many shareware packages contain very innovative software and occasionally will become established products (e.g., PaintshopPro). An interesting development in the commercial components market is ''try before you buy." This is a familiar concept for software applications and many are available as timeor functionlimited evaluation copies. It is more difficult to provide this for small components, but we expect it to become a major aspect of purchasing large components. It will not be worth dealing with any company that does not allow you to try out its component first. We have very loosely used the term buy in regard to components, but in the software industry the concept of licensing is more familiar. Chávez, Tornabene, and Wiederhold [4] draw some distinctions between components and software applications that may well impose more arduous requirements on vendors. They argue the use of licensed components presupposes certain assumptions: that the components will not be used standalone, that to be worth licensing the components will be of such a size that they will form a significant part of the final application, that the components will have been fully tested, and that the components will be flexible and adaptable. They also point out the rather restrictive licenses currently attached to software applications will not be suitable for components. Much wider rights will be necessary to combine, integrate, and reuse the component. They suggest that licenses may have to be much more specific to the particular component and potential users should be very skeptical of standard licenses. These contractual issues also raise interesting questions about payments, royalties, and liability. These are all new areas that the commercial component industry will need to face up to quickly.
Page 232
Subscribe to Components An alternative model to acquiring components by buying the source code or executable is the concept of subscribing to a component when needed. This concept has gained in popularity following the rise of distributed component technologies and the WWW. The idea is that components would be called via a component broker as and when needed. The broker would identify the appropriate component, based on some type of servicelevel agreement, and route the call to that component. The user would then pay, based on some preagreed charging mechanism, based on number of times accessed, CPU time used, or perhaps by subscription. It is possible to imagine all sorts of charging regimes and discount mechanisms. At one level it has been suggested that PC users will no longer buy a full suite of office software (word processors, spreadsheets, presentation packages, and so forth) with lots of facilities they never use. Instead they would buy a core package with basic facilities and, on the occasions when they needed advanced functions, the software would call out over the Internet to access components that would deliver the required function. The user would be prompted for a credit card number or maybe they would have previously set up an account and indicated what functions they might want access to. At one time it was imagined that these subscription components might be Java aplets or Java Beans that would be downloaded and run on a virtual Java machine on the calling platform. The logical extension of this idea was the network PC that would only contain a Java virtual machine and would download all the required software, either at bootup or as required. However, the concept of downloading significant code to run locally has not been popular, particularly with corporate users, and the concept of the network PC has not developed the way all the hype would have led us to believe. Thus subscription components would have to operate more like corporate client/server systems and provide meaningful services executed on the server. The development of Enterprise Java Beans is seen by many as a key enabler for this market to develop. Of course the component or service would need to be of the right granularity and provide the right measure of valueadd. It is unlikely that a network servicebased suite of office software will be performant and cost effective. Specialized business and technical services are more likely candidates, particularly in areas where purchased software is expensive and may be infrequently used. Adapt Components If the components available inhouse or from commercial sources do not meet our exact needs then we have the possibility of taking them and adapting them.
Page 233
Adaptive reuse is the most common form of reuse in software engineering and is particularly prevalent in the informal reuse we see when engineers swap pieces of code with one another. Of course it is code that is being swapped, adapted, and reused rather than components. If we wish to undertake adaptive reuse on components, then they must have been supplied as White Box components so that the full source code is available. Alternatively one would need to go back to the supplier and ask them to adapt the code. At the current state of maturity of CBD, with limited availability of commercial components, adapting those few that do exist is on the face of it an attractive idea. However, it can lead to many problems. We saw some of the problems of creating multiple versions of components when we discussed creating generic components, and we will highlight this again later when we discuss configuration management. Adapting components should be avoided if at all possible. One alternative to modifying the component source code is to wrap the existing component with a layer of additional code that adds new functionality. Effectively we make a new component, but without changing the original. This is a much cleaner approach and it is relatively easy to cope with upgrades and modifications to the original component. It is analogous with subclassing in the OO domain and is most frequently used when data translation or type mapping is required. For instance a financial component may expect customer information in the form "last name" in one field with "forename" in another. If your company standard is for "last name, forename" then it is of course trivial to produce a customer component that makes the translation and embeds the financial component within it. This method is to be preferred to source code modification, but only certain sorts of modifications can be handled this way. Harvest Components Often when looking for available components we may become aware that the function wanted already exists in one of the organization's systems, but it is not available as an actual component. There are two ways of trying to extract a component from the system: harvesting and legacy wrapping. The term harvesting has an ecofriendly appeal to it and simply means extracting the code from an existing system to make a new component. We might do this to enable a specific piece of function to be used elsewhere or to reengineer the system into a componentbased structure. Often a number of variants of an existing system are required. Rather than have completely separate versions of the entire software, it can be more effective to identify the key areas of variation and break these into components. The first step is to establish what elements of the system are required, and whether they are logically distinct. The next step is to consider whether the
Page 234
components need to be physically distinct. If so, then the issue of separability needs to be addressed. Can the component be isolated from the system? What interface will it need? Such components need to be precisely specified both in terms of what they do, and how they do it. Once separated they could well exist on open repositories, being a general resource for the enterprise. But the separability and specification of such components is poorly understood, and requires further research. Harvesting can be a very pragmatic way of moving to a componentoriented approach as it is not necessary in the first instance to break the whole of the system into components. The core of the software that remains unchanged between the variants can be left untouched with just those areas containing the variations being turned into components. There can be considerable cost benefit in taking this approach in contrast to trying to justify the cost of a complete componentization of the system. Most CBD and more general software engineering methods are tailored towards creating componentbased systems from scratch. While there has been a lot of work in academia looking at legacy reengineering, there is very little help available to aid people trying to do the pragmatic valueadded harvesting we have described. British Telecommunications have experimented with several different techniques, and we talk a bit more about these later. Wrap Legacy System The alternative to harvesting a component from a system is to use the function it provides in situ. This is a very common approach to handling evolution of legacy systems and has been around for a lot longer than CBD. Essentially, simple access is provided to the piece of legacy function through a client/server networkbased interface. The user makes a call to this service without having to be aware that the function is embedded with a much larger system. We described this type of approach when discussing embedded system components in Chapter 3. The advantage of legacy wrapping is that at some point the function can be harvested from the legacy and made available as a standalone component. Users can swap to the new component just by changing their network call address. In fact this could even be done automatically without the user being aware. A popular way of wrapping is now to make large applications Web enabled. Developing, say, a simple formsbased WWW front end to an ordering system can be a very quick way of providing a component type access to a legacy system. At the simplest level, this can be compared to the rise in screenscraping that occurred a few years ago. It can be just as dangerous with significant issues of performance, resilience, network traffic, and usability patterns. More robust solutions are now making their way into all areas of business, and
Page 235
the challenge is for an enterprise to architect such solutions rather than have them grow in a piecemeal and uncontrolled way. Verify and Validate Components Having identified a collection of available components and arranged for others to be developed or bought, we are almost ready to assemble them into the required system. We are now faced with an interesting question: Do we take it on trust that these components will do what we expect of them or are we going to check them before we assemble them together? The electronics industry was faced with exactly the same dilemma in the late 1960s and early 1970s. At that time it was quite common for large electronic assembly companies to install complex automated test equipment. This was used not only for testing inhouse developed items, but also boughtin components and subassemblies. Actually these type of problems can be traced back even before this to more general industrial engineering. Again it was quite common for companies to have metrication departments that would check the dimensions and tolerances on all significant purchased mechanical items to make sure that they would actually fit together. At that time there were few quality standards for industry grade electronic components against which components could be bought. However, standards were put in place by the military (and later the aerospace and automotive industries). These standards were very rigorous and buying components to these standards attracted a significant premium. However, the approach used by manufacturers to meet these standards had the knockon effect of increasing the standard or all supplied components. When producing integrated circuits, it was often too expensive to have a production line just to produce military standard components and then another for commercial grade. Also the nature of IC production was such that the performance of components (particularly operating speed versus temperature performance) could be described by statistical functions. So manufacturers had one production line and put in place rigorous test procedures, selecting the best components for military use and the remainder for industrial use. In order to guarantee sufficient yield to supply the growing needs of the military, the quality standards of the whole line had to go up. This meant that the mean quality levels of industrial integrated circuits rose significantly. This, along with rapidly improving manufacturing techniques, meant that the quality of electronic assemblies and subsystems also rose. The end result was that the electronic manufacturing industry no longer needed goods inwards testing, except in very special cases or where there were known problems. From this example, and similar experience in other industries, we can see a general trend. When the industry is immature it is necessary to test everything
Page 236
at all stages to guarantee any chance of the pieces fitting together. As manufacturing processes improve, testing in the middle stages declines until we are left with a twostage approach. Vendors test their components on completion of manufacture and assemblers test their product after assembly. As always the comparison with hardware and software is not exact. The testing of integrated circuits we have described was largely performance testing rather than functional testing. Generally, if integrated circuits worked at all, they did what they were supposed to and it was performance that was the issue. Incorrect function or erratic behavior was usually a sign of a manufacturing defect rather than a design error. Often the component would fail completely well before the performance limits were exceeded. As we have observed before, software tends to exhibit more rich functionality than hardware, and there is more potential for design and coding errors. The difficulty of providing rigorous specifications for components adds to this problem. There is no real analogue to the manufacturing defects seen in hardware. Errors by compilers or automatic code generators would be the closest. This suggests that improvements in production processes reducing the need for component testing will not have the same impact on software as it did on hardware. Does this mean we should always test components before assembly? In the extreme the answer must be yes. Until rigorous standards for component specification and accepted quality standards for component production are in place, we really have no idea whether a component will work. The only useful measure we might have is the number of times the software has been reused. We might assume that a frequently reused component has a greater quality because the more it has been used, the more likely all its bugs will have been found, and, that people would not keep using it if the quality was poor. However, we must be very careful about making such assumptions. Lots of people may be attracted to a very useful component and suffer its bugs because of its usefulness. We only have to look at some of the market leading software products for desktop PCs to see that scale of use does not guarantee low defect rate. Deliver Components to Repository Once we have all our components we need to put them somewhere ready for assembly and reuse. Of course that does not mean we have to physically put them all in the same place, but it would be good to have a central data repository to hold the specifications, models, test results, pointers to location, and other useful information about the component. If we have been developing using a particular manufacturer's tool set then most likely this will provide some sort of repository environment.
Page 237
If tools from a number of vendors have been used then an independent repository is required. If information on components for a complete organization needs to be described then something more sophisticated may be needed. We will discuss repositories later. Systems Assembly and Delivery It is debatable whether some of the previous stages in the life cycle that we have described are in fact distinct stages. However, it is very likely that the systems assembly and delivery stage will be separate. After all, the whole point of a component approach is to be able to separate assembly from construction. The key phases in this step might be as shown in Figure 8.5. Extract Components from Repository This step should be trivial. We should not be concerned here about searching for the right component. That should have been taken care of in the production engineering phase. We should know exactly which components we want. Ideally the development tool or CASE environment should have recorded this information and have links via the repository to the exact location of the component. We may be taking source code from the repository for assembly with other source code, or we may be getting information about the location and type of distributed components so that the right client calls can be embedded. Assemble Components If we have achieved our vision for the ideal of CBD then this should also be a trivial step. The design and analysis stages will have determined that the design and interaction of components will deliver the system functionality required. The production engineering phase will ensure the right components have been procured. Assembly should now be just a mechanical, maybe even automatic, step. If we are talking about assembling relatively smallgrained, maybe GUItype components, then this step may well be simple. Increasingly, visual design environments are becoming available, particularly for VB and Java, that enable systems to be constructed simply by drag and drop of components. But with larger grained software components and particularly with system components, then assembly will not be that straightforward and will be more a case of component integration. Integrate Components What is the difference between assembling components and integrating components? In our view, component assembly is a fairly mechanical step which is just the final stage of a procedural decomposition. By the time we get to assembly,
Page 238
Figure 8.5 A simple CBD software life cycle: system assembly and delivery.
Page 239
we know that when the bits go together, we will end up with the right system. In an ideal world all systems would be built this way. The process of integrating components reminds us that life is not that simple. As soon as we go beyond smallscale developments we know that the analysis and design phases will not be able to produce a system definition that is sufficient and complete. We know that all the components that will be needed will not be available. We also know that we will not be able to exactly specify what we want and that when they are delivered they will not fit together without further work. Essentially integration admits the likelihood that further intellectual input will be required at very late stages in the process. It admits that glue code will have to be produced to get the bits to fit and various fiddling around will be necessary to make the system work. Of course working this way adds cost and risk, particular to major developments. However, what we have tried to show in this book is that there are two ways to try and manage and reduce this risk. One is by making increasing use of larger grained components and the second is by using techniques to go about integration in a much more rigorous and engineered way (e.g., by the use of the interface equation—see Appendix A). Integrating software components is hard enough, but when those components are themselves large and complex applications, it is a difficult and time consuming task. Production of glue code can often seem more complex than the applications that it actually glues together. It also raises all of the configuration and management issues for software components, but on a much larger scale. Senior management and chief executives often poorly understand the difficulties encountered in largescale integration. Having signed the business cases to buy these applications, they expect them to work out of the box and become frustrated at the lengthy implementation delays and increasing costs. This prompts the cry, "Why can't we buy everything from one source and let the supplier solve all these problems?" Companies such as SAP and Baan have capitalized on this, becoming market leaders in selling "enterprise resource planning" applications. The term ERP is somewhat of an anachronism now as ERP vendors provide systems that can support the whole of business operations. They also provide domainspecific frameworks for common business types (e.g., telecommunications and warehousing). However, even the ERP market is not immune from the march of components. Vendors need ways of making their product set more flexible so that product offerings can be more tailored to customers' needs. Also customers want to be able to mix and match between different vendors. They want to be able to buy the main business framework from one vendor, but plug in their favorite human resources package from another vendor. Moreover, few companies have the opportunity to start from a green field site where they can put in
Page 240
a new set of systems from scratch. Most companies have their legacy of support and specialist systems that have to be integrated into the ERP world. A whole new industry of enterprise application integration (EAI) companies has grown up to provide integration service at the ERP level. Just as with component technology, everyone suddenly wants to describe their products as providing EAI. Standards in this area have been few and far between. At the component technology level, the main vendors will sign up to support COM or CORBA and more recently Java Beans. Increasingly, however, these component wars are being seen as irrelevant as it becomes clear that it will be necessary to support them all. Even if a company does choose to opt for one flavor there is a strong likelihood that mergers, collaborations, and changing market patterns will rapidly make the decision out of date. Vendors of distributed middleware products that provide isolation from the lower level component technologies are taking the higher ground in claiming EAI support. Of course we are still talking of interface standards at the lower levels. At the higher levels of semantic integration, the open applications group (OAG) is working on standards for application integration as we described in Chapter 5. The most ubiquitous interface standard is current SAP's BAPI that provides an API for interfacing to its R3 product. Many application vendors support BAPI interfaces and most EAI companies will work at this level. SAP is a major player in the OAG, and it is very likely that they will set the de facto standard in one form or another. There are still many integration issues to solve. Most large applications expect to be the controlling application. Connecting a number of these leads to the question, who is in control? The usual ways of separating out control leave no way of tracking a job through the integrated maze and understanding the current status and dependencies of a particular job. The use of workflow systems has significant potential here, effectively providing an engine that does the control calling functions built into components. But again large application packages have job control built into them, effectively leading to multiple interacting workflow systems. This whole area of large component integration is one of the most interesting and challenging facing major organizations. We believe the most farreaching developments in the software industry over the next few years will occur in this area rather than at the component technology level. Configuation Management Configuration management is one of the most significant challenges for componentbased approaches to meet. We have included it as part of the system delivery stage, but as can be seen from Figure 8.1, it is an activity that underpins all activities. It is not as interesting as many of the more intellectual issues con
Page 241
cerned with architectures, design abstraction, and component specifications and as a result is frequently ignored. However, until the problem is cracked, CBD will not progress far. We saw how the search for component abstraction ended up creating multiple versions of components. This creates significant management problems in dealing with all these versions. What happens when the component has to be upgraded, perhaps as a result of a bug fix or change in some business rule or technology? Which version of the component do you change? The obvious answer is the most recent one, but what happens to those using the earlier versions? Are they forced to upgrade to latest and who pays for them to make the changes to their system to accommodate the new component? If by chance they do not really need to upgrade, because the bug or change does not affect them, they may choose to go on using the original version. This then means the distance between their version and the latest becomes even wider. The situation is analogous to upgrades of operating systems. Often systems administrators will take the decision not to upgrade because they see no value or more often because key software they are running is not yet supported on the new operating system. Such decisions can be tricky, but it is usually straightforward to do appropriate risk and commercial analysis. Vendors of mainstream operating systems will usually support perhaps up to three historic releases, but at some point the systems will have to be upgraded, otherwise, support contracts become invalid. The situation with components is much worse. It is much more likely that version of components will proliferate much more quickly than operating system releases. Moreover, while most systems only have one operating system, they may have tens, hundreds, and even thousands of components. Even worse there will almost certainly be complex dependencies between component releases. Managing such a system becomes a nightmare for vendors and users. Smallscale component projects often fail once the number of components becomes larger than can be managed in someone's head. Once we try and manage systems as components at an enterpriselevel, the problem is magnified further. We have dependencies between systems, software releases on those systems, between components within those systems, between other applications, and with data. British Telecommunications has several thousand operational support systems. The largest of these, the customer service system (CSS), has 28 operational systems. At any one time, there can be up to three different operational software releases in place across these systems. Just configuration managing the dependencies between systems at this level of granularity is a major undertaking without considering the component releases within them. There has been a rapid rise of interest in repository systems to hold specification and design information about components, but there are no repository
Page 242
products currently on the market which have made any significant attempt to address configuration management issues. Most will refer users to separate configuration management products, but these are software configuration management products. They are not specifically designed to handle component configuration and certainly not the enterprise scales the problems described. There is some move to combine software repositories with the more industrial strength repositories used for data warehousing. Whatever the direction, vendors must make significant progress in this area before widespread component use can take off. System VV&T In smallscale componentbased developments, this stage should again be relatively straightforward. Components will have been built to a known specification and standard and tested before assembly. If the analysis and design stages have been done correctly then we could expect this stage to be straightforward validation and verification. With the larger scale and integration type component developments we have previously described, then this phase will be much more arduous and intensive. There are no specific componentbased approaches that help here, but we would expect that the use of welltested components would reduce the testing burden. Return to Previous Phases As Necessary We have described this CBD software production model as a set of distinct phases following one after another in a waterfall type approach. We do not advocate that real development is actually like that, and we would expect to see significant iteration, particularly around the earlier stages. In fact the ideal model would probably be to have significant iteration around the analysis, specification, and design stages, in order to get the specification right. To have limited iteration around the design and production engineering stages in order to get a practical solution that can be built with available resources. And, finally, to have minimal iteration around build and assembly phase which ideally should be totally distinct. Deliver At last we can deliver the system or product and wait for the money to come rolling in. We have now completely finished the cycle—well almost, we still have to think about maintenance. Component Maintenance Maintenance has always been the ugly duckling of software engineering [15], the thing that people least want to do and do not want to pay for. Maintenance
Page 243
for component engineering is just as important, if not more so. The key phases in this step might be as shown in Figure 8.6. Measure and Evaluate Component Use This is a very useful aid to managing components. One of the main reasons for using components is to be able to reuse them so it seems to make sense to measure to what extent reuse is achieved. It is also important to understand how components have been reused, particularly to see if there are trends that suggest certain types of components are reused in certain ways, or maybe those particular domains use components in particular ways. This is vital market analysis for component vendors but just as important for organizations using enterprisewide components. It is also essential to build a complete list of component users so that they can be informed of upgrades and improvements. For companies using legacy wrapping to make available embedded components, it is important to make them aware of system changes. Larger systems' issues may force changes to wrapped legacy components and users may have to accommodate changes even when there is no specific benefit. Good data is the key to successful component management. Abstract New Components We discussed earlier the difficulties of getting the right abstraction for newly created components. As components become increasingly reused and more and more components are added to the inventory, then it will become easier to see what the correct abstraction should be and where the component boundaries are. Making use of this information to make sensible changes and produce a coherent range of components can only be done as part of maintenance. Individual development projects are unlikely to have the wideranging viewpoint and time to make such changes. However, this is still the problem of who pays for such work. Essentially it is infrastructure expenditure and will have to be paid for from overheads or levies. Forwardthinking vendors will be able to justify the expenditure on the basis of improving the product line, but for reuse within an organization it is a lot harder to make a business case. Maintain Existing Components There are many reasons why it might be necessary to maintain existing components. There may be bug fixes necessary, or changes in business rules or the commercial environment. Technologies may change and offer improved performance or make the old components obsolete. As we have seen, there are
Page 244
Figure 8.6 A simple CBD software life cycle: component maintenance.
Page 245
many configuration management problems connected with the issue of new component releases so it is important to ensure that changes are worthwhile. Software maintenance is difficult at the best of times, but the use of components may exacerbate the difficulties. The very nature of component approaches is that the development of the component is separated from the assembly process. The source code may not be readily available and when using commercial components, the vendors may be unwilling or unable to make the required changes. Jeffrey Voas [5] puts it very succinctly when he says, ''When you build a Windows NT application, you have effectively teamed with hundreds of Microsoft developers. However, when the application needs maintenance, you become a one man team." A major problem occurs when a vendor of commercial components ceases to support the component or goes out of business. Voas calls this frozen functionality. An advantage of the component approach is that, in principle, the frozen component can be replaced with one from another manufacturer. However, finding such a replacement can be difficult in immature and specialized markets. The alternative is for the user to try and build the component themselves or to try and acquire the source code and undertake the necessary maintenance. The component industry is becoming increasingly aware of these problems. It realizes that for components to be used by large corporate users for mission critical applications then service and quality guarantees must be provided. For instance, component vendor ComponentSource has set up a source code ESCROW service. Users of components from vendors who have agreed to offer this service can subscribe to the ESCROW service which will give them access to the source code if the vendor ceases to support it. ComponentSource has stated [6]: We have found that many Corporates require a higher level of service from organizations that they work with, specifically in their main business systems. When a Corporate evaluates third party software—functionality and quality of the product are only one aspect of the decision process.
ComponentSource also offers a subscription service for guaranteed upgrades and updates and is looking at component certification. ComponentSource is also fairly unique in offering a 30day refund policy for purchased components where there is no evaluation version available. Even if the component vendor continues to trade and support the component there is no guarantee that the vendor's vision for the evolution of the component will be the same as the user's. If the vendor changes strategy and brings out new versions of the component that do not suit the users, then what
Page 246
are they to do? They can choose to carry on using the old version component, but what happens when the vendor no longer supports it? This can be a tricky problem. The vendor might be unwilling to provide source code, and ESCROW agreements may not trigger because the company and components still exist. This is particularly a problem with largescale components and applications. Users should be very wary of using such components in ways that are significantly different from those suggested by the vendor as there is more likely to be a divergence of long term vision. Another related problem is when a component vendor is taken over by or merged with another company. Again, the new company may have a difference in vision about how the product should develop. They may also have a different approach to software engineering and may not deliver to the quality and commercial standards expected. In a very mature market these sorts of problems might be mitigated by the ease at which users might find alternative components. However, with the current state of the commercial software component market, users may find themselves locked in to unsatisfactory products. Manage the Inventory and Component Use It is important that all the tasks and issues that are described in this maintenance phase are managed in a coherent way. It is the whole inventory that needs to be managed, not just individual components. Changes should be managed in a proactive way. Users should be given advanced warning of changes and made aware of how long support for old components will be available. The maintenance team should look ahead for major changes in the business environment (millennium, European Monetary Union, changes in stock market rules, and so forth) and undertake risk analysis and contingency planning. Update Systems with New Component Versions Inevitably systems will have to be modified to accept new components. Either because it is advantageous for system performance or function, or, because of major environmental changes. It may also be forced on system users because of planned withdrawal of components as described above. Such changes have to be planned into the system's software release cycle and the necessary funding obtained. This may be quite difficult when there is no perceived benefit to the endusers. Wellplanned inventory management that gives good notice of forthcoming changes and wellmanaged withdrawal processes will present the image of a professional engineering discipline. Such a professional approach is much more likely to be respected and taken notice of by accountants and system plan
Page 247
ners than a slipshod outfit that is always panicking users with forced component changes. Withdraw Components Many companies have wellpracticed procedures for introducing new software and systems, but forget to consider the need for withdrawal. This is often where hopelessly outofdate and expensive to maintain pieces of equipment are still on the product list. This is not quite so common with commercial software vendors whose business is predicated on releasing new versions, but is much more common inside large organizations. It is an essential part of inventory management to identify the useful life of components and have a planned withdrawal process. Users should be made aware of when support for components will be withdrawn and provided with upgrade paths and alternatives. Generate Costs Models One of the major difficulties encountered when trying to introduce CBD is being able to put together a coherent business case that senior management will accept. This is largely due to the lack of serious historic cost figures for the introduction of components and any subsequent benefit. For many organizations the introduction of CBD is largely an act of faith, often championed by a forward thinking senior manager. Once an organization starts using a component approach in earnest it is vital to start collecting financial and usage data to enable a cost model to be put together. Only then will future developments be able to be truly justified on the basis of cost benefit. Rapid Application Development and CBD We have described a sequential waterfall type process but indicated that in practice it may be highly iterative. One of the most popular ways of undertaking iterative development is to use rapid applications development (RAD) techniques. How does RAD fit into the component development life cycle and does it add any new issues? It can be argued that RAD is a hindrance to CBD, but that CBD is an enabler for RAD. Does this make any sense? The concept of RAD techniques such as DSDM [3] is of very short release cycles (maybe two to four weeks) where developers work closely with customers on prioritized requirements. It works best on projects that require a high proportion of GUI development, simply because the customers can see some progress in a short timescale. It does not work so well with designs involving highly complicated algorithms because there is little to see. Often it is used for
Page 248
prototyping requirements and interface look and feel as a prelude for the main development phase. However, as with all prototyping methods there is the danger of the prototype ending up as the final system. Critics of RAD methods also argue that testing is not carried out in a rigorous manner. As we have seen from the description of the CBD life cycle, the emphasis is on business analysis, architectural, and component design. In the ideal situation, component development and assembly should be a mechanical process with the addition of very limited intellectual input. The analysis and design phases may be much longer than in conventional methods. This approach does not fit well with RAD cycles that aim to go through all phases in only a couple of weeks. The focus of each RAD cycle tends to be on a different area of prioritized functionality, and the GUI aspects of a CBD project may be quite small. There is the danger that the system will be developed piecemeal without sufficient attention to the overall structure and design. Of course it is perfectly possible to have early RAD cycles that concentrate just on analysis and design. However, there is still the pressure to push in and start cutting code. In a true CBD approach that is the last thing we want to do! Of course if lots of components are available this is a great boon to the RAD method as it can be a quick way to assemble them together into something useful. This is particularly easy when visual assembly tools are available. It can be argued that RAD only came about in the first place because of the availability of visual environments like Visual Basic and the availability of GUI components. Hence CBD is an enabler to RAD and maybe RAD would not exist if it were not for CBD. In small developments where the whole process may be done by one or two people using a common tool and where two to four week life cycles are the norm, then RAD could be a significant advantage. In larger projects, particularly involving systems integration, it has less value. This is why we believe that RAD could be a hindrance to CBD, but that CBD aids the RAD process. RAD prototyping certainly has a significant role to play in the analysis and design stages, but we think it is of less benefit at the design and development stages. Methods for CBD We have placed much emphasis on the need for wellexecuted business analysis, specification, and design in a component approach. In this section we will look at what current software engineering methods offer and at the emerging componentoriented methods.
Page 249
There are essentially three types of method: • Structured; • Objectoriented; • Componentoriented. Structured methods tend to use a waterfall type approach with one stage following another. Objectoriented methods by comparison are much more incremental and iterative. Class models are refined throughout the development process. Objects are much more like components and techniques such as subclassing, polymorphism, and overloading can be used to add additional functionality without major effect on the existing model. A componentbased approach, as we have already seen, is also a structured approach. Business analysis and architecture drive it. It attempts to separate the analysis, specification, and design phases from component development, and component development from assembly. It does not of course preclude an OO approach. Many (maybe most) CBD developments are OO. The possible tension between the two approaches arises because CBD aims to encapsulate function in large components that have loose couplings. OO is also about encapsulation, but within objects, and there tends to be highly complex coupling between objects. This makes it more difficult in OO to separate the component development and component assembly phases. In the late 1980s British Telecommunications developed its own sixphase generic software life cycle called TELSTAR [7]. In 1998 BT did a survey of software engineering methods and compared them to the six phases of TELSTAR. These phases do not exactly map to the five phases we described above, but are similar enough to give a useful indication. • Business analysis: builds business model, identifies business objectives and requirements; • Project definition: defines system requirements in outline terms, produces business case, obtains project authorization; • Requirements: captures and understands requirements, formally agrees with client; • Design: transforms requirements into a design; • Construction: implements the design in an executable, fully tested system; • Installation: prepares for and installs the system (runs in parallel with the requirements, design, and construction phases).
Page 250
Table 8.7 below shows some of the main software support for these phases. It can be seen that the OO method evaluated concentrated on the design and development phases, but omitted business analysis. CBD methods were very varied with some concentrating just on design and development, while others covered the whole life cycle. We will look at these in a bit more detail. Structured Methods Information Engineering The information engineering (IE) method evolved from work done at CACI (Consolidated Analysis Centres, Inc.) in the 1970s. It is an integrated, structured approach to developing information systems. Table 8.7 Method Support for CBD Methods
Business Analysis
Project Definition
Requirements
Design
Construction
Installation
Structured Methods Telstar
Yes
Yes
Yes
Yes
Yes
Yes
Information
Yes
Yes
Yes
Yes
Yes
Yes
Engineering
Yourdon
DSDM
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
OO Methods Booch
CBD Methods Catalysis
Yes
COOL:
Yes
Yes
Yes
Yes
Yes
Yes
Methods
Framework
Yes
Yes
Yes
Yes
Yes
Yes
Unified
Yes
Select Perspective
Yes?
Page 251
The method is based upon the premise that the data used by a business rarely changes; from this the method follows a topdown evaluation of the business requirements. This datacentered approach has been enhanced to include functional analysis. It advocates the separation of data and function that can be at odds with the data encapsulation ideas of object and component technology. On the other hand it recognizes that most large organizations model their data that way. It may be said that IE takes a broad approach by starting at the business need, viewing the project from the business problem rather than the development solution. The method encourages building systems to an agreed technical and data architecture; duplication should therefore be minimized. High user participation in the development is encouraged: the method promotes the idea of having client and user representation within the development team trained in the techniques and conversant with the method. Information engineering has no related project management techniques, but lends itself to any project management method through its highly structured stage framework. Yourdon The Yourdon method is a collection of techniques bound within a framework. Originally a data processing method, it was extended to cover realtime applications in the early to mid1980s. It would more accurately be called realtime structured analysis and design. Yourdon uses similar techniques to other development methods and supports three views of the systems: control, data, and process. It uses topdown functional decomposition and refinement of the problems. Yourdon is flexible: it allows adaptations and can be extended to suit many applications. The scope of the method and tool support provided covers requirements analysis to the beginning of construction. The method uses familiar notations like DFDs and ERDs. Realtime aspects are supported by the use of state transition diagram, control flow diagrams, and various timebased matrices. Yourdon can either be used in a topdown or bottomup fashion as part of the iterative refinement to gain first the logical view and then the physical view of the system. Yourdon does not cover requirements capture or testing, it needs to be augmented with other techniques from other methods that cover these activities. DSDM The dynamic systems development method (DSDM) [3] is a nonproprietary rapid application development method which has been developed through
Page 252
capturing the experiences of a large consortium of vendor and user organizations. It provides a framework for building and maintaining systems that meet tight time constraints through the use of incremental prototyping in a controlled project environment. The method not only addresses the developers view of RAD but those of all the other parties who are interested in effective system development, including users, project managers, and quality assurance personnel. DSDM provides a holistic approach to software development in a RAD project environment. DSDM provides a software development life cycle supported by all the necessary controls needed to ensure success. OO Methods Booch The Booch method recommends four distinct stages—analysis, design, evolution, and modification. Unlike the traditional waterfall life cycle, Booch recommends and allows a continuous iteration to take place throughout the whole life cycle. The above stages have evolved into a widely used practicable method that consists of the following: • Requirements analysis provides basic definition of a systems functions. • Domain analysis defines the logical structure of the system. • System design defines the physical structure of the system and maps the logical structure to it. This stage results in the production of working executable code. Requirements analysis employs use cases to describe system functions. During this stage, end users together with analysts produce and review the use cases that define the systems operation. All the functions of the system together with scope are documented here. The stage should deliver a definition of the responsibilities of the system and a function statement giving all the use cases of the system. Domain analysis defines an objectoriented model of the problem domain and includes all data and operations that the system requires. This stage produces class diagrams, class specifications, object scenario diagrams, and the data dictionary. During this stage the following steps are performed: definition of classes, definition of relationships and operations, definition of attributes and inheritance, and validation and iteration. System design is the stage of defining the implementation necessary to allow the object and classes to be coded and executed. It results in an implementation
Page 253
that performs the functions and stores the data, defined in the domain analysis. This stage produces the architectural description, executable release descriptions, class category diagrams, design class diagrams, and design object scenario diagrams. The steps required here are to determine the initial architecture, plan the executable releases, develop the executable releases, and refine the design. CBD Methods Catalysis Desmond D'Souza and Alan Cameron Wills developed the Catalysis method [8] in 1998. Catalysis is particularly well suited to building components with objects, having an emphasis on defining component interfaces. Its aim is to provide a set of techniques based on UML modeling to support objectoriented design, provide a method for componentbased development, and provide support for reuse, traceability, and the use of process patterns. It is based on three primary modeling constructs: • Type; • Collaboration; • Refinement. The aim of Catalysis is to provide an "unambiguous interface specification." Use case models define the specification with type models defining the vocabulary in the use case models. A form of instance diagram know as a snapshot is used to show states immediately before and after a use case. Use case can be strung together in an operational sequence and represented on a sequence chart. The level of detail at which these models operate can vary from the very abstract to the highly complex. Models can be refined into more detailed models in a typical structured decomposition. Catalysis also supports the use of frameworks and patterns by providing notations and methods of framework and pattern composition. On the face of it, Catalysis stills looks targeted at component development using object oriented techniques. However, a major attraction of Catalysis is that it has been taken up by Sterling Software as the foundation for their COOL:Spex product. COOL:Spex is not specifically an OO tool and thus its implementation of Catalysis can be expected to be a lot more generic. Cool:Methods Framework Sterling Software developed a methods framework in 1998 to contain its existing and new methods supported by their tools. The COOL Methods Framework contains five generic development life cycles, which define routes, tasks,
Page 254
techniques, and concepts for a project to follow. The life cycles are shown in Table 8.8. Componentbased development was primarily undertaken using the COOL:GEN product and using a set of standards and guidelines known as CBD96. Sterling Cool:Spex COOL:Spex is not actually a method, but a component modeling tool from Sterling Software that implements the Catalysis method and incorporates elements of the COOL method framework. While it forms part of the COOL product range, it can be used standalone for component specification. This separation highlights one of the key precepts of Sterling's approach to CBD, that the implementation of a component is separate from its specification. The use of COOL:Spex does not mandate an objectoriented approach. Based on the UML notation and built on the Microsoft Repository, component models can be exchanged with other CASE tools to give a choice of implementation paths or multiple implementations. As well as offering support for some of the standard diagram types found in UML, COOL:Spex provides additional diagrams from the Catalysis method aimed specifically at supporting component design: the collaboration diagrammer, interface diagrammer, and type diagrammer. • Collaboration diagrammer: models the static and dynamic aspects of a domain. Shows interactions between elements within the domain and Table 8.8 COOL:Methods Lifecycle
Enterprise systems planning
Identifies and prioritizes an enterprise business process, based upon the goals of the enterprise. It can be extended to identify and prioritize software and technical assets.
Business process improvement
Analyzes critical business processes and proposes new designs for this; this may involve designing new or modified applications.
Software asset improvement
Examines current software assets and envisions the future assets as architectures.
Technical infrastructure improvement
Examines current technical infrastructure and envisions the future technical architecture.
Software development
A generic process for delivering componentbased applications.
Page 255
allows the definition of behavior (joint actions) that has not yet been assigned to an interface. Actions can be joined together to form more complex behavior. The behavior shown in the models can be refined into a new model and an association matrix tool records the relationships between the models. • Interface diagrammer: defines the component interfaces as those services that will be provided by the interface. The interface type model uses UML to define the terms in which the interface is specified. A key part of the interface diagram is the definition of the preconditions and postconditions. The preconditions describe those conditions that the user of the component must establish before the component can perform its function. The postconditions define the results of the component providing its service. The preand postconditions can be described in English language or more formally using the UML Object Constraint Language (OCL). • Type diagrammer: defines the terms that are common across an interface and helps identify the components in the environment. COOL:Spex also introduces its own component architecture diagrammer and use case diagrammer [9]. • Component architecture diagrammer: a model of the dependencies between components. It shows how the various components are related and provides an overview of the structure of the software, at either the individual project level or enterprisewide. It identifies all the design constraints in the software structure. • Use case diagrammer: functional requirements expressed as use cases which identify key interactions between the users and the system. Associated with COOL:Spex is the Advisor. The Advisor is a combination of online and WWWbased help that provides CBD development scenarios, COOL:Spex techniques, general CBD guidelines (including CBD96), and a glossary of terms. Sterling aims to make the combination of COOL:Spex and Advisor not just another tool, but a complete source of information, method, and support for undertaking CBD. Sterling describes COOL:Spex as the leading component specification tool, but at the time of writing it is the only tool of its kind. No doubt there will soon be imitators, but by building a tool on such a solid foundation of method and experience. Sterling has set the standard to beat. Sterling has recently been awarded a U.S. government NIST Advanced Technology Program
Page 256
(ATP) research contract for research into CBD techniques [10]. Key areas of investigation include use of patterns and frameworks, component specification, and enhanced modeling for component architectures. No doubt the benefits of this research will find its way into their products in the future. The Unified Process (Objectory) Ivar Jacobson, Grady Booch, and James Rumbaugh of Rational Software Corporation have developed the objectory process. They later changed the name to the unified process to make part of a whole with the unified modeling language UML. They describe the unified process as A new product which unifies best practices from several software development disciplines—including business modeling, requirements management, componentbased development, data engineering, configuration and change management, and test—into a consistent framework covering the full life cycle.
A book by the three describing the process is due out shortly, but the process is also embodied in what Rational describes as ''an online, searchable, knowledge base in HTML format that provides universal, platformindependent access across a corporate Intranet." They claim the process brings together the best aspects of OMT, Booch, and OOSE and use case. Some of the key characteristics of the process are: • Componentbased; • Usecase driven; • Architecturecentered; • Iterative and incremental process; • Based on UML. It is aimed at being a process framework that can be configured to best support teams involved in projectbased component development. The parentage of the product (and it seems that it may be more like a product than a method) gives is very strong OO flavor. It remains to be seen how useful it will be as a more general CBD method. Select Perspective Paul Allan and Stuart Frost of Select Software Tools developed the SELECT Perspective [11]. This approach is a pragmatic amalgam of several techniques and approaches, which include:
Page 257
• Object modeling technique (OMT); • Use cases; • Business process modeling; • Dynamic systems development method (DSDM), (provides a framework for the iterative and incremental process). The approach uses use cases, UML, and a servicebased layered approach to architecture: • Business processes; • User services; • Business services; • Data services. SELECT also provides a set of componentbased software tools known as the SELECT Component Factory (SCF), which provides support for the SELECT Perspective method in addition to wrapping tools and code generation. The method is a much more pragmatic and application integration oriented approach and not as focused on OO techniques as perhaps Catalysis. A Generic Approach to CBD The Need for an Overall Approach We have seen that there are several emerging methods that offer support for a componentbased approach. None of them can be considered to be a generic method for CBD. Most of them are strongly OO oriented and generally they are intended to be implemented using a particular vendor's tool set. None of them have a long track record in delivering CBD projects, although the authors and their associated companies all have strong track records in their own fields. While we are confident that each of these will be effective in delivering componentbased projects, it does require that users commit to a particular approach at the outset. The problem is that it is difficult for people new to CBD to make that choice. We have also seen throughout this book that there is considerable variability in the definitions of key concepts. We believe it would be useful to have an agreed generic approach and definition of terms which people could compare with their own needs. By using a suitability filter similar to that we introduced at the start of the chapter, users could then identify the more specific
Page 258
techniques they need. They could then look to vendors for the product set that best matched their requirements. Despite the immense amount of interest in CBD and the number of vendors claiming CBD support, it is interesting to note that there is no generally agreed rigorous method for doing CBD in the same way that there is, for instance, for OO. Maybe that is because CBD is a different sort of thing than OO, using as it does best practice from many parts of software engineering. CBD is more of a change of emphasis to existing approaches rather than an entirely new approach. Nor are there any companies who offer an offtheshelf method to transforming existing applications to componentbased architectures. For many large companies, this is the application of CBD that they may wish to implement first. An Experiment in Component Approaches A project in British Telecommunications in the late 1990s was to take an existing legacy billing system and break it into components. The business drive for this legacy transitioning was to enable a number of variants of the system to be made available as commercial products. The products were to be sold in different countries and it seemed selfevident that supporting different tariff and tax regimes might be best handled by encapsulating these functions into some key components. The main development project decided that because of the immaturity of CBD, attempting such an approach in the time available was too great a risk. Instead conventional software developments produced separate versions of the system. However, it was decided to undertake a research project to do a paperbased investigation of approaches taken by other companies, to tackle the problem of systems migration, and also to evaluate any specific methodologies offered by vendors. Because of the research nature of project, there was a limit to what could be done, but in the end three approaches were investigated: • A fusion of the Sterling Software Transition Solution Method [12], the Catalysis Method [8], and practical experience of using CBD on BT pilot projects; • An OO approach using elements of UML and Catalysis; • A pattern language approach using the ADAPTOR pattern language [13]. The three approaches were all very different but had the potential to be compatible with each other. The patterns approach addresses the overall techni
Page 259
cal and organizational issues which must be addressed in the migration process. It does not address lower level detailed analysis but could provide the overall framework that organizes either of the other two approaches that focus on the detailed analysis. The UML and inhouse approaches both address the analysis and modeling of the system which identify the components of the system and how they interwork together. These two approaches come from different backgrounds, the former from object oriented and the later from the information engineering methods. However, they both end up with similar results, namely a set of components and an architecture. Although this was only a paper exercise, the results were interesting and encouraging. No one method was evidently superior to the others. Each method had its strengths in particular areas and each method produced similar results. At first sight it was disappointing that there was no one method that would be the obvious choice. However, on reflection, it seemed more useful to have a number of approaches that could be mixed and matched to suit particular requirements and circumstances. Case Study Results As a further step to trying to develop a more generic approach to CBD within BT, it was important to bring in practical experience from those already doing CBD within the company. A case study template was designed, based on the suitability filter, to understand why people were doing CBD, whether these reasons matched their original aims, and how they had gone about it. The results of the case studies echoed the findings of the legacy transition research. None of the teams had used any CBDspecific methods or tools but had employed traditional techniques, usually a business process design phase followed either by structural decomposition or object oriented analysis and design. Each project had used its own architecture. Reuse was stated as one of the prime drivers for the component approach and all projects stated they had been successful. Several teams raised the problem of widescale reuse and examples where this had been done successfully usually involved goodwill and personal involvement of component designers. Interestingly, none of the projects raised issues about component technologies (CORBA, DCOM, and so forth). It seems that the obsession of the technical press with these issues may be misplaced. Developers already recognize that the key issues are about architecture, defining greater value business components, and maintenance. The main issue consistently raised was of
Page 260
configuration management with comments such as "tricky," and "reuse makes it a nightmare." A Process Framework for CBD BT decided that rather than wait for the marketplace to mature, with fully developed methods, tools, and processes, what was needed was a pragmatic approach to give guidance to those who wanted to start CBD immediately. The first step was to develop a suitability filter similar to the one described earlier to enable people to better understand why they wanted to do CBD. The second step was a review of potential methods and an attempt to map their key characteristics onto a generic life cycle model. The table given earlier shows an overview of the mapping. A much more detailed analysis of the methods was done with the aim of extracting their key features and distilling these into a generic method or process. The aim was that the model would give options for methods and techniques that could be used for various stages of the life cycle. These would be selected based on the answers from questions in the suitability filter. Thus each project would build its own customized CBD process model, tailored to its specific needs, available tools, skills, and circumstances. However, at the time, few of the methods described above were actually available and all were in very early stages in their maturity. BT approached a number of vendors to identify what methods for CBD they recommended. The responses were unclear, and even when asked to highlight the key differences between their CBD approach and traditional methods there was little tangible response. Our conclusion was that they all offered methods for utilizing their own products, but none of them were the same, nor appeared to conform to any standard. There was no definitive or generalized CBD process defined, and it was not possible to build the model of appropriate techniques that we had hoped for. Instead BT put together its own generic model of a CBD process (Figure 8.7). It introduces more emphasis on flow and iteration than the simple model that we introduced at the start of the chapter. It also introduces the concept of twin track development whereby the component assembly can be done in parallel with the component development and provision. This recognizes one of the main advantages of the componentbased approach. Once a component has been specified, and its interfaces defined, the rest of the development can continue in the knowledge that the component when delivered will conform to this contract. While it was initially disappointing that we could not map specific methods and techniques into this model, in the longterm leaving it as a method
Page 261
Figure 8.7 British telecommunications CBD process framework.
only may be more useful. Mandating a specific technique or even tool often challenges developers to argue why it is not applicable to them. They are more likely to buy in to a generic, but sensible, model. Moreover, it will remain current while tools and techniques come and go. Butler Forum Universal Component Concepts Initiative The Butler CBD Forum have also recognized that although software tool vendors are keen to push their own methods and techniques for CBD, what users want is an independent process that will allow components from a variety of different vendors to be assembled. They have therefore established an initiative to develop a set of universal component concepts [14] that cover the essential steps in componentbased development that can be agreed upon by all. BT was a leading member of the Forum, and the BT Process Framework has been made available to members who are in the process of drilling down and defining each of the stages in more detail. Each stage description will have a set of input, or preconditions, a set of output, or postconditions, and a relationship to other process stages.
Page 262
Describing Components Architectural mismatch has been one of the major barriers to widespread software reuse. Sharing a component with a colleague on the same team or in the same part of the organization is not too difficult. There is a strong likelihood that such people will have a similar understanding of the domain in which they work. They will use similar tools, maybe use the same design methods, and have similar knowledge of the organization's technical infrastructure. Thus they will take the component's specification and probably make the same technical assumptions about what the specification means and how the component is intended to work. Even if they do not get it exactly right the first time they will probably quickly understand where they went wrong. Sharing such a component with somebody more dispersed in the organization or outside of it, or selling that component as a commercial venture presents a significant challenge. How are those implicit assumptions made explicit? How are the nonfunctions requirements specified? How can the vendor make sure the user makes appropriate use of the component? This is in the vendors interest just as much as the user's. The vendors do not want their reputation tarnished by users giving bad press to components that they did not know how to use properly. Table 8.9 shows some current approaches to describing components. UML is becoming the natural choice as a modeling notation for component development based on underlying OO distributed technology. The wide tool support and the exchange of models through the Microsoft Repository are obvious attractions. It is not yet clear how it will scale to describe much larger and complex components, how it will support transactionbased systems, or how well support for process flow will be implemented. However, the extensibility of the notation and the weight of the major vendors behind it almost guarantee its leading role. UML does not directly address how to address large grained semantic descriptions or how to express nonfunctional Issues. At the moment these tend to be expressed in plain English. These are easy to understand, but also easy to misinterpret and open to ambiguity. For the use of complex commercial components to proliferate, such descriptions will need to be tightened up. There are currently no standards or methods defined for doing this. While XML could be a notation for doing this, it is not necessarily the right one and certainly does not address the real issues of how to define those semantics and quality standards. The Open Applications Group (OAG) is working in this area with a view to be able to integrate largescale applications. A few years ago formal mathematical methods were popular in academia and in some military developments. Their attraction was that rigorous mathe
Page 263 Table 8.9 Describing Components Describing Components Interface definition languages, e.g., Good for small grained components. Usually targeted at IDL and Java RMI specific distributed component technology. Can be used for automatic generation. Do not scale for large components. Component definition languages, e.g., CDIF and CDL
Have not become established. OMG working in this area. Not clear how well they will scale.
Formal methods
Mathematically rigorous and formally vertifiable, but not popular due to mathematical skill required, complexity, and difficulty in understanding.
Components models
Tend to be vendor specific, but UML making more open.
Modeling languages
Strong support for UML as de facto standard. Extensible, but some vendor specific extensions emerging. Very 00.
Frameworks
Assumptions about using components are carried in the framework and accepted when the framework is chosen.
Semantic languages, e.g., XML
Claimed by many to be the answer to defining data models at the semantic level. Great danger that this is being hyped up and is not appropriate for all the uses it will be put to.
Domainspecific components
Assumptions are implicitly well understood in the domain. Components written by domain for the domain are more likely to work.
matical techniques could be used to verify that system descriptions were complete and correct. They never found favor in general industrial use because of the detailed mathematical knowledge required to use them. Moreover, such formal descriptions are very hard for lay people to read which defeats part of the object of producing high quality specifications. We will look at the problem of providing a universal language of systems in more detail in Chapter 9. At present the most pragmatic approach is to have components that are either so ubiquitous they are well understood by all (e.g., GUI widgets), or so specialized that only those few who have the detailed knowledge of the domain need to use them. The most difficult area is the very general components that might be used by many, but where there is no clear common understanding of what the component should or could do.
Page 264
Repositories, Indexing, and Retrieval If components cannot be managed, they will not be used and reused. Not very long ago, it was assumed the repository would be the answer to the reuse problem. It was claimed that people do not use components because they could not find them. Hence the main role for the repository would be to allow people to search for and identify useful components that could be assembled into the desired system. It was assumed that people would submit useful components to the repository as and when they became available. The main problem was seen as finding the right ones, and people started to suggest ideas for semiintelligent search algorithms. However, the problem holding back reuse is the implicit knowledge of developers that is inaccessible to other users. There is a need for domains, models, frameworks, plugins, and so forth to reduce search space and make repositories practical. As we have been at pains to stress in this book, components must be based around an architecture and have been designed to fit together. Assembling randomly generated components together will not work in any significant scale. Thus the original concept of components was misguided. Fortunately the software industry has woken up to this, and the role of repositories has changed. They are now seen as the core around which the design and analysis phases revolve. The release of the Microsoft Repository has firmly established the role of the repository as the vehicle for promoting design, object, and component reuse. Moreover, the use of UML as the underlying modeling notation allows the exchange of data between different modeling environments rather than locking the user into a proprietary toolset. This is of vital importance if components are going to find significant application at all levels of major organizations. Although UML is not in itself a component modeling notation, many of its more recent features (e.g., implementation diagrams) are targeted towards supporting components. However much vendors would like to lock major customers into a single toolset, large organizations will always want the freedom to use a range of different tools. Not only is this desirable, but in practice it is essential because, for instance, the types of modeling tools that are used for highlevel business modeling are nearly always different to those being used for codelevel object modeling. There are few companies that provide integrated toolsets that span this whole range of applications, and, even when they do their expertise is usually in one particular area. Thus to get the best of breed tools, it is necessary to select from a number of different vendors. Those vendors who actively support the exchange of models and data with other vendors will be much more attractive to major users than those who pretend to do everything and try to lock the users into a single tool set.
Page 265
Funding Component Approaches It almost seems as though we start nearly every section with words to the effect that "this issue is one of the most important challenges for a componentbased approach." However, the issue of funding definitely is "the" major issue. Most users and vendors will agree that a component approach costs more in the short term than conventional methods. As an act of faith everyone believes CBD should be cheaper in the long term, but few have been doing it long enough to really find out. There are one or two contract software development organizations that claim it costs no more. They argue that having components enables them to put together systems for customers quicker and cheaper with components than without. They are clearly much more advanced than most, but even for them it must have cost more in the early phases. They either used some sort of internal venture capital to get started or were prepared to borrow the money against future earnings. If you are producing commercial components, then you can amortize your development costs based on forecast sales. As with any product you are in danger of being undercut by someone selling a similar, but less extensive and hence cheaper product. But this is business and you have to do your calculations and balance components cost against your margins, selling proposition, brand identity, and marketing budget. If you are building components for inhouse use, then the problem is much harder. Many companies are used to the concept of capitalizing major purchases and writing off equipment and physical infrastructure expenditure. However, few are used to doing this for software and those that do will be doing it for large application packages and definitely not for small software components. Most software development shops are cost recovery centers and will not be allowed to make profits that they can use to fund infrastructure development. So who will pay for the development of your universal guaranteed generic component? If you are lucky, the internal customer for the first project that needs it will have lots of money and generous timescales and, knowingly or unknowingly, they will foot the bill. If you are very lucky you may even find a customer who does not mind you selling their component to lots of other users. Some customers will genuinely understand the strategic benefit of infrastructure development and be prepared to fund the development for the greater good. However, in our experience these types of customer are rare. With ever tightening budgets and tighter timescales, internal customers just want you to deliver what they want and will assume some (usually mythical) central corporate body is paying for infrastructure. This tends to proliferate development of
Page 266
stovepipe approaches. The same customer will complain bitterly at a later date when told the cost of interconnecting their product or service with another which was of no interest to them a few months earlier. The answers to these problems are not easy. Seemingly the most attractive way forward is for large organizations to emulate the commercial market and allow development outfits to run as a business—not just as a cost recovery business, but making profits (or charging a levy if that seems more acceptable) and using this to improve the efficiency of their business. In practice this is difficult to do because the customer/supplier relationship is always artificial. In the real world a business would only take on work that was commercially attractive, technically viable, and had proper requirements and sensible deliverable dates. If customers came with more fluid proposals then, if the work was accepted at all, they would be charged a premium. Moreover, if the customers did not pay they could be sued. In the corporate world these options are not feasible. The development shop is not generally allowed to turn away major projects. Very rarely does it get the opportunity to renegotiate the delivery dates and costs, and, when the budget does not materialize it cannot sue. In practice we do not know of any organization that has funded significant component development this way. The most successful examples of component development for use in the corporate environment usually involve a large act of corporate faith to just get on and do it, or at the very least corporate pump priming to get the component production business started. Often this approach will be stimulated by the need to make a major change to the business, perhaps as a result of a takeover or merger. Successful Use of Components As we have seen, it is one thing to decide that a components approach is what you want and another to make it work. There are many practical issues that need to be addressed, many of which we have described in this chapter. To support these concepts, ideas, and guidelines, we present a review of key issues that need to be tackled if good intent is to be turned into operational advantage. Business Issues Business Function and Product Lines It is vital that businesses take a component approach to their products and establish product lines of related products based on common components. The computer systems that support the business should then be organized as com
Page 267
ponents that appropriately relate to the product line and the business function required. The most successful companies implementing component approaches today are those that have established strong product lines with systems to the products. This is of vital importance in business where products are not distinct physical products, but where the operational support systems deliver the products (e.g., utilities, financial). Component Approach to Process Increasingly businesses are defining their requirements in terms of business process definitions (e.g., value chains). Reuse at the process level, and the automatic reuse of the supporting systems, must be the ultimate goal of any business. Business analysis is a major part of CBD, and we see process analysis and modeling becoming increasingly important. Cost and Time Establishing a funding model that will support the longterm production of a multilayered reusable infrastructure is currently one of the major challenges for establishing CBD in any major organization. If successfully implemented, the promise of CBD is that it will produce a reduction in overall development times. However, in the shorter term on initial developments, timescales may be longer and costs higher. There are some CBD practitioners who claim no additional costs for developing reusable components, but most quote anything from two to five times the costs. A typical rule of thumb is that a component must have been reused three times before it is sufficiently mature to be stable for general reuse. In these circumstances tribal attitudes can easily be established with departments being unwilling to fund a development in the hope that someone else will pay for it instead. Those that do lead the way often will not share with those who have not paid anything. Organization The organization of the component producing parts of the enterprise is a key element and closely related to component funding models. A preferred model is the separation of the component construction teams from the component assembly teams. Typically a component architect is usually attached to the construction team to ensure the reusability of components being created and to spot wider opportunities for reuse. Cultural Issues Reuse on a Larger Scale CBD is not just the reuse of bigger things (than code objects), it is more about a strategy and a method for developing and assembling (and reusing) components
Page 268
at a number of different levels or layers. The issue is how to buy, design, and build these components so not only can they be easily reused, but more importantly, they can easily be enhanced, altered, and rebuilt as business needs change. The issue is as much about architectures to support CBD as it is about the components themselves. Component Awareness A major issue is how to publicize the availability of components and how to describe them in an accessible, but common and complete format. Codelevel components are frequently not reused because of different viewpoints (people want apparently different components). The ''notinventedhere syndrome" often is excused because it is easier to build a new one than work out how to reuse an existing component. Repositories are often thought to be the answer, but as we have seen, while they are enablers, they do not solve the fundamental problem. Once again architectures and frameworks are the key. Avoiding Legacy Systems A danger for CBD is that, rather than reducing the legacy system problem (by decomposing and encapsulating), it produces a legacy problem of its own. It may create components that are so useful and ubiquitous that changing them has a major impact, or limitations in defining and implementing them means nobody has sufficient grasp of the consequences to risk making the change. Reuse works best with components that are stable and well known, inevitably meaning components may be seen as old technology, not as the cutting edge. Impractical on a EnterpriseWide Scale Is it possible to implement an enterprisewide CBD approach in a large company? The problem space may just be too large for people to know what is going on. Stovepipes are attractively quick and cheap, and the potential additional cost of duplication may be outweighed when the business need is met, contrasted with the time and effort of building an infrastructure which misses the deadline. However, stovepipes may never be capable of putting together the complex global services that are increasingly being sought. The question is how to build a largescale reusable infrastructure that can react to the rapid changes in market requirements and technology. Timing is a major issue; the view is often that we cannot wait for a generic component, something must be built quickly. Enterprisewide reuse is very difficult. It is probably better to focus on key areas. Reward and Recognition The rationale for the software industry is entirely geared to producing the latest release in the latest language which will always be slicker, faster, more excit
Page 269
ing than the last. But business users want stable, reliable, costeffective systems that do the job. We need to move toward a way of working that rewards quality and reuse, that penalizes building something new that could have been bought or reused. Do not recode your systems into the latest language because it's new. If it's not broke, don't fix it. If the architecture is not right, recoding in a new language will not fix it. We remember a university researcher who could not understand why we were not attracted to the software tool he had just written that would take all our legacy systems and automatically decompose them into Java. Technical Issues ObjectOriented Techniques CBD should be a multilayered approach where the lowest level may well be objects, but higher level components are systems, services, processes, and so forth. There is no doubt that objectoriented techniques are an important enabler for CBD, but CBD is not OO and object orientation is not the central idea of CBD. ThreeTier Architecture There is nothing about CBD that implies threetier or distributed client/server architecture. A layered model of CBD is a conceptual architecture for assembling systems and does not directly map onto the logical or physical tiers in a threetier architecture. The threetier model of separating data and presentation from the mid tier business logic is a sensible approach, but does not address any further how systems should be constructed. The rise of Enterprise Java Beans may have a significant effect on component approaches to threetier. Architectures, Standards, and Control Successful CBD requires architectures into which the components fit, standards for the interfaces between components, and a degree of control over the way new applications are developed and assembled. A culture needs to be established where standards and control are dynamic, rather than bureaucratic, and seen as helping people to build systems more effectively rather than hindering them. Quality and Reliability While reuse of mature and reliable components should offer quality improvements, it does not follow automatically that componentbased reuse necessarily improves quality or reliability. There are also technical problems with code level reuse (e.g., undocumented restrictions on interrupt handling and callbacks) and
Page 270
other implicit functional knowledge only known by the developer in the context of the particular application domain. It has been shown that decomposing components to a small level of granularity can actually start to increase the defect rate. Introducing a new component into a previously reliable system can compromise this reliability unless the new component is of an appropriate standard. Granularity What are the best type and size of components to use and be reused? Smaller components allow systems to be built that are modular and allow a system to be configured in more flexible ways, but they are harder to manage and do not add much value. Large components are easier to manage, add value, but are difficult to describe and less flexible. What sort of processes and services make good components? Should service components be arranged in a hierarchy? It remains to be seen whether largescale business components become available. Summary In this chapter we have described the basic steps that are necessary for componentbased approach to systems assembly and integration. Componentbased development is still immature and at this stage more applicable to componentbased software than systems. Nevertheless we can see that, in principle, systems could be assembled from components. The question is to what extent the approach will scale up and what additional challenges are posed. We have shown that much of CBD is about a new perspective on current best practice, but we have also described some of the new issues and challenges that such an approach brings. We have seen that methods and tools are starting to become available, but they are very much aligned to key vendors and the approaches implicit in their products. What is needed is a more universal and generic understanding of how to go about CBD, and we have described how British Telecommunications and the Butler Forum have started towards such a view. Many popular books on CBD and related techniques describe either a very academic approach to CBD or a very technologydriven approach. They tend to give the impression that everyone is already doing CBD, and there is not much more to it than choosing whether to go DCOM, CORBA, or EJB and then doing a conventional objectoriented design. We hope we have given a much more pragmatic view and identified what some of the real issues are. We have not gone into great detail about tools and techniques. Most of them are not CBD, but variations on OO. Readers can find that detail elsewhere and our bibliography gives some suggestions for further reading.
Page 271
Final Checklist We hope potential users are not put off CBD by the many unresolved issues. CBD is no different from the rest of software engineering in that respect, and CBD is no silver bullet. Table 8.10 and 8.11 give some final checks that might be helpful. Table 8.10 Axioms for CBD CBD Axioms • Those best at CBD already use good, solid software engineering practice. CBD is a slight change in technical perspective, not a new approach. • Beware; even thought the technical change is slight, there can be significant changes needed to culture, management, and funding. • If you aren't doing good software engineering, do not try CBD, it will not make things any better.
Table 8.10 Checklist for Doing CBD
To do CBD
Make sure you are doing good quality software engineering first—processes such as those explained in Chapter 4 should be in place.
Make sure you understand why you want to do CBD.
Choose the right approaches. Use the questions at the beginning of this chapter and from Chapter 7 as a start.
Use an architecture; there are plenty to choose from in Chapter 5.
Spend the most time on analysis and design. Do not be too keen to cut code; CBD is about avoiding that.
Do not get hung up about what technology or language to use; the whole point of CBD is that is should not matter.
Use commercial components with care; look for guarantees of quality and supply.
If you have no idea where to start, then look to the key vendors who are providing CBD consultancy and tools.
Remember, nobody sells methods. Consultants want you to buy their time to tell you how to do it; vendors sell tools and that's what they are trying to lock you into.
Keep an eye on bodies such as OMG, OAG, Butler Forum, and so forth for good practice, generic approaches, and standards.
Do not get carried away by the hype; learn from the mistakes of others (and preferably not your own)!
Page 272
If you want still want to do CBD, then make sure you follow the guidelines in Table 8.11. Good luck! References [1] Butler Group CBD Forum. (http://www.butlergroup.com) [2] Norris, M., P. Muschamp, and S. Sim, "The BT Intranet—Information by Design," IEEE Computer, March 1999. [3] Dynamic Systems Development Method (DSDM) Consortium. (www.dsd.org) [4] Chávez, A., C. Tornabene, and G. Wiederhold, "Software Component Licensing: A Primer." IEEE Software, July/August 1998. [5] Voas, J. "Maintaining ComponentBased Systems." IEEE Software, July/August 1998. [6] ComponentSource. (www.componentsource.com) [7] TELSTAR, Principles of Systems Engineering. Version O.A, BT Systems Engineering, June 1992. [8] Cameron, W. A. Catalysis: Technical Briefing. TriReme International Ltd. [9] Sterling Software, Methods and Tools for ComponentBased Development, October 1998. [10] Sterling Software, CBD in Practice: A Review of Sterling Software's Past, Present and Future Directions in CBD, June 1998. [11] Allen, P., S. Frost, and E. Yourdon, ComponentBased Development for Enterprise Systems—Applying the SELECT PERSPECTIVE, Cambridge University Press, 1998. [12] Texas Instruments Software (now Sterling Software), The Transition Solutions Process Version 2.1, June 1996. [13] O'Callaghan, A., ADAPTOR: A Pattern Language for the Reengineering of Systems to Object Technology, 1998. [14] The Universal Component Concepts (UCC) initiative. Butler CBD Forum. (http://www.butlerforums.com/cbdindex.htm) [15] Norris, M., Survival in the Software Jungle, Norwood, MA: Artech House, 1995.
Page 273
9— Interfaces and Integration—Standards and Formality The world is richer than it is possible to express in any single language. Ilya Prigogine and all, hierarchitectitiptitploftical, with a burning bush abob off its baubletop James Joyce, Finnegans Wake
Now the whole earth had one language and few words. And as men migrated from the east, they found a plain in the land of Shinar and settled there. And they said to one another, 'Come, let us make bricks, and burn them thoroughly.' And they had brick for stone, and bitumen for mortar. Then they said, 'Come, let us build ourselves a city, and a tower with its top in the heavens, and let us make a name for ourselves, lest we be scattered abroad upon the face of the whole earth.' And the Lord came down to see the city and the tower, which the sons of men had built. And the Lord said, 'Behold, they are one people, and they have all one language; and this is only the beginning of what they will do; and nothing that they propose to do will now be impossible for them. Come, let us go down, and there confuse their language, that they may not understand one another's speech.' So the Lord scattered them abroad from there over the face of the earth, and they left off building the city. Therefore its name was called Babel, because there the Lord confused the language of all the earth; and from there the Lord scattered them abroad over the face of the earth. Genesis 11
Page 274
So even God does not like standards! But then, neither do systems engineers, or so it seems. But while God's motives were to effectively divide and conquer, systems engineers generally agree that we need to find that common language again if we are to be in a position to build today's systems equivalent of the Tower of Babel (Figure 9.1). Nobody doubts that we have to be able to communicate accurately and that the information that engineers exchange should be unambiguous and open to analysis. The problem is how best to achieve this. So, in this chapter, we look at the state of play and consider the prospects for finding the Babel fish (the ultimate translation device from Douglas Adams' Hitchhiker's Guide to the Universe). Despite numerous attempts to establish a universal Esperanto for the systems world, we are in fact no nearer than we were twenty years ago. Indeed, it may be argued that with more and more specification, design, and coding languages being developed (such exercises have proved to be ideal Ph.D. dissertation material), the picture is becoming even more confusing. As soon as a likely standard emerges, Java, for example, various suppliers start to develop their own variants and we're back to square one. This may suit God, and it may suit software developers, but it is a serious hindrance when it comes to integrating largescale systems and defining the interfaces. Indeed, system integrators are often left with the task of building a functioning system from the equivalents of K'nex, Lego, and Meccano! Or put another way, they are often faced with the problem of fitting a square peg into a round hole. The point is that these are not going to fit together very well, and that a large amount of additional engineering would be needed to ensure that a fully functioning and stable structure is produced. And such problems are not confined to software engineering. The International Space Station (ISS) has highlighted similar problems in the aerospace industry. It is well documented that the United States and Russia (and the former USSR) adopted quite different approaches to space flight. For example, when the Space Shuttle docked with Mir (the Russian space station) an elaborate docking interface had to be developed at significant cost. That the legacy of past programs was responsible for this is not surprising and perfectly understandable. Clearly, for the ISS such interface problems cannot be allowed to occur. A generic design philosophy needed to be established. But in fact the same reasoning applies to businesses and explains equally well why we have such a plethora of systems engineering techniques and methods. The avid reader might recall from Chapter 1 how the concept of a computer network arose from initially standalone, bespoke systems. Each of these systems was designed to fulfill a particular function within a particular organization. The systems developed to control stock levels in a warehouse were very different from the systems for human resources, or the process control software for a chemical processing plant.
Page 275
Figure 9.1 The Tower of Babel.
Page 276
While each of these systems was standalone and isolated from the rest of the world, systems engineers did not really need to worry about the fact that each was developed using a distinct language and design methodology, but the globalization of business and the emergence of the Internet have changed all that. From the engineering point of view, we live in a very different world and the tools and techniques required to construct these new systems should either be global standards (very hard to achieve) or have the ability to interface seamlessly (also very hard to achieve). Regarding the later point, we have shown in this book that techniques do exist to facilitate this interworking between different language, but unless you were already familiar with this work the authors are sure that you will have found it hard to follow. It is in fact a difficult and complex mathematical problem. The essential message here is that there is no easy solution. The Nature of Languages Is such a standard even possible or required? As with the Prigogine quote, some argue that the range of systems required in today's business, defense, and government domains is simply too diverse to be adequately covered by a single language. Realtime systems, batch processing, transaction processing, client server, peertopeer, mainframe, interactive, and so forth all have certain idiosyncrasies which would be difficult to capture in one language. Such a language would have to be very rich in terms of syntax and semantics. Now in fact we do have such a language—mathematics. As far as we can tell mathematics is the only true universal language. Scientists confidently expect that if we ever do communicate with an alien intelligence, it will initially be via mathematics. Hence, it is not surprising that many of the languages developed to solve the problems of system specification and design have emerged from mathematical logic, discrete mathematics, and automata theory. The problem with this approach is that for a large proportion of the population, mathematics is considered difficult. And these notations do look intimidating. Look at this relatively trivial VDM [1] specification, for example. VDM Example—Creating and Removing Bank Accounts RemoveAccount(account_number : Accno)
ext
wr bankc : Bank
pre
account_number / dom bankc let act = bank(account_number) in
Page 277
act.balance = 0 post
bankc = {account_number}
NewAccount(name : Name, overdraft_limit : Amount) account_number : Accno ext
wr bankc : Bank
pre
true
post
account_number dom bankc = bankc † {account_number (nm, 0 overdraft_limit, [] )}
mkAccount
Or this simple CSP [2] specification: CSP Example—A Simple Specification of the Process Diagram in Figure 9.2
where
Figure 9.2 A simple process diagram.
Page 278
You might think that you could decipher this diagram reasonably quickly, particularly if you are a trained mathematician, but if you have no mathematical background, it will probably appear almost meaningless. Now imagine being faced with twenty, thirty, even one hundred pages of such terminology. Remember, designers must understand every detail of the system before they can begin to put together a structure for the system. The problem is that a specification tells a software systems designer what a system should do. It has to be, therefore, intelligible and understandable, but mathematics is a very precise language with little redundancy. Redundancy is what makes a natural language so easy to use (you could leave out quite a few words and letters in a typical sentence of English and still retain the meaning). Redundancy, however, induces ambiguity and that in turn results in imprecise meaning and as such the wrong software system. Thus we have a dichotomy: understandability or precision. As yet no one has really managed to develop a notation that sufficiently accommodates both. No one had a greater understanding of languages than James Joyce, and in Finnegans Wake [3] he concocts a language that is extremely rich and which contains several layers of semantics. Each and every sentence has multiple meanings, but it requires a very special wordsmith indeed to break Joyce's code. Some extraordinary people have produced a universal code book (or Rosetta stone) for Finnegans Wake—monumental works in themselves! Likewise, in his book Contact [4], Carl Sagan describes the retrieval of a message from another civilization which contains a detailed specification of how to build a cosmological transport device and visit the alien civilization. Sagan clearly gave much thought to the type of specification language necessary to convey all the information required. He describes a language which operates at several different levels simultaneously, akin to the language of Finnegans Wake. Maybe that is what we, as systems engineers, should be aiming for, but there is no sign of such a language materializing, and even if it did, wouldn't we be back to square one anyway—for who would understand it? Yet another small group of specialists? That was exactly the same problem that forced industry (in part) to pass on formal methods in the 1980s. As you can see, the problem of specifying a system on paper so that all the intricacies of the system and the customer requirements are captured, but which is understandable by most system engineers, is a major headache. Inherent Complexity An interesting notion developed by Ashby [5] and later Beer [6] concerns ''the law of requisite variety." In essence, this states that in order for two systems to
Page 279
interact they must be of at least the same complexity. This is a simplification and the concept is in fact quite deep, but what it does say is that complexity is inherent and that a formal description system must be at least as complex. The message from this is that we will never get rid of complexity—difficult systems will always be difficult systems until we find new ways of doing business. However, the more we know, the greater the diversity of applications, and the more complex systems become. (The telecommunications network is an excellent example of this.) Everything in the software industry seems to be getting more complicated. For a simple life or system a simple language will do, but as we are all too aware, life is not simple anymore. Assuming that complexity is intrinsic, how do we manage it? Certainly topdown design and breaking the system up into manageable bits will help. System engineers have known this for many years. But eventually, you have to deal with the nuts and bolts and tackle the details of the program or interface. In the next section we will review, at a cursory level, the types of languages which have been developed over the past two decades. Formality and Protocols There is a plethora of notations, methodologies, and languages to specify and design software systems. In this section we will try to categorize them into a few broad classes. This is not intended to be a critique of the notations. Our intention is merely to provide a flavor of what is available both in industry and academia. In terms of problem domain, we will focus on a particular business where virtually all of the issues discussed elsewhere in this book can be found in one convenient package—telecommunications networks. The concern is not the hardware, but the management systems and protocols without which the network could not operate. When it comes to managing the growing complexity of communication networks, only one thing is certain—there is no room for mistakes. Downtime costs money, either directly or indirectly. Yet, recent surveys of IT managers show that 91 percent of those responding experience total or partial LAN failure at least once a year. That might be just about acceptable yesterday, but for today and tomorrow, where the network will most likely be a platform for ebusiness, such failures could be very expensive indeed. In addition, the telecommunications network is constantly evolving. Almost daily, there is something new. Keeping everyone in an organization connected brings constant challenges as well as opportunities. A network may have multiple sites with hundreds of workstations. It may have different cabling sys
Page 280
tems at different facilities. An organization may be building its own intranet, or looking for ways to connect with clients and colleagues through the Internet. The network is absolutely vital to such businesses, distributing both core functions, process applications, and linking backend systems. So communications systems are some of the most complex that are currently built, if not the most complex. They contain large amounts of hardware and software, are highly concurrent, and involve communication between components governed by formal rules known as protocols. While such systems are not as critical in terms of failure as, say, nuclear power station safetycritical systems, they still exhibit a very high economic penalty for systems failure. The combination of complexity and high cost of failure makes telecommunication systems good candidates for the application of formal methods. The early hardwaredominated telecommunication systems were complex enough; however, the introduction of stored program control systems, although greatly improving flexibility, has resulted in much greater increase in complexity. A further push towards complexity has been the move to integrated networks and services such as ISDN, IN, and LANs. While the growth of the Internet has benefited greatly from a de facto standard—TCP/IP—this growth has resulted in an explosion of diverse applications and functions. A major issue that permeates communication system development concerns protocols and the management of interfaces, not only within a system but also between a system and another system. Protocols are typically used to enable such communication. A protocol is a set of rules which allows communication between different system components that often have widely differing properties, for example, I/O port characteristics. Although the trend to partition a communication system into aspects which describe its logical, physical, and conceptual behavior—for example, via the OSI model—has undoubtedly led to a better understanding of systems, we are still faced with major problems. The service concept is one of the most important architectural concepts of OSI. It has been introduced in order for systems to be considered from the user's point of view with all low level details removed. What remains is the abstract behavior of the system as perceived by an external observer (the user). But, as indicated earlier, you cannot ignore the details. At some point they have to be addressed, and that is the hard part. Protocols can be extremely complex and, while to a certain degree, informal design techniques have been successful in this area, it has also become apparent that they have also yielded a disturbing number of protocols with undesirable behavior. In defining a protocol, it is important to describe the context in which it is to operate and the services it is to support. The context is provided by the overall architectural (such as OSI), and by the particular layer in which
Page 281
the protocol will reside. The architecture will provide the protocol engineer with a set of constraints in terms of other interfacing protocols and routing strategy. The protocol's behavior—in terms of the services it is to provide—is determined by the service specification. This will describe what data is to be transported and security requirements, and provide a general functional specification. Hence, while the architecture can be seen as determining the external aspects of a protocol (e.g., interfaces), it is the service specification which determines its internal structure. As a result, a protocol specification should include: 1. A general description of the purpose of a layer and the services it provides; 2. An exact specification of the service to be provided; 3. An exact specification of the service provided by the layer below, which is required for the correct and efficient operation of the protocol; 4. The internal structure of the layer in terms of entities and relations; 5. A description of the protocol(s) used between the entities, including an overall informal description of the operation of the entities, a list of the types and formats of messages exchanged between the entities and rules governing the reaction of each entity to user commands, messages from other entities, and other internal events. A formal method of software development relies on the use of discrete mathematics for the functional specification and design of a system. A formal method also uses proof for validation, for example, in order to check whether a design is a correct reflection of a functional specification. The use of formal or semiformal techniques for communications applications is not new. System X (System X is a major telephone system originally developed by the British company GPT Telecommunications), for example, was completely specified using semiformal message sequence charts and data flow diagrams [7] in 1979. The early pedigree of this project makes it, in terms of formality, or at least semiformality, an innovative project, an example where the nature of telecommunications systems has forced developers to be more innovative than the vast majority of IT suppliers. There are two general points concerning the use of formal methods: 1. It must be said that the impact of formal methods on protocol standards is negligible with natural language and diagrams being used in the main.
Page 282
2. There is an understandable trend of using either existing notations or hybrids of existing notations for protocol specification. It is questionable whether this trend addresses the problems of protocol specification. The analysis of protocols centers on the behavior of the communicating protocol entities and the underlying medium, that is, the set of possible sequences of events that can occur when the entities execute. Protocol properties are often specified in terms of assertions associated with a transition system. Such properties are classified as either safety or liveness properties. A protocol is safe if no undesirable event ever occurs, for example, deadlock. It is live if all desired events eventually happen, for example, completion, recurrence, or progress. Safety properties can be described as invariant assertions associated with transition system models, or as relations (simulations or bisimulations) between a proposed and a required behavior. Liveness properties generally require temporal reasoning; hence, the use of temporal logic becomes necessary. Temporal logic formulas which capture the liveness conditions are added to a specification and the state transition system checked to see if it is a model for these formulas. So each component of a (possibly) heterogeneous system has been formally specified. But what next? How can we use all this formality? Issues such as deadlock, safety, and liveness have already been mentioned. You can use mathematics to analyze the system to determine the state of those properties (although it generally is not easy). But what about traditional old testing? How can the mathematicians help make that process easier—if at all? By testing we mean that subset of activities which give rise to the execution of an interface with test data in order to check that a protocol meets the relevant section of the service specification. The advantage of adopting a formal approach is that it presents the possibility, at least in principle, of automatically generating test suites from the relevant specifications. This assumption is based on the idea that it is possible to define mathematically provable transformations on a specification to produce a test suite. There are two major aspects to protocol testing: 1. Static confomance testing: defining the allowed minimum capabilities of an implementation to facilitate internetworking. The term static refers to the fact that it is the structure (defined by the associated program control flow, for instance), which is the key property. Static tests do not require the protocol to be in operation. It merely requires the source code or some other (static) representation. 2. Dynamic conformance testing: defining the set of allowable behaviors of an implementation in the instances of communication to check dynamic behavior. Dynamic testing requires the protocol to be oper
Page 283
ational—these tests can only be carried out when the protocol is in use (usually by simulating usage in a controlled and deterministic manner). Because of the increasing complexity of modern protocols, generating tests by manual or ad hoc means is a task fraught with difficulty. So the ability to automate this part of the process would be seen as a significant step forward. As yet, this breakthrough has yet to be achieved. But where significant advances have been achieved is in the area of protocol conversion [8], where a wide variety of techniques has been developed. With such a wide range of techniques available, none of which is totally general, it would appear that the system engineer's best hope is to develop a protocol conversion toolkit. There are a number of important points that have emerged from research into protocol conversion: 1. There are few tools which enable the software or hardware engineer to provide a specification of an interface given some inputoutput specifications. Those that do exist are inefficient (being nondeterministic polynomial NP or worse) and of limited applicability. 2. There is little theory concerning the underlying mechanisms which are used for the automatic derivation of interface specifications. This could be a reason why the algorithms that have been developed are NP. 3. There is an open question which concerns the utility of current communications specification languages with respect to protocol conversion. For example, we feel that SDL, for instance, is too cumbersome a medium to be used for developing theories, techniques, and tools which would enable the automatic generation of interface specifications. The issue is: Can appropriate techniques be developed to include data in the interface specification at a sufficiently high level of abstraction that the process of protocol derivation is not inefficient? 4. There is a deep research question which is to do with the fact that we do not yet fully understand the inheritance properties of the interface specification. For example, can you assert that an interface specification does not contain undesirable properties such as deadlock when automatically generated from two specifications which are deadlockfree? 5. Tools that have been developed are not based on practical notations such as LOTOS. Most tools that have been developed have been based on more abstract notations such as CCS.
Page 284
6. Many of the tools that have been developed for protocol conversion have had as their aim the fully automatic derivation of interface specifications. As a consequence of this, these tools have only really been found applicable to very small protocols of three orders of magnitude in size less than practical protocols. There is scope for examining the impact of heuristic, semiautomatic techniques which may scale up better. 7. The issue of scalability is critical. Of the techniques that have been discussed, very few, if any, can claim to have been applied to industrial scale examples. Industrialstrength protocol conversion exercises which have been published show no evidence of nonstandard techniques being used. Formal Methods What is also clear is that this area of application is bedeviled with the same problems that occur generally with formal methods: gaps in theory, many of which have been outlined in this chapter, and a comparative lack of concrete demonstrations that scaling up to reallife problems is possible. We regard the latter as the most serious. Few techniques have addressed the problems of applying formal notations to largescale applications. In the author's view the comparative lack of progress of formal methods in communications applications are those often cited for other areas: • Lack of trained staff; • Problem of customer liaison; • High training costs. There are a number of promising developments which affect formal methods which will increase their application in a number of areas, not just telecommunications applications. First, there is the integrated methods approach being pioneered by various establishments. This approach combines standard structured methods such as SSADM and Yourdon Structured Development with formal specification languages such as VDM and Z. With this approach one uses formal languages only when absolutely necessary; otherwise the simpler structure methods are used. This is eminently sensible: many communications systems which could benefit from being developed formally contain large chunks of noncritical code which could be developed using less exact techniques.
Page 285
Second, initiatives such as the British MoD's Defence Standard 0055, which makes mandatory the use of formal methods for the critical components of a system may yet have an impact. Although intended for the defense industry, success in this area may provide incentives for industry in general. Third, in the United Kingdom with mathematical methods of software development being on every computing curriculum, formal methods will become a more and more viable technology as graduates with the requisite skills trickle into industrial projects. La Trobe University has, for instance, developed an undergraduate protocol development training course based on formal methods. It remains to be seen whether initiatives such as this will provide the necessary levers in terms of industrial applications. Fourth, in the United States where university computing departments have traditionally had an arms length relationship to formal methods of development, a number of special issues of prestigious journals such as IEEE Software and IEEE Transactions on Software Engineering have devoted special issues to the subject. However, the author feels that these positive aspects which have emerged over the last ten years are not enough to ensure the expansion in use of formal methods in the telecommunications industry. There is a clear need for some demonstration projects which are able to provide a high degree of assurance to the industrial communications community that real economic benefit accrues from formal methods. Of the areas surveyed in this chapter it is felt that this case can be made most easily in the areas of conformance testing and protocol conversion. Not only do we feel that this work contains scalable theories, but they do not require the use of the large numbers of trained personnel, something which has acted as a drag on the adoption of formal methods in large scale. It is tempting to regard protocol conversion as a relatively unimportant area. However, computer communication systems permeate almost all corners of the globe. Yet, achieving useful communication between systems generally remains a nontrivial problem. So if mathematics is so hard to use should we not be looking at other methods? The short answer is yes we should, but we also need to recognize that the basic principle behind formal notations is sound. A recent addition to the already large encyclopedia of systems languages is UML (Unified Modeling Language). There are many positive things being said of UML and it might, just maybe, be the answer to the systems engineers' dreams. However, UML is still a very formal notation. Are there domains within software engineering, particularly interface design, where we can dispense with mathematics altogether? A likely candidate is user interface design. There are many components and techniques available to the graphical user interface designer including:
Page 286
• Mouse: single click, double click, drag and drop; • Keyboard: accelerators, mnemonics, arrow keys, tab key; • Single, multiple, extended selection; • Popup menus; • Icons; • Overlapping windows; • Menu bars; • Toolbars; • Push buttons; • Notebooks; • List boxes; • Dropdowns; • Spin buttons; • Sliders; • Many different fonts and character sizes; • Colors; • Sounds. The abundance of user interface methods and techniques, coupled with the business objectives of increased productivity and reduced training time make the design of the user interface a complex and timeconsuming part of the development process. Although you are unlikely to find any mathematics, it is often the case that specialist test laboratories have to be built so that the user's behavior can be observed in controlled conditions. Iterative usercentered design following proven user interface design principles and techniques is required to create a usable interface. The industry is now realizing that user interface design requires special skills quite separate from those required for software development. Whereas industry requires skilled mathematicians to formally specify a system such as a protocol, psychologists are usually employed for user interface design. While it may lack the objectivity and conciseness of mathematics, the role of the psychologist is no less complex. Successful designers have mastered not only the hundreds of user interface design rules and guidelines, but more importantly, have a detailed understanding of user interface design principles, the building blocks upon which successful userbased software applications are built. However, because humans are involved and they are notoriously unpredictable, traditional development life cycle approaches are not always useful. More often than not, a prototype
Page 287
based approach is used where the development team can use real experiments to see if their ideas are working. This Lego approach is also finding favor within the software industry where it is called componentbased development. An interesting possibility that is beginning to emerge is to use UML to specify the function and interface requirements for a component and to use components to assemble software systems. Summary In order to overcome current limitations and develop a software technology suitable for problem solving, it is essential to recognize that software complexity grows with both hardware sophistication, the number of interfaces, and application domain. Through the development of specification languages such as UML that can be (at least indirectly via a code generator) executed to achieve rapid prototyping and the declarative view (that is, functional and logic programming), specification and programming have, to some extent, been converging. In fact, recently even text documents have been growing smarter. The huge popularity of presenting documents on the Internet through the HTML (and more recently XML) language is because this permits us to incorporate some elements of computation directly into our text. When one activates a link in a Web document, a computation closely analogous to a remote procedure call is performed. Of course, the procedure calls are very inflexible in the current setting, but it does not take much imagination to foresee many other possibilities. It is quite likely that something akin to HTML or XML, used to link components from various component repositories around the world, will provide the universal linking, or interface, method so sought after by those in the software engineering community. The programmer's dream is to discover the ideal programming environment that supports problem solving without writing programs as such and for this environment to be universally accepted. But achieving such standardization will be fraught with difficulty. Whereas God decided that the construction of a Tower of Babel was potentially harmful to their health, there are many businesses with invested interests in making sure that their approach becomes the standard. They will certainly view the adoption of a competitor's methods as harmful to their health. Hopefully, there will be a breakeven point where it will be in everybody's interests to standardize— but do not hold your breath!
Page 288
References [1] Jones, C., Systematic Software Development Using VDM, PrenticeHall, 1986. [2] Hoare, C., Communicating Sequential Processes, PrenticeHall, 1985. [3] Joyce,J., Finnegans Wake, Penguin, 1992. [4] Sagan, C., Contact, Pocket Books, 1997. [5] Ashby, W.R. Introduction to Cybernetics, Chapman & Hall, 1965. [6] Beer, S., Diagnosing the System for Organizations, Wiley, 1988. [7] Norris, M.T., M.W. Shields, and J. Ganeri, ''A Theoretical Basis for the Construction of Interactive Systems," BT Technical Journal, 5(2), 1987. [8] Green, P., "Protocol Conversion," IEEE Trans on Communications, 34(3), 1986.
Page 289
10— From Here to Eternity You can't depend on your eyes when your imagination is out of focus. Mark Twain
All industries mature over time. Communications is no exception. It has certainly come a long way in a relatively short time; the Internet and PC are still teenagers and the World Wide Web a mere toddler. Even the good old IBM mainframe is little more than a vibrant middleaged performer compared to most other engineering or technical artifacts. In many ways the capabilities of the communications industry have advanced faster than our ability to harness them. There may be plenty of drive and energy behind the advances in computers and telecommunications, but has there been the time to acquire the Methuselahlike wisdom of other engineering disciplines? It took chemists and physicists hundreds of years to hone their skills. Perhaps there is a lesson of history that we could (and should) be heeding. Figure 10.1 provides a general picture of how engineering disciplines evolve. Moving from left to right, the picture indicates that people get good at doing something (i.e., they develop a craft), this becomes more systematic (i.e., can be reproduced on a commercial basis), and, finally, a formal basis is found, and this underpins professional engineering. So where are we with softwarerich systems? Truth be told, we are probably moving along the commercialization limb. The evidence for this would be that we have standards such as ISO 9001 for software production, measures such as the Capability Maturity Model to assess commercial standing, and a raft
Page 290
Figure 10.1 The evolution of an engineering discipline.
Page 291
of technical standards and tools to aid production (see Appendix B). It could be argued that data normalization techniques, mathematically based specification techniques (such as VDM, Z, and CSP), and object orientation provide the basis for professional engineering. The state of practice and sheer scope of engineering in softwarerich systems make this statement more of a wish than a reality, though. Figure 10.2 gives a good idea of where we really are. To get to professional engineering—that is, largescale, masscustomized products with routine delivery schedules—we would need to blend the science in the formal definition and generation of interfaces with componentization and systematic integration. Lost Horizon So far, we have said little about the scale of engineering that we are tackling. It is a major factor in the state of the industry. This is, perhaps, best illustrated by looking at the growth in software embedded in everyday devices and services. A recent advertisement for cars made the claim that there was more software in its engine management system than in NASA's lunar landing module. This is only the tip of the iceberg. Some telephone exchanges now have over 20 million lines of code, fighter planes several million, and TVs up to a million. Even the humble shaver can have several tens of thousands! And all of this code is proprietary, despite the commonality of application across each of the areas. The point being made is that softwarerich technology is already so complex and moving so fast that it will have to be engineered; handcrafting is no longer up to the job. There are a number of reasons for the everincreasing amount of software in just about every electrical product in the marketplace. Part of it is due to the fact that applications are networked but there are other reasons. Will this upwards spiral of complexity abate? The smart money is on: • All companies coming to rely on networks and computers. The majority of the working population will be using computing peripherals as the tool of work. All companies will use them extensively and the variety of computer peripherals will grow (a phone, a laptop, a notepad, a pager, and so forth). The role of the simple telephone within businesses will reduce significantly (although voice communications will remain strong).
Page 292
Figure 10.2 The state of play in the software industry.
Page 293
• Computer peripherals being embedded in society. All people will be using computing tools in their private lives as well as their work. In the home, children will use them to interact with their friends and school; parents will use them for leisure, work, and home management; and they will be major tools in enabling their interaction with their many communities. • A mobility tsunami on the way. People will be much more mobile. At present, mobile data services, beyond the short message services currently in GSM, are not widespread. Deployment of the Universal Mobile Telecommunications Services (UMTS) infrastructure is in its infancy (although these services will quickly be taken up by global companies for their highly mobile workforce). However, the massmarket takeup of mobile services is about to start, the value is clear, the technology developing, and the market will flourish quickly. • An electronic business tsunami about to hit. Electronic commerce (e.g., trade, payment, and so forth) between businesses will be ubiquitous in support of a dynamic and changing service economy. The lack of universal standards has not yet enabled costeffective solutions for the mass market, but things are aligning, ready for a mass takeup and use in the near future. In short, if anything, complexity will rise. There is no return path. So, if complexity is going to be a fact of life, let us think a bit more about its nature. A useful start might be to make a simple statement that complexity arises in two basic ways. First, there is inherent complexity, where something just is intricate. In this case, we have atomic functions or entities that cannot be reduced to simpler ones. The second form of complexity is complexity of scale. In this instance, the root cause is replication. In this instance, there are lots of the same or similar entities. Each may be simple in its own right, but when assembled, a complex whole emerges. Maturity in any engineering discipline is signed by the ability to manufacture, and software is characterized by great freedom given to the programmer (who produces the end system). The software engineer has no real Lego bricks that can be used to constrain complexity to scale complexity (that is, loads of components). Thus, the end result is typically one of atomic complexity. The industry has tried to move from atomic to scale complexity through methods such as SSADM, JSD, and SDL, but people can too readily work around these. The only real way to constrain complexity is to limit choice, and
Page 294
the major problem then moves to integration and the design of the interfaces rather than the design of the components. Looking again at another engineering discipline, the main problem in bridge building once standard types and components were established became the design and placement of the approach roads. For some time now, there have been set types of bridges. For each one we know how to construct it, which questions to ask, what calculations to make, and what materials to use. We also understand how the (many) bits fit together to form the product. The result is a complex entity built of many simpler bits, but the way in which they are assembled is a science. It was not always thus. I.K. Brunel's first attempts fell down. Fortunately, he lived to learn from the experience. It could be said that the plethora of APIs now available helps to move us away from atomic complexity. The truth of the matter is that they only provide a set of interfaces, no guarantee that the components will work together. So lots of interfaces just give more complexity, a host of bits that do not necessarily operate as a cogent whole because they only cooperate where they are joined. It is for this reason that the interface equation has to consider the possible states behind each interface. Applying the interface equation glues two components into one new one (albeit bigger). Treasure Island In earlier chapters we illustrated components with reference to the car industry. This was not a random choice. The U.S. MIL standards for software production use the same ideas, as shown in Figure 10.3. In this model, each of the boxes constitutes a configuration item, some simple, some complex. In effect a component is managed and maintained as a separate entity. Again it is the interfacing of the components that makes the components into a system and the integration process and interface theories explained in the main part of the book that come into play. Summary This chapter takes stock of the information and software industry in terms of its maturity and likely evolution. In doing this we have introduced two key influences: • Engineering maturity: the natural evolution of ideas and practice in any technical discipline;
Page 295
Figure 10.3 A system as a set of configuration items.
Page 296
• Complexity: the fact that most of the IT industry is beyond the stage of handcrafting and there is no way back. The content of this book is a consequence of the former and has been driven by the latter. Our point in this chapter has been that the world of communications is now so complex that the production techniques behind it have to get more sophisticated. Componentbased engineering with a focus on interfaces and integration provides the basis for this sophistication. And when the imagination is in focus, practical progress can be made. Selected Bibliography European Information Infrastructure, ETSI Sixth Strategic Review Committee, May 1995. "The World We Are In," Prospect Magazine (http://www.prospectmagazine.co.uk/), November 1996. The Henley Centre, Media Futures, 1997, pp. 226–236.
Page 297
A— The Interface Equation—A Formal Theory of Interfaces The more original a discovery, the more obvious it seems afterwards. Arthur Koestler
Given the everincreasing complexity of modern high technology products, it is becoming more and more important to adopt a scientific approach to their design and construction. A key part of this is the development of wellfounded, formal methods that provide a reproducible and reliable solution to some part of the overall problem. The creation of methods, algorithms, and techniques is not (usually) the user's concern, more a specialist activity carried out in the corridors of academia. The practitioner does, however, want to be assured that the theories that they are given and come to rely on produce correct results. This appendix explains the ideas and steps in the development of one theory that fits into the designer's toolbox, the interface equation. The mechanics of its derivation may not be of direct relevance to the design, integration, and system problems faced in everyday life, but it is worth knowing how systemlevel techniques are derived, at least in outline. And so to the detail. This appendix may be tough going in places, but the final result meets Einstein's quality criterion of giving a fairly simple (but not overly simple) answer to a complex problem.
Page 298
Introduction So, is it possible to use formal, mathematical techniques to develop interfaces? This question has been an active area of research among academics and industrial scientists for many years [1–3]. The answer is yes, but it is not a simple matter [4]. That the mathematics is not straightforward is hardly surprising. We are, after all, dealing with a very complex problem. What is probably more surprising is that such a theory exists at all, though its application to real problems has been limited. In this appendix we are going to look at one approach, the interface equation. We will provide an overview of the interface equation and the underlying mechanisms associated with its solution. Details will be kept to a minimum for the sake of clarity; we anticipate that a good understanding of high school mathematics will be sufficient. We initially adopt a wide perspective and examine a number of approaches that could be described as representing, although not explicitly, the general class of interface equationtype theories. The focus, however, is on the work of Norris, Shields, Martin, and Pengelly [5–8], since it is the belief of the authors that this represents the best developed theory for solving the equation to date. This work developed the notion of an interface equation, a formal mathematical machine for interfacing two (or more) systems. By way of an example, we concentrate on protocol conversion as this represents the classic interface problem. It also takes us back to an earlier example in the book, that of Mrs. Smith and Mr. Lee. In many ways the technical aspects are not important. The message that we want to get across in this chapter is that systems integration and interfaces are not a black art, that there is a clear and unambiguous formality underpinning them. The other message, though, is that this formality is not simple and that there is a price to pay if we are to try and automate interface development. It is a difficult problem by any standards. Formal Methods The use of formal methods in systems engineering has been a rich area of study, particularly during the late 1980s. It was also evident that protocol conversion is an ideal application for formal methods, which are perceived to offer the best chance in terms of developing a completely general and concise calculus of interfaces [8– 10]. The overriding problem, it seems, in terms of generating a grand unified theory, is the absence of a suitable widespectrum language, a general language we can use for any problem that we may encounter. Even within the protocol conversion domain, no single formal notation has emerged as being sufficiently general for all problems. Given that, we will use a well
Page 299
known language called CCS (Calculus of Communicating Systems) which is general enough for our purposes [11]. An interface equation is an equation of the form (Mp|Mr)\A ~ Mq, where|is parallel composition, A the set of hidden actions (the events by which Mp and Mr communicate), and ~ is some notion of equivalence. We will not expand greatly on this notion of sameness; suffice it to say that there are many definitions depending on which characteristics are being equated. The situation is very similar to the various morphisms in mathematics (e.g., isomorphism, homomorphism, homeomorphism, and so forth). Note also the term machine, which is used as an abbreviated form of finite state machine. Mp and Mq are the two known protocols or machines; Mr is the interface machine to be found that satisfies the equation, if a solution exists. The first attempts at solving the interface equation were relatively crude. The first algorithm was called the discarding algorithm. This, in essence, looked at all possibilities. This was later improved resulting in the constructive algorithm. The latest solution technique uses the idea of quotient machines, a rather obscure term, but what it means is that common elements of one protocol and another are factored out, leaving the difference. The next section investigates the discarding algorithm and constructive algorithm. Before exploring these algorithms, it is worth spending a little time acquainting oneself with CCS. CCS is, on the face of it, quite a simple language. However, it is extremely powerful and well suited to the sort of analysis undertaken in the proceeding sections. A CCS formula, or specification as it is also called, looks like this:
This simple example says that P1 either does an action and goes to state P2 or it does a and goes to P3. We can represent this graphically in a very intuitive way, as can be seen in Figure A.1.
Figure A.1 A graphical representation of P1 = R2 + R3.
Page 300
There is a special action called tau ( ) which occurs when two machines synchronize, or talk to each other. Note that . that is an action and its complementary action (denoted by the bar) combine to form the tau action. A tau action is, in a sense, silent and cannot generally be observed as it is internal; it occurs inside the machines. Interested readers should consult Milner's excellent book for further details [11]. Solving the Interface Equation The interface equation, in its various guises, can also be viewed in more general terms as an alternative representation of the {em quotient machine} problem. The idea here is that the interface equation can be viewed as an equation of the sort a*b = c. We all know from elementary school mathematics that if a andc are known, then we can find b by dividing c by a. A similar thing can be done here, though it is somewhat more complicated. The essence of the concept is that in the interface equation shown above, Mr can be expressed as the quotient of Mq over Mp. In truth the situation is more complicated than this statement would seem to suggest. That the machines communicate with each other means that Mp cannot simply be factored out from Mq. The interface equation is formally defined: An interface equation is an expression of the form
where 1. 2. 3.
—the Mp and Mq machines cannot communicate. —Mp has no actions in A. —Mq has no actions in A.
The resulting machine Mr must satisfy the following: 1. Mr is a solution to the equation (Mp|X)\A ~ Mq—(M|Mr)\A ~ Mq 2. (r) (p) { }—nontrivial and means that actions of the Mp and Mr machines are distinct. Informally, we wish to find a machine (short for state machine) X which wraps itself around Mp so that externally (from the point of view of an ob
Page 301
server) Mp looks like Mq, even though internally they may be very different. Another way to look at it is to consider machine X as a translator attached to Mp so that any outgoing messages (or actions) are translated into Mq actions. To see this assume that one has a solution to the interface equation (Mp|Mr)\A ~ Mq. Let us denote its starting state by r. Then any sequence of transitions of the composite machine (p|r)\A to some arbitrary state (p'|r')\A must be accompanied by a corresponding sequence of transitions from the q state to a state q' such that (Mp'|Mr')\A ~ Mq'. More formally if (p|r)\A {s} (p'|r')\A, where s is some sequence of visible actions, then there must exist a state q' such that q {s} q' and (p'|r')\A ~ q'. Similarly, if q {s} q' then there must exist states p' and r' such that (p|r)\A {s} (p'|r')\A and (p'|r')\A ~ q'. For a given state r' of the solution there may be several combinations of states of the known machines, p' and q', for which (p'|r')\A ~ q'. If one collects together all such pairs then one can associate each state r' of the solution with a set of state pairs (p',q') such that (p'|r')\A ~ q'. That is, for each state r' of the solution one can associate a set K(r') given by K(r') = {(p',q') : (p'|r')\A ~ (q'} and (p'|r')\A {s} (p'|r')\A and q {s} q' and s is some sequence of observable actions. These K(r)sets are important. They are discussed later in this chapter. Using the sets K(r') for each state r' of the solution, it is possible to define derivations between the sets in an analogous manner to derivations between the states themselves. These derivations and the notion of observational equivalence give rise to the presence of certain conditions that these Ksets must satisfy individually and collectively in order that a solution to the interface equation exists, the socalled Icompleteness and Ocompleteness conditions. These conditions are rather technical in nature; the interested reader should consult the references. If we denote R(p) as the set of states reachable from p and R(q), the set of states reachable from q, then the process of finding a solution can be expressed as the determination of a subset of the power set P(R(p) R(q)) which satisfy these Icompleteness and O completeness conditions. The Discarding and Constructive Algorithms In this section we will examine the solution mechanism developed by Shields and Martin. There are a number of basic concepts which need to be covered before one can attempt to understand the theory. The solution centers on the construction of special sets, the Ksets, which need to meet certain conditions (Icompleteness and Ocompleteness) if a solution is to be found. Ksets are fundamental to the discarding and constructive algorithms. The internal structure of Ksets is intimately related to the structure of Mp. In addition, each Kset is associated with one, and only one, state of Mr. Each Kset consists of a set
Page 302
of triples (p, q, r). Since each Kset is associated with one and only one r, the r is usually dropped from the triple and attached to the K, so a distinct Kset is labeled K(r). The Ksets partitions R(p) R(q), so all ordered pairs (p,q) K(r) have the property that (p|r)\A ~ q. Let Mp be a machine which can exist in the state p. The R(p) is defined to be the set of all states reachable from the state p. The set (p) is defined to be the set of all actions which are possible ' ' p' R(p). Let A be a set of actions, then R(p) is the set of all states reachable from p by sequences, s, of actions not contained in A; i.e., s ( (p)A)*. K(r)sets by themselves are not sufficient to generate a solution. Two new concepts, Icompleteness and Ocompleteness, provide the necessary and sufficient conditions to find a solution. This is an important step, which together with Ksets, provide the cornerstones for the theory. The definitions for Icompleteness and O completeness are given below. K is said to be Icomplete if " (p',q') K, if p'
{µ}p" for some p" where µ A and µ ; then q'
{µ}q" for some q".
For Ocompleteness, we need to consider transitions between Ksets (and hence between states in the Mr machine). There are two types since Mr will either do a transition which does not involve communication with Mp or does communicate (synchronize) with a transition in Mp. We refer to these transitions as µ,O (where µ (q) (p) { }) and µC (where µ A) respectively. The difference is that if we have a K {µ,O} K' then every pair (p,q) K will have a µ,O transition to a pair (p',q') K'. A µ,C transition, on the other hand, need only be between one element of K and one element of K'. We can now define ocompleteness. K is said to be Ocomplete wrt (with respect to) S, where S is a set of Ksets, if " (p',q') K if µ (q) and q' A, p0 . . .pn, p'0 . . .p'm R(p) and K0 . . .Kn, K'0 . . .K'm S such that:
{µ}q" then m,n 0, µ1 . . .µn, µ'1 . . .µ'm
Page 303
The set S is referred to as a system. It is a set of Ksets which together form a potential solution. Informally, one can view S as a sort of cover over R(p) R(q). The discarding algorithm starts with S as the set of all possible Ksets, some of which are discarded as the algorithm proceeds. The constructive algorithm adopts a more general definition of the Ksets and applies certain conditions which immediately rule out certain sets. In this case S has fewer, but larger Ksets. In general S will be too small and require additional points to be added, thus the term constructive. In both cases, however, the set S is gradually refined until the complete set satisfies I completeness and Ocompleteness These Icompleteness and Ocompleteness relations are necessary and sufficient conditions for solution (if it exists). The essential point is that every state of the unknown machine can be expressed as a subset of the Cartesian product of the two known machines, subject to these Icompleteness and Ocompleteness conditions. Intuitively, the Icompleteness and Ocompleteness conditions amount to establishing a path of actions between any two ordered pairs in R(p) R(q), which may be in different Ksets. A Kset is essentially Icomplete if it is connected, that is, there is a path connecting any two elements of K. But if the two pairs in question lie in different Ksets then we will need transitions from one Kset to another to establish the path. This is summarized in Figure A.2. Since each Kset corresponds to a unique state r of Mr, then these transitions between the Ksets are in fact the transitions of Mr. One can view Icompleteness as being associated with the reachability (or connectedness) of the Mp machine and ocompleteness with the connectedness of the Mr machine. Icompleteness says that a transition in Mq is equivalent to a sequence of transitions in Mp which lie entirely within one Kset. Ocompleteness says that we must add actions from Mr to find an equivalent sequence of transitions if Mp is unable to perform the appropriate action(s). If a set of Ksets have sufficient transitions so that any two pairs may be connected, then they are said to be Icomplete and Ocomplete. Informally, what we have is a path searching algorithm which attempts to find a suitable path from one pair to another—if it gets to a point where it can go no further, it starts again and tries a different route. A Kset which blocks any such path is said to compromise a potential solution. Thus a solution is said to have been found when we have a set of Ksets which is both Icomplete and Ocomplete. Another way of looking at it is to say that a solution has been found when we have an uncompromised set of Ksets. Let S be a potential solution of the interface equation (a set of Ksets). We say that K compromises S if K is either not Icomplete or K is not Ocomplete wrt S. Let there be m states of Mp and n states of the Mq machine, then there will in general be 2mn different sets of such state pairs that one could create.
Page 304
Figure A.2 In this example Mq does an implicit q transition from q to q'. Mp cannot carry out this transition, hence it must be done by Mr. Here we see that a series of transi tions, both within and between Ksets needed to find an equivalent transition in (Mp|Mr)\A. In this way (Mp|Mr)\A ~ Mq, if this can be done for all transitions in Mq. In other words, the Ksets are Icomplete and 0complete.
Of these sets, however, only a few will correspond to states of the solution discussed above. The essential process of devising an algorithm to solve the interface equation involves finding those sets of the 2mn possible which can be identified with a solution. If during this process there remain no sets which contain the pair (p,q) (where p and q refer to the starting states of the known machines), then no solution is possible. Otherwise one will eventually obtain a system of Ksets all of which will satisfy the necessary conditions. It is then a simple step to assign each set a state name (with associated behavior) which corresponds to a solution of the interface equation. The main practical difficulty with the discarding algorithm is the huge number of sets that one needs to consider initially. Shields' work was extended by determining the necessary and sufficient conditions for a solution in the general case (i.e., full nondeterminism is accounted for) and devising an algorithm which operates in the opposite way to the discarding algorithm. That is, a system is built of sets until we obtain a system which satisfies the required
Page 305
conditions. This is the constructive algorithm. By analyzing the structure of the solution, it is possible to rule out certain kinds of sets at a very early stage during the construction process which would have involved a large amount of effort in discarding algorithm. The way the constructive algorithm works can be summarized as follows. Firstly, the two machines are reduced to a minimal form where redundant states and actions, from the point of view of observational equivalence, are deleted. With the machines in such a form, then any set corresponding to a solution state is such that for each state of Mp, for example, p', there is one and only one pair of the form (p',q') (where q' is some state of the reduced Mq) in that set. Instead of generating all the sets of P(R(p) R(q)), two relations on the elements (p,q) of R(p) R(q) are introduced which must be satisfied by those elements to be part of a solution. These two relations are variants of the Icompleteness, and Ocompleteness conditions, and are referred to as I'completeness, and O'completeness. These are formally defined as follows: Let W R(p) (R(q) then W is said to be I'complete if " (p',q') W and " p" : p" R(p') then (p",q") W : (p',q') ~ (p'',q"). This set W is a sort of super Kset (see below). Given S P(R(p) R(q)) then (p',q') K' S (denoted by the triple (p',q',K')) is said to be O'complete in S if " µ q' and q" R(q) q'
{µ}q" n > 0,
p0,p1 . . . pn, p(n+1) R(p), K0, K1 . . . Kn, Notice that these are more general (weaker) than their respective counterparts in the discarding algorithm. By applying these weaker conditions the resulting sets are in general quite large, we could call them super Ksets. The elements that do not satisfy these conditions are discarded until a subset of P(R(p) R(q)) is found that satisfies the conditions, denote this by S'. S' is in fact a set of super Ksets. If S' does not contain an element (p,q) K S' then no solution is possible. This step of the algorithm quickly deletes pairs (elements) which cannot form part of the solution. The next step involves using the equivalence class, which is defined over the I' completeness, condition to partition the set S'. Let (p',q'), (p",q") S R(p) R(q), then define the relation {S} by the following: (p',q') {S}(p",q") if
Page 306
p' RA(p") or p" RA(p') and if p' RA (p") then L{p",p'} = Ls((p",q"), (p',q')), if L{p",p'}>1 then " µ ((p)A) if p'' {µ}p' then q" {µ}q', and 1. if p" RA(p') then L{p',p"} = Ls((p',q'), (p",q')), if L{p',p"}>1 then µ ((p)A) if p' 2. if p' = p" then q' = q" and " µ ((p)A ) if p' {µ}p' then q'
{µ}p" then q' {µ}q" and
{µ}q'.
where L is the length of a trace (of states) and L is the length of a (corresponding) sequence of actions. The relation {S} is symmetric, but not transitive or reflexive. In order to generate an equivalence class, we need to find the transitive closure of {S}. Define S{p,q} to be the largest subset of R(p) R(q) which is I'complete. Let {S{p,q}} be the transitive closure of {S{p,q}} (note that {S{p,q}} is a reflexive relation) then define (p,q) to be the set of equivalence classes given by WR{S{p,q}} on S{p,q}. then K is I'complete. This partition gives the basic form of the sets which will form the solution, if it exists. O'completeness is then applied over this partition. So the sequence is to first check for I'completeness, generate S and partition S to form Ksets, and then to check for O'completeness. As stated above, the Ksets generated by the discarding algorithm obey the Icompleteness and Ocompleteness conditions which exists between the sets. In the constructive algorithm, one uses a weaker version of these conditions so that sets (and hence the pairs that they contain) which clearly do not form part of the solution can be computed quickly and hence eliminated. This is an iterative process in that the deletion of a pair may affect the conditions of other pairs which would subsequently result in their deletion. The process is continued until all pairs satisfy the required conditions. Under certain circumstances a solution may be obtained already at this stage which involves polynomial time algorithms from graph theory. In general, further processing is required in the form of an extraction process. This is achieved through an approach which is equivalent to a tree search with backtracking. Essentially, particular subsets are chosen in turn from the above partition; in general there will be a number of choices that can be made. After each subset
Page 307
is chosen, some processing is required to ensure that the necessary conditions still hold. Eventually a situation is reached where no more subsets can be chosen. If the required conditions are not satisfied, then the process is backtracked to choose another subset from the list of choices. The efficiency of this part of the algorithm is dependent on good heuristics to make the best choice. In the examples that have been considered, it was found that fairly simple heuristics are sufficient to obtain a solution quite quickly. However, in the general case more sophisticated heuristics may be necessary (or the need for human intervention) in order to guide the system towards a potential solution. Notice that a fully extracted set of I'complete and O'complete Ksets is in fact Icomplete and Ocomplete, since Icompleteness and Ocompleteness are properties of the solution, not the method used to derive it. If the algorithm reaches a point where no more sets can be chosen and all the conditions are satisfied, then again in certain circumstances this situation results in a solution to the interface equation. This condition is: all states of Mp are reachable from any other state by sequences of actions not contained in A. For classes of interface equation that do not satisfy this condition, then further processing may be required. For certain classes of interface equation, the constructive algorithm as currently implemented can obtain complete solutions far quicker than the discarding algorithm. Furthermore, the algorithm is more widely applicable than the method given in the concurrency workbench in that nondeterminism in the machines can be taken into consideration. The current algorithm is particularly suited to solving interface equations which are such that each state of the p machine is reachable by actions not contained in A. In such cases the size of the sets obtained during the solution is well defined. In the more general case, the algorithm requires a procedure for merging and splitting the sets during the construction procedure. This introduces further complexity into the solution procedure and would also greatly benefit from good heuristics. A Small Example Having established an informal description of the interface equation, a small example can now be investigated. In this example we will use the discarding algorithm to generate a solution. This example is by necessity more rigorous than our previous informal discussion. Consider two machines, Mp and Mq, given by the following agents. Mp is given by:
Page 308
Mq is given by:
Let A be given by A={ }. The problem is to find a machine Mr which has a state r1 such that (p1|r1)\A ~ q1. One first notes that q2 ~ q4 since there is a cycle between these states. (A cycle is a sequence of states q'1 . . . q'n : q'1 { }q'2 { }. . . { }q'n { }q'1.) Hence Mq can be reduced to a machine with observationally distinct states to obtain, after relabeling:
Clearly q1~q'1 and so we consider Mq to be given by the above specification. In the subsequent analysis, the primes are dropped. The solution, if it exists, will be an uncompromised system S of Icomplete and Ocomplete Ksets. The first step is to start constructing the Ksets, which we also require to be Icomplete. Before doing this we need a few more definitions. Let (p',q'),(p",q") R(p) R(q), µ
Define
(p) (q){ }and define
Page 309
These sets define types of neighborhood associated with each ordered pair. Define W {i}(p',q') by the following:
In general for a given (p',q'), there will be a number of W (p',q') depending on which subsets are chosen in i of the above definition. The superscript i is used to distinguish between these subsets. When there is no danger of ambiguity, the superscript is dropped. We give the set of all W {i}(p',q') a special name: Define (p,q) = { {B in X} | X { W {i}(p',q') | (p',q') R(p) R(q) }} A Kset is an element of . Let S be a system and K S then W (p,q) K. We define a decomposition of S into S' by the following conditions: • W (p,q) S' • if K { ,Y}K' exists for some K, K' S and L K with L S' : L ,Y derivation from L to K' is also contained in S'.
{ ,Y}K' then the set given by the minimal union of W from K' containing the image of the
• if K { }K' exists for some K, K' S and L K with L S' then the set given by the minimal union of W from K' containing the image of the derivation from L to K' is also contained in S'. Let (p',q') R(p) R(q). Define (p',q') (p',q')
K and (p",q")
K' : (p',q')
then (p'',q") K' : (p',q')
{ ,C}(p",q") if p' { ,C}(p",q") " (p',q')
{ ,C}(p",q").
If K, K' (p,q) then define K { ,C}K' if: K if
Page 310
We shall now obtain a system S which contains no compromising Ksets. By definition the system must contain a set W (p1,q1) which we calculate first. By definition (p1,q1) W (p1,q1. We need to calculate all nonempty IX(p1,q1,p') where X {( ,I)( ,P)}, p' R(p1), (p1). These are I{a,I}(p1,q1,p1) and I{a,I} (p1,q1,p2) which are given by:
By way of an example, we will look at the derivation of I{a,I}} (p1,q1,p1) more closely. We wish to find the set of pairs for which there is an alpha transition from (p1,q1) to (p1,q'). ® a ® a ® a Hence, we need to find the set of qstates reachable, via an alpha, from q1. Referring to machine, it can be seen that p1 { }p2 and that q1 { }q1, q1 { }q2, ® t ® a Þ a and q1 { }q3 { }q3 (or q1 { }q3). Thus we arrive at the set given above. The process is repeated for all IX sets. Î Î Now, for any (p',q'1)(p',q'2) K S where S is an uncompromised system, then q'1 ~ q'2. This follows since (p' | r(K)\A ~ q'1 and (p' | r(K)\A ~ q'2. Thus since W (p1,q1) does not compromise the system and no pair of states are observationally equivalent, we can only have one element contained in W (p1,q1). By definition Î Ç (p1,q1) W (p1,q1), and so we can only have W (p1,q1) I{a,I} (p1,q1,p1)={(p1,q1)}. We cannot have (p2,q1) nor (p2,q3) contained in W (p1,q1) since by Ç È definition W (p1,q1) would not then be Icomplete. We thus have W (p1,q1) (I{a,I} (p1,q1,p1) I{a,I} (p1,q1,p2))={(p1,q1)(p2,q2)}. m t One must now consider the ,I and ,P derivations from (p2,q2), obtaining:
Ç Ç Using the same arguments as before, one can only have W (p1,q1) I{b ,I} (p2,q2,p1)= W (p1,q1) I{t,P} (p2,q2,p1)={(p1,q1)}. Thus for W (p1,q1), we have the only choice:
This is certainly Icomplete, and so we now need to find sets to add to our system to make it Ocomplete. Î ® a ® a ® m First consider (p1,q1) W '(p1,q1). Clearly W '(p1,q1) is Ocomplete with regards the q1 { }q1 and the q1 { }q2 derivations. The q1 { }q3 de
Page 311
rivation indicates that a ,O derivation is required since (p). Conditions 1, 2, 4, and 5 in the definition for Ocompleteness obviously do not apply since p1 does not an implicit derivation to p2 where A. Also since (p) then we must have condition 3. We thus consider possible and ,O derivations from W '(p1,q1). The images possible of a ,O derivation from W (p1,q1) are {(p1,q3)(p2,q2)(p2,q3)} where we use the notation to denote any nonempty subset of the elements contained within. Now (p2,q3) cannot be contained within this set because it would not then be Icomplete (since p2 { } but q3 does not do a ) and so the only image possible is {(p1,q3)(p2,q2)}. Next we calculate W (p1,q3) and (p2,q2). One obtains:
È Since we require W (p1,q3) to be Icomplete then W (p1,q3)cap I{a,I} (p1,q3,p2)={(p2,q2)}. Also since our required set, which is of the form W (p1,q3) W (p2,q2), contains (p1,q3) and no pairs of q1, q2, and q3 are observationally equivalent to each other, then the only elements that can be considered for the set from are {(p1,q3),(p2,q2)}. If we identify this set with r2 and W '(p1,q1) with r1, i.e.:
® m ® m ® m we have K1 { }K2 (r1 { }r2). K2 satisfies the Ocompleteness of K1 for the derivation q1 { }q3. The final derivation from q1 that we need to consider is ® t q1 { }q3. Þ e t Þ e To satisfy Ocompleteness, we require (note p1 has no tau actions or actions contained in A) K1 { }K3 no actions or actions contained in A : K1 { }K3 with Î t W (p1,q3) K3. The images possible of a derivation from (p1,q1) are given by {(p1,q1)(p1,q3)(p2,q2),(p2,q3)(p2,q1)}. Again Icompleteness allows only (p2,q2) ® t ® t in the second square brackets. Thus we can satisfy Ocompleteness for the q1 { }q3 derivation (K1 { }K1 gives no new information) only if we have K3=K2 ® t and so we have r1 { }r2.
Page 312
We now need to look at (p2,q2) K1 and consider derivations from q2. They are:
Taking cases 1–5 in turn, we show that Ocompleteness is already satisfied. • p2
{ }p1 and (p1,q1) K1p2
{ }p1 and (p1,q1) K1
• In this case we have K1 { ,O}K2 and (p2,q2) K2 • Here we only have p2 { }p1 and (p1,q3) K1. However, Ocompleteness is satisfied because we have p2 { }p1, K1 { }K2 and (p1,q3) K2 • Here Ocompleteness is satisfied because p2
{ }p1, K1 { }K2 and (p1,q3) K2
• Here we clearly need a ,O derivation at some stage. It is not possible to have K1 { ,O} because q1 cannot do an implicit h, but K2 does have a h,O derivation, the images of which are {(p1,q1)(p1,q3),(p2,q2) (p2,q3)(p2,q1)}. For reasons stated previously, due to Icompleteness only the (p2,q2) term in the second set of square brackets leads to an uncompromised system. Considering the options in the first set of square brackets, one has the choice K2 { ,O}K2 or K2 { ,O}K1. Both these choices satisfy the Ocompleteness condition. That is, if K2 { ,O}K2, then we haveK1 { }K2 { }K2 and (p2,q2) K2 or if K2 { ,O}K1, then wehave K1 { }K2 { }K1 and (p2,q2) K1. Having satisfied the Ocompleteness of K1, we now need to look at K2. Clearly, K2 is Icomplete. The element (p1,q3) satisfies Ocompleteness using the sets and derivations already generated, provided we have K2 { }K1. The agent q2 in the second element has the derivations listed 1–6 above and the first of these does not satisfy the Ocompleteness condition with the results obtained so far. We therefore require further derivations from K2. The only possible tau derivations from K2 is K2 { }K2 which does not help and so an , C derivation from K2 are given by {(p2,q1)(p2,q2)(p2,q3)}. We cannot have a set containing
Page 313
(p2,q1) because it would not then be Icomplete, and so we are left with two possible choices: position.
derivation does not help since it leaves us in the same
However, the , (p2,q2) K1 and (p2,q2) satisfy the Ocompleteness conditions in K1 then (p2,q2) satisfies the Ocompleteness conditions in r2. We therefore have that the system consisting of the sets K1 and K2 contains sets which are both I and Ocomplete and since (p1,q1) K1, then it constitutes an uncompromised system. The transitions between the Ksets are thus:
These translate immediately to the following Mr machines. It so happens that the r1 in each of these generated solutions is observationally equivalent although in general this may not necessarily be so for multiple solutions. Well, that was something of a tour de force, wasn't it! And this example is rather trivial. Industrial problems would involve thousands of states. Even with this small example, the details associated with the underlying theory are quite complex. It is perhaps not surprising that the algorithmic complexity of this approach is certainly NP and possibly EXP. Clearly the process of formalizing and then automating the interface development process has some way to go before it reaches the point where it can be sensibly downstreamed to industry. Conclusion This chapter has presented an overview of the interface equation. By presenting the basic theory, we have laid the foundations for the work that follows—all the basic tools required have been covered. The simple example presented in this chapter demonstrates the principles involved. As can be seen the underlying approach is one of set generation and path searching. As such, it is perhaps not surprising that the complexity of the problem increases exponentially with the number of states of the respective machines. The discarding algorithm, being very much concerned with the behavioral (dynamic) properties of the machines, ignores many of the structural (static) clues inherent within the problem. It is a bruteforce approach which considers all possibilities, discarding those combinations which do not lead to a solution.
Page 314
The constructive algorithm goes some way towards addressing the weaknesses of the discarding algorithm by including some aspects of structure, primarily in the derivation of the Ksets. If Mp is reachable by actions not in A, then the algorithm can have a considerable impact on the processing times, perhaps halving the time required by the discarding algorithm. However, if the reachability condition is not met then the algorithm could actually be worse than the discarding approach. This weakness could be addressed via a suitable mergesplitting algorithm, but this itself is likely to be computationally hard. The discarding algorithm then presents an essentially behavioral approach to the problem of solving the interface equation. By considering some of the structural properties within the protocols, the constructive algorithm is, for a specific class of problems, a significant improvement. More recent work has shown that there is another approach which would appear to be more general. It is based on the observation that, from a topological point of view, the graph of Mr is similar (in a precise sense in fact) to Mq divided by Mp (see Figure A.3). Mr is a sort of quotient machine, and it turns out that, with some modifications, graph quotient algorithms [12,13] can be used to generate the interface specification. Indeed, this approach points to a much deeper relationship between automata, dynamics and topology [14].
Figure A.3 The quotient structure of the machines Mq, Mp, and Mr.
Page 315
References 1] Calvert, K., and S. Lam, "Formal Methods for Protocol Conversion," IEEE Selected Areas in Comms., 8(1), 1990. [2] Calvert, K., and S. Lam, "Formal Methods for Protocol Conversion," IEEE Selected Areas in Comms., 8, 1, 1990. [3] Okumura, K., A Formal Protocol Conversion Method, Proc. ACM SIGCOMM'86 Symp., 1986. [4] Lam, S., and A. Shanker, "A Theory of Interfaces and Modules I—Composition Theorem," IEEE Trans on Software Eng., 20, 1, 1994. [5] Norris, M. T., R. Everett, G. Martin, and M. Shields, "A Method for the Synthesis of Interactive System Specifications," Proc. Global Telecoms. Conf., 1988. [6] Norris, M. T., and M. W. Shields "The Interface Equation," Mathematics Bulletin, 25, 6, 1989. [7] Shields, M. W., "Implicit System Specification and the Interface Equation," The Computer Journal, 32, 5, 1989. [8] Pengelly, A., and D. Ince, "A Graph Theoretic Solution to the Interface Equation," Applications of Combinatorial Mathematics, 1995. [9] Green, P., "Protocol Conversion," IEEE Trans on Comm., 34, 3, 1986. [10] Lam S., "Protocol Conversion," IEEE Trans. Software Eng., 14, 3, 1988. [11] Milner, R., A Calculus of Communicating Systems, SpringerVerlag, 1980, 92, Lecture Notes in Computer Science. [12] Aurenhammer, F., J. Hagauer, and W. Imrich, "Cartesian graph factorisation at logarithmic cost per edge," Computational Complexity, 1992, p. 2, 331–349. [13] Winkler, P., "Factoring a Graph in Polynomial Time," European J. Combinatorics, 8, 209–212, 1987. [14] Arbib, M., and E. Manes, "Adjoint Machines State—Behavior Machines and Duality," Journal of Pure and Applied Algebras, 1975, p. 6.
Page 317
B— Standards, Organizations, and Initiatives All government, indeed every human benefit and enjoyment, every virtue, and every prudent act is founded on compromise and barter. Edmund Burke
In a complex world, some measure of standardization has to be welcome. When it comes to the communications industry, the standards themselves are a further source of complexity. One of the wry observations on the industry is that the nice thing about standards is that there are so many of them! This appendix explains a little of what each one does. No great depth—a little inaccuracy saves loads of explanation—but enough to aid understanding along with references to more detail if you need to know it. Architectural Tradeoff Analysis (ATA) Initiative The Architectural Tradeoff Analysis Initiative from the Software Engineering Institute (SEI) builds on their earlier work developing a Software Architecture Analysis Method (SAAM) to evaluate software architectures. They have now developed this further into a Architecture Tradeoff Analysis (ATA) Method. The method can be used to assess architectures against a range of different quality attributes and to provide analysis of patterns and architectural styles. • http://www.sei.cmu.edu/activities/ata/ata_init.html
Page 318
Business Application Programming Interfaces (BAPI) BAPI produced by SAP AG allow users to interface with and extend the SAP R/3 Enterprise Resource Planning (ERP) application. While BAPI is a proprietary API, SAP/R3 is one of the leading vendors of largescale business frameworks and is engaged in a major drive to componentize its products. While SAP R/3 does not yet allow components from other vendors to truly plug and play into the framework, there are a large number of thirdparty software vendors whose products integrate with SAP R3 and companies who specialize in SAP/R3 integration. There are over 200 BAPIs across all SAP R/3 business application areas. They are Microsoft COM/DCOM compliant and will be CORBA compatible in the future. SAP is a founder member of the Open Applications Group (OAG) and has committed to make all relevant BAPIs compliant with the OAG's Integration Specification (OAGIS). The largescale use of SAP/R3 and strong links with the OAG may well lead to the BAPIs becoming de facto standards in the ERP business framework market. • http://wwwext.sap.com Business Object Component Architecture (BOCA) Produced by the OMG Business Object Domain Task Force (BODTF), BOCA is an architectural template that specifies, but separates, application domain semantics from specific technology implementations. BOCA comprises: • Domain semantic model: metamodel describing interaction of domain objects, based on the OMG object model; • Component definition language (CDL): textual representation of the contract among cooperating domain objects or components; • Contract: configures and structures domain concepts, a model instantiating the domain semantic metamodel; • Framework: an execution environment for implementing domain components. • Digre, T., "Business Object Component Architecture," IEEE Software, September/October 1998. • http://www.omg.org/techprocess/meetings/schedule/Business_Objects_RFP.html
Page 319
Business Object Domain Taskforce (BODTF) The term business object is often referred to without any clear definition of what the term means. The OMG has business objects as part of their Object Management Architecture (OMA) and have set up the Business Object Domain Taskforce (BODTF, formerly BOMSIG) to define and promote their use. Specifically they state their aims to facilitate and promote: • The use of OMG distributed object technology for business systems; • Simplicity in building, using, and deploying business objects; • Interoperability between independently developed business objects; • Adoption and use of common business object and application component standards; • Evaluation of proposals for OMG specifications for objects, frameworks, services, and architectures applicable to a wide range of businesses. The OMG defines a business object as "a representation of a thing active in a business domain." It goes on to say, "Business objects can be viewed as Modeling Objects used in the design process (or BPR) and as (run time) objects in the information system." Business objects may represent common entities such as customer or be domain specific. Despite a lot of effort in trying to use IDL and extensions to IDL to define business components, nothing substantial has yet emerged. The taskforce is currently voting on new proposals to define the way forward, with the use of CDL being one possibility. • http://www.omg.org/techprocess/sigs.html#bomsig Component Definition Interchange Format (CDIF) The Component Definition Interchange Format (CDIF) is a family of standards produced by a division of the Engineering Industries Association (EIA) to define standards for the exchange of information between modeling tools and between repositories, and to define the interfaces of the components to implement this architecture. First released in 1991, CDIF provided a textbased notation for describing semantic and presentation information for the most commonly used modeling standards based on the CDIF Integrated MetaModel. In collaboration with the OMG, CDIF has defined how the metamodel of the Unified
Page 320
Modeling Language (UML) can be used with the CDIF Transfer Format or the CORBA bindings to interchange object analysis and design models in a toolindependent manner. As a result, CDIF is now being incorporated into the OMG's XMI standard. • http://www.eia.org/eig/cdif/index.html Component Definition Language (CDL) The Component Definition Language (CDL) is part of the OMG's Business Object Component Architecture (BOCA). CDL is a textual representation of the semantic relationships between components at the domain or business level. It can describe the interfaces of domain components, the relationship between components, and the collective behavior of groups of components. It is a superset of the OMG's Interface Definition Language (IDL) with specific extensions to support the representation of domain semantics including constraints, rules, states, attributes pre and postconditions, exceptions, events, and so forth. For implementation CDL must be mapped to a target framework. CDL has been proposed as a mechanism for domainspecific specifications being produced by the various OMG task forces. However, most task forces are using IDL at this stage. • http://www.omg.org COM/DCOM The Component (or Common) Object Model (COM) and Distributed COM (DCOM) specifications cannot be regarded as standards as such because they are the proprietary offering from a single vendor, Microsoft. However, the ubiquity of Microsoft software on the desktop means that client software in a distributed client/server architecture will almost certainly be a Microsoft application or run on a Microsoft operating system and need to interface with Microsoft applications. Thus in any largescale enterprise it is very difficult to ignore the need to interface with Microsoft Object Models. COM objects can be any one of a variety things: a C++ object, an OLE component, a piece of code, or a complete application such as one of the Microsoft Office products. The main characteristic of a COM object is its interfaces. There are some basic interfaces which all objects must implement, but beyond that the user can have any number of interfaces. It does not have the component or interface specification language like
Page 321
CORBA's IDL, but is a binary standard. Once a COM or DCOM object is complete, it is compiled and supplied as a binary executable for the particular environment for which it is intended. Initially intended for application on a single machine using the Windows operating system, it was expanded to allow communication with components on other Windows NTbased systems and thus DCOM was introduced in 1996. • www.microsoft.com/com CORBA The Common Object Request Broker Architecture (CORBA) is a standard produced by the OMG to allow applications to communicate with one another independent of location, platform, or vendor. Effectively a messaging bus for communications between objects, CORBA 1.1 was introduced in 1991 and defined the IDL and the APIs that enable client/server objects to interact using a specific implementation of an ORB. CORBA 2.0, adopted in December of 1994, specifies how ORBs from different vendors can interoperate. CORBA is a client/server middleware that enables a client to invoke a method on a server object, which can be on the same machine or across a network. The ORB intercepts the call and is responsible for finding an object that can implement the request, pass it the parameters, invoke its method, and return the results. The client does not have to be aware of where the object is located, its programming language, its operating system, or any other system aspects that are not part of an object's interface. CORBA also provides a set of services that can be used by developers (naming, event notification, security, persistence, and so forth) and facilities. Facilities are frameworks (OpenDoc, UML, and MOF). On top of these layers are domainspecific frameworks of business objects. OMG Special Interest Groups and Task Forces have been set up to agree to the standards for these domains. Primarily aimed at object technology, CORBA is one of several competing sets of distributed technology standards that will provide the plumbing for a true distributed componentbased approach. CORBA 3 is soon to be released and will include support for multiple interfaces (like DCOM), DCOM/CORBA interworking, and asynchronous messaging. One of the most important new developments will be the CORBA component model. • http://www.omg.org/corba/whatiscorba.html
Page 322
CORBA Component Model The CORBA component model is a common modeling standard being developed by the OMG, based on IDL, that would allow components to be specified in a technology independent format that would allow subsequent code generation into other component implementations such as ActiveX or Enterprise Java Beans (EJB). • http://www.omg.org/corba/whatiscorba.html Distributed Computing Environment (DCE) Produced by the Open Software Foundation (OSF, now the Open Group), DCE defined a vendorindependent standard for making secure, controlled remote procedure calls (RPCs) between processes running on different systems. It also introduced the concept of naming services with unique identifiers to make it easy to locate distributed resources. DCE paved the way for, and is being replaced by, the type of distributed environments that we now see in DCOM and CORBA. DCE was the first to introduce an IDL. • http://www.osf.org/dce/index.html Department of Defense (DoD) Software Technology Programs The U.S. Department of Defense (DoD) has the following architecturefocused software technology programs: • CARDS: an Air Force Program aimed at promoting systematic software reuse by DoD and U.S. government agencies. CARDS has built an operational domain specific, architecturebased reuse library from information defined in the PRISM Generic Command Center Architecture which is built on the STARS Reuse Library Framework. A prototype component assembly system allows the qualification of components and the incorporation of qualified components off the shelf (COTS) from commercial software vendors. CARDS is also doing analysis on Architecture Description Languages (ADLs) with the aim of providing guidance to potential users. • DISA: Defense Information Systems Agency (DISA), part of the DoD Software Reuse Initiative. DISA's Software Systems Engineering Direc
Page 323
torate's Software Reuse Program (SRP) is responsible for the Defense Software Repository System (DSRS), an automated repository designed primarily to help software developers in the reuse of software assets. Assets in the repository include: requirements, design specifications, architectures, design diagrams, source code, documentation, test suites, and repository support items. • DSSA: an ARPA program to develop standard DomainSpecific Software Architectures (DSSAs) in areas relevant to the DoD. In particular it analyzes the effect of architecture choice on system performance and builds component assembly tools to encourage reuse. • PRISM: development of an objectoriented architecture to encourage reuse of COTS by allowing integration through wrappering. • STARS: an ARPA program aimed at accelerating change in DoD software development to take up domainspecific and reusebased ideas with the aim of increasing software productivity, reliability, and quality. Demonstration projects are applying distinct architecturecentric reuse processes, methods, and tools, based on common conceptual foundations. Also see ''Software Architecture Technology Guide." • http://wwwast.tdsgn.lmco.com/arch/arch008.html ESPRIT Software Evolution and Reuse (SER) A European initiative comprised of seven projects aimed at supporting software evolution, reuse introduction and management, maintenance, and software enhancement. • REBOOT: reuse based on objectoriented techniques; • PROTEUS: support for system evolution; • RECYCLE: support for maintenance and enhancement activities; • EUROWARE: enabling reuse over wide areas; • EUROBANQUET: managing the evolution of banking applications; • COSMOS: measurement of application quality and complexity; • SCALE: support for largescale reuse activities. • http://dis.sema.es/projects/SER/sermain.html
Page 324
Interface Definition Language (IDL) IDLs provide a standard way of defining the interface between objects independently of the technology in which the components will be implemented. IDLs were first used in the Distributed Computing Environment (DCE) specifications and now both the OMG's OMA and Microsoft's COM have IDLs, although there is no commonality between the three. OMG's CORBA IDL provides a set of bindings to commonly used languages (e.g., C, C++, Java). IDL interface definitions are compiled using an IDL compiler and deployed into an ORB interface repository. IDL creates stubs which interface client software with the ORB and appear like a local object to the client software. IDL also creates skeletons which are the same sort of things as stubs, but are located at the server. CORBA also allows a Dynamic Invocation Interface (DII) which allows the connection between client and server objects to be established at run time. CORBA 2 only allows objects to have a single interface, but inheritance allows subinterfaces to be specified. COM, on the other hand, allows any number of interfaces. IEEE P1471: Recommended Practice for Architectural Description This recommendation is produced by the IEEE Architecture Planning Group (APG) to set the direction for the next generation of architecturerelated standards and practices for the IEEE. Aimed at "softwareintensive" systems, that is, those of particular complexity, it standardizes conventions for architectural descriptions. It is not intended to replace other standards but to be an organizing framework. It defines a basic framework for an architecture and defines what it means by specific views and viewpoints. A view is a collection of models that represent the whole system from a particular perspective (operations, technology, and so forth). A viewpoint is a template for a view and can be reused across many architecture descriptions. Some of these viewpoints correspond to UML notations, enabling UML to be used to produce a compliant architecture. The recommendation does not proscribe which views to use or which are important. It is up to the architect to decide this based on the requirements. The recommendation takes the view that there is more to architecture than just the structure and organization of components and that many nonphysical and nonfunctional aspects are equally important. Draft version 4 of the recommended practice went to a first IEEE ballot in December 1998. • http://www.pithecanthropus.com/~awg/
Page 325
INCOSE Systems Architecture Working Group The International Council on Systems Engineering (INCOSE) is an international organization formed to "develop, nurture, and enhance a systems engineering approach to multidisciplinary systems development." The Systems Architecture Working Group (SAWG) was formed to advance the architectural practices needed to produce successful systems. Current key activities include: • Baselining of Architecture Process Guide; • System Architecture Evaluation Criteria; • System Element Representation Symbols and Taxonomy Guidebook; • System Architecture Role for Systems with Evolutionary Development; • User Guide for System Architecture Tool Selection. • http://www.incose.org/cmtes/sawg.html Microsoft Repository The Microsoft Repository is a proprietary offering from Microsoft that holds information models, "metadata," to describe a variety of software artifacts. It comprises a set of COM interfaces used to describe information models, and a repository engine, sitting on a SQL database that stores the models. Although a proprietary product, its use is becoming widespread because of its power in allowing information on software items such as objects, components, database schemas, and WWW pages to be exchanged in a standard format between software tools from many vendors. The repository can store many different types of models, but it was Microsoft's decision to support the UML standard which has established it as almost the de facto standard for the underlying repository for the tools of many vendors wishing to use UML. However, the Microsoft Repository does support nonstandard extensions to UML, and it remains to be seen how vendors will use these extensions and how much divergence there is from UML. • http://msdn.microsoft.com/repository Meta Object Facility (MOF) The Meta Object Facility (MOF) provides a set of CORBA interfaces that can be used to define and manipulate a set of metamodels. These metalmodels
Page 326
contain information that describes other information. The MOF uses the UML notation and is a key building block in the construction of CORBAbased distributed development environments. It provides a standard for publishing metainformation and aims to integrate object repositories, object modeling tools, and metadata management in distributed object environments. Also see XMI. • http://www.dstc.edu.au/MetaObjectFacility/MOFAQ.html • http://www.omg.org/news/pr97/umlprimer.html NIST Advanced Technology Program—CBD The U.S. National Institute of Standards and Technology (NIST) operates an Advanced Technology Program which includes a componentbased software focused program. Its aim is to develop and promote technologies for the systematic software component reuse across a broad range of applications. Specifically it is working to: • Enable automated assembly of software components to improve quality, reliability, and reduce time to develop and test new systems; • Enable increased productivity for software users through increased quality of components and dependability of systems; • Enable systematic reuse of components and increased interoperation of software and encourage the market in reusable components; • Address key issues such as revenue collection, commercial security, specification, performance, and so forth. Sterling Software have been awarded a major contract by NIST to look at many of the issues of taking requirements and business process models through to practical CBD implementations. Other parts of the work are looking at a Business Object Component Specification. • http://www.atp.nist.gov/atp/focus/cbs.htm Open Applications Group Integration Specification (OAGIS) The Open Applications Group's Integration Specification (OAGIS) is targeted at integrating the business objects which contain the main business functions that occur within an enterprise. The OAG has created an architecture termed
Page 327
the Business Object Document (BOD) to provide business object integration across heterogeneous environments, including multiple hardware platforms, operating systems, databases, middleware, and so forth. The OAG says that although its architecture is very object oriented, it "does not require classical objectoriented technologies to be implemented." The OAG does not suggest, at least at this stage, that OAGIS will enable plugandplay interoperability. It says that business objects can be more difficult to integrate than other types of objects because of the various data architectures required for differing business objects. Each business object is not discrete, but part of a larger, implied business data model and process model. Their technical architecture is intended to be technology sensitive, but not technology specific. OAGIS defines business processes that are used for communication between OAGcompliant software so that users can be assured that information sent from one business application has been received and successfully processed by the receiving application. The OAG also aims to standardize the core functionality of business processes. Processes unique to a particular implementation are mapped individually, yet will use the OAG's architecture to communicate. A metadata architecture is used for building the BOD based on the OAGIS Specification as well as XML. • http://www.openapplications.org/oagis/loadform.htm Open Applications Group Middleware API Specification (OAMAS) Developed by the OAG, the Middleware API Specification (OAMAS) is a proposal for a common way of connecting business applications to each other at the technical level. It is not built on a specific architecture or middleware but, like OAGIS is "technology sensitive, but not technology specific." • http://www.openapplications.org/oamas/loadform.htm Object Management Architecture (OMA) Object Management Architecture (OMA) is a set of standards produced by the OMG to create a componentbased software marketplace by encouraging the introduction of standardized object software. It is particularly aimed at establishing CORBA as the "middleware that's everywhere." Some of the key standards include:
Page 328
• CORBA; • IIOP (Internet InterORB Protocol); • IDL (Interface Definition Language); • UML; • Meta Object Facility (MOF); • Transaction Processing Standard; • XMI. • http://www.omg.org Object Management Group (OMG) Founded in May 1989 by eight companies, 3Com Corporation, American Airlines, Canon, Inc., Data General, HewlettPackard, Philips Telecommunications N.V., Sun Microsystems, and Unisys Corporation, as a nonprofit corporation, it now includes over 800 members. OMG is based in the United States, with marketing partners in the United Kingdom, Germany, Japan, India, and Australia. It was set up to promote standards and guidelines for developing and managing distributed objects and, more recently, componentbased developments. Its main aim is to develop an Object Management Architecture (OMA) and establish CORBA as the "middleware that's everywhere." OMG standards are open standards that can be implemented by vendors in a variety of ways. The OMG operates a number of task forces and special interest groups to develop standards for domainspecific interfaces including: • Business objects; • Manufacturing; • Electronic commerce; • Telecommunications; • Financial; • Medical. • http://www.omg.org Open Applications Group (OAG) The Open Applications Group (OAG) is a nonprofit consortium of application software vendors, formed in February 1995 to create common standards for the
Page 329
integration of enterprise business applications. Work includes integration between enterprise planning and managing systems, integration to other business applications, and integration to execution systems. Their work is complementary with that of the OMG and the Open Group. Key standards include: • OAGIS—Open Applications Group Integration Specification; • OAMAS—Open Applications Group Middleware API Specification. • http://www.openapplications.org Open Distributed Processing (ODP) Published by the International Organization for Standardization (ISO) and the International Telecommunications Union (ITU) as international standard ISO/IEC 10746 (ITU X.900), 1995, ODP is a reference model that provides a framework for the standardization of open distributed processing (ODP). Based on five viewpoints (enterprise, information, computation, engineering, and technology), the architecture defines terminology, conformance approach, and viewpoint correspondence rules for traceability that support distribution, interworking, and portability. Each viewpoint has its language defining the concepts and rules for applying them. The standard contains specifications of the required characteristics that qualify distributed processing as "open." • http://www.iso.ch:8000/RMODP/part3/0.html Open Group The Open Group is an international consortium of vendors and customers from industry, government, and academia. It defines standards to support corporate IT users. Its IT DialTone initiative infrastructure aims to help organizations to evolve their computing architectures to meet the needs of those participating in a global information infrastructure. One of its key standards is the Open Group Architectural Framework (TOGAF). The Open Group contains what was formerly called the Open Software Foundation (OSF), which was responsible for the introduction of the DCE standard. • http://www.opengroup.org
Page 330
The Open Group Architectural Framework (TOGAF) The Open Group Architectural Framework (TOGAF) is a tool to help define an architecture for an information system. It is based on the Technical Architecture Framework for Information Management (TAFIM), developed by the United States Department of Defense. TOGAF is not an architecture itself, nor does it define an architecture. It provides guidelines for developing specific architectures through a set of services, standards, design concepts, components, and configurations. There are two main parts to the framework: the TOGAF Foundation Architecture and the TOGAF Architecture Development Method (ADM). The Foundation Architecture itself is comprised of the Technical Reference Model (TRM), describing generic platform services, and the Standards Information Base (SIB), a database of standards that can be used to define the particular services and other components of an organizationspecific architecture. The Architecture Development Method shows how to apply the foundation architecture to produce a specific architecture for an organization. TOGAS also has the concept of an Architecture Continuum and a Solutions Continuum. The Architecture Continuum defines the relationships and rules between the various entities that the architecture describes. The Solutions Continuum shows how the Architecture Continuum is implemented within the organization and, in particular, it defines the reusable building blocks. • http://www.opengroup.org/public/arch/ Rational Unified Process The Unified Process is based on Rational's earlier Objectory Process and incorporates the UML standard. It aims to unify best practice from several software development disciplines including business modeling, requirements management, componentbased development, data engineering, configuration, and change management, and to test into a consistent framework covering the full software development life cycle. The process is supported by Rational's suite of development tools, and although not a componentbased development method as such, both the process and tools are aimed at supporting componentbased approaches. • http://www.rational.com/products/rup/index.jtmpl
Page 331
SEI Software Architecture Technology Initiative The SEI has established the Software Architecture Technology Initiative (SATI) to coordinate SEI architecturerelated efforts. SATI is working to build knowledge on domain specific architectures, identify best practice, and provide guidance on selecting Architectural Description Languages. The SEI WWW site contains a valuable information resource on many aspects of software architecture including a comprehensive bibliography. Some of its key activities include: • Product line practice initiative; • Architecture tradeoff analysis initiative (ATA); • COTSbased systems. • http://www.sei.cmu.edu/architecture/ Software Architecture Analysis Method (SAAM) An architectural analysis method developed by the SEI, SAAM uses a set of scenarios to judge how well an architecture performs across a wide range of attributes as opposed to trying to evaluate the architecture against each attribute individually. Such scenarios can represent real world examples of the challenges that software systems have to face. SAAM can be used to assess architectures under development or alternatively to appraise the architectures of commercial software systems. They have now developed this further into a Architecture Tradeoff Analysis (ATA) Method. The method can be used to assess architecture against a range of different quality attributes and to provide analysis of patterns and architectural styles. • http://www.sei.cmu.edu/architecture/scenario_paper/index.html Software Architecture Technology Guide The Software Architecture Technology Guide has been developed by a U.S. Department of Defence (DOD) program called STARS to provide DoD software developers with an overview of software architecture technology aimed at domainspecific reuse. The guide provides information on architectural
Page 332
concepts, representations, and initiatives. It also provides links and guidance to WWW and printed information. • http://wwwast.tdsgn.lmco.com/arch/guide.html Telecommunications Information Networking Architecture (TINA) TINA is a telecommunications domainspecific open software architecture developed by a consortium (TINAC) of over 40 of the world's leading network operators, telecommunications equipment, and computer equipment manufacturers. It works to integrate World Wide Web, multimedia, and current computer technologies with the more traditional and mature telecommunications technologies. TINA states it main goals as: • To make it possible to provide versatile multimedia and information services; • To make it easy to create new services and to manage services and networks; • To create an open telecommunications and information software component marketplace. The common architecture around which new services can be built and deployed is derived from an objectoriented based analysis and is implemented by software components in a distributed processing environment. There is a strong decoupling between the applications themselves and the distributed processing environment (DPE) which, although derived from objectedoriented analysis, is not necessarily objectoriented in implementation. TINA is a fourlayer architecture comprising: • Hardware layer: processors, memory, communication devices; • Software layer: operating systems, communications, and other support software; • DPE layer: provides support for distributed execution of telecommunications applications; • Telecommunications applications layer: implements the capabilities provided by the system. TINA is further divided into four subarchitectures.
Page 333
• Computing architecture: concepts and the CORBAbased DPE; • Service architecture: principles for providing services; • Network architecture: model managing telecommunication networks; • Management architecture: principles for managing software systems. • http://www.tinac.com/ Unified Modeling Language (UML) UML is a language for specifying, visualizing, constructing, and documenting the artifacts of software systems and for business modeling. It represents a collection of best engineering practices that have proven successful in the modeling of large and complex systems. The UML was developed by Rational Software and its partners and it is the successor to the modeling languages found in the Booch, OOSE/Jacobson, OMT and other methods. Many companies are incorporating the UML as a standard into their development process and products, which cover disciplines such as business modeling, requirements management, analysis and design, programming, and testing. A major attraction of UML is that it facilitates the integration of software development tools from a wide range of manufacturers who conform to the standard. By using a common, UMLcompliant repository (e.g., the Microsoft Repository), UML models can be exchanged between the different tools. While primarily a standard for object modeling, it is becoming increasingly important for componentbased development as it offers the possibility of standardizing component descriptions. It is claimed that ''it supports higherlevel development concepts such as collaborations, frameworks, patterns, and components" although it is early days for specific support for components. UML was adopted as an OMG standard in 1997. • http://www.omg.org/news/pr97/umlprimer.html • http://www.rational.com/uml/index.jtmpl Workflow Management Coalition The Workflow Management Coalition was founded in 1993 as an international organization of over 130 workflow vendors, users, analysts, and university/ research groups. Its aim is to promote the use of workflow systems, and it has defined standards for interoperability and connectivity between workflow
Page 334
products, process modeling tools, and software development environments. The coalition has produced a Workflow API (WAPI) specification covering: • Workflow client application interface; • Workflow interoperability specification; • Audit data specification; • Process definition import/export specification; • Application invocation APIs. Future work is aiming to build a common object model. • http://www.aiim.org/wfmc/ WorldWide Institute of Software Architects The Institute of Software Architects, Inc. is a nonprofit corporation founded to accelerate the establishment of the profession of software architecture and to provide information and services to software architects and their clients. They have identified eight key phases for the involvement of the software architect in a project: 1. Predesign; 2. Domain analysis; 3. Schematic design; 4. Design development; 5. Project documents; 6. Staff or contracting; 7. Construction; 8. Postconstruction. • http://www.wwisa.org/ World Wide Web Consortium The World Wide Web Consortium (W3C) was founded in October 1994 to promote standards for the World Wide Web and guide its evolution to achieve
Page 335
its full potential. Hosted by institutions in the United States, Europe, and Japan and funded by member organizations, it has established major standards in the use of the WWW and provides a vast amount of information and sample applications for developers. • http://www.w3.org XMI The XMI Metadata Interchange format has been submitted to the OMG by IBM, Unisys, and other industry leaders as a proposed standard for interchange of object design and programming artifacts. It integrates XML, UML, and MOF and replaces previous proposals for CDIF, XML, and UOL. It aims to enable easy interchange of metadata between tools and metadata repositories (based on OMG MOF) over the Internet. The major initial use of XMI will be to interchange UML models between modeling tools (based on the OMG UML) and repositories (based on OMG MOF and UML). As well as straightforward data interchange between files systems, it can also support information streamed across the Internet from a database or repository. It is intended to encourage collaborative software development of applications by distributed teams. The OMG is planning to release XMI as an open standard in early 1999. • http://www.software.ibm.com/ad/features/xmi.html • http://www.omg.org/archives/orbos/msg00702.html Extensible Markup Language (XML) XML, developed by the XML Working Group (originally known as the SGML Editorial Review Board) formed under the auspices of W3C in 1996, is one of the growing set of Internet interoperability standards. It describes a class of data objects called XML documents and partially describes the behavior of computer programs which process them. It uses the Internet HTTP (Hypertext Transfer Protocol) for data exchange, but has a higher level metadata representation. It is becoming of major importance to component integration because it is a standard for describing and exchanging information between applications and environments that are based on different technologies and have different semantic representations. It is likely to become a standard for interconnecting largescale business applications that currently rely on proprietary stands and opens up at
Page 336
the least the possibility of a plugandplay environment at this scale of component. Also see XMI. • http://www.w3.org/TR/RECxml • http://www.oasisopen.org/cover/
Page 337
Glossary Language is the dress of thought. Samuel Johnson
With computing systems and software pervading more and more areas of personal and business life, its associated vocabulary has grown ever more difficult to follow. In particular, the convergence of computing and telecommunications has led to a very broad and often confusing set of terms that are assumed by many to be well understood. In truth, even some of the more familiar terms can have more than one interpretation. Here we list many of the keywords, abbreviations, buzzwords, and concepts that are freely bandied around these days. Many of those listed below have been introduced in the main text, some have not. In both cases the aim is to unveil the thoughts that lie behind the words by giving some indication of their context and application. 0–9 2tier Twotier client/server applications consist of a single client and server pairing. Often the server is a central database with most of the application code placed in the client.
Page 338
3tier With 3tier client/server systems, the application code is partitioned according to the type of processing performed. This may be, but does not have to be, distributed across three different physical systems, typically comprising a presentation layer, a business application layer, and a data layer. Seen as the future for all business systems, its nature has changed dramatically with the use of the Internet and intranets for delivering the front end of business applications. 3270 An IBM block mode terminal, once ubiquitous as the user interface to the IBM mainframe. Many PCs have 3270 emulation capability so that they can access applications originally built for block mode terminals. 4GL Fourthgeneration language. A term usually applied to languages such as SQL designed to allow databases to be interrogated. A Abstract Containing less information than reality. Abstraction A representation of something that contains less information than the something. For example, a data abstraction provides information about a reference in the outside world without indicating how the data is represented in the computer. ActiveX Microsoft technology for embedding information objects and application components within one another. For example, an ActiveX button can be embedded in an HTML page that is displayed in a browser window. Address A common term used in computers, telecommunications, and data communication to designate the destination or origination of data or terminal equipment in the transmission of information. Types of address include hardware addresses (e.g., 0321.6B11.3310, for an Ethernet card), logical address (e.g., 132.146.29.11, a TCP/IP address for a workstation), or a personal address (mnorris@ iee.org, to reach an individual).
Page 339
Agent In systems and network management, a term usually applied to a server specialized from performing management operations on the target system or network node. Agent A more recent use of the word (sometimes prefixed with the word intelligent) used to describe a semiautonomous program that roams through a computing network collecting and processing data on behalf of its originator, sending back results as necessary. Algorithm A group of defined rules or processes for solving a problem. This might be a mathematical procedure enabling a problem to be solved in a definitive number of steps. A precise set of instructions for carrying out some computation (e.g., the algorithm for calculating an employee's takehome pay). ANSA Advanced Networked Systems Architecture, a research group established in Cambridge, U.K., in 1984 that has had a major influence on architectures for distributed systems. API Application programming interface. Software designed to make a computer's facilities accessible to an application program. It is the definition of the facilities offered to a programmer. All operating systems have APIs; in a networking environment, it is essential that various machines' APIs are compatible, otherwise programs would be exclusive to the machines on which they reside. APPC Advanced ProgramtoProgram Communication. An application program interface developed by IBM. Its original function was in mainframe environments enabling different programs on different machines to communicate. As the name suggests, the two programs talk to each other as equals using APPC as an interface designed to ensure that different machines on the network talk to each other. Applet A small(ish) software component of little use on its own but which may be plugged in to a form part of a larger application. Used with World Wide
Page 340
Web applications, Java, and mobile code environments to provide downloadable components. Application A collection of software functions (and possibly components) recognized as delivering most of what is needed to support a particular domain. Often available as commercial products and reused as Black Box components. Application generators Highlevel languages that allow rapid generation of executable code, sometimes referred to as 4GLs, Focus being a typical example. Application program A series of computer instructions or a program which when executed performs a task directly associated with an application such as spreadsheets, word processing, database management. Applications software The software used to carry out the application's task. Architecture A highlevel conceptual representation showing how systems and components in a domain relate to one another and may be assembled into more complex systems. Any given domain may have a number of different architectures representing different viewpoints. When applied to computer and communication systems, it denotes the logical structure or organization of the system and defines its functions, interfaces, data, and procedures. In practice, architecture is not one thing but a set of views used to control or understand complex systems. A loose definition is that it is a set of components and some rules for assembling them. Architecture style A set of components, topological layouts, set of interaction mechanisms, environments, and possibly technology (e.g., CORBA). Automata A machine (such as a computer) that operates to a fixed set of instructions. Typically, automata do something, and then wait for an external stimulus before doing something else. The way in which automata interact is the subject of a welldeveloped branch of mathematics known as automata theory. Automation Systems that can operate with little or no human intervention. It is easiest to automate simple me
Page 341
chanical processes, hardest to automate those tasks needing common sense, creative ability, judgment, or initiative in unprecedented situations. B BGP Border Gateway Protocol. This is the protocol used in TCP/IP networks for routing between different domains. Binding The process whereby a procedure call is linked with a procedure address, or a client is linked to a server. In traditional programming languages, procedure calls are assigned an address when the program is compiled and linked. This is static binding. With late, or dynamic, binding, the communicating parties are matched at the time the program is executed. BOCA Business Object Component Architecture. A framework developed in the Object Management Group for componentbased engineering. Bug An error in program or a fault in equipment. Origin of the term is not universally agreed upon but popular belief is that the first use in a computing context can be attributed to ViceAdmiral Grace Murray Hopper of the U.S. Navy. In the early days of valvebased electronic computing, she found that an error was caused by a genuine bug—a moth fluttering around inside the machine. C C A widely used programming language originally developed by Brian Kernighan and Dennis Ritchie at AT&T Bell Laboratories. C became most widely known as the language in which the UNIX operating system was written. C++ A programming language based upon C but adding many enhancements particularly for objectoriented programming. It has probably now surpassed C in popularity and provides the basis for
Page 342
Java, the subset of C++ now widely used for building mobile code. Caching This is a process by which data requested by the operating system of a computer is retrieved from RAM instead of from a hard disk (or some other mass storage media). Caching algorithms will check if the requested data is in its cache (or RAM). The significance of this is that RAM access is an order of magnitude faster than today's mass storage devices, so the more accesses to the cache, the faster overall system performance will be. CASE ComputerAided Software. Engineering tools which support the software engineer in the design process and take away much of the drudgery of writing code. CBD Componentbased development. An approach to software engineering that encourages the creation of reusable components and the production of new software systems through the assembly of preexisting components. CERN The European laboratories for particle physics. Home of the HTML and HTTP concepts that underpin the popular Mosaic and Netscape browsers. CGI Common Gateway Interface. A protocol associated with file servers for the World Wide Web. CGI is the logical interface between an HTTP server and an application server. It allows information (e.g., records taken from a database) to be presented to a user in a standard format. Client Usually synonymous with a PC. A client is an entity—for example a program, process, or person—that is participating in an interaction with another entity and is taking the role of requesting (and receiving) the required service. Client/server The division of an application into (at least) two parts, where one acts as the client (by requesting a service) and the other acts as the server (by providing the service). The rationale behind client/ server computing is to exploit the local desktop processing power, leaving the server to govern the centrally held information.
Page 343
CMOS Complimentary metal oxide semiconductor. An electronic device technology. Code A computer program expressed in the machine language of the computer on which it will be executed, that is, the executable form of a program. More generally, a program expressed in representation that requires only trivial changes to make it ready for execution. COM Common Object Model. The expansion of the Component Object Model (see next entry) to add support for distribution. COM was jointly developed by Microsoft and Digital (see also DCOM). COM Component Object Model. The nondistributed framework underlying Microsoft's OLE object technology. Components Selfcontained, recognizable entities that perform wellunderstood functions and can be assembled via known interfaces with other components to build something more complex. Components are often reused and can be replaced with an identical functioning component without affecting overall operation. Computer A piece of hardware that can store and execute instructions (i.e., interpret them and cause some action to occur). Concurrency The case when two or more systems cooperate on a task in parallel. Concurrent operation can be efficient but is prone to undesirable states (such as deadlock, where all parties are waiting, or livelock, where there is a repeated sequence of activity with no exit defined). Configuration A collection of items that bear a particular relation to each other (e.g., the data configuration of a system in which classes of data and their relationships are defined). Cookie A token of agreement between cooperating programs that is used to keep track of a transaction. At a more concrete level, a cookie is a fragment of code that holds some information about your local state—your phone number or home page reference, for instance. You probably have cookies
Page 344
that you do not know about. The Netscape and Explorer browsers both support them, with the cookie being presented to the server to control your dialogue. CORBA Common Object Request Broker Architecture. Strictly, the name of a framework specification produced by the Object Management Group that describes the main components of a distributed object environment. Informally used to denote any of a number of related specifications produced by the OMG. COTS Components off the shelf. The idea that you can construct a networked computing system by selecting ready made piece parts from a catalog—in much the same way that you would put together a designer bicycle from the forks, wheels, handlebars, and so forth on offer. Together with plugandplay operation, COTS holds the promise of fast, customized technical solutions. D Daemon A program that lies dormant, waking up at regular intervals or waiting for some predetermined condition to occur before performing its action. Supposedly an acronym rationalized from disk and execution monitor. Data Usually taken to mean the same as information. Data is the raw input which, once interpreted and processed, can be used to provide information. For instance, a spreadsheet contains data and the fact that it shows your company to be in profit is the information. Database A collection of interrelated data stored together with controlled redundancy to support one or more applications. On a network, data files are organized so that users can access a pool of relevant information. Database server The machine that controls access to the database using client/server architecture. The server part of the program is responsible for updating records,
Page 345
ensuring that multiple access is available to authorized users, protecting the data, and communicating with other servers holding relevant data. DBMS Database management system. A set of software used to set up and maintain a database that will allow users to call up the records they require. In many cases, a DBMS will also offer reportand applicationgenerating facilities. DCE The distributed computing environment. A set of definitions and software components for distributed computing developed by the Open Software Foundation, an industry led consortium. It is primarily a remote procedure call with a set of integrated services, such as security, time, and directory. DCOM Distributed Component Object Model. Microsoft's upgrade to its initial version of COM for a distributed environment. Deadlock A condition where two or more processes are waiting for one of the others to do something. In the meantime, nothing happens. An undesirable condition that needs to be guarded against, especially in the design of databases. Design (n.) A plan for a technical artifact, or (more verbosely) a complete set of plans showing exactly how a system will be manufactured. It is that set of information that is necessary and sufficient to ensure that the correct thing is built to the specification. Design (v.) To create a design; to plan and structure a technical artifact. In software engineering, the phase that is often preceded by implementation. Design Process The process of converting a requirements specification to a set of complete manufacturing plans within the context of a chosen architecture and production environment. Directory A directory provides a means of translating information from one form to another (e.g., someone's name into their telephone number). In a distributed system directory services are a key component. They often perform much the same function
Page 346
as a telephone directory, translating from a symbolic name to a network address. A wellknown example is DNS, which translates Internet names (
[email protected]) and dot, or IP, addresses (142.119.42.17). Distributed computing A move away from having large centralized computers such as minicomputers and mainframes, it brings processing power to the desktop. Often used as a synonym for distributed processing. Distributed database A database that allows users to gain access to records, as though they were held locally, through a database server on each of the machines holding part of the database. Every database server needs to be able to communicate with all the others as well as being accessible to multiple users. DLL Dynamic link library. A set of software utilities that are bound with source code when it is compiled. DNS Domain Name Server. The method used to convert Internet textual addresses—for example, gatekeeper.dec.com—to their corresponding numerical addresses. Domain In the broad context, this is a wellunderstood area of common interest within which common technical terms are understood and common components can be practically applied. When applied to networked communication systems, a domain is part of a naming hierarchy. A domain name consists, for example, of a sequence of names or other words separated by dots. DPE Distributed Processing Environment. One of the various standards in this area. E EJB Enterprise JavaBeans. Components written in the Java programming language intended to be run within a serverbased environment (e.g., a WWWserver or database). EJBs run within a container on the server and appear as objects to the outside world. Clients locate the EJB via the Java Naming and Directory Interface (JNDI).
Page 347
Email or email Common shorthand for electronic mail. Enterprise A term (usually used a descriptor for network or computing) to denote the resources deployed to suit the operating needs of a particular organizations. ESIOP EnvironmentSpecific InterORB Protocol. A protocol defined by the OMG for communication between ORBs. Ethernet A local area network (LAN) characterized by 10 Mbps transmission using the CSMA/CD (collision sense multiple access with collision detection) access method. Ethernet was originally developed by and is a registered trademark of Xerox Corporation. F FDDI Fiber distributed data interface. An American National Standards Institute (ANSI) LAN standard. It is intended to carry data between computers at speeds up to 100 Mbps via fiberoptic links. It uses a counterrotating token ring topology and is compatible with the first, physical, level of the ISO sevenlayer model. Federation A union of otherwise largely independent systems to support some common purpose. Federated systems share some basic agreements or protocols to enable them to work together, but are operated and managed autonomously. File server A machine in a LAN dedicated to providing file and data storage to other machines in the network. Framelets Smallscale framework that implements a domainspecific requirement or pattern. Framework A partial product assembly that can be completed, extended, and customized to meet a customer's exact requirements but is based on a common core. Frontend The presentation element of client/server system. Usually a desktop PC running a graphical user interface.
Page 348
FTAM File Transfer, Access, and Manipulation. A protocol entity forming part of the application layer enabling users to manage and access a distributed file system. FTP File Transfer Protocol. The Internet standard (as defined in the RFC series) highlevel protocol for transferring files from one computer to another. A widely used de facto standard. Anonymous FTP is a common way of allowing limited access to publicly available files via an anonymous login. G Gateway Hardware and software that connect incompatible networks, enabling data to be passed from one network to another. The gateway performs the necessary protocol conversions. GIOP General InterORB Protocol. A protocol defined by the OMG for communication between ORBs. GUI Graphical user interface. An interface that enables a user to interact with a computer using graphical images and a pointing device rather than a characterbased display and keyboard. Such interfaces are also known as WIMP interfaces—WIMP standing for Windows, Icons, Menus, and Pointers. The most common pointing device is that electronic rodent, the mouse. H Hardware The physical equipment in a computer system. It is usually contrasted with software. Heritage system A euphemism for legacy system, cherished system, and Millstone. Heterogeneous Of mixed or different type. Homogeneous Of the same type. Hostage data Data which is generally useful but held by a system which makes external access to the data difficult or expensive. HTML Hypertext Markup Language. HTML is the language used to describe the formatting in WWW
Page 349
documents. It is an SGML document type definition. It is described in the RFC series of standards. HTTP Hypertext Transfer Protocol. The basic protocol underlying the World Wide Web (see WWW). It is a simple, stateless requestresponse protocol. Its format and use are rather similar to SMTP. HTTP is defined as one of the Internet's RFC series, generated by the IAB. I IAB Internet Architecture Board. The influential panel that guides the technical standards adopted over the Internet. Responsible for the widely accepted TCP/IP family of protocols. More recently, the IAB have accepted SNMP as their approved network management protocol. Idiom Pattern for implementing a solution in a specific language with a certain style. Effectively an implementation pattern. IDL Interface Definition Language. A notation that allows programs to be written for distribution. An IDL compiler generates stubs that provide a uniform interface to remote resources. IDL is used in conjunction with remote procedure calls. IETF Internet Engineering Task Force. A very active body parented on the IAB, responsible for many of the de facto standards used on the Internet. IIOP Internet InterORB Protocol. A protocol defined by the OMG for communication between ORBs. Inheritance In object orientation, this is a relationship between classes that a subclass inherits from its superclass. Inheritance is, along with encapsulation and polymorphism, a basic property of all objects. Interface The boundary between two things: typically two programs, two pieces of hardware, a computer and its user, or a project manager and the customer. The channel between the two entities is a conduit through which information passes. Information can consist of data or commands. An
Page 350
API defines the commands and data that when sent through the channel enable a software application to be controlled. Internet A concatenation of many individual TCP/IP sites into one single logical network all sharing a common addressing scheme and naming convention. Interoperate The ability of computers from different vendors to work together using a common set of protocols. Suns, IBMs, Macs, PCs, and so forth all work together allowing each to communicate with and use the resources of the other. Intranet A closed internet system running within an organization connected to the Internet, and protected from unauthorized access, via a firewall. Increasingly, business are using intranets to provide employees with desktop access to key business systems. ISO International Organization for Standardization. J Java A programming language and environment for developing mobile code applications. Java is a subset of the C++ language and is widely used to provide mobile code application for use over the Internet. JavaBean Components written in the Java programming language, originally intended to be delivered over the Internet and run on a desktop client PC. See also EJB. JNDI Java Naming and Directory Interface. See EJB. JVM Java Virtual Machine. The ubiquitous engine for running Java code, in essence a software CPU. The idea is that any computer can equip itself with a JVM, a small program that allows Java applets (which are widely available over the Internet) to be downloaded and used. K Kermit A communications protocol developed to allow files to be transferred between otherwise incom
Page 351
patible computers. Generally regarded as a backstop: if all else fails Kermit will get the files from A to B. Kernel The level of an operating system that contains the systemlevel commands—the functions hidden from the user. This program is always running on a processor. L LAN Local area network. A data communications network used to interconnect data terminal equipment distributed over a limited area. Language An agreedupon set of symbols, with the rules for combining them and meanings attached to them that is used to express something (e.g., the Pascal programming language, a jobcontrol language for an operating system, and a graphical language for building models of a proposed piece of software). Legacy system A system which has been developed to satisfy a specific requirement and is, usually, difficult to substantially reconfigure without major reengineering. Life cycle A defined set of stages through which a development passes over time, from requirements analysis to maintenance. Common examples are the waterfall (for sequential, staged developments) and the spiral (for iterative, incremental developments). Life cycles do not map to reality too closely but do provide some basis for measurement and, hence, control. M Mainframe A computer (usually a large one and often manufactured by IBM) that provides a wide range of applications to connected terminals. Messaging Exchanging messages. Often but not limited to the context of electronic mail. Message passing Communication through the exchange of messages. Although not a rigorously used term,
Page 352
message passing systems usually have the connotation of realtime immediate message exchange. Message queuing A message passing technology augmented by a storeandforward capability. Method A way of doing something—a defined approach to achieving the various phases of the life cycle. Methods are usually regarded as functionally similar to tools (e.g., a specific tool will support a particular method). Methodology Strictly, the science or study of methods. More frequently used as a more important sounding synonym for method, process, or technique. Middleware Software that mediates between an application program and an underlying set of utilities such as databases, networks, or servers. It manages the interaction between disparate applications across the heterogeneous platform, masking diversity from the programmer. Object Request Brokers such as CORBA are an example of middleware, as they manage communication between objects, irrespective of their location. Mobile code Programs capable of being run on many different systems (e.g., Java can run on any machine equipped with a Java Virtual Machine). Mobile code is written once, used anywhere, and gets around the need for porting work to be done every time that the program encounters a different type of computer. Model An abstraction of reality that bears enough resemblance to the object of the model that we can answer some questions about the object by consulting the model. Modeling Simulation of a system by manipulating a number of interactive variables; can answer ''what if?" questions to predict the behavior of the modeled system. A model of a system or subsystem is often called a prototype. Modularization The splitting up of a software system into a number of sections (modules) to ease design, coding, and so forth. Only works if the interfaces between the modules are clearly and accurately specified.
Page 353
MOM Messageoriented middleware. A term used to describe commercial message passing and message queuing products. MP Abbreviation used for both multiprocessing and message passing. Multiplexing The sharing of common transmission medium for the simultaneous transmission of a number of independent information signals. Multiprocessing Running multiple processes or tasks simultaneously. This is possible when a machine has more than one processor or processing is shared among a network of uniprocessor machines. See also multitasking and multithreading. Multiprocessor A single computer having more than one processor and capable of executing more than one program at once. Multitasking Performing (or seeming to perform) more than one task at a time. Multitasking operating systems such as Windows, OS/2, or UNIX give the illusion to a user of running more than one program at once on a machine with a single processor. This is done by "timeslicing," dividing the processor into a small chunks which are allocated in turn to competing tasks. Multithreading Running multiple threads of execution within a single process. This is a lower level of granularity than multiprocessing or multitasking. Threads within a process share access to the process memory and other resources. Threads may be "timesliced" on a uniprocessor system or executed in parallel on a multiprocessor system. N Network A general term used to describe the interconnection of computers and their peripheral devices by communications channels, for example, public switched telephone network (PSTN), packet switched data network (PSDN), local area network (LAN), and wide area network (WAN).
Page 354
Network interface The circuitry that connects a node to the network, usually in the form of a card fitted into one of the expansion slots on the back of the machine. It works with the network software and operating system to transmit and receive messages over the network to other connected devices. Network operating system A network operating system (NOS) extends some of the facilities of a local operating system across a LAN. It commonly provides facilities such as access to shared file storage and printers. Examples include NetWare and LAN Manager. Network topology The geometry of the network relating to the way the nodes are interconnected. NFS Network file system. A method, developed by Sun Microsystems, that allows computers to share files across a network as if they were local. Nonproprietary Software and hardware that is not bound to one manufacturer's platform. Equipment that is designed to the specification that can accommodate other companies' products. The advantage of nonproprietary equipment is that a user has more freedom of choice and a larger scope. The disadvantage is that when it does not work, you may be on your own. O Object An abstract, encapsulated entity which provides a welldefined service via a welldefined interface. An object belongs to a particular class which defines its type and properties. One object can inherit properties from another, and objects can evolve to do specific tasks. Object orientation A philosophy that breaks a problem into a number of cooperating objects. Objectoriented design is becoming increasingly popular in both software engineering and related domains, for example in the specification of componentbased systems.
Page 355
Object program The translated versions of a program that has been assembled or compiled. Nothing to do with objectorientation. OLE Object linking and embedding. Microsoft's proprietary object component technology. Often compared to CORBA. OMA Object Management Architecture (OMA). OMG Object Management Group. An industry consortium responsible for the CORBA specifications. Open system A much abused term. The usual meaning of an open system is one built to conform to published, standard specifications or interfaces, for example, POSIX. Openness is rather like beauty in that it is often in the eye of the beholder. Operating system Software such as VME, MVS, OS/2, Windows, VMS, MSDOS, or UNIX that manages the computer's hardware and software. Unless it intentionally hands over to another program, an operating system runs programs and controls system resources and peripheral devices. OSI Open Systems Interconnection. A model to support the interworking of telecommunications systems. The ISO Reference Model consisting of seven protocol layers. These are the application, presentation, session, transport, network, link, and physical layers. The concept of the protocols is to provide manufacturers and suppliers of communications equipment with a standard that will provide reliable communications across a broad range of equipment types. Also more broadly applied to a range of related computing and network standards. OSS Operational support systems. These are all of the behindthescenes systems that allow a service to operate reliably and profitably. The usual spread of OSS includes billing, fault and problem management, provisioning, network and service management, customer handling, and management information.
Page 356
Overloading A term used in objectoriented software development to describe the use of one identifier that serves more than one purpose. P Packet A unit of data sent across a network. The basis for all of the modern data communication networks, a common format for communications between computers. Parallel processing Performing more than one process in parallel. Usually associated with computerintensive tasks which can be split up into a large number of small chunks which can be processed independently on an array of relatively inexpensive machines. Many engineering and scientific problems can be solved in this way. It is also frequently used in highquality computer graphics. Parameter A variable whose value may change the operation but not the structure of some activity (e.g., an important parameter in the productivity of a program is the language used). Also commonly used to describe the inputs to and outputs from functions in programming languages. In this context they may also be known as "arguments." Pattern An approach to solving a given type of problem by using analogy/comparison with established or existing solutions of a similar type (and progressing through the application of templates or reference designs). Peer to peer Communications between two devices on an equal footing, as opposed to host/terminal or master/slave. In peertopeer communications both machines have and use processing power. Pipe A feature of many operating systems, a pipe is a method used by processes to communicate with each other. When a program sends data to a pipe, it is transmitted directly to the other process without ever being written onto a file. Polling The process of interrogating terminals in a multipoint network in turn in a prearranged sequence
Page 357
by controlling the computer to determine whether the terminals are ready to transmit or receive. If any problems are detected with the normal sequence of operations, the polling sequence is temporarily interrupted while the terminal transmits or receives. Polymorphism A term used in objectoriented software development to describe an object that can be used in different ways, according to context. Port (n.) A device which acts as an input/output connection. Serial communication ports or parallel printer ports are examples. (v.) To transport software from one system to another different system and make the necessary changes so that the software runs correctly, taking account of the specific calls and structures used on that system. POSIX Portable Operating System Interfaces. A set of international standards defining APIs based upon the UNIX operating system. Process The usual term for a program currently being run by an operating system. A process is assigned resources such as memory and processor time by the operating system. The term task is sometimes used as a synonym. See also multiprocessing, multitasking, and multithreading. Processor That part of a computer capable of executing instructions. More generally, any active agent capable of carrying out a set of instructions (e.g., a transaction processor for modifying a database). Production engineering Part of the design process that ensures that a design can be manufactured to specification with the tools and techniques available. Product line A series of designs showing how a family of strongly related products are to be built from common components. If the product line is the set of related products that address a common market segment then a product family is the set of related products based on common components. Proprietary Any item of technology that is designed to work with only one manufacturer's equipment. The
Page 358
opposite of the principle behind Open Systems Interconnection (OSI). Protocol A set of rules and procedures that are used to formulate standards for information transfer between devices. Protocols can be lowlevel (e.g., the order in which bits and bytes are sent across a wire) or highlevel (e.g., the way in which two programs transfer a file over the Internet). Prototype A scaleddown version of something, built before the complete item is built, in order to assess the feasibility or utility of the full version. Q Quality assessment A systematic and independent examination to determine whether quality activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives. Quality system The organizational structure, responsibilities, procedures, processes, and resources for implementing quality management. Quality system standards A quality system standard is a document specifying the elements of a quality system. The ISO 9001 standard (which is generally used to control software development) is a widely known and used quality standard. R Remote procedure call An RPC provides a distributed programming mechanism where an apparently local procedure call in a client causes an implementation of the procedure provided by a server to be invoked. Repository A data store holding (or pointing to) software and systems entities that designers and developers could reuse in the process of delivering new systems solutions. The repository provides services to manage the creation, use, versions, maintenance, translation, and viewing of these entities.
Page 359
Reuse The process of creating software systems using existing artifacts rather than starting completely from scratch. Code, components, designs, architectures, operating systems, patterns, and so forth are all examples of artifacts that can be reused. Also methods and techniques to enhance the reusability of software. generative reuse Reuse of the process (method, design, tools, and so forth) used to create a component. local reuse Reuse of components within a product, product line, or by a small team in several products. People working closely together will have implicit knowledge of how to use the component. domain reuse Systematic reuse of wellunderstood common components across a specific area of interest, often in specific environments. People working in the domain will have implicit knowledge of how to use the component. Sometimes called vertical reuse. global reuse Widespread reuse across domains, organizations, environments, and geography. All the knowledge needed to use the component has to be made explicit. Sometimes called horizontal reuse. RFC Request for Comment. A longestablished series of Internet "standards" documents widely followed by commercial software developers. As well as defining common Internet protocols, RFCs often provide the implementation detail to supplement the more general guidance of ISO and other formal standards. The main vehicle for the publication of Internet standards, such as SNMP. S Screen scraping A method of accessing a server where the client presents itself as being a direct interface to a human user. The client "reads" information from the "screen" presented by the server and "sends'' information as "keystrokes" from the pretend user. Server An object which is participating in an interaction with another object, and is taking the role of providing the required service.
Page 360
Service A piece of work done; a facility provided. In the context of components, a service is the process and interface through which the predefined function of a component is accessed. Typically services are used to "wrap" legacy systems and enable imbedded legacy functions to be used as if they were standalone components. Session The connection of two nodes on a network for the exchange of data—any live link between any two data devices. SGML Standard Graphical Markup Language. An international standard encoding scheme for linked textual information. HTML is a subset. Signaling The passing of information and instructions from one point to another for the setting up or supervision of a telephone call or message transmission. Silver bullet The ultimate cureall for all of the ills of the software industry—the magic panacea. Over the years there have been many silver bullets that promised to remove all of the pain: objectorientation and Java are probably the most recent saviors. In reality, there is no magic to make inherent complexity easily controllable, as explained in Fred Brooks' seminal paper "No Silver Bullet." Software glue Common term for the software that binds together applications, operating systems, and other elements. The closest practical instantiation of software glue is probably the middleware developed for distributed systems (e.g., DCE, CORBA) under the auspices of the OMG and others. SOM This is IBM's objectoriented development environment that allows users to put together class libraries and programs. Associated with SOM is an OMG CORBA conformant object request broker (known as DSOM) for building distributed applications. See also COM, the Microsoft equivalent of SOM. SQL Structured Query Language. A widely used means of accessing the information held in a database. SQL enables a user to build reports the data held.
Page 361
Stateful When applied to a server, this term implies that the server maintains knowledge about and context for its clients between requests. Stateless When applied to a server, this term implies that a server maintains no knowledge about its clients between requests—each request is handled in isolation from all preceding and following requests. Stovepipe An independent set of systems supporting a new requirement represented by a vertical stack of functionality in a layered architecture with few interconnections with other systems in the architecture. Contrasts with using existing systems and components in the architecture. Synchronization The actions of maintaining the correct timing sequences for the operation of a system. Synchronous transmission Transmission between terminals where data is normally transmitted in blocks of binary digit streams, and transmitter and receiver clocks are maintained in synchronism. Syntax The set of rules for combining the elements of a language (e.g., words) into permitted constructions (e.g., phrases and sentences). The set of rules does not define meaning (this is covered by semantics), nor does it depend on the use made of the final construction. System A collection of independently useful objects which happen to have been developed at the same time. A collection of elements that work together, forming a coherent whole (e.g., a computer system consisting of processors, printers, disks, and so forth). System assembly Another name for system integration. System design The process of establishing the overall (logical and physical) architecture of a system and then showing how real components are used to realize it. System integration The process of bringing together all of the components that form a system with the aim of showing that the assembly of parts operates as expected. This usually includes the construction of
Page 362
components to carry out missing or forgotten functions and the glue to interconnect all of the components. T Topology A description of the shape of a network, for example, star, bus, and ring. It can also be a template or pattern for the possible logical connections onto a network. Trading Matching requests for services to appropriate suppliers of those services based on some constraints. Transaction A single, atomic unit of processing. Usually a single, small "parcel" of work which should either succeed entirely or fail entirely. Transaction processing Originally a term that mainly applied to technology concerned with controlling the rate of enquiries to a database. Specialist software—known as a TP monitor— allowed potential bottlenecks to be managed. Transparency Distribution transparencies provide the ability for some of the distributed aspects of a system to be hidden from users. For example, location transparency may allow a user to access remote resources in exactly the same way as local ones. TTL TransitorTransitor Logic. An electronic device technology, but more specifically a family of electronic logic components produced by Texas Instruments. U, V UML Unified Modeling Language. The objectoriented notation adopted by the OMG and devised by Booch, Rumbaugh, and Jacobson. UML unifies several of the main (and formerly disparate flavors) of objectoriented notation. UNO Universal Network Objects. A standard defined by the OMG for communication between ORBs. URL Uniform Resource Locator. Essentially, this is the form of address for reaching pages on the World
Page 363
Wide Web. A typical URL takes the form http://www.interesting.com/. Value chain A process description of a business' key activities showing where business functions add value to the products or services the business provides. Vendor independent Hardware or software that will work with hardware and software manufactured by different vendors—the opposite of proprietary. Virtual device A module allocated to an application by an operating system or network operating system, instead of a real or physical one. The user can then use a computer facility (keyboard, memory, disk, port) as though it was really present. In fact, only the operating system has access to the real device. Virtual machine A software program that provides an implementation of an abstract processing environment. It supplies an execution engine for other programs compiled into byte code which it interprets. Virus A program, passed over networks, that has potentially destructive effects once it arrives. Packages such as VirusGuard are in common use to prevent infection from hostile visitors. VLSI Very large scale integration. A general name for the technique of fabricating a large number of electronic devices onto a silicon wafer which led to the computer chips so ubiquitous in modern life. W W3, or WWW Common abbreviations for World Wide Web. Window A flow control mechanism the size of which determines the number of packets that can be sent before an acknowledgment of receipt is needed and before more can be transmitted. Windows A way of displaying information on a screen so that users can do the equivalent of looking at several pieces of paper at once. Each window can be manipulated for closer examination or amendment. This technique allows the user to look at two files at once or even to run more than one
Page 364
program simultaneously. Also the generic name (though not a registered trademark) for Microsoft's family of operating systems—Windows 97®, Windows NT®. X, Y, Z XML Extensible Markup Language. Developed by the Worldwide Web Consortium for describing and exchanging information between applications and environments that are based on different technologies and have different semantic representations. It uses the Internet HTTP (Hypertext Transport Protocol) for data exchange, but has a higher level metadata representation. X/Open An industry standards consortium that develops detailed system specifications drawing on available standards. It has produced standards for a number of distributed computing technologies. X/Open also licenses the UNIX trademark and thereby brings focus to its various flavors (e.g., HPUX, AIX from IBM, Solaris from SUN, and so forth). Yahoo Yet Another Hierarchically Organized Offering. One of the many search utilities that can be used to trawl and crawl the information held on the World Wide Web. Zip A compression program, from PKWare, to reduce files to be sent over a network to a more reasonable size. This was originally popularized on MSDOS but has now spread to other operating systems.
Page 365
About the Authors Mark Norris is an independent consultant with over 20 years experience in software development, computer networks, and telecommunications systems. He has managed dozens of projects to completion, from the small to the multimillion dollar, multisite, and he has worked for periods in Australia and Japan. He has published widely over the last ten years with a number of books on software engineering, computing, project and technology management, telecommunications, and network technologies. He lectures on network and computing issues, has contributed to references such as Encarta, and is a visiting professor at the University of Ulster and a fellow of the IEE. Mark plays a mean game of squash but tends not to mix this with other forms of interfacing. Mark can be found at
[email protected]. Rob Davis graduated with a first class honors in electrical and electronic engineering and held a number of engineering design posts in industry before joining British Telecommunications pie in 1984 to work on optical fiber digital line systems. In 1988, he joined BT Laboratories to manage computeraided engineering for BT Network Systems. More recently, Rob led a team in the Intelligent Systems Unit looking at applying knowledgebased systems technology to decision support tasks, including the use of expert systems and casebased reasoning for help desk applications. This work was later applied to BT's customer handling systems. In his current work, he leads the ComponentBased Systems Engineering (CoBaSE) team at BT Laboratories, looking at how reusable components can be systematically used to rapidly assemble new systems. Other interests include philosophy of the mind, explanations of consciousness, traveling (in comfort) to interesting places, and learning to fly a helicopter. Alan Pengelly has over 18 years of experience in the software and systems industry. He joined BT in 1988 after a number of years in the defense industry. He also works as a parttime consultant. He has a B.A. honors degree in mathematics and a Ph.D. in computer science for work on interface theory and design. He has published over 35 technical and managerial papers and regularly
Page 366
lectures on software project management and system design both in the United Kingdom and overseas. His particular area of interest is the relationship between information systems and organizational structure. Alan is a keen aviator and cyclist, though more often than not these days he can be found at cart racing circuits around the United Kingdom supporting his young son Christopher! Alan can be contacted at
[email protected].
Page 367
Index 2tier client/server applications, 337 3tier architecture, 59–60, 198 client/server, 62 defined, 198, 338 illustrated, 198 as technical issue, 269 See also Architectures 4GL, 338 A Abstract design, 227–29, 338 Acceptance stage defined, 83 inputs, outputs, checks, actions for, 84 See also Integration process ActiveX, 338 Adapting components, 232–33 Adaptive reuse, 44, 233 ADAPTOR pattern language, 258 Addresses, 338 Advanced PeertoPeer Networking (APPN), 188 Agents, 339 Algorithms constructive, 301–7 defined, 339 discarding, 23, 301–7 Antipatterns, 123 Applets, 339–40 Application programming interfaces (APIs), 193–95 builtin, 17 business (BAPIs), 135, 318 defined, 16, 339 development, 194 elements, 16–17, 193 example illustration, 17 file transfer, 194 increasing use of, 17 interest in, 195 invoking, 193 message delivery, 194 RPC, 193, 194 SQL, 193, 194 types of, 193–94 Workflow (WAPI), 334 See also Interfaces Applications architecture, 110 defined, 340 network, 190 software, 340 Architects construction phase, 146 defined, 75 design development phase, 145 domain analysis phase, 143 postconstruction phase, 146
Page 368
predesign phase, 143 project documents phase, 146 role of, 143–46 schematic design phase, 143–45 staff/contracting phase, 146 See also Architectures Architectural description languages (ADLs), 119 Architectural layers, 110–12 application, 110 business, 110 illustrated, 111 technology, 110 views and, 111, 112 Architectural mismatch, 111, 127 Architectural models, 110–19 Butler, 113–14 layer, 110–12 layers of generality, 112–13 Rational 4+1 View, 114–17 structure, 117–18 TOGAF, 118–19, 329 Architectural structures, 117–18 defined, 17 structure list, 117–18 See also Architectural models Architectural styles, 108–9 defined, 108, 340 elements, 108 Architectural views, 109–10 illustrated, 110 layer model, 111, 112 list of, 109 Architectures, 97–148, 152 3tier, 59–60, 62, 198, 338 antipatterns, 123 application, 110 business, 110, 131–38 choosing, 139–40, 215–17, 222 component, 43 defined, 38, 105–6, 340 defining, 92–93 domainspecific, 119 evaluating, 140–41 frameworks, 125–28 future direction of, 146–47 middleware, 191 need for, 142–43 network, 167, 187–90 organization use of, 145 OSS, 49 patterns, 120–23 pattern systems, 123–25 product lines/families, 128–30 role, 139 stovepipe, 130–31 structures in, 120–31 successful use of, 141 technology, 110 using, 141–43 See also Components; Integration; Interfaces Architecture Tradeoff Analysis (ATA) method, 141, 144 defined, 317 illustrated, 144 See also Software Engineering Institute (SEI) Assembly, 43 car components, 30–32, 98–99 CBD systems, 237–42 component, 43, 237 defined, 38 PC, 36–37, 99–100 Automata, 340 B Binding, 341 Black Box components, 44–45 database/spreadsheet components as, 49 defined, 44 integrity, 46 use, 44 See also Software components Border Gateway Protocol (BGP), 341 Bugs, 341 Build stage defined, 83 inputs, outputs, checks, actions for, 85 See also Integration process Business analysis, 205–7 Business application programming interfaces (BAPIs), 135 defined, 318 OAGIS compliance, 135
Page 369
Business architecture, 110, 131–38 frameworks, 136–38 IBM San Francisco, 138 OAG and, 135–36 See also Architectures Business components, 63 defined, 131 deployment, 133 generic, 133 Business frameworks, 136–38 defined, 136 development, 137–38 IBM San Francisco, 137 independent, 137 See also Frameworks Business issues, 266–67 component approach and, 267 cost and time and, 267 organization and, 267 product lines and, 266–67 See also Component use Business object component architecture (BOCA), 139 defined, 318, 341 elements, 318 Business objects, 138–39 BOCA, 139, 318, 341 BODTF, 139, 319 business components vs., 138 common (CBOs), 138 defined, 319 facility (BOF), 138 Business process reengineering (BPR), 207 Business rules, 64 Butler CBD Forum, 11, 104–5, 204, 261 Butler model, 113–14 commoditization, 114, 116 defined, 113 illustrated, 113 market for components, 115 See also Architectural models C C++ language, 341–42 Caching, 342 Calculus for Communicating Systems (CCS), 23, 299 Capability Maturity Model, 289 Car assembly components, 30–32, 98–99 chassis and, 36 component list, 30 design, 33–36 elements, 32, 35 engine, 30, 33 interfaces, 31 offtheshelf, 33 parts and, 32, 34 reuse potential, 35 selfcontainment, 31 See also Components CARDS, 322 Catalysis, 253 Cease stage defined, 88 inputs, outputs, checks, actions for, 91 See also Integration process C language, 341 Class libraries, 47–48 Collaboration diagrammer, 254–55 in three C's, 159 Commercial components, 225–26 Commercial off the shelf (COTS), 27, 64–65 defined, 344 domains, 64 market, 65 move to larger scale components, 65 Common Gateway Interface (CGI), 342 Common Object Request Broker Architecture. See CORBA Communication, 159 Communication systems complexity, 279, 280 development issues, 280 formal development methods, 281–82, 284 See also Protocols Community of interest network (CoIN), 155, 156, 157, 161, 166 architecture illustration, 164 establishment of, 164 organization structure, 170 project, 161–62 Complexity, 296
Page 370
atomic, 293 communication system, 279, 280 information system, 162 protocol, 280 scale, 293 Component architecture diagrammer, 255 Componentbased development (CBD), 26, 66, 203–61 adapt components, 232–33 approach to, 220–21 architecture choice, 215–17 axioms for, 271 business analysis, 205–7 buy/tender for components, 231 checklist, 271 choice of component approach, 210–11 componentbased engineering vs., 204 componentize level, 218–19 component maintenance, 242–47 concept meaning, 28 defined, 342 deliver components to repository, 236–37 design, building, procurement, 229–37 design component, 229 develop/test components, 229–31 generic approach to, 257–61 harvest components, 233–34 methods for, 248–57 perceived benefit of, 212–13 preliminary component search, 213–15 process framework, 260–61 production engineering, 222–29 quantifiable benefits of, 214–15 RAD and, 247–48 requirements capture, 207–10 software life cycle, 205, 206 subscribe to components, 232 successful examples of, 29 system analysis and design, 217–22 system design and specification, 205–22 systems assembly/delivery, 237–42 system specification, 211–13 verify/validate components, 235–36 wrap legacy system, 234–35 Componentbased development, 287 Component Definition Interchange Format (CDIF), 319–20 defined, 319 Integrated MetaModel, 319–20 Component Definition Language (CDL), 139, 320 defined, 320 implementation, 320 Component maintenance, 242–47 abstract new components, 243 cost model generation, 247 existing components, 243–46 illustrated, 244 manage inventory/component use, 246 measure/evaluate component use, 243 update systems, 246–47 withdraw components, 247 See also Componentbased development (CBD) Component model, 54–65 business components, 63 component technology, 55–58 COTS, 64–65 enterprise components, 63–64 illustrated, 55 midtier components, 58–60 software components, 58 system components, 60–63 Component Object Model (COM), 55, 56, 199–200, 320–21 COM+, 56 defined, 199, 343 Distributed (DCOM), 56, 105, 199–200, 320–21 illustrated, 200 objects, 320, 321 See also Component technologies Componentoriented methods, 253–57 Catalysis, 253 COOL:Spex, 253, 254–56 COOL Methods Framework, 253–54 SELECT Perspective, 256–57 unified process, 256 See also Methods Components, 25–66, 151 abstracting, 243 adapting, 232–33
Page 371
approach funding, 265–66 assembly, 43, 237 attributes, 41–44 automatically generated, 42 awareness, 268 benefits of using, 28–29 businesslevel, 63 buying/tendering, 231 car assembly example, 30–32 classifying, 223–25 clearly identifiable attribute, 41 commercial, 225–26 commoditization, 114, 116 concept meaning, 28 cost and, 28 defined, 29–30, 343 delivering, to repository, 236–37 describing, 262–63 design compromises and, 29 designing, 229 developing/testing, 229–31 embedded, 42, 61 encapsulated attribute, 41–42 enterprise, 63–64 extracting, from repository, 237 flexibility and, 29 functional interrelation between, 106 hardware, 52–54 harvesting, 233–34 hierarchy/structure of, 40 importance of, 28 inhouse, 225 instantiated ondemand, 42 integrating, 237–40 interface, 43–44 layered model, 54–65 logical, 38, 43, 52 maintenance and, 29 material, 38 measure/evaluating, 243 midtier, 58–60 military, 34 parts, 38 personal computer assembly, 36–37 physical, 42, 52 physically realizable attribute, 42–43 plugable, 50–51 predefined attribute, 43 preinstantiated, 42 preliminary search for, 213–15 quality and, 28 reason for using, 28–29, 216 redefining design/specification to match, 226–27 reliability and, 28 replaceable, 43 reusable attribute, 44 selfcontained, 31, 41 status, 44 subcomponents, 38 subscribing to, 232 system design vs., 32–37 systemlevel, 60–63 time to market and, 28 transparency, 45 TTL, 53 unavailable, 227 verifying/validating, 235–36 versions, updating, 246–47 views of, 39–44 withdrawing, 247 See also Architectures; Component maintenance; Integration; Interfaces; Software components Component technologies, 55–58 COM, 55, 56 CORBA, 56–57 DCOM, 56 Enterprise Java Beans, 57–58 Component use, 266–70 business issues, 266–67 cultural issues, 267–69 managing, 246 technical issues, 269–70 Compositional reuse, 44 Computeraided software engineering(CASE) tools, 107, 211, 213, 342 Computers, 2–3 ambiguity/redundancy and, 21 defined, 343 interaction mechanisms, 14–15 Concurrent systems, 4, 343 Configuration defined, 343
Page 372
management, 240–42 Constructive algorithm, 301–7 defined, 304–5 discarding algorithm weaknesses and, 314 functioning of, 305 See also Interface equation Cookie, 343–44 COOL:Spex, 253, 254–56 Advisor, 255 collaboration diagrammer, 254–55 component architecture diagrammer, 255 defined, 253 interface diagrammer, 255 type diagrammer, 255 use case diagrammer, 255 use of, 254 See also Componentoriented methods COOL Methods Framework, 253–54 Coordination, 159 CORBA, 56–57, 105, 107, 195–98, 203, 321–22 component model, 322 CORBA 3, 321 defined, 56–57, 195, 321, 344 EJB convergence, 58 illustrated, 197 interfaces, 325 as middleware, 57 ORB, 56–57, 195–96 services, 321 specification, 197 vision behind, 196 Cost models, generating, 247 Cultural issues, 267–69 avoiding legacy systems, 268 component awareness, 268 enterprisewide scale, 268 reuse on large scale, 267–68 reward and recognition, 268–69 See also Component use Customer service system (CSS), 241 D Databases, 48–50 as Black Box component, 49 defined, 344 distributed, 346 as Glass Box component, 49 as legacy system, 50 server, 344–45 See also Software components Decision support systems (DSS), 168 Defense Information Systems Agency(DISA), 322–23 Department of Defense (DOD) software technology programs, 322–23 Describing components, 263–64 Design abstract, 227–29 defined, 38, 106, 345 patterns, 123 process, 345 Designer role, 75 Design stage defined, 79 inputs, outputs, checks, actions for, 81 See also Integration process Desktop technology, 167 Development stage defined, 79 inputs, outputs, checks, actions for, 82 See also Integration process Digital electronic circuit, 101 componentoriented, 103 interconnected, 102 Directories, 345–46 Discarding algorithm, 23, 301–7 behavioral approach, 314 difficulty, 304 weakness, 314 Dispersal, 155, 156 Distributed COM (DCOM), 56, 105, 107, 199–200, 320–21 defined, 56, 200, 345 illustrated, 200 objects, 321 Distributed computing environment (DCE), 198–99 client/server application support, 199 defined, 198–99, 322, 345 IDL, 322 illustrated, 199 Distributed Processing Environment (DPE), 346 Domains
Page 373
analysis, 252 COTS, 64 defined, 38, 51, 346 reuse, 39, 359 DomainSpecific Software Architectures (DSSAs), 119, 323 Dynamic conformance testing, 282–83 Dynamic systems development method (DSDM), 210, 217, 251–52 defined, 251–52 holistic approach, 252 E Embedded components, 61 Engineering, 179–202 applications, 190 architectures, 187–90 connection control, 184–87 evolution, 290 interface software/standards, 190–200 maturity, 293, 294 network, 180–84 Enterprise application integration (EAI), 240 Enterprise components, 63–64 business rules and, 64 defined, 63 See also Component model Enterprise Java Beans (EJB), 57–58 class, 58 CORBA convergence, 58 defined, 58, 346 Enterprise resource planning (ERP), 63, 134, 239, 318 applications, 135 market, 239 systems, 134–35 vendors, 135 Environment, 39 ESPRIT software evolution and reuse (SER), 323 Ethernet, 184, 347 Evolution plan, 93 Executive information systems (EIS), 168 Extensible markup language (XML), 134, 335–36 defined, 335, 364 documents, 134, 335 Extranets, 51 complications, 159 illustrated, 155 IPbased infrastructure, 162 F Formal methods, 284–87, 298–300 in communication applications, 284 defined, 281 developments affecting, 284 problems, 284 use of, 281–82, 298 Frameworks, 51, 125–28 attraction of, 122, 126 business, 136–38 CBD process, 260–61 concept, 125 custom, 125 defined, 38 as infrastructure, 126–27 in larger view, 126 in realworld application implementation, 126 Visual Basic, 51 See also Architecture Funding, component approaches, 265–66 G Generative reuse, 44, 359 Generic approach, 257–61 case study results, 259–60 experiment, 258–59 need for, 257–58 process framework, 260–61 See also Componentbased development (CBD) Glass Box components, 45–46 databases/spreadsheet components as, 49 defined, 45 user danger, 46 See also Software components Global communications, 1–3 Global reuse barriers, 45–46 defined, 39, 359 See also Reuse Granularity, 270 Graphical user interfaces (GUIs), 13, 348 Gray Box components, 46–47 defined, 46 See also Software components Groupware, 158–59
Page 374
H Hardware components, 52–54 Harvesting components, 233–34 Hypertext Markup Language (HTML), 348–49 Hypertext Transfer Protocol (HTTP), 335, 349 I IBM Sysplex technology, 9 Icompleteness, 301, 303, 306, 307, 308, 310 Idioms, 123 IEEE Architectural Planning Group (APG), 324 Indexing, 264 Information systems building blocks, 154 complexity, 162 dispersal and, 155, 156 integration, 168 technology mix, 165 Inhouse components, 225 Installation stage defined, 88 inputs, outputs, checks, actions for, 89 See also Integration process Institute of Software Architects, 143, 334 Integrated methods approach, 284–85 Integration, 69–95, 152 architecture definition, 92–93 closed interfaces and, 73 component, 237–40 concepts, 75–78 coping strategies, 75–78 defined, 3, 69–70 diverse data and, 73 enterprise application (EAI), 240 evolution plan, defining, 93 with existing installations, 70 information systems, 168 interfaces enabling, 6–7 loose, 74–75 map strategy, 76 multiple access requirements and, 71–73 nonstandard users and, 74 as plug and play, 7–8 practical, 88–94 problems, 4, 88, 92 as project management phase, 170 scrap strategy, 76 stovepipe designs, 70–71 strategy, publishing, 92 success, 4 template, 77 tight, 74–75 tools, 9–10 trap strategy, 76 unwrap strategy, 77–78 VLSI, 36, 363 wrap strategy, 76 See also Architectures; Components; Interfaces Integration process, 78–88 acceptance stage, 83, 84 build stage, 83, 85 cease stage, 88, 91 defined, 78 design stage, 79, 81 development stage, 79, 82 installation stage, 88, 89 operation stage, 88, 90 release stage, 83, 87 requirements stage, 78–79, 80 test stage, 83, 86 See also Integration Integrator role, 75 Interactive speech interface, 5 Interface Definition Language (IDL), 13, 58, 133, 324 defined, 324, 349 interface definitions, 324 Interface diagrammer, 255 Interface equation, 9–10, 22–24, 297–314 algorithms, 23, 301–7 as alternative representation, 300 application, 23 classes, 307 constructive algorithm, 301–7 defined, 9 discarding algorithm, 301–7 example, 299, 300 initial development of, 22 input machine specification, 23 operation, 9–10 solving, 299, 300–313 theory, 10
Page 375
Interfaces, 3–7, 151 ad hoc development, 22 aircraft, 4 API, 13, 16–18, 193–95 car assembly component, 31 closed, 73 complexity and, 13–24 component, 43–44 defined, 3, 5, 349–50 design problem, 21–22 developing/generating, 9 enabling integration, 6–7 GUI, 13 guises, 13 humantocomputer, 4 importance of, 5–6 interactive speech, 5 interpreter as, 5 latent traits, 21 messaging, 20 middleware, 18–19 network, 354 protocols, 4, 15–16 real world, 8–9 RPC, 19–20 scenario, 165 signaling, 195 types of, 4–5, 14–20 See also Architectures; Components; Integration International Council on Systems Engineering (INCOSE), 325 International Space Station (ISS), 274 Internet, 51, 350 Intranets, 51 defined, 160, 350 firewall, 160 moving to internets, 158 J Java Beans, 57–58, 107, 350 defined, 57 Enterprise (EJB), 58 Java Naming and Directory Interface (JNDI), 346, 350 Java Virtual Machine (JVM), 350 K Ksets, 301–2, 303, 306, 307, 309 constructing, 308 See also Interface equation L Languages 4GL, 338 defined, 351 Finnegans Wake, 278 mathematics as, 276–78 nature of, 276–78 pattern, 124–25 Large scale integration (LSI), 53 Layered model. See Component model Layers of generality, 112–13 defined, 112–13 illustrated, 112 See also Architectural models Legacy systems, 50, 60–63 avoiding, 268 client/server, 62 defined, 351 front end, 76 functions, 77–78 integration, 77–78 keeping, 76 logical components and, 52 reengineering, 76 scraping, 76 wrapped, 61, 234–35 Lego approach, 287 Local area networks (LANs), 165, 181–84 bus, 181–82 defined, 351 Ethernet, 184 mesh, 181 polling, 183 ring, 182 routing requirements, 184 star, 182–83 token passing, 183 See also Networks; Wide area networks (WANs) Local reuse, 39, 359 Logical components defined, 38, 52 legacy systems and, 52 modeling uses, 52 See also Components M Mailboxes, 20
Page 376
Mass customization, 156 Mathematics, 276–78 Messageoriented middleware (MOM), 191–92, 353 Message switching, 186 Messaging API (MAPI), 20 defined, 20 example, 20 Meta Object Facility (MOF), 325–26 Methods, 249–57 ATA, 141, 144, 317 componentoriented, 249, 253–57 formal, 281–82, 284–87, 298–300 OO, 252–53 structured, 249, 250–52 types of, 249 Metropolitan area network (MAN), 181 Microsoft Repository, 264, 325 Middleware, 18–19 application models, 18 architecture, 191 complexity, 19 defined, 18, 190, 352 forms, 191–92 messageoriented (MOM), 191–92, 353 ORB, 192 purpose of, 192 requirements, 191 role, 191 views of, 19 See also Interfaces Midtier components, 58–60 Military components, 34 Modeling, 352 Multiprocessing, 353 Multitasking, 353 Multithreading, 353 N Networks, 165–66 applications, 190 architecture, 167, 187–90 defined, 353 features, 180 interface, 354 interface software/standards, 190–200 LAN, 181–84 operating systems, 189–90, 354 peertopeer, 187–88 serverbased, 188–89 WAN, 184 NIST Advanced Technology Program, 326 O Object Management Architecture (OMA), 319, 327–28 defined, 327 standards, 328 Object Management Group (OMG), 134, 197, 319 defined, 328 interface standards development, 328 Objectoriented (OO) methods, 47, 48, 249, 252–53, 269 defined, 249, 354 domain analysis, 252 modeling constructs, 253 requirements analysis, 252 system design, 252–53 Object request broker (ORB), 56–57 defined, 57, 195 illustrated, 196 implementations, 57, 196 middleware, 192 See also CORBA Objects, 47–48 business, 138–39 COM/DCOM, 320, 321 defined, 354 Ocompleteness, 301, 302, 303, 306, 307, 308, 311 satisfying, 311–12 See also Interface equation Online analytical processing (OLAP), 171 Open Applications Group (OAG), 135–36, 240, 262, 318, 328–29 application integration and, 135–36 business object document (BOD), 136 defined, 328–29 OAGIS, 135, 136, 318, 326–27, 329 OAMAS, 135, 136, 327, 329 Open Distributed Processing (ODP), 329 Open Systems Interconnection (OSI), 355 Operating systems, 48 defined, 355 network, 189–90, 354 pipe, 356
Page 377
Operational support system (OSS), 130 architecture, 49 defined, 355 Operation stage defined, 88 inputs, outputs, checks, actions for, 90 See also Integration process Outlook, 15 P Packet switching, 187 Parts car assembly, 32, 34 defined, 38 See also Components Pattern languages, 124–25 Patterns, 120–23 anti, 123 application, 120 architectural, 123 defined, 38, 120, 356 design, 123 discovery, 121 elements, 120 idioms, 123 modelviewcontroller, 122 OO and, 120–21 as predefined component set, 121 in software engineering, 121 See also Architecture Pattern systems, 123–25 Peertopeer networks, 187–88, 356 Personal computer assembly, 36–37, 99–100 model, 36–37 VLSI circuit components, 36 Physical components, 52 Physical realization, 42–43 Pipe, 356 Plugable components, 50–51 Plug and play, 7–8 Polling, 356–57 PRISM, 323 Procedures, 47 Process diagram, 277 Process framework, 260–61 British telecommunications, 261 See also Componentbased development (CBD) Production engineering, 222–29 abstract design, 227–29 commercial component identification, 225–26 component classification, 223–25 defined, 222–23 design/specification redefinition, 226–27 illustrated, 224 inhouse component identification, 225 unavailable component specification, 227 See also Componentbased development (CBD) Product lines, 99, 128–30 approach proponents, 129 business function and, 266–67 defined, 38, 357 families, 129 Swedish, 129 See also Architecture Project management, 169–71 detailed design, 170 installation and training, 170–71 integration and testing, 170 preparation, 169 responsibilities, 175–76 roles/phases, 172–74 solution design, 169–70 Protocol conversion, 283–84, 298 Protocols, 15–16, 279–84 analysis, 282 behavior, 281 communication, 15 complexity, 280 defined, 15, 280, 358 defining, 280–81 effects of, 16 elements, 15–16 interface, 4, 15–16 properties, 282 testing, 282–83 See also Interfaces Q Quality system, 358 R Rapid application development (RAD), 210 CBD and, 247–48 cycles, 248 enabler, 248 technique concept, 247
Page 378
Rational 4+1 View, 114–17 defined, 114–16 scenarios, 116, 117 views, 114–16 See also Architectural models Rational Unified Process, 330 Release stage defined, 83 inputs, outputs, checks, actions for, 87 See also Integration process Remote procedure calls (RPCs), 19–20 to application program, 19–20 defined, 19, 358 implementations, 19 middleware, 191 stubs and, 20 transactional (TRPC), 93–94 See also Interfaces Repositories defined, 358 delivering components to, 236–37 extracting components from, 237 Microsoft, 264, 325 UMLcompliant, 333 Request for Comments (RFCs), 359 Requirements analysis, 252 capture, 207–10 Requirements stage defined, 78–79 inputs, outputs, checks, actions for, 80 See also Integration process Retrieval, 264 Reuse adaptive, 44, 233 barriers, 45–46 car assembly component, 35 compositional, 44 defined, 359 domain, 39, 359 flavors, 29 generative, 44, 359 global, 39, 359 local, 39, 359 maintenance and, 29 on larger scale, 267–68 test coverage accumulation and, 28 time to market and, 28 See also Components Routers, 6 S Scenarios, 164–68 desktop technology, 167 interfaces, 165 networks, 165–66 processes, 167–68 Rational 4+1 View, 116, 117 server technology, 166–67 Security, 161 SELECT Perspective, 256–57 Component Factory (SCF), 257 techniques/approaches, 256–57 See also Componentoriented methods Serverbased networks, 188–89 advantage, 189 defined, 188 illustrated, 189 server purposes, 189 See also Networks Servers defined, 359 technology, 166–67 Software architecture analysis method (SAAM), 141, 331 Software Architecture Technology Guide, 331–32 Software components, 39–44, 58 Black Box, 44–45 class libraries, 47–48 in component model, 58 databases, 48–50 examples, 47–51 generic types of, 44–47 Glass Box, 45–46 Gray Box, 46–47 hardware components vs., 52–54 integrating, 237–40 objects, 47–48 operating systems, 48 overloading, 46 plugable, 50–51 procedures, 47 specific types of, 47–51 spreadsheets, 48–50 subroutines, 47
Page 379
technologies, 55–58 White Box, 45 See also Component Software Engineering Institute (SEI), 130 ATA method, 141, 144 Product Line Practice Initiative, 130 Reuse and Product Lines Working Group, 128 SAAM, 141, 331 SATI, 331 Workshop, 129, 130 Software glue, 360 Software maintenance, 245 Spreadsheets, 48–50 as Black Box component, 49 as Glass Box component, 49 as legacy system, 50 See also Software components Standards Information Base (SIB), 330 STARS, 323 Static conformance testing, 282 Stovepipe designs, 70–71, 130–31 defined, 131, 361 illustrated, 72, 132 problem, 131 Structured development method (SDM), 219 Structured methods, 250–52 defined, 249 DSDM, 210, 217, 251–52 information engineering, 250–51 Yourdon, 251 See also Methods Structured Query Language (SQL), 360 Subcomponents defined, 38 plugin, 50 See also Components Subroutines, 47 Subscription components, 232 Switching, 186 message, 186 packet, 187 Systems analysis, 217–22 complexity, 278–79 components, 60–63 defined, 361 design, 217–22, 252–53, 361 integration. See Integration as set of configuration items, 295 Systems assembly/delivery, 237–42 component assembly, 237 component extraction, 237 component integration, 237–40 configuration management, 240–42 delivery, 242 illustrated, 238 system VV&T, 242 See also Componentbased development (CBD) System X, 281 T Technical issues, 269–70 architectures, standards, control, 269 granularity, 270 OO techniques, 269 quality and reliability, 269–70 threetier architecture, 269 See also Component use Technical Reference Model (TRM), 330 Technology architecture, 110 component, 55–58 defined, 39 desktop, 167 server, 166–67 Telecommunications Information Networking Architecture (TINA), 119, 332–33 defined, 332 fourlayer architecture, 332 goals, 332 subarchitectures, 333 Testing dynamic conformance, 282–83 as project management phase, 170 protocol, 282–83 static conformance, 282 Test stage defined, 83 inputs, outputs, checks, actions for, 86 See also Integration process The Open Group Architectural Framework (TOGAF), 118–19, 329
Page 380
Architecture Development Method (ADM), 330 defined, 330 Threelayered strategy, 163 Tower of Babel, 274, 275 Training, as project management phase, 170–71 Transactional RPC (TRPC), 93–94 Transaction processing (TP) monitor, 191 TransistorTransistor Logic (TTL) components, 53 building, 53 defined, 362 factors, 100 Type diagrammer, 255 U Unavailable components, 227 Unified modeling language (UML), 133, 147, 257, 259, 333 compliant repositories, 333 defined, 333, 362 encyclopedia edition, 285 large grained semantic descriptions and, 262 for modeling notation, 262 use of, 264, 333 Unified process, 256 Use case diagrammer, 255 User interface design, 285–86 V Verifying/validating components, 235–36 Very largescale integration (VLSI) components, 36 defined, 363 Virtual corporation, 160 device, 363 Visual Basic DLL/VBX components, 53 framework, 51 libraries, 27 plugin components, 50–51 W White Box components, 45 Wide area networks (WANs), 184 illustrated, 185 routing requirements, 184 See also Local area networks (LANs); Networks Workflow Management Coalition, 333–34 defined, 333–34 Workflow API (WAPI), 334 World Wide Web (WWW), 59–60, 134, 363 Consortium (W3C), 334–35 frontend, 160, 234 products built on, 209 X XMI Metadata Interchange format, 335 Y Yourdon method, 251
Page 381
Recent Titles in the Artech House Telecommunications Library Vinton G. Cerf, Senior Series Editor Access Networks: Technology and V5 Interfacing, Alex Gillespie Achieving Global Information Networking, Eve L. Varma, Thierry Stephant, et al. Advanced HighFrequency Radio Communications, Eric E. Johnson, Robert I. Desourdis, Jr., et al. Asynchronous Transfer Mode Networks: Performance Issues, Second Edition, Raif O. Onvural ATM Switches, Edwin R. Coover ATM Switching Systems, Thomas M. Chen and Stephen S. Liu Broadband Network Analysis and Design, Daniel Minoli Broadband Networking: ATM, SDH, and SONET, Mike Sexton and Andy Reid Broadband Telecommunications Technology, Second Edition, Byeong Lee, Minho Kang, and Jonghee Lee Communication and Computing for Distributed Multimedia Systems, Guojun Lu Communications Technology Guide for Business, Richard Downey, Seán Boland, and Phillip Walsh Community Networks: Lessons from Blackburg, Virginia, Second Edition, Andrew Cohill and Andrea Kavanaugh, editors ComponentBased Network System Engineering, Mark Norris, Rob Davis, and Alan Pengelly Computer Telephony Integration, Second Edition, Rob Walters Desktop Encyclopedia of the Internet, Nathan J. Muller Enterprise Networking: Fractional T1 to SONET, Frame Relay to BISDN, Daniel Minoli
Page 382
FAX: Facsimile Technology and Systems, Third Edition, Kenneth R. McConnell, Dennis Bodson, and Stephen Urban Guide to ATM Systems and Technology, Mohammad A. Rahman A Guide to the TCP/IP Protocol Suite, Floyd Wilder Information Superhighways Revisited: The Economics of Multimedia, Bruce Egan Internet email: Protocols, Standards, and Implementation, Lawrence Hughes Introduction to Telecommunications Network Engineering, Tarmo Anttalainen Introduction to Telephones and Telephone Systems, Third Edition, A. Michael Noll IP Convergence: The Next Revolution in Telecommunications, Nathan J. Muller The Law and Regulation of Telecommunications Carriers, Henk Brands and Evan T. Leo Marketing Telecommunications Services: New Approaches for a Changing Environment, Karen G. Strouse Mutlimedia Communications Networks: Technologies and Services, Mallikarjun Tatipamula and Bhumip Khashnabish, editors Packet Video: Modeling and Signal Processing, Naohisa Ohta Performance Evaluation of Communication Networks, Gary N. Higginbottom Practical Guide for Implementing Secure Intranets and Extranets, Kaustubh M. Phaltankar Practical Multiservice LANs: ATM and RF Broadband, Ernest O. Tunmann Protocol Management in Computer Networking, Philippe Byrnes Pulse Code Modulation Systems Design, William N. Waggener Service Level Management for Enterprise Networks, Lundy Lewis SNMPBased ATM Network Management, Heng Pan
Page 383
Successful Business Strategies Using Telecommunications Services, Martin F. Bartholomew Telecommunications Department Management, Robert A. Gable Telecommunications Deregulation, James Shaw Understanding Modern Telecommunications and the Information Superhighway, John G. Nellist and Elliott M. Gilbert Understanding Networking Technology: Concepts, Terms, and Trends, Second Edition, Mark Norris Videoconferencing and Videotelephony: Technology and Standards, Second Edition, Richard Schaphorst Visual Telephony, Edward A. Daly and Kathleen J. Hansell WideArea Data Network Performance Engineering, Robert G. Cole and Ravi Ramaswamy Winning Telco Customers Using Marketing Databases, Rob Mattison WorldClass Telecommunications Service Development, Ellen P. Ward For further information on these and other Artech House titles, including previously considered outofprint books now available through our InPrintForever® (IPF®) program, contact: Artech House 685 Canton Street Norwood, MA 02062 Phone: 781–769–9750 Fax: 781–769–6334 email:
[email protected] Artech House 46 Gillingham Street London SW1V 1AH UK Phone: +44(0)207596–8750 Fax: +44(0)207630–0166 email: artech
[email protected] Find us on the World Wide Web at: www.artechhouse.com
