Workflow Handbook 2005

Workflow Handbook 2005

Workflow Handbook

2005 _________________________________________________

Published in association with the

Edited by

Layna Fischer Future Strategies Inc., Book Division Lighthouse Point, Florida

Workflow Handbook 2005 Copyright © 2005 by Future Strategies Inc.

ISBN 0-9703509-8-8 05 04 03 02 1 2 3 4 5 All brand names and product names mentioned in this book are trademarks or service marks of their respective companies. Any omission or misuse should not be regarded as intent to infringe on the property of others. The Publisher recognizes and respects all marks used by companies, manufacturers and developers as a means to distinguish their products. The “WfMC” logo and “Workflow Management Coalition” are service marks of the Workflow Management Coalition. http://www.wfmc.org. Neither the editor, Workflow Management Coalition, nor Future Strategies Inc., accept any responsibility or liability for loss or damage occasioned to any person or property through using the material, instructions, methods, or ideas contained herein, or acting or refraining from acting as a result of such use. The authors and the publisher expressly disclaim all implied warrantees, including merchantability or fitness for any particular purpose. There will be no duty on the authors or Publisher to correct any errors or defects in the software.

Published by Future Strategies Inc., Book Division 2436 North Federal Highway #374 Lighthouse Point FL 33064 USA 954.782.3376 fax 954.782.6365 [email protected] Cover design by Pearl & Associates

All rights reserved. Manufactured in the United States of America. No part of this work covered by the copyright hereon may be reproduced or used in any form or by any means—graphic, electronic, or mechanical, including photocopying, recording, taping, or information storage and retrieval systems—without written permission of the publisher, except in the case of brief quotations embodied in critical articles and reviews. Publisher’s Cataloging-in-Publication Data Library of Congress Catalog Card No. 2005902352 Workflow Handbook 2005: /Layna Fischer (editor) p. cm. Includes bibliographical references, glossary, appendices and index.

ISBN 0-9703509-8-8 1. Business Process Management. 2. Workflow Management. 3. Technological Innovation. 4. Information Technology. 5. Total Quality Management. 6. Organizational Change 7. Management Information Systems. 8. Office Practice_Automation. 9. Business Process Technology. 10. Electronic Commerce. 11. Process Analysis Fischer, Layna

TABLE OF CONTENTS FOREWORD

7

Jon Pyke, Chair WfMC, United Kingdom

INTRODUCTION

9

Layna Fischer, General Manager Workflow Management Coalition, United States

SECTION 1—THE WORLD OF WORKFLOW WORKFLOW IN THE WORLD OF BPM: ARE THEY THE SAME?

17

Charlie Plesums, WfMC Fellow, United States

BPM—TOO MUCH BP, NOT ENOUGH OF THE M

23

Derek Miers, Enix Consulting, United Kingdom

INTEGRATED FUNCTION AND WORKFLOW

31

Chris Lawrence, Old Mutual, South Africa

BUSINESS ACTIVITY MONITORING AND SIMULATION

53

Joseph M. DeFee, CACI and Paul Harmon, Business Process Trends, United States

BUSINESS PROCESS IMPROVEMENT THROUGH OPTIMIZATION OF ITS STRUCTURAL PROPERTIES

75

Vladimír Modrák, Technical University of Košice, Slovakia

ENHANCING AND EXTENDING ERP PERFORMANCE WITH AN AUTOMATED WORKFLOW SYSTEM 91 Robert J. Kearney, Image Integration Systems, Inc., USA

NARROWING THE SEMANTIC GAP BETWEEN BUSINESS PROCESS ANALYSIS AND BUSINESS PROCESS EXECUTION 103 Dr. Setrag Khoshafian, Pegasystems Inc., USA

USING SOA AND WEB SERVICES TO IMPROVE BUSINESS PROCESS FLOW

113

Zachay B. Wheeler, Roberta Bortolotti, SDDM Technology, United States

WORKFLOW AND BUSINESS RULES–A COMMON APPROACH

129

Heinz Lienhard and Urs-Martin Künzi,ivyTeam-SORECOGroup, Switzerland

STATE OF BPM ADOPTION IN ASIA Ken Loke, Bizmann System (S) Pte Ltd., and Dr. Pallab Saha,, Institute of Systems Science, National University of Singapore

141

TABLE OF CONTENTS SECTION 2—WORKFLOW STANDARDS BUSINESS PROCESS METAMODELS AND SERVICES

159

Jean-Jacques Dubray, Attachmate, United States

WORKFLOW AND SERVICE-ORIENTED ARCHITECTURE (SOA)

179

Arnaud Bezancon, Advantys, France

A COMPARISON OF XML INTERCHANGE FORMATS FOR BUSINESS PROCESS MODELLING 185 Jan Mendling and Gustaf Neumann, Vienna University of Economics and Business Administration; and Markus Nüttgens, Hamburg University of Economics and Politics

HOW TO MEASURE THE CONTROL-FLOW COMPLEXITY OF WEB PROCESSES AND WORKFLOWS 199 Jorge Cardoso, Department of Mathematics and Engineering, University of Madeira, Portugal

AN EXAMPLE OF USING BPMN TO MODEL A BPEL PROCESS

213

Dr. Stephen A. White, IBM Corp., United States

A SIMPLE AND EFFICIENT ALGORITHM FOR VERIFYING WORKFLOW GRAPHS

233

Sinnakkrishnan Perumal and Ambuj Mahanti, Indian Institute of Management Calcutta, India

ASAP/WF-XML 2.0 COOKBOOK—UPDATED

257

Keith D Swenson, Fujitsu Software Corporation, United States

SECTION 3—DIRECTORIES AND APPENDICES WfMC Structure and Membership Information

281

Appendix—Membership Directory

283

Appendix-Officers and Fellows

289

Appendix—Author Biographies

307

Index

317

Other Resources

320

Foreword Jon Pyke, WfMC Chair, United Kingdom Thank you for supporting the work of the Workflow Management Coalition. It never ceases to amaze me just how much progress can be made in a 12month period. My concerns during 2004 were that the ever-increasing numbers of standards bodies in my industry were doing a great job in confusing the market place, holding back growth and putting us in danger of becoming just another over-hyped trendy sector. Well, things have certainly changed, and changed for the better. I am no longer concerned for the future of Workflow and Business Process Management technology. Over the past year the discussions over which standard fits where has abated, and we now have a clear understanding of direction. The increase in the term BPM is however causing confusion in the minds of many—does it mean Business Process Measurement? Business Process Modeling? or Business Process Measurement? It is clear that all of these terms are valid so the challenge now is to ensure that those responsible for the development of products, associated standards and the promotion of three-letter acronyms articulate the message crisply and clearly and ensure they say what they mean—and that’s part of the job of this publication. The development of standards is now moving at some considerable pace, and this is especially true with the XPDL standard also known by WfMC as Interface 2. Soon to see its second major version released, XPDL is recognized as a key component of the standards landscape. The working group responsible for the standard, led by Robert Shapiro, has made some significant advances especially in mapping the specification into the works of other standards groups such as the BPMI. The growing list of XPDL supporters and implementations from both the vendor and user community can be found on our website. Led by Keith Swenson, WfMC TC Chair, several times during 2004 the WfMC assembled a live demonstration of products that implemented the Wf-XML 2.0 web commerce protocol. Wf-XML is a protocol for process engines that makes it easy to link engines together for interoperability. Wf-XML is built upon OASIS ASAP, so it was simultaneously a demonstration of ASAP interoperability. The demonstration showed six different implementations of the new web services protocol and the exchange of data across the Internet. These demos were both in front of the live audience and online with observers around the world via a webcast, each time with sold-out capacity. A live demonstration led by Keith Swenson took place in Pisa, Italy in October 2004 at which we hosted a local BPM Workshop open to the first 100 attendees, as well as another 600 on line. Participants who implemented the protocol included ADVANTYS, Fujitsu, HandySoft and TIBCO who demonstrated the scenarios of Customer, Retailer and Manufacturer. ASAP and Wf-XML are designed to be used by non-programmers. This is the key.

1

FOREWORD More details on the demonstrations are available elsewhere in this book in the chapter: ASAP/Wf-XML 2.0 Cookbook—Updated by Keith Swenson. Significant milestones for WfMC in 2004 have been the continuing adoption of our standards and specifications by government and big business worldwide. Notably, the United Kingdom e-Gov National Workflow project issued a workflow official standards guide assisted greatly by David Hollingsworth, TC Chair from the Workflow Management Coalition. More information on this workflow guide is available on their website, www.workflowNP.org.uk. The WfMC Standards Reference Model has proved its importance in other areas of technology, most notably the ISO Seven Layer reference model for computer communications. The members of the Workflow Management Coalition hope you enjoy our Workflow Handbook 2005 and find it useful as you explore workflow and business process management and their many diverse benefits. Our thanks go to everybody who contributed to this important body of work and to Layna Fischer, WfMC General Manager for her role as chief editor and publisher. Jon Pyke, Chair WfMC

2

Introduction Layna Fischer, General Manager Workflow Management Coalition Welcome to the Workflow Handbook 2005. This edition offers you:: • SECTION 1: The World of Workflow covers a wide spectrum of viewpoints and discussions by experts in their respective fields. Papers range from an examination of the workflow in the world of BPM to Web Services workflow architectures and Business Process Management Technology and Business Rules. • SECTION 2: Process Standards deals with a comparative analysis of XML standards, with a visionary look into the future of the serviceoriented architecture. Several examples detail important step-by-step instructions of generating processes, such as using the BPMN specification to model a BPEL process. The ASAP/Wf-XML 2.0 Cookbook has been updated following several successful live demonstrations of the protocol. • SECTION 3: Directory and Appendices—an explanation of the structure of the Workflow Management Coalition and references comprise the last section including a membership directory.

Section 1—The World of Workflow WORKFLOW IN THE WORLD OF BPM: ARE THEY THE SAME?

17

Charlie Plesums, WfMC Fellow, United States

This introductory chapter describes how workflow management systems are no longer just a simple inventory of work to be processed, or a simple routing system, but have become sophisticated process management tools. System tools have emerged to help analyze and design complex new business processes. Other tools, the invocation engines, run the process as defined. Specifically these engines invoke transactions on systems both internally and across many organizations—suppliers, partners, and customers. Business Process Management—BPM—is born.

BPM—TOO MUCH BP, NOT ENOUGH OF THE M

23

Derek Miers, Enix Consulting, United Kingdom The problem with many BPM deployments is that they often overlook the reason why this technology is needed in the first place—to support the achievement of business objectives. The re-emergence of business processes as a core discipline in modern business management is fairly clear. But in order to really derive the maximum benefit from BPM initiatives, firms need to manage the people interface more carefully.

INTEGRATED FUNCTION AND WORKFLOW

31

Chris Lawrence, Old Mutual, South Africa Mr. Lawrence discusses designing and building computer systems based on the process modeling methodology called ‘integrated function and workflow’ (IFW) systems. A key claim of the approach presented in this chapter is that

9

INTRODUCTION it keeps the business model and the solution model aligned because they are one and the same model. The subprocess concept and construct is an important factor in that alignment—which is effectively the alignment between what and how.

BUSINESS ACTIVITY MONITORING AND SIMULATION

53

Joseph M. DeFee, CACI and Paul Harmon, Business Process Trends, United States Gartner suggests that BAM will become a major corporate concern in the next few years. Most large organizations will at least explore the possibility of improving business process management by creating systems that provide a broad overview of a process that can provide near-real-time information and advice to process managers. A variety of techniques will be used. The authors believe that simulation-based BAM will prove to be the most powerful and flexible approach to BAM and will increasingly be relied on by those with the more complex processes.

BUSINESS PROCESS IMPROVEMENT THROUGH OPTIMIZATION OF ITS STRUCTURAL PROPERTIES 75 Vladimír Modrák, Technical University of Košice, Slovakia With the growing requirements for the improvement of business activities within organizations, aspects of changes and new concepts of process structures are becoming a topical problem. These aspects are evenly important from standpoint objectives of the first out of two phases of workflow management (WfM). The processes of change have been addressed mostly at the level of administrative business processes (BP). Apart from main intention to present a practicable approach to the measurement of the structural complexity of business processes, the chapter also outlines some conceptual aspects of the effectiveness of the creation of practical tools for business processes redesign consisting of modeling and subsequent analyzing of processes structural attributes.

ENHANCING AND EXTENDING ERP PERFORMANCE WITH AN AUTOMATED WORKFLOW SYSTEM 91 Robert J. Kearney, Image Integration Systems, Inc., USA ERP systems are most commonly and correctly perceived and utilized as transaction processing machines. In that role they excel. Workflow systems, integrated with the ERP system, can function as the data delivery mechanism for ERP transactional processing. Conversely, ERP transactional processing is but one of the many activities in the workflow. Mr. Kearney describes how the integrated result provides capabilities that have been missing with ERP alone: standardization and automation of entire business processes, effective involvement and interaction with the business experts, and, the creation and capture of all relevant business process information. The improved business processes enable the promised economies of scale from centralized ERP processing.

NARROWING THE SEMANTIC GAP BETWEEN BUSINESS PROCESS ANALYSIS AND BUSINESS PROCESS EXECUTION 103 Dr. Setrag Khoshafian, Pegasystems Inc., USA The business process management (BPM) industry is growing rapidly, surpassing the expectations of even its most ardent supporters. Like most new technologies, BPM is enduring its own growing pains thanks to convergence, consolidation, and accelerated adoption. One of the critical areas of conver-

10

INTRODUCTION gence that has not received sufficient attention is the semantic gap and interoperability challenges between business process analysis (BPA) tools and intelligent BPM engines. This interoperability challenge is further aggravated by the lack of robust business rules modeling tools. Business rules are now regarded as essential components of a next generation BPM (intelligent or smart BPM). Even though there are various BPM standardization efforts, the semantic gaps between BPA and run-time intelligent BPM engines are considerable. This paper addresses these semantic gaps and identifies solutions for continuous and iterative development of complex intelligent BPM applications.

USING SOA AND WEB SERVICES TO IMPROVE BUSINESS PROCESS FLOW

113

Zachay B. Wheeler, Roberta Bortolotti, SDDM Technology, United States Case Study: District of Columbia's Oracle Database, Presentation Layer, SOAP/XML. Tasked with the analysis of improving and automating the current business process of license issuance for the Department of Consumer and Regulatory Affairs (DCRA) of the District Columbia, the SDDM Technology team developed Business Logic and Data Access Tiers. The solution to the District of Columbia business license problem was provided by using Service-Oriented Architecture and Web Services, taking advantage of a wide variety of available technologies. WS SOA allows business people in the DC government to consider using an existing application in a new way or offering it to a partner in a new way, thus potentially increasing the transactions between agencies.

WORKFLOW AND BUSINESS RULES–A COMMON APPROACH

129

Heinz Lienhard and Urs-Martin Künzi,ivyTeam-SORECOGroup, Switzerland The authors propose a BPM approach for addressing processes, Web Services and the use of business rules by the processes, starting from graphical models. Transparent, easy to manage and mathematically sound solutions are obtained in a coherent way. The authors show that practical experience with Business Rule Management within BPM will have a beneficial influence on the further development of BPM technology. What is already possible to do now will become very easy to do in the future, e.g. totally integrated calls to rule management from process elements (like “event starts,” “process triggers,” “decisions (gateways in BPMN)” etc). As well, rule inference may become a natural part of these systems.

STATE OF BPM ADOPTION IN ASIA

141

Ken Loke, Bizmann System (S) Pte Ltd., and Dr. Pallab Saha,, Institute of Systems Science, National University of Singapore There will be an exponential growth in the adoption of BPM technologies within ASEAN companies, say the authors. The way businesses and the marketplace are evolving will fuel this adoption. Acadamicians, consultants and solutions vendors are working together to bridge viable deliverables in various forms for ASEAN companies to adopt BPM. Companies have recognized the need to equip their workforce on process excellence. The intense competition and time to market factors further put pressure for companies optimize their current processes. The authors show how observing corporate governance in different degrees is something that most enterprises are taking seriously.

11

INTRODUCTION Section 2—Workflow Standards BUSINESS PROCESS METAMODELS AND SERVICES

159

Jean-Jacques Dubray, Attachmate, United States The software industry has long searched for a computing model where business or work processes would be explicit and where customers could change the business processes without significant coding projects. Programming languages like WS-BPEL, Service orientation and web service technologies represent a major architectural advance to create a new generation of business process engines that can integrate with a wide variety of business functions and across business boundaries going far beyond the original concepts of business process orchestration that were defined in the late ninetiesi and have hardly evolved since then. Mr. Dubray contends that this new generation of process engines is expected to manage end-to-end business processes while being far more flexible, far more business savvy and far more integrated with all aspects of IT as was laid out the business vision in the past 20 years. These concepts are poised to revolutionize software engineering and the way we build business applications.

WORKFLOW AND SERVICE-ORIENTED ARCHITECTURE (SOA)

179

Arnaud Bezancon, Advantys, France Service-Oriented Architecture is clearly the solution for organising information systems, responding on various levels to new development and communication challenges in applications. The work involved in system migration and choosing the appropriate moment to effect this migration are the main obstacles to rapid implementation in companies. Bezancon maintains that workflow, BPM and SOA are therefore not competitors but the proliferation of marketing and techniques surrounding automation of processes are such that solutions are particularly difficult to understand from the client company’s point of view. He predicts that in this particular context, those solutions presenting tools which are easiest to implement and use will almost certainly have the highest rate of success.

A COMPARISON OF XML INTERCHANGE FORMATS FOR BUSINESS PROCESS MODELLING 185 Jan Mendling and Gustaf Neumann, Vienna University of Economics and Business Administration; and Markus Nüttgens, Hamburg University of Economics and Politics This paper addresses heterogeneity of business process metamodels and related interchange formats. It presents different approaches towards interchange format design and effects of interchange format specification first. The authors derive the superset of metamodel concepts from 15 currently available XML-based specifications for business process modeling. These concepts are used as a framework for comparing the 15 specifications. The authors aim to contribute to a better comparison of heterogeneous approaches towards BPM, with the hope that this may finally result in a BPM reference metamodel and a related general interchange format for BPM.

HOW TO MEASURE THE CONTROL-FLOW COMPLEXITY OF WEB PROCESSES & WORKFLOWS 199 Jorge Cardoso, Department of Mathematics and Engineering, University of Madeira, Portugal The major goal of this chapter is to describe a measurement to analyze the control-flow complexity of Web processes and workflows. The measurement is to be used at design-time to evaluate the complexity of a process design

12

INTRODUCTION before implementation. In a competitive e-commerce and e-business market, organizations want Web processes and workflows to be simple, modular, easy to understand, easy to maintain and easy to re-engineer. To achieve these objectives, one can calculate the complexity of processes. The complexity of processes is intuitively connected to effects such as readability, understandability, effort, testability, reliability and maintainability. While these characteristics are fundamental in the context of processes, Prof. Cardoso illustrates that no methods currently exist that quantitatively evaluate the complexity of processes.

AN EXAMPLE OF USING BPMN TO MODEL A BPEL PROCESS

213

Dr. Stephen A. White, IBM Corp., United States The Business Process Modeling Notation (BPMN) has been developed to enable business user to develop readily understandable graphical representations of business processes. BPMN is also supported with appropriate graphical object properties that will enable the generation of executable BPEL. Thus, BPMN creates a standardized bridge for the gap between the business process design and process implementation. This paper presents a simple, yet instructive example of how a BPMN diagram can be used to generate a BPEL process.

A SIMPLE AND EFFICIENT ALGORITHM FOR VERIFYING WORKFLOW GRAPHS

233

Sinnakkrishnan Perumal and Ambuj Mahanti, Indian Institute of Management Calcutta, India The main contribution of this chapter is that a new workflow verification algorithm has been proposed to verify structural conflict errors in workflow graphs. This algorithm is presented along with visual step-by-step trace of the algorithm, correctness and completeness proofs and complexity proofs. Workflow verification issues have been solved in a simple and elegant manner by the proposed algorithm. The authors show how this algorithm is much easier to understand, as it uses search based techniques like DepthFirst Search and has significant advantages in terms of time complexity when compared to other workflow verification algorithms available in the literature.

ASAP/WF-XML 2.0 COOKBOOK—UPDATED

257

Keith D Swenson, Fujitsu Software Corporation, United States Wf-XML is a protocol for process engines that makes it easy to link engines together for interoperability. Wf-XML 2.0 is an updated version of this protocol, built on top of the Asynchronous Service Access Protocol (ASAP), which is in turn built on Simple Object Access Protocol (SOAP). This chapter is for those who have a process engine of some sort, and wish to implement a WfXML interface. At first, this may seem like a daunting task because the specifications are thick and formal. But, as you will see, the basic capability can be implemented quickly and easily. This article will take you through the basics of what you need to know in order to quickly set up a foundation and demonstrate the most essential functions. The rest of the functionality can rest on this foundation. The approach is to do a small part of the implementation in order to understand how your particular process engine will fit with the protocol.

13

INTRODUCTION Section 3—Directory and Appendices •

The Authors’ Appendix provides the contact details and biographies of the valuable contributors to this book. Each is a recognized expert in his or her respective field. You may contact them if you wish to pursue a discussion on their particular topics.

•

The chapters on the WfMC Structure and Membership describe the Coalition’s background, achievements and membership structure and sets out the contractual rights and obligations between members and the Coalition

•

WfMC Membership Directory: WfMC members in good standing as of February 2005 are listed here. Full Members have the membership benefit of optionally including details on their products or services. The WfMC invites you to delve into the information presented in whatever manner suits your reading or research style and knowledge level. Our thanks and acknowledgements extend to not only the authors whose works are published in this Handbook, but also to the many more that could not be published due to lack of space. Layna Fischer, Editor and Publisher General Manager, WfMC

14

Section 1

The World of Workflow

Workflow in the World of BPM Are They the Same? Charlie Plesums, WfMC Fellow, United States WORKFLOW “In the Middle Ages, monks sat at tables carefully copying the scriptures. The father superior would make the assignments, perhaps giving the illuminated first page of a section to the most skilled artist, perhaps assigning the proofreading tasks to the elderly scholar with trembling hands.1” Little has changed in centuries—the process is established, work is assigned and tracked, performance is checked, and results delivered. Often it is done by manual effort of supervisors (even today). But automated tools have emerged to assist.

WORKFLOW MANAGEMENT Document imaging emerged as a practical technology in the mid-1980s. When the images were stored in a computer system, there no longer was any paper to drop in somebody’s in-box—no natural way to assign and track the work. The initial simple workflow management tools routed the work to the person (or sequence of people) that needed to process the documents. The early workflow management systems were intimately associated with the content—the document—that was being moved or routed. The workflow management tools evolved to assign priorities to different types of work, to balance the distribution of work among multiple resources (people) who could handle the work, to support interruptions in the work (I’ll finish it in the morning), and to handle reassignment (I’m sick and am leaving now). Not all work was associated with a document, so most workflow management systems were adapted (if necessary) to process work that had no documents, images, or attachments—pure process, without content. For example, renewing an insurance policy or changing a credit limit. Not all processing steps required a person. Therefore most current workflow management systems support “straight through” processing or robotics. For example, •

As part of a larger process, one person may enter data from an order or application. When the entry is done and necessary data is available, the workflow system may automatically requests a credit report from a system in another company. When the report is received, the manual or automated process continues.

•

The entire process may be automated. An address change may be detected (perhaps on the payment stub), so is automatically sent to an OCR process. If the address recognized from the form is a valid address in the postal database, the change is made, and a confirmation

1

“Introduction to Workflow, Workflow Handbook 2002, page 19

17

WORKFLOW IN THE WORLD OF BUSINESS PROCESS MANAGEMENT letter is sent to the customer. A person is only involved in an error—if the data cannot automatically be verified. Thus workflow management systems are no longer just a simple inventory of work to be processed, or a simple routing system, but have become sophisticated process management tools. They originally assigned documents to people for processing, so most products have special strength assigning longer tasks to people, with one or many attached documents, but the viable workflow management systems have gone far beyond their historical origins.

BUSINESS PROCESS MANAGEMENT Business has become more complex in recent years. Many processes extend outside the organization—even outside the enterprise. Some of the steps that traditionally were handled internally are now being “outsourced” to “business partner” companies or individual contractors. Suppliers of products and services can be more economical and predictable when they are integrated into the larger process. System tools have emerged to help analyze and design these complex new business processes. Other tools, the invocation engines, run the process as defined. Specifically these engines invoke transactions on systems both internally and across many organizations—suppliers, partners, and customers. Business Process Management—BPM—is born. The new BPM tools have been defined “top down” rather than simply evolving (like most workflow management tools). The techniques used to define the process are more rigorous—some people take pride in being able to describe the mathematical foundation behind the various techniques. Graphical maps, modeling, and simulation are common tools to define the process. In production, the invocation engines will use the latest technology (including the Internet) to pass data and invoke processes on local and remote systems within an organization, between different organizations within an enterprise, and between enterprises.

ARE BPM AND WORKFLOW THE SAME? Both Business Process Management and Workflow allow a process to be defined, tested, and used. BPM originally focused on computer transactions— the large number of rapid business processes most often handled entirely by machine. Workflow originally focused on content that required human judgment or processing, often distributed among large numbers of people, with each process taking a relatively long time, thus being subject to interruption. Both BPM and Workflow can handle the entire range of business processes. The question is how well each product works in each situation. BPM is focused on defining the overall business process, and managing and tracking that process wherever it goes, often through multiple organizations, different computer systems, multiple locations, and even different enterprises. However, each transaction is normally quick—the request to get a credit report may need to be queued until morning, but when run, the response is immediate. There may be only one system that performs a particular transaction, and when it is run, it is rarely interrupted. Since processing is fast, most can be processed on a first-in, first-out basis. BPM is optimized for processes that are automatic, not involving human interaction. Workflow is focused on managing the process also, but often has components of uncertainty and delay such as those associated with people. We

18

WORKFLOW IN THE WORLD OF BUSINESS PROCESS MANAGEMENT may have to send the "work" to someone for approval, but that person may be interrupted (suspending the work while they take a phone call, go to a meeting, or even return from vacation). There may be many (even hundreds) of different processors (people or systems) that could handle that particular piece of work, and any one processor may be able to process many different types of work (but not the same combination as the next processor). The total work must be equitably distributed among the many processors. Some of the processing may take many minutes or hours, so more important work (the bigger deal or the better customer or the older work) may be given higher priority—be assigned first. The workflow is often modified as the person seeks additional information (such as a lab analysis or a legal opinion) for this particular case. Quality needs to be checked—not just is the system operating reliably, but are the people making mistakes. (Beginners may have 100 percent of their work checked, but an expert may only have a few percent checked.) Most workflow products can invoke programs automatically (without human intervention) like the BPM engines, but are optimized for the multiple processors and delays in the process. But many of the workflow products on the market today aren't (yet) as focused on the cross-enterprise or inter-enterprise processes as the BPM tools. We could conclude that both workflow management products and BPM products manage a business process, so in that sense they are the same. These products may be very different internally—and thus the strengths and weaknesses of those products in a particular application can be very different. •

Many workflow vendors have integrated BPM products into their workflow packages, but the degree of integration is sometimes limited—the product still is primarily a workflow tool.

•

Many BPM vendors have discovered the special needs of workflow, and have either integrated a workflow product into their BPM packages, or have built-in rudimentary workflow functions.

EMBEDDED WORKFLOW Many computer systems, such as those that process orders or administer insurance policies, have recognized the value of workflow. Therefore many of these systems have embedded workflow among the functions provided by their system. The good news is that this is a fast and convenient way to get started in workflow. The bad news is that it may only have a minimal set of workflow functionality—just enough to perform that specific application in an unsophisticated way. The workflow functions of a dedicated enterprise workflow system are likely to be much more robust than those that are embedded within a specific application. One person rarely works with a single application all day, every day—some of their time may be spent substituting for a supervisor, or quality checking other people’s work. Some companies mix assignments between “back office mail processing” and “front office call center” work. Some time will be spent in training or even developing new work management procedures. Work may arrive from the telephone, internal or external email, from the paper inbox, or from the workflow system. People do not do well balancing the work from multiple sources—while working on email, they will likely respond to all the email, even if more important work is waiting in the workflow system or inbox. This leads to an argument for a separate enterprise (or desktop) work-

19

WORKFLOW IN THE WORLD OF BUSINESS PROCESS MANAGEMENT flow systems, that balance the work between multiple business functions and systems.

STANDARDS The Workflow Management Coalition reference model, below, defines five components of workflow—the five interfaces to the workflow enactment services (what BPM calls the invocation engine). The standards associated with these interfaces are listed in the standards section on the WfMC website at www.wfmc.org. Understanding the five components helps distinguish between BPM and Workflow systems. Process Definition Tool

1

Workflow Enactment Services A1

Administration & Monitoring

A2

4

5 Workflow Engine(s) Vendor A

2

B

C

Other Workflow Enactment Services

3

Workflow Client Application

Invoked Applications

WfMC Reference Model WfMC Interface 1 is the process definition. This is the starting point for BPM and the particular BPM component that has received a lot of analysis and development. Workflow products vary in the techniques and tools used to define the process—some products use third party tools, if any, for analysis of the process, or use custom programs or simply manually generated tables to define the business process for the processing engine. WfMC Interface 2 is the client application—the program that invokes the workflow process. Workflow tools have a strong history of human interface, so may be stronger in this aspect; BPM is traditionally focused on processes

20

WORKFLOW IN THE WORLD OF BUSINESS PROCESS MANAGEMENT that require less human interaction, and may have a stronger programming interface. Some BPM products have no human interface at all—any human interaction must be custom programmed. WfMC Interface 3 is the interface to the programs invoked by the business process. The first workflow systems presented documents to people for processing in their traditional way (none of the processes to be performed by the people were automatically invoked). Workflow vendors quickly learned that they needed at least a minimal interface to expedite the process—start the right program for the user, even if the data all needed to be entered manually. Eventually most workflow tools evolved, and can invoke local and remote processes, but that interface must often be customized—it is an “add on” to many systems. On the other hand, BPM is built to automatically invoke existing systems, using consistent, modern interfaces. BPM products generally excel here, except when dealing with archaic legacy systems that don’t support the modern interfaces and tools. WfMC Interface 4 allows one workflow system to talk to another workflow system… to initiate work on another system (perhaps a different vendor’s system in a different enterprise), to track the status (progress) of that work, to cancel the work if necessary, and to inquire or be notified when the work is complete. BPM invocation engines generally don’t talk to other engines, but would invoke a transaction at another enterprise, and that transaction might, in turn, initiate a local process. Either approach gets the job done. WfMC Interface 5 is for administration and monitoring of the system, including logging, monitoring the performance and backlogs, and adjusting the resources. Both BPM and Workflow have these functions. But this is an area that deserves attention.

CAUTION One organization decided that their systems needed to be modernized across the entire enterprise, using new technology to link the home office and regional offices to their vendors and customers. They selected a BPM product, installed all the system components, and trained all the staff. The first application dealt with processing documents, from the customer through several regional offices, to home office specialists. Their BPM product didn’t naturally link to documents, but that could be added. Their BPM product didn’t have a user interface for the regional and home office staff to query the status of their work queues, but that could be built. Their BPM product didn’t have the ability to suspend work, allow a supervisor to override the assignment of work, or to automatically prioritize work (some work was more urgent than others). All of these requirements could be added to the platform provided by the BPM product. But this process could have easily been implemented by many traditional workflow systems, with little or no customization. The BPM solution may have been the best overall solution for the company, but workflow tools would have been much better to solve this first application. In another environment the problem might have had minimal human interaction, electronic data rather than documents, and processes that involve a more complex organizational structure. A workflow tool may accomplish the job, but BPM products might be easier to implement and perform better for this process.

21

WORKFLOW IN THE WORLD OF BUSINESS PROCESS MANAGEMENT There is no simple answer. This certainly doesn’t mean that Workflow is better than BPM, although it probably would have been for the one business process in the first case. Not does it mean that BPM is better than workflow, even though it might have been so in the second case. It does mean that the business requirements need to be honestly identified, and both BPM and Workflow products need to be considered.

CONCLUSION The key issue is the business process, and how that process works for the business users, partners, suppliers, and customers. BPM and Workflow are both technologies that manage the definition, implementation, and operation of the business processes. One is not good and the other bad, but they come from a different origin, and thus have different strengths. The key is to look beyond the product name, and find the functions that will best serve the business.

22

BPM—Too Much BP, Not Enough of the M Derek Miers, Enix Consulting, United Kingdom INTRODUCTION In the 1990s, I used to say, “The problem with workflow is that you don’t want it (work) to flow—you want it to get done.” The point was that, rather than having cases of work ping-ponging all over your business, the emphasis should be on getting work done. Generally, the reason companies were deploying workflow technology was to drive operational efficiency—to save cost while improving consistency and quality. Workflow also promised a way of dealing with the cultural malaise that had infected organizations over a certain size. But the reality of deployment in many large financial services businesses was that they just got into a bigger mess faster. Sure, many tasks were automated and streamlined to remove delays from the process. But the culture of the organization was seldom fundamentally changed. The body politic of the firm just absorbed the technological change and carried on as before. What firms were often missing was a methodology for dealing with the dayto-day grind of management—production and how to get the most out of the available resources. Even now, most firms still have only a superficial appreciation of how much work their employees are capable of handling. With a workflow implementation, the problem shifted. The visible backlogs of bulging in-trays were now hidden within the shared queues on the system. Individuals were driven (handed work) by the system but were seldom measured or managed in terms of what they could realistically achieve. Deployments took little account of the skill levels of the individuals. Some could achieve work at a tremendous rate, while others struggled to achieve the norm. At the team level, there was no sense of driving performance or the achievement of goals or business targets. Out-of-the-box, management information was simply unavailable in workflow products. While the system logged the history of all work (audit trail), it was left to the customer to write suitable programs that provided managers with effective information—i.e., while ‘management information’ was often promised, it was seldom delivered.

BPM TODAY And it is not much different today. All that has really changed are the terms. Now it’s called Business Process Management (BPM) rather than workflow. But in the same way that workflow implementation often missed the point, so do many BPM deployments—too much emphasis on the BP and not enough on the M. The problem with many BPM deployments is that they often overlook the reason why this technology is needed in the first place—to support the achievement of business objectives. They set out to deliver the ability to ensure that work is ‘done’—consistently, on time, and correctly. Yet they miss

23

BPM—TOO MUCH BP, NOT ENOUGH OF THE M one of the key ingredients to corporate success—the day-to-day management of the people involved. This is not about the life-cycle management of the process itself, which is still important. It is about the management of the people who work within the process—what their collective efforts can achieve, where they are struggling, how much work is coming down the pipe, and what they have to get out the door today, tomorrow, this week, or by the end of the month. The capability to steer the detailed operations of the business by driving its business processes (through BPM) is one thing, but developing an effective production management discipline is another. Technological support for this aspect is usually left to manual spreadsheets—an afterthought, developed by the managers themselves. While this may be good enough for a couple of small teams, it just doesn’t scale to 300 teams of 20 people. Indeed, most white-collar businesses have no accurate idea of what sort of work throughput is possible with the resources they already have. Executives continually hear the cry for more staff, yet they don’t really know how much work the current employees can deliver. Our research in major financial services firms has shown that as much as 30-40 percent additional productivity is possible when a disciplined production management approach is employed (over and above the benefits possible from the core workflow/BPM implementation). This research was based on a series of detailed interviews with key executives within the core business and IT management functions of major financial services firms in the UK and Europe. Firms ranged in size from 120 to several thousand employees. The primary focus of the analysis was to understand the critical success factors associated with major BPM deployments—how these affected productivity and the goals of executive management. In all cases, the businesses concerned had already deployed business process support environments. We really wanted to understand the issues that affected how one firm’s BPM implementation was more successful than that of another. What did they do different? What ‘best practices’ were developed? How was it driven and managed? We found that senior managers often bought into the idea of a BPM deployment based on not only increased productivity and better consistency, but also on the Holy Grail—better ‘Management Information.’ They really wanted the ability to look into the end-to-end profitability of a product or service, drilling down to assess, compare, and contrast the impact of one channel over another, or how individual departments were performing against Service Level Agreements (SLAs). They wanted the ability to track and monitor their business, identifying teams that were failing or quickly spotting bottlenecks that were impacting performance. They also wanted the ability to ensure adequate adherence to new regulatory requirements.

A CASE STUDY Halifax plc (now merged with the Bank of Scotland to create HBOS) is an internationally famous financial services brand. Responding to the challenges of the modern high street, their re-organization involved taking the back office functions out of their many branches and creating centralized administration centers. To support this exercise, process support systems were implemented to drive work from the front office operations in the high street directly into the back office.

24

BPM—TOO MUCH BP, NOT ENOUGH OF THE M The initial emphasis was on the BPM technology implementation, its features and usability at the front end. Significant benefits derived from the initial consolidation project with around a 15 percent reduction in costs. While some new management information was made available as a result, there was little change in the way that first line managers operated across the business. However, through a subsequent project, focused on the introduction of production management disciplines, the company transformed the whole culture of management. Rather than looking at team performance from a historical point of view (as was the norm), managers now predict what work is possible with the resources at their disposal. Previously, managers looked at what level of achievement they had delivered and then justified why they couldn’t handle any more. To support this new project, the core BPM system was extended with an innovative Management Information application that also took over the allocation and distribution of work, marrying the needs of the case and business priorities back to the skills of the employees in the individual teams. First line managers are now driven to understand how much work they have in the system and what is likely to arrive. In turn, this has allowed them to think more deeply about the performance of the individual team members, assessing their skills and personal development in a more holistic way. Individuals are assigned work within their capabilities and monitored against performance—in terms of task completion but also qualitatively. Managers are held accountable against weekly plans and asked to predict productivity over the ensuing 12 weeks. A league table is maintained, and individual team leaders are now incentivised to (over)achieve realistic performance targets through the staff under their control. The end result is a further 20 percent productivity improvement over the previous year alone. Week on week output is still rising and the costs of doing business are being driven ever lower. With over 2000 fulltime staff in the back office alone, that 20 percent improvement equates to 400 man-years—a big impact on the bottom line of the business. Moreover, the company has achieved a real transformation in management culture, building a virtuous circle of corporate performance, team working, and personal development. But none of this would have been possible without an alignment of incentives with the enhanced technology support environment. The product used to extend the BPM environment, Work Manager from eg Solutions in the UK (www.eguk.co.uk), provided a further layer of sophistication over and above the core capabilities of the BPM Engine. It allowed the business to develop a single view of work in the business, integrating the core BPM environment 1 with other workflow systems and applications.

We have since discovered that this same application is relatively widely used in the UK financial services market, working alongside leading BPM products such as Staffware, FileNet, AWD, and eiStream (now Global 360). In some implementations, Work Manager is integrated with two or three different BPM environments, providing a single view of all work in the business and allowing management to track work across processes rather than just functional silos.

1

25

BPM—TOO MUCH BP, NOT ENOUGH OF THE M In the words of the Head of Retail Processing for the bank (talking about the BPM deployment), “You end up with brilliant processes, but the people involved can’t necessarily handle them. The organizational culture is left a mile behind, and the people side suffers. We had the systems and process part working well, but the behavior and people side slipped. The real issue was developing a new set of management behaviors.”

DON’T FORGET THE PEOPLE There is an important lesson here. In a great many BPM projects, there is plenty of emphasis on the Business Process, but the wider holistic aspects of people, culture, and production management are often overlooked, yet they are just as important. As far as functionality goes, it is not a huge difference, but the lack of a ‘manage & control’ and ‘understand & improve’ culture makes a big difference to performance and bottom line profitability. BPM cannot be considered complete without it. All of the over-performing companies we surveyed sought to create a culture of continuous improvement. At the core of that objective were two common strategies: • •

The application of ‘production management’ techniques, and The extraction of quality ‘management information.’

From the executive point of view, the overall objective was generally sublime operational efficiency—reducing the level of resources required to deliver value. They also wanted more meaningful information to support decisionmaking and better adherence/compliance with corporate policies and procedures.

PRODUCTION MANAGEMENT & REGULATORY COMPLIANCE Production Management is really all about seven things: Measure; Plan; Communicate; Allocate; Monitor; Analyze, and Improve. While all this just sounds like plain, old-fashioned common sense to the seasoned manager, it has almost been lost in the hullabaloo created around the Business Process end of BPM. Having understood what their people are capable of and planned accordingly, team leaders need to track and monitor how well they do against targets. People need to know where they fit. When they understand this, they are far more motivated, especially if they can clearly grasp the basis for their individual targets and see that their achievements are fairly reflected. One mortgage firm we spoke with had achieved a particularly dramatic rise in productivity. With a new set of work practices and service level targets in place for six months, throughput rose by an average 221 percent per day (in the number of applications processed), and the number of actual completions carried out increased by a massive 429 percent. These changes have translated into a more profitable company with lending increasing from £220 million in the whole of 2001, to £620 million of completions in the last nine months of 2002 alone. In the words of another senior manager (from one of the world’s leading multinational insurance groups): “Our people now understand that a service level agreement is not a target that we aspire to, but a benchmark below which we will not fall. Introducing SLAs is an opportunity to make rapid and

26

BPM—TOO MUCH BP, NOT ENOUGH OF THE M lasting cultural changes within an organization, whilst bringing people with you. Everyone knows what he or she has to achieve. They now understand what ‘good’ looks like.” Of course, there is more to it than saying managers need to monitor their employees. What came out in our interviews again and again was the need to align the people to the work in hand and then monitor them individually. This implies assessing an individual’s skills and competencies against that required by the task. However, understanding people’s expertise and proficiencies is almost an art in itself. While this is relatively easy with a small team, when you consider the scale required for several thousand customer service staff and their associated back office support, a technological foundation is a must. On the other hand, the cost of getting this alignment wrong is usually not considered until it is too late. In the UK mortgage market during the late 90s, many firms ‘oversold’ endowment related insurance policies that paid off the mortgage at term. Across the industry, the costs associated with handling related complaints have now run into hundreds of millions of pounds. If firms had delivered that work to suitably qualified personnel in the first place, they could well have avoided the current cost repercussions that are having a serious impact on business and product profitability. Sure, some of the issues I am highlighting with this example are related to initial product design and a ‘responsible’ sales process, but the whole financial services industry is now governed by a much stricter regulatory regime. Firms have to meet stringent new targets on handling customer complaints within a given deadline—i.e. managers need to know when they are unlikely to achieve this. But more importantly, organizations must now ensure that the individuals handling sensitive parts of the sales and administrative processes are qualified to undertake the work. Stricter regulatory regimes are not limited to just the financial services industry. With the introduction of the Sarbanes Oxley legislation in the US, most large firms need to ensure compliance in all sorts of ways. Firms are also struggling with the new focus on transparency and compliance. The widely held perception is that greater control of the process will ensure regulatory compliance. While the underlying BPM technology provides an effective audit trail (logging the history of all work items), seemingly small errors in the way a case is handled can have a massive impact on the brand. But the reality is that routing a work item through the business, using rules to get it to the right role (job title), is just the first step. Modern financial markets demand transparency and accountability—on what we do, how we act, and the decisions we make—through every level of the business. With this transparency comes increased risk to the public ‘trust’ in the brand. But transparency is only about finding out afterwards—management via the rear view mirror. What is needed is a more proactive approach. A methodology that ensures employees have the right capabilities and training to undertake the work in hand. And to realistically achieve this goal, a more holistic view of the business is required.

27

BPM—TOO MUCH BP, NOT ENOUGH OF THE M MANAGEMENT INFORMATION As the use of procedural rules to route work moves all firms towards commoditization, differentiation will increasingly be based on how the processes are managed. Managing the life cycle of the process itself is certainly important, but we found that managing the groups of people interacting with those BPM systems is of equal importance. And managing people in such an environment requires that decisions are based upon good information. Virtually every BPM solution one looks at promotes the idea that, by using their technology, customers will get better management information. The audit trail provides all the information you’ll need—a complete history of the work item including who handled it when, what information was changed, etc. But because these products are focused on the needs of the business process life cycle, businesses are left to reflect on their own special needs for Management Information. With the right information gathering and reporting regime, the business can more easily sense important changes in the market and customer behavior, changing the process if necessary. But all that process flexibility will not do you much good unless you optimize the resources at your disposal. Based on a better understanding of operational capacity, managers can make more informed decisions, adjusting resource levels in different departments; effectively load balancing the business in line with peaks and troughs of demand. While this sounds a little like a return to the days of Bigger People Reductions (BPR), during our interviews we found that, with more accurate Management Information, a greater sense of reality prevailed. In some instances, this led to redeploying or recruiting extra staff to manage the work within the desired service and quality levels. Employees are now focused on targets set against several factors including effectiveness, quality, and product knowledge. With the core BPM environment, all that was really monitored was throughput. Firms seeking this sort of sophistication have three alternative options— build your own (i.e., write suitable programs on top of the BPM package), deploy a package with the core Production Management and Management Information components, or plug in a high-end ‘Business Intelligence’ application such as Hyperion or Comshare. While many BPM vendors claim to provide the requisite Management Information, most leave it to the customer to write suitable programs to reflect their own special reporting needs. Some BPM tools do provide their own built-in ‘analytics’ capabilities capturing average cycle times of processes and activities, or how long work items wait before moving on to the next activity. This information is useful for finding process bottlenecks, but often does little for the day-to-day grind of extracting management information to support production or supervision at the team and individual level. When it comes to building your own management information, the problem is that you first of all have to understand all the key business issues and problems. Moreover, a deep appreciation is needed of the relationship between day-to-day operations and effective reporting. You also need to be aware of the technological implementation issues and then develop your own methodology for deployment amongst the workforce. Indeed, developing an

28

BPM—TOO MUCH BP, NOT ENOUGH OF THE M appropriate implementation methodology that embeds the cultural change is a major part of the battle. With BI solutions, the emphasis is largely on financial reporting. Products tend to focus on the provision of an executive dashboard with a 360-degree view of all metrics. This information is largely underpinned by operational metrics, but its purpose is primarily overall financial performance. Again, support for the day-to-day needs of production management is largely lacking. When considering packaged implementation, we found only one product that seemed to cover the whole spectrum. Work Manager from eg Solutions provided a pretty comprehensive approach. Their customer references pointed to the product’s superior work allocation features, providing integrated management information with features for monitoring work throughput at the individual and team levels. This has left managers with more time to address process bottlenecks rather than simply detecting them. Managers were also more able to focus on value-added functions, so time saved on routine tasks such as work allocation was channeled into addressing customer needs.

CONCLUSION The re-emergence of business processes as a core discipline in modern business management is fairly clear. But in order to really derive the maximum benefit from BPM initiatives, firms need to manage the people interface more carefully. Through a focus on this area, successful firms have derived as much as 40 percent additional productivity improvement over and above that achieved by initial process automation using a BPM engine. In order to achieve these sorts of benefits, firms need to adopt a wider, holistic set of principles surrounding the BPM initiative. They need to institutionalize this as a way of thinking. Thinking that delivers real, tangible results and longterm benefits.

29

Integrated Function and Workflow1 Chris Lawrence, Old Mutual, South Africa I want to talk about designing and building computer systems based on the process modelling methodology previously outlined in this book. I shall call them ‘integrated function and workflow’ (IFW) systems. The significance of each of the words in this description should become clear as the rest of this book unfolds, so I won’t try to steal my own thunder by attempting a summary explanation now. It is possible to design computer systems in which the concepts of process, subprocess, task, and so on are themselves the basic building blocks. The immediate advantage of this is that the eventual system design matches the business design. This simple correspondence brings enormous benefits, with implications on a number of levels. Some of the implications can be quite profound. But before explaining how to design a computer system from a business process model, I need to introduce a specialised piece of software which makes it possible. This is the ‘workflow engine’ or ‘process engine’. But before I do that, I must talk a bit about current ‘workflow systems’. I am not claiming that any component calling itself a ‘workflow engine’ will do the job. But nor am I claiming that the kind of workflow engine the approach needs is radically different from any workflow engine currently available. Most if not all of the features which an IFW system needs in its workflow engine would be found somewhere in today’s ‘workflow technology’. I will not be describing workflow engines, or any particular workflow engine, at the technical level. But I will cover the essential business functionality of the kind of workflow engine an IFW system needs. I want to do this in enough depth to make it quite clear what architectural component is responsible for what feature of any eventual solution. Workflow systems Software systems configured as workflow engines have been around for a while, and there are many ‘workflow systems’ on the market. There is an entire industry dedicated to designing and perfecting workflow systems, and to deriving and maintaining standards in workflow technology. The standards include protocols allowing workflow systems and workflowenabled applications to talk to each other. To an extent however this industry has a particular kind of ‘workflow system’ in mind. This is primarily a separate application operating alongside (and therefore ‘external’ to) other business applications, and not necessarily communicating with them. This isn’t to say it rules out the type of workflow which is fully integrated into the business application, but that the paradigm case is of functional separation.

Excerpted with permission from the forthcoming book Make work make sense: An introduction to business process architecture by Chris Lawrence

1

31

INTEGRATED FUNCTION AND WORKFLOW There are a number of reasons for this, some good, some not so good. One good reason is that a business or organization may have a number of system applications, for example a core administration system, an accounting package, and, say, a human resources/payroll system. They may then have a single external workflow system which manages work done on (ie user interactions with) all these systems. If an entire business process spans a number of systems, the advantage of a functionally separate workflow system is that the entire process can be implemented in the one workflow implementation. This brings consistency and comprehensiveness. A less good reason is historical. Workflow is a relatively new technology— newer than a lot of the mainframe systems which dominate many of our banks, insurance companies and government departments. So there was obviously a ready market for workflow systems designed to operate in parallel with, or ‘on top of’, and therefore external to, those legacy systems. Turning now to what these packaged ‘workflow’ applications do, I think it is fair to say that their principal functionality focuses on the routing of links to electronic (scanned or text) documents and the management of work related to those documents. As a result there are now many reasonably sophisticated administration operations which see themselves as functioning as ‘paperless offices’. But their business processes are actually implemented in a mixture of procedure manuals, independent electronic models, people’s heads, and these ‘workflow’ packages. Their business processes are only partially implemented in the routing of links to electronic documents. This is because sometimes the two are the same and sometimes they are not:

There is another type of workflow which is a lot more ‘internal’. Many application systems have a ‘workflow component’ which may be of greater or lesser sophistication. This workflow component often consists of little more than control over the ‘status’ (or ‘status code’) of a particular master record, for example an insurance policy master record. This will exercise a degree of ‘workflow control’ over (say) new business administration, but the same control may not be extended to other areas—claims, investment instructions and so on. This second type of workflow is often called ‘embedded’ workflow—for fairly obvious reasons. This book is principally concerned with a third type of workflow. I shall call this ‘intrinsic’ rather than embedded or external workflow. Architecturally, both embedded and external workflow are afterthoughts. Embedded work-

32

INTEGRATED FUNCTION AND WORKFLOW flow is an afterthought built into the data and functional design of an administration system which is normally functionally complete and coherent without it. External workflow is even more of an afterthought, one that is literally layered onto a pre-existing application—regardless of how much ‘integration’ is then done to make the two layers communicate with each other. Perhaps the best way to characterise intrinsic workflow is to turn the IT clock back, and try to imagine what might have happened if ‘workflow’ systems had appeared first, and record-keeping systems only afterwards. Systems might then have been designed with their primary objective being to model, support, control, direct and record the work passing through the business, and only their secondary objective being to carry out the data processing, record keeping and document generation which was the work itself. Although the intrinsic workflow design pattern did not apply in the past, it will be increasingly important in the future. Even where external workflow makes sense as a unifying layer over disparate legacy systems, intrinsic workflow can serve as a model to which de facto enterprise (compromise?) architectures will aspire. Specification and interoperability standards will be as necessary in this world as in the current world. Where appropriate in this book I will employ the standards and terminology of the Workflow Management Coalition (WfMC), which is where the words ‘process’, ‘subprocess’ and ‘task’ originate. (WfMC uses ‘task’ as an alternate to ‘activity’.) Having said that I am using ‘subprocess’ in a slightly specialised sense. The reason for this I hope I shall make plain later on, when we get onto incremental automation. Further comparisons between external and ‘intrinsic’ workflow are drawn in a later section entitled “IFW and ‘classic’ workflow compared”. Workflow engine The WfMC literature includes a detailed survey of features displayed by a workflow engine or, more broadly, a ‘workflow enactment service’. I do not want to replicate these descriptions here. Instead I want to discuss in general business terms the features which a workflow engine would need in order to animate and articulate the kind of systems this book is about. It is unlikely that anything here will contradict a standard like that of the WfMC. These features are summarised below. Data The workflow engine will need its own database to support a number of specific data structures. I will go through these in turn. Process model These will be primarily the concepts already introduced, plus some supporting entities: • • • •

Process (instance) Process type • Eg order handling; insurance claim. Subprocess (instance) Subprocess type

33

INTEGRATED FUNCTION AND WORKFLOW • Eg take order; check order. Task (instance) Task type • Eg manual take order; automatic check order; automatic follow up. • Task class • Eg manual; automatic etc (others may be needed later). The data model diagram below shows the relationships between these entities. • •

The individual instances of the subject entity will be the individual cases—eg order 123 going through the order handling process. There will also need to be rules, attributes, and possibly additional entities, to allow processes of different types to be assembled from the appropriate subprocesses and tasks, in the right sequences, and with the correct routing. Users and access There will be more to say about tasks later on, including the features of manual versus automatic tasks. For now we shall just say that the main functionality of the systems discussed here will be at task level. So this will be where information about different users’ access rights will need to be applied. Different users will have access to different task types. The data model therefore needs to be extended:

34

INTEGRATED FUNCTION AND WORKFLOW

(In practice access may also need to operate at levels lower than task type. An example is where user A may be able to authorise requests captured by user B, and user B may be able to authorise requests captured by user A; but neither user A nor user B can authorise requests they themselves captured.) Workflow information ‘Workflow information’ here means information about where all the current process instances are at any time, and historic information about what task in what subprocess was performed when for what subject entity. In the case of manual tasks, the historic information would also include what user was involved at what point. So ‘current state’ information would be for example that subject entity 123 is in process ABC and has a current workflow status of ‘awaiting X’, where X is the subprocess (or task within the subprocess) which that process instance has reached.

35

INTEGRATED FUNCTION AND WORKFLOW Historic information would be for example: Process instance

Process

Sub process

Task

User

Date/ time started

Date/time finished

Task result*

Order 123

Order

Take order

Manual take order

Fred

10-1-2001; 09:00:00

10-1-2001; 09:30:00

Next Subprocess

Order 123

Order

Check order

Auto check order

N/A

10-1-2001; 09:31:00

10-1-2001; 09:31:02

Next Task

Order 123

Order

Check order

Manual correct errors

Fred

10-1-2001; 10:00:00

10-1-2001; 10:15:00

Next Task

Order 123

Order

Check order

Auto check order

N/A

10-1-2001; 10:16:00

10-1-2001; 10:16:02

Next Subprocess

Order 123

Order

Check credit rating

Auto credit check

N/A

10-1-2001; 10:20:00

10-1-2001; 10:20:02

Next Subprocess

…etc

Task result If, as a result of what has taken place in the task, the next move is to the next subprocess, then we shall say the task result is ‘next subprocess’. If instead the result is that the next move is to another task in the same subprocess, we shall say the result is ‘next task’. In practice the result ‘next task’ may need to be supplemented with an indication of which ‘next task’ to route to, as there could be more than one choice, but we’ll keep it simple for now. There may also be other possible task results later, but for the moment we shall only consider these two. So for example the only possible task result for ‘Manual take order’ is ‘next subprocess’, as there is nowhere else for the order to go. Then if the order passes all the validation rules in ‘Automatic check’, the task result is again ‘next subprocess’. But if as a result of errors the order has to be routed to the Manual correct errors task, then the task result is ‘next task’. System information We said above that in an actual IFW system the functionality itself is all contained at task level. In fact even that wasn’t strictly true. Just as a process is effectively a container for one or more subprocesses, and a subprocess is a container for one or more tasks, so a task is a container for one or more programs—and it is those programs which are the real functionality level. To prevent any misconceptions, the expression ‘one or more programs’ is only so as not to make assumptions about the implementation environment. Each task would normally be implemented by one ‘logical’ program, even if this consists of a number of different components, subroutines and so on.

36

INTEGRATED FUNCTION AND WORKFLOW The same logical program could however easily implement more than one task. The data model is therefore further extended:

Functionality The workflow engine will also need to ‘do’ a number of things, as well as ‘understand’ its database. Handle tasks A fundamental thing the workflow engine will have to do is handle task instances: both manual and automatic tasks. Both classes have slightly different requirements depending on whether or not they are the first in the process. To explain how the workflow engine handles various categories of tasks will involve explaining the generic features of the tasks themselves. This makes up a big part of the design of an IFW system. Manual task at the start of a process A manual task at the start of a process would normally be the way the relevant process begins—for example entering an order or an insurance claim. In fact most manual tasks in IFW systems, and in particular manual tasks which start processes, are like traditional on-line data entry programs whose job is to capture data and create and/or update records in a production database. And just as traditional data entry programs need to be made available to certain users and not others depending on the users’ access rights, so manual tasks which start processes will normally be accessible from some sort of

37

INTEGRATED FUNCTION AND WORKFLOW menu customised in accordance with the logged-in user’s access rights. In an IFW system this is typically a function of the workflow engine itself. So the workflow engine would be able to present each logged-in user with the list of processes that user is allowed to start: a ‘process menu’. Then, by the time the relevant manual task has completed, and as a result has captured enough data to start the process, the workflow engine would have created an instance of the correct process in its database. It would then follow the routing rules for the process of that type to determine what the next task should be. Following the order process example, since the subprocess ‘take order’ only has that one manual task, the routing will be to the next subprocess, in which the first task is the automatic check task. Having said all this, it is also possible for a manual task at the start of a process to behave like one after the start of a process (see below) and not be started from the process menu. There will be an example of this in the case study. Automatic task at the start of a process This would normally be where a process is not only triggered by another process, it also uses data captured by that other process. The first task of the triggered process can therefore be an automatic task which operates on the data captured by the process which triggered it. There is an example of this in one of the main processes in the case study. Automatic task after the start of a process The automatic ‘check order’ task is an example of an automatic task after the start of a process. The initial manual task will have completed, and the workflow status of the relevant subject entity will have been set to ‘awaiting check order’. The workflow engine makes the next task available, which is ‘Automatic check’. ‘Make available’ means effectively ‘place on the job queue’. The workflow engine therefore needs scheduler logic to decide what task instance gets processed next. This will use data including priority or frequency parameters held at the task type and/or task instance level. A fairly simple type of logic is that the workflow engine processes task instances in strict order of ‘effective date/time’. Thus each task instance gets created with an effective date/time, and the scheduler processes any task whose effective date/time is less than or equal to the current date/time, in strict order of effective date/time. Task instances can be created to run at a particular time in the future by setting their effective date/time in the future. One common way of doing this for automatic follow-up functionality is by setting a ‘time constraint’. The ‘rule’ would typically operate at task (type) level—eg to set the effective date/time of all instances of that task type to a week following the current date/time. The constraint itself (the future effective date/time) would be at task instance level. So for example a task type of ‘auto follow up’ might have a time constraint of seven days. An instance of task: Auto follow-up created at (say) 10:00:00 am on 3 March 2003 would have its effective date/time set to 10:00:00 am on 10 March 2003. The workflow engine would run the task at (or as soon as possible after) 10:00:00 am on 10 March 2003.

38

INTEGRATED FUNCTION AND WORKFLOW The more ‘normal’ task types which are not post-dated in this way would therefore have a default time constraint of zero. It is possible to implement other types of constraints, for example conditional constraints (only action if such and such is true). It is also possible to implement more complicated priority algorithms into the workflow engine if required. An automatic task is like a batch program processing a file of data with only one logical record in the file. In outline, the responsibilities of the workflow engine in this context are as follows: • • •

Decide when it is the turn of this automatic task instance to run. When its turn comes, call the appropriate (logical) program and cause it to be supplied with the appropriate data. The ‘appropriate data’ will be identified from the subject entity. When the program has processed that instance, act on the task result—if ‘next subprocess’, then route to the next subprocess; if ‘next task’, then route to the appropriate next task in the same subprocess. (In practice ‘route to’ means creating a record on the appropriate workflow table identifying the next task instance—either the first task of the next subprocess or the relevant next task in the same subprocess—and therefore placing the next task instance on the job queue.)

Manual task after the start of a process Take the example of the Manual correct errors task in the Check order subprocess. The Automatic check task has run and found errors in the captured data. It returns a result of ‘next task’. The only possible next task is Manual correct errors. Just as in the case of an ‘Automatic task after the start of a process’ a task instance record will need to be put on the appropriate workflow table. This will identify the subject entity instance (either directly or via identification of subprocess and process), the task type, and the task class. In this case the task class is ‘manual’. Therefore a user has to action it. Which user? Any user with the appropriate access rights—controlled by the task access table. How will these users access the task instance, and how do they know it is there to be actioned? This brings us to another important workflow feature, that of the ‘in tray’. This is another familiar way in which workflow systems interact with users (and vice versa). Whereas the ‘process menu’ enables users to start processes (of types they are allowed to start), the ‘in tray’ presents users with instances of (manual) tasks from processes which have already been started. The instances will be of task types the users are allowed to perform. They may well be from processes someone else (or something else) has started. An example of a ‘Manual task after the start of a process’ is ‘manual underwriting’. Someone else (eg a new business clerk) may have started a new business process for application number 123. When application number 123 reaches the underwriting subprocess the business rules decide it needs to be underwritten manually, by an underwriter of level 2. The routing within the

39

INTEGRATED FUNCTION AND WORKFLOW process will then ensure that a task instance for application number 123 will appear in the in tray of one or more underwriters of level 2. Different workflow systems distribute work in different ways, and they may be flexible in how they do this. In one paradigm the task instance for application number 123 will appear in the in tray of all level 2 underwriters. As soon as one of them ‘picks’ application number 123 off his in tray it will no longer be available for any others. In another paradigm there could be a work distribution algorithm which allocates work across users in accordance with skill levels, current workloads etc. Another variant of the distribution algorithm paradigm is where there is no in tray as such. The workflow engine not only allocates manual task instances between individuals, it also controls the order the individuals process the tasks. In this paradigm the user is only given one task instance at a time. See also ‘Distribute work’ below. It is worth mentioning that there are two overall logical categories of manual tasks which are ‘waiting’ to be performed. One is of tasks which can be done immediately—for example the manual underwriting case. The other category is of manual tasks which can only be performed when a particular external event occurs. An example is of a manual task which can only be performed when a reply is received from a customer. Manual task instances also sometimes need to be post-dated—ie created so they appear on user’s work queues (in trays) not immediately but at some time in the future. As with automatic tasks, this is done by applying a ‘time constraint’ at the task type level. Distribute work Work allocation algorithms have already been mentioned (‘Manual task after the start of a process’ above). Workflow systems will often have functionality to allow managers and supervisors to manipulate in trays, reallocate work and alter task access rights to ease bottlenecks and generally spread work appropriately across the available resources. Record data Another important job the workflow engine has is to record information about tasks as they happen. Information about subprocesses and processes can normally be derived from task-level data. Example data items are: • • • • • •

Task type • Eg manual underwriting; automatic check order Task class • Manual or automatic Date + time started Date + time completed User • (Or, more generically, ‘resource’) Task result • Next task; next subprocess;…

40

INTEGRATED FUNCTION AND WORKFLOW Display data It may not necessarily be the responsibility of the workflow engine itself to display data. But it will need to make the data available for query and display. This will cover both historic data (work which has happened) and data about work still to be done. This data will need to be accessed for a number of reasons, including both work management (how much work is there ‘in the system’; what has been allocated to whom) and also customer service (contact management; history of work done for particular customers; and so on). Incremental automation We can now draw a number of threads together to introduce another powerful feature of the architectural approach. The example will be the order process discussed earlier. We need to imagine a context where the system has been implemented initially with a relatively low level of automation, but with an intention to enhance the system incrementally thereafter. This ‘minimal’ principle can apply to all processes in the system, and all subprocesses and tasks in each process. But to illustrate what I mean I shall focus on one particular subprocess, that of ‘check credit rating’. For convenience the relevant process diagram is repeated below. Check credit rating Take order

Check order

+

+

Automatic credit check

Match against stock

Authorise order

Despatch order

+

+

+

pass 1 or 2

approved (rule 3)

meets criteria of 3, 4 or 5

Manual credit check

written to customer (rule 4 or 5)

Automatic follow up

Manual record documents

I now want to consider how the ‘Check credit rating’ subprocess may be developed incrementally, by going through a number of iterations. For reasons of illustration, the example will be fairly extreme.

41

INTEGRATED FUNCTION AND WORKFLOW There were some fairly intricate business rules involved in the ‘check credit rating’ subprocess. Incremental automation will to a large extent mean progressive implementation of those rules. Iteration 1 In the first iteration the subprocess would consist of just two tasks, both of which would have minimal functionality: Check credit rating Take order

Check order

+

+

Automatic credit check

Match against stock

Authorise order

Despatch order

+

+

+

Manual credit check

Automatic credit check This task would do literally nothing. Every case would pass straight through it. Manual credit check This task would consist of a screen identifying the order, the customer etc, and a yes/no button signifying the order either passes the credit check or it doesn’t. If yes, the process continues. If no, the order is cancelled, which terminates the process. The user would need to telephone or write to the customer to tell him the bad news. Other than the identity of the order and the customer, all the information the user would need to take into account before making the yes/no decision would be external to the system—on paper, in other systems etc. All the business rules described earlier would need to be applied ‘manually’. The reader will note that the Manual credit check task (however minimal) is a necessary component, because the credit check has to be performed, and control has to be exercised at this point in the process. In this iteration (in this example) the Automatic credit check is theoretically redundant (as it does nothing). But it was put there so that it could be built on in the future. Iteration 2 In the next iteration the subprocess might consist of the same two tasks, but the functionality of both could be enhanced. Automatic credit check The first business rule mentioned was: If the customer has sent cash with the order, then pass credit check. Assuming the ‘cash received’ amount was captured in the initial subprocess (Take order), this rule could be programmed into the automatic task:

42

INTEGRATED FUNCTION AND WORKFLOW If cash received >= order total, then Task result = next subprocess; otherwise Task result = next task. Manual credit check In the first iteration all instances would have gone to Manual credit check, and would therefore need to be worked on by a user. In the second iteration, the only ones which will need to be touched by a user would be those which did not pass business rule 1 above. The Manual credit check screen could be left as it was in the first iteration. Or it could be made slightly more sophisticated. As well as showing the identity of the order and the customer, and allowing the user to make a yes/no decision, it might help to show how close the instance came to passing, for example by showing the cash received and the order total and maybe the difference between the two. The very simple increase in automation between iterations 1 and 2 demonstrates the principle behind its economic justification. In iteration 1, 100 percent of instances go to Manual credit check. Let’s say for the sake of argument that in iteration 2 only 70 percent of instances go to Manual credit check. If the cost of changing the system from iteration 1 to iteration 2 is less than the saving made by automating 30 percent of instances (over a particular payback period), then the enhancement is justified. The key thing is to know in advance what the saving ought to be, which is why measurement is important. Iteration 3 etc In subsequent iterations further business rules could be implemented which will progressively reduce the proportion of instances which need to be handled manually. In view of the suggestion in iteration 2 to show how close the instance came to passing, rule 1 could be altered as follows: If (cash received + tolerance) >= order total, then Task result = next subprocess; otherwise Task result = next task. This would effectively automate the manual judgment in cases where the credit exposure is negligible. The second rule was: If the customer has not sent cash with the order, and the amount of the order is less than or equal to the total current unused credit, and the customer is not in arrears, then pass credit check. It will be evident that this rule would be straightforward to implement arithmetically—assuming the relevant data items are available. A rule like this one however may call for more sophisticated treatment: If the customer has not sent cash with the order, and the amount of the order is less than or equal to the total current unused credit, but the customer is currently in arrears, then write to the customer to request payment before accepting the order. We shall assume all the following data items are available to the system: cash received order total total current unused credit

43

INTEGRATED FUNCTION AND WORKFLOW total unpaid invoices greater than x days old customer name and address It should then be possible for the system to generate a letter to the customer. In the diagram below, the Manual credit check task would be responsible for generating the letter—for example by having a ‘generate letter’ button, and providing an opportunity for a user to check and/or amend the letter. An alternative design would give the Automatic credit check task the job of producing the letter, which might then go out immediately with no human intervention. Check credit rating Automatic credit check

pass 1 or 2

approved (rule 3)

meets criteria of 3, 4 or 5

Manual credit check

written to customer (rule 4 or 5)

Automatic follow up

Manual record documents

And so on. The principle should be established by now how this kind of architecture allows for targeted, incremental automation. Development effort can be concentrated on one or more processes, on one or more subprocesses, or on one or more tasks—depending on where automation makes the most economic sense. To a large extent, incremental automation means incremental implementation of business rules. (The automation does not have to be progressive—we could also literally get the rules wrong or not quite right the first time, and adjust them the next time. The architecture allows for ‘learning’ as much as it allows for incremental automation.)

44

INTEGRATED FUNCTION AND WORKFLOW ‘Raw’ and ‘implemented’ business rules We mentioned before that it was possible to distinguish between a ‘raw’ business rule and an ‘implemented’ business rule. We introduced rule 1 like this: If the customer has sent cash with the order, then pass credit check. This is what I mean by a ‘raw’ business rule. It is expressed purely in business terms. It could be (and very often will be) something which a person might follow. We then redescribed rule 1 like this: If cash received >= order total, then Task result = next subprocess; otherwise Task result = next task. We also said it could be programmed into the automatic task ‘Automatic credit check’. This is what I mean by an ‘implemented’ business rule. It is one which is now coded somewhere and somehow into the system, so it can be used to generate automatic behaviour. The words ‘somewhere and somehow’ are important. It could be ‘hard coded’ or programmed so as to be available to one or more system components (for example the component which implements the automatic task ‘Automatic credit check’). It could be implemented in data terms. For example if the minimum premium on policy type X is £50, then this might be implemented by setting a ‘minimum premium’ field on a ‘policy type’ record to £50 for policy type X. Or it could be implemented in workflow and routing. After all, the order process flow described in words (first take the order; then check the order; then check the customer’s credit rating; and so on) is only another set of (raw) business rules. They just happen to be rules implemented in the process model component of the system. Very often a raw business rule will be implemented by a combination of two or all three of these (program logic, data, process model). As far as implementing rules in program logic is concerned, the statement about a rule being ‘programmed so as to be available to one or more system components’ needs some explanation. It is generally desirable to reuse system components, and business rules are no exception. For a rule implemented in program logic, this means the same rule being run, or called, or applied, by more than one task. In data terms therefore, one task can apply many rules; but equally one rule can be applied by many tasks. So to extend part of a diagram we have used before:

45

INTEGRATED FUNCTION AND WORKFLOW Before leaving this subject it is worth mentioning that as with workflow and business process management there is a growing industry relating to academic research, standards and interoperability concerning the concept of a business rule and how types of business rule are to be categorised and logically analysed. See for example www.businessrulesgroup.org. I do not wish to stray into this heady atmosphere. I would however like to mention a recent review of business rules by CJ Date2. Date’s approach to business rules is to reduce them as much as possible to a subset of data (‘…business rules really ought to be part of the data model’). This is in pursuit of the ideal of ‘declarative’ rather than ‘procedural’ programming. I do not wish to argue the rights and wrongs of any particular way of implementing business rules in computer system design. The important objective is to achieve the correct results in an efficient and maintainable manner. From a theoretical standpoint however I do question an attempt to reduce all business rules to rules about well-formed data, integrity constraints, and data transformation. This is related to comments we made very early on (see ‘Process, data and work: Process and data’ above). It may be that all computer systems transform data. There may also be significant categories of computer systems for which it makes sense to view their prime function as that of transforming data. From the viewpoint of software engineering it may be true that for systems which are primarily mechanisms for transforming data the declarative approach to implementing rules is the best on offer. However, the systems I am talking about in this book are not primarily mechanisms for transforming data. Their transformation of data is a means to an end. They are primarily mechanisms for getting work done. The business rules are primarily rules about how that work is to be done. It may be useful to reduce some of those rules to rules about well-formed data, integrity constraints, and data transformation. But not all of them. The important word is ‘useful’. A rule which says an underwriter of level 2 or above must approve all life assurance proposals above £500,000 sum assured is not primarily a rule about well-formed data. It may be possible to express it as a rule about wellformed data—for example: IF Proposal.SumAssured > £500,000 THEN IF Proposal.Underwriter is level 2 or above THEN Proposal.Status can be ‘Approved’ END IF END IF

CJ Date: What Not How: The Business Rules Approach to Application Development, Addison-Wesley, 2000.

2

46

INTEGRATED FUNCTION AND WORKFLOW But expressing it like this risks losing sight of the dynamic features and implications. The implications are to do with work, routing, communication. If there isn’t an underwriter of level 2 or above where does the proposal go? How do we make sure it doesn’t get forgotten about? Perhaps it might be useful at this stage to embark on a brief review of the ‘systems I am talking about in this book’. I shall do this under the familiar aspects of data and functionality. High-level architecture The diagram below is intended as a high-level view of an IFW-architected system.

The diagram is intended to bring out several key points. Process model underpins entire design The process model (a finite set of processes, broken down into subprocesses and tasks, and interacting in specific ways) is what the workflow engine operates on and animates. But the process model is also the organising principle of the application system itself. However the application system is physically configured, it is logically partitioned into components which are primarily linked to tasks. Programs are not assembled into batch runs or (generally) called by on-line menus. They are linked to tasks (which are in turn linked to subprocesses

47

INTEGRATED FUNCTION AND WORKFLOW and processes) and are initiated by the workflow engine to process one instance of work at a time. This is a significant difference between IFW architecture and the currently more common ‘layered’ approach in which a generic workflow solution operates in parallel with a totally differently architected administration system. The latter would typically not be structured around the business process model. (See ‘Integrated function and workflow: Workflow systems’ above.) Below is an equivalent high-level view of a ‘layered’ approach.

Workflow and application functionality separate Although the process model in an IFW system both ‘instructs’ the workflow engine and ‘structures’ the application components, the workflow and application functionality each keeps to its own domain. The workflow engine almost operates as a ‘higher-order’ operating system for the application functionality. Workflow and application data separate Just as the workflow and application functionality are separate, so are the workflow and application logical databases. Workflow data is all about production instances (new business process for policy 123; order check subprocess for order number 456; etc). The applica-

48

INTEGRATED FUNCTION AND WORKFLOW tion data is the familiar production database of entities and attributes: customer tables, policy tables, transaction files. There clearly need to be areas of overlap, in the key or index data: policy numbers, agent numbers, order numbers—anything which is needed to identify instances of work. Subject entities in fact. It is this shared key data which supports the powerful reporting and analytical potential. Synthesising information The application database knows all about customers, orders, policy applications, names and addresses, credit limits. The workflow database knows all about business processes currently being performed and (more significant from a management information viewpoint) processes which have been performed in the past. These two domains of data can be synthesised to provide insight into, for example: •

Which customers (buying which products) generate the most complaints (require the most service attention, etc); • Which financial products take longest (and cost the most) to put into force (to underwrite, to arrange payment, etc); • Which areas of the business have the highest levels of manual work (and might therefore repay investment in greater levels of automation, after taking into account the differential costs of labour, etc); • Who did what on what case; who spends most of his time on what product; whose work needs to be checked and redone most often; … • …and so on. The potential of the architecture is limited only by imagination and economic common sense. Some at least of that imagination and common sense should be applied to the distribution of work. Distribution of work As long as there are appropriate ways of distributing system functionality, the architecture also provides scope for distributing work (or ‘work’) among people and other resources around the world and along the relevant value chain. The value chain can include geographically separate components of the same business, but also customers, agents, partners and suppliers, including business process outsource (BPO) providers. So for example the initial data entry of a switch instruction on a unit-linked investment contract could be done over the internet by a broker, or the investor himself, or by a company employee at a sales branch. The same principle could apply to placing a sales order. Authorisation tasks in the same process could be handled at a head office, by a GUI screen on a local area network or intranet. Since the workflow component is ring-fenced it can conform to appropriate interoperability standards (WfMC, BPMI, etc) and communicate with other workflow architectures. Batch routines in legacy systems can create instances of processes (eg policy or pension maturities). Accounting journal entries generated at instance level from an IFW system can be aggregated and/or summarised to feed on a daily/weekly/monthly basis into a batch-architected general ledger package.

49

INTEGRATED FUNCTION AND WORKFLOW There is just as much potential in the realms of distribution and interoperability as there is in targeted and incremental automation. How best to exploit that potential will depend on what the business or organization who owns the processes is aiming to achieve. So we shall soon do just that—look at what a particular business or organization may want to achieve, and how it might achieve it. But before that I want to round off this Theory section by looking at some remaining differences between the ‘IFW’ architectural approach and classic ‘extrinsic’ workflow. IFW and ‘classic’ workflow compared I do not want to delve into the mathematical theory underpinning workflow management systems. I shall however take as my example ‘generic external workflow system’ not any particular proprietary product but a ‘model’ as discussed in the theoretical literature. The main comparative features will be as applied to business applications and implementation projects. The first point I want to make is that in pure workflow terms it is generally possible to translate the two paradigms into each other. The main difference apparent in the diagrammatic representation of a classic workflow process and an IFW process is that classic workflow doesn’t have the IFW-style ‘subprocess’. Since in an implemented workflow solution a subprocess is ultimately only a ‘container’ for one or more tasks, this should not matter—in an implemented solution. I shall take as an example a completed ‘Order process’. In IFW format (and simplified BPMN notation) this would appear something like this: Take order Manual take order

Check order

Check credit rating

Automatic check

Match against stock Automatic credit check

Automatic match

Authorise order

Despatch order

Automatic authorise

Manual despatch

pass 1 or 2 Manual correct errors

Manual match

approved (rule 3)

meets criteria of 3, 4 or 5

Manual credit check

Manual authorise 1 Manual authorise 2

written to customer (rule 4 or 5)

Automatic follow up

Manual record documents

The same process expressed in ‘classic’ workflow notation would appear something like this:

50

INTEGRATED FUNCTION AND WORKFLOW

The point is not that one is ‘right’ and the other is ‘wrong’. As an implementation the classic schema is just as ‘right’, especially if the automatic and manual tasks do exactly the same in both cases. But the classic schema is normally (although not necessarily) associated with a context where the ‘manual’ tasks are the focus of attention, and are either ‘truly manual’, or are tasks associated with separate administration systems. The automatic tasks are normally concerned with routing, and therefore route the work (request) automatically by the application of coded rules. Those rules typically only have a limited amount of data available to them. There is something ‘producer-centric’ about this paradigm, as the accent is on getting the most out of human resources, and limiting the overhead of routing between them. But, as I said, this is not a necessary feature of the ‘classic’ workflow schema. One could imagine a workflow system configured in the classic format, but with no linked ‘administration’ system. And then more and more functionality and data storage is built into that workflow system so that in the end the workflow system and the administration system are one. The result would be an IFW system, even though it originated as a workflow ‘skeleton’ designed in the classic format. If this were done, it would have been without the ‘subprocess’ construct. What then is the point of the ‘subprocess’?

51

INTEGRATED FUNCTION AND WORKFLOW The subprocess does not serve a crucial system-architectural function in the implemented solution. The solution will work without it. But it serves a business-architectural function. This is best brought out by the following observations, which are all different aspects of the same thing: •

The subprocess is concerned with the what rather than the how. At the subprocess level what is important is that a transition has occurred between one business status and another. How it happened is not relevant. This accords with the customer-centric viewpoint. The customer is not interested in how his claim got approved, just that it did get approved. • Continuing the theme of the what versus the how, a process model which stops at the subprocess level is almost by definition a model of the business at the logical level. • The subprocess level can be important for measurement. Measurement at subprocess level can be both ‘natural’ and familiar. Yes it might be possible and important to count the number of cases waiting for initial assessment, waiting for medical evidence requested by the underwriter, or waiting for a final underwriting decision, but it is also important to provide subtotals at subprocess level: how many cases are ‘in underwriting’? • In terms of solution design the subprocess level very often corresponds to particular domains of data and functionality—which span across the component tasks. The partitioning referred to in the last bullet above is not as evident from a ‘classic’ workflow schema. In the example above the only indications that ‘Auto authorise’, ‘Manual authorise 1’ and ‘Manual authorise 2’ are linked are (i) the word ‘authorise’; and (ii) the fact that they are in a loop ‘controlled’ by ‘Auto authorise’. Reason (i) is completely arbitrary. Reason (ii) is also fairly arbitrary, in that ‘Manual authorise 1’ and ‘Manual authorise 2’ (in either classic or IFW schemas) do not have to route back through ‘Auto authorise’ to make ‘Auto authorise’ the only entry and exit point. Both manual authorisation tasks could quite legitimately route direct to ‘Manual despatch’. If so this would link both ‘Manual authorise 1’ and ‘Manual authorise 2’ closer to ‘Manual despatch’ than to each other. But as soon as one considers the business content the link between (say) ‘Auto authorise’, ‘Manual authorise 1’ and ‘Manual authorise 2’ is evident. The sort of data items processed automatically in ‘Auto authorise’ (cash amount of order; customer’s outstanding credit; etc) will typically be the ones needing to be displayed and considered ‘manually’ by ‘Manual authorise 1’ and ‘Manual authorise 2’. I do not want to labour the point. A key claim of the approach presented in this book is that it keeps the business model and the solution model aligned because they are one and the same model. The subprocess concept and construct is an important factor in that alignment—which is effectively the alignment between what and how.

52

Business Activity Monitoring and Simulation Joseph M. DeFee, CACI and Paul Harmon, Business Process Trends, United States 1. MANAGING A BUSINESS IN REAL TIME Companies have always depended on processes. Historical processes may not have been as well-analyzed as they are today, but there have always been business procedures designed to turn inputs into outputs in an efficient manner. Just as there have been processes that defined how materials flowed from their arrival to assembly and then to shipping, there have always been communication and control systems that attempted to monitor the process flows and deal with events that threatened to upset the expected flow. Consider one example of how processes have historically been managed. Imagine a small hospital of 30 years ago. As we have today, this hospital had a Customer Lifecycle Process that managed customers from admission through treatment to discharge. (See Figure 1.) The first subprocess or activity was probably Admissions. As patients came through the front door they were documented. The patient’s medical history was determined, credit was established, and the patient was assigned to a specific ward for treatment. Over the course of time, the hospital had established expectations. In a normal week, roughly the same number of patients entered as were discharged, maintaining a predictable need for doctors, nurses, medicines, and beds. Hospital Administrator

Activity Supervisor

Activity Supervisor

Activity Supervisor

Activity Supervisor Outprocessing

Admission

Actual Process Activities being Executed in Real Time

Figure 1. A manual patient-lifecycle process with reports delivered by phone. (Bold lines indicate flow of patients. Dashed lines indicate flow of communication and control information.)

Consider what happened when a serious local infection manifested, or when a major fire or traffic accident resulted in a large number of patients arriving at once. When a sudden, unexpectedly large number of patients arrived in the lobby, the supervisor in charge of admissions picked up the phone and called the administrator to alert her that there was a problem. The administrator would normally ask the nature of the emergency, and then consider

53

BUSINESS ACTIVITY MONITORING AND SIMULATION possible actions. If the emergency was an accident, it was easier to deal with, in the sense that a few phone calls could probably determine the extent of the accident and number of patients that would be arriving at the hospital in the course of the next hour or two. Calls to other supervisors would result in still other calls to change shifts and make more doctors and nurses available. Still other phone calls would result in shifts of bed assignments to assure that the emergency trauma ward had enough beds available. In the course of an hour or so, the hospital would adjust its activities and assign new resources to assure that the patient lifecycle process would continue to function effectively. A harder problem would be an increase in flu patients. In this case, instead of getting a large, but known, increase in admissions over the course of a few hours, the hospital would need to deal with a number of unknowns. Admissions would begin to increase slowly, and then, as the flu spread, the increase would grow daily. Many variables would determine the overall course of the flu. Severity, the susceptibility of particular groups—like older people or youngsters, whether school was in session, the availability of flu shots, and many other things could limit the spread, or control the duration of hospital flu patients. Complicating matters, the flu might infect doctors and nurses, making it harder to smoothly adjust staffing schedules. Although most flu epidemics pass without serious consequences, there have been especially virulent epidemics, like the one following World War I, which killed millions of people. The alert hospital administrator has to try to plan for a variety of different scenarios, and then adjust her actions as she acquired more data on the development of the flu in the hospital’s community, and in the nation as a whole. Different industries have different kinds of problems. Most, however, have processes that are designed to run within set parameters, and those processes have communication and control systems in place to handle exceptional periods. Most exceptions are easy to understand and deal with, while some are much more challenging, involving, as they do, more complex interactions among variables over a longer period of time. Historically, companies have relied on smart, experienced managers to gather appropriate data, interpret it correctly, and take decisions to minimize the effect of the changed circumstances on the daily functioning of company processes. As companies have become larger and processes have been dispersed over wider geographical areas, managing large business processes has become more difficult. In the past 30 years, most companies have installed computers and used them to collect data, and, in some cases, to automate processes. Thus, for example, our hospital admission office now enters new patient data via a computer terminal and can often access data from customer databases to determine a new patient’s medical history and credit. Since most supervisors have access to computers, it is often possible for ward supervisors to check the admissions database to determine how many new patients will be arriving in their ward in the next hour. Similarly, it is possible for an administrator to check historical data and generate a report that describes how many patients were admitted during the flu season last year or during the last 10 flu epidemics. In essence, computers, that were originally installed to facilitate or automate the flow of patients, parts, or assemblies through a process, can also be used to facilitate monitoring and communication, and some can even support

54

BUSINESS ACTIVITY MONITORING AND SIMULATION managers who have to make decisions to maintain the efficiency of a process in unusual circumstances. During the last few decades, most companies have also become more sophisticated in their management of processes. To counteract a tendency toward departmental functions that don’t communicate as efficiently as they might, most companies have designated managers who are responsible for largescale business processes. In product-oriented companies, these managers are often termed line-managers. In other cases, managers are assigned to coordinate processes that cross functional lines, like our patient lifecycle process. To support these managers, who are often responsible for managing processes that occur at several different locations and over long periods of time, software vendors are working to create tools that pull together all of the relevant information, highlight problems, anticipate problems, and assist making decisions to assure rapid successful adjustments of the process flow.

2. BUSINESS ACTIVITY MONITORING In 2002, the Gartner Group coined the term Business Activity Monitoring (BAM) to refer to software products that aimed at “providing real-time access to critical business performance indicators to improve the speed and effectiveness of business operations.”[1] In the past year the term BAM has become quite popular. Before using the term, however, it’s important to emphasize that BPM is a misleading term. The emphasis should have been on Business PROCESS Monitoring. Unfortunately, Gartner already used the acronym BPM to refer to Business Process MANAGEMENT, so Gartner apparently used “Activity” to get a unique, new acronym. BAM has caught on, and we’ll use it throughout this paper, but readers should remember that the emphasis in BAM is on pulling together information about large-scale processes, rather than on monitoring small-scale activities. BAM and Other Decision Support Technologies As we have already suggested, using computers to help smooth the flow of items through a process is nothing new. For many years, software designers have built triggers and alerts into software applications. Thus, for example, if admissions exceed some set number, an administrative terminal may sound an alert. One only has to think of an operator at a power plant to understand how a computer can provide an operator with a wide variety of alerts and even provide diagnostic information to assist the operator in his or her job performance. Similarly, marketing groups have analyzed data from sales for decades to determine shifts in customer preferences. In the past decade, many companies have invested in large Data Warehouses, that consolidate data from many smaller databases, and Business Intelligence (BI) applications that use special algorithms to search massive amounts of data, looking for patterns that humans might overlook. The results of these efforts usually find their way to senior managers who set strategy or design new products. IT groups have also used ERP and EAI tools to analyze the flow of data between application components. The emphasis, in the case of IT, has been on fast, efficient data processing and smoothly functioning middleware and not on drawing any broader meaning from the data. Still, it’s easy to imagine how transaction data, relabeled, and provided within the broader context of

55

BUSINESS ACTIVITY MONITORING AND SIMULATION

Who is Information Delivered To

a business process model, could help business process managers understand how a process is working. BAM proposes something that falls in-between the immediate feedback that alert signals and triggers can provide operators and supervisors and the long term trend reports that database reports and BI can provide senior managers. (See Figure 2.) BAM aims at providing a process manager with a broad overview of a major business process. As in the case of our example, it seems to provide a hospital administrator with an overview of the current status of the patient lifecycle process. Or, it seeks to provide a factory administrator with an overview of how an entire production line is functioning. Senior Staff Manager

Strategic Planning Systems

BAM Systems

Process Manager

Operator or Supervisor

Alarm and Process Control Systems

No Delay

Little Delay

Long Delay

Latency of Information Delivery Figure 2. One way of organizing the monitoring and decision support systems in use today.

Figure 2 provides one way of summarizing the range of monitoring and decision support systems in use today. Process control systems that provide information to operators and provide alerts to supervisors are mostly real-time systems that report to employees and supervisors who are very close to a specific activity. Similarly, systems that gather data, analyze it over hours, days, or weeks and report to senior staff managers are mostly designed to aid in future planning. BAM systems are newer and less widely deployed. They aim to fill the middle-ground between activity specific and strategic planning systems by providing business process managers with near-realtime information about an entire process. Properly done, they allow the process manager to initiate changes in specific activities that keep the entire process running smoothly. The Functions Required for an Effective BAM System A BAM system can not simply provide the administrator with the kinds of raw data or the signals that it provides plant operators or IT managers, or the administrator would be overwhelmed with inputs. Instead, someone must design a filtering system that draws data from a wide range of sources

56

BUSINESS ACTIVITY MONITORING AND SIMULATION and then massages it so that only truly significant data reaches the administrator. On the other hand, the BAM system can’t spend too long in massaging the data, or it will be out of date, like the BI systems that provide trend data to strategists, and only useful for future planning. A good BAM system should provide the administrator with enough information to enable good decisions, and it should provide the information in something close to real time so that decisions, when needed, can be taken in time to actually affect the ongoing performance of the process flow. Any effective BAM system requires a collection of modules. (See Figure 3.) There are different ways the various modules can perform their functions, but all of the functions must be present if the BAM system is to perform as its strongest advocates suggest it will. The first set of modules must convert data about actual events into digital information. In most cases this can be done by simply monitoring databases and transaction events that occur as software is used to automate a process. Thus, the same data the administration clerk enters into the computer, as he signs in a new patient, can feed a monitoring system that keeps track of the rate of patients entering the hospital. Hospital Administrator Business Activity Monitoring System Level 4 BUSINESS DASHBOARD

A User Friendly Interface that Presents Information on the Process in Close to Real Time, Provides Alerts, and Recommends Actions When Necessary.

Level 3 DECISION SUPPORT PROGRAM

Some Program for Analyzing the Data on Changes and then Determining Appropriate Actions the Manager Might Take

Level 2 PROCESS MODEL TO PROVIDE CONTEXT

Process Model of a Hospital Admission

Software Components and Databases that Automate Portions of the Hospital Admission System

Level 1 DIGITALIZED INFORMATION AVAILABLE

ACTIVITY Actual People and Process Activities being Executed in Real Time

Figure 3. Modules required for a serious BAM system.

The second level or set of modules required for BAM must provide some context for the digital data being accumulated. The BAM system may depend on an explicit model of the actual business process, as illustrated in Figure 3,

57

BUSINESS ACTIVITY MONITORING AND SIMULATION or it may simply depend on a series of equations that establish relationships between data sources. One way or another, however, the system must be able to organize the data to reflect the process it is monitoring. The analysis of the relevance of the data, the generation of information about trends, and intelligent action suggestions all depend on an analysis of the process and the relationships between process elements. Using its understanding of the process, the BAM system must apply some kind of logic to the data to identify problems, diagnose them, and to recommend managerial actions. For example, the BAM system might apply a set of business rules. One rule might state that whenever patient admissions increase by more 10% of the expect rate for a given period, a signal should be set. Another rule might state that whenever a signal is set as a result of a 10% increase, the patients’ ward assignments should be analyzed to determine if the increase in patients was random, or if there was a significant increase of patients for a particular ward. Still another rule might say that whenever patients were assigned to a given ward in excess of some historical number, the rule should post a suggestion on the administrator’s BAM monitor that specific changes are to be made in the staffing of the ward. Rule-based systems can be used to accomplish a number of different tasks. For our purposes here, they simply provide an example of one way the process data can be analyzed so as to generate action recommendations. Finally, any BAM system needs some way of presenting the information to an administrator. Most vendors speak of these monitor displays as “dashboards.” The term is meant to reflect the fact that the displays often have dials, gauges or other graphic devices that alert senior managers to changing conditions. Equally important, however, is a context for the information presented. If the manager is simply monitoring the admissions process, then several gauges might be adequate to let the manager know what is happening. If the manager is managing the entire customer lifecycle process, however, the manager is probably going to need some general picture of the process as a whole to pinpoint the problem area. For example, admissions might be normal, but discharges might drop, creating a shortage of beds for the new patients. Or, abnormal delays in obtaining lab test results may delay procedures that result in longer patient stays. So, some kind of graphic should probably help the manager pinpoint the area of concern. Then, within each area, data ought to be summarized. Finally, if alerts are displayed, information about the nature of the problem and possible corrective actions should also be presented. 3. Supporting Managerial Decision Making At this point, let’s focus on what we termed Level 3 in Figure 3, the specific approaches that are available to analyze data and generate actionable recommendations to process managers. There are, in essence, three more or less independent techniques that a BAM system might use to analyze data in order to make recommendations to a manager: Rule-Based Systems, Business Intelligence-Based Systems or Simulation Systems. (See Figure 4.)

58

BUSINESS ACTIVITY MONITORING AND SIMULATION

Hospital Administrator Business Activity Monitoring System

Findings Presented to Administrator

Current Data Compared With Historical Data by a BI engine to Identify Patterns

Current Data Analyzed With Business Rules To Identify Actions

Data About Historical Activities

Data About Activities

Current Data Analyzed By A Simulation To Identify Future Consequences

Simulation Model Uses Current and Historial Data to Generate Simulations

Activities Occuring

Figure 4. Sources and nature of decision analysis.

Rule-Based Systems The most straightforward approach, as we suggested earlier, is to use a set of rules to analyze existing data. Whenever the current data triggers a rule, it fires, either generating a recommendation, or triggering still other rules that ultimately lead to a recommendation. There are different types of rule-based systems. The simplest are rule systems that are embedded in data base management programs. The more complex rely on an inference engine that processes rules which are held in a rule repository. BI-Based Systems An alternative approach is to rely on historical data and Business Intelligence (BI) techniques. In this case, the BAM system might compare current data to historical data in an effort to identify a pattern. Such an approach might, for example, identify a slight increase in the use of certain medicines, correlate that with the season and a slight rise in admissions, and detect the onset of a flu epidemic before the doctors recognize that they are facing an epidemic. Using the same approach, the system might suggest to the administrator shifts in staffing and drug orders to bring today’s activities in line with the staffing and drug order patterns that were relied upon during the last three years’ epidemics. BI is an umbrella term used to refer to a broad collection of software tools and analytic techniques that can be used to analyze large quantities of data. The data used by BI systems is usually stored in a data warehouse. A data warehouse consists of the data storage and accompanying data integration architecture designed specifically to support data analysis for BI. The data

59

BUSINESS ACTIVITY MONITORING AND SIMULATION warehouse integrates operational data from various parts of the organization. Unlike operational databases, which typically only include current data, a data warehouse incorporates historical information, enabling analysis of business performance over time. Data warehousing is considered an essential, enabling component of most BI and analytic applications. BI usually relies on pattern matching algorithms derived from Artificial Intelligence (AI) research, or, in some cases on specially designed rule-based systems. Simulation-Based Systems Simulation systems rely on a process model and a set of assumptions about how work flows through the process. The assumptions are often based on knowledge of historical flows, but can be combined with current data. In essence, a simulation system projects future states of affairs. Thus, a trend that may be too small to attract attention today, may, if unchecked, result in major problems in a few days or months. A simulation system can run repeatedly simulations with the latest data and alert managers of potential problems before they occur. It can also be used to determine how proposed changes will affect the process in the future. Mixed Systems In many cases, BAM products will combine various analytic techniques. Thus simulation systems could also employ rules to facilitate certain kinds of analysis. Similarly, rule systems might also employ BI and other techniques. Table 1 provides a summary of some of the advantages and disadvantages associated with each analytic approach. Advantages

Rule-Based Systems That Analyze Current Data

BI Systems That Use Historical Data

Simulation Systems That Project Consequences

Disadvantages

- Can be provided for very specific tasks and can operate independent of other systems. - Can be easily defined and tested. - Is a well understood approach.

- Can become complex if the range of variables are extensive - Can become complex to test and maintain over time as the business changes - Difficult to graphically validate relationships to processes

- Have powerful algorithms for analyzing trends and patterns - Can pull together data from EAI and ERP systems and from best-of-breed applications.

- Work best when used in conjunction with large amounts of data - Must create Data Warehouse as precondition to using BAM capabilities - Aren't designed to use a process context for reporting - Have not been designed to operate in real-time.

- Provide capability to model highly complex and dynamic processes - Provides better insight into the future predicted state of the business, based on validated process flows - Can provide unique insight into longer range situations. - Notices gradual changes not so readily identified by other techniques. - Can take advantage of both BI and Rule-Based approaches to provide even better simulation results at run time.

- Simulation technology is not so well understood at the user level - Development of complex simulations requires specialized knowledge. - Requires an initial analysis and specification of a business process model.

Table 1. Comparison of Analysis Methods

60

BUSINESS ACTIVITY MONITORING AND SIMULATION Obviously there is no one best analytic technique for all situations. As a generalization, rule-based techniques are best for narrowly focused, specific analysis. BI techniques are best when there is a lot of historical data and you are reasonably sure that future situations will be like those that occurred in the past. Simulation techniques are best for more complex and changing situations. Early Examples of BAM Offerings Consider some early examples of vendors who are creating BAM solutions. The ERP vendors offer application modules. Their modules are designed to store information in a common database. Most integrate the flow of information between modules with a workflow system and rules. Depending on their existing technology, most have begun to create interfaces for business managers that are driven by information from the database. They have difficulty including data about applications other than their own. A good example of this approach is SAP’s BAM offering. Similarly, workflow vendors are well-positioned to enter the BAM market. Workflow tools depend on an initial analysis of the flow of material and information between activities. They usually supplement their models of a process with rules to make it easy to control and alter the flow. Although most workflow systems are small in scope, some cover entire business processes, and some are capable of interacting with non-automated processes by providing workers with tasks and recording when they indicate they have completed the tasks. For a good example of a BAM solution being offered by a workflow vendor, see SeeBeyond's BAM offering. EAI vendors offer systems that integrate a variety of different applications. They also depend on workflow-like systems, supported by rules, to describe how various applications are related and how to manage the flow of data between applications. EAI vendors have also begun to create BAM interfaces that allow managers to see how applications are functioning. These systems usually have trouble including information about human activities that may play a role in a large-scale process. A good example of this approach is TIBCO’s BAM offering. IBM acquired Holosofx, a business process modeling vendor, and has begun to integrate this business modeling tool into their WebSphere middleware environment. Holosofx, at the moment, relies on its strength in monitoring IBM MQSeries workflow data, but it can also monitor other middleware data flows to provide a manager with a management dashboard. Expect to see a variety of BAM offerings from Business Process Modeling tool vendors and from middleware vendors. The problem with most of these approaches is that they have to be hand-tailored for each application. In addition, a number of Data Warehouse/Business Intelligence vendors have begun to offer BAM modules. In this case the vendors are already storing data and already have powerful BI tools to examine the data. What they usually don’t have is a process model to provide context for the data, nor are they adept at providing analysis in near-real time. However, most are working on BAM extensions to their suites. A good example of this approach is the Business Objects BAM offering. Finally, a BAM solution is being developed by at least one Simulation vendor and that is the focus of this white paper. Like workflow, EAI, and business process modeling tools, simulation tools already rely on the creation of models of processes. Simulation tools are especially flexible in their modeling ca-

61

BUSINESS ACTIVITY MONITORING AND SIMULATION pabilities, since they are often used to model large-scale processes. Simulation tools normally rely on rules that incorporate statistical assumptions and on historical data to execute their models and generate data on possible future scenarios. Most of the current products rely on rules to offer limited decision support. The two exceptions are the DataWarehouse/BI tools that rely on their BI algorithms to identify historic patterns and the simulation tools that can use current and historical data to run scenarios and project future states based on current trends. Obviously, there are many combinations that are also possible. Many Business Modeling tools also support limited simulation, and, increasingly, most of these tools can be integrated with offerings from more powerful rule-based tools like those from Pegasystems and Fair-Issac. There are no mature BAM tools. All of the offerings, to date, are early products that have been assembled from the features of the vendor’s current product. As the market grows and matures, more comprehensive and specialized BAM offerings will appear. 4. Simulation At this point, since our primary focus is on the use of simulation to support BAM, let’s consider simulation in more detail. Simulation is the use of computing to mimic the behavior of a real-world system or process. Simulations are represented and executed from models that are abstractions of those real-world systems or processes. Simulation modeling is a broad topic that in its entirety is beyond the scope of this paper. We will focus on discrete-event simulation models since they are more likely to add value to business process analysis, business process management, business monitoring and decision support. Discrete event simulation models are based on events that occur within and are acted upon in a business process. By using random occurrences of those events, the simulation can mimic the dynamic behavior of the business. There are commonly two types of implementations of discrete event simulation models: • Probabilistic—the use of probability distribution functions to represent a stochastic process (this type of implementation is also commonly known as Monte Carlo) • Deterministic—the events that are input into the simulation will produce the same set of results over time This section will focus primarily on probabilistic discrete event simulation models. These models are used to define the business work steps, and specifically, the entities that flow through the business, as well as the resources required to perform each work step. Once the process modeler has created the basic process diagram, a simulation expert must enter information about the flow of events. The timing and occurrence of the events are based on probability distribution functions, which reproduce the behavioral dynamics of a real world business process. The developer of the simulation must choose probability functions that reflect the behavior of a given process. The process model and the information about the events are entered into a software program, which can then “execute” the simulation. By entering initial data, and executing the system, the modeler can determine future states of the process. As the simulation executes, the events are generated, the entities flow through the process, the delays are sampled, and the resources are

62

BUSINESS ACTIVITY MONITORING AND SIMULATION used—all using probability distributions to produce the real-world randomness of the business process. Consider a simulation of our hospital patient lifecycle process. We have a probabilistic function that determines how many patients who enter Admissions are routed to the maternity ward. This function is based on historical data. If we indicate that 100 patients enter on Monday, our system will automatically assign a portion of them to the Maternity ward. If we indicate that 1000 patients enter on Tuesday, the same formula will assign a proportionally large number to the Maternity ward. As the number of patients entering the Maternity ward increases, resources, ranging from beds and rooms to doctors and nurses, must be increased. By running different simulations we can determine just what resource would run out first and develop a plan to deal with the constraint if we expect that we might one day have that number of Maternity cases. A Detailed Simulation Study Now consider a more detailed simulation. In this case, consider what happens to patients entering the emergency room who need operations. This will illustrate how a discrete event simulation model can successfully support business process analysis. The purpose of this model was to consider the potential business impacts of opening a new emergency room facility that increased the number of treatment rooms by fifty percent. As with any good simulation model, the primary goals of the simulation were clearly stated. They were: • Examine the resource impacts (resource utilization, cost, etc.) from opening the new facility. Resources include physicians, nurses, support staff, supplies, and facilities. • Examine the impacts to patient treatment cycle time. This particular hospital maintains an average treatment cycle time of 2.1 hours per patient on average. The new facility should seek to improve the cycle time at best and maintain the cycle time at worst case. • Examine the cost of specific activities in the operation (activity based cost). You will notice that the goals of the simulation are kept to a few key business metrics. This is one of the most important things to remember when applying simulation technology—don’t try to model the whole world all at once. Models should be built incrementally or evolved to solve increasing complex problems over time. The measurements need to have a business focus and not focus on technical problems that the management is not immediately concerned with. The model is made up of three basic ingredients: entities, activities or work steps, and resources. The primary entities in the model are the types of patients that may arrive randomly at the door of the emergency room. The model makes assumptions about the number of individuals who will show up within any given period, and what types of problems they will have. These probabilistic assumptions are based on historical data. Some types of patients arrive more often, on average, than others. Some will have higher priority than others, based on severity of the illness or injury. The work steps include the primary activities performed, ranging from triage to initial evaluation, to treatment, to out-processing. The amount of time to perform each work step depends on the type of problem, and this, again is based on historical data and is represented as a probabilistic function that introduces a realistic variation into each patient’s treatment. For example, the amount

63

BUSINESS ACTIVITY MONITORING AND SIMULATION of time to perform the triage may be a random sample from an Exponential or Poisson distribution. The Normal distribution would have a mean value and a standard deviation to represent, statistically, what occurs in the real world. The resources required to perform each work step are assigned and include such attributes as the number available, cost, and planned down time. The number of each resource type may also be selected from a probability distribution in cases where a business has variations in the number of resources available. In other words, the model will consider things like employees taking leave or breakdowns in machinery. Figure 5 illustrates a screen shot of the emergency room process model.

Figure 5—SIMPROCESS® Screen Showing Top Level Process Flow Model

Notice that the model provides a concise, high-level view of the process. Each of the major sub-processes shown in Figure 5 has been defined in more detailed process models. This is another important aspect of simulation models. Where possible, the overall model should be kept as general as possible to assure it can be easily understood and that it will generate answers to the important business questions. Models with too much detail actually make it harder to identify and answer important questions. At any point in the analysis, if more granularity is needed, you can always drill down to the lower processes and model their sub-processes and activities to get the results needed. This technique allows you to have some processes modeled in detail while others are simply pass-through boxes. It lets you see the whole process at the higher levels and drill down in later spirals of analysis, if needed, without having to change the high level model. The patients (the key entities) in the model are categorized into three types: • Level 1—the most critical or severe, needing immediate attention • Level 2—critical patients needing attention but are probably in a lifethreatening condition • Level 3—patients that do not need immediate attention The three types of patients arrive at the door at varying rates. Figures 6 through 8 below show pop-up Windows that SIMPROCESS presents to analysts who are entering the information needed to set up a simulation. The ®

64

BUSINESS ACTIVITY MONITORING AND SIMULATION three Windows require the entry of information about the activity that occurs at the emergency room entry door. In this case, the analyst has decided to enter three different rates to reflect the patient flow historically experienced by the hospital. Figures 6 and 7 focus on the arrival rates for Level 1 patients.

Figure 6—SIMPROCESS® Screen Showing Entity Definition

In Figure 6 we define the three types of patient entities that will flow through our business model. Since the patient types, Level 1, 2 and 3, arrive at the hospital at varying rates and at varying quantities over time, we must define separate inter-arrival schedules for them in SIMPROCESS®. In Figure 6 we have set up three schedules and named them Level 1, 2, and 3 to correspond to the patient entity types. By choosing the Level 1 schedule in the Figure 6 dialog and choosing the Edit function, we can now describe the inter-arrival schedule for the Level 1 patients, as depicted in Figure 7.

Figure 7—SIMPROCESS™ Screen Showing Entity Schedules

The dialog presented in Figure 7 allows us to create schedules within schedules. That is, the inter-arrival of Level 1 patients vary depending on the time of day it is. In this model, we have defined three inter-arrival schedules for

65

BUSINESS ACTIVITY MONITORING AND SIMULATION the patients: Day Shift, Evening Shift, and Morning Shift. By selecting the shifts in Figure 7 and selecting the Edit function, we can define how the Level 1 patients will arrive for each daily period, or shift. This concept can be extended, as necessary, to include varying the schedules for weekends, time of month, or season. This is an important and powerful capability when using simulation models since this mimics how the business really encounters patients statistically during different periods of time. Once a shift is selected from the choices in Figure 7, the dialog box shown in Figure 8 is presented allowing the user. This dialog box allows the user to define the types of patients (entities) that could appear during that shift. Patient quantity and rate of arrival are defined as well. Since the hospital manages by shifts and keeps their metrics by shift, it only makes sense to have the simulation model be consistent with the shift schedules.

Figure 8 –SIMPROCESS® Screen Showing Patient Arrival Rates

The inter-arrival rates for Level 1 patients are 0.109375 patients on average every hour. This is based on the past history at the hospital having an average of 111 total patients per day with Level 1 patients making up 2.775 of those on average. For the day shift, it is 0.875 patients over an 8-hour period resulting in the 0.109375 per hour average. You will notice the use of a probability distribution function (Poisson) to provide a representative statistical curve of how the patients arrive. If we just use the average, then our model would not be probabilistic and would not provide a true representation of how things really occur. If events occurred in the real world at steady states, then simulation analysis would not be needed. However, that is not the case. Not only do the patients arrive at random, resources fail (x-ray machine breaks), go down (person takes leave or sick), are occupied or busy (physician takes varying times to diagnose and treat the patient), or deplete (for consumable resources, e.g., oxygen canisters) at random rates. Likewise, the time it takes to perform work steps varies and is not accurately represented by a steady state average. Once the entity (patient) arrival rates have been defined and the process flow diagram (see Figure 5) has been developed, the timing and resources on the work steps must be set to account for the time delay that is required and exactly what resources are assigned or consumed to perform the work steps.

66

BUSINESS ACTIVITY MONITORING AND SIMULATION Figure 9 shows a pop-up Window that an analyst could use to define the delay time and the resources required to perform one of the treatment steps (Perform Treatment) in this model.

Figure 9—SIMPROCESS® Screen Showing Activity Delay Time

In this model, we have defined the time to perform the treatment work step as a probability distribution function. We have used the Normal distribution function with an average of one hour and a standard deviation of 0.25 hours. When the simulation is run, the entities (patients) will flow through the system and when they reach this work step, the time allotted to perform the task will be randomly sampled from the Normal probability distribution function. This will create the randomness in the time it takes to do the task just as the probability distribution function in Figure 8 defined the interarrival rates of patients.

Figure 10—SIMPROCESS® Screen Showing Required Resources

Figure 10 shows the resources required to perform the actual treatment. Resources are globally defined in SIMRPOCESS™ and can be used or assigned to any task in the process model where they may be needed. The number of resources available as well as the number required to perform a

67

BUSINESS ACTIVITY MONITORING AND SIMULATION certain work step can also be randomly sampled as described earlier using probability distribution functions. You will notice that in this model, the number of nurses required is sampled using a probability distribution function to simulate how the real world process works since the same number of resources is not always the same, depending on the type of treatment. Notice that the physician resource is a shared resource since physicians bounce between five or six patients on average. This model is sampling a Poisson distribution with a mean of 20 percent of a physician’s time on average being consumed by a single patient. Additionally, resources can be required, based on different scenarios. In Figure 10, we have specified that all the resource requirements (physicians and nurses in this case) are required at the same time to perform the work step. This means the entity (patient) will wait (this is where queuing and bottlenecks are discovered) at this work step until both resource types are available simultaneously before the task can start. In some cases, we may list substitutable resources and set the requirements to “Any One Member” to specify that the work step can start as soon as any one of the resource types are available to do the job. The use of the “Reserve as Available” option in Figure 10 allows us to lock one of the resources needed as soon as it becomes available and wait until the others are available and lock them until all resources requirements are met. As can be seen in the use of probability distribution functions for all the key components of a SIMPROCESS® model (entities, activities, and resources), we can simulate even the most complex dynamic behavior of any business process. Recall the goals of the simulation model. We used the model to do “what if” scenarios that will be influenced by the new emergency room facility and 50 percent increase in treatment rooms. This model looks rather simple on the surface, but is a very powerful tool to do the “what if” analysis and determine if the business metrics and goals can be met. Figure 11 is a plot of the treatment cycle time (one of their most important business performance metrics) with the old facility plotted on the left and the new (50 percent additional rooms) plotted on the right. The plots in Figure 11 are clipped for the first 100 hours of simulation time. Before

After

Figure 11—Two SIMPROCESS® Screens Showing Patient Treatment Cycle Time: Before and After

Multiple simulations of “what if” scenarios were used to ensure the hospital could achieve its treatment cycle time business goals. For example, if the number of physicians, nurses, and lab technicians remain the same, the ad-

68

BUSINESS ACTIVITY MONITORING AND SIMULATION ditional rooms will cause the treatment cycle time to go above three hours. That is due to patients getting in the rooms without enough resources to treat them, hence waiting in the room for additional time (treatment time is counted from the time the patient gets to a room until the time they leave the treatment room). The simulations quickly uncovered these types of problems and allowed the hospital to play additional “what if” games to find the right balance of resources and costs to make optimal use of the new treatment facilities and resources. The objective of all this is being able to avoid embarrassing impacts to treatment of patients (for this particular hospital, cycle time is one of their marketing nuggets) or worse, poor utilization of a new facility from an operating cost standpoint. As can be seen from the “after” plot in Figure 11, the treatment cycle time is actually improved when adjustments are made to the physician and nurse resources. In the old facility, adding resources would not have improved treatment time since the bottleneck was the number of room resources. 5. Using Simulation for BAM Section 4 focused on how simulation models are used in traditional business process analysis (BPA). Models are developed and validated for an existing business process (an As Is model). Then, various changes (What If models) were imagined and tested, via simulation, to see if they would improve the efficiency of the process. In the simulation process, bottlenecks and specific inefficiencies were identified and eliminated. Although this use of simulation is very common and valuable, most organizations use the simulation models developed in this manner during a limited improvement project and then set them aside as the new process is implemented. It is possible, however, to use simulation to support BAM systems. In essence, the new process model is maintained in the simulation environment, and new simulations, using the latest data, are run periodically. Triggers or rules are used to identify problems. The current level of admissions at our hospital, for example, may be slightly higher than it was a week ago, but not high enough to trigger an alert. On the other hand, if a simulation is run using the past month’s data, it may be determined that an underlying trend is present that will result in unacceptable rates of admissions in three weeks. Figure 12 suggests how a Simulation Environment might be linked to software applications and databases to allow the simulation system to be run with real-time data. In effect, data generated by the actual process would be used to run simulations. The simulation system would have alerts and rules to identify problems and suggest alternatives. In some cases, the simulation system might include alternative activities. If the system determined that the normal process would generate problems, it might try pre-packaged alternative approaches to a specific activity to see if a particular set of changes would result in an acceptable projection. This is one way in which a simulation might be able to combine alerting a manager about problems with suggested remedies.

69

BUSINESS ACTIVITY MONITORING AND SIMULATION Hospital Administrator Business Activity Monitoring System BUSINESS DASHBOARD

SIMULATION PROGRAM

PROCESS MODEL TO PROVIDE CONTEXT

A User Friendly Interface that Presents Information on the Process in Close to Real Time, Provides Alerts, and Recommends Actions When Necessary.

Simulation Program is part of Modeling Program and Makes Decisions by Generating Projections to See if Any Fire Triggers or Rules.

Process Model of a Hospital Admission

alternative approaches

Software Components and Databases that Automate Portions of the Hospital Admission System

DIGITALIZED INFORMATION

ACTIVITY Actual People and Process Activities being Executed in Real Time

Figure 12. A BAM System Based on a Simulation System.

The approach described in Figure 12 goes beyond what is available in today’s simulation products. In effect, it changes simulation from something done to test alternatives, and makes it, instead, a way of dynamically determining what will happen in the future if the current state of the process is allowed to continue without change. Obviously, this diagram greatly simplifies what is involved in using simulation for BAM. The developers need, for example, to identify the events or data items that will be monitored. Similarly, they need to insert triggers or create rules to determine when managers should be alerted. And they need to determine how frequently the simulation should be run, and how far ahead it should project. These are all decisions that will need to be made in the context of a specific company process. These decisions are not unique to the use of simulation. The same events, triggers, and information monitoring will need to be defined for any BAM solution. Simulation is merely providing an additional dimension to already useful BAM solutions. Rule-based systems, by themselves, only look at the present for problems. BI systems use historical data to look for current patterns that might suggest problems, but can only identify problems that have already occurred before, in the past. Simulation systems can combine the best of both and add the ability to look for future states that suggest problems and then dynamically try alternative assumptions to identify changes that the manager could make today to avoid the undesired future state. Companies that have already used simulation and have been happy with the results find the possibility of reusing their simulation investment to create powerful BAM systems exciting.

70

BUSINESS ACTIVITY MONITORING AND SIMULATION CACI, for example, has used simulation in the loop for real-time decision making for customers such as the Department of Defense. For example, DOD training systems are a good example of using simulations along with feeds from of real-time operational systems to create scenarios for training purposes. The result is a hybrid tool that operates partially off validated probabilistic models along with real world events at real time. This concept, when extended to more mainstream solutions such as BAM, opens up some interesting ways of significantly improving the benefits of BAM (especially if process modeling investments have already been made with BPA models). 6. A Case Study If we consider the hospital example described early in this paper, we can get an idea of how this capability could be put to practical use. The process simulation models were already built as part of a business process improvement project and were extended to serve as a key ingredient in a simulationbased BAM solution. Let’s imagine a scenario of where this particular hospital—which has operational information systems that capture the key events in the business activities such as patient sign-in, capture initial triage data, resource clockin/out, and post treatment data capture—can actually provide data to the simulation models in real-time. The data feeds can be done at periodic intervals to get a “look ahead” on the impacts to patient treatment cycle time based on the current resources, patients received, expected future patient arrivals (see section 5 for simulation of patient arrivals based on empirical data), and the validated standard processes documented in the simulation modeling tool. Remember, the patient inter-arrival rates were based on probability distributions, as well as the activity delays and resource assignments. These probability distributions provided us a powerful tool to mimic real world dynamics; however, if an unexpected peak occurs in any of the variables, the probability distribution function chosen may not have predicted a worse-case situation. These spikes may be statistically insignificant over a long period of time but it could skew the performance of the business for several days or weeks, impacting the business goals as well as the P&L. When using the actual real world data feed along with the probability distribution for future expected events, we get a hybrid model that is based on both real and simulated data. This is technically a pre-load of the model with current data while the simulation will used expected events as it simulates to the future, for example: two weeks, one month, six months, etc. The result is the capability to alarm management with data to make decisions (such as call in temporary resources or change to a crisis process) based on the simulated future. One might ask what the benefits of this would be over traditional BAM type dashboards. The difference is that the current data and rules associated with alerting management from the traditional BAM solution may see only gradual changes in the variables of the business performance and may not have enough insight into the long range impact to know to alert management. With the simulation-based BAM, we add another dimension to the BAM solution. We can look several weeks or months into the future and predict what impacts may be experienced in overall key business performance measurements while there is still time to affect those impacts. The hospital example we used in this paper was based on the customer’s very important business metric of average patient treatment cycle time. If certain run-time

71

BUSINESS ACTIVITY MONITORING AND SIMULATION events drive the average up, due to unforeseen peaks, the hospital loses its ability to use its performance metric as a marketing tool. There are many other metrics that could be looked at, such as efficient resource utilization. In many cases, staffs are overworked and exhausted before management is aware of it, and exhausted staffs can create even more business performance problems such as quality of service. Another example situation that could arise in the hospital model is if a flu outbreak starts an upward spike of Level 1 and Level 2 patient types that are normally not expected. Since Level 1 treatments can disrupt the primary process flow due to priority of patient care, and Level 2 treatment takes longer on average to administer, the BPA model would not have uncovered those scenarios unless an explicit “what if” experiment were run. With the simulation-based BAM solution, the actual data affects the simulation and, in theory, “reprograms” the simulation and gives a better future picture of the organizational performance. The proposed simulation-based BAM does not have to have user intervention as in BPA simulation analysis. Once the model is built, it runs completely in the background and presents management dashboard data that is in a business metrics form. However, another consideration is to let the simulation run through multiple “fall back” or “crisis” process alternatives based on the real-time data, and present the user with choices that would then, in turn, be related back into BPM solutions for temporary process adjustments. Figure 13 is an example of the type of performance metrics that could be placed on a management dashboard. The dashboard in this example is showing both the real time data that a traditional BAM would provide in the top half of Figure 13. The bottom half is a simulation of five days into the future using the validated business process model.

Figure 13. Management Dashboard

The dashboard information is described as follows: • The actual real-time reporting (traditional BAM scenario) of business information is depicted in the top half of Figure 13 and includes the following graphical gadgets: • A meter to the left of the dashboard that depicts the running average of patients per day for the past month. The colors on the rim of the meter

72

BUSINESS ACTIVITY MONITORING AND SIMULATION are used to help the manager quickly see the numbers that represent critical values (i.e. red indicates a critical situation). • A bar graph that depicts the total patients for the last month for each type of patient. • A thermometer gadget that depicts the utilization of treatment rooms. The fill in the thermometer changes colors to indicate critical values. • A trace plot on the far right that depicts the average treatment cycle time of all patients over the last month. • Two text values that depict the current levels of the most critical resources—nurses and physicians. • The text field to indicate the date of the dashboard information. • A text field that indicates the average treatment cycle time up to the last hour of reporting. This is important to see the difference from the 30-day average and the last hour or so of activity. This field is used in conjunction with the next field. • A text field that is used with the previous field and indicates the average treatment cycle time for the previous 24 hours in this example. Obviously, this time span can be set to a wider range based on the business being monitored. • The projected (simulated) business information is depicted in the bottom half of Figure 13 for five days into the future (Feb 5th in this example) and includes the following graphical gadgets: • A meter to the left of the dashboard that depicts the simulated average of patients per day up through February 5th. • A bar graph that depicts the simulated total patients expected based on the past month and the probability distribution function in the model. • A thermometer gadget that depicts the predicted utilization of treatment rooms. • A trace plot on the far right that depicts the predicted average treatment cycle time of all patients up through February 5th. • Two text values that depict the predicted levels of the most critical resources—nurses and physicians. • The text field to indicate the date of the simulation for the dashboard information (February 5th in this example). • A text field that indicates the expected average treatment cycle time for the last 24 hours of actual data and up through February 5th of simulated data. The data reported in Figure 13 was based on certain events occurring in real time and as can be seen from comparing the data, the impact is minimal and probably not alarming to management based on the traditional BAM data. However, when the simulated data up through February 5th is provided, you can see the urgency of the problem as the key business metrics are affected. For example, the average treatment cycle time goes from 2.14 hours average over the last month to 2.83 hours for the last 24 hours of real-time data up through February 5th. An interesting piece of information is that if you focus only on the last 24 hours in the actual data (2.09 hours average which is less than the monthly average), you can see how not including simulated data can delay alarming management to a building problem. The events that caused the situation to become critical are as follows:

73

BUSINESS ACTIVITY MONITORING AND SIMULATION •

Starting on January 15th the average arrival of patients increased by 6 per hour. This was due to a flu and cold outbreak. • On January 30th, one of the nurses became ill and had to take emergency leave for up to 10 days. • On January 31st at 4:00pm, one of the physicians and one additional nurse had to take emergency leave due to exhaustion and illness. Notice the number of nurse and physician resources on the dashboard have decreased from 7 and 3 to 5 and 2 respectively. • An additional 10 patients arrived on January 30th due to two separate accidents. • An additional 20 patients arrived on January 31st at 11:00am due to multiple accidents. Since the traditional BAM reporting does not simulate ahead to predict the queuing theory problems of the growing arrival rates and the drop in resources, as of 5:00pm on January 31st the actual data is not showing any major impact to the business metrics. The simulated BAM data, however, predicts significant problems growing rapidly over the next several days and risks driving the average patient treatment time significantly above the goals set by management. If the problem is allowed to spike the treatment time, it could take a couple months of improved performance to pull the average back into line with management goals. Simulation-based BAM solutions are achievable today. CACI’s model-viewcontroller (MVC) architecture provides for separation of the simulation from the front-end analysis tool, therefore providing a server-based, GUI-less, and scalable capability. It provides connectors to outside applications through Java-based remote calls and/or XML, which are needed to feed the real-time data to the simulation tool. The example above is easily implemented with any vendor operational application with simple messaging capability. 7. Conclusion Gartner suggests that BAM will become a major corporate concern in the next few years. Most large organizations will at least explore the possibility of improving business process management by creating systems that provide a broad overview of a process that can provide near-real-time information and advice to process managers. A variety of techniques will be used. Some “BAM” systems will, in fact, monitor subprocesses. Some will use rules to alert managers about specific real-time problems. Some will be based on simulation engines and use models that allow the system to project future events from the current state of the process and then dynamically generate alternative options to identify what changes, taken today, would maintain the process in the most efficient manner over the long term. We believe that simulation-based BAM will prove to be the most powerful and flexible approach to BAM and will increasingly be relied on by those with the more complex processes. Notes [1] Gartner Group. Business Activity Monitoring: The Data Perspective. February 20, 2002. SIMPROCESS® is a registered Trademark of CACI. For more information about SIMPROCESS®, check www.simprocess.com Although not emphasized in this article, SIMPROCESS produces XPDL, the WfMC’s XML process language.

74

Business Process Improvement through Optimization of its Structural Properties Vladimír Modrák, Technical University of Košice, Slovakia PROCESS IMPROVEMENT THROUGH PROCESS CHANGES With the growing requirements for the improvement of business activities within organizations, aspects of changes and new concepts of process structures are becoming a topical problem. These aspects are evenly important from standpoint objectives of the first out of two phases of workflow management (WfM). The processes of change have been addressed mostly at the level of administrative business processes (BP). Such situation partially accords with a logical succession of reengineering framework steps and corresponds with advancements of information technology infrastructures. On the other hand, less attention has been devoted to the changes at the shop-floor level. Despite this, the redesign of the shop floor infrastructures is equally legitimated. Approaches to the business process improvement (BPI) can generally be divided into two categories: improvement of the operational properties of BP and improvement of the structural properties of BP. Both of them are useful for the administrative and manufacturing BP1 While the first approach is oriented on dynamic parameters of BP, the approach based on structural analysis deals with the static properties of BP. This chapter is concerned with the second category of BP properties from the point of view of measuring and benchmarking. Its main scope is to present a practicable approach to the structural complexity measurement of business processes.

MODELING OF BUSINESS PROCESS STRUCTURES Classification concept of business process structures One of the important roles of Business Process Reengineering (BPR) should be building a logistical concept of organization, which should involve coordination and management of all material and information flows. Especially higher-level modeling of business process structures substantially support the company’s successful running. Such process models structures normally start by establishing a framework for the systematic classification of company processes. A classification framework for the systematic rebuilding of processes can be built from three hierarchical levels, which are (bottom to top)[18]: • Elementary process (EP) represented by a set of complex tasks, consisting of the smallest elements-activities;

1 1 1Existing

distinctions between administrative BP and production BP resulted in the creation of different concepts of administrative and production workflow systems (see for example [14],[17]).

75

BUSINESS PROCESS IMPROVEMENT •

Integrated process (IP) which represents a set of two or more elementary processes with the purpose to create the autonomic organizational unit at the second hierarchical level;

•

Unified enterprise process (UEP), which consists of one or several integrated processes to the extent that is conditioned by its capability to flexibly and effectively secure customers’ requirements. The application of classification approach in the modeling of business processes structures is further shown by the process mapping technique that was inspired by other methods for analyzing and designing of information systems [1], [2], [8] [15]. Business process mapping Our process mapping technique is based on process decomposition that is resulting in a set of business structure models, which are represented by diagrams in the order given: system diagram, context diagram, commodity flow diagrams, state transition diagrams. An example of the first three simplified diagrams is illustrated on Figure 1.

Figure 1. A fragment of the process map described by the hierarchical diagrams According to the procedure outlined for redesigning enterprise processes, the first step of this method is the creation of a System Diagram. Its purpose is to separate so-called Unified Enterprise Processes (UEP) from the original arrangement of processes. Subsequently, relations between them and the

76

BUSINESS PROCESS IMPROVEMENT environment of the enterprise are specified. The environment is represented in the diagram by External Entities (EE), with which the system communicates, while their content is not a subject of analysis in the following steps. They usually represent the initial source of commodity, or their end consumer. In the fact it represents the starting base of modeling processes, from which other diagrams are derived using the principle of process decomposition. Context Diagrams are created for each UEP on the basis of the System Diagram. Individual Context Diagrams express relations of the given Unified Enterprise Process with its environment. These surrounding elements, irrespective of whether they represent objects outside the enterprise or internal processes, are approached as External Entities. It means that External Entities are considered in the same way as internal processes of the System Diagram. System decomposition at the given level emphasizes the need for the creation of an equal customer approach in which there should not be differences between internal and external customers. The essence of the Commodity Flow Diagrams is gradual decomposition of UEP, up to the level of so-called elementary or primitive processes. The diagrams at the first stage of decomposition start from Context Diagrams. In the sense of the proposed classification, the mutual links of Integrated Processes (IP) are described in them. Despite that, the Commodity Flow Diagrams do not provide any details about modeled processes; their purpose is to provide an general overview of the follow-up of processes, which allows their owners at different levels to see the boundaries of their own as well as subsequent processes. Commodity Flow Diagrams of the second stage are constructed in an analogous way as Commodity Flow Diagrams of the first stage. It is the last stage of commodity flow diagrams because the elementary processes, which present the objects of modeling, are considered to be the primitive processes. The objective of the State Transition Diagram is the description of the dynamics of elementary processes by modeling states in which objects can be presented and transition periods between these states. These diagrams also describe events that initiate transitions between states and conditions for the realization of these transitions. A comprehensive view of the business processes structures in the sense of the approach outlined helps to make the process models transparent and usable resulting in workflow models. Without a classification framework for defining the basic types of processes, the overall view of the business processes structures could become disorganized. Structuring of business process models Models of business process structures are often used in very different ways. Technically, horizontal structuring and vertical structuring are mostly recognized [10]. Horizontal structuring helps to handle each customer engagement with guaranteed service. Vertical structuring serves to distinguish different levels of details, or abstraction levels. In the first stage of our attempt to formulate and apply metrics for BPI in more detail, the issues of the horizontal structuring will be addressed. Subsequently, evaluation of the structural properties of the vertical levels, based on degree of structure centralization, will be outlined. According to Franken et al [7] models of business processes can be divided into extensive and intensive model structures. In extensive model structures of the business processes are the processes described as an integrated whole. With such a model, investigation is not concentrated on internal process structure and its behavior. An extensive

77

BUSINESS PROCESS IMPROVEMENT model describes the process from the viewpoint of its environment, which usually consists of external or internal suppliers and customers. From the business structure models that have been outlined by the process mapping technique two classes of diagrams, the system diagram and the context diagram, belong to this category of business process models. The above-mentioned diagrams are pertinent to the evaluation of the structural properties based on vertical structured models. On the other hand, the intensive structures models of business processes are used to describe the interactions of entities inside the part of the system that is being investigated. Such models describe business processes from the viewpoint of their internal objects, sources and other entities such as the staff, technological components, protocols, etc. The rest of business structure models: commodity flow diagrams, state transition diagrams from the process mapping technique can be incorporated into this category of business process models. Intensive models of processes structures can be additionally divided into: • Workflow-oriented model structures, which represent the behavior of a business process from the perspective of a single item that passes through the process. •

Functional-oriented model structures that reveal the behavior of functional units (departments) from the viewpoint of subsequent business functions. These models represent the obsolete functional approach of organization structuring. The commodity flow diagram, represent ting the first sub-category of models, will be subjected to the evaluation of structural properties using a set of proposed indicators in subsequent paragraphs.

ATRIBUTES OF PROCESS STRUCTURES AND THEIR MEASUREMENT Process structure modeling by commodity flows The business process can be understood in general as sets of logically related, structured activities that produce a specific service or product for a particular customer or customers. Depending on the medium that is the subject of the operation, it is possible to differentiate between material, information or energy processes. On the basis of the above mentioned, it is possible to analyze these processes individually or in a complex way. In the application of reengineering, the attention is focused especially on the effectiveness of the key business processes, and only later on processes of lower significance are taken into account. Inputs of such processes are usually material and/or information items—simply marked as commodities. They are re-transformed in the framework of processes into new commodities, through which the system gets closer to its goal. Having in mind that aspect, the process or its parts can be understood as a sequence of states, from the initial state to the final one, in which the process is completed. Commodity flows are usually represented by material flows, from their sources to the places, where they are consumed. Commodity flows are often split up in complicated ways into the places of their transformation, and they are concentrated into the places of their finalization and consumption. The main task of the modeling processes as commodity flows is to relate the basic objectives, resources, and limitations of the system. In line with that,

78

BUSINESS PROCESS IMPROVEMENT such models should allow for analysis for completeness, absence of conflicts, viability and effectiveness of individual functions. Modeling on this basis can be considered as decisive in terms of the aims of BPR and can be useful for various purposes. Some of them are: •

Examination of such models from the point of view of effectiveness and efficiency. Analysis of process models oriented toward the model improvement is usually based on process simulation [13].

•

Process improvement with the aim to utilize the results of process model analysis in order to increase process flexibility or/and productivity [19].

•

Process enactment. Business process models can be used for driving real processes in workflow-oriented style [4]

•

Process improvement based on optimization of its structural properties. Overview of the approaches to the measurement of BP structural properties BP structural analysis and assessment methods were emphasized using various perspectives by researchers during the last few years. It is an opportune moment for the recognition of traditional structural metrics oriented to process efficiency and new kinds of structural metrics, which deal with process effectiveness. Metrics for effectiveness should be primarily focused on structural properties with ability to influence outcomes of BP. Such structural properties of general interest can be relatively easily extracted from reengineering principles. They include such properties as simplicity, integrity, flexibility, (de) centralization, viability and others. Aggregated structural metrics of BP consisting of three of these properties Simplicity, Integration and Flexibility have been developed by Tjaden [24]. Measuring the process simplicity versus process complexity is based on enumeration of activities, material flows and number of persons performing activities. To quantify the relative degree of flexibility of a process a set of so called Flexibility Properties are used. Thesis similar to the ideas concerning process flexibility was developed earlier by Hayes et al [9]. The process integrity in this approach has to be first measured separately for each material flow and subsequently these individual values are combined into overall value for a set of activities. This aggregated approach in given segment of attributes contributes to the metrics of BP structural properties. Vanhoucke et al [26] performed a study on morphological and topological indicators of a network that can be used for testing software packages to generate networks, which could also be applied to business processes metrics. The aspects of process complexity and position of static models of BP structures were analyzed by Fathee et al [6]. The influence of process complexity on the use of static process model was stressed in their work. Research on implementing and comparison the potential complexity measures was presented by Latva-Koivisto [13]. In his work wide ranging alternative complexity measures such as Coefficient of Network Complexity, Cyclomatic Number, Complexity Index, Restrictiveness estimator (RT) and Numbers of the trees in a graph (T) are employed. Two of them namely “RT” and “T” will be used in the next section for the testing of distinct complexity indicators on different process graphs.

79

BUSINESS PROCESS IMPROVEMENT EVALUATION OF STRUCTURAL PROPERTIES FOR BUSINESS PROCESSES ON A HORIZONTAL LEVEL Theoretical basis and methodology Business processes are usually modeled as process charts, in which the activities that form the process and dependencies between them are composed. In the case of process models using commodity flows, the first activity creates output that is used as an input for the second activity. Within the outlined BP modeling method by the process mapping technique (see Figure 1), individual diagrams were designed based on the principles of the graph theory. The fundamental concept of graph theory is the graph G = (V, E) that consists of a set of vertices V(G) together with a set of edges E(G). The number of vertices in a graph will be denoted n while the number of edges will be denoted m. Each link connects two points, which are its endpoints. The points connected by a line are said to be adjacent. In contrast, a link that has a given point as an endpoint is said to be incident upon that point. Two lines that share an endpoint are also said to be incident [3]. In the proposed approach the structural properties of BP will be investigated by means of a topological analysis in terms to which the basic elements of the process structure—vertices (or nodes) and edges (or links) are subjected. The starting point of this analysis is an adjacency matrix of a graph, which is a binary n x n matrix A in which aij = 1 and aji = 1 if vertex vi is adjacent to vertex vj , and aij = 0 and aji = 0 otherwise. At the beginning we will convert a process map represented by the set of above-mentioned diagrams to directed graphs, in which the flow of movement follows the direction of the edges. The Digraphs will be composed as activity-on-node (AoN), in which the activities of the processes are modeled by nodes and dependencies as links. Since all the initial parameters of the investigated graphs are known with certainty, the problem is deterministic. As regards the basic research methodology that has been used, the motivation of research direction has been based on the application of general axioms of graph theory for specified problem domains, while extraction of findings has some features of a partially inductive approach. Determination of the indicators for process complexity The most frequent structure attribute of business processes in the scope of BPR is undoubtedly complexity. Scofield [21] describes business models and enterprises that tend to be more complex than anyone wants. He adds that an organization must understand the causes of enterprise complexity in order to build better models and longer-lasting architectures. This section outlines a possible way for a better understanding of causes of process complexity by treating the potential aggregate indicator of process complexity that consists of three separate sub-indicators, which are Binding of structure, Diameter of network and Structure diversity. Determined indicators are selected for investigating processes on the horizontal level. Binding of structure As one of many possible indicators of the complexity of the business processes structures the ‘redundancy measure index‘ of the structure linkage will be considered. This is based on the concept of graph binding. This term means the least possible number of linkage graphs, the reduction of which would lead to an incomplete graph containing isolated nodes. An incomplete graph is the opposite of a graph in which all vertices are adjacent to all oth-

80

BUSINESS PROCESS IMPROVEMENT ers. The minimum number of edges for graph binding is n—1. That is valid for both digraphs and non-directed graphs. Within digraphs, each link (i,j) has one element in the adjacent matrix aij = 1. Within the non-directed graph, each edge has two elements, which is valid for aij = aji To determine the measure of the binding structure, the following indicator expressing a relative measure of the size of the number of the “m” edges that occur within a given structure can be applied:

B=

m −1 n −1

(1)

With the minimum number of links, the value of this relation equals zero. In connection with the above described processes modeling technique, the indicator of the structure binding can be used in the analysis of the processes of the UEP type to interpreting the internal structure of integrated processes (IP). For this purpose we will convert commodity flow diagrams of the first decomposition stage from Figure 1 to the process structure that is described in Figure 2a. For process analysis by this indicator, only the internal structure of the investigated process shown in Figure 2b is relevant excluding the relations of the process to its immediate environment. The index value of the structure binding “B” of a given process, when m = 15 and mmin = 7 by formula (1) equals 1.14. The reduction of the structure linkages in conformity with the principles of reengineering can be obtained by the purposeful integration of compatible processes either sequentially or parallel. In this case, the integration of the processes IP31 all the way to IP 34 will be represented through the process IP 31-4 and similarly IP 37 and IP 38 will be integrated to the process IP 37-8. The new process structure is shown in Figure 2c. The index value obtained by this transformation will be reduced to B = 0. 67.

Figure 2. An example of the internal process structure reduction Diameter of network The diameter of a network or a graph’s diameter also appears to be a pertinent indicator for the comparison of the complexity of the business processes structures. This indicator is commonly defined in the graph theory as the longest shortest path in the network. That is, if the length (in point-topoint hops) of the shortest path between i and j is Li,j, then the diameter of the network, directed or undirected graph, L = maxi,j(Li,j). When applying this indicator to the processes at the same level as in the previous case, it is reasonable to test structures including those elements of

81

BUSINESS PROCESS IMPROVEMENT the environment, which interact directly with the elements of the internal structure of the process. The process analysis, using this indicator, will consider the initial state of the original structure of the UEP3 process that is shown in Figure 3a.

Figure 3. The network structures before and after integration of selected processes If we assume that “I ” represents a set of output nodes (from which the path is initiated) and “J” represents a set of input nodes (in which the performance of the network finishes) then for I={2’, 3’, 5’, 6’, 8’} and J={2’’, 3’’, 5’’, 6’’, 8’’} the following distance matrix can be created:

The matrix for case a) results in max Lij = L2’6’’ = L3’2’’ = L3’3’’ = L3’5’’ = L5’6’’ =L6’6’’ = 5, thus L = 5. A more favorable value of this indicator for the given process can be obtained by the purposeful integration, which is again represented by the joining of the sequentially arranged processes IP 37 and IP 38 into the process IP 37-8 and by the joining of the processes from IP31 all the way to IP34 into the process IP 31-4. Through such a modification of the process structure, the network structure presented in Figure 3b can be obtained. Then the Diameter of network for the changed structure will be calculated in the same way by distance matrix where the numbers of the graph nodes “I” and “J” remaining in an original state: Based on this, the new measured value L equals 4; this means that according to this indicator the business process complexity is reduced.

STRUCTURE DIVERSITY The formalization of the process structure diversity is based on the supposition that the investigated network structure can be represented as a transformation process of input effects into output ones encompassing distribu-

82

BUSINESS PROCESS IMPROVEMENT tion activities. When determining one of the possible indicators of the process structure complexity, it is supposed that the more heterogeneous transition paths from input nodes to output nodes imply a more complex process structure. Based on these suppositions, a measure of the degree of structure diversity can be assessed by the following indicator:

1 D= n1 .n2

n1

n2

i =1

j =1

∑ ∑c

ij

−1

(2)

in which “n1, n2” are numbers of initial and final nodes of the process structure and “cij” represents number of heterogeneous paths from the i-th input node to the j-th output node of the process (without any possibility to pass twice through the same node within one route). If the process structure does not contain alternative transition ways from input nodes to output nodes within the structure, then the structure diversity indicator D = 0. In order to apply the structure diversity indicator to the original structure of the UEP3 process shown in Figure 3a and to that obtained after the integration of the processes (Figure 3b), the values cij which are given in the following matrices, are determined first. Applying the values cij in the equation 2, the following figures of the indicators D are obtained: • D = (0.04 * 86) -1 = 2.44 • D = (0.04 * 29) -1 = 0.16 Aggregate indicator of process complexity In order to compare structural properties of the same process before and after the (re) design it is useful to combine the above sub-indicators “B”, “L”, “D” of process structure complexity into an aggregate indicator. Figure 4 shows the required changes of the structural properties of the business processes to achieve the optimization.

Figure 4. Comparison of structural properties before and after the BP (re) design

The individual complexity values of the sub-indicators “B”, “D”, “L” are expressed as the proportional reductions in Table 1. Based on this, the average proportional reduction of complexity equals 51.56 %.

83

BUSINESS PROCESS IMPROVEMENT Table 1 Sub-indicators of procThe index ess structure complexity values before (re) design

The index values after (re) design

Proportional reduction of complexity

Binding of structure “B” 1.14

0.67

41.23 %

Structure diversity

0.16

93.44 %

4

20.00 %

“D” 2.44

Diameter of network “L”

5

Establishing an Aggregate complexity indicator “AC” by using above subindicators means finding a formula, which expresses the composite proportional reduction of complexity of the business process structural properties. The following expression for an Aggregate complexity indicator can be formulated:

AC = log(( B + L + D ) / 3)

(3)

Using equation 3, the value of proportional reduction of complexity in the above case between before and after (re) design, is calculated as 54.61%.

COMPARISON OF PROCESS COMPLEXITY INDICATORS This section considers the comparison of the complexity measures for describing the structural attributes of a business process models. Alternatives measures to above indicators are the Restrictiveness estimator and the Numbers of the trees in a graph.

ALTERNATIVE INDICATORS OF PROCESS COMPLEXITY Restrictiveness estimator This indicator was originally defined and presented by Thesen [23] and applied to project networks measurements. According to De Reyck [5] the Restrictiveness estimator (RT) is practically the same measure as Order strength, which was defined by Mastor [16]. t is defined as the number of existing direct and indirect precedence relations divided by the theoretical maximum number direct and indirect precedence relations and therefore ranges from 0 to 1. Later Schwindt [20] applied this indicator to measure network complexity. Formally RT is expressed by the equation:

RT =

2∑ rij − 6(n − 1) (n − 2)(n − 3)

(4)

where rij is an element of the reachability matrix R = [rij ], such that rij =1 if there is a path from the vertex vi to vj , otherwise rij = 0, and n is a number of vertices in a graph. According to Latva-Koivisto [12], the RT indicator could also be used for cyclic graphs, even though this measure was not originally specified for cyclic graphs. Because the previously analysed graphs in Figure 3 belong to cyclical graphs, this assumption will be verified for these two cases of network infrastructure. Number of trees in a graph The second indicator that appears to be potent as alternative indicators of process complexity is the Number of trees in a graph (T). This indicator, based on the supposition that the number of distinct trees of graph reflects

84

BUSINESS PROCESS IMPROVEMENT the complexity of a graph, was developed by Temperley [22]. According to Latva-Koivisto this indicator, along with the Restrictiveness estimator, are potential complexity measures for business process models. Hence the decision to compare them with the proposed indicators. The Number of trees in a graph indicator is calculated using so called treegenerating determinant, which is defined for any graph containing no slings, and not more than one undirected or two directed lines joining a pair of points. The formal expression of this index is rather complex. Temperley and Latva-Koivisto deal with the relevant theoretical comments on this indicator and the calculation details. Comparison of different indicators In order to assess the relevance of the proposed sub-indicators and the AC indicator for the selected complexities they have been evaluated for a set of selected graphs. The group of graphs has been designed to reflect the typical variety of business process structures. For that reason previously analysed graphs of a more general form, together with the graphs used by Kaiman [11], and Latva-Koivisto, have been applied. The first two graphs, shown in Figure 5, represent the real-life process graphs, while other graphs were based on artificial process graphs. In total 8 graphs have been used in the order that does not completely depend on their expected complexity. The graphs are shown in Figures 5 to 10.

Figure 5. Real-life process graphs: Graph 1 (b), Graph 2 (a)

Figure 6. Latva-Koivisto’s process graphs: Graph 3 (a), Graph 4 (b)

85

BUSINESS PROCESS IMPROVEMENT

Figure 7. Kaiman’s process graph; Graph 5

Figure 8. Kaiman’s process graph; Graph 6

Figure 9. Kaiman’s process graph; Graph 7

Figure 10. Kaiman’s process graph; Graph 8 Summary of alternative measures Table 2 shows the results of the implementation of complexity indicators. The empty boxes in the table for the RT measure in the case of Graphs 1 and 2 are there because the values of that measure did not fall within the value interval [0,1]. Specifically, the value 1.095 was calculated for Graph 1 and 1.067 for Graph 2. Table 2 Graph No.

Number of edges

Number of nodes

Complexity subindicators B

L

D

AC

RT

T

1

10

9

0.67

4

0.16

0.207

-

12

2

28

18

1.14

5

2.44

0.456

-

1280

86

BUSINESS PROCESS IMPROVEMENT 3

15

10

0.67

3

6

0.508

0.250

7

4

15

10

0.67

3

7

0.551

0.536

48

5

30

22

0.43

8

21

0.992

0.611

384

6

31

22

0.48

9

50

1.297

0.789

768

7

29

22

0.38

10

44

1.258

0.884

256

8

22

22

0.05

19

1

0.825

0.989

2

On the basis of the evaluation of Table 2, it is possible to state the following: • The values of indicators AC and RT for Graphs 3 through 6 show comparable tendencies. For graphs 7 and 8 the trend is different. •

The T indicator does not provide a real difference in values in the case of Graphs 1 and 2. The difference between the values is too large.

•

The values of the indicators AC and T for Graphs 3—8 have approximately the same tendency, even in the case of Graph 7, whose complexity is apparently lower than in the case of Graph 6. That could signal some advantages for the indicator AC compared to RT.

•

The indicator T is apparently not applicable for cyclical graphs with multiple input and output nodes. The above analysis implies that the parallel use of alternative indicators may lead to an objective evaluation of the evaluation of the structural properties of BP in terms of their optimization. On the other hand, the results confirm the more or less known fact that a universal measure for the assessment of the complexity of process structures is probably not attainable. The complexity of the structures should therefore be assessed on the basis of predefined criteria, which differ for real processes according to their purpose. The proposed indicator AC, from this perspective, can be considered as a specific indicator for the assessment of the complexity of BP, based on the principles of reengineering. For that reason it may be used effectively in the optimization of structural properties of business processes.

EVALUATION OF STRUCTURAL PROPERTIES FOR BUSINESS PROCESSES ON A VERTICAL LEVEL The previous paragraph dealt with the analysis of the structural properties of the business processes on a horizontal level. This enables the comparison of various processes of the same class, or the improvements of processes after being redesigned. The presented example of the analysis is related to the latter possibility. The following section describes the procedure of the analysis of structural properties based on the principles of reengineering, where the criteria of suitability at the individual levels of the process models are different. With an analysis of this type, it is possible to consider the measure of irregular loading of the structure elements by means of the following indicator. Degree of the Structure Centralization In order to define the degree of centralization of the process structure, it is possible to use the so-called index of centralization “α”, which expresses the measure of (de) centralization of the structure elements. The index value can be obtained by means of the relation:

87

BUSINESS PROCESS IMPROVEMENT

α=

n 1 ∑ (V (k ) − V (i)), ( n − 1)(V (k ) − 1) i =1

(5)

i≠k

where V(i) = vi + vi expresses the total number of input edges and output edges of the i-th node, V(k) = max V(i). Index “α” can obtain two limiting values: • α = 1 in the case that the structure is centralized to the maximum degree •

α = 0 in the case that the structure is decentralized to the maximum degree This index will be used in the following example in order to evaluate the level of the degree of centralization of the modeled process structure based on the first three stages of the structured process decomposition shown in Figure 1. The following extracted process structures are analyzed: • System diagram, •

Context diagram for the UEP3 process, and the

• Commodity flow diagram of the 1st stage for the same process. Directed graphs will be used to determine the internal structure of the given process (the vertices without gray shading). The block diagrams of these processes are summarized in Table 3 together with the relevant characteristics. Table 3

88

BUSINESS PROCESS IMPROVEMENT When this indicator is used for the evaluation of structural attributes of the business processes according to the above- mentioned process mapping technique, it is necessary to use different criteria: • the system diagram structure should have a decentralized structure to the maximum degree, •

the context diagram structure should have a strong centralized structure,

•

and commodity flow diagrams should have significantly decentralized structure.

CONCLUSION

Apart from main intention to present a practicable approach to the measurement of the structural complexity of business processes, the chapter also outlined some conceptual aspects of the effectiveness of the creation of practical tools for business processes redesign consisting of modeling and subsequent analyzing of processes structural attributes. The modeling procedure presented based on structural process decomposition and has been verified by the solution of many practical problems of process redesign. Based on experience, this stage is the key to the way of successful enterprise reengineering and the subsequent workflow management. Undoubtedly the position of business process metrics is of major importance for both stationary and dynamic properties of the process structures.

REFERENCES: [1] Ashworth, C. and Goodland, M. SSADM A Practical Approach, McGrawHill, 1990. [ 2] Avison, D.E and Fitzgerald, G. Information Systems Development, Blackwell, 1991. [3] Borgatti, S.P. Graph theory, http:/www.analytictech.com/networks/graphtheorychap.doc Accessed December, 2004. [4] Deiters, W. Gruhn, V. Process management in practice- Applying the FUNSOFT net approach to large scale Processes, In: Special issue on process technology/Automated Software Engineering 5, 1998, 7-25. [5] De Reyck, B. On the Use of the restrictiveness as a measure of Complexity for Resource-Constrained Project Scheduling. Research Report 9535, Department of Applied Economics, Katholieke Universiteit Leuven, Belgium, 1995. [6] Fathee, M. M. The effects of complexity on business processes reengineering: values and limitations of modeling and simulation technologies, In Proceedings of the 30th conference on Winter simulation, 1998, 13391345. [7] Franken, H.M., de Weger, M.K. and Jonkers, H. Structural and quantitative perspectives on business process modelling and analysis. In: Proceedings of the 11th European Simulation Multiconference, Istanbul, June 1-4, 1997. Society for Computer Simulation International, Ghent, Belgium, 1997, 595-599. [8] Gane, C. and Sarson, E. Structured Systems Analysis: Tools and Techniques, Prentice Hall International, Inc., 1978. [9] Hayes, R., Wheelwright, S., & Clark, K. Dynamic Manufacturing. New York, NY: The Free Press, 1988.

89

BUSINESS PROCESS IMPROVEMENT [10] Jonkers, H. The application of hybrid modeling techniques for business process performance analysis In: A.R. Kaylan and A. Lehmann (eds.), Proc. 11th European Simulation Multiconference, Istanbul, Turkey, June 1997, 779-786. [11] Kaimann, R.A. Coefficient of Network Complexity and Coefficient of Network Complexity: Erratum. Management Science Vol. 21, No. 2, pp. 172177 and No. 10, pp. 1211-1212, October 1974. [12] Latva-Koivisto, A. Finding a complexity measure for business process models, Research Report, Helsinki University Of Technology, February , 2001,1-25 http://www.hut.fi/~alatvako/Kompleksisuus-erikoistyo_2001-0213.PD Accessed December, 2004. [13] Law, A. M. and Kelton, W. D. Simulation Modeling & Analysis (Second Edition), McGraw-Hill, 1991. [14] Liu, L., Pu, C., Ruiz, A. A systematic approach to flexible specification, composition, and restructuring of workflow activities. Journal of Database Management, 15 (1), 2004, 1-40. [15] Marca, D. A. SADT. Structured Analysis and Design Technique, McGrawHill. 1988. [16] Mastor, A. A. An experimental and comaparative evaluation of production line balancing techniques, Management Science 16, 1970, 728-746. [17] McReady, S. There is more than one kind of workflow software. Computerworld. November 2, 1992, 86–90. [18] Modrák, V. Evaluation of Structural Properties for Business Processes, In: 2004, {roceedings of the 6th International Conference on Enterprise Information Systems (ICEIS), Porto, 2004, 619-622. [19] Paper, D. and Dickinson, S. A Comprehensive Process Improvement Methodology: Experiences at Caterpillar's Mossville Engine Center (MEC), In: Cases on Information technology management, Idea Group Publishing, 1997 [20] Paulk M.C., Curtis B., Chrissis M.B. and Weber, C.V. Capability Maturity Model SM for Software, Version 1.1, Technical Report CMU/SEI-93-TR-024 ESC-TR-93-177, February 1993. [21] Schwindt, C. A new problem generator for different resource-constrained project scheduling problems with minimal an maximal time lags, WIOR – Report-449, University of Karlsruhe, 1995 [22] Scofield, M. Enterprise models: Anticipating Complexity, Enterprise Reengineering, Reengineering Resource Center Jan./Feb., 1996. [23] Temperley, H.M.V): Graph Theory and Applications. Ellis Horwood Ltd., England, 1981. [24] Thesen, A. heuristic scheduling of activities under resources and precedence restrictions, Management science 23, 1976, 412-422. [25] Tjaden, G.S. Business process structural analysis, Working paper, Georgia Tech Research Corp., October, 1999, 1-25. http://www.ces.btc.gatech.edu/report3.html Accessed December 2002 [26] Vanhoucke, M., Coelho, J., Tavares, L., Debels, D., "On the Morphological Structure of a Network", Working paper, October 2004. http://econpapers.hhs.se/paper/rugrugwps/04_2F272.htm, Accessed November, 2004.

90

Enhancing and Extending ERP Performance with an Automated Workflow System Robert J. Kearney, Image Integration Systems, Inc., USA A PRACTICAL VIEW OF THE BENEFITS OF WORKFLOW SYSTEMS IN ERP ENVIRONMENTS The effective integration of comprehensive, independent workflow systems with Enterprise Resource Planning systems can produce significant improvements in those business processes implemented with the ERP. This synergism fully delivers on the economies of scale promised for centralized ERP processing, while insuring and simplifying the requisite participation of the "expert," often decentralized, knowledge workers. Practical limitations of most ERP systems are considered, as is the manner in which automated workflow overcomes those limitations to effect greater business process improvement. The results at two businesses are considered, each of which benefited from faster, less costly and more controllable business processes.

ENTERPRISE RESOURCE PLANNING Over the last few decades, enterprise resource planning (ERP) changed and now defines the business processing landscape. Virtually every transactional business process is now “automated” by an ERP system, or the related systems for supply chain management (SCM) and customer relationship management (CRM). And the more recent advent of business-to-business (B2B) capabilities with the internet has, in some well-defined situations, extended ERP coverage across and beyond the borders of the enterprise. The efficiencies created continue to boost productivity. Without ERP the business models for many of today’s corporations would change dramatically, if in fact they could exist at all. ERP is a necessity for success in today’s business environment. The transactional volumes in sales, accounting, production, inventory, distribution and the like, that today’s businesses routinely process might be impossible otherwise. So in what way is the “promise” of ERP unfulfilled?

ERP OR ENTERPRISE TRANSACTIONAL PROCESSING Simply put, while ERP is necessary it is not sufficient. The implicit “promise” of ERP included: 1. standardization of the entire business process 2. a central repository of all relevant business process information 3. active, direct support of and participation by the business “experts” The shortfall against these expectations is principally due to the fact that for most organizations ERP is more accurately “ETP: enterprise transaction processing.” As such, the ERP system is not part of the business process or the associated information until a transaction is processed. Many, if not most, business processes are initiated by activities at the physical edge of the enterprise, but ERP does not usually extend that far. This part of the

91

ENHANCING AND EXTENDING ERP PERFORMANCE process, from the outside world to the ERP transaction, usually is standardized, but not by ERP. Most ERP transactional interfaces are necessarily designed for the data entry processor, not the business expert. Industries are replete with examples of organizational layers, changes and additions with the primary objective being to buffer the business expert from ERP, and sometimes, the ERP from them; the very people who need to see, create, approve, initiate and authorize the life-blood information of the enterprise. These business experts are often managers and supervisors of departments, functions or projects. But may also be those business knowledge workers in purchasing, production, sales administration, distribution, etc., on which the enterprise depends to operate effectively and profitably. The ERP system is certainly a great place to find discrete data, but only in the context of the transactional process. It wasn’t long ago that what is now Information Technology was justifiably called “data” processing. Most interaction with ERP still remains within the constraints of the transactional data processing interfaces. So for all the tremendous benefits that most ERP systems provide, they can not “internalize” much of the vital business process information; nor encourage and support the interaction of the business experts; nor automate and standardize entire business processes.

INDEPENDENT WORKFLOW SYSTEMS The development of workflow systems to automate and improve business processes has been, in part, driven by recognition of that which ERP systems can not provide well, or at all. In fact it is not unusual for businesses to implement an independent workflow system with, or immediately following implementation of a new ERP system. At the very least they recognize the potential for increased returns on their ERP investment by adding workflow to extend and enhance their new ERP after its use has stabilized. And for an increasing number of companies, new business processes are designed with the workflow system as a requisite partner with the new ERP system, and they are implemented together. Independent (i.e. not built into the ERP system) workflow products historically have been far easier to configure and use than workflow imbedded in the ERP. Their flexibility and orientation to the entire process, not just transaction processing, enable independent workflow products to be “business expert friendly.” The workflow engine is about the process; the ERP engine is about the transaction. Most ERP systems include at least some form of workflow, but it is either very tightly coupled with the ERP transactional processing or has only a subset of standard workflow functionality. In the first instance, development and maintenance of the workflow schema usually requires considerable efforts from specially trained and thus scarce and/or expensive IT development resources. In the second, the workflow is of very limited value and rarely capable of effecting entire processes. In neither case is it possible to respond quickly and effectively to the changing process requirements that characterize today’s business. Activity driven workflow systems have also been available for some time and, like ERP systems, are now mature and commonly accepted in a variety of business arenas. In ERP business environments they are now frequently be-

92

ENHANCING AND EXTENDING ERP PERFORMANCE ing used to support and supplement ERP capabilities. The workflow system complements the ERP in that it can begin the business process by recognizing an event at the edge of the enterprise, and subsequently manage the capture, creation and validation of transactional information through the process activities. The active participation of the business expert is supported with more usable, business-intuitive interfaces that need not be constrained by the requirements for ERP transactional data entry.

WORKFLOW: THE ERP DATA DELIVERY SYSTEM In a workflow-centric environment, ERP processing is just another automated activity in the workflow schema. Conversely, in an ERP-centric environment, workflow is the delivery mechanism by which the data required for transactional processing are provided. While ERP processing requires only data, the expert routinely requires knowledge of the business context within which these data exist in order to exercise his expertise. The initiating document often provides that, at the very least acting as the data “container.” With an improved perspective, informed operational decisions can be made: approve, order, pay, release, produce, etc., with each such decision ultimately leading to an ERP transaction. It is still common for the document from the world outside the ERP system to be paper, which is why “imaging” is often part of the business process improvement effort. From a processing perspective, it would be far more efficient if all business were transacted electronically, as B2B. But the “paperless” office remains an elusive target. That is not likely to change any time soon, and in fact the volume of paper generated by business continues to increase. By capturing the external document as an electronic object or image (whether from scanner, fax, email or web) the workflow system can better inform the expert, effectively internalizing that added information for both immediate and continuing use.

INTEGRATION OF WORKFLOW AND ERP SYSTEMS As opposed to stand-alone workflow systems, workflow in the ERP environment is specifically designed and structured to integrate and co-exist with ERP systems. Stated narrowly, the purpose of this integration is to enable workflow to deliver transactional data to the ERP system. Note also that this integration does not, and should not include transactional processing within the workflow system, as opposed to the ERP system(s). Any duplication creates the potential for conflicting and invalid results, subverting business process improvement at least. The ERP system is a powerful and finelytuned transaction processing machine that needs to be supported, not replaced. Sometimes this support is required for more than a single instance of one ERP system, and often for multiple, different ERP systems. A common business process can be established across these platforms by deploying a common workflow scheme that incorporates the ERP platforms as different processing activities. Effective integration of workflow with the ERP system will vary depending on the application, but often provides: 1. validation of new data against existing ERP records 2. extraction of related (process relevant) data from the ERP

93

ENHANCING AND EXTENDING ERP PERFORMANCE 3. automated presentation of the transactional data to the ERP data entry staff 4. automated naming (indexing) of all originating and supporting documents as part of the ERP transactional process 5. automated submission of transactional data directly to the ERP, through an ERP utility or edited interface 6. detection and response to changed status of a business condition in the ERP Items 1, 2 and 6 are implemented with customizable workflow agents (and/or exit programs) require the workflow system manufacturer have reasonable expertise with the particular ERP system. The effective integration for items 3, 4 and 5 commonly necessitates configurable integration modules, requiring considerable ERP expertise and often stringent testing for certification by the ERP manufacturer. This ensures that the workflow system does not in any way corrupt ERP transactional data or processing. The proper design and implementation of the workflow system in an ERP environment requires this level of integration to fulfill the promise of ERP. As illustrated below, this results in processes that are faster (origination to completion), cheaper (reduced manual content) and better (visible and measurable).

THE SHARED SERVICES CENTER MODEL The Shared Services Center (SSC) is a good business process environment in which to observe the effects of ERP processing with integrated workflow processes. The SSC is commonly established to achieve economies of scale that come from centralization of a corporation’s “admin” and processing functions to one physical location. The accounting, finance and human resources staffs, for example, support geographically disbursed business units from the SSC. ERP processing is done by this SSC staff with the remote users networked for various levels of ERP involvement, depending on their function. Centralization of this type is routinely practiced in industries such as construction and property management, where there is a portfolio of farflung, often changing business units (e.g. projects, properties). With an SSC, processing efficiencies stem from a centralized staff and their use of the ERP system, plus productivity increases from standardization of the entire business process with workflow. Process effectiveness is due to the workflow system as a delivery mechanism; delivering the right data to the right people and place at the right time. Through the workflow, the transactional data and documents are readily accessible by internet or private network to the business experts regardless of location. The full benefits of centralization can now be realized since, on the one hand, economies of scale are available, and on the other, neither geography nor the ERP interface is a constraint to the participation of de-centralized business experts. Experienced business systems professionals may recognize these heretofore irreconcilable factors as the root of the long-standing centralize versus decentralize systems argument. The effective use of workflow can render those arguments moot.

94

ENHANCING AND EXTENDING ERP PERFORMANCE CASE STUDIES Two cases are presented. They are illustrated by “swim lane” diagrams with activities segregated by type: manual, workflow or ERP. For some measures of productivity improvements exceeded 50 percent. Cycle times for transactional processing and costs per transaction were reduced, while the visibility and management control of the process increased dramatically. For a multinational manufacturing company, the major benefits derive from the workflow automation which streamlined a comprehensive multi-level management approval process for vendor invoice processing and payment through the ERP. And from the increased capabilities for fully automated, nearly hands-free processing. At a large construction company the workflow system enables the ERP-based, mission-critical pass-through-billing process. This process starts with receipt of diverse expense items in Accounts Payable and is completed when the documents are “passed through” to support and justify a customer’s billing in Accounts Receivable. The workflow system implemented by both companies is DocuSphere®, a business process improvement software product from Image Integration Systems.

THE MANUFACTURING COMPANY This large, multi national company has production facilities and administrative offices across North America and Europe. The entire enterprise is served by a single ERP system. The challenge was to control and reduce AP transactional costs and to standardize the entire AP process for all locations. After extensive analysis it was concluded that this was best achieved by establishing a SSC, there by eliminating the need for processing at each office. The resulting economies of scale would significantly reduce “manual content” per transaction, with commensurate reductions in staff and costs. It required that all vendor documents be sent directly to the SSC, and development of a comprehensive and controllable delivery process by which the transactional information and documents (vendor invoices, delivery tickets, etc.) could be presented to the ERP system ready to process (voucher). “Ready to process” includes review and approval by the business experts who generate and are ultimately responsible for the transaction, most of whom are in other offices, and in fact in other countries. And virtually none of these experts were users of the ERP system. Figure 1 represents the process for vouchering (i.e. make ready to pay) vendor invoices that were not generated by purchase orders. For many companies this type of transaction is most problematic in that the ERP system has no prior knowledge of the transaction, as it would if it were related to an ERP-generated purchase order. Note that for purposes of illustration, exception handling activities are not shown. They are a necessary part of the workflow process, and often require involvement by both business experts and AP processing experts. Many of the process activities implemented with workflow replace what would otherwise be manual activities. Only five of these 17 “delivery” activities (from numbers 4 through 20) remain manual. And these manual activities are all performed by the business experts, who possess the requisite business information. Workflow further supports the process by interrogating the ERP system with automated agents to validate and acquire process relevant data. (Activities 4, 7, 11, 14). The delivery is completed in activities

95

ENHANCING AND EXTENDING ERP PERFORMANCE 18 to 20, wherein the workflow system creates and submits voucher records (reflecting ERP specifications for transactional data) using an ERP utility for batched input. Documents related to each resulting transaction are automatically named (indexed) with the transactional data by the workflow system. As a result of the integration module functions, they are then accessible (viewable) from within both the ERP and workflow system. The transaction processing and reporting is then done by the ERP system, with AP staff review, to complete the process. Simply comparing the number of activities by type (manual, workflow, ERP) doesn’t fully measure the importance of each type. But it is instructional to note that, of 25 activities for this process, four are manual activities within AP, five are by business experts and three are ERP data processing activities. The remaining 13 process activities are provided by the workflow system. A note on activity number 3, “auto data capture.” To further reduce manual activities, this activity incorporates a relatively new capability; automated or advanced data capture (ADC). ADC is based on optical character recognition technology, and provides rules-based methods to extract data from scanned documents. A significant proportion of ERP data entry fields can be captured automatically, further replacing manual data entry. With auto voucher (1820) the resulting transactions can appear to be virtually hands-free. ADC is shown in workflow for convenience and arguably warrants a swim lane of its own. The results and benefits directly from, or made possible by, the workflow system are: 1. the process is fully defined and standardized by workflow, improving aspects of governance and regulatory compliance. 2. performance metrics for each activity and the process as a whole, are visible to and measurable by the process owners. 3. reduced manual content per transaction, resulting in overall productivity increases of over 50 percent 4. directly involved business experts, interacting appropriately and efficiently, reducing elapsed process time and error frequency. 5. reduced total process time; transactions that required weeks are now complete in days, while those that took days are often complete in hours. The existing ERP system continues to provide effective, robust, high-volume transaction processing, but contributes to the incremental benefits above only as a necessary sub-set of activities for processing within the workflow process.

96

ENHANCING AND EXTENDING ERP PERFORMANCE

Figure 1. SSC AP Voucher Process Activities

97

ENHANCING AND EXTENDING ERP PERFORMANCE THE CONSTRUCTION COMPANY Confronted by significant growth challenges, this company replaced their business systems with a single integrated ERP system. To fully satisfy their business objectives the new business processes necessarily included implementation of independent workflow with the new ERP system. This was particularly important for a common type of customer billing process often referred to as pass-through billing. This is illustrated with the simple example of “cost plus” contracts, wherein the customer agrees to pay the building contractor for all the legitimate construction project expenses, passed through to the customer, plus say 10 percent more, the contractor’s project profits. Processing problems, particularly with projects such as the design and construction of large manufacturing facilities, stemmed from the sheer volume of project expense items from AP, Payroll and similar ERP transactional processes. Understandably, the customer demands original documentation to justify the billed expenses. As a consequence, it could be several weeks after an expense item was paid before the customer was billed, this delay caused by the effort to find and organize all the originating documents (e.g. vendor invoices). This delayed creation of the bill, added time and unrecoverable expense (people), and increased the opportunity for error, including under-recovery as legitimate expense items were easily missed. Figure 2 represents the entire inter-departmental process, starting in AP and completing in Billing (Accounts Receivable). As in the first case, the combination of workflow and new ERP sytems allowed centralization and elimination of AP processors at job sites and regional offices. For simplicity Figure 2 is restricted to non-PO invoices, although in practice there are many PO-driven vendor invoices for construction projects, which the workflow handles with analogous process activities. The AP process issues and benefits in the first (manufacturer) case are equally evident here: standardized processes, performance metrics, reduced costs and total process time, and direct involvement of the business experts, such as project managers at the job sites. In fact, total AP process time is even more important in this case as it contributes directly to the dead time between the business action that creates the vendor expense and the revenue from billing the customer for that expense. In the pre-workflow process the elapsed process time to voucher entry (AP activity 13) averaged over three weeks. It has been reduced to under two weeks. Management is also currently evaluating ADC and automatic batch vouchering, as described earlier, with the potential to further reduce time and cost.

98

ENHANCING AND EXTENDING ERP PERFORMANCE

Figure 2. AP and Pass-Through Billing Activities

99

ENHANCING AND EXTENDING ERP PERFORMANCE Vendor invoices are manually vouchered by AP processors using one of the standard ERP data entry screens. As with automatic batch vouchering, the transactional documents are automatically indexed with relevant transactional data. In this instance the processor does data entry by “key from image,” working from the electronic image of the vendor invoice, and related documents, as there is no longer a paper document needed or available. Further, the workflow activities, rules and roles can be configured in a variety of ways to support virtually any business requirement for distribution to the AP processors of documents to be worked. This is most often done by “pushing” the next invoice document from a common work list to a processor who has become idle. As a consequence, the work load is spread evenly over the entire group of AP processors. The major benefits derive from billing the customer faster and more accurately, and thus recognizing the revenue sooner. This is something of a twofor-one result in that the new process implemented with workflow reduces costs in the AP department while independently increasing revenues in the Billing department. The reduced time to bill is a function of both the ERP system, which has a very effective pass-through billing module, and the workflow system. The originating document was earlier discussed as the data “container” providing context to the expert. In this case that business expert is in the customer’s organization, and has a right to the documents for review before payment. The workflow activities (numbers 4 and 5 in AR) organize all the supporting transactional documents required for the bill by using agents to interrogate the bill detail generated by the ERP system. Since the billing detail includes some of the same data that indexes these documents (e.g. voucher number), they can be organized and presented to match structure previously established within workflow (AR activity 1). Often iterations (drafts) are required before the bill includes all expense items, and only those items, that satisfy the terms and conditions of the customer’s contract. As a last activity in the workflow, the processors select the distribution medium for the customer invoice; print to paper or write to CD. The availability of a CD with the customer’s bill and all supporting details and documents, provides obvious benefits as compared to paper. In summary, the AP benefits in this case are much the same as in the first case. In addition however, the customer billing results and benefits due directly to the workflow system are: 1. more than a 60 percent reduction in total time to bill, from over six weeks to under three weeks on average. 2. qualitatively, an increase in completeness and accuracy in the customer bills, reducing disputed items and speeding receipts. Because time is money, these translate to significant revenue improvement.

THE FUTURE The foregoing is based on characteristics of the vast majority of ERP systems as installed and utilized today. New ERP vendor initiatives with their own workflow, business process management and similar products and architectures will undoubtedly change today’s landscape, presumably for the better. It is reasonable to assume that the flexibility and capability of independent workflow systems will increasingly appear both within ERP systems, and as

100

ENHANCING AND EXTENDING ERP PERFORMANCE complementary products from the ERP vendor. Over time therefore, the sources of effective workflow systems for use in ERP environments may change somewhat, but the business requirements will not. However, at least for the time being, this trend will have little or no effect on the tens of thousands of businesses that use, and for some time will continue to use, today’s ERP systems.

SUMMARY AND CONCLUSION ERP systems are most commonly and correctly perceived and utilized as transaction processing machines. In that role they excel. Workflow systems, integrated with the ERP system, can function as the data delivery mechanism for ERP transactional processing. Conversely, ERP transactional processing is but one of the many activities in the workflow. The integrated result provides capabilities that have been missing with ERP alone: standardization and automation of entire business processes, effective involvement and interaction with the business experts, and, the creation and capture of all relevant business process information. The improved business processes enable the promised economies of scale from centralized ERP processing.

101

Narrowing the Semantic Gap between Business Process Analysis and Business Process Execution Dr. Setrag Khoshafian, Pegasystems Inc., USA ABSTRACT The business process management (BPM) industry is growing rapidly, surpassing the expectations of even its most ardent supporters. Like most new technologies, BPM is enduring its own growing pains thanks to convergence, consolidation, and accelerated adoption. One of the critical areas of convergence that has not received sufficient attention is the semantic gap and interoperability challenges between business process analysis (BPA) tools and intelligent BPM engines. This interoperability challenge is further aggravated by the lack of robust business rules modeling tools. Business rules are now regarded as essential components of a next generation BPM (intelligent or smart BPM). Even though there are various BPM standardization efforts, the semantic gaps between BPA and run-time intelligent BPM engines are considerable. This paper will address these semantic gaps and identify solutions for continuous and iterative development of complex intelligent BPM applications.

INTRODUCTION The business process management market has grown steadily over the past five years. Most organizations have successfully built and deployed BPM applications with tangible returns on investment. However, as BPM starts to go mainstream, it faces challenges in the proliferation of BPM tools and solutions. Among the different components of BPM suites 1 , you have BPM engines, BPA tools, enterprise integration, business rules engines, and more. A comprehensive BPM solution will have a lifespan of several years and will, like all other software applications, go through several versions and iterations. If several tools are used to implement the solution, they need to work together seamlessly from modeling to design to deployment to execution to monitoring and back to modeling. But that is difficult to achieve. Let us take two of the components involved in modeling, designing and execution of business process solutions: business process analysis and business process management systems. There are four main approaches to interoperability between BPA and BPMS: •

Paper-based and modeling artifacts as starting point for requirement/analysis documents: This common approach has no pretenses of interoperability. The various implementers of BPM applications take whatever is produced through BPA tools—paper documents and electronic documents. The realm of the modeling and analysis tool comes from the business analysts. The execution tools, on

The credit for the BPM suite approach goes to Jim Sinur from Gartner. There are fundamental components of the BPM suite including Business Rules, Enterprise Application Integration, Analysis and Simulation, and Business Activity Monitoring.

1

103

NARROWING THE SEMANTIC GAP

•

•

•

the other hand, are more IT-driven and detailed. This chasm between business owners, who want to model and analyze, and IT, who designs, implements and deploys, is critical. The chasm gets accentuated when modeling/analysis and then design/implementation/deployment are done through different tools, each with its own platform, semantic models, versioning conventions, meta-model, etc. At best, you get requirement documents produced from business owners and thrown over the wall for IT to implement. Export/Import Models Using Standards: The second approach uses standards for interoperability. However, in BPM—as in many other technologies—the “nice” thing about standards is that there are so many to choose from. The standards bodies attempt to be complementary and co-exist to solve real problems. At least that’s the theory. Workflow Handbooks over the past several years feature articles on standardization progress. The three most pervasive standards are the UML activity diagram notation, the BPMN notation, and the BPEL Web Services Execution Language. There are many—too many—other standards (XPDL, JSR 207, BPSS, WSCDL, and many more). Interoperability and Partnerships between Modeling and Execution Platforms: The third approach is to have “partnerships” between modeling and execution platform vendors. This approach pairs the higher-level modeling offered by modeling and analysis tools with the lower-level IT design primitives of the platform. This is a patch-work integration at best, involving two and sometimes three products with predictable challenges for synchronization, maintenance, and flexibility. Advanced BPM applicaitons evolve continuously. Comprehensive and Unified Intelligent Business Process Management Platforms: A fourth approach is to use an intelligent and unified BPM platform that encompasses modeling, simulation, piloting, deployment and execution capabilities. There is a fundamental and real-world assumption here: a comprehensive BPM application is not a one-shot deal. You need to maintain, fix, improvement, enhance, and extend. It is even more critical for BPM since you can incrementally build bigger, richer, and more advanced applications from your building blocks: decision rules, or processes, or integration components. A unified platform avoids many of the pitfalls of BPM-BPA interoperability—especially when there are continuous planned and unplanned changes in the BPM applications.

EMERGENCE OF AGILE AND ITERATIVE APPROACHES Fifty years of computer science has shown the value of close affinity between modeling and execution. The more you can avoid mapping and transformations, the better. There are emerging requirements. We need to be able to handle uncertainty. We need to implement and deploy incrementally. And we need to maintain increasingly high levels of quality and performance. The delay between modeling and piloting should be reduced. The traditional “waterfall” methodology, with sequential and extended periods of analysis, then design, then implementation, then testing and deployment is ineffective. As a result, we now see growing adoption of methodologies (such as Unified Software Development Process, Extreme Programming, or Scrum) that attempt to address the continuous and iterative improvement lifecycle and address the waterfall approach’s limitations. These methodologies encourage agility, quick wins, unplanned and constant iterative change. While they may differ in their

104

NARROWING THE SEMANTIC GAP approaches to team formation and project planning, they all seek to avoid long phases and gaps from modeling and analysis to design and implementation, support incremental, iterative, and fast turn around for modifications to existing solutions. The underlying platforms and tools used within this kind of methodology obviously should not raise technical impediments to the iterative and continuous improvement approach. This implies the semantics of the modeling artifacts in the analysis phases should be very close if not identical to the semantics of the design and implementation constructs. The semantic gaps need to be eliminated.

BUSINESS PROCESS ANALYSIS TOOLS BPA tools are used to model and analyze processes. After discovering the processes and rules of the application, the processes are exported and deployed in a business process management system. The BPMS can provide more detailed design of the processes imported from the BPA tool. In business processes analysis and modeling, there are a number of tools and notations for modeling the processes. These tools provide various modeling constructs to design business process applications at a higher level and then deploy them to a process engine. Using a BPA tool you can model organizations, networks, data and, perhaps most importantly for this chapter, processes. The tools and notations could be arranged in various perspectives (e.g. business, technology, etc.). Here are some of the modeling constructs that BPA tools support: •

•

•

•

•

•

Strategy Diagrams: BPA tools often have high-level graphical representations for corporate goals and strategies—for cause-and-effect analysis, or strategic goal setting or as a general framework. Examples here include Balanced Scorecard and Fishbone illustrations. Process Diagrams: This is perhaps the most important component for BPM. The process diagrams typically include swim-lanes, tasks or activities and the overall flow of the process. Processes can have subprocesses and you can also specify simulation parameters and simulate the process. Business Policies and Decision Rules: The processes provide the procedural flow models of work, involving human participants, systems, and trading partners. You also need to model the business policies and decision rules. Rules such as risk level determinations, service level agreements and approval levels are examples of business policies and decision rules. These can be associated with processes or can pertain to the application or even the enterprise as a whole. Class or Data Modeling: This allows the modeler to provide the analysis of the business classes. Typically UML notation is used to represent the class hierarchies. Depending upon the specific capabilities of the modeling tool, the classes can be used in processes. Interaction Models: A number of interaction models are used to represent either use case diagrams, sequence diagrams or object interaction diagrams. These depict external actors using the system or internal objects interacting with one another. Organizational Model: The organizational model is used to represent the various organizational units and their relationships. Usually this is illustrated as a hierarchy.

105

NARROWING THE SEMANTIC GAP There are dependencies and associations between these models. For instance, a process model will use the organization model (the participants in the flows—typically in the swim lanes), the class or information model and also the business policies. The strategy model will use both the processes and policies:

INTELLIGENT BUSINESS PROCESS MANAGEMENT SYSTEMS2 An intelligent business process management system has several components. Why do we call it intelligent BPM? Primarily because it is a unified system that can handle different types of rules. In [Khoshafian 2004] 3 we defined intelligent BPM through: Intelligent Business Process Management = Flows (workflows) + Process Rules + Practice Rules + System Rules In addition, intelligent BPM will allow you to program both procedurally and declaratively. Examples of the former include all conventional and objectoriented programming languages as well as workflow constructs. Examples of the latter include the declarative rule and constraint systems. Intelligent BPM reminds us that processes and decision rules are two sides of the same coin. Today we take database management systems and content / document management systems for granted. We have also done reasonably well with digitizing our procedural logic and procedural programming. However, until

A word of caution: in some cases “Business Process Management” and “Business Process Management Systems” have been given different semantics or connotations. We think this is confusing and within this paper BPM and BPMS are synonyms (this is similar to “Database Management” and “Database Management Systems” – in most contexts they are synonyms).

2

[Khoshafian, 2004] “Web Services Orchestration and Management through Intelligent BPM,” by S. Khoshafian. Appeared in Workflow Handbook 2004 Edited by Layna Fischer, Future Strategies Inc., Lighthouse Point, Florida.

3

106

NARROWING THE SEMANTIC GAP recently, we had not done so well in digitizing the declarative rules in the context of BPM applications. Even as BPM systems emerged we soon realized that the rules that drive the business are often in people’s heads or in volumes of paper documents or in application programs. When we say “intelligent” BPM we mean exactly that—digitizing the scattered rules found in documents, program spaghetti code or in peoples’ heads. You still have knowledge workers participating in the processes and authoring the rules but now the system either replaces rote manual labor or guides, assists and complements the worker to get her/his job done more effectively. This self-guided assistance also applies to the back-end systems and trading partner activities. The following illustrates the various components of an intelligent business process management system: •

•

•

•

Integration : Business process management includes enterprise as well as business-to-business integration components. You can have back-end applications (such as CRM, HR, ERP applications), in-house applications, and newly developed components (e.g. EJB components) become participants in your processes and decision rules. BPM systems use applications as well as technology adapters to realize integration. Examples of technology adapters include EJB, MQ, JSM, .NET, and perhaps the most popular is Web services through SOAP interactions. Design: Most of the aforementioned modeling features found in BPA tools often exist in intelligent BPM suites, though usually with a different emphasis: details for deployment and execution. The design is more detailed since now you are focusing on execution. You can model the flows. You can also design your classes and author your decision rules. Other constructs include your integration services or connectors. BPM design can also include strategy models. You can simulate either your existing (“as is”) or designed (“to be”) processes. Performance analysis allows you to make modifications in terms of participants, flow logic, or digitization of your decision rules. Presentation rules: design also includes other components that you typically do not find in BPA. For instance, your forms and overall portal GUI is designed through the intelligent BPM tool. The design of object types, processes, integration components, and business rules could be performed in thick client platforms with an explicit deployment phase or in a thin client (browser based) environment with immediate run-time piloting and execution. Increasingly the trend is to have thin client design environments in conjunction with popular tools that are readily available on every client platform—such as Visio and Excel. Execution: The rules and flows are compiled and executed by the underlying execution engine. The best option is to have one core execution engine (e.g. a rule engine) that treats flows and processes uniformly and supports a consistent mathematical model of execution and resolution. The engine keeps track of the status of various processes, participants, and other objects that are involved in intelligent business process automation. This engine communicates and stores the process states in underlying relational database management systems. These process states are then used to analyze the performance of entire applications, specific processes and participants in processes. The engine supports security and access control. It also has transactional semantics. The execution engines

107

NARROWING THE SEMANTIC GAP

•

run within a multi-tier application server architecture (e.g. J2EE). They will have both client and server tiers. Portals: Intelligent BPM products also offer out-of-the box portals for different communities. These communities include end users, business owners and business analysts. The portals provide the run-time front end to the processes and business rules. Typically, end users will have access to a work list that displays the list of items assigned to the user. Management portals include business activity and business performance monitoring capabilities. The BAM reports graphically display the performance of various processes and applications as well as process participants.

STANDARDIZATION EFFORTS The previous sections provided an overview of the features offered by business process analysis tools and business process management platforms, respectively. The primary target audience of BPA tools is business analysts who model the information, processes, organization and strategies of the enterprise. BPA tools do not include process execution. They use higher level artifacts and models that are supposedly closer to the business owner or analyst’s perspective. However, as business process management systems evolve, they too are providing modeling, design, deployment and execution primitives. The overall iterative and continuous improvement trend of the industry is to reduce and eliminate the gap from modeling to design to execution and then back to modeling. Furthermore, the fewer mapping limitations or anomalies between modeling and design and implementation the better.

Given these different tools, on the surface a “logical” answer or suggestion would be: Why don’t we use standards? How about a standard for process definitions that is exported from one tool and imported into another. In fact, there is an abundance of BPM standards. These standards can be categorized into three basic categories: notation, process definition, process execution. This is not a comprehensive list, but it shows how the three areas are addressed by various standardization bodies and initiatives. We have provided here a higher level and integrated view of the standard categories. In other

108

NARROWING THE SEMANTIC GAP illustrations the process definition and executions are partitioned between “internal” and “external” sub-categories. Internal implies potentially modeled, deployed and executed process standard. The external sub-category deals with business-to-business (B2B) exchanges for interoperability especially between different process engines or platforms. Mapping Models The alternative Notation, Process Definition, and Process Execution standards can be mapped onto the three layers correspond to Object Management Group’s(OMG) model driven architecture (MDA). This architecture provides three layers of abstraction for models: a computation independent model (CIM) which is oriented to model businesses, a platform independent model (PIM) and a platform specific model (PSM).

The overall premise is that you have business models that can be represented independently of underlying software models or implementations. Notations or modeling languages such as BPMN or UML can be used to represent certain aspects of CIM bsuiness models. CIM concentrates on the business use cases, the required business results, and the business processes to achieve them, independent of software or underlying implementation systems. In an iterative development methodology the CIM will correspond to business modeling and requirements phases. But you will need to carry out business modeling and requirement processing for each iteration. In PIM, the goal is to provide more details for the software models without staying independent of underlying specific platforms. For instance, execution languages such as BPEL can be used to capture execution of Web Services choreographies, independent of the underlying business process management system. The last layer PSM provides detailed specifications on implementation platforms, and it is at this layer that you have specific extensions and capabilities from various vendors. So what is a “platform?” Typically in a large and complex enterprise architecture you have several layers. So a PIM from one perspective could be a PSM from another. A business process management system could be supported on a variety of platforms (e.g. WebSphere, WebLogic etc.). Hence the PSM models for the BPM application is the application server infrastructure where you execute your BPM applications. However, for other domains, such as component computing, the PIM could be the detailed design models of the components while the PSM could be either a J2EE or .NET platform—

109

NARROWING THE SEMANTIC GAP corresponding to the code. In MDA architecture you will then provide mappings. Here again, every time you use mappings there is the danger of loosing semantics. Once again this could be problematic if you are continuously improving and iteratively building your application. Mapping between CSM and PIM could be problematic if multiple tools are used. Metamodels Common metamodels and standardized exchange formats between models facilitate language sharing between tools, communities, and products, as well as mapping between various platforms and layers. OMG has introduced a four-layered metamodel framework and an exchange standard (XMI) for interoperability between tools that support common metamodels. The modeldriven architecture (MDA) framework is also attempting to standardize on a meta-model for processes and organizational models. The MDA framework supports a four-layered metadata architecture: 1. M3: MetaMeta Model, the most abstract layer specifying the Meta Object Framework (MOF). 2. M2: MetaModels for specific domains or disciplines: for instance data warehousing. 3. M1: Models, which are instances of the metamodel in M2. For example a process and practice rules complying to M2. 4. M3: instances of models. For instance a specific row in a table, an object instance or a process instance. Metamodels standardization provides value in “speaking the same language” between various tools. For instance OMG’s Business Integration Task Force is working on a metamodel (M2) for business process definitions. However, especially due to significant differentiations and extensions by various underlying platforms, it is still difficult to sustain interoperability simply by relying on the metamodel standards.

THE SEMANTIC GAP So how about the interoperability between BPA and intelligent BPM platforms? Business process applications are not “one shot” deals. There is a continuous round-trip to be sustained throughout the BPM applications’ life cycles. This lifecycle is illustrated as follows:

110

NARROWING THE SEMANTIC GAP Now if we were to map this onto tools what we get is:

Typically the BPMN or UML notation is used within the BPA modeling tool. BPA tools and vendors produce a number of artifacts—including flows or processes—and throw them over the wall. Usually they throw BPEL. There are other options or standards such as XPDL or XMI. BPM execution platforms incorporate additional details to the imported artifacts, to enable them for execution. This is of course one approach and it is marred by semantic gap challenges. Supporting standards or common metamodels or an MDA approach does not mean the respective tools automatically comply with the same underlying standards or meta-models. In addition, BPM vendors typically provide their own extensions and features as differentiators. Unified Intelligent BPM A unified, intelligent BPM system supports all the phases of iteraive and conituous development: modeling, analysis, design, implementation, deployment and testing. It also supports a unified platform to handle flows, policies, and integration components. A single object model and platform avoids many of the pitfalls of the semantic gap beween various components or tools. In the continuous round-trip and iterative development, you need a platform that allows quick building and demonstrating of wins, then sustains your applications’ iterative improvements and extensions. So there are two fundamental features of unified intelligent BPM platforms that support and satisfy the requirements for agility, handling of uncertainty, and continuous improvements of advanced intelligent BPM applications: •

•

Continuous Round-Trips: You must be able to go from modeling to design to implementation to deployment and then back to iterative improvements continuously and iteratively. With multiple products, and asynchronous import/export, differences in the underlying semantic models are inevitable. Unified and Integrated Platform: An intelligent BPMS has several components that must operate in the context of the same unified platforms. Within a single unified platform you can handle simple and

111

NARROWING THE SEMANTIC GAP complex rules in the context of flows and processes, using a unified information, integration, and organization models as illustrated here:

CONCLUSION Even though there are significant and important efforts to provide BPM standards and support interoperability between modeling and execution platforms, the semantic gaps are inevitable when one uses multiple products and this proves to be an impediment to continuous improvement and iterative development. Standards can be useful in bootstrapping and perhaps in the initial phases of a project. Metamodels can provide a common language between systems and perhaps facilitate integration. However, a process model or definition of a detailed design and execution environment is not a “one shot” deal. Instead, BPM applications need to be implemented for continuous change, they need to anticipate uncertainty and unplanned change, and they need to deliver quick wins. We need to reduce or eliminate transformations, mappings and import/export roundtrips between disparate systems. For productivity gains we need to use the full power and capabilities of the underlying platforms. This is an “on demand” age. Narrowing this gap between the world of modeling, which usually belongs to the business owners, and the design and execution environment that pertains to IT, is a new (at least “re-newed’) and compelling imperative. This imperative is best addressed through unified intelligent BPM platforms that provide a single environment for business owners, process architects, managers and users alike, without the demands and pitfalls of transformations, while supporting immediate deployment and execution of planned and unplanned changes in processes or decision rules.

112

Using SOA and Web Services to Improve Business Process Flow Case Study: District of Columbia's Oracle Database, Presentation Layer, SOAP/XML

Zachay B. Wheeler, Roberta Bortolotti, SDDM Technology, United States ABSTRACT SDDM Technology, small business enterprise, was tasked with the analysis of improving and automating the current business process of license issuance for the Department of Consumer and Regulatory Affairs (DCRA) of the District Columbia. DCRA is the business regulatory agency for the District of Columbia and provides license and licensee information to various local and federal agencies. SDDM Technology responded to this challenge, after careful analysis, by recommending, developing and implementing a Service-Oriented Architecture (SOA) and Web Services (WS) approach. The improved business process of issuing business license, by DCRA, entailed the development of a web-based (intranet) application using innovative technologies. The development of an n-tier application was essential and the SDDM Technology team was responsible for developing Business Logic and Data Access Tiers. The main challenges faced were the integration within distinctive platforms along the application architecture tiers including Java, XML, .NET and Oracle technologies; the handling of requests and data manipulation to the Web services focusing on increasing the performance; and the availability of the business rules ensuring flexibility for future enhancements in a useful enterprise IT environment. The SDDM Technology team applied Microsoft's .NET technology to develop the business rules. Specifically, an objectoriented approach was applied to the development of the business layer using VB .NET. ADO.NET was used in conjunction with Oracle packages to access and manipulate data from the Data Tier (Oracle database). Data was requested from and passed to the Presentation Layer (Java technology) using SOAP/XML. In short, the solution to the District of Columbia business license problem was resolved using Service-Oriented Architecture and Web Services, taking advantage of the available technologies mentioned in this abstract. Keywords: VB.NET, Oracle, SOAP, XML, SOA, Web Services

BACKGROUND The Department of Consumer and Regulatory Affairs (DCRA) is the business regulatory and enforcement branch of the District of Columbia. The DCRA is responsible for the issuance and enforcement of all licenses in the District of Columbia. Over the course of time however, the responsibility of issuing license was allocated to several other agencies in the District. For example, the Department of Health (DOH) became responsible for issuing dog licenses, pharmacy licenses and several other health licenses; the Department of Mental Health (DMH) became responsible for issuing mental health and mental health community licenses; other agencies became responsible for issuing agency specific licenses.

113

USING SOA AND WEB SERVICES In regard to license issued by the DCRA, prior to 2004, the agency faced a problem which would make a business owner have to apply for and maintain multiple business licenses at a particular address. For instance, a hotel owner that maintained a restaurant, cigarette vending machine, massage parlor, and food vending machine would have to obtain and maintain four separate business licenses. Each license would have its own fee and renewal period; hence the business owner would have four separate times of the year that he would have to interact with the Department of Consumer and Regulatory Affairs to renew a license. The issuance, maintenance and tracking of business license became a daunting task, the process became a burden to the business community and the efficiency of the Department of Consumer and Regulatory Affairs work force decreased significantly. In the summer of 2003, the Basic Business License (BBL) legislation was passed by the City Council and readily adopted by the DCRA and several related license issuing agencies. The BBL legislation is based on two specifics: Return all license issuance, regardless of license type, to the DCRA; and allow the business community to apply for and maintain one business license regardless of the number of business license types held by the business owner at that particular premise address. The business requirements for the development of this application were determined to be the following: • Intranet application should be the initial focus • The application should be at a minimum 3 tiers • The ability to share data between agencies over the Wide Area Network (WAN) • The ability for each agency to maintain a local copy of its data • The ability to extend proposed intranet application to the public for online license issuance and tracking. • The data repository should be Oracle (a standard eventually adopted by the city) It became apparent that neither the DCRA, outside agencies nor the District of Columbia had adopted a technology standard for application development. Hence, systems were written in a variety of development languages such as PowerBuilder, Java, .NET, VB, C++, and Fox Pro residing on disparate platforms such as UNIX and Microsoft. Also, a variety of data repositories where used: Oracle, SQL Server, Access, Excel, Fox Pro and Sybase just to name a few. The task of developing the application was split between three companies, and the solution architecture based on the use of web services in order to show flexibility in integrating different programming languages and platforms used throughout the DC Government network structure. The use of protocols based on the XML language to issue business licenses or to exchange data with another Web Service constituted a main requirement in order to extend this application to inter-organization communication for the overall system. FileNet, Inc., the company responsible for the presentation layer, developed this tier using java technology. SSDM Technology, on the other hand, used Microsoft .NET technology to build the Business Layer and developed the Data Access Layer using ADO.NET and Oracle stored proce-

114

USING SOA AND WEB SERVICES dures. The stored procedures were wrapped in Oracle packages. And, finally, Peake Technology developed the conceptual, logical and physical data model. Analysis of Challenges and Solution for the Business Process The design and analysis of a solution for this case led the SDDM Technology team to focus on having a result in which the work was done better, faster, more reliably and for less money. In the business process of license issuance in the District of Columbia, the solution was to draw a strong relationship between the workflow and web services. So, in order to face the challenges presented by this relationship, the implementation of the workflow was designed to answer questions such as who? What? When? in the business process. The use of Unified Model Language (UML) played an important role in analyzing the business process, depicting who were actors, systems, subsystems, services (use cases), and workflow in the system and their roles; what actions and transactions users make and if these transactions were manual or automatic; and, finally, when the users start or end a work, the order the transactions are made and if they are done sequentially or in parallel. There were several challenges that the SDDM Technology project team had to overcome. Of those challenges the concept of interoperability was the biggest hurdle (see Interoperability Challenges). Having a considerable challenge of integrating the workflow of this system with others in the District of Columbia in order to create a collaborative application built of Web Services, the business process became a set of tasks performed by Web Services in which workflow control and interaction are an inevitable demand. From a 100,000 ft perspective or enterprise level, the web service-SOA approach is utilized for simple data exchange between agencies, the public and the DCRA staff.

Figure 1: Simplified View of Interaction

115

USING SOA AND WEB SERVICES Based on Figure 1, many different clients will interface with and request data from the DCRA system. With regards to the agencies that issue their specific license, each would have their own individual business logic however some business rules generated by the DCRA could be reused, thus reducing future development time. The process of code reuse was accomplished by restricting the processing/business logic to the WS. The business logic was decoupled and modularized for autonomy and functional independence. This naturally led to ntier application architecture with the WS acting as a service layer. Using the modular decomposition approach each major license process or function was decomposed into 20 web methods or services. Because of the granular decomposition of the workflow (business rules) into services the services are able to be used individually or within and between services and applications. Interoperability Challenges The implementation of the WS-SOA approached presented many challenges to the development team. A short list of the major challenges is presented below. The integration within distinctive platforms along the application architecture tiers including Java, XML, .NET and Oracle technologies. There was a need for a technology platform- and language-independent, that leads to task-oriented development and workflows, loose-coupling and able to adapt existing applications to changing business conditions. The handling of requests and data manipulation to the Web services focusing on increasing the performance. SDDM Technology confronted three main data exchange challenges: primitive data type mappings, in order to guarantee that the correct data type of one platform is referenced to the same data type in the other; non-existent data types, which consist in handling data types that exists in one platform but not in the other; and complex data types, which consider the exposure of complex data type so that the other platform can use it. And the availability of the business rules ensuring flexibility for future enhancements in a useful enterprise IT environment. The use of an integration or migration strategy allows a service-oriented solution. Interoperability Resolution The interoperability challenges led to a solution which used web services. The use of web services not only affected how the application was designed but also imposed a variety of strategic and technical factors that met the requirements and purpose of this project. Seeking an inter-application and inter-organization communication by abstracting proprietary technology and establishing universal integral framework for the District of Columbia, the web services consisted solely of XML Technologies. XML schemas were used to ensure type and class compatibility, specifying the format for XML documents during the integration of the n-tier application. Interoperability between Presentation and Business Layers For requests and data manipulation, XML serialization intended to take complex data type used in our application and encode, save, transfer and decode it. With serialization, both presentation layer and business layer understand a particular data type before they attempt to exchange it and to establish a connection between them. In this case, the serialization occurred in the .NET platform

116

USING SOA AND WEB SERVICES

Figure 2 depicts the structure of this n-tier application. It is worth mentioning that the JAVA and .NET technologies come from different backgrounds and present some distinction on data types. So, the use of XSDs helped the two platforms agree on the format of an XML document, allowing mapping of a class to a defined XML Schema. This solution assured that each platform could exchange XML data with each other. A common data format was defined before development, and XML serialization in conjunction with XML Schemas were used to exchange data between .NET and Java. In this scenario where each application was built from scratch, the following solution was adopted to integrate the presentation and business tiers: • Use of XSD to define common or shared types and then generate platformspecific code from those shared types. • Creation of a central XSD repository for the development teams developing the presentation and business layers to provide for consistency in generating types across applications. • Use of elements that XSD recognizes and publish. The service-oriented architecture (SOA) describes the District of Columbia entire system of services dynamically looking around for each other, getting together to perform some application, and recombining in many ways. This model encourages the reuse of components and technology that evolves the way applications are designed, developed and put to use. The SOA model represents their distributed application across the network, allowing distributed communications of services such as the use of an Enterprise Service Bus (ESB), which is a common distribution network for service communication.

THE DCRA WORKFLOW AS-IS Business Process In the initial stages of the project SDDM Technology went beyond observing the license issuance process. SDDM staff members participated in the As Is license issuance process. An overview of some of the various business processes for issuing, renewing and exchanging license data is described below.

117

USING SOA AND WEB SERVICES ISSUE NEW LICENSE • • • • •

• • • •

• • • • • •

The client (business owner) enters the One-Stop Center (License Issuance Center). The client interacts with DCRA CSR (Customer Representative) and inquires about the particular license(s) that they are interested in obtaining. The client is handed an information (application) packet and the details are further explained to him, in regards to the necessary documents that are needed to obtain a license. The client leaves and obtains the necessary documents and fills out the application. The client returns with all of the necessary documents completed. These documents include license type approval from different agencies. For example, the Department of Health, Metropolitan Police Department, Department of Public Works, etc….. The client interacts with DCRA CSR and the CSR reviews the application package for completeness. The CSR logs the client in the IO system and hands the client a ticketed number. The client is called to different CSR for application processing. The CSR records the information in an access database with PowerBuilder front-end (the database was not designed with data relationships i.e. oneto-one, one-to-many, or many-to-many hence all data is saved into a single table). The CSR looks on a paper based fee chart to determine and calculate the fee. The CSR adds the fee to the application. The CSR prints the invoice. The client takes the invoice to the cashier (associated with a different agency) and pays the invoice amount (partial payment is not allowed therefore he pays the bill in full). The client returns to the One-Stop Center waits for the CSR to finish with the current client hands the stamp paid invoice to the CSR that processed the application. The CSR reviews the stamped receipt. The CSR prints the license and gives it to the client.

Renewal License Process The renewal process of the BBL can be initiated by several different routes that converge into the same process. The difference between the routes is how the client responds (walk-in, mail in or lockbox) to the renewal letter. • The Task Leader contacts the Information Technology (IT) department and requests that renewal bills be generated for a particular time period. Typically, they are sent out sixty (60) days prior to the renewal date. • The client responds by sending the payment either by waking in, sending the payment through mail or paying through the bank (lockbox). If updated documents are needed the client sends the documents through mail or walks them in. • The CSR reviews (if necessary) the updated documents for completeness. If they are not complete then the CSR types or fills out a deficiency letter by hand and sends the documents back to the client, however the CSR keeps the payment unless it is insufficient.

118

USING SOA AND WEB SERVICES • • • • • •

The CSR records how the payment was received using the modification module of the PowerBuilder application. The CSR calculates the fee using a paper chart to identify the correct amount of the fee. The CSR records the amount of the payment and using the update module adds the date payment received and how it came in (walking, mail in or lockbox). The CSR changes the status to paid in the system. The CSR prints the license. The CSR mails the license out to the client at the end of the day or gives it to the client (walk-in).

External Agency Process The AS-IS process for external agencies is bi-directional. In case one, the external agency wants information from the DCRA. In case two the DCRA wants license, review, inspection information from that agency. Case One • The external agency contacts the IT department of DCRA and submits a request for information about a particular business or business category(s). • The request is logged into the report request system by the IT CSR. • The IT CSR gives a paper copy of the request to the IT Report CSR (a single staff member dedicated to generate reports). • The IT Report CSR, using Visual Basic for Application Interfaces, connects to the access database defined in the previous processes and generates the report. • The requesting agency is then sent a copy of the data in .mdb, .xls format on a floppy or zip disk or paper report is generated and mailed to the agency. Case Two • The DCRA management contacts the external agency. • The external agency sends an .xls file or paper based report to the requesting manager through email.

TO-BE BUSINESS PROCESS After reviewing the processes described above it is obvious that many of the processes could be automated and after further review many of the process could and would be decoupled into individual services many of which can be used throughout the district for fee calculation, data exchange, license issuance, departmental reviews and departmental inspections. In short, WS-SOA was ideal for this particular enterprise. An overview of the services is provided using the Unified Modeling Language (UML) to define the various systems, subsystems, services and workflow. SDDM Technology relied heavily on the UML Use Case, Use Case Diagram, State Chart and Activity Diagrams for workflow modeling. The following Activity Diagram provides a general overview of the generic application process with interaction from the CSR, Presentation Layer and Web Service.

119

USING SOA AND WEB SERVICES

The following use case diagram provides general overview of the basic services developed.

IMPLEMENTATION In short, granularized independent business services will allow business logic reuse for other DC agencies and future applications at the DCRA.

120

USING SOA AND WEB SERVICES In fact, there are 20 web methods (services) defined for the DCRA WS. A snapshot of some of the services are provided below.

An example of the AddApplication service signature is provided and shows the calling interface with the complex XML data structure below. Once the AddApplication process has completed the AddApplication Response is returned to the calling interface. AddApplication Web Method (Service) Signature Add new Application information in the system. Test The test form is only available for methods with primitive types or arrays of primitive types as parameters. SOAP The following is a sample SOAP request and response. The placeholders shown need to be replaced with actual values. POST /BBLFINAL/BBL.asmx HTTP/1.1 Host: localhost Content-Type: text/xml; charset=utf-8 Content-Length: length SOAPAction: "http://tempuri.org/BBLFINAL/BBl/AddApplication"

121

USING SOA AND WEB SERVICES string string string string string string string string string string string string int int string string string string string string string string string string string int string string string int int string string string string string

122

USING SOA AND WEB SERVICES string string string string string string int string string string int int string string string string string string string string string string string int string string string int int string string string string string string string string string string

123

USING SOA AND WEB SERVICES string int string string string int int string string string string string string string string string string string int string string string int int string string string string dateTime string string dateTime string string string string int int string

124

USING SOA AND WEB SERVICES string string string string string string string string string string int string dateTime string dateTime string string string string string string string string int dateTime dateTime string string int dateTime dateTime string string string dateTime

125

USING SOA AND WEB SERVICES dateTime string string string string string dateTime dateTime string string string string string string string string string string int int string int int int string string string int int string HTTP/1.1 200 OK

126

USING SOA AND WEB SERVICES Content-Type: text/xml; charset=utf-8 Content-Length: length string



Once the AddApplication method is invoked and the place holders are filled with actual data, it acts as an agent of the application and is used to communicate and transfer data to the appropriate class. In this case, the AddApplication service of the WS is called and invokes the AddBaseData class which contains the method addNewApplication. Public Function AddApplication(ByVal oAddApp As AddBaseData) As String Try Return oAddApp.addNewApplication(oAddApp) Catch ex As Exception HandleException(ex) Finally End Try End Function If data access is needed then the data access layer is invoked and communication between the data access layer and data store is initiated. This communication remains open until the data access layer request is completed. The data access layer then responds to the calling application with the required data.

DATA ACCESS LAYER SDDM Technology created the data access layer using the Oracle Data Provider for .NET (ODP.NET) and VB.NET. The ODP.NET provider was capable of handling the various data types inherent to .NET. SDDM Technology was particularly impressed with ODP.NET’s ability to handle datetimestamp data types. In addition, the ODP.NET provider provided a better performance (speed) than the Oracle Managed Provider for .NET 1.1 Framework. Each table was wrapped into a class, each field was defined as a property in the given class and the respective methods were associated with each class i.e. add, update, delete, and retrieve. The individual class properties were used to validate basic business rules such as data type checking, null value validation, and field size checking.

127

USING SOA AND WEB SERVICES Oracle stored procedures were created to increase data access performance. Since the business logic was handled by the WS, the stored procedures were created for the basic tasks of adding, updating, deleting, and retrieving data. These individual procedures were grouped into four separate Oracle packages: pac_AddData, pac_UpdateData, pac_DeleteData, and pac_GetData. In addition, a package was created for generating and retrieving sequence values from the database. Example of Code Snippet of the basic package to Add Data for the License Periods of a License: drop package pac_AddData; CREATE OR REPLACE PACKAGE pac_AddData AS PROCEDURE addBBLLicPeriodsData(v_BBLAppID IN NUMBER, v_UpdateSeq IN NUMBER, v_IsCurrent IN VARCHAR2, v_IssueTMS IN DATE, v_IssueUserID IN VARCHAR2, v_EffectiveDate IN DATE, v_ExpiryDate IN DATE); END pac_AddData; -CREATE OR REPLACE PACKAGE BODY pac_AddData IS --Begin adding data to BBL_License_Periods table PROCEDURE addBBLLicPeriodsData(v_BBLAppID IN NUMBER, v_UpdateSeq IN NUMBER, v_IsCurrent IN VARCHAR2, v_IssueTMS IN DATE, v_IssueUserID IN VARCHAR2, v_EffectiveDate IN DATE, v_ExpiryDate IN DATE) IS BEGIN insert into BBL_License_Periods (BBL_APPLICATION_ID, UPDATE_SEQUENCE, IS_CURRENT, ISSUE_TMS, ISSUE_USER_ID, EFFECTIVE_DATE, EXPIRY_DATE) values (v_BBLAppID, v_UpdateSeq, v_IsCurrent, v_IssueTMS, v_IssueUserID, v_EffectiveDate, v_ExpiryDate); END addBBLLicPeriodsData; END pac_AddData;

CONCLUSION In summary, adopting and implementing the Web Service Oriented Architecture approach accomplished the goals set out by the DCRA. In the future, extensibility and scalability will be easier since many of the services are reusable in the area of license issuance and permits issuance. The ability to exchange data among the various District agencies and the public was great