This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
> Q5 ; ... Q5 : = Queue . c r e a t e <... >( M a x C a p a c i t y . QUEUE ) ; CAD . b i n d D i s p a t c h i n g Q u e u e <... >(Q5 , 5 ) ; ... thread threadgroup5 () { P a i r . t y p e<EventHandlerEnum , CAD . Event > p a i r ; EventHandlerEnum h a n d l e r ; CAD . E v e n t e v e n t ;
}
l o c l o c 0 : l i v e { h a n d l e r , e v e n t } when Queue . s i z e <... >(Q5) > 0 do i n v i s i b l e { p a i r : = Queue . g e t F r o n t <... >(Q5 ) ; Queue . dequeue <... >(Q5 ) ; h a n d l e r : = P a i r . f i r s t <... >( p a i r ) ; e v e n t : = P a i r . s e c o n d <... >( p a i r ) ; } goto l o c 1 ; l o c l o c 1 : l i v e {} i n v i s i b l e i n v o k e v i r t u a l f ( h a n d l e r , e v e n t ) goto l o c 0 ;
Fig. 8. Bogor dispatch queue and thread model for ModalSP (excerpts).
the Bogor model, queues are modeled using Queue and Pair extensions. Figure 8 illustrates the 5 hertz rate queue of pending event dispatches and the thread, threadgroup5, that cyclically dequeues dispatch pairs and invokes the component event handler encoded in each pair (note that pair type declarations are elided (i.e., <...>) for improved readability). Each correlator is represented as a deterministic finite-state automaton whose transition function is encoded as a static transition table. For each correlator, there is a single state variable that holds the current correlator state. Since the structure of correlators is fixed for a given system, the transition tables are not held in the state vector. 5.4
Summary of Data Portion of State-Vector
To summarize the modeling strategy discussed above, we present the state vector components related to data state of Cadena systems. The observable state of a Cadena assembly is comprised of all non-fixed system data. As we have noted above, correlator transition tables, subscriber lists, and component connection information are all fixed and are not considered part of the observable state. Definition 1. Cadena Data States are tuples (c, r, a, t, p) where: c = (c1 , . . . , ck ) stores the data states of component instances, each of which is comprised of a, possibly empty, set of mode attributes as defined by ci ’s component type. r = (qr1 , . . . , qrn ) are rate-specific queues of pairs, (c, e), recording the dispatch of event e to port c. a = (a1 , . . . , al ) stores the current states of each of the event correlation recognition automata. t records an abstraction of time used to trigger timeouts. p records the priority of the current thread being executed.
Model-Checking Middleware-Based Event-Driven Real-Time
171
The initial state is defined to have instance modes set to their initial values, correlation automata set to their start state, rate specific queues to be empty, t = 0, and the priority variable is set to the highest priority. The values of local variables in component handlers and methods and in the implementation of push methods and rate-specific threads cannot be observed outside their method activation by other threads or by property observables and are also not considered part of the observable state. Local variable are held in the state vector, but only during the corresponding method activations. In addition to the data state described above, Bogor maintains control information in the form of a program counter and a method call-stack frame for each thread. 5.5
Strategies for Modeling Scheduling and Time
The behavior of Cadena systems is driven by the triggering of middleware timeouts as described in Section 4 and is controlled by the scheduling policies of the thread-pool in the real-time event channel. Finding an effective strategy for modeling these timeouts and thread-scheduling is a central issue in the construction of Cadena models. When analyzing concurrent systems, most model-checkers do not attempt to exploit knowledge of specific timing or scheduling strategies but instead explore all possible interleavings of concurrent actions. If we followed this approach, we would allow timeout events to occur non-deterministically between every system transition and we would allow actions from different threads to be interleaved non-deterministically without consideration of priorities or other scheduling constraints. While such a strategy is sound in that it covers all possible system behaviors, the number of states generated makes it impractical for all but the smallest systems. In the subsections below, we describe several strategies that we use to reduce infeasible interleavings. Each strategy incorporates constraints based on observations about priority scheduling and timeout policies implemented by the real-time middleware. Priority-based scheduling: Having the model-checker non-deterministically explore interleavings without considering thread priorities obviously introduces schedules that are infeasible in the actual system, e.g., a schedule that continues to execute transitions from a lower priority thread even though a higher-priority thread is enabled. Inter-rate-group timeout constraints: Having the model-checker nondeterministically generate timeout events introduces schedules that are infeasible in the actual system, e.g., a 5 Hz timeout event should not occur more frequently than a 20 Hz timeout event. We present strategies that reduce infeasible interleavings by taking into account the appropriate relative frequency of timeout events, i.e., by taking into account constraints that exist between timeouts of different rate groups. Intra-rate-group timeout constraints: Having the model-checker nondeterministically generate timeout events introduces infeasible schedules
172
X. Deng et al.
where a timeout for a rate group r occurs before all events in the current frame for r are dispatched or before the previous timeout from group r is even dispatched. We constrain the generation of time-out events to ensure that timeouts from the same rate group are not triggered “too quickly”. This strategy constrains the occurrence of timeouts by considering the relative lengths of the real-time frames and constrains scheduling by considering priority information. Lazy-time with priority scheduling: In addition to the techniques used in the strategy above, this strategy also considers timing estimates for system transition which allows additional infeasible schedules to be removed from consideration. 5.6
Representing Priority-Based Scheduling Information
Bold Stroke systems are priority scheduled based on the results of rate monotonic analysis of a set of harmonic rate groups. The CAD call connectEvent(), illustrated in Figure 7, assigns a rate, and hence a priority, to each component handler for a given event. The default non-deterministic scheduling policy in Bogor is implemented by a module that calculates the set of enabled transitions in a given state and passes that set to the state exploration module, which explores each possible outgoing transition. When reporting our experiments, we refer to models that use this strategy as priority-unaware. For Cadena models, a Bogor plugin is used that intercepts the set of enabled transitions in a given state, selects the transitions with the highest priority and passes these transitions on to the state exploration module. As expected, this yields dramatic reductions in the state space, as shown in Section 6, and also improves the precision of the state space since only infeasible schedules are eliminated (i.e., ones on which a lower-priority transition executes when a higher-priority transition is enabled). We refer to models that use this strategy as priority aware. Variations of this plugin are used in the following models to allow for interleaving of timeouts with the highest-priority enabled transition. 5.7
Representing Intra-rate-group Timing Constraints
The treatment of time, t, determines, in part, the fidelity of the model with respect to the real system’s behaviors. If detailed timing information is available one can keep track of time as component actions are executed and use that time value to trigger periodic events. However, even when timing information is not available, one can still reduce the occurrence of timeout events based on both intra- and inter-rate-group constraints. Intra-rate-group constraints that we consider involve the notion of frame overrun. A frame overrun occurs when a timeout event er for rate group r occurs before all events e triggered directly or indirectly by the previous timeout for r are processed by the rate group’s thread tr . In normal situations, a timeout er occurs and is dispatched, other events arrive in the event channel’s dispatch queues (including those associated with r), and thread tr becomes idle after all
Model-Checking Middleware-Based Event-Driven Real-Time
173
events associated with r have been dispatched. The time that tr remains idle waiting for the next r timeout is called slack time. If a system has a frame overrun error, a thread tr has no slack time – it is unable to finish all of its work before the next timeout er arrives. Note that exploring the state-space of systems where arbitrary frame overruns are modeled results in a huge number of additional system behaviors that would very likely be infeasible if actual timing data were considered (timing data would allow us to conclude that in most cases frame overruns do not occur). While frame overruns are a real source of bugs in Bold Stroke systems, engineers have other tools and methods for detecting these types of errors. Accordingly, we will reduce the state space that we explore using two strategies. The first strategy which we call no overruns assumes that no frame overruns occur at all. This is implemented by having the model-checker scheduler only emit a timeout event for rate group r if there are no enabled transitions associated with rate group r – which models the situation where tr has become idle because its associated dispatch queue is empty. The second strategy which we call limited overruns is implemented by having the model-checker scheduler only emit a timeout event er if there is no other timeout event remaining in the r dispatch queue (but other non-timeout events may still be waiting in the queue for dispatch). Intuitively, this model includes overruns that only spill over into the very next frame but does not include overruns where processing is ‘late’ by more than one additional frame. 5.8
Representing Inter-rate-group Timing Constraints
The strategies related to frame overrun in the previous section constrain timeout events by considering when they should occur relative to other timeouts from the same rate group. We now describe a strategy which we call the relativetime (RT) strategy that constrains the issuing of timeout events by considering when a timeout for r should occur relative to a timeout for a different rate group r . Specifically, we take advantage of the fact that in rate-monotonic scheduling theory (which is used in Bold Stroke systems), the frame associated with a rate can be evenly divided into some whole number of r -frames for each rate r that is higher than r. In the example system of Figure 3, the frame of the slowest rate (1 Hz) can be divided into 5 5 Hz frames, and each 5 Hz frame can be divided into 4 20 Hz frames. The longest frame/period (the frame associated with the lowest rate) is called the hyper-period. The relative-time model enforces the following constraints related to issuing of timeouts: – a single timeout is issued for the slowest rate group in the hyper-period, – timeouts for rate groups, ri and rj where ri > rj , are issued such that ri /rj timeouts of rate ri are issued in a rj frame. These constraints determine the total number and relative ordering of instances of timeouts that may occur in the hyper-period.
174
X. Deng et al.
CAD . Component Timer ; ... Timer : = CAD . c r e a t e C o m p o n e n t ( ” Timer ” ) ; CAD . d e c l a r e E v e n t S o u r c e P o r t <EventType >(Timer , ” timeOut5 ” , EventType . TimeOut ) ; ... thread timerThread () { l o c l o c 0 : l i v e {} when t r u e do { t i m e : = ( t i m e + 1 ) % 2 0 ; } goto l o c 0 ; } ... thread timeOutSenderThread ( ) { ... l o c l o c 1 : / / 5 Hz t i m e o u t c a s e when t i m e % ( 2 0 / 5 ) = = 0 do i n v i s i b l e { } goto l o c I n v o k e 5 ; when t i m e % ( 2 0 / 5 ) ! = 0 do i n v i s i b l e { } goto l o c 2 ; ... loc locInvoke5 : l i v e { localTime} i n v i s i b l e i n v o k e p u s h O f P r o x y ( Timer , ” timeOut5 ” , CAD . c r e a t e E v e n t <EventType > ( EventType . TimeOut ) ) goto l o c 2 ; ... l o c l o c 2 : / / 1 Hz t i m e o u t c a s e ... }
Fig. 9. Timer and TimeOutSender thread models for ModalSP (excerpts).
Figure 9 shows the Bogor code for two threads that are used to model this strategy. Thread timerThread increments an abstraction of time where each ’tick’ (i.e., each increment of the time variable) represents the passing of time corresponding to the shortest frame in the system (e.g., in the ModalSP, each tick represents a 20 Hz frame). The time variable wraps around every 20 ticks which corresponds to the fact that there are 20 Hz frames in the 1 Hz hyperperiod. Thread timeOutSenderThread models the behavior of the rate-specific timer threads in the middleware discussed in Section 4. This thread monitors time and when it observes a change in the time value, it passes through a case statement to see which timeout events should be dispatched at that point. Since a time tick represents the period of the shortest frame, a new timeout event for the fastest rate is issued on each pass through the case statement. In our example system, the 5 Hz timeout happens every fourth tick. To represent the occurrence of a timeout, the thread enqueues the timeout event through the standard push call. From the explanation above, it is clear that the RT model only establishes the occurrence of timeouts relative to each other – it does not relate timeout occurrences to the time required by component event handlers and method execution. Thus, it is now important to understand when timeout actions may occur with respect to actions that occur inside of component handlers, i.e., when can these actions be interrupted by timeouts. To see that the model safely approximates all interleavings of timeouts and component actions (given the constraint on no frame overruns) consider Figure 10. This figure illustrates four points during a system execution which contains 5 Hz and 10 Hz rate processing. The 10 Hz and 5 Hz timeouts are queued together (e.g., at the point 1) since they both have frames that begin at the
Model-Checking Middleware-Based Event-Driven Real-Time
10Hz and 5Hz
175
10Hz timeout
10Hz
10Hz execution step 5Hz timeout 5Hz execution step
10Hz priority delay interleaving
5Hz time 1
2
3
4
Fig. 10. Relative-time Environment.
same point. However, the 10 Hz timeout event is dispatched first due to its higher priority. Once the all the actions associated with 10 Hz component processing complete (e.g., at point 2), the model-checker scheduler begins consideration of lower priority actions and the 5 Hz timeout is dispatched leading to 5 Hz component processing. Our no overruns assumption entails that processing the 10 Hz component actions does not require more time than the period of the 10 Hz frame — thus, the next 10 Hz timeout cannot occur before point 2. Since we are not modeling the actual time required for carrying out component actions, it impossible to determine the relationship between the time required for 5 Hz component action processing (e.g., the duration from point 2 to 4) and the time until the next 10 Hz timeout (e.g., the duration from point 2 to 3). To safely cover all possibilities, we must allow for any relationship between these durations. To model all such relationships, we adapt Bogor’s standard scheduling module to consider all interleavings of enabled timeouts with the enabled transitions. On the right in Figure 10 the interleavings of the 10 Hz timeout and the enabled transitions performed during 5 Hz component processing are illustrated. The dark grey double circle represents the dispatching of the leftmost 10 Hz timeout event. This is followed by three dark grey circles representing transitions in 10 Hz component processing: the first branch point represents the choice between the next 10 Hz timeout (on the left) or dispatching the already queued 5 Hz timeout event (on the right). If 5 Hz processing is selected then the choice between the 10 Hz timeout and 5 Hz processing repeats for the next enabled 5 Hz transition illustrated as a light grey circle. 5.9
Lazily-Timed Components
In the relative-time model, timeouts are arranged in a proper order and ratio with respect to each other, but there are no constraints that guarantee that, e.g., the interval between time outs is appropriate for the correspond period. This means that the model may have interleavings in which a timeout, e.g., for ri , occurs prematurely with respect to an action sequence whose duration is less than period(ri ). For example, if the 5 Hz component processing (i.e., from point 2 to 4) in Figure 10 is guaranteed to be less than the time to the next
176
X. Deng et al. 10Hz
10Hz priority delay
5Hz time 1
2
3
Fig. 11. Lazily-timed Environment.
10 Hz timeout (i.e., point 4 comes before point 3) then the interleavings of 10 Hz timeouts with 5 Hz processing in the RT model will be infeasible. The lazilytimed (LT) component model addresses this by leveraging worst-case estimates of the running time of components; these will be available for Cadena systems to support rate monotonic analysis. This model can be configured for whatever granularity of timing information is available. Here we consider worst-case timing estimates for event handlers. Conceptually, the estimates are used to determine whether a handler can run without interruption before the next timeout occurs and, if not, the model non-deterministically interleaves action sequences from the handler with timeouts and higher-priority actions that follow from timeouts. This model modifies the data associated with time to record the intra-hyperperiod (IHP) time normalized by the least common factor of all handler durations and timeout periods, the guards in timeOutSenderThread from Figure 9 are adjusted accordingly, and each component handler is modified to include an increment of time. Figure 11 illustrates how these increments are performed. It shows the execution of 5 Hz component processing subsequent to completion of 10 Hz processing in a frame. There are two cases: (1) the worst-case time estimate of the 5 Hz processing (i.e., which runs up to point 2) is less than or equal to the next timeout (i.e., timeout occurs at point 3) or (2) it is not (i.e., timeout occurs at point 1 and interrupts the 5 Hz actions). In case (1), the IHP time is incremented by the worst-case timing estimate of the currently running 5 Hz event handler and the state space exploration algorithm proceeds; note that there is no branching in the state space for this case. In case (2), the IHP time is incremented to the next timeout (i.e., point 1), a non-deterministically chosen prefix of the currently running 5 Hz handler is executed, and then the 10 Hz timeout is performed. By choosing a prefix of the handler actions, we are modeling all possible distributions of timing across the actions of the handler. The remaining portion of the handler is left for the state-space exploration algorithm after the 10 Hz timeout, and subsequent 10 hertz processing is performed. The difference between point 2 and point 1 (i.e., the worst-case execution time of the handler remaining time of the 10 Hz frame) is assigned to that remaining portion as its duration. This model can be seen as a refinement of the RT model. It eliminates interleavings when the timing estimates guarantee that a group of highest-priority enabled transitions are guaranteed to complete before the next timeout. In the example in Figure 10, if the right-most three light-grey circles correspond to a
Model-Checking Middleware-Based Event-Driven Real-Time
177
5 Hz component handler body whose worst-case execution bound is less than the time to the next 10 Hz timeout, then there would be no branching in that portion of the state space (i.e., the lower two left outgoing arcs to 10 Hz timeouts are eliminated).
6
Experimental Results
Table 1 shows the results of evaluating our strategies using four example systems provided by Boeing engineers. As an example of how to read a system description, the ModalSP scenario that we have used as an example has three threads (for rate groups 1 Hz, 5 Hz, and 20 Hz), 8 components, an event correlation (e/c), and 125 events being generated per one second hyper-period (hp). For each scenario, we give data for five models that incorporate the modeling strategies presented in the previous section. – (R) is the reference model. There is no scheduling policy for the thread groups in the scenario (it is priority unaware and has no intra-rate-group timing constraints). Since a completely interleaved execution is infeasible to check, the relative time constraints are used though. – (RT-1) uses two policies: priority aware scheduling and the relative time environment where we implement the no frame overruns strategy for the highest-priority thread only. – (RT-2) is like (RT-1), but also assumes there are no frame overruns for all threads. – (LT) is like (RT-2) but uses the lazy time environment model. For each example, we collect the number of transitions trans, states, time, and memory consumption mem at the end of the search. The numbers of transitions and states are both listed because some of steps in the model are marked as invisible (atomic) for which Bogor will not save the states. The experiments were run on a Pentium 4 2.53 GHz with 1.5Gb RAM using the Java 2 Platform. Bogor’s collapse compression[17] and heap symmetry [18] and process symmetry [2] reductions are used in all of the experiments. Each of the experiments represents a complete exploration of the state-space of the system. From the table, the state space generally decreases from model (R), (RT-1), (RT-2), to (LT). This shows that by incorporating more knowledge (e.g., the scheduling policy) of the model that is being checked, less states need to be explored. For example, Medium, the largest scenario that we have, cannot be model checked using Bogor or our previous dSpin implementation [14] without employing the reduction strategies used in (RT-2) and (LT). For Basic the states are the same for model (RT-1) and (RT-2) because it only has a single thread (thus, there is no interleaving). Model (R) has a larger number of states because the lack of constraints allows the timeout to occur even when events associated with the current frame are still being dispatched. Model (LT) has two more states than (RT-2) due to the overhead introduced by the timing transitions. Bogor runs out of memory checking ModalSP (R) (at 3 million states) and Medium (RT-1) (at 13 million states). It is interesting that the states for ModalSP
178
X. Deng et al. Table 1. Experiment Data. Example System Basic Scenario Threads: 20Hz Components: 3 Events: 2 per .05sec hp Multi-Rate Scenario Threads: 20Hz, 40Hz Components: 6 Events: 6 per .05sec hp ModalSP Scenario Threads: 1Hz, 5Hz, 20Hz Components: 8 (e/c) Events: 125 per 1sec hp Medium Scenario Threads: 1Hz, 20Hz Components: 50 Events: 820 per 1sec hp
(R) trans 111 states 20 time .16 sec mem .51Mb trans 1.36M states .12M time 5 min mem 16Mb trans o.m. states 3M+ time o.m. mem o.m. trans o.m. states — time o.m. mem o.m.
(RT-1) 42 12 .11 sec .5Mb 7.5K 1.5K 1.9 sec .77Mb .92M 20.9K 20 sec 4.1Mb o.m. 13M+ o.m. o.m.
(RT-2) 42 12 .09 sec .5Mb .98K .1K .38 sec .61Mb 38.2K 9.1K 8.59 sec 1.61Mb 3.79M .74M 29 min 71.8 Mb
(LT) 44 14 .11 sec .51Mb .15K 33 .19 sec .61Mb 6.27K 1.56K 2.11 sec 1.45Mb .36M 74.5K 3 min 21.5Mb
(R) require more memory than the states for Medium (RT-1). This is an effect of the collapse compression that is used. Specifically, there are three threads in ModalSP (R), but only two threads in Medium (RT-1). In addition, ModalSP (R), which has fewer scheduling constraints, allows more interleaving than Medium (RT-1). Thus, the collapse compression can save more in Medium (RT-1) than ModalSP (R), because there are more similar state bit patterns in Medium (RT-1) than in ModalSP (R).
7
Related Work
Garlan and Khersonsky [11] describe an approach for checking publish-subscribe systems (which they refer to as implicit invocation systems) using SMV. We build on their key insights of factoring system models into two parts: (1) a reusable model of run-time event-delivery infrastructure, and (2) application dependent, user-specified component models. However, we support much more directly the forms of component structure and connections (i.e., CCM structures and object references) and event-delivery mechanisms (i.e., RT CORBA middleware) found in real systems. This advance is achieved by leveraging Bogor’s direct modeling of OO concepts (as compared to the SMV input language which provides very little support for modeling programming language features) and Bogor’s extension mechanism (which allows complex middleware behavior to be captured internally to the model-checker). In addition, [11] does not provide any performance data, nor does that work consider any state-space reductions based on priorities, scheduling, or timing constraints that seem critical for scaling to realistic applications.
Model-Checking Middleware-Based Event-Driven Real-Time
179
There has been a large body of work on timing and schedulability analysis for component-based systems (see also [12] in this volume). As these techniques have matured, they have been integrated into environments that support the development of real-time systems. For example, MetaH [25] and Geodesic [6] are frameworks that support the reuse of components written in Ada and Java, respectively, in real-time systems. These frameworks include a range of timing analyses and automatically generate infrastructure code that coordinates the execution of component code in a way that achieves the system’s timing requirements. Cadena is complementary to this work in that it targets logical properties of a system using both light-weight and heavy-weight analysis techniques. Ptolemy [20] is a framework that allows a wide variety of formal descriptions of components and their behavior to be integrated into a single system. User’s provide sufficient detail in these descriptions to allow implementations to be automatically generated. Ptolemy provides a run-time infra-structure to mediate between components that have different execution models. In contrast, Cadena models intentionally leave out detail in order to provide more abstract system descriptions that are amenable to analysis for large systems. While Cadena provides some code generation capabilities, we do attempt to generate component method implementations. Model checking [4] has become extremely popular as a technology for analyzing behavioral models of software artifacts. Researchers have extracted such models from source code (e.g., [15,5]), UML design artifacts (e.g., [21,19]) and architectural descriptions (e.g., [1]). The difficulty with all applications of model checking is scaling it up to apply to realistically large and complex systems. Recent years have seen an enormous amount of research on the systematic abstraction of models to enable more tractable reasoning. We take a different approach in Cadena by exploiting the natural abstractions that arise when developing high-level design models of systems.
8
Conclusions
We believe the idea of a flexible model-checking framework that allows domainspecific extensions (such as the ones that we have used for encoding models of CORBA communication layers) can be a very effective approach for modelchecking modern distributed system designs and implementations. We are currently working with Boeing engineers to incorporate other forms of domain information (e.g., common specification idioms) and other forms of light-weight checking (e.g., interface protocol checking, refinement checking) and static analysis into Cadena.
Acknowledgements Thanks to various members of the Boeing Bold Stroke team including David Sharp, Carol Sanders, Wendy Roll, and Dennis Noll for their comments on the work described in this paper.
180
X. Deng et al.
References 1. R. Allen and D. Garlan. A formal basis for architectural connection. ACM Transactions on Software Engineering and Methodology, July 1997. 2. D. Bosnacki, D. Dams, and L. Holenderski. Symmetric spin. In International Journal on Software Tools for Technology Transfer. Springer-Verlag, 2002. 3. G. Brat, K. Havelund, S. Park, and W. Visser. Java PathFinder – a second generation of a Java model-checker. In Proceedings of the Workshop on Advances in Verification, July 2000. 4. E. Clarke, O. Grumberg, and D. Peled. Model Checking. MIT Press, 2000. 5. J. C. Corbett, M. B. Dwyer, J. Hatcliff, S. Laubach, C. S. P˘ as˘ areanu, Robby, and H. Zheng. Bandera : Extracting finite-state models from Java source code. In Proceedings of the 22nd International Conference on Software Engineering, June 2000. 6. D. de Niz and R. Rajkumar. Geodesic - a reusable component framework for embedded real-time systems. Technical report, Carnegie Mellon University, 2002. 7. C. Demartini, R. Iosif, and R. Sisto. dspin : A dynamic extension of SPIN. In Theoretical and Applied Aspects of SPIN Model Checking (LNCS 1680), Sept. 1999. 8. W. Deng, M. Dwyer, J. Hatcliff, G. Jung, Robby, and G. Singh. Model-checking middleware-based event-driven real-time embedded software (extended version). Forthcoming – April 2003. 9. B. Doerr and D. Sharp. Freeing product line architectures from execution dependencies. In Proceedings of the Software Technology Conference, May 1999. 10. Eclipse Consortium. Eclipse website. http://www.eclipse.org, 2001. 11. D. Garlan and S. Khersonsky. Model checking implicit-invocation systems. In Proceedings of the 10th International Workshop on Software Specification and Design, Nov. 2000. 12. G. Goessler and J. Sifakis. Composition for component-based modeling. In Proceedings of the First International Symposium on Formal Methods for Components and Objects, volume (this volume) of Lecture Notes in Computer Science. Springer, 2002. 13. T. H. Harrison, D. L. Levine, and D. C. Schmidt. The design and performance of a real-time corba event service. In Proceedings of the 1997 ACM SIGPLAN conference on Object-oriented programming systems, languages and applications, pages 184–200. ACM Press, 1997. 14. J. Hatcliff, W. Deng, M. Dwyer, G. Jung, and V. Prasad. Cadena: An integrated development, analysis, and verification environment for component-based systems. In Proceedings of the 25th International Conference on Software Engineering (to appear), 2003. 15. K. Havelund and T. Pressburger. Model checking Java programs using Java PathFinder. International Journal on Software Tools for Technology Transfer, 1999. 16. G. J. Holzmann. The model checker SPIN. IEEE Transactions on Software Engineering, 23(5):279–294, May 1997. 17. G. J. Holzmann. State compression in SPIN: Recursive indexing and compression training runs. In Proceedings of Third International SPIN Workshop, Apr. 1997. 18. R. Iosif. Symmetry reduction criteria for software model checking. In Proceedings of Ninth International SPIN Workshop, volume 2318 of Lecture Notes in Computer Science, pages 22–41. Springer-Verlag, Apr. 2002.
Model-Checking Middleware-Based Event-Driven Real-Time
181
19. D. Latella, I. Majzik, and M. Massink. Automatic verification of a behavioural subset of UML statechart diagrams using the SPIN model-checker. Formal Aspects of Computing, 11(6):637–664, 1999. 20. E. A. Lee. Overview of the ptolemy project. Technical Report UCB/ERL M01/11, University of California, Berkeley, Mar. 2001. 21. J. Lilius and I. P. Paltor. vUML: A tool for verifying UML models. In Proceedings of the 14th IEEE International Conference on Automated Software Engineering, 1999. 22. Robby, M. B. Dwyer, and J. Hatcliff. Bogor: An extensible and highly-modular model checking framework. In Proceedings of the 2003 ACM Symposium on Foundations of Software Engineering (FSE 2003), 2003. 23. Robby, M. B. Dwyer, and J. Hatcliff. Bogor Website. http://www.cis.ksu.edu/bandera/bogor, 2003. 24. H. Sipma. Event correlation: A formal approach. Technical Report Draft, Stanford University, July 2002. 25. S. Vestal. Metah user’s manual. http://www.htc.honeywell.com/metah, 1998.
Equivalent Semantic Models for a Distributed Dataspace Architecture Jozef Hooman1,2 and Jaco van de Pol2 1
University of Nijmegen, Nijmegen, The Netherlands [email protected] http://www.cs.kun.nl/∼hooman/ 2 CWI, Amsterdam, The Netherlands [email protected] http://www.cwi.nl/∼vdpol/
Abstract. The general aim of our work is to support formal reasoning about components on top of the distributed dataspace architecture Splice. To investigate the basic properties of Splice and to support compositional verification, we have defined a denotational semantics for a basic Splice-like language. To increase the confidence in this semantics, also an operational semantics has been defined which is shown to be equivalent to the denotational one using the theorem prover PVS. A verification framework based on the denotational semantics is applied to an example of top-down development and transparent replication.
1
Introduction
The general aim of our work is to support the development of complex applications on top of industrial software architectures by means of formal methods. As a particular example, we consider in this paper components on top of the software architecture Splice [Boa93,BdJ97] which has been devised at Thales Nederland (previously called Hollandse Signaalapparaten). It is used to build large and complex embedded systems such as command and control systems, process control systems, and air traffic management systems. The main goal of Splice is to provide a coordination mechanism between loosely-coupled heterogeneous components. In typical Splice-applications it is essential to deal with large data streams from sensors, such as radars. Hence Splice should support real-time and high-bandwidth distribution of data. Another aim is to support fault-tolerance, e.g., it should be possible to replicate components transparently, i.e. without affecting the overall system behaviour. Splice is data-oriented, with distributed local databases based on keys. It uses the publish-subscribe paradigm to get loosely-coupled data producers and consumers. An important design decision is to have minimal overhead for data management, allowing a fast and cheap implementation that indeed allows huge data streams. For instance, Splice has no standard built-in mechanisms to ensure global consistency or global synchronization. If needed, this can be constructed for particular data types on top of the Splice primitives. Note that this is quite F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 182–201, 2003. c Springer-Verlag Berlin Heidelberg 2003
Equivalent Semantic Models for a Distributed Dataspace Architecture
183
different from Linda [Gel85] and JavaSpaces [FHA99], which have a central data storage. The latter also has a transaction mechanism to handle distributed access. Our aim is to reason about components of the distributed dataspace architecture Splice in a compositional way. This means that we want to deduce properties of the parallel composition of Splice-components using only the specifications of the externally visible behaviour of these components. Compositionality supports verification of development steps during the development process. Many examples in the literature show that it is convenient to specify components using explicit assumptions about the environment. Concerning Splice, in [HH02] we propose a framework with an explicit assumption about the quality of data streams published by environment and a similar commitment of the component about its produced data. When putting components in parallel, assumptions can be discharged if they are guaranteed by other components. Reasoning with assumption/commitment [MC81] or rely/guarantee [Jon83] pairs, however, easily leads to unsound reasoning. There is a danger of circular reasoning, two components which mutually discharge each others assumptions, leading to incorrect conclusions. Hence it is important to prove the soundness of the verification techniques. Correctness of compositional verification rules is usually based on a denotational semantics which assigns a meaning to compound constructs based on the meaning of its constituents. Earlier work on the verification of Splicesystems [HvdP02] was based on a complex semantics with environment actions and its correctness was not obvious. In this paper we define a denotational semantics for a simple Splice-like language which is more convenient as a basis for compositional reasoning using assumptions about the environment. It is, however, far from trivial that this semantics captures the intuitive understanding of the Splice architecture. Hence, we also provide a more intuitive operational semantics and prove formally that it is equivalent to the denotational one. Moreover, we show by a small example of transparent replication how the denotational framework can be used to support verification. Both versions of the semantics have been formulated in the language of the interactive theorem prover PVS [OSRSC01]. The equivalence result has been checked completely using PVS. Also the example application (transparent replication) has been verified in detail using PVS. Related to our semantic study is earlier work on the semantics of Splice-like languages such as a transition system semantics for a basic language of write and read statements (without query) [BKBdJ98] and a comparison of semantic choices using an operational semantics [BKBdJ98,BKZ99]. In previous work on a denotational semantics for Splice [BHdJ00] the semantics of local storages is not very convenient for compositional verification; it is based on process identifiers and a partial order of read and write events with complex global conditions. New in this paper is an operational semantics that deals with local time stamps and their use for updating local databases. We define an equivalent denotational semantics which includes assumptions about the environment of a
184
J. Hooman and J. van de Pol
application process
application process
...... read
read
write
write local database
local database
......
agent send
agent receive
send
receive
Fig. 1. Splice applications.
component. Moreover, we show that this denotational semantics forms the basis of a formal framework for specifying and verifying Splice applications. This paper is structured as follows. Section 2 contains a brief informal explanation of Splice. In Sect. 3 we present a formal syntax of the Splice primitives considered in this paper. The operational and denotational semantics of this language are defined in Sect. 4 and Sect. 5, respectively. The main outline of the equivalence proof can be found in Sect. 6. A framework for specification and verification is described in Sect. 7 and applied to an example with top-down design and transparent replication in Sect. 8. Finally, Sect. 9 contains a number of concluding remarks.
2
Informal Splice Introduction
The Splice architecture provides a coordination mechanism for concurrent components. Producers and consumers of data are decoupled. They need not know each other, and communicate indirectly via the Splice primitives; basically readand write- operations on a distributed dataspace. This type of anonymous communication between components is strongly related to coordination languages such as Linda [Gel85] and JavaSpaces [FHA99]. These languages, however, have a single shared dataspace, whereas in Splice each component has its own dataspace, see Fig. 1. Communication between components takes place by means of local agents. A data producer writes data records to the other dataspaces via its agent. A data consumer uses its agent to subscribe to the required types of data; only data which matches this subscription is stored. Data items may be delayed and re-ordered and sometimes may even get lost. It is possible to associate certain quality-of-service policies with data delivery and data storage. For instance, for a particular data type, delivery maybe guaranteed (each item is delivered at least once) or best effort (zero or more times). Data storage can be volatile, transient, or persistent.
Equivalent Semantic Models for a Distributed Dataspace Architecture
185
Each data item within Splice has a unique sort, specifying the fields the sort consists of and defining the key fields [BdJ97]. In each local dataspace, at most one data item is present for each key. Basically, a newly received data item overwrites the current item with the same key (if any). To avoid that old data items overwrite newer information (recall that data may be delayed and re-ordered), data records include a time stamp field. A time stamp of a data item is obtained from the local clock of the data producer when the item is published. At the local storage of the consumer, data items are only overwritten if their time stamp is smaller than that of a newly arrived item (with the same key). This overwriting technique reduces memory requirements and allows a decoupling of frequencies between producers and consumers. It also reduces the number of updates to be performed on the dataspace, as not all received records get stored. The timestamps improve the quality of the data stored, as no record can be overwritten by older data. To program components on top of Splice, a Splice API can be called within conventional programming languages such as C and Java. Splice provides, for instance, constructs for retrieving (reading) data from the local dataspace, and for publishing (writing) data. Read actions contain a query on the dataspace, selecting data items that satisfy certain criteria. Data has a life-cycle attribute, such as “read”, “unread”, “new”, and “update”, which may be used for additional filtering.
3
Syntax of a Simple Splice-Like Language
In this section we define the formal syntax of a very simple Splice-like language. We have embedded the basic Splice primitives in a minimal programming language to be able to high-light the essential features and to prove equivalences between various semantic definitions in a formal way. In Sect. 7.1 we show that it is easy to extend the language with, e.g. if-then-else and loop constructs. Lifecycle attributes are not included in the current version, because we detected some problems with the informal definition (as mentioned in Sect. 9). We consider only one sort. Let Data be some data domain, with a set KeyData of key data and a function key: Data → KeyData. Assume a given type LocalT ime to represent values of local clocks (in our PVS representation we choose the natural numbers). The type DataItems of time-stamped data items, consists of records with two fields: dat of type Data and ts of type LocalT ime. A record of type DataItems can be written as (#dat := ..., ts := ...#), following the PVS notation. Hence, for di ∈ DataItems, we have dat(di) ∈ Data and ts(di) ∈ LocalT ime. The function key is extended to DataItems by key(di) = key(dat(di)). Let SV ars be a set of variables ranging over sets of elements from DataItems. A data expression e yields an element from Data (dependent on the current value of the program variables). Henceforth we typically use the following variables ranging over the types mentioned above:
186
– – – – –
J. Hooman and J. van de Pol
d, d0 , d1 , . . . over Data lt, lt0 , lt1 , . . . over LocalT ime dati, dati0 , dati1 , . . . over DataItems diset, diset0 , diset1 , . . . over sets of DataItems x, x0 , x1 , . . . , y, y0 , y1 , . . . over SV ars
A query q ⊆ P(diset) specifies sets of (time-stamped) data items (it may also depend on program variables). A few examples: – q1 = {diset | for all dati ∈ diset: ts(dati) > 100} = 0} – q2 = {diset | for all dati ∈ diset: dat(dati) = ø and diset = x} – q2 = {diset | diset For simplicity, we do not give the syntax of data expressions and queries here. The syntax of our programming language is given in Table 1. Table 1. Syntax Programming Language. Sequential program S ::= Write(e) | Read(x, q) | S1 ; S2 Process
P ::= S | P1 P2
Informally, the statements of this language have the following meaning: – Write(e) publishes the data item with value e (in the current state) and the current time stamp (from the local clock). We model best effort delivery; a data item arrives 0 or more times at each process, where it might be used to update the local storage if there is no current value with the same key which has a larger or equal time stamp. – Read(x, q) assigns to x some set of data items from local storage that satisfies query q (if there are several sets satisfying q, the choice is non-deterministic). For instance, for query q1 above, we assign to x a set of data items from local storage such that each data item has time stamp greater than 100. If there are no such items in local storage, x becomes the empty set. Note that query q3 above does not allow the empty set; then the read statement blocks until the local storage contains a set of items satisfying the query. Hence a read statement may be blocking, depending on the query. – S1 ; S2 : sequential composition of sequential programs S1 and S2 . – P1 P2 : parallel composition of processes. A process is a sequential program or a parallel composition of processes. As a very simple example, consider a few producers and consumers of flight data. Let Data be a record with two fields: flightnr (a string, e.g. KL309) and pos a position in some form, here a natural number for simplicity. The flight number is the key, that is, key(dati) = flightnr(dati). Consider a producer of flight data
Equivalent Semantic Models for a Distributed Dataspace Architecture
187
P1 = Write((#flightnr := KL567, pos := 1#)) ; Write((#flightnr := LU 321, pos := 6#)) ; Write((#flightnr := KL567, pos := 2#)) ; Write((#flightnr := KL567, pos := 3#)) and two consumers: C1 = Read(x1 , true) ; Read(y1 , q1 ) ; Read(z1 , q1 ) C2 = Read(x2 , q1 ) ; Read(y2 , q2 ) whose queries are specified as follows: = ø and for all dati ∈ diset : flightnr(dati) = KL567} q1 = {diset | diset = ø and for all dati ∈ diset : flightnr(dati) = KL567 and q2 = {diset | diset for all dati1 ∈ x2 : ts(dati) > ts(dati1 )} Consider the process P1 C1 C2 and assume there are no other producers of data. Note that the producer does not specify the local time stamp explicitly; this is added implicitly. Recall that the items produced by P1 may arrive at a different order at the consumers, and they may arrive several times. Variable x1 may be empty (if no data item has been delivered yet – note that this read is not blocking) or a set with one or two elements, at most one for each flight number. For instance, it may contain position number 3 for KL567. Variable y1 will be a singleton, since the local storage contains at most one item with flight number KL567 and the second read is blocking (the query requires a non-empty set). If there is a position for KL567 in x1 , then the position in y1 will be greater or equal (lower values are produced earlier, hence have a smaller local clock value, and thus they cannot overwrite greater values). Similarly for z1 , where the position is greater or equal than the one in y1 . It is possible that z1 = y1 . For consumer C2 the second read action requires a newer time stamp, hence we always have y2 = x2 and the position in y2 is at least 2.
4
Operational Semantics
We define an operational semantics for a process S1 ... Sn of the syntax of Sect. 3. where the Si are sequential programs. We first define an operational status of a sequential process (Def. 1) and a configuration (Def. 3) which represents the state of affairs during operational execution of a process. Next computation steps are defined (Def. 5), leading to the operational semantics (Def. 6). Let DataBases be the type consisting of sets of data items with at most one item for each key, i.e. DataBases = {diset ⊆ DataItems | for all dati1 , dati2 ∈ diset: key(dati1 ) = key(dati2 ) → dati1 = dati2 } Definition 1 (Operational Status). An operational status of a sequential process, denoted os, os0 , os1 , .., is a record with three fields, st, clock and db: – st : SV ars → ℘(DataItems), represents the local state, assigning to each variable a set of data items; – clock ∈ LocalT ime, the value of the local clock;
188
J. Hooman and J. van de Pol
– db ∈ DataBases, the local database, a set of data items (at most one for each key) representing local storage. Definition 2 (Variant). The variant of local state st with respect to variable x ∈ SV ars and value diset ⊆ DataItems, denoted diset if y = x st[x := diset], is defined as (st[x := diset])(y) = st(y) if y =x Similarly, the variant of a record r with fields f1 . . . fm is defined by v if fi = f r[f := v](fi ) = =f fi (r) if fi Produced data items are sent to an underlying network. Here this is represented by N , a set of data items, i.e. N ⊆ DataItems. Note that we do not use a multiset, although a particular item might be produced several times. The use of a set is justified by the fact that data items may always be delivered more than once and hence are never deleted from N . Definition 3 (Configuration). The state of affairs of a process S1 ... Sn during execution is represented by a configuration: (S1 , os1 ), ..., (Sn , osn ), N It denotes for each sequential process Si , the status osi and the remaining part Si that still has to be executed, and the current contents N of the network. For convenience, we introduce the empty statement E indicating that the process has terminated. An execution of S1 ... Sn is represented by sequence of configurations C0 −→ C1 −→ C2 −→ ... where C0 = (S1 ; E, os1 ), ..., (Sn ; E, osn ), ø with, for all i, db(osi ) = ø. The idea is that each step in the sequence C0 −→ C1 −→ C2 −→ ... represents the execution of an atomic action by some process i. We define the update of a database. Definition 4 (Update Database). The update of database db using a new database db1 , denoted UpdateDb(db, db1 ) is defined by dati ∈ UpdateDb(db, db1 ) iff – either dati ∈ db and for all dati1 ∈ db1 with key(dati1 ) = key(dati) we have ts(dati1 ) ≤ ts(dati), – or dati ∈ db1 and for all dati0 ∈ db with key(dati0 ) = key(dati) we have ts(dati0 ) < ts(dati). Computation steps are defined formally as follows. Definition 5 (Computation Step). We have a step of process i, i.e., (S1 , os1 ), .., (Si , osi ), .., (Sn , osn ), N −→ (S1 , os1 ), .., (Si , osi ), .., (Sn , osn ), N iff one of the following clauses holds: = E and Si = Si , N = N , st(osi ) = st(osi ), clock(osi ) = (Update) Si clock(osi ), and there exists a database db1 ⊆ N that is used to update
Equivalent Semantic Models for a Distributed Dataspace Architecture
189
db(osi ) such that an element of db1 is added if its key is not yet present and, otherwise, it replaces an element of db(osi ) with the same key if its local time-stamp is strictly greater. Formally, using Def. 4, there exists a database db1 ⊆ N such that db(osi ) = UpdateDb(db(osi ), db1 ). For simplicity, we did not change the local clock (it is not needed), but alternatively we might require clock(osi ) ≥ clock(osi ). Note that the network has not been changed, since data items might be used several times for an update (modeling the fact that an item might be delivered by the network several times). (Write) Si = Write(e) ; Si , st(osi ) = st(osi ), db(osi ) = db(osi ), clock(osi ) > clock(osi ), and N = N ∪ {(v, clock(osi ))} where v is the value of e in the current state. Note that, given a syntax of expressions, it would be easy to define the values of expressions and queries in the current status (see, e.g. [dRdBH+ 01]). (Read) Si = Read(x, q); Si , N = N , db(osi ) = db(osi ), clock(osi ) = clock(osi ), and there exists a set of data items diset ⊆ db(osi ) satisfying query q and such that st(osi ) = st(osi )[x := diset], i.e. assigning diset to x. For two configurations C1 and C2 , define C1 −→k C2 , for k ∈ IN, by C1 −→0 C1 and C1 −→k+1 C2 iff there exists a configuration C such that C1 −→k C and C −→ C2 . Define C1 −→∗ C2 iff there exists a k ∈ IN such that C1 −→k C2 . Typically, the operational semantics yields some abstraction of the execution sequence, depending on what is observable. Here we postulate that only the set of produced data items in the last configuration of an execution sequence is (externally) observable. Definition 6 (Operational Semantics). The operational semantics of a program S1 ... Sn , given an initial operational status os0 , is defined by O(S1 ... Sn )(os0 ) = {N ⊆ DataItems | (S1 ; E, os0 ), .., (Sn ; E, os0 ), ø −→∗ (E, os1 ), .., (E, osn ), N , with db(os0 ) = ø } Thus the operational semantics of a program yields a set of sets of produced data items, where each set of produced data items represents a possible execution of the program. Example 1. Let 0 be the query specifying that the value 0 should be read, similarly for 1. Observe that, for any os0 , O((Read(x, 0) ; Write(1)) (Read(x, 1) ; Write(0)))(os0 ) = ø.
5
Denotational Semantics
We define the denotational semantics of a program, given an initial status, i.e. the state of affairs when execution starts. To support our aim to reason with assumptions about the items produced by the environment, such assumptions are
190
J. Hooman and J. van de Pol
included in the status. The semantics yields a set of statuses, each representing a possible execution of the program. To achieve compositionality and to describe a process in isolation, it is quite common that information has to be added to the status to express relations with the environment explicitly. Here we add, e.g. the set of written data items and the set of items that are assumed to be produced by the environment. Moreover, we need a way to represent causality between the written items. As shown below, this is needed to assign a correct meaning to the program (Read(x, 0); Write(1)) (Read(x, 1); Write(0)). Here this is achieved by the use of conceptual logical clocks that are added to the status of each process. They are updated similarly to Lamport’s logical clocks [Lam78], which ensures that there exists a global, total order on the produced items. Let T ime be the domain of logical clock values, here we use the natural numbers. We add a logical clock value to the data items produced. Type ExtDataItems, with typical variables edi, edi0 , edi1 , . . ., consists of records with two fields: – di of type DataItems, and – tm of type T ime. Field tm represents the logical moment of publication. It can be used to construct a global partial order on the produced data items that reflects causality. Field selector di is extended to remove the logical clock values from a set of extended data items. For a set ediset ⊆ ExtDataItems define di(ediset) = {di(edi) | edi ∈ ediset}. A status, typically denoted by s, s0 , s1 , .., representing the current state of affairs of a program, is a record with six fields. In addition to the three fields of the operational status: – st : SV ars → ℘(DataItems), the local state (values of variables); – clock ∈ LocalT ime, the value of the local clock; – db ∈ DataBases, the local database (a set of data items, unique per key); we have three new fields: – time ∈ T ime, the logical clock, to represent causality; – ownw ⊆ ExtDataItems, the set of extended data items written by the program itself in the past; – envw ⊆ ExtDataItems, the set of extended data items written by the environment of the program; this is an assumption about all items produced (including present and future). Below we define a meaning function M for programs by induction on their structure. The possible behaviour of a program prog, i.e. a set of statuses, is defined by M(prog)(s0 ), where s0 is the initial status at the start of program execution. Note that this includes an assumption about all data items that have been or will be produced by the environment. The semantics will be such that if s ∈ M(prog)(s0 ) then
Equivalent Semantic Models for a Distributed Dataspace Architecture
191
– ownw(s) equals the union of ownw(s0 ) and the items written by prog. – envw(s) = envw(s0 ); the field envw is only used by prog to update its local storage. Although all items are available initially, constraints on logical clocks prevent the use of items “too early”. Next we define M(prog) by induction on the structure of prog. Write. In the semantics of the write statement the published item, extended with the current value of the local clock, is added to the ownw field. The local clock is increased to ensure that subsequent written items get a later time stamp. In a similar way, a logical clock value has been added. M(Write(e))(s0 ) = {s | clock(s) > clock(s0 ), time(s) > time(s0 ), ownw(s) = ownw(s0 ) ∪ {(#di := dati, tm := time(s)#)}, where dati = (#dat := v, ts := clock(s0 )#), with v the value of e in s0 , and s equals s0 for the other fields (st, db and envw) } Read. To define the meaning of a read statement, we first introduce an auxiliary Update function which may update the local database with data items written (by the process itself or by its environment), using UpdateDb of Def. 4. Update(s0 ) = {s | there exists an ediset ⊆ ownw(s0 ) ∪ envw(s0 ) and a database db1 ⊆ di(ediset), such that db(s) = UpdateDb(db(s0 ), db1 ), time(s) >= time(s0 ), for all edi ∈ ediset, we have tm(edi) < time(s), and s equals s0 for the other fields (st, clock, ownw and envw) } Then the read statement Read(x, q) first updates the local storage and next assigns to x a set of data items that satisfies the query q. M(Read(x, q))(s0 ) = {s | exists s1 ∈ Update(s0 ) and diset ⊆ db(s1 ) such that diset satisfies query q and st(s) = st(s1 )[x := diset], and s equals s1 for the other fields (clock, db, time, ownw and envw) } Note that we only represent terminating executions; blocking has not been modeled explicitly. Sequential Composition. Since we only model terminating executions, the meaning of the sequential composition S1 ; S2 is defined by applying the meaning of S2 to any status that results from executing S1 . In Sect. 7.1 we show how this can be extended to deal with non-terminating programs. M(S1 ; S2 )(s0 ) = {s | exists s1 with s1 ∈ M(S1 )(s0 )∧ s ∈ M(S2 )(s1 )} Parallel Composition. To define parallel composition, let init(s0 ) be the condition db(s0 ) = ø ∧ ownw(s0 ) = ø. Moreover, we use s + ediset to add a set
192
J. Hooman and J. van de Pol
ediset ⊆ ExtDataItems to the environment writes of s, i.e. envw(s + ediset) = envw(s) ∪ ediset and all other fields of s remain the same. In the semantics of P1 P2 , starting in initial status s0 , the main observation is that envw(s0 ) contains only the data items produced outside P1 P2 . Hence the semantic function for P1 is applied to s0 where we add the items written by P2 to the environment writes. Similarly for P2 . Then parallel composition is defined as follows: M(P1 P2 )(s0 ) = {s | init(s0 ) and there exist s1 and s2 with s1 ∈ M(P1 )(s0 + ownw(s2 )), s2 ∈ M(P2 )(s0 + ownw(s1 )), ownw(s) = ownw(s1 ) ∪ ownw(s2 ), envw(s) = envw(s0 )} Example 2. Consider again the program of Example 1: (Read(x, 0) ; Write(1)) (Read(x, 1) ; Write(0)) Without using logical clocks, the semantics of parallel composition would allow for this program a status where envw = ø and ownw contains 0 and 1 (each component produces the item required by the other one). This, however, does not correspond to the operational semantics which yields the empty set. In the current version of the semantics, we can indeed show that, for any s0 , M((Read(x, 0) ; Write(1)) (Read(x, 1) ; Write(0)))(s0 ) = ø.
6
Equivalence of Denotational and Operational Semantics
In this section we first define what it means that the operational and the denotational semantics of Sect. 4 and 5, resp., are equivalent. Next we give an outline of how we proved this equivalence formally. Note that equivalence is far from trivial, since there a number of prominent differences: – The operational semantics allows updates of the local database at any point in time, whereas in the denotational semantics we minimized the number of updates to keep verification simple, allowing it only once, immediately before reading items. – The parallel composition of the denotational semantics is defined by a few recursive equations and it is not clear whether this indeed corresponds to operational execution. – The denotational semantics has additional fields in a status and the read statement contains an additional check on logical clock values. – The underlying network is modeled in different ways. Equivalence is based on what is externally observable, i.e. two semantics functions are equivalent if they assign the same observable behaviour to any program. For the denotational semantics, we choose the same notion of observable behaviour as has been used in the operational semantics, namely the set of published data items. For a set D of denotational statuses, define the observations
Equivalent Semantic Models for a Distributed Dataspace Architecture
193
by Obs(D) = {di(ownw(s)) | s ∈ D}. Define for an n-tuple (s1 , . . . , sn ) of statuses Obs(s1 , . . . , sn ) = di(∪i,1≤i≤n ownw(si )), and for a set T of such tuples Obs(T ) = {Obs(s1 , . . . , sn ) | (s1 , . . . , sn ) ∈ T }. To relate the operational and the denotational semantics, we use a function Ext to extend an operational status to a status of the denotational semantics; Ext(os) copies the fields st, clock, and db of os and it sets the fields time to 0 and ownw and envw to ø. This leads to the main theorem. Theorem 1. O(S1 ... Sn )(os) = Obs(M(S1 (S2 (... Sn )...))(Ext(os))) We give an outline of the proof that has been checked completely using the interactive theorem prover PVS. Below we only give the main steps, for instance, ignoring details about initial conditions. The proof uses three intermediate versions of the semantics and corresponding lemma’s: – M which is the same as M except that also the write-statement is preceded by an update action. Lemma 1. Obs(M(S1 (S2 (... Sn )...))(Ext(os))) = Obs(M (S1 (S2 (... Sn )...))(Ext(os))) Note that M and M are defined for the parallel composition of two processes. But we derive a similar formulation for n processes: Lemma 2. Obs(M (S1 (S2 (... Sn )...))(Ext(os))) = Obs({(s1 , . . . , sn ) | init(Ext(os)), envw(Ext(os)) = ø and for all i, 1 ≤ i ≤ n, si ∈ M (Si )(Ext(os)[envw := ∪j=i ownw(sj )]) }) – OD which extends the operational semantics O to the status of the denotational semantics (adding time, ownw and envw). Moreover, the network N is removed. This is achieved by defining the atomic steps of a single sequential process as (S, s) −→ediset (S , s ), where ediset represents the set of items written in the step (a singleton if S starts with a write statement, the empty set otherwise). This also includes update steps, similar to the update inside a read statement of the denotational semantics, so including a condition of the values of logical clocks. Based on this step relation for one process, we have a step of n parallel processes, denoted (S1 , s1 ), ..., (Sn , sn ) −→ (S1 , s1 ), ..., (Sn , sn ), if there = i, Sj = Sj and exists an i with (Si , si ) −→ediset (Si , si ), and for all j sj = sj + ediset. This leads to the following semantics for n processes: OD(S1 ... Sn )(s0 ) = {(s1 , . . . sn ) | init(s0 )∧ (S1 ; E, s0 ), ..., (Sn ; E, s0 ) −→∗ (E, s1 ), ..., (E, sn ) } Lemma 3. O(S1 ... Sn )(os) = Obs(OD(S1 ... Sn )(Ext(os)) – OS which extends the operational semantics of a sequential process to the status of the denotational semantics, using the relation (S, s) −→ediset (S , s ) mentioned above. Lemma 4. OS(S) = M (S), for sequential programs S.
194
J. Hooman and J. van de Pol
Now, by the lemmas 1, 2, 3, and 4, theorem 1 reduces to the following lemma. Lemma 5. Obs(OD(S1 ... Sn )(Ext(os)) = Obs({(s1 , . . . , sn ) | init(Ext(os)), envw(Ext(os)) = ø and for all i, 1 ≤ i ≤ n, si ∈ OS(Si )(Ext(os)[envw := ∪j=i ownw(sj )]) }) Proof. We give the main ideas of the proof, showing that the sets are contained in each other. ⊆ Suppose (s1 , . . . sn ) ∈ OD(S1 ... Sn )(Ext(os)), that is, (S1 ; E, Ext(os)), ..., (Sn ; E, Ext(os)) −→∗ (E, s1 ), ..., (E, sn ). We have proved by induction on the number of steps in the execution sequence that this implies, for all i, 1 ≤ i ≤ n, (Si ; E, Ext(os)[envw := ∪j=i ownw(sj )]) −→ediset1 . . . −→edisetk (E, si ), that is, si ∈ OS(Si )(Ext(os)[envw := ∪j=i ownw(sj )]). ⊇ Assume, for all i, 1 ≤ i ≤ n that si ∈ OS(Si )(Ext(os)[envw := ∪j=i ownw(sj )]). Thus we have an operation execution (Si ; E, Ext(os)[envw := ∪j=i ownw(sj )]) −→ediset1 . . . −→edisetk (E, si ) for each of the sequential processes. We have to show, (S1 ; E, Ext(os)), ..., (Sn ; E, Ext(os)) −→∗ (E, s1 ), ..., (E, sn ), i.e., we have to show that these sequential executions can be merged into a global execution sequence for the parallel program. Basically, this is done by induction on the total number of steps in all sequential executions. The global execution sequence is constructed by selecting for each step the process with the lowest logical time after its next possible step. This construction is far from trivial, since the local, sequential executions start with all available environment writes, whereas in the global execution a process can only use what has been produced up to the current moment. However, the constraints on the logical clocks (that have been included in this extended operational semantics), ensure that only items produced before its current logical time are used. Formally, this is captured by the property that if (S, s+ediset) −→ediset1 (S , s ) (representing a step of a local process) and for all edi ∈ ediset, tm(edi) ≥ time(s ) then (S, s) −→ediset1 (S , s [envw := envw(s)]) (i.e. it can be used in the global sequence). Since we execute the process with the earliest logical time, we can also show that all items produced later cannot have a smaller logical time stamp.
7
Verification Framework
In this section we provide a framework that can be used to specify and verify processes, as shown in Sect. 8. First, in Sect. 7.1, the programming language is extended with a number of useful constructs. Section 7.2 contains the main specification and verification constructs.
Equivalent Semantic Models for a Distributed Dataspace Architecture
195
7.1 Language Extensions To deal with some more interesting examples, the simple programming language of Sect. 3 is extended with an assignment, if-then-else construct, and infinite loops. Accordingly, the denotational semantics of Sect. 5 is extended. Since infinite loops introduce non-terminating computations, we add one field to the status: – term ∈ {true, f alse}: indicates termination of the process; if it is false all subsequent statements are ignored. Henceforth, we assume that s0 is such that term(s0 ) = true, i.e. after s0 we can still execute subsequent statements. The definition of sequential composition has to be adapted, since it is possible that the first process does not terminate and thus prohibits execution of the second process. M(S1 ; S2 )(s0 ) = {s | s ∈ M(S1 )(s0 ) ∧ ¬term(s)}∪ {s | there exists an s1 with s1 ∈ M(S1 )(s0 )∧ term(s1 ) ∧ s ∈ M(S2 )(s1 )} The meaning of the skip statement can be defined easily. M(Skip)(s0 ) = {s0 } Similarly we can easily define x := e where x ∈ Svars and e an expression yielding a set of data items. We also add variables that range over data items and define assignments for such variables. Next we define an if-then-else statement, where b is a boolean expression. M(If b Then S1 Else S2 Fi)(s0 ) = {s | if b is true in st(s0 ) then s ∈ M(S1 )(s0 ) else s ∈ M(S2 )(s0 ) } Finally, we define an infinite loop by means of an infinite sequence of statuses s0 , s1 , s2 , . . ., where si is the result of executing the loop body i times, provided all these executions terminate. Otherwise the term-field of si is false. The written items are collected by taking the union of the produced items in each execution of the body. Note that only the own and environment writes are relevant. M(Do S Od)(s0 ) = {s | there exists a sequence s0 , s1 , s2 , . . . such that for all i, if term(si ) then si+1 ∈ M(S)(si ) else term(si+1 ) = f alse, and ownw(s) = ∪{i|term(si )} ownw(si+1 ) and envw(s) = envw(s0 ) } 7.2
Specification and Verification
To obtain a convenient specification and verification framework, we define a mixed formalism in which one can freely mix programs and specifications, based on earlier work [Hoo94]. Specifications are part of the program syntax; let p, p0 , p1 , . . . , q, q0 , q1 , . . . be assertions, that is, predicates over statuses. A specification is a “program” of the form Spec(p, q) with the following meaning.
196
J. Hooman and J. van de Pol
M(Spec(p, q))(s0 ) = {s | (p(s0 ) implies q(s)) and envw(s) = envw(s0 ) } Next we define a refinement relation ⇒ between programs (which now may include specifications). Definition 7 (Refinement). For any two programs P1 , P2 , we define P1 ⇒ P2 iff for all s0 , we have M(P1 )(s0 ) ⊆ M(P2 )(s0 ). Note that it is easy to prove that the refinement relation is reflexive and transitive. We have the usual consequence rule. Lemma 6 (Consequence). Spec(p, q).
If p → p0 and q0 → q then Spec(p0 , q0 ) ⇒
Based on the denotational semantics for Splice, we checked in PVS the soundness of a number of proof rules for programming constructs. For instance, for sequential composition we have a composition rule and a monotonicity rule which allows refinements in a sequential context. Lemma 7 (Sequential Composition). (Spec(p, r); Spec(r, q)) ⇒ Spec(p, q). Lemma 8 (Monotonicity of Sequential Composition). If P3 ⇒ P1 and P4 ⇒ P2 then (P3 ; P4 ) ⇒ (P1 ; P2 ). The reasoning about parallel composition in PVS mainly uses the semantics directly. We only have a monotonicity rule, which forms the basis of stepwise refinement of components. Lemma 9 (Monotonicity of Parallel Composition). If P3 ⇒ P1 and P4 ⇒ P2 then (P3 P4 ) ⇒ (P1 P2 ).
8
Verification Example
To illustrate our reasoning framework, we first show top-down development of a typical application consisting of a producer, a transformer, and a consumer in Sect. 8.1. Next, in Sect. 8.2, we consider transparent replication. The general question is whether we can replace a single process by two, or more, copies of the same process without having to change the environment. So, given the monotonicity rules mentioned above, we look for sufficient conditions for having P P ⇒ P . We investigate this for the concrete case study developed in Sect. 8.1. 8.1
Top-Down Development
Here we consider a concrete and simple case study, which is prototypical for the application area, with the following processes: – Producer: provides monotonically increasing data (here simply natural numbers) with name SensData.
Equivalent Semantic Models for a Distributed Dataspace Architecture
197
– Transformer: assuming it gets increasing SensData items, it provides monotonically increasing data with name Intern. – Consumer: assuming the environment provides increasing data with name Intern, it produces monotonically increasing Display items. To formalize this, we define the following types and functions: – DataN ame = {SensData, Intern, Display}, with typical variable dn. – DataV al = IN. – Data is a type of records with two fields: name of type DataN ame and val of type DataV al. Let dvar be a variable of type Data. – KeyData = DataN ame and key(dvar) = name(dvar). To formulate the specifications, first a few preliminary definitions are needed, where wset is a set of extended data items: – N ameOwnw(dn)(s) = ∀edi ∈ ownw(s) : name(edi) = dn – SensData(wset) = {edi | edi ∈ wset ∧ name(edi) = SensData} Similarly, we have Intern(wset) and Display(wset). – Increasing(wset) = ∀edi1 ∈ wset, edi2 ∈ wset : (val(edi1) < val(edi2) ↔ ts(edi1) < ts(edi2)) Let pre be an assertion expressing that db = ø, ownw = ø, and all variables ranging over sets of data items are initialized to the empty set. It is used as a precondition for the processes. The top-level specification of the overall system is defined as follows. postT opLevel(s) = envw(s) = ø → Increasing(Display(ownw(s))) T opLevel = Spec(pre, postT opLevel) We implement this top-level specification by the parallel composition of the three processes mentioned above: Producer Transformer Consumer We first specify the processes, without considering their implementation, and show that these specifications lead to the top-level specification. The producer writes an increasing sequence of sensor data. postP rod(s) = N ameOwnw(SensData)(s)∧ Increasing(ownw(s)) P rod = Spec(pre, postP rod) The consumer should produce an increasing sequence of Display data, assuming the environment provides increasing internal values. postCons = N ameOwnw(Display)(s)∧ Increasing(Intern(envw(s))) → Increasing(Display(ownw(s))) Cons = Spec(pre, postCons) To connect producer and consumer, we specify the transformer as follows.
198
J. Hooman and J. van de Pol
postT rans = N ameOwnw(Intern)(s)∧ Increasing(SensData(envw(s))) → Increasing(Intern(ownw(s))) T rans = Spec(pre, postT rans) We have proved in PVS that this leads to a correct refinement of the top-level specification. Theorem 2. (P rod (T rans Cons)) ⇒ T opLevel Next we consider the implementation of the three components. Let d be a variable ranging over Data, whereas dset and dold are variables ranging over sets of data items. P roducer = d := (#name := SensData, val := 0#) ; Do Write(d) ; d := (#name := SensData, val := val(d) + 1#) Od Note that the producer writes all natural numbers, which is not required by the specification. Also note that subscribers to data with name SensData will usually read only a (increasing) subsequence of these items. We have proved in PVS that this is indeed a correct refinement. Lemma 10. P roducer ⇒ P rod Similarly we provide a program for the transformer and show that it is a correct implementation. Let q(dn, old) be the query that specifies a set of data items with name dn and local time stamp different from those in variable old (which is initially empty). The set is allowed to be empty, to avoid blocking computations; otherwise it will be a singleton with a new, unread, item. The item that has been read is transformed using a function T r(dn, dset) which, for simplicity, just changes the name of the record of the item in dset into dn. Transformer = Do Read(dset, q(SensData, old)) ; If dset =ø Then Write(T r(Intern, dset)) ; old := dset Else Skip Fi Od Lemma 11. Transformer ⇒ T rans Similarly for the consumer. For simplicity, we used the same name transformation, which makes it possible to derive the correctness of transformer and consumer from a single proof that was parameterized by the name of the data sort. Consumer = Do Read(dset, q(Intern, old)) ; If dset =ø Then Write(T r(Display, dset)) ; old := dset Else Skip Fi Od Lemma 12. Consumer ⇒ Cons Finally observe that top-level theorem 2 and the lemmas 10, 11, and 12 for the components lead by the monotonicity property (lemma 9) to (P roducer (Transformer Consumer)) ⇒ T opLevel
Equivalent Semantic Models for a Distributed Dataspace Architecture
8.2
199
Transparent Replication
While investigating transparent replication, we observed that the current version of the transformer specification is not suitable. The next lemma expresses that we cannot replicate the transformer in a transparent way. Lemma 13. ¬((T rans T rans) ⇒ T rans) The lemma has been proved by a concrete counter example in PVS. The basic idea is that the two transformers each have increasing output, but - because they may write this output at different moment - the local time stamps in these sequences might be different and hence merging these output streams need not lead to an increasing sequence. The main problem is that the items produced get their local time stamp from the local clock when the item is written. This need not be related to the temporal validity of the data. Hence we propose an alternative specification for the transformer where the local time stamp is just copied from the incoming data item (as we will see later, this also requires a modified write statement). M aintainT s(s) = ∀edi ∈ ownw(s) : ∃edi1 ∈ envw(s) : ts(edi) = ts(edi1 )∧ name(edi1 ) = SensData∧ val(edi) = val(edi1 ) postT ransN ew = N ameOwnw(Intern) ∧ M aintainT s T ransN ew = Spec(pre, postT ransN ew) This indeed refines the transformer specification: Lemma 14. T ransN ew ⇒ T rans Moreover, this specification can indeed be replicated. Lemma 15 (Transformer Replication). (T ransN ew T ransN ew) ⇒ T ransN ew Hence we have (by theorem 2, lemma 14 and monotonicity): (P rod (T ransN ew Cons)) ⇒ T opLevel and by lemma 15 and monotonicity: (P rod ((T ransN ew T ransN ew) Cons)) ⇒ T opLevel It remains to implement T ransN ew. Important part of this specification is that it just copies the local time stamp (the ts-field). This, however, cannot be implemented with the current write statement, since it always use the current value of the local clock as its time stamp (the ts field is set to clock(s0 )). Therefore we introduce a new write statement Write(e, texp) which has an additional parameter, a time expression texp, which specifies the ts value, i.e. in the semantics of this extended write statement the ts-field in the data item gets the value of texp. Query Qtrans specifies a set of data items that is either empty or a singleton containing an item with name SensData. Expression MkIntern(dset) changes the name of the item in dset to Intern and copies it value. Now the write statement has a time expression T s(dset) which just yields the ts-field of the data item in dset.
200
J. Hooman and J. van de Pol
NewTransformer = Do Read(dset, Qtrans) ; If dset =ø Then Write(MkIntern(dset), T s(dset)) Else Skip Fi Od Lemma 16. NewTransformer ⇒ T ransN ew So, finally, we have (P roducer(NewTransformerConsumer)) ⇒ T opLevel and (P roducer ((NewTransformer NewTransformer) Consumer)) ⇒ T opLevel.
9
Concluding Remarks
We have defined a new denotational semantics for the main primitives of the industrial software architecture Splice. This architecture is data-oriented and based on local storages of data items. Communication between components is anonymous, based on the publish-subscribe paradigm. New is especially the modeling of time stamps, based on local clocks, and the update mechanism of local storages based on these time stamps. Moreover, the denotational semantics supports convenient compositional verification, based on assumptions about the data items produced by the environment of a component. Causality between data items is represented by logical time stamps. To simplify verification, we minimized the number of updates of the local storages and tried a short, but non-trivial, formulation of parallel composition. To increase the confidence in this denotational semantics, we also formulated a rather straightforward operational semantics. Using the interactive theorem prover PVS, we formally showed that the operational semantics is equivalent to the denotational one. The proof revealed a number of errors in earlier versions of the semantics, e.g. concerning the precise interpretation of logical time stamps representing causality. As a side-effect, the proof resulted in a number of useful properties. Classical ones, such as associativity of sequential and parallel composition, but also more Splice-oriented properties such as idempotence of updates. We also observed that the current definition of the notion of a lifecycle in Splice breaks idempotence of updates. Since it indicates a conceptual error in the definition, we have removed the lifecycle from the current formalization. A topic of current work concerns the equivalence classes induced by the semantics, i.e. which programs obtain the same semantics? The question is whether the denotational semantics is fully abstract with respect to the operational one, that is, does it only distinguish those programs that are observably different in some context? We have shown how the semantics can be used as a basis for a formal framework for specification and compositional verification. The study of transparent replication clarified the use of local time stamps and led to an improvement of the write primitive. In future work we intend to investigate the use of clock synchronization.
Equivalent Semantic Models for a Distributed Dataspace Architecture
201
References BdJ97.
M. Boasson and E. de Jong. Software architecture for large embedded systems. IEEE Workshop on Middleware for Distributed Real-Time Systems and Services, 1997. BHdJ00. R. Bloo, J. Hooman, and E. de Jong. Semantical aspects of an architecture for distributed embedded systems. In Proc. of the 2000 ACM Symposium on Applied Computing (SAC 2000), volume 1, pages 149– 155. ACM press, 2000. BKBdJ98. M.M. Bonsangue, J.N. Kok, M. Boasson, and E. de Jong. A software architecture for distributed control systems and its transition system semantics. In Proc. of the 1998 ACM Symposium on Applied Computing (SAC ’98), pages 159 – 168. ACM press, 1998. BKZ99. M. M. Bonsangue, J.N. Kok, and G. Zavattaro. Comparing coordination models based on shared distributed replicated data. In Proc. of the 1999 ACM Symposium on Applied Computing (SAC’99). ACM Press, 1999. Boa93. M. Boasson. Control systems software. IEEE Transactions on Automatic Control, 38(7):1094–1106, July 1993. dRdBH+ 01. W.P. de Roever, F. de Boer, U. Hannemann, J. Hooman, Y. Lakhnech, M. Poel, and J. Zwiers. Concurrency Verification, Introduction to Compositional and Noncompositional Methods. Cambridge Tracts in Theoretical Computer Science. Cambridge University Press, 2001. FHA99. E. Freeman, S. Hupfer, and K. Arnold. JavaSpaces: Principles, Patterns, and Practice. Addison-Wesley, Reading, MA, USA, 1999. Gel85. D. Gelernter. Genarative communication in Linda. Transactions on Programming Languages and Systems, 7(1):80–112, 1985. HH02. U. Hannemann and J. Hooman. Formal reasoning about real-time components on a data-oriented architecture. In Proc. of 6th World Multiconference on Systemics, Cybernetics and Informatics (SCI02), volume XI, pages 313–318, 2002. Hoo94. J. Hooman. Correctness of real time systems by construction. In Formal Techniques in Real-Time and Fault-Tolerant Systems, pages 19–40. LNCS 863, Springer-Verlag, 1994. HvdP02. J. Hooman and J. van de Pol. Formal verification of replication on a distributed data space architecture. In Proc. of the 2002 ACM Symposium on Applied Computing (SAC 2002), pages 351–358, 2002. Jon83. C.B. Jones. Tentative steps towards a development method for interfering programs. ACM Transactions on Programming Languages and Systems, 5(4):596–619, 1983. Lam78. L. Lamport. Time, clocks, and the ordering of events in a distributed system. Communications of the ACM, 21(7):558–565, 1978. MC81. J. Misra and K.M. Chandy. Proofs of networks of processes. IEEE Transactions on Software Engineering, 7(7):417–426, 1981. OSRSC01. S. Owre, N. Shankar, J.M. Rushby, and D.W.J. Stringer-Calvert. PVS System Guide. SRI International, Computer Science Laboratory, Menlo Park, CA, version 2.4 edition, December 2001. http://pvs.csl.sri.com.
Java Program Verification Challenges Bart Jacobs, Joseph Kiniry, and Martijn Warnier Dep. Comp. Sci., Univ. Nijmegen P.O. Box 9010, 6500 GL Nijmegen, The Netherlands {bart,kiniry,warnier}@cs.kun.nl http://www.cs.kun.nl/∼{bart,kiniry,warnier}
Abstract. This paper aims to raise the level of verification challenges by presenting a collection of sequential Java programs with correctness annotations formulated in JML. The emphasis lies more on the underlying semantical issues than on verification.
1
Introduction
In the study of (sequential) program verification one usually encounters the same examples over and over again (e.g., stacks, lists, alternating bit protocol, sorting functions, etc.), often going back to classic texts like [11,14,15]. These examples typically use an abstract programming language with only a few constructs, and the logic for expressing the program properties (or specifications) is some variation on first order logic. While these abstract formalisms were useful at the time for explaining the main ideas in this field, they are not very helpful today when it comes to actual program verification for modern programming and specification languages. The aim of this paper is to give an updated collection of program verification examples. They are formulated in Java [13], and use the specification language JML [19,21]. The examples presented below are based on our experience with the LOOP tool [3] over the past five years. Although the examples have actually been verified, the particular verification technology based on the LOOP tool does not play an important rˆ ole: we shall focus on the underlying semantical issues. The examples may in principle also be verified via another approach, like those of Jack [6], Jive [23], Krakatoa [10], and KeY [2]. We shall indicate when static checking with the ESC/Java [12] tool brings up interesting semantical issues in our examples1 . When ESC/Java checks an annotated program, it returns one of three results. The result “passed” indicates that ESC/Java believes the implementation of a method fulfills its specification; in that case we’ll say ESC/Java accepts the input. A result of “warning” indicates that an error potentially exists in the program but ESC/Java was unable to prove its existence definitively (e.g., a counter-example or instruction trace). Finally, the message “error” indicates a specific bug in the program, typically a potential run-time violation like a NullPointerException or an ArrayIndexOutOfBoundsException. 1
We use the final binary release (v. 1.2.4, 27 September 2001) of ESC/Java.
F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 202–219, 2003. c Springer-Verlag Berlin Heidelberg 2003
Java Program Verification Challenges
203
Of course the examples below do not cover all possible topics. For instance, floating point computations and multi-threading2 are not covered. In general, our work has focused on Java for smart cards and several examples stem from that area. Other examples are taken from work of colleagues [4], from earlier publications, or from test sets that we have been using internally. They cover a reasonable part of the semantics of Java and JML. Most of the examples can be translated easily in other programming languages. The examples do not involve semantical controversies, as discussed for instance in [5]. These examples are not meant to be canonical or proscriptive; they simply give a flavor of the level of completeness (and thus, complexity) necessary to cover modern programming language semantics. The verification examples are organised in the following categories. – – – – – –
Control flow and side-effects Overflow and bitwise operations Static initialization Inheritance Non-termination Specification issues
The organization of the paper roughly follows this categorization. Each example is accompanied by a short explanation of why the example is interesting and the challenges involved. But first we briefly describe the specification language JML.
2
JML
The Java Modeling Language, JML [19], is a behavioral interface specification language designed to specify Java modules. It can be used for classes as a whole, via class invariants and constraints, and for the individual methods of a class, via method specifications consisting of pre-, post- and frame-conditions (assignable clauses). In particular, it is also possible within a method specification to describe if a particular exception may occur and which post-condition results in that case. JML annotations are to be understood as predicates, which should hold for the associated Java code. These annotations are included in the Java source files as special comments indicated by //@, or enclosed between /*@ and */. They are recognised by special tools like the JML run-time checker [9], the LOOP compiler, and the Krakatoa verification condition generator. Class invariants and constraints are described as follows. /∗@ /∗@ 2 @ invariant <predicate> @ constraint
For a meta-theory on multithreaded Java programs see [1] in this volume.
204
B. Jacobs, J. Kiniry, and M. Warnier
exceptional) of methods, provided it holds before. Thus, invariants are implicitly added to postconditions of methods and constructors, and to preconditions of normal (non-constructor) methods. A constraint is a relation between two states, expressing what should hold about the pre-state and post-state of all methods. In this paper we use no constraints, and only one invariant (in Subsection 3.6). However, in practice they contain important information. Next we give an example JML method specification of some method m(). 2
4
6
/∗@ behavior @ requires <precondition>; @ assignable
Such method specifications may be understood as an extension of correctness triples {P }m{Q} used in Hoare logic, because they allow both normal and exceptional termination. Moreover, the postconditions in JML are relations, because the pre-state, indicated by \old(--), may occur. We shall see many method specifications below. JML is intended to be usable by Java programmers. Its syntax is therefore very much like Java. However, it has a few additional keywords, such as ==> (for implication), \old (for evaluation in the pre-state), \result (for the return value of a method, if any), and \forall and \exists (for quantification). This paper will not pay much attention to the semantics of JML (interested readers should see [21] for such). Hopefully, most of the JML assertions are self-explanatory. However, there are three points that we would like to mention. – In principle, expressions within assertions (such as an array access a[i]) may throw exceptions. The JML approach, see [21], is to turn such exceptions into arbitrary values. Of course, one should try to avoid such exceptions by including appropriate requirements. For instance, for the expression a[i] one should add a != null && i >= 0 && i < a.length, in case this is not already clear from the context. This is what we shall always do. – JML uses the subtype semantics for inheritance, see [22]. This means that overriding methods in subclasses should still satisfy the specifications of the overridden ancestors in superclasses. This is a non-trivial restriction, but one which is essential in reasoning about methods in an object-oriented setting. However, it does not hold for all our examples (see for instance Subsection 3.8). In that case we simply write no specification at all for the relevant methods. – JML method specifications form proof obligations. But also, once proved, they can be used in correctness proofs of other methods. In that case one first has to establish the precondition and invariant of the method that is called, and subsequently one can use the postcondition in the remainder of
Java Program Verification Challenges
205
the verification (which will rely heavily on the called method’s assignable clause). An alternative approach is to reason not with the specification, but with the implementation of the method that is called3 . Basically, this means that the body of the called method gets substituted at the appropriate place. However, this may lead to duplication of verification work, and makes proofs more vulnerable to implementation changes. But if no specification is available (see the previous point), one may be forced to reason with the implementation. In the examples below we shall see illustrations of method calls which are used both by specification and by implementation. For readers unfamiliar with JML, this paper may hopefully serve as an introduction via examples. More advanced use of JML in specifying API-components may be found for instance in [24] 4 .
3
Verification Challenges
This section describes our Java+JML examples in several subsections. Our explanations focus on the (semantic) issues involved, and not so much on the actual code snippets. They should be relatively self-explanatory. 3.1
Aliasing and Field Access
Our first example, seen in Figure 1, might seem trivial to some readers. The return expression of the method Alias.m() references the value of the field i of the object c value via an aliased reference to itself in the field a. We present this example because it represents (in our view) the bare minimum necessary to model a language like Java. ESC/Java has no problem verifying this program. Either the implementation or specification of the constructor of class C can be used to verify method Alias.m(). 3.2
Side-Effects in Expressions
One of the most common abstractions in program verification is to omit sideeffects from expressions in the programming language. This is a serious restriction. Figure 2 contains a nice and simple example from [4] where such side-effects play a crucial rˆ ole, in combination with the logical operators. Recall that in Java there are two disjunctions (| and ||) and two conjunctions (& and &&). The double versions (|| and &&) are the so-called conditional operators: their second argument is only evaluated if the first one is false (for ||) or true (for &&). 3 4
This only works if one actually knows the run-time type of the object on which the method is called. See also on the web at www.cs.kun.nl/∼ erikpoll/publications/jc211 specs.html for a specification of the Java API for smart cards.
206
2
B. Jacobs, J. Kiniry, and M. Warnier
class C { C a; int i ;
4
6
8
10
}
/∗@ normal behavior @ requires true; @ assignable a , i ; @ ensures a == null && i == 1; @∗/ C() { a = null; i = 1; }
12
class Alias { /∗@ normal behavior @ requires true; 16 @ assignable \nothing; @ ensures \ result == 4; 18 @∗/ int m() { 20 C c = new C(); c.a = c; 22 c. i = 2; return c.i + c.a. i ; 24 } } 14
Fig. 1. Aliasing via Field References.
In case the field b in Figure 2 is true, method m() yields f() ∨ ¬f() = false and ¬f() ∧ f() = true, going against standard logical rules. The verification of the specification for method m() may use either the implementation or the specification of f(). 3.3
Breaking out of a Loop
While and for loops are typically used for going through an enumeration, for instance to find or modify an entry meeting a specific condition. Upon hitting this entry, the loop may be aborted via a break statement. This presents a challenge for the underlying control flow semantics. Figure 3 presents a simple example of a for loop that goes through an array of integers in order to change the sign of the first negative entry. The two lines of java code are annotated with the loop invariant, with JML-keyword maintaining stating what holds while going through the loop, and the loop variant, with JML-keyword decreasing. The loop variant is a mapping to the natural numbers which decreases with every loop cycle. It is used in verifications to show that the repetition terminates.
Java Program Verification Challenges
207
boolean b = true; boolean result1, result2; 2
/∗@ normal behavior @ requires true; @ assignable b; 6 @ ensures b == !\old(b) && \result == b; @∗/ 8 boolean f() { b = !b; return b; } 4
/∗@ normal behavior @ requires true; 12 @ assignable b, result1 , result2 ; @ ensures (\old(b) ==> !result1 && result2) && 14 @ (!\ old(b) ==> result1 && result2); @∗/ 16 void m() { result1 = f () || ! f (); result2 = ! f() && f (); } 10
Fig. 2. Side-effects and Conditional Logical Operators.
The result of ESC/Java on this program is not very interesting because of its limited handling of loops: they are executed (symbolically) only once by default. In general, this may indicate a basic problem with the invariant, but the coverage is far from complete5 . 3.4
Catching Exceptions
Typical of Java is its systematic use of exceptions, via its statements for throwing and catching. They require a suitable control flow semantics. Special care is needed for the ‘finally’ part of a try-catch-finally construction. Figure 4 contains a simple example (adapted from [17]) that combines many aspects. The subtle point is that the assignment m+=10 in the finally block will still be executed, despite the earlier return statements, but has no effect on the value that is returned. The reason is that this value is bound earlier. 3.5
Bitwise Operations
Our next example in Figure 5 is not of the sort one finds in textbooks on program verification. But it is a good example of the ugly code that verification tools have to deal with in practice, specifically in Java Card applets6 . It involves a “command” byte cmd which is split in two parts: the first three, and last five bits. Depending on these parts, a mode field is given an appropriate value. This 5
6
As an aside: ESC/Java has difficulty with this example due to limitations in its parser as quantified expressions cannot be used in ternary operations. If we rewrite the specification of negatefirst as a conjuction of disjoint implications, ESC/Java accepts the program. ESC/Java does not handle a bitwise operator like signed right shift (>>) correctly.
208
B. Jacobs, J. Kiniry, and M. Warnier
int [] ia ; 2
/∗@ normal behavior @ requires ia != null ; @ assignable ia [∗]; 6 @ ensures \ forall int i; 0 <= i && i < ia.length ==> @ (\old(ia [ i]) < 0 && 8 @ (// i is the first position with negative value @ \ forall int j; 0 <= j && j < i ==> \old(ia[j]) >= 0)) 10 @ ? ( ia [ i] == −\old(ia[i ])) @ : ( ia [ i] == \old(ia[ i ])); 12 @∗/ void negatefirst () { 14 /∗@ maintaining i >= 0 && i <= ia.length && @ (\ forall int j; 0 <= j && j < i ==> 16 @ (ia [ j] >= 0 && ia[j] == \old(ia[j ]))); @ decreasing ia .length − i ; 18 @∗/ for(int i = 0; i < ia .length ; i++) { 20 if ( ia [ i ] < 0) { ia [ i] = −ia[ i ]; break; } } } 4
Fig. 3. Breaking out of a Repetition. int m; 2
/∗@ normal behavior @ requires true; @ assignable m; 6 @ ensures \ result == ((d == 0) ? \old(m) : \old(m) / d) @ && m == \old(m) + 10; 8 @∗/ int returnfinally (int d) { 10 try { return m / d; } catch(Exception e) { return m / (d+1); } 12 finally { m += 10; } } 4
Fig. 4. Return within try-catch-finally.
happens in a nested switch. The specification is helpful because it tells in decimal notation what is going on. 3.6
Class Invariants and Callbacks
Class invariants are extremely useful in specification, because they often make explicit what programmers have in the back of their mind while writing their code. A typical example is: “integer i is always non-zero” (so that one can safely divide by i).
Java Program Verification Challenges
2
209
static final byte ACTION ONE = 1, ACTION TWO = 2, ACTION THREE = 3, ACTION FOUR = 4; private /∗@ spec public @∗/ byte mode;
4
/∗@ behavior @ requires true; @ assignable mode; 8 @ ensures (cmd == 0 && mode == ACTION ONE) || @ (cmd == 16 && mode == ACTION TWO) || 10 @ (cmd == 4 && mode == ACTION THREE) || @ (cmd == 20 && mode == ACTION FOUR); 12 @ signals (Exception) @ ((cmd & 0x07) != 0 || (cmd != 0 && cmd != 16)) 14 @ && @ ((cmd & 0x07) != 4 || (cmd != 4 && cmd != 20)); 16 @∗/ void selectmode(byte cmd) throws Exception { 18 byte cmd1 = (byte)(cmd & 0x07), cmd2 = (byte)(cmd >> 3); switch (cmd1) { 20 case 0x00: switch (cmd2) { case 0x00: mode = ACTION ONE; break; 22 case 0x02: mode = ACTION TWO; break; default: throw new Exception(); } 24 break; case 0x04: switch (cmd2) { 26 case 0x00: mode = ACTION THREE; break; case 0x02: mode = ACTION FOUR; break; 28 default: throw new Exception(); } break; 30 default: throw new Exception(); } // ... more code 32 } 6
Fig. 5. Typical Mode Selection Based on Command Byte.
The standard semantics for class invariants is: when an invariant holds in the pre-state of a (non-constructor) method, it must also hold in the post-state. Note that this post-state can result from either normal or exceptional termination. An invariant may thus be temporarily broken within a method body, as long as it is re-established at the end. A simple example is method decrementk in Figure 6. Things become more complicated when inside such a method body the class invariant is broken and another method is called. The current object this is then left in an inconsistent state. This is especially problematic if control returns at some later stage to the current object. This re-entrance or callback phenomenon is discussed for instance in [25, Sections 5.4 and 5.5]. The commonly adopted solution to this problem is to require that the invariant of this is established before a method call. Hence the proof obligation in a method call a.m() involves the invariants of both the caller (this) and the callee (a).
210
2
B. Jacobs, J. Kiniry, and M. Warnier
class A { private /∗@ spec public @∗/ int k, m; B b;
4
/∗@ invariant k + m == 0; @∗/ 6
/∗@ normal behavior @ requires true; @ assignable k , m; @ ensures k == \old(k) − 1 && m == \old(m) + 1; @∗/ void decrementk () { k−−; m++; }
8
10
12
14
16
18
20
}
/∗@ normal behavior @ requires b != null ; @ assignable k , m; @ ensures true; @∗/ void incrementk () { k++; b.go(this); m−−; }
class B { /∗@ normal behavior 24 @ requires arg != null ; @ assignable arg.k , arg.m; 26 @ ensures arg.k == \old(arg.k) − 1 && @ arg.m == \old(arg.m) + 1; 28 @∗/ void go(A arg) { arg.decrementk(); } 30 } 22
Fig. 6. Callback with Broken Invariant.
This semantics is incorporated in the translation performed by the LOOP tool. Therefore we can not prove the specification for the method incrementk in Figure 6. However, a proof using the implementations of method go and decrementk is possible, if we make the additional assumptions that the run-time type of the field b is actually B, and that the method incrementk is executed on an object of class A. These restrictions are needed because if, for instance, field b has a subclass of B as run-time type, a different implementation will have to be used if the method go is overridden in the subclass. ESC/Java warns about the potential for invariant violation during the callback. Another issue related to class invariant is whether or not they should be maintained by private methods. JML does require this, but allows a special category of so-called ‘helper’ methods which need not maintain invariants. We don’t discuss this matter further.
Java Program Verification Challenges
2
211
class C { static boolean result1, result2 , result3 , result4 ;
4
6
8
10
12
14
16
}
/∗@ normal behavior @ requires !\ is initialized (C) && @ !\ is initialized (C1) && @ !\ is initialized (C2); @ assignable \ static fields of (C), @ \ static fields of (C1), @ \ static fields of (C2); @ ensures result1 && !result2 && result3 && result4; @∗/ static void m(){ result1 = C1.b1; result2 = C2.b2; result3 = C1.d1; result4 = C2.d2; }
18
class C1 { static boolean b1 = C2.d2; static boolean d1 = true; 22 } 20
24
class C2 { static boolean d2 = true; static boolean b2 = C1.d1; 28 } 26
Fig. 7. Static Initialization.
3.7
Static Initialization
Figure 7 shows an example of static initialization in Java (due to Jan Bergstra). In Java a class is initialized at its first active use (see [13]). This means that class initialization in Java is lazy, so that the result of initialization depends on the order in which classes are initialized. The rather sick example in Figure 7 shows what happens when two classes, which are not yet initialized, have static fields referring to each other. In the specification we use a new keyword \static_fields_of in the assignable clause. It is syntactic sugar for all static fields of the class. The first assignment in the body of method m() triggers the initialization of class C1, which in turn triggers the initialization of class C2. The result of the whole initialization is, for instance, that static field C2.b2 gets value false assigned to it. This can be seen when one realizes that the boolean static fields from class C1 initially get the default value false. Subsequently, class C2 becomes initialized and its fields also get the default value false. Now the assignments
212
B. Jacobs, J. Kiniry, and M. Warnier
in class C2 are carried out: d2 is set to true and b2 is set to false. Note that d1 is still false at this stage. Finally the assignments to fields in class C1 take place, both resulting in value true. One can see that the order of initializations is important. When the first two assignments in the method body of m() are switched, class C2 will be initialized before class C1 resulting in all fields getting value true. ESC/Java cannot handle this example as it cannot reason about static initialization. It provides no warnings for potential run-time errors in static initializers or in initializers for static fields. 3.8
Overriding and Dynamic Method Invocation
The example in Figure 8 is usually attributed to Kim Bruce. It addresses an issue which is often thought of as confusing in programming with languages which support inheritance. The overriding of the method equal makes it hard to tell which implementation is called: the one in the subclass or in the superclass. When a method is overridden, the run-time type of an object decides which implementation is called. This phenomena is also called late-binding. In the example three different objects are created, and the question is which equal method will be used. Notice that the equal methods in Figure 8 have no specifications. According to the behavioural subtyping semantics used in JML, the equal method in the subclass ColorPoint should also satisfy the specification of the equal method in superclass Point. This makes it impossible to prove a precise specification of the equal method in class ColorPoint. Therefore we proved the specification of method m() by using the implementations of the equal methods. 3.9
Inheritance
The program in Figure 9 is from [16] and was originally suggested by Joachim van den Berg. On first inspection it looks like the method test() will loop forever. The method test() calls method m() from class C, which calls method m() from class Inheritance, since ‘this’ has runtime-type Inheritance. Due to the subtype semantics used in JML for inheritance, we cannot write specifications for both of the m() methods with which we can reason. Therefore we can only prove the specification of method test() by using the method implementations. 3.10
Non-termination
The example in Figure 10 (due to Cees-Bart Breunesse) shows a program that does not terminate. The specification asserts that the program does not terminate normally or with an exception. The JML keyword diverges followed by the predicate true indicates that the program fails to terminate. The reader can easily see that this
Java Program Verification Challenges class Point { 2
int equal(Point x) { return 1; }
4
}
6
class ColorPoint extends Point {
8
}
int equal(Point x) { return 2; }
10
int r1,r2,r3,r4,r5,r6,r7,r8,r9; 12
/∗@ normal behavior @ requires true; @ assignable r1 , r2 , r3 , r4 , r5 , r6 , r7 , r8 , r9; @ ensures r1 == 1 && r2 == 1 && r3 == 2 && @ r4 == 2 && r5 == 2 && r6 == 2 && @ r7 == 1 && r8 == 2 && r9 == 2; @∗/ void m() { Point p1 = new Point(); Point p2 = new ColorPoint(); ColorPoint cp = new ColorPoint(); r1 = p1.equal(p1);r2 = p1.equal(p2);r3 = p2.equal(p1); r4 = p2.equal(p2);r5 = cp.equal(p1);r6 = cp.equal(p2); r7 = p1.equal(cp);r8 = p2.equal(cp);r9 = cp.equal(cp); }
14
16
18
20
22
24
26
Fig. 8. Overriding and dynamic method invocation.
2
class C { void m() throws Exception { m(); } }
4
6
class Inheritance extends C { void m() throws Exception { throw new Exception(); }
8
10
12
14
}
/∗@ exceptional behavior @ requires true; @ assignable \nothing; @ signals (Exception) true; @∗/ void test () throws Exception { super.m(); }
Fig. 9. Overriding and Dynamic Types.
213
214
B. Jacobs, J. Kiniry, and M. Warnier
class Diverges{ 2
4
6
8
10
12
14
}
/∗@behavior @ requires true; @ assignable \nothing; @ ensures false ; @ signals (Exception e) false ; @ diverges true; @∗/ public void m(){ for (byte b = Byte.MIN VALUE;b <= Byte.MAX VALUE;b++) ; }
Fig. 10. A Program that does not Terminate.
program does not terminate. Since Byte.MAX VALUE + 1 = Byte.MIN VALUE the guard in the for loop will never fail. Note that in order to verify this program both overflowing and non-termination have to be modeled appropriately. ESC/Java does not handle non-termination. 3.11
Specification
The final example exemplifies two commonplace complications in reasoning about “real world” Java. Representations of integral types (e.g., int, short, byte, etc.) in Java are finite. A review of annotated programs that use integral types indicates that specifications are often written with infinite numeric models in mind [7]. Programmers seem to think about the issues of overflow (and underflow, in the case of floating point numbers) in program code, but not in specifications. Additionally, it is often the case that specifications use functional method invocations. Methods which have no side-effects in the program are called called “pure” methods in JML and “queries” in the Eiffel and UML communities7 . The example in Figure 11 highlights both complications, as it uses method invocations in a specification and integral values that can potentially overflow. The method isqrt(), which computes an integer square root of its input, is inspired by a specification (see Figure 12) of a similar function included with early JML releases [20]. Note that the iabs() method in Figure 11 is used in the specification of isqrt() to stipulate that both the negative and the positive square root are an acceptable return value, as all we care about is its magnitude. Actually, our implementation of isqrt() only computes the positive integer square root. 7
There is still debate in the community about the meaning of “pure”. Many Java methods, for example, which claim to have no side-effects, and thus should be pure, actually do modify the state due to caching, lazy evaluation, etc.
Java Program Verification Challenges
215
/∗@ normal behavior @ requires true; @ assignable \nothing; 4 @ ensures \ result == ((x >= 0 || @ x == Integer.MIN VALUE) ? x : −x); 6 @∗/ /∗@ pure @∗/ int iabs(int x) { 8 if (x < 0) return −x; else return x; } 2
/∗@ normal behavior @ requires x >= 0 && x <= 2147390966; 12 @ assignable \nothing; @ ensures iabs(\ result ) < 46340 && 14 @ \ result ∗ \ result <= x && @ x < (iabs(\ result ) + 1) ∗ ( iabs(\ result ) + 1); 16 @∗/ int isqrt (int x) { 18 int count = 0, sum = 1; /∗@ maintaining 0 <= count && 20 @ count < 46340 && @ count ∗ count <= x && 22 @ sum == (count + 1) ∗ (count + 1); @ decreasing x − count; 24 @∗/ while (sum <= x) { count++; sum += 2 ∗ count + 1; } 26 return count; } 10
Fig. 11. Dependent Specifications and Integral Types.
2
4
6
/∗@ normal behavior @ requires x >= 0; @ ensures Math.abs(\result) <= x && @ \ result ∗ \ result <= x && @ x < (Math.abs(\result) + 1) ∗ @ (Math.abs(\result) + 1); @∗/ Fig. 12. JML Specification of Integer Square Root from [20].
The original specification included in early JML releases is provided in Figure 12. This specification uses the Math.abs() method instead of our iabs() method. We use our own equivalent implementation of square root out of convenience, not necessity. The definition of JML states that expressions in the requires and ensures clauses are to be interpreted in the semantics of Java. Consequently, a valid im-
216
B. Jacobs, J. Kiniry, and M. Warnier
plementation of the specification in Figure 12 is permitted to return Integer.MIN VALUE when x is 0 (as already noted in [20]). This surprising situation exists because Java’s integral are bounded and because the definition of unary negation in the Java Language Specification is somewhat unexpected. Because integral types are bounded, expressions in the postcondition, specifically the multiplication operations, can overflow. Additionally, the two implementations of integer absolute value both return a value of Integer.MIN VALUE when passed Integer.MIN VALUE. While this is the documented behavior of java.lang.Math.abs(int), it is often overlooked by programmers because they presume that a function as mathematically uncomplicated as absolute value will produce unsurprising, mathematically correct results for all input. The absolute value of Integer.MIN VALUE is equal to itself because Math.abs() is implemented with Java’s unary negation operator “-”. This operator, defined in Section 15.15.4 of the Java Language Specification, silently overflows when applied to maximal negative integer or long [13] 8 . The precondition of isqrt() in Figure 11 is explained in Figure 13. We wish to ensure that no operation, either in the implementation of isqrt() or in its specification, overflows. The critical situation that causes an overflow is when we attempt to take an integer square root of a very large number. In particular, if we attempt to evaluate the postcondition of isqrt() for values of x larger than 2,147,390,966 an overflow takes place. The small but critical interval between the precondition’s bound 2,147,390,966 and Integer.MAX VALUE = 2,147,483,647 is indicated by the dark interval on the right of Figure 13: to check the postcondition, the prospective root (the arrow labeled 1) must be determined, one is added to its value (arrow 2), and the result is squared (arrow 3). This final result will thus overflow. Indeed, (46, 340 + 1)2 > 2, 147, 483, 647. 2.
3.
Integer.MAX_VALUE
0 1.
Fig. 13. The Positive Integers.
The erroneous nature of a specification involving potential overflows should become clear when one verifies the method using an appropriate bit-level representation of integral types [18]. Unfortunately, such errors are not at all apparent, even when performing extensive unit testing, because the boundary conditions for arithmetic expressions, like the third term of the postcondition of isqrt() in Figure 11, are rarely automatically derivable, and full state-space coverage is simply too computationally expensive. 8
Interestingly, the documentation for java.lang.Math.abs() did not reflect this fact until the 1.1 release of Java.
Java Program Verification Challenges
217
Specifications involving integral types, and thus potential overflows, are frequently seen in application domains that involve numeric computation both complex (e.g., scientific computation, computer graphics, embedded devices, etc.) and relatively simple (e.g., currency and banking). The former category are obviously challenging due to the complexity of the related data-structures, algorithms, and their specifications, and the latter are problematic because it is there that implementation violations have egregious (financial) consequences. This specification raises the question: What is the appropriate model for arithmetic specifications9 ? Another challenge highlighted by this example might be called “formalism completeness”. Many semantic formalisms of modern programming languages like Java do not attempt to specify the semantics of complicated features like method invocation. Even fewer attempt to incorporate such semantics in method specifications, as used here. For example, ESC/Java is unable to deal with this example because of such interdependencies. This is a significant limitation as many, if not most, method specifications rely upon pure or JML helper methods. This is also a weakness of the LOOP tool, as dealing with (pure) method calls in specifications is tedious in verifications10 . In general, the semantics of method invocation in specifications is still an unclear issue at this time.
4
Conclusions
The starting point of this paper is the observation that the classical examples in (sequential) program verification are no longer very relevant in today’s context. They need to be updated in two dimensions: language complexity and size. This paper focuses on complexity, by presenting a new series of challenges, written in Java, with correctness assertions expressed in JML. The examples incorporate some of the ugly details that one encounters in real-world programs, and that any reasonable semantics should be able to handle. The fact that these example programs are small does not mean that we think size is unimportant. On the contrary, once a reasonably broad semantic spectrum is covered, the next challenge is to scale up one’s program verification techniques to larger programs. With our tools we are currently verifying programs with hundreds of lines of code.
Acknowledgments We want to thank Patrice Chalin for insightful comments on an earlier version of this paper. 9 10
We do not have answers for these questions, though investigations are underway [8]. The verification of the method isqrt() from Figure 11 uses the implementation of the absolute value method iabs().
218
B. Jacobs, J. Kiniry, and M. Warnier
References ´ 1. Erika Abrah´ am Mumm, Frank S. de Boer, Willem-Paul de Roever, and Martin Steffen. A Tool-supported Proof System for Multithreaded Java. In FMCO 2002 proceedings, Lect. Notes Comp. Sci. Springer, Berlin, 2003. 2. W. Ahrendt, Th. Baar, B. Beckert, R. Bubel, M. Giese, R. H¨ ahnle, W. Menzel, W. Mostowski, A. Roth, S. Schlager, and P.H. Schmitt. The KeY tool. Technical Report 2003-5, Chalmers and G¨ oteborg University, 2003. http://i12www.ira.uka.de/˜key. 3. J. van den Berg and B. Jacobs. The LOOP compiler for Java and JML. In T. Margaria and W. Yi, editors, Tools and Algorithms for the Construction and Analysis of Systems, number 2031 in Lect. Notes Comp. Sci., pages 299–312. Springer, Berlin, 2001. 4. J. Bergstra and M. Loots. Empirical semantics for object-oriented programs. Artificial Intelligence Preprint Series nr. 007, Dep. Philosophy, Utrecht Univ. http://preprints.phil.uu.nl/aips/, 1999. 5. E. B¨ orger and W. Schulte. Initialization problems in Java. Software—Concepts and Tools, 20(4), 1999. 6. Lilian Burdy, Jean-Louis Lanet, and Antoine Requet. JACK (Java Applet Correctness Kit), 2002. www.gemplus.com/smart/r_d/trends/jack.html. 7. P. Chalin. Back to basics: Language support and semantics of basic infinite integer types in JML and Larch. Technical Report 2002.003.1, Computer Science Department, Concordia University, 2002. www.cs.concordia.ca/˜faculty/chalin/. 8. P. Chalin. Improving JML: For a safer and more effective language. Technical Report 2003-001.1, Computer Science Department, Concordia University, March 2003. 9. Y. Cheon and G.T. Leavens. A runtime assertion checker for the Java Modeling Language (JML). In H.R. Arabnia and Y. Mun, editors, International Conference on Software Engineering Research and Practice (SERP ’02), pages 322–328. CSREA Press, Las Vegas, 2002. 10. E. Contejean, J. Duprat, J.-C. Filliˆ atre, C. March´e, C. Paulin-Mohring, and X. Urbain. The Krakatoa tool for JML/Java program certification. Available via the Krakatoa home page at www.lri.fr/˜marche/krakatoa/, October 2002. 11. E.W. Dijkstra. A Discipline of Programming. Prentice Hall, 1976. 12. C. Flanagan, K.R.M. Leino, M. Lillibridge, G. Nelson, J.B. Saxe, and R. Stata. Extended static checking for Java. In Proceedings of the 2002 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), volume 37(5) of SIGPLAN Notices, pages 234–245. ACM, 2002. 13. J. Gosling, B. Joy, G. Steele, and G. Bracha. The Java Language Specification Second Edition. The Java Series. Addison-Wesley, 2000. http://java.sun.com/docs/books/jls/second_edition/html/ j.title.doc.html. 14. D. Gries. The Science of Programming. Springer, 1981. 15. C.A.R. Hoare. An axiomatic basis for computer programming. Commun. ACM, 12:576–580, 583, 1969. 16. M. Huisman and B. Jacobs. Inheritance in higher order logic: Modeling and reasoning. In M. Aagaard and J. Harrison, editors, Theorem Proving in Higher Order Logics, number 1869 in Lect. Notes Comp. Sci., pages 301–319. Springer, Berlin, 2000.
Java Program Verification Challenges
219
17. B. Jacobs. A formalisation of Java’s exception mechanism. In D. Sands, editor, Programming Languages and Systems (ESOP), number 2028 in Lect. Notes Comp. Sci., pages 284–301. Springer, Berlin, 2001. 18. B. Jacobs. Java’s integral types in PVS. Manuscript, http://www.cs.kun.nl/˜bart/PAPERS/, 2003. 19. G.T. Leavens, A.L. Baker, and C. Ruby. JML: A notation for detailed design. In H. Kilov and B. Rumpe, editors, Behavioral Specifications of Business and Systems, pages 175–188. Kluwer, 1999. 20. G.T. Leavens, A.L. Baker, and C. Ruby. Preliminary design of JML: A behavioral interface specification language for Java. Techn. Rep. 98-06, Dep. of Comp. Sci., Iowa State Univ. http://www.cs.iastate.edu/˜leavens/JML.html, 1999. 21. G.T. Leavens, E. Poll, C. Clifton, Y. Cheon, and C. Ruby. JML reference manual (draft). www.jmlspecs.org, 2002. 22. B. Liskov and J.M. Wing. A behavioral notion of subtyping. ACM Trans. on Prog. Lang. & Syst., 7(16(6)):1811–1841, 1994. 23. J. Meyer and A. Poetzsch-Heffter. An architecture for interactive program provers. In S. Graf and M. Schwartzbach, editors, Tools and Algorithms for the Construction and Analysis of Systems, number 1785 in Lect. Notes Comp. Sci., pages 63–77. Springer, Berlin, 2000. 24. E. Poll, J. van den Berg, and B. Jacobs. Specification of the JavaCard API in JML. In J. Domingo-Ferrer, D. Chan, and A. Watson, editors, Smart Card Research and Advanced Application, pages 135–154. Kluwer Acad. Publ., 2000. 25. C. Szyperski. Component Software. Addison-Wesley, 1998.
ToolBus: The Next Generation Hayco de Jong1 and Paul Klint1,2 1
2
Centrum voor Wiskunde en Informatica Kruislaan 413, P.O.Box 94079 1090 GB Amsterdam, The Netherlands Informatics Institute, University of Amsterdam, The Netherlands {Hayco.de.Jong,Paul.Klint}@cwi.nl
Abstract. The ToolBus is a software coordination architecture for the integration of components written in different languages running on different computers. It has been used since 1994 in a variety of projects, most notably in the complete renovation of the Asf+Sdf Meta-Environment. In this paper we summarize the experience that has been gained in these projects and sketch preliminary ideas how the ToolBus can be further improved. Topics to be discussed include the structuring of message exchanges, crash recovery in distributed applications, and call-by-value versus call-by-reference.
1
Generic Language Technology
Our primary interest is generic language technology that aims at the rapid construction of tools for a wide variety of programming and application languages. Its central notion is a language definition of some programming or application language. The common methodology is that a language is identified in a given domain, that relevant aspects of that language are formally defined and that desired tools are generated on the basis of this language definition. This generative approach is illustrated in Fig. 1. Using a definition for some language L as starting point, a generator can produce a range of tools for editing, manipulating, checking or executing L programs. Language aspects have to be defined, analyzed, and used to generate appropriate tooling such as compilers, interpreters, type checkers, syntax-directed editors, debuggers, partial evaluators, test case generators, documentation generators, and more. Language definitions are used, on a daily basis, in application areas as disparate as Cobol renovation, Java refactoring, smart card verification and in application generation for domains including finance, industrial automation and software engineering. In the case of Cobol renovation, the language in question is Cobol and those aspects that are relevant for renovation have to be formalized. In the case of application generation, the language in question is probably new and has to be designed from scratch. 1.1
One Realization: The Asf+Sdf Meta-environment
The Asf+Sdf Meta-Environment [1,2] is an incarnation of the approach just described and covers both the interactive development of language definitions and the generation of tools based on these language definitions. F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 220–241, 2003. c Springer-Verlag Berlin Heidelberg 2003
ToolBus: The Next Generation
221
Fig. 1. From language definition to generated programming environment.
In this paper we are primarily interested in the software engineering aspects of building such a system. Starting point is the Asf+Sdf Meta-Environment as we had completed it in the beginning of the 1990’s. This was a monolithic 200 KLOC Lisp program that was hard to maintain. It had all the traits of a legacy system and was the primary motivation to enter the area of system and software renovation. 1.2 Towards a Component Based Architecture We give a brief time line of the efforts to transform the old, monolithic, implementation of the Meta-Environment into a well-structured, component-based, implementation. In 1992, first, unsuccessful, experiments were carried out to decompose the system into separate parts [3]. The idea was to separate the user-interface and the text editor from the rest of the system. The user-interface was completely re-implemented as a separate component and as text editor we re-used Emacs. In hindsight, we were unaware of the fact that we made the transition from a completely sequential system to a system with several concurrent components. Unavoidably, we encountered hard to explain deadlocks and race conditions. In 1993, a next step was to write a formal specification of the desired system behavior [4] using PSF, a specification language based on process algebra and algebraic specifications [5]. Simulation of this specification unveiled other, not yet observed, deadlocks. Although this was clearly an improvement over the existing situation, this specification approach also had its limitations and drawbacks: – The specification lacked generality. It would, for instance, have been a major change to add the description of a new component. – The effort to write the PSF specification was significant and there was no way to derive an actual implementation from it. In 1994, the first version of the ToolBus was completed [6,7]. The key idea was to organize a system along the lines of a software bus and to make this bus programmable by way of a scripting language (Tscript) that was based on ACP (Algebra of Communicating Processes, [8]). Another idea was to use a uniform data format (called
222
H. de Jong and P. Klint
ToolBus terms) to exchange data between ToolBus and tools. At the implementation level, Tscripts were executed by an interpreter and communication between tools and ToolBus took place using TCP/IP sockets. In this way, multi-language, distributed, applications could be built with significantly less effort than using plain C and sockets. Based on various experiments [9,10,11,12], in 1995 a new version of the ToolBus was designed and implemented: the Discrete Time ToolBus [13,14,15]. Its main innovations were primitives for expressing timing considerations (delay, timeout) and for operating on a limited set of built-in data-types (booleans, integers, reals, lists). The Discrete Time ToolBus has been used for the restructuring of the Asf+Sdf MetaEnvironment [16]. A first version was released in 2001 [2]. In the meantime, the exchange format has also evolved from the ToolBus terms mentioned above to ATerms [17]: a term format that supports maximal subterm sharing and a very concise, sharing preserving, binary exchange format. ATerms decrease memory usage thanks to sharing and they permit a very fast equality test since structural equality can be replaced by pointer equality thanks to the maximal subterm sharing. Another line of development is the ToolBus Integrated Debugging Environment (Tide) described in [18]. Today, beginning 2003, it turns out that the original software engineering goals that triggered the development of the ToolBus have been achieved and that the MetaEnvironment can now be even further stretched than anticipated [19]. Therefore, it is time for some reflection. What have we learned from this major renovation project and what are the implications for the ToolBus design and implementation? 1.3
Plan of this Paper
In Sect. 2 we discuss component coordination, representation and computation and introduce the ToolBus: our component coordination architecture. Following, in Sect. 3, we demonstrate some of the ToolBus-features by means of an example. In Sect. 4 we show how we used the ToolBus in the Asf+Sdf Meta Environment to migrate from a monolithic to a distributed architecture. Then, in Sect. 5 we elaborate on the various issues that we would like to tackle in a next generation of the ToolBus. We conclude the paper with an overview of the current status of our current implementation of this next generation ToolBus (Sect. 6) and some concluding remarks (Sect. 7).
2 The TOOLBUS Architecture In [20] it was advocated that the overall architecture of a software system can be improved by separating coordination from computation. In addition to this, we also distinguish representation and use the following definitions: – Coordination: the way in which program and system parts interact (using procedure calls, remote method invocation, middleware, and others). – Representation: language and machine neutral data exchanged between components. – Computation: program code that carries out a specialized task.
ToolBus: The Next Generation
223
Fig. 2. Separating coordination from computation.
Fig. 3. The ToolBus architecture.
The assumption is now that a rigorous separation of coordination, representation and computation leads to flexible and reusable systems. This subdivision is sketched in Fig. 2. Our ToolBus approach follows this paradigm and is illustrated in Fig. 3. The goal of the ToolBus is to integrate tools written in different languages running on different machines. This is achieved by means of a programmable software bus. The ToolBus coordinates the cooperation of a number of tools. This cooperation is described by a Tscript that runs inside the ToolBus. The result is a set of concurrent processes inside the ToolBus that can communicate with each other and with the tools. Tools can be written in any language and can run on different machines. They exchange data by way of ATerms. A typical cooperation scenario is illustrated in Fig. 4. A user-interface (UI) and a database (DB) are combined in an application. Pushing a button in the user-interface leads to a database action and the result is displayed in the user-interface. In a traditional approach, the database action is directly connected to the user-interface button by means of a call-back function. This implies that the user-interface needs some knowledge about
224
H. de Jong and P. Klint
Fig. 4. A typical cooperation scenario.
the database tool and vice versa. In the ToolBus approach the two components are completely decoupled: pushing the button only leads to an event that is handled by some process in the ToolBus. This process routes the event to the database tool (likely via some intermediary process) and gets the answer back via the inverse route. This implies that the configuration knowledge is now completely localized in the Tscript and that UI and DB do not even know about each others existence. The primitives that can be used in Tscripts are listed in Table 1.
3 An Example: The Address Book Service To make the scenario from Fig. 4 more concrete, we describe the construction of an address book holding (name, address) pairs. Typical uses include creating a new address, finding an address based on the name, etc. First we consider some aspects of the User Interface. An instance of the UI connects to the ToolBus and during the subsequent session, the user can: create a new entry in the address book database; delete an existing entry from the database; search for an entry in the database; update an existing entry in the database. Each of these use cases can be described as a ToolBus process which, together with a process that explains how these use cases interact, form the ToolBus script describing our Address Book Service. 3.1 TOOLBUS Processes for the Address Book Service The ADDRESSBOOK process tells the ToolBus that an instance of our address-book tool is to be executed, followed by a loop which invokes one of the processes CREATE,
ToolBus: The Next Generation
225
Table 1. Overview of ToolBus primitives. Primitive delta + . * create snd-msg rec-msg snd-note rec-note no-note subscribe unsubscribe snd-eval rec-value snd-do rec-event snd-ack-event if ... then ... fi if ... then ... else ... fi || let ... in ... endlet := delay abs-delay timeout abs-timeout rec-connect rec-disconnect execute snd-terminate shutdown attach-monitor detach-monitor
Description inaction (“deadlock”) choice between two alternatives (P1 or P2 ) sequential composition (P1 followed by P2 ) iteration (zero or more times P1 followed by P2 ) process creation send a message (binary, synchronous) receive a message (binary, synchronous) send a note (broadcast, asynchronous) receive a note (asynchronous) no notes available for process subscribe to notes unsubscribe from notes send evaluation request to tool receive a value from a tool send request to tool (no return value) receive event from tool acknowledge a previous event from a tool guarded command conditional expressions communication-free merge (parallel composition) local variables assignment relative time delay absolute time delay relative timeout absolute timeout receive a connection request from a tool receive a disconnection request from a tool execute a tool terminate the execution of a tool terminate ToolBus attach a monitoring tool to a process detach a monitoring tool from a process
DELETE, SEARCH or UPDATE in each iteration. This construction, using the + operator ensures that at this level, the sub-processes can be regarded atomically. This means that for example no DELETE will happen during an UPDATE. process ADDRESSBOOK is let AB : address-book in execute(address-book, AB?) . ( CREATE(AB) + DELETE(AB) + SEARCH(AB) + UPDATE(AB) ) * delta endlet
226
H. de Jong and P. Klint
The operating system level details of starting the tool are defined in a separate section (one for each tool if multiple tools are involved): tool address-book is { command = "java-adapter -class AddressBookService" }
In this case, the ToolBus is told that our tool is written in Java, and that the main class to be started is called AddressBookService. The CREATE process can be described as a ToolBus process as follows: process CREATE(AB : address-book) is let AID : int in rec-msg(create-address) . snd-eval(AB, create-entry) . rec-value(AB, new-entry(AID?)) . snd-msg(address-created(AID)) endlet
The request to create a new address book entry is received and delegated to the tool, so it can update its state. In this case, our tool yields a unique id for reference to the new entry, which is returned as the result of the creation message. Note that communication between processes involves the matching of the arguments of snd-msg and rec-msg. The same holds for the communication between a process and a tool using snd-eval and rec-value. In all these cases, a result variable of the form V? gets a value assigned as the result of a successful match. The DELETE process differs only from the CREATE process in that it does not need a return value: ... rec-msg(delete-address(AID?) . snd-do(AB, delete-entry(AID)) . snd-msg(address-deleted(AID)) ...
The SEARCH process in our example implements but a single query: finding an address book entry by name. It shows how different results from a tool-evaluation request can be processed in much the same way that different messages are handled. Upon receiving a find-by-name message from another process, this request is delegated to the tool. Depending on whether or not the entry exists in the database, the tool replies with a found or a not-found message, respectively. This result is then propagated to the process that sent the initial find-by-name message.
ToolBus: The Next Generation
227
process SEARCH(AB : address-book) is let Aid : int, Name : str in rec-msg(find-by-name(Name?)) . snd-eval(AB, find-by-name(Name)) . ( rec-value(AB, found(Aid?)) . snd-msg(found(Aid)) + rec-value(AB, not-found) . snd-msg(not-found) ) endlet
The UPDATE process is more interesting. It shows that each update of an address entry is guarded. A process wanting to update an entry first has to announce this fact by sending an update-entry message, before it can do one or more updates to the entry. It then finishes the update by sending an update-entry-done message. Because matching snd-msg and rec-msg messages are connected synchronously, it is not possible for one update transaction to interfere with another. After a sender and receiver of the update-entry message are connected, all other processes that want to send a update-entry message have to wait until the receiving process is ready to receive an update-entry message again. This message pair thus acts as a very primitive locking scheme. More elaborate schemes are very well possible, but are not discussed in this paper. Summarizing, the UPDATE process shows that outside the implementation of the address book service, we can enforce the order in which certain parts of the service are invoked, as well as mutual exclusion of some of its sections. process UPDATE(AB : address-book) is let AID : int, Name : str, Address : str in rec-msg(update-entry(AID?)) . ( rec-msg(set-name(Name?)) . snd-do(AB, set-name(AID, Name)) + rec-msg(set-address(Address?)) . snd-do(AB, set-address(AID, Address)) ) * rec-msg(update-entry-done(AID)) endlet
3.2 TOOLBUS Process for the User Interface Because users can connect at any time to the ToolBus to start a session with the Address Book Service, the ToolBus itself does not execute instances of the UI (as it did with the
228
H. de Jong and P. Klint
address book tool). Instead UITool instances can connect, make zero or more requests to the service, and disconnect at their convenience. A ui tool is declared to exist, but no operating system level details are provided. The following definition of the UI process shows how UI requests for the creation of a new entry and a name-change can be realized: tool ui is { /* the ToolBus does not execute ui-instances */ } process UI is let UITool : ui, AID : int, Name : str in rec-connect(UITool?) . ( rec-event(UITool, create-address) . snd-msg(create-address) . rec-msg(address-created) . snd-ack-event(UITool, create-address) + rec-event(UITool, update-name(AID?, Name?)) . snd-msg(update-entry(AID)) . snd-msg(set-name(Name)) . snd-msg(update-entry-done(AID)) . snd-ack-event(UITool, update-name(AID, Name)) + ... /* more UI requests */ ) * rec-disconnect(UITool) endlet
4 Application to the ASF+SDF Meta-environment As explained in Sect. 1.2, the ToolBus has been used to restructure the Asf+Sdf MetaEnvironment. It consists of a cooperation of 27 tools ranging from a user-interface, graph browser, various editors, compiler and interpreter, to a parser generator and a repository for parse trees. A simplified view is shown in Fig. 5. Our insight can be further increased by considering some statistics. Table 2 shows the relative sizes of the various implementation languages used in the Meta-Environment. In the column language the various languages are listed. In column KLOC the size (in Kilo Lines Of Code) is given for each language. The result is 107 KLOC for the whole system of which 4.6% are Tscripts. If we consider the fact that Asf+Sdf specifications are compiled to C code, another view is possible as well: 12 KLOC of Asf+Sdf generates 170 KLOC of C code. Taking this generated code into account, the total size of the whole system amounts to 277 KLOC of which 1.8% are Tscripts. This is compatible with the expectation that Tscripts are relatively small and form high-level “glue” to connect much larger components.
ToolBus: The Next Generation
229
Fig. 5. Architecture of the Asf+Sdf Meta-Environment. Table 2. Facts concerning implementation languages.
†
Language KLOC† Generated KLOC Total KLOC ASF+SDF 12 170 (C) C 80†† Java, Tcl/Tk 5 Makefiles, etc 5 Tscript 5 Total LOC: 107 170 277 Tscript 4.6% 1.8% Kilo Lines of Code excluding third party code such as emacs, dot, and the like. †† This includes 10 KLOC (C code) for the ToolBus implementation itself.
Part of the generated C code is currently done by ApiGen [21]. This is an API generator which takes an Sdf grammar as input and generates a C library which gives type-safe access to the underlying ATerm representation of the parse trees over this grammar. Another conclusion from these facts is that low-level information for building the software (makefiles and configuration scripts) are of the same size as the high level Tscripts. This points into the direction that the level of these build scripts should be raised. This conclusion will, however, not be further explored in this paper. Another view is given in Table 3 where the frequency of occurrence of Tscript primitives is shown. Clearly, sequential composition (.) is the dominant operator and sending/receiving (snd-msg, rec-msg) messages is the dominant communication mechanism, followed by communication with tools (snd-do, snd-eval). It may be surprising that parallel composition (||) is used so infrequently. However, one should be aware that at the top level all ToolBus processes run concurrently and that || is only used for explicit concurrency inside a process. The level of concurrency is therefore approximately 100 (104 process definitions and 3 explicit || operators). Empirical evidence shows that: – The ToolBus-based version of the Asf+Sdf Meta-Environment is more flexible as illustrated by the fact that clones of the Meta-Environment start to appear for other languages than Asf+Sdf. Examples are Action Semantics [22] and Elan [23].
230
H. de Jong and P. Klint Table 3. Facts concerning Tscript primitives. Primitive Number of occurrences process definitions 104 tool definitions 27 . (sequential composition) 4343 + (choice) 341 * (iteration) 243 || (parallel composition) 3 snd-msg 555 rec-msg 541 snd-note 100 rec-note 24 snd-do/snd-eval 220 rec-event 56 create 58
– Various components of the Asf+Sdf Meta-Environment are being reused in other projects [10,24].
5
Issues in a Next-Generation TOOLBUS
The ToolBus has been used in various applications of which the Meta-Environment is by far the largest. Some of the questions posed by our users and ourselves are: – I find it difficult to see which messages are requests and which are replies; can you provide support for this? See Sect. 5.1. – If a tool crashes, what is the best way to describe the recovery in the Tscript? See Sect. 5.2. – I have huge data values that are exchanged between tools and the ToolBus becomes a data bottleneck; can you improve this? See Sect. 5.3. – The ToolBus and tools are running as separate tasks of the operating systems. Would it not be more efficient to run ToolBus and tools in a single task? See Sect. 6. 5.1
Undisciplined Message Patterns
The classical pattern of a remote procedure call is shown in Fig. 6: a caller performs a call to a callee. During the call the caller suspends execution and the callee executes until it has computed a reply. At that point in time, the caller continues its execution. Compare this simple situation with general message communication as shown in Fig. 7: the caller continues execution after sending a message msg1 to Callee1 and may even send a message msg2 to Callee2. At a certain point in time Callee2 may send message msg3 back to Caller. In this case, the three parties involved continue their execution while messages are being exchanged and there is no obvious pairing of calls and replies.
ToolBus: The Next Generation
231
Fig. 6. Communication pattern for remote procedure call.
Fig. 7. Communication pattern for general messages.
In the ToolBus case, a snd-msg and a rec-msg can interact with each other if their arguments match. A typical sequence is: Process A: snd-msg(calculate(E)) . ... other actions ... rec-msg(value(E, V?))
Process B: rec-msg(calculate(E?)) . ... actions that compute value V ... snd-msg(value(E, V))
What we see here is that a form of call/reply regime is encoded in the messages: process B returns the value V that it has computed as snd-msg(value(E, V)). The E is completely redundant but serves as identification for process A to which message this is an answer. The call/reply regime is thus implicitly encoded in messages. This makes error handling harder (which reply did not come?) and makes the Tscripts harder to understand. This is particularly so, since unstructured combinations of snd-msg/rec-msg and sequential composition, choice, iteration and parallel composition are allowed.
232
H. de Jong and P. Klint
The only solution for the above problems is to limit the occurrences of snd-msg or rec-msg in such a way that a form of very general call/reply regime is enforced. Our approach is to syntactically enforce that snd-msg/rec-msg or rec-msg/snd-msg may only occur in (possibly nested) pairs and that in between arbitrary operations are allowed. In fact, the matching snd-msg or rec-msg may be an arbitrary expression provided that all its alternatives begin with a matching snd-msg or rec-msg. We replace thus snd-msg(req(E)) . arbitrary process expression . rec-msg(ans(A?)) by the syntactic construct snd-msg(req(E)) { arbitrary process expression } rec-msg(ans(A?)) and also allow more general cases like: snd-msg (req(E)) { arbitrary process expression} ( rec-msg(ans(A?)) + rec-msg(error(M?) ) It is an interesting property of Process Algebra that every process expression can be normalized to a so-called action prefix form: a list of choices where each choice starts with an atomic action. An action prefix form has the following structure: a1 .P1 + a2 .P2 +...+ an .Pn . Using this property we can formulate the most general constraint that we impose on occurrences of snd-msg and rec-msg. Consider P1 { Q } P2 and let P1 ’ and P2 ’ be the action prefix forms of, respectively, P1 and P2 . Our requirement is now that each choice in P1 ’ starts we a snd-msg and each choice in P2 ’ with a rec-msg, or vice versa. Note that this constraint can be checked statically. 5.2
Exception Handling
Exception handling is notorious for its complexity and impact on the structure of program code. The mainstream exception handling approach as used in, for instance, Java associates one or more exception handlers with a specific method call. If the call completes successfully, the handlers are ignored. If the call raises an exception, it is checked whether this exception can be handled locally by one of the given handlers. If not, the exception is propagated to the caller of the current code. This model does, however, not work well in a setting where multiple processes are active and the occurrence of an exception may require recovery in several processes. Local Exception Handling. We start with the simpler case of local error handling and introduce the disrupt operator (>>) proposed in LOTOS [25]. A process algebra variant of this operator is described in [26]. It has the form P >> E, where P describes the normal processing and E the exceptional processing. It adds the exception E as alternative to each atomic action in P. If the action prefix form of P is a1 .P1 + a2 .P2 +...+ an .Pn , then P >> E ≡ (a1 + E).(P1 >> E) +.. + (an + E).(Pn >> E).
ToolBus: The Next Generation
233
Global Exception Handling. Global exception handling in distributed systems is a very well-studied subject from the perspective of crash recovery and transaction management in distributed databases. An overview of rollback-recovery protocols in message-passing systems is, for instance, given in [27]. In the context of system reliability, the notion of a recovery block has been introduced by Randell [28]. Its purpose was to provide several alternative algorithms for doing the same computation. Upon completion of one algorithm, an acceptance test is made. If the test succeeds, the program proceeds normally, but if it fails a rollback is made to the system state before the algorithm was started and one of the alternative algorithms is tried. In [29] this idea is applied to backtracking in string processing languages. It turns out that the preservation of the system state can be done efficiently by only saving updates to the state after the last recovery point. Recovery blocks also form the basis for Coordinated Atomic Actions described in [30]. Recovery blocks are intended for the error recovery in a single process. They can be generalized to conversations between more than one process: several processes can enter a conversation at different times but they can only leave it simultaneously, when all participating processes satisfy their acceptance test. In case one participant fails to pass its test, each participant is rolled back to the state when it entered the conversation. We are currently studying this model since it can be fit easily in the ToolBus framework and seems to solve our problem of global exception handling. It is helpful that a backtrack operator similar to the one described in [29] has also been described for Process Algebra [31]. What remains to be studied is how the recovery of tools has to be organized. Most likely, we will add a limited undo request to the tool interface to recover from the last few operations carried out by a tool. 5.3
Call-by-Value Versus Call-by-Reference
Background. The concepts of call-by-reference and call-by-value are well-known in programming languages. They describe how an actual parameter value is transmitted from a procedure call to the body of the called procedure. In the case of call-by-reference, a pointer to the parameter is transmitted to the body. Call-by-reference is efficient (only a pointer has to be transmitted) and the parameter value can be changed during execution of the procedure body (via the pointer). In the case of call-by-value, a physical copy of the parameter is transmitted to the procedure body. Call-by value is less efficient for large values and does not allow the called procedure to make changes to the parameter value in the calling procedure. These considerations also apply to value transmissions in a distributed setting, with the added complication that values can be accessed or modified by more than one party. Call by reference (Fig. 8) is efficient for infrequent access or update. It is the prevalent mechanism in, for instance, CORBA [32]. However, uncontrolled modifications by different parties can lead to disaster. Call-by-value (Fig. 9) is inefficient for large values and any sharing between calls is lost. To us, this is of particular interest, because we need to preserve sharing in huge parse trees. In the case of Java RMI [33], value transmission is achieved via serialization and works only for communication with other Java components. Using IIOP [34] communication with non-Java components is possible.
234
H. de Jong and P. Klint
Fig. 8. Call-by-reference in a distributed application.
Fig. 9. Call-by-value in a (Java-based) distributed application.
Current ToolBus approach. Currently, the ToolBus provides a transport mechanism based on call-by-value as shown in Fig. 10(a). It is transparent since the transmitted values are ATerms (see Sect. 1.2) that can be exchanged with components written in any language. Since pure values are exchanged, there is no need for distributed garbage collection. Note that the call-by-reference model can easily be mimicked in the ToolBus. For instance, one tool can maintain a shared database and can communicate with other tools using record keys and field names so that only the values of record fields have to be exchanged (as opposed to complete records or even the complete database). In this way the access control to the shared database can be spelled out in detail and concurrency conflicts can be avoided. This solves one of the major disadvantages of the pure call-byreference model in a distributed environment. The downside is, however, that the ToolBus becomes a data bottleneck when huge values really have to be transmitted between tools. Currently, two workarounds are used. A first workaround is to get temporary relief by sending compressed values rather than the values themselves. A second workaround is to store the large value in the filesystem and to send a file name rather than the file itself. It does scale, but it also creates an additional inter-tool dependency and assumes that both tools have access to the same shared file system. We will now first discuss how related frameworks handle call-by-reference and then we come back to implications for the ToolBus design. In particular, we will discuss channel-based transmission as already shown in Fig. 10(b). 5.4
Related Frameworks: Java RMI, RMI-IIOP and Java IDL
Given our needs and desires for a next generation ToolBus it is interesting to see what other solutions are applied in similar projects. In this section, we briefly look at three related mechanisms:
ToolBus: The Next Generation
235
Fig. 10. Value-based (a) versus channel-based (b) transmission in the ToolBus.
– Java Remote Method Invocation (RMI) which connects distributed objects written in Java; – Java RMI over Internet Inter-ORB Protocol (IIOP) which is like RMI, but uses IIOP as the underlying protocol; – Java IDL which connects Java implementations of CORBA interfaces. Java RMI. Java Remote Method Invocation is similar to the ToolBus architecture in the sense that it connects different tools, possibly running on different machines. It differs from the ToolBus setting because it is strictly Java based: only components written in Java can communicate via RMI. For components to work together in RMI, first a remote interface is established. This is a Java interface that has a “real” implementation in the tool (or server) and a “stub” implementation on the client sides (Fig. 11). The interface is written by the programmer as opposed to the generated interfaces in a ToolBus setting where they are derived from the communication patterns found in the ToolBus script. The stubs in the RMI setting are then generated from this Java interface using rmic: the RMI compiler. Stubs act as a client-side proxy, delegating the method call via the RMI system to the server object. In RMI, any object that implements a remote interface is called a remote object. In RMI, arguments to or return values from remote methods can be primitive data (e.g. int), remote objects, or serializable objects. In Java, an object is said to be serializable if it implements the java.util.Serializable interface. Both primitive data and serializable objects are passed by value using Java’s object serialization. Remote objects are essentially passed by reference. This means that changes to them are actually performed on the server, and updates become available to all clients. Only the behavior that was defined in the remote interface is available to the clients. RMI programmers should be aware of the fact that any parameters, return values and exceptions that are not remote objects are passed by value. This makes it hard to understand when looking at a system of RMI objects exactly which method calls will result in a local (i.e. client side) state change, and which will have global (server side) effect.
236
H. de Jong and P. Klint
Fig. 11. Client-server model in RMI framework.
Consider, again, our address book example. If the AddressBookService is implemented as a remote object in RMI, then client-side invocations of the setAddress method will cause a global update. If, on the other hand, the AddressBookEntries are made serializable and instances of this class are returned as the result of a query to the AddressBookService, then updates on these instances will have a local state change only. Finally, before two RMI components can connect, the server side needs to register itself with an rmiregistry, after which the client needs to explicitly obtain a reference to the (remote) server object. Java RMI over IIOP. By making RMI programs conform to some restrictions, they can be made available over the Internet Inter-ORB Protocol (IIOP). This means that functionality offered by the RMI program can be made available to CORBA clients written in any (CORBA supported) language. The restrictions are mostly namespace oriented: programmers need to take special care not to use certain names that might collide with CORBA generated names, but some reservations should also be made regarding sharing preservation of object references. References to objects that are equal according to the == operator in one component, need not necessarily be equal in a remote component. Instead the equals method should be used to discern equality. RMI over IIOP is best used when combining several Java tools for which the programmer would like to use RMI, and some tools written in another CORBA-supported language need to use (some of) the services provided by the Java tools. The component’s interface is established by writing a Java interface, just as in plain RMI. Java IDL. Apart from Java RMI, which is optimized for connecting components that are all written in Java, there is also a connection from Java to CORBA using the Java Interface Definition Language (IDL). This alternative to Java RMI is for Java programmers who want to program in the Java programming language, based on interfaces defined in the CORBA Interface Definition Language. Using this bridge, it becomes possible to let Java components communicate with CORBA objects written in any language that has Interface Definition Language (IDL) mappings.
ToolBus: The Next Generation ToolBus Architecture Component coordination Interface Tscript GC yes parameters / by-value return values language any with TB adapter component yes coordination
RMI Client Server Java Interface yes local: by-value remote: by-ref only Java no
RMI-IIOP Client Client Java Interface no local: by-value remote: by-ref CORBA objects if interface in Java no
237
Java IDL Client Server IDL no depends on signature any with IDL binding no
Fig. 12. Related architectures: a feature overview.
Instead of writing a Java interface as is done in RMI, in Java IDL the definition is written in IDL: a special purpose interface language used as the base for CORBA implementations. This IDL definition is then used to generate the necessary stubs (client side proxies to delegate method invocations to the server) and skeletons, holder and helper classes (server side classes that hide low-level CORBA details). Feature summary. Fig. 12 shows some of the similarities and differences in ToolBus, RMI, RMI-IIOP and Java IDL. – RMI, RMI-IIOP and Java IDL make an explicit distinction between client and server sides of a set of cooperating components. In the ToolBus setting all components are considered equal (and none are more equal than others). – In RMI and RMI-IIOP, the programmer writes a Java interface which describes the component’s incoming and outgoing method signature, from which stubs and skeletons are generated. In Java IDL a CORBA interface is written. In the ToolBus setting, these signatures are generated from the ToolBus script which describes much more of the component’s behavior in terms of method call interaction, rather than just method signatures. – The ToolBus takes care of garbage collection of the ATerms that are used to represent data as it is sent from one component to another. RMI allows programmers access to Java’s Distributed Garbage Collection API. In RMI-IIOP and Java IDL however, this is not possible, because the underlying CORBA architecture is used, which does not support (distributed) GC, but places this burden entirely on the developer. – In the ToolBus all data is sent by-value. RMI and RMI-IIOP use both pass-by-value and pass-by-reference, depending on whether the relevant data is serializable (it is a primitive type, or it implements Serializable) or is a remote object. In Java IDL the components abide by IDL prescribed interfaces. Determination of whether a parameter is to be passed by-value or by-reference is made by examination of the parameter’s formal type (i.e. in the IDL signature of the method it is being passed to). If it is a CORBA value type, it is passed by-value. If it is an ordinary CORBA interface type (the “normal” case for all CORBA objects), it is passed by-reference.
238
H. de Jong and P. Klint
– The ToolBus allows components in any language for which a ToolBus adapter exists. Programming languages such as C and Java are supported, but adapters also exist for a wide range of languages and applications, including e.g., Perl, Prolog, MySQL, Tcl and Asf+Sdf. In RMI, only Java components can be connected; in RMIIIOP the service is implemented in Java, its functionality (client-side) is available to CORBA clients. The Java IDL framework is fully CORBA compliant. – Only the ToolBus has coordination support for component interaction. In the three other cases any undesired sequence of incoming and outgoing method calls will have to be prohibited by adding code to the component’s internals. Whereas RMI, RMI-IIOP and Java IDL just perform the wiring that connects the components, the ToolBus also provides workflow support. In relation to this workflow support, it would be interesting to compare the ToolBus to related workflow description languages such as the Business Process Modeling language [35] and the Web Services Description Language [36]. Implications for the ToolBus Approach. To overcome the problems of value-based transmission, we envisage the introduction of channels as sketched in Fig. 10(b). This model is inspired by the second workaround mentioned at the end of Sect. 5.3 and is completely transparent for the user. The idea is to stick to the strict call-by-value transmission model, but to implement the actual value transmission by data communication between sending tool and receiving tool thus offloading the ToolBus itself. Via the ToolBus, only an identification of the data value is transmitted between sender and receiver. The downside of this model is that it introduces the need for distributed garbage collection, since a value may be distributed to more than one receiving tool and the sender does not known when all receivers have retrieved their copy. Adding expiration times to values or reference counting at the ToolBus level may solve this problem.
6
Current Status
The current ToolBus was first specified in Asf+Sdf and has then been implemented manually in C. Its primary target was the renovation of the Asf+Sdf Meta-Environment. The next generation ToolBus is being implemented in Java and aims at supporting larger applications such as, for instance, a multi-user game site like www. gamesquare.nl with thousands of users. High performance and recovery of crashed game clients are then of paramount importance. The Java implementation is organized in such a way that the actual implementation of tools is as much hidden as possible. This is achieved by introducing the interface ToolInterface that describes the required ToolBus/tool interaction. This interface can be implemented by a variety of classes: ClassicToolBusTool: This implements the ToolBus/tool communication as used in current applications. The tool is executed as a separate, operating system level, process and the ToolBus/tool communication is achieved using sockets. JavaTool: This implements a new model that addresses one of the issues mentioned in Sect. 5: when ToolBus and tool run on the same computer and the tool is written
ToolBus: The Next Generation
239
in Java, then the tool can be loaded dynamically in the executing ToolBus, e.g. using Java Threads. In this way, the overhead of interprocess communication can be eliminated. JavaRMITool: This is a special case where a Java tool runs on another computer. SOAPTool: This implements communication with a tool that has a SOAP interface. A prototype implementation is under development that allows experimentation with the features mentioned in this paper.
7
Concluding Remarks
In this paper we have reflected on our experiences over the past years with the use of the ToolBus as a means to refactor a previously monolithic system: the Asf+Sdf Meta Environment. This real test case of the ToolBus has taught us some of its shortcomings: its data bottleneck in case very large data items are sent using pass-by-value, maintenance issues related to undisciplined message passing and questions such as how to deal with exceptions caused by e.g. crashing tools. Some of the ideas we showed in this paper could be implemented by changing or extending the Tscript (e.g. to implement a call-reply regime as discussed in Sect. 5.1), others will also require extending the ToolBus and the tool-adapters (e.g. to detect crashed tools in combination with exception handling as discussed in Sect. 5.2). We have also studied some related ideas and frameworks and we are now in a position where we have a new prototype of the ToolBus in Java, with a very open structure which allows for all sorts of experiments and case studies based on the experience we have with the existing ToolBus and the ideas presented in this paper.
Acknowledgments We thank Pieter Olivier for his contribution and input to the many interesting and fruitful discussions we have had about ToolBus related issues, and his efforts to get www.gamesquare.nl ToolBus enabled.
References 1. Klint, P.: A meta-environment for generating programming environments. ACM Transactions on Software Engineering and Methodology 2 (1993) 176–201 2. van den Brand, M.G.J., van Deursen, A., Heering, J., de Jong, H.A., de Jonge, M., Kuipers, T., Klint, P., Moonen, L., Olivier, P.A., Scheerder, J., Vinju, J.J., Visser, E., Visser, J.: The ASF+SDF Meta-Environment: a Component-Based Language Development Environment. In Wilhelm, R., ed.: Compiler Construction (CC ’01). Volume 2027 of Lecture Notes in Computer Science., Springer-Verlag (2001) 365–370 3. Bakker, H.C.N., Koorn, J.W.C.: Building an editor from existing components: an exercise in software re-use. Technical Report P9312, Programming Research Group, University of Amsterdam (1993) 4. van Vlijmen, S.F.M., Vriend, P.N., van Waveren, A.: Control and data transfer in the distributed editor of the ASF+SDF meta-environment. Technical Report P9415, University of Amsterdam, Programming Research Group (1994)
240
H. de Jong and P. Klint
5. Mauw, S., Veltink, G.J.: A process specification formalism. Fundamenta Informaticae XIII (1990) 85–139 6. Bergstra, J.A., Klint, P.: The ToolBus: a component interconnection architecture. Technical Report P9408, University of Amsterdam, Programming Research Group (1994) 7. Bergstra, J.A., Klint, P.: The ToolBus coordination architecture. In Ciancarini, P., Hankin, C., eds.: Coordination Languages and Models. Volume 1061 of Lecture Notes in Computer Science. (1996) 75–88 8. Bergstra, J.A., Klop, J.W.: Process algebra: specification and verification in bisimulation semantics. In Hazewinkel, M., Lenstra, J.K., Meertens, L.G.L.T., eds.: Mathematics & Computer Science II. Volume 4 of CWI Monograph., North-Holland (1986) 9. Olivier, P.A.: Embedded system simulation: testdriving the ToolBus. Technical Report P9601, University of Amsterdam, Programming Research Group (1996) 10. Dams, D., Groote, J.F.: Specification and Implementation of Components of a muCRL toolbox. Logic Group Preprint Series 152, Utrecht University, Dept. of Philosoph (1995) 11. Lisser, B., van Wamel, J.J.: Specification of components in a proposition solver. Technical Report SEN-R9720, Centrum voor Wiskunde en Informatica (CWI) (1997) 12. Diertens, B.: Simulation and animation of process algebra specifications. Technical Report P9713, Programming Research Group, University of Amsterdam (1997) 13. Bergstra, J.A., Klint, P.: The discrete time ToolBus. Technical Report P9502, University of Amsterdam, Programming Research Group (1995) 14. Bergstra, J.A., Klint, P.: The discrete time ToolBus. In Wirsing, M., Nivat, M., eds.: Algebraic Methodology and Software Technology. Volume 1101 of Lecture Notes in Computer Science., Springer-Verlag (1996) 286–305 15. Bergstra, J.A., Klint, P.: The discrete time ToolBus—a software coordination architecture. Science of Computer Programming 31 (1998) 205–229 16. van den Brand, M.G.J., Heering, J., Klint, P.: Renovation of the ASF+SDF meta-environment current state of affairs. In Sellink, M.P.A., ed.: Proceedings of the 2nd International Workshop on the Theory and Practice of Algebraic Specifications. electronic Workshops in Computing, Springer-Verlag (1997) 17. van den Brand, M.G.J., de Jong, H.A., Klint, P., Olivier, P.A.: Efficient Annotated Terms. Software, Practice & Experience 30 (2000) 259–291 18. Olivier, P.A.: A Framework for Debugging Heterogeneous Applications. PhD thesis, University of Amsterdam (2000) 19. Brand, M.G.J.v.d., Moreau, P.E., Vinju, J.J.: Environments for Term Rewriting Engines for Free. In: Rewriting Techniques and Applications. Lecture Notes in Computer Science, Springer Verlag (2003) To appear. 20. Gelernter, D., Carriero, N.: Coordination languages and their significance. Communications of the ACM 35 (1992) 96 21. de Jong, H.A., Olivier, P.A.: Generation of abstract programming interfaces from syntax definitions. Technical Report SEN-R0212, St. Centrum voor Wiskunde en Informatica (CWI) (2002) Submitted to Journal of Logic and Algebraic Programming. 22. Mosses, P.D.: System demonstration: Action semantics tools. In van den Brand, M.G.J., L¨ammel, R., eds.: Proceedings of the Second Workshop on Language Descriptions, Tools and Applications (LDTA 2002). Volume 65.3 of Electronic Notes in Theoretical Computer Science. (2002) 23. Brand, M.G.J.v.d., Ringeissen, C.: ASF+SDF parsing tools applied to ELAN. In Futatsugi, K., ed.: Third International Workshop on Rewriting Logic and its Applications (WRLA’2000). Volume 36 of Electronic Notes in Theoretical Computer Science., Elsevier Science Publishers (2001)
ToolBus: The Next Generation
241
24. Blom, S.C.C., Fokkink, W.J., Groote, J.F., van Langevelde, I., Lisser, B., van de Pol, J.C.: µCRL: A toolset for analysing algebraic specifications. In Berry, G., Comon, H., Finkel, A., eds.: Proc. of the CAV 2001. Volume 2102 of LNCS., Springer (2001) 250–254 25. Brinksma, E.: On the Design of Extended LOTOS–A Specification Language for Open Distributed Systems. PhD thesis, University Twente (1988) 26. Diertens, B.: New features in PSF I – Interrupts, Disrupts, and Priorities. Technical Report P9417, Programming Research Group, University of Amsterdam (1994) 27. Elnozahy, E.N.M., Alvisi, L., Wang, Y., Johnson, D.B.: A survey of rollback-recovery protocols in message-passing systems. ACM Computing Surveys (CSUR) 34 (2002) 375–408 28. Randell, B.: System structures for software fault tolerance. IEEE Transactions on Software Engineering SE–1 (1975) 21–232 29. Klint, P.: A Study in String Processing Languages. Volume 205 of Lecture Notes in Computer Science. Springer-Verlag (1985) 30. Zorzo, A., Romanovsky, A., Xu, J., Randell, B., Stroud, R., Welch, I.: Using coordinated atomic actions to design dependable distributed object systems. In: OOPSLA’97 Workshop on Dependable Distributed Object Systems. (1997) 31. Bergstra, J.A., Ponse, A., van Wamel, J.: Process algebra with backtracking. In: REX School/Symposium. (1993) 46–91 32. Object Management Group (OMG): The Common Object Request Broker: Architecture and Specification. (1999) http://www.omg.org/technology/documents/formal/corba_2.htm. 33. Sun MicroSystems Inc.: Java Remote Method Specification. (2003) http://java.sun.com/j2se/1.4/docs/guide/rmi. 34. Object Management Group: CORBA IIOP Specification. (2003) www.omg.org/technology/documents/formal/corba_iiop.htm. 35. Business Process Management Initiative: Business Process Modeling Language. (2002) http://www.bpmi.org/bpmi-downloads/BPML1.0.zip. 36. W3C: World Wide Web Consortium: Web Services Description Language. (2001) http://www.w3.org/TR/wsdl.
High-Level Specifications: Lessons from Industry Brannon Batson1 and Leslie Lamport2 1 2
Intel Corporation Microsoft Research
Abstract. We explain the rationale behind the design of the TLA+ specification language, and we describe our experience using it and the TLC model checker in industrial applications–including the verification of multiprocessor memory designs at Intel. Based on this experience, we challenge some conventional wisdom about high-level specifications.
1
Introduction
The first author is a computer architect with a master’s degree in electrical engineering. His work focuses on designing, implementing, and validating multiprocessor cache-coherence protocols. He has worked on TLA+ formal specifications for the cache-coherence protocols of two Digital/Compaq Alpha multiprocessors, and he is currently using TLA+ to model protocols on future Intel products. The second author is a computer science researcher who began verifying concurrent algorithms over 25 years ago [12]. About ten years ago, he devised TLA, a logic for reasoning about concurrent algorithms [15]. He later designed TLA+, a complete high-level specification language based on TLA [17]. The two authors view formal verification and TLA+ from two different, complementary vantage points. In this paper, we try to synthesize our two views to explore the rationale behind TLA+, describe our experience using it in industry, and derive some lessons from this experience. When discussing our individual experiences, we refer to the first and second authors as BB and LL, respectively. We begin by describing TLA+ and TLC, the TLA+ model checker. We then describe how TLA+ has been used at Digital/Compaq and at Intel. We next explore how our experience contradicts some conventional wisdom about specification, and we end with some simple conclusions.
2 2.1
TLA+ Desiderata
TLA+ is a high-level language for describing systems – especially asynchronous concurrent and distributed systems. It was designed to be simple, to be very expressive, and to permit a direct formalization of traditional assertional reasoning – the style of reasoning begun by Floyd [5] and Hoare [9] and extended to concurrent programs by Ashcroft [2], Owicki and Gries [21], Pnueli [24], and F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 242–261, 2003. c Springer-Verlag Berlin Heidelberg 2003
High-Level Specifications: Lessons from Industry
243
others [3,11,12,22]. Making it easy, or even possible, to build tools was not a design criterion for the language. The desire to formalize assertional reasoning, especially for liveness properties, led LL to base TLA+ on TLA (the Temporal Logic of Actions) [15], a simple variant of linear-time temporal logic [24]. To be practical, a temporal logic must be based on an expressive language for writing elementary, non-temporal expressions. The desire for simplicity and expressiveness led to the use of ordinary first-order logic and set theory for this underlying language of expressions. 2.2
From Math to TLA+
First-order logic and set theory provide a formalization of ordinary mathematics. TLA adds to them modalities for expressing temporal properties. Temporal modalities are useful for describing liveness (eventuality) properties. However, temporal logic, like any modal logic, is more complicated than ordinary math. TLA was therefore designed to put most of the complexity, both in describing a system and in reasoning about it, into the realm of ordinary math rather than into temporal logic. LL originally assumed that first-order logic and set theory extended with the temporal operators of TLA would provide the semantic basis for a language, but that a practical language would require conventional programming-language constructs such as assignment statements. However, not wanting to introduce unnecessary constructs, he decided to begin writing formal specifications using only mathematics, and to add other constructs as needed. To his surprise, he discovered that he did not need those conventional constructs. Instead, he added to TLA+ only the following extensions to ordinary mathematics: Unambiguous Syntax. A formal language must be unambiguous, meaning that it must be possible for a program to parse it. This led to eliminating from TLA+ two common practices of mathematicians: using juxtaposition as an operator and the overloading of operators. Mathematicians write the product of x and y as xy; in TLA+ it is written x ∗ y. (One could require a space between x and y to distinguish this product from the single variable xy, but that would make parsing difficult.) Mathematicians frequently overload operators—for example, f −1 could mean either the inverse of f or f raised to the power −1. There is no overloading of operators in TLA+. (The use of types can make some instances of overloading unambiguous; but for reasons explained below, TLA+ is untyped.) New Constructs. TLA+ borrows a few useful constructs from computer science—for example, allowing if/then expressions like if x = 0 then 1/x else 0 Also, mathematicians have no notation for explicitly describing a function— for example, the function whose domain is the set of reals and that maps every number to its negative. In TLA+, this function is written [x ∈ Real → −x ]
244
B. Batson and L. Lamport
(A computer scientist might write this as a λ expression, but TLA+ avoids the keyword lambda because of its potentially confusing connotations.) Definitions. Mathematicians have no standard convention for defining operators. They typically write something like “let ◦ be defined by letting a ◦ b equal . . . , for any a and b.” In TLA+, one writes: ∆
a ◦ b = ... Support for Large Specifications. Mathematicians typically introduce new variables informally as needed. This casual introduction of variables could lead to errors in large specifications, so TLA+ requires that variables be declared before they are used. Moreover, mathematicians will write “let x be an element of S ” even though two pages earlier they had defined x to have some other meaning. Formalizing this requires some method of restricting the scope of a declaration. TLA+ does this through the use of modules, which provide a mechanism for structuring large specifications. The mathematical operation of substituting expressions for variables is expressed in TLA+ by instantiating a module with expressions substituted for its declared variables. Support for Large Formulas. For a mathematician, a 20-line formula is large. In a specification, a 200-line formula is not unusual. To aid in writing long formulas, TLA+ allows bulleted lists of conjuncts and disjuncts, using indentation to eliminate parentheses [14]. For example, ∧ reqQ[p][i ].type = “MB” ∧ ∨ DirOpInProgress(p, reqQ[p][i ].adr ) ∨ reqQ[p][j ].adr = reqQ[p][i ].adr means (reqQ[p][i ].type = “MB”) ∧ ( (DirOpInProgress(p, reqQ[p][i ].adr )) ∨ (reqQ[p][j ].adr = reqQ[p][i ].adr )) TLA+ also has a let/in construct for making definitions local to an expression. This permits structuring an expression for easier reading as well as combining multiple instances of the same subexpression. TLA. TLA+ extends ordinary math by adding the modal operators of TLA. The most important of these is prime ( ), where priming an expression makes it refer to the value of the expression in the next state. For example, x = x + 1 is an action, a predicate on a pair of states (called the current and next state), that is true iff the value of x in the next state is one greater than its value in the current state. Although formally a modal operator, expressions with primes obey the rules of ordinary mathematics, where x is treated like a new variable unrelated to the variable x . TLA also has a few simple temporal operators, used mostly for expressing liveness properties. As we will see, these operators appear in only a small part of a specification. We have listed the ways in which TLA+ differs from math as used by ordinary mathematicians. Where practical, TLA+ maintains the richness and economy of
High-Level Specifications: Lessons from Industry
245
inline AppendNum(n) { i = 0; do :: i < MaxSeqLen && seq[i] != 0 && seq[i] != n -> i++ :: else -> break od; if :: i >= MaxSeqLen || seq[i] != 0 :: else -> seq[i] = n fi } Fig. 1. Part of a toy specification written in Promela. ∆
AppendNum(n) = ∧ ∀ i ∈ 1 . . Len(seq) : n = seq[i] ∧ seq = Append (seq, n) ∧ num = num Fig. 2. The TLA+ version of the piece of specification in Figure 1.
ordinary mathematical notation. For example, while a textbook on first-order logic typically defines only a simple quantified formula such as ∃ x : exp, mathematicians typically write formulas like: ∃ x , y ∈ S , z , w ∈ T : exp TLA+ allows this kind of richer syntax. On the other hand, mathematicians do not use extraneous keywords or punctuation. TLA+ maintains this simplicity of syntax; for example, successive statements are not separated by punctuation. This syntactic economy makes TLA+ specifications easy for people to read, but surprisingly hard for a program to parse. 2.3
A Brief Taste of TLA+
To provide a sense of how a specification written in TLA+ compares to one written in a typical specification language used in industry, Figures 1 and 2 give small parts of two versions of a toy specification. The first version is written in Promela, the input language of the Spin model checker [10]. (It was written by Gerard Holzmann, the designer of the language.) The second is written in TLA+. The figures contain corresponding parts of the two specifications, although those parts are not completely equivalent. (One significant difference is mentioned in Section 3 below.) Figure 3 shows a small part of a TLA+ specification of a real cache-coherence protocol for a multiprocessor computer. Observe that it looks very much like the piece of toy specification of Figure 2; it is just a little more complicated. In both examples, the TLA+ specification uses only simple mathematics.
246
B. Batson and L. Lamport
∧ req.type = “MB” ∧ ∀ i ∈ 1 . . (idx − 1) : ∧ reqQ[p][i].type = “MB” ∧ DirOpInProgress(p, reqQ[p][i].adr ) ∧ ∀ j ∈ 1 . . (i − 1) : reqQ[p][j ].adr = reqQ[p][i].adr ∧ ¬ ∃ m ∈ msgsInTransit : ∧ m.type ∈ {“Comsig”, “GetShared”, “GetExclusive”, “ChangeToExclusive”} ∧ m.cmdr = p Fig. 3. A small piece of a real TLA+ specification.
Crucial to the simplicity of TLA+ is that it is based on the ordinary mathematics used by ordinary mathematicians. Computer scientists have devised many forms of weird mathematics. They have introduced bizarre concepts such as “nondeterministic functions”, leading to strange formalisms in which the formula A = A is not necessarily true. Ordinary mathematics was formalized about a century ago in terms of first-order logic and (untyped) set theory. This is the formal mathematics on which TLA+ is based. The use of ordinary mathematics in TLA+ led BB to remark: if I want to find out what an expression in a TLA+ specification means, I can just look it up in a math book. Computer scientists and engineers, accustomed to computer languages, are likely to be surprised by the expressiveness of TLA+. BB describes the power of TLA+ in this way: A single line of TLA+ can do powerful manipulations of complex data structures. This allows me to focus on the algorithm without getting lost in bookkeeping tasks. TLA+ tops even perl in this regard. Unlike perl, however, TLA+ is unambiguous. The simplicity and expressiveness of TLA+ is not the result of any cleverness in the design of the language; it comes from two thousand years of mathematical development, as formalized by great mathematicians like Hilbert. TLA+ is described in a recent book by LL, which is available on the Web [17]. 2.4
The Structure of a TLA+ Specification
TLA+ does not enforce any particular way of structuring a specification, allowing declarations and definitions to appear in any order as long as every identifier is declared or defined before it is used. Moreover, by partitioning it into separate modules, one can structure a specification to allow reading higher-level definitions before reading the lower-level definitions on which they depend. Logically, most TLA+ specifications consist of the following sections: Declarations. The declarations of the constant parameters and the variables that describe the system. There are typically a dozen or so declared identifiers.
High-Level Specifications: Lessons from Industry
247
Definitions of Operations on Data Structures. These define mathematical operators used to describe operations specific to the particular system. For example, a specification of a system that can perform a masked store to a register might define MaskedStoreResult(curr , val , msk ) to be the new value of a register, whose current value is curr , after storing to it a value val through a mask msk . A specification usually has only a few such operator definitions, each just a few lines long. The operator MaskedStoreResult is more likely to be used only in the definition that describes the masked store action, in which case it would probably be defined locally in a let/in expression. The Initial Predicate. The definition of a formula that describes the possible initial values of the variables. It is a conjunction of formulas x = . . . or x ∈ . . . for each variable x , where the “. . .” is usually a simple constant expression. The Next-State Action. The definition of a formula containing primed and unprimed variables that describes the system’s possible next state as a function of its current state. It is generally defined as a disjunction of subactions, each describing one type of system step. For example, in the specification of a mutual-exclusion algorithm, the next-state action might have a disjunct ∃ p ∈ Proc : EnterCS (p) , where EnterCS (p) describes a step in which a process p enters its critical section. The definition of the next-state action comprises the bulk of the specification. Liveness. The definition of a temporal formula specifying the liveness properties of the system, usually in terms of fairness conditions on subactions of the next-state action. It typically consists of about a dozen lines. The Specification. This is the one-line definition ∆
Spec = Init ∧ 2[Next]v ∧ Liveness that defines Spec to be the actual specification, where Init is the initial predicate, Next is the next-state action, Liveness is the liveness formula, and v is the tuple of all variables. For a high-level specification that describes a system’s correctness properties, Spec is the inner specification in which internal variables are visible. The true specification would be obtained by hiding those internal variables. If h is the tuple of all internal variables, then Spec with those variables hidden is represented by the TLA formula ∃ h : Spec. For technical reasons, TLA+ requires that this formula be defined in a separate module from the one defining Spec. However, hiding the internal variables is done for philosophical correctness only. The TLC model checker cannot handle the TLA hiding operator ∃ , and in practice, one uses the inner specification Spec. The only temporal operators that appear in the entire specification are the ones in the liveness formula and the single 2 in the final specification. The rest of the specification—usually about 99% of it—consists entirely of ordinary math,
248
B. Batson and L. Lamport
with no temporal operators. Moreover, engineers generally do not use the liveness property. The complexity of model checking liveness properties is inherently much greater than that of checking safety properties, which means that liveness can be checked only for extremely small models of real systems. Engineers therefore usually do not even write the liveness property; instead their specification is ∆ Spec = Init ∧ 2[Next]v The only temporal operator in this specification is the single 2. 2.5
The Size of Specifications
In principle, one could write specifications of any size, from tiny toy examples to million-line monsters. But tiny toys are of no practical interest, and the module structure of TLA+ is probably inadequate for handling the complexity of specifications longer than ten or twenty thousand lines. We have identified the following three classes of applications for which TLA+ is useful, each with a surprisingly narrow range of specification sizes: Abstract Algorithms. These are the types of concurrent algorithms that are published in journals—for example, the Disk Paxos algorithm [6]. Their specifications seem to require a few hundred lines of TLA+. Interesting algorithms simple enough to have much shorter specifications seem to be rare, while an algorithm with a much longer specification is probably not abstract enough for journal publication. Correctness Properties. These are descriptions of the properties that protocols or systems should satisfy. One example is a description of the memory model that a cache-coherence protocol is supposed to implement [20]. Their specifications also seem to require a few hundred lines of TLA+. The requirements of a real system are seldom simple enough to have a shorter specification, while a statement of correctness requiring a much longer specification would probably be too complicated to be useful. High-Level Protocol or System Designs. These describe the high-level designs of actual systems or protocols. We know of no published example; such designs are usually proprietary. We have found that these specifications are about two thousand lines of TLA+. Any such specification is an abstraction of the actual lower-level implementation. Engineers want to describe their design in as much detail as they can. However, if the specification takes much more than two thousand lines, then the design is too complicated to understand in its entirety, and a higher-level abstraction is needed.
3
TLC: The TLA+ Model Checker
TLA+ was not designed with tools in mind; LL believed that a practical model checker for it was impossible and advised against trying to write one. Fortunately, Yuan Yu ignored him and wrote TLC, an explicit-state model checker for TLA+ programmed in Java [26].
High-Level Specifications: Lessons from Industry
249
TLA+ is an extremely expressive language—for example, it can easily be used to specify a program that accepts an arbitrary Turing machine as input and tells whether or not it will halt. No model checker can handle all TLA+ specifications. TLC handles a subset of TLA+ that seems to include most specifications of algorithms and correctness properties, as well as all the specifications of protocol and system designs that engineers actually write. Those few specifications arising in practice that TLC does not handle can be easily modified, usually by changing only a few lines, so they can be checked by TLC. Explicit-state model checking is possible only for bounded-state specifications. Most high-level specifications are not bounded-state because the state contains data structures such as unbounded queues. We want engineers to use the TLA+ specification as the official high-level specification of their system, and a major goal of TLC is that the specification should not have to be changed to allow it to be checked. So TLC accepts as input a TLA+ specification and a configuration file that defines a finite model. The configuration file instantiates the constant parameters of the specification—for example, instructing TLC to replace the parameter Proc that represents the set of processors with a set containing three elements. The configuration file can also specify a constraint on the state space, instructing TLC to explore only states satisfying the constraint. As an example, we return to the toy specification, part of which is specified in Figures 1 and 2. In that specification, the variable seq represents a queue that can grow arbitrarily large. To model check it with TLC, we write a simple module that imports the original specification, declares a constant MaxSeqLen, and defines a constraint asserting that the length of seq is at most MaxSeqLen. We then instruct TLC to check the specification using that constraint, substituting a specific value for MaxSeqLen. We do this for increasing values of MaxSeqLen until we are confident enough that the specification is correct, or until the space of reachable states becomes so large that it takes TLC too long to explore it. In contrast, note in Figure 1 that, to model check the specification with Spin, the parameter MaxSeqLen had to be made part of the actual Promela specification. Operations on many common data types are not built into TLA+, but are instead defined in standard modules. For example, the natural numbers are defined in the Naturals module to be an arbitrary set satisfying Peano’s axioms, and arithmetic operation are defined in the usual way in terms of the next-number function. A specification that uses operators like + on natural numbers imports the Naturals module. It would be rather difficult for TLC to compute 2 + 2 from the definition of + in that module. Instead, such arithmetic operations are programmed in Java using TLC’s general module overriding mechanism. When a specification imports a module named M , TLC looks for a Java class file named M.class. If it finds one, it replaces operators defined in M with their Java implementations in M.class. There are Java implementations for common operators on numbers, sequences (lists), finite sets, and bags (multisets) that are defined in the standard modules. Ordinary users can also write their own Java class files to provide more efficient implementations of the operators that they define. However, we know of no case where this was necessary, and we know of only one user (a researcher) who has done it.
250
B. Batson and L. Lamport
TLC has a primitive command-line interface. Debugging is done by adding print statements to the specification. Although this violates the principle of not having to modify the specification to check it, the print statements are usually removed as soon as initial debugging is completed and simple “coding” errors corrected. Design errors are generally found by examining error traces. We hope eventually to add a graphical user interface. TLC is coded in Java. It is a multithreaded program and there is a version that can use multiple machines. For checking safety properties, it obtains close to an n-fold speedup with n processors when run on Alphas using a high quality Java runtime. However, we have found that the poor implementation of multithreading in many Java runtimes can significantly reduce the speedup. The largest case we know of was one with 900 million reachable states that took about two weeks on a four-processor machine. The expressiveness of TLA+ makes it essentially impossible to compile TLA+ specifications into efficient code. Therefore, TLC must interpret specifications. We guess that this makes TLC about ten times slower than explicit-state model checkers that require specifications to be written in a low-level, compilable language. Because TLC maintains its data structures on disk, it has essentially no space limitations for checking safety properties. The goal of TLC is to help engineers find bugs in their designs. Experience tells an engineer what kind of bugs a particular finite model is and is not likely to find. For example, if a cache-coherence protocol handles different addresses independently, then it may suffice to check models with only a single memory address. A specification is an abstraction, and checking it can find only errors that are present in that abstraction. Lower levels of detail introduce many other sources of error not reflected in the specification. In most industrial applications, engineers need cost-effective methods of finding bugs; they are not seeking perfection.
4
TLA+ Use in Industry
4.1 Digital/Compaq/HP TLA+ and TLC were conceived at the Digital (later Compaq) Systems Research Center. The first serious industrial use of TLA+ was for specifying and writing part of a hand proof of the cache-coherence protocol of a multiprocessor codenamed Wildfire, based on the Alpha EV6 processor [7]. The Wildfire experience inspired Yuan Yu to write TLC. It also persuaded the verification team to write a TLA+ specification of the cache-coherence protocol of the next generation Alpha, the EV7. The initial specification viewed an entire processor chip as a single component; the level of abstraction was later lowered to model protocol interactions between on-chip components as well. TLC checked important properties of the protocol and helped find several bugs. TLC and the EV7 protocol specification were also used as the basis of a project to improve test coverage for hardware simulation [25]. The processor design team for the next Alpha processor, the EV8, began using TLA+ to write the official specification of its cache-coherence protocol. However, development of that processor was cancelled.
High-Level Specifications: Lessons from Industry
251
TLA+ and TLC were also applied by Compaq engineers to the cache- coherence protocols of two Itanium-based processors. Researchers used TLC to debug a bus protocol proposal and to help develop database recovery and cache management protocols. TLC is now used routinely by some researchers to check the concurrent algorithms that they develop. The use of TLA+ and TLC at Digital and Compaq, some of which continued at HP, is described in [18]. 4.2
Intel
We now describe the use of TLA+ and TLC by BB and his colleagues at Intel. The actual specifications are proprietary and have not been viewed by anyone outside Intel. Overview of the Problem. Designing a complex system starts with a problem statement and an appropriate set of boundary conditions. A component of a computer system is initially represented abstractly as a black box, with assertions about its functionality and with some guidelines on performance, cost, and complexity. The engineering process involves iteratively refining this abstract model into lower-level models. Each lower-level model is a representation of the design at a certain level of abstraction, and it has a specific purpose. Some models are meant to evaluate tradeoffs between scope, performance, cost, and complexity. Others carry the design down to the low level of detail needed to manufacture the component. The engineering process therefore creates multiple representations of a design. Validation entails checking these multiple representations against one another. Designers of digital systems have good tools and methods for validating mid-level functional models, written in a hardware description language (HDL) like VHDL or RTL, against lower-level models such as circuit net-lists. However, they have not had as much success checking the higher-level functional representations of the design against one another, and against the initial problem statement and functional assertions. For some components, there is an intuitive correlation between the highlevel notions of correctness and the HDL model; such components tend not to be difficult to validate. Other components, like multiprocessor cache-coherence protocols, are sufficiently complex that checking the HDL model against the problem statement is quite challenging. We need formal techniques from the world of mathematics to perform this high-level validation. Although formal methods are based on mathematics, engineers view them differently from the way mathematicians do. To engineers, formal verification is simply another imperfect validation tool (albeit a powerful one). A TLA+ specification is only an abstraction of the actual system, and model checking can usually validate the specification only for a highly restricted set of system parameters. Validating the specification therefore cannot guarantee that there are no errors in the system. For engineers, formal verification is a way of finding bugs, not of proving correctness.
252
B. Batson and L. Lamport
The main benefit of applying TLA+ to engineering problems comes from the efficiency of the TLC model checker in reaching high levels of coverage and finding bugs. A secondary benefit we have encountered is the ability of TLA+ and TLC to provide good metrics for the complexity of a design. Complexity is a major consideration in evaluating design tradeoffs. However, unlike performance or cost, engineers have not historically had a good way to quantify algorithmic complexity before attempting to validate a design. TLA+ encourages designers to specify the design abstractly, suppressing lower-level details, so the length of the specification provides a measure of a design’s complexity. TLC reports the size of the reachable state space, providing another measure of complexity. Experience and intuition will always have a place in evaluating complexity, but TLA+ and TLC provide robust and impartial input to the evaluation. Having this input early in the design process is of considerable value.
Designing with TLA+. The core group at Intel started using TLA+ at Compaq while working on the Alpha EV7 and EV8 multiprocessor projects described above. From that experience, the Alpha engineers learned that multiprocessor cache-coherence protocols are an ideal candidate for formal methods because most of the protocol bugs can be found at a high level of abstraction. They also learned that the true value of TLA+ and TLC would be realized when (a) they were applied early enough in the design to provide implementation feedback, and (b) the implementation was based directly on the specification that had been verified. On the EV8 project, the TLA+ specification was completed before the design was stable, and it provided important feedback to the designers. When the engineers from the Alpha group joined Intel, they began applying their experience in writing TLA+ specifications when collaborating with other Intel engineers on cache-coherence protocols for future Intel products. Intel engineers are now using TLA+ as an integral part of the design process for the protocols that are under study. Whiteboard Phase. Designing one cache-coherence protocol from scratch provided the engineers with the opportunity to evaluate TLA+ as a prototyping platform for complex algorithms. Work on this protocol started by exploring the design space on a whiteboard for about two months. In this phase, basic message sequencing was determined, as were some coarse notions of what state had to be recorded at the protocol endpoints. A basic direction was set, based on the guidelines for functionality, performance, and cost. Because of their background, engineers tend to visualize an algorithm in terms of a particular implementation. They are better at gauging implementation complexity than at measuring algorithmic complexity. One benefit of having engineers write formal specifications is that it helps them learn how to think about a protocol abstractly, independent of implementation details. We found that, even in the whiteboard phase of the protocol design, the Intel engineers were able to make some judgments on complexity by asking themselves, “How would I code this in TLA+ ?”.
High-Level Specifications: Lessons from Industry
253
The whiteboard phase produced a general understanding of the protocol philosophy, an understanding of the constraints placed on the communication medium, the basic message flows, and coarse ideas on what state needed to be maintained. The next step was to introduce the rigor of a formal specification. TLA+ Scratchpad Phase. The TLA+ scratchpad phase of the project involved formally describing the abstract system, with appropriate state variables representing high-level components. This phase took about two months, starting with the initial design of the protocol. The difficulty lay not in the use of TLA+— engineers frequently learn new programming languages—but rather in (a) determining the layer of abstraction and (b) exploring the protocol’s corner cases. Task (a) is where TLA+ forces engineers to think about the protocol abstractly, which they often find unnatural. Their ability to think abstractly improves with experience writing TLA+ specifications. Task (b) is inevitable when documenting a protocol formally, as it forces the designers to explore the corner cases. During the scratchpad phase, the designers had to return to the whiteboard a few times when they encountered new race cases while writing the specification. The actions that formed the major blocks of the specification were chosen early; very few changes were made later. The Intel engineers adopted a methodology used in the earlier Alpha specifications, in which the decomposition of high-level named actions is based on classifying the protocol messages that they process. This methodology has led to fairly readable specifications, since it means that each action changes only a few local state variables. It encouraged the protocol specifications to be designed in a modular way, which also enabled the inter-module interfaces in the specification to be similar to their low-level counterparts in the implementation. Running TLC. The initial week or so of running TLC was spent finding and fixing typographical errors and type mismatch problems. This time could probably have been shortened by doing more syntax checking when writing the specification, which is what one often does when programming. The next four weeks saw a continuous process of running TLC, finding bugs, fixing them, and re-running TLC. During this phase of the project, many assumptions and assertions about the protocol were brought into question. This had the effect of educating the engineers about the protocol they had designed. We have found that TLC can be a useful learning tool if we use in-line assertions and global invariants to check everything we think is true. The Intel engineers were able to develop an intuitive understanding of the correctness of the protocol by developing meaningful global invariants and having TLC check them. If an assertion or invariant fails, TLC generates a counterexample that is useful for visualizing a difficult race case. These counterexamples are such a powerful teaching aid that the Intel engineers have developed tools to translate the TLC output into nicely formatted protocol flow diagrams that are easier to read. Another useful feature of the TLC model checker is its coverage checking. TLC can print the number of times each action was “executed”. This provides a simple way to identify holes in coverage. Much of the effort expended by the engineers in debugging the specification was spent eliminating each of these
254
B. Batson and L. Lamport
holes, or convincing themselves that it represented an action that could never happen. The performance of the model checker was sufficient to debug a large protocol specification. The engineers determined a base configuration that would “execute” all the actions and that displayed all interesting known cases. This configuration could be run on a four-processor machine in about a day, enabling fast turn-around on bug fixes. Larger configurations were periodically run as sanity checks on the smaller ones. The engineers would also run TLC in simulation mode, which randomly and non-exhaustively explores the state space, allowing them to check much larger configurations. Such random simulations are similar to the ones engineers typically perform on lower-level models, but it has the advantage of being several orders of magnitude faster because it is based on the abstract TLA+ model, and it provides a robust metric for coverage. Optimizing with TLC. Once the initial protocol specification was successfully checked by TLC, the Intel engineers were able to use it as a test bed for exploring optimizations. TLA+ is an ideal language to explore changes because its expressiveness usually allows the new version to be written quickly. Model checking the modified specification with TLC not only checks functional correctness, but it also measures any increase in the state space. Such an increase implies additional algorithmic complexity. The engineers spent several months exploring additions to the protocol, testing them with TLC. As a general rule, they would consider adopting only those optimizations that did not appreciably expand the state space. The insight that TLA+ and TLC gave into the complexity of modifications to the protocol was invaluable in iterating towards an optimal solution that adequately weighed algorithmic complexity along with factors like cost and performance. A significant optimization was later made to the protocol. This optimization followed the normal design cycle described above, though on a compressed schedule. With the original design yielding a good starting point, the entire cycle (whiteboard phase, TLA+ coding, and verification with TLC) was done within six weeks. This modification was accomplished by a recent college graduate with an undergraduate degree in engineering. He was able to learn TLA+ well enough within a matter of weeks to do this work. Feedback on TLA+ Syntax. The feedback we have received from engineers about the TLA+ language has been mostly positive. Engineers are usually able to pick up and understand a specification within a couple of days. One mistake we made was to present TLA+ to hardware designers as similar to a programming language. This led to some frustration. A better approach seems to be to describe TLA+ as being like a hardware description language. Engineers who design digital systems are well acquainted with methods for specifying finite-state machines, with the associated restrictions of allowing a primed variable to be assigned a value only once within a conjunction, not allowing a primed variable to appear in a conjunction before the assignment of its value, etc. To an engineer, TLA+ looks like a language for specifying finite-state machines.
High-Level Specifications: Lessons from Industry
255
While writing the protocol specification at Intel, BB was impressed by the ease of specifying complex data structures in TLA+ as sets and tuples. The part of the specification that described and manipulated data structures was a small part of the complete protocol specification. This compact specification of “bookkeeping tasks”, along with the overall expressiveness of TLA+, won over the engineers who were accustomed to using more clumsy functional languages for specifying complex algorithms. For the algorithmic specification, TLA+ naturally encourages nested disjunctions of conjunctions (known to engineers as sums of products of expressions). This method for specifying Boolean formulas has both advantages and disadvantages. One advantage is that it allows expressive comment blocks and assertions to be inserted in-line with a nested conjunct. A disadvantage is that this tends to lead to large specifications. The engineers are experimenting with the use of TLA+ operators to encode large blocks of regular Boolean disjunctions as truth tables, which engineers find more natural to work with.
5
Some Common Wisdom Examined
Based on our experience using TLA+, we now examine the following popular concepts from the world of programming: types, information hiding, object-oriented languages, component-based/compositional specifications, hierarchical description/decomposition, and hierarchical verification. Most of these concepts were mentioned in this symposium’s call for papers. We do not question the usefulness of these concepts for writing programs. But high-level specifications are not programs. We find that in the realm of high-level specifications, these ideas are not as wonderful as they appear. 5.1
Types
Very simple type systems are very restrictive. Anyone who has programmed in Pascal has written programs that were obviously type-correct, but which were not allowed by Pascal’s simple type system. Moderately complicated type systems are moderately restrictive. A popular type system is that of higher-order logic [8]. However, it does not allow subtyping. With such a type system, an integer cannot be a real number. When writing Fortran programs, one gets used to 1.0 being unequal to 1. One should not have to put up with that kind of complication in a specification language. Subtyping is provided by predicate subtyping, perhaps best known through its use in the PVS verification system [23]. We will see below a problem with PVS’s predicate subtyping. Moreover, predicate subtyping is not simple. It has led to several bugs that caused PVS to be unsound. For a typed language to be as expressive as TLA+, it will need an extremely complicated type system, such as that of Nuprl [4]. Engineers have a hard enough task dealing with the complexity of the systems that they design; they don’t want to have to master a complicated type system too.
256
B. Batson and L. Lamport
A specification consists of a large number of definitions, including many local ones in let/in expressions. Although an operator may have a simple type, it is often hard or impossible to declare the types of the operators defined locally within its definition. Even when those type declarations are possible, LL has found that they clutter a specification and make it harder to read. (Early precursors of TLA+ did include type declarations.) Any information contained in a type declaration that is helpful to the reader can be put in a comment. The main virtue of types, which makes us happy to bear the inconvenience they cause when writing programs, is that they catch errors automatically. (That advantage disappears in type systems with predicate subtyping, in which type checking can require manually guided theorem proving.) However, we have found that the errors in a TLA+ specification that could have been found by type checking are generally caught quite quickly by running TLC with very small models. The problems with types are discussed at length in [19]. We want to emphasize that we do not dispute the usefulness of types in programming languages. We prefer to program in strongly typed languages. We are questioning the use of types only in a high-level specification language. 5.2
Information Hiding
We have learned that programmers should hide irrelevant implementation details. However, a high-level specification should not contain implementation details. Such details will appear in a well-written specification only if an inexpressive language requires high-level concepts to be described by low-level implementations. TLA+ provides users with powerful mathematical objects like sets and functions; they don’t have to encode them in arrays of bits and bytes. Such “bookkeeping details” do not occur in specifications written in a truly high-level language like TLA+, so there is no need to hide them. 5.3
Object-Oriented Languages
The mathematical concept underlying object oriented programming languages can be described as follows. A program maintains identifiers of (references to) objects. There is a function Obj that maps the set ObjectId of object identifiers to a set Object of objects. An object-oriented language simply hides the function Obj , allowing the programmer to write o.field instead of Obj [o].field , where o is an object identifier. Eliminating explicit mention of Obj can make a specification look a little simpler. But it can also make it hard to express some things. For example, suppose we want to assert that a property P (obj ) holds for every object obj . (Usually, P (obj ) will assert that, if obj is a non-null object of a certain type, then it satisfies some property.) This is naturally expressed by the formula ∀ o ∈ ObjectId : P (Obj [o]) It can be difficult or impossible to express in a language that hides Obj .
High-Level Specifications: Lessons from Industry
257
Object-orientation introduces complexity. It raises the problem of aliasing. It leads to the confusing difference between equality of object identifiers and equality of objects—the difference between o1 = o2 and o1.equals(o2). You can’t find out what o1.equals(o2) means by looking it up in a math book. Object-oriented programming languages were developed for writing large programs. They are not helpful for two-thousand-line programs. Object orientation is not helpful for two-thousand-line specifications. 5.4
Component-Based/Compositional Specifications
A high-level specification describes how the entire system works. In a TLA+ specification, a component is represented by a subexpression of the next-state relation—usually by a disjunct. We can’t understand a formula by studying its subexpressions in isolation. And we can’t understand a system by studying its components in isolation. We have known for 20 years that the way to reason about a distributed system is in terms of a global invariant, not by analyzing each component separately [13]. Many tools have been developed for debugging the low-level designs of individual hardware components. Engineers need a high-level specification to catch bugs that can’t be found by looking at individual components. 5.5
Hierarchical Description/Decomposition
Hierarchical description or decomposition means specifying a system in terms of its pieces, specifying each of those pieces in terms of lower-level pieces, and so on. Mathematics provides a very simple, powerful mechanism for doing this: the definition. For example, one might define A by ∆
A = B ∨C ∨D and then define B , C , and D. (TLA+ requires that the definitions appear in the opposite order, but one can present them in a top-down order by splitting the specification into modules.) Building up a specification by a hierarchy of definitions seems simple enough. But a specification language can make it difficult in at least two ways: – It can restrict the kinds of pieces into which a definition can be broken. For example, it might require the pieces to be separate processes. There is no reason to expect that splitting the system into separate processes will be the best way to describe it. – It can use a strict type system. For example, suppose x is a variable of type real number, and we want to define an expression A by ∆
A = if x = 0 then B else C where B is defined by ∆
B = 1/x
258
B. Batson and L. Lamport
This is a perfectly reasonable definition, but PVS’s type system forbids it. PVS allows the expression 1/x only in a context in which x is different from 0. This particular example is contrived, but TLA+ specifications often contain local definitions in let/in expressions that are type-correct only in the context in which they are used, not in the context in which they are defined. How Intel engineers use and don’t use hierarchical decomposition is somewhat surprising. As we observed above, the major part of a TLA+ specification is the definition of the next-state action. Intel engineers use the common approach of decomposing this definition as a disjunction such as ∆
Next = ∃ p ∈ Proc : A1 (p) ∨ . . . ∨ An (p) where each Ai (p) describes a particular operation performed by process p. They also use local definitions to make a definition easier to read. For example, an action Ai (p) might be defined to equal let newV = . . . newW = . . . in . . . ∧ v = newV ∧ w = newW ... This allows a reader to scan the in clause to see what variables the action changes, and then read the complex definitions of newV and newW to see what the new values of v and w are. What the Intel engineers do not do is use hierarchical decomposition to hide complexity. For example, they would not eliminate common subexpressions by writing ∆ SubAction(p) = . . . ∆
A1 (p) = . . . ∧ SubAction(p) ∧ . . . ∆
A2 (p) = . . . ∧ SubAction(p) ∧ . . . if the two instances of SubAction(p) represent physically distinct components. The Intel engineers rely on the TLA+ specification to gauge the complexity of their designs, using the number of lines in the specification as a measure of a design’s complexity. This is possible because TLA+ does not introduce the extraneous details needed by lower-level languages to encode higher-level concepts. 5.6
Hierarchical Verification
Hierarchical verification works as follows: To show that a system described by the specification Sys implements the correctness properties Spec, we write an intermediate-level spec Mid and show that Sys implements Mid and Mid implements Spec. In TLA+, implementation is implication. To show that a specification ∃ x : F implements a specification ∃ y : G, we must show
High-Level Specifications: Lessons from Industry
259
(∃ ∃ x : F ) ⇒ (∃ ∃ y : G) Here, x and y are tuples of variables, which we assume for simplicity to be distinct. By simple logic, we show this by showing F ⇒ (G with y ← exp) for some tuple of expressions exp, where with denotes substitution (which is expressed in TLA+ by module instantiation). The tuple exp is called a refinement mapping [1]. To show that Sys implies Spec, we must show Sys ⇒ (ISpec with h ← exp) where ISpec is the inner specification, with internal variables h visible. To use hierarchical verification, we find an intermediate-level inner specification IMid , with internal variables m visible, and show: Sys ⇒ (IMid with m ← exp1) IMid ⇒ (ISpec with h ← exp2) When the verification is done by TLC, there is no reason for such a decomposition; TLC can verify directly that Sys implements ISpec under the refinement mapping. When the verification is done by mathematical proof, this decomposition seems reasonable. However, as LL has argued elsewhere [16], it is just one way to decompose the proof; it is not necessarily the best way. Unfortunately, the whole problem of verifying that a high-level design meets its specification is not yet one that is being addressed in the hardware community. Thus far, engineers are checking only that their TLA+ design specifications satisfy an incomplete set of invariants. Checking that they satisfy a complete specification is the next step. Engineers want to do it, but they have to learn how—which means learning how to find a correct refinement mapping. TLC has the requisite functionality to do the checking, but only doing it for real systems will tell if it works in practice. The reader can get a feeling for the nature of the task by trying to solve the Wildfire Challenge Problem [20].
6
Conclusions
Our industrial experience with TLA+ has led us to some simple, common-sense conclusions: – Buzzwords like hierarchical and object-oriented are to be viewed with suspicion. – A language for writing high-level specifications should be simple and have effective debugging tools. – Only proofs and model checking can catch concurrency bugs in systems. For the vast majority of applications, proofs are not a practical option; engineers have neither the training nor the time to write them.
260
B. Batson and L. Lamport
– A specification method cannot be deemed a success until engineers are using it by themselves. TLA+ and TLC are practical tools for catching errors in concurrent systems. They can be used very early in the design phase to catch bugs when it is relatively easy and cheap to fix them. Writing a formal specification of a design also catches conceptual errors and omissions that might otherwise not become evident until the implementation phase. TLA+ is not just for industrial use. Anyone who writes concurrent or distributed algorithms can use it. We invite the reader to give it a try.
References 1. Mart´ın Abadi and Leslie Lamport. The existence of refinement mappings. Theoretical Computer Science, 82(2):253–284, May 1991. 2. E. A. Ashcroft and Z. Manna. Formalization of properties of parallel programs. In Machine Intelligence, volume 6. Edinburgh University Press, 1970. 3. K. Mani Chandy and Jayadev Misra. Parallel Program Design. Addison-Wesley, Reading, Massachusetts, 1988. 4. R. L. Constable, S. F. Allen, H. M. Bromley, W. R. Cleaveland, J. F. Cremer, R. W. Harper, D. J. Howe, T. B. Knoblock, N. P. Mendler, P. Panagaden, J. T. Sasaki, and S. F. Smith. Implementing Mathematics with the Nuprl Proof Development System. Prentice-Hall, 1986. 5. R. W. Floyd. Assigning meanings to programs. In Proceedings of the Symposium on Applied Math., Vol. 19, pages 19–32. American Mathematical Society, 1967. 6. Eli Gafni and Leslie Lamport. Disk paxos. To appear in Distributed Computing., 2002. 7. Kourosh Gharachorloo, Madhu Sharma, Simon Steely, and Stephen Van Doren. Architecture and design of AlphaServer GS320. In Anoop Gupta, editor, Proceedings of the Ninth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS IX), pages 13–24, November 2000. 8. M. J. C. Gordon and T. F. Melham. Introduction to HOL: A Theorem Proving Environment for Higher Order Logic. Cambridge University Press, 1993. 9. C.A.R. Hoare. An axiomatic basis for computer programming. Communications of the ACM, 12(10):576–583, October 1969. 10. Gerard Holzmann. The model checker spin. IEEE Transactions on Software Engineering, 23(5):279–295, May 1997. 11. Simon S. Lam and A. Udaya Shankar. Protocol verification via projections. IEEE Transactions on Software Engineering, SE-10(4):325–342, July 1984. 12. Leslie Lamport. Proving the correctness of multiprocess programs. IEEE Transactions on Software Engineering, SE-3(2):125–143, March 1977. 13. Leslie Lamport. An assertional correctness proof of a distributed algorithm. Science of Computer Programming, 2(3):175–206, December 1982. 14. Leslie Lamport. How to write a long formula. Formal Aspects of Computing, 6:580– 584, 1994. First appeared as Research Report 119, Digital Equipment Corporation, Systems Research Center. 15. Leslie Lamport. The temporal logic of actions. ACM Transactions on Programming Languages and Systems, 16(3):872–923, May 1994.
High-Level Specifications: Lessons from Industry
261
16. Leslie Lamport. Composition: A way to make proofs harder. In Willem-Paul de Roever, Hans Langmaack, and Amir Pnueli, editors, Compositionality: The Significant Difference (Proceedings of the COMPOS’97 Symposium), volume 1536 of Lecture Notes in Computer Science, pages 402–423. Springer-Verlag, 1998. 17. Leslie Lamport. Specifying Systems. Addison-Wesley, Boston, 2002. A link to an electronic copy can be found at http://lamport.org. 18. Leslie Lamport, John Matthews, Mark Tuttle, and Yuan Yu. Specifying and verifying systems with TLA+ . In Proceedings of the Tenth ACM SIGOPS European Workshop, pages 45–48, Saint-Emilion, France, September 2002. INRIA (Institut National de Recherche en Informatique et en Automatique). 19. Leslie Lamport and Lawrence C. Paulson. Should your specification language be typed? ACM Transactions on Programming Languages and Systems, 21(3):502– 526, May 1999. 20. Leslie Lamport, Madhu Sharma, Mark Tuttle, and Yuan Yu. The wildfire verification challenge problem. At URL http://research.microsoft.com/users/ lamport/tla/wildfire-challenge.html on the World Wide Web. It can also be found by searching the Web for the 24-letter string wildfirechallengeproblem. 21. Susan Owicki and David Gries. Verifying properties of parallel programs: An axiomatic approach. Communications of the ACM, 19(5):279–284, May 1976. 22. Susan Owicki and Leslie Lamport. Proving liveness properties of concurrent programs. ACM Transactions on Programming Languages and Systems, 4(3):455–495, July 1982. 23. Sam Owre, John Rushby, Natarajan Shankar, and Friedrich von Henke. Formal verification for fault-tolerant architectures: Prolegomena to the design of PVS. IEEE Transactions on Software Engineering, 21(2):107–125, February 1995. 24. Amir Pnueli. The temporal logic of programs. In Proceedings of the 18th Annual Symposium on the Foundations of Computer Science, pages 46–57. IEEE, November 1977. 25. Serdar Tasiran, Yuan Yu, Brannot Batson, and Scott Kreider. Using formal specifications to monitor and guide simulation: Verifying the cache coherence engine of the Alpha 21364 microprocessor. In In Proceedings of the 3rd IEEE Workshop on Microprocessor Test and Verification, Common Challenges and Solutions. IEEE Computer Society, 2002. 26. Yuan Yu, Panagiotis Manolios, and Leslie Lamport. Model checking TLA+ specifications. In Laurence Pierre and Thomas Kropf, editors, Correct Hardware Design and Verification Methods, volume 1703 of Lecture Notes in Computer Science, pages 54–66, Berlin, Heidelberg, New York, September 1999. Springer-Verlag. 10th IFIP wg 10.5 Advanced Research Working Conference, CHARME ’99.
How the Design of JML Accommodates Both Runtime Assertion Checking and Formal Verification Gary T. Leavens1 , Yoonsik Cheon1 , Curtis Clifton1 , Clyde Ruby1 , and David R. Cok2 1
Department of Computer Science, Iowa State University 226 Atanasoff Hall, Ames, Iowa 50011-1041 USA {leavens,cheon,cclifton,ruby}@cs.iastate.edu phone: +1 515 294 1580, fax: +1 515 294 1580 2 Eastman Kodak Company Research & Development Laboratories 1700 Dewey Avenue, Building 65, Rochester, New York 14650-1816 USA [email protected] phone: +1 585 588 3107, fax: +1 585 588 3269
Abstract. Specifications that are used in detailed design and in the documentation of existing code are primarily written and read by programmers. However, most formal specification languages either make heavy use of symbolic mathematical operators, which discourages use by programmers, or limit assertions to expressions of the underlying programming language, which makes it difficult to write complete specifications. Moreover, using assertions that are expressions in the underlying programming language can cause problems both in runtime assertion checking and in formal verification, because such expressions can potentially contain side effects. The Java Modeling Language, JML, avoids these problems. It uses a side-effect free subset of Java’s expressions to which are added a few mathematical operators (such as the quantifiers \forall and \exists). JML also hides mathematical abstractions, such as sets and sequences, within a library of Java classes. The goal is to allow JML to serve as a common notation for both formal verification and runtime assertion checking; this gives users the benefit of several tools without the cost of changing notations.
1
Introduction
The Java Modeling Language, JML [55, 54], is the result of a cooperative, international effort aimed at providing a common notation and semantics for the specification of Java code at the detailed-design level [58]. JML is being designed cooperatively so that many different tools can use a common notation for Hoarestyle behavioral interface specifications. In this paper we explain the features of JML’s design that make its assertions easily understandable by programmers and suitable for both runtime assertion checking and formal verification. F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 262–284, 2003. c Springer-Verlag Berlin Heidelberg 2003
How the Design of JML Accommodates
1.1
263
Background
By a Hoare-style specification we mean one that uses pre- and postconditions to specify the behavior of methods [34, 43, 44]. A behavioral interface specification language (BISL) is a specification language that specifies both the syntactic interface of a module and its behavior [33, 48, 52, 85]. JML, the interface specification languages in the Larch family [33,48,52,85] and RESOLVE/C++ [22,73] are BISLs. Most design by contract languages and tools, such as Eiffel [70, 71] and APP [77], are also BISLs, because they place specifications inside programming language code. By contrast, neither Z [80, 79, 87] nor VDM [6, 27, 74, 43] is a BISL; they have no way to specify interface details for a particular programming language. OCL [82, 83] is a BISL for the UML, but the UML itself is language-independent; this poses problems for a Java programmer, because the UML does not have standard notations for all details of Java method signatures. For example, the UML’s syntax for specifying the signatures of operations has no standard notation for declaring that a Java method is strictfp or for declaring the exceptions that a method may throw [7, pp. 128-129] [49, p. 516]1 . Also the OCL has no standard constraints that correspond to JML’s exceptional postconditions. Because BISLs like JML specify both interface and behavior, they are good at specifying detailed designs that include such Java details. This makes JML well suited to the task of documenting reusable components, libraries, and frameworks written in Java. 1.2
Tool Support
Because BISLs are easily integrated with code, they lend themselves to tool support for activities related to detailed design, coding, testing, and maintenance. An important goal of JML is to enable a wide spectrum of such tools. Besides tools that enforce JML’s semantics (e.g., type checking), the most important JML tools help with the following tasks. Runtime checking and testing. The Iowa State group provides (from www. jmlspecs.org): – the jmlc runtime assertion checking compiler [13], which generates class files from JML-annotated Java sources2 , and – the jmlunit tool [14], which uses the runtime assertion checker to generate test oracle code for JUnit tests. Documentation. David Cok provides the jmldoc tool, also available through www.jmlspecs.org, which generates HTML documentation similar to that 1
2
Larman notes that the UML has some nonstandard ways to specify the exceptions that a method may throw, by either using Java’s own syntax directly or by using a “property string”. Besides this runtime assertion checking work at Iowa State, which relies on adding instrumentation to compiled code, Steven Edwards’s group at Virginia Tech is working on a wrapper-class based approach to runtime assertion checking that will allow instrumentation of programs for which source code is not available.
264
G.T. Leavens et al.
produced by javadoc [29], but containing specifications as well. The generated documentation is useful for browsing specifications or posting to the web. Static analysis and verification. The following tools are prepared by our partners at Compaq and the University of Nijmegen: – The ESC/Java tool [28,65,66] statically checks Java code for likely errors. ESC/Java understands a subset of JML annotations. – The LOOP tool [37,38,40,42] assists in the formal verification of the correctness of implementations from JML specifications, using the theorem prover PVS. In addition, the Daikon dynamic invariant detector [23,72] outputs invariants for Java programs in a subset of JML, and the Korat automated testing tool [8] uses the jmlunit tool to exercise the test data it derives. In this paper, we discuss how JML meets the needs of tools for runtime assertion checking, documentation, static analysis, and verification. We focus on runtime assertion checking and formal verification, which we consider to be the extremes of the spectrum of tools that a BISL might support. The tasks of runtime assertion checking and formal verification have widely differing needs: – Runtime assertion checking places a high premium on executability. Many specification languages intended for runtime assertion checking, such as Eiffel [70, 71] and APP [77], only allow assertions that are completely executable. This is sensible for a language that is intended only to support runtime assertion checking and not formal verification. – On the other hand, formal theorem proving and reasoning place a high premium on the use of standard mathematical notations. Thus, most specification languages intended for formal reasoning or verification, such as VDM, the members of the Larch family, and especially Z, feature a variety of symbolic mathematical notations. Many expressive mathematical notations, such as quantifiers, are impossible, in general, to execute at runtime. Again, including such notations is sensible for a language intended only to support formal theorem proving and reasoning and not runtime assertion checking. 1.3
Problems
We begin by describing the problems that arise when addressing the needs of the range of tools exemplified by runtime assertion checking and formal verification. 1.3.1 Notational Problem. It is often said that syntax does not matter; however, our experience with Larch/Smalltalk [11] and Larch/C++ [12, 50, 51, 53, 56] showed that programmers object to learning a specialized mathematical notation (the Larch Shared Language). This is similar to the problems found by Finney [26], who did a preliminary experiment demonstrating that the symbolic notation in Z specifications may make them hard to read. Conversely, in executable languages like Eiffel and APP, programmers feel comfortable with the use of the programming language’s expressions in assertions. Such an assertion
How the Design of JML Accommodates
265
language is therefore more appealing for purposes of documentation than highly symbolic mathematical notations. To summarize, the first problem that we address in this paper is how to provide a good syntax for specification expressions. Specification expressions are the syntactic forms that are used to denote values in assertions. By a good syntax we mean one that is close enough to programming language expressions that programmers feel comfortable with it and yet has all of the features necessary to support both runtime assertion checking and formal verification. 1.3.2 Undefinedness Problem. Expressions in a programming language may abruptly terminate (e.g., throw exceptions) and may go into infinite loops; consequently, they may have undefined values from a strictly mathematical point of view. Programming languages typically provide features to deal explicitly with such undefinedness. For example, Java provides short-circuit versions of boolean operators (such as && and ||) that allow programmers to suppress evaluation of some subexpressions. We want both programmers and mathematicians to use JML’s notations; hence, JML’s specification expressions should not only look like Java’s expressions and use Java’s semantics, but should also validate the standard laws of logic. However, because of a potential for undefinedness, Java expressions do not satisfy all the standard rules of logic; for example, in Java the conjunction E1 && E2 is not equal to E2 && E1 , although in logic they would be equal. To resolve this conflict, we are willing to accept a slightly different semantics for assertion evaluation as long as programmers are not too surprised by it. Thus, the second problem we address in this paper is how to find a semantics for expressions used in assertions that validates standard laws of logic and yet does not surprise programmers and is still useful for runtime assertion checking. 1.3.3 Side Effects Problem. Another important semantic issue is that expressions in a programming language like Java (and most others, including Eiffel) can contain side effects. Side effects have a very practical problem related to runtime assertion checking. It is generally assumed that assertions may be evaluated or skipped with no change in the outcome of a computation, but an assertion with side effects has the potential to alter the computation’s outcome. For example, an assertion with side effects might mask the presence of a bug that would otherwise be revealed or cause bugs that are not otherwise present. Because one of the principal uses of runtime assertion checking is debugging and isolating bugs, it is unacceptable for side effects from assertion checking to alter the outcome of a computation. Thus, the third problem that we address in this paper is how to prevent side effects in assertions while still retaining as much of the syntax of normal programming language expressions as possible. 1.3.4 Mathematical Library Problem. Most specification languages come with a library of mathematical concepts such as sets and sequences. Such concepts are especially helpful in specifying collection types. For example, to specify
266
G.T. Leavens et al.
a Stack type, one would use a mathematical sequence to describe, abstractly, the states that a stack object may take [35]. VDM, OCL, Z, and the interface specification languages of the Larch family all have libraries of such mathematical concepts. They also are standard in theorem provers such as PVS. However, as discussed in Section 1.3.1, we want to limit the barriers that Java programmers must overcome to use JML. Thus, the fourth problem that we address in this paper is how to provide a library of mathematical concepts in a way that does not overwhelm programmers, and yet is useful for formal verification. 1.4
Other Goals of JML
In addition to providing solutions to the preceding four problems, the design of JML is guided and constrained by several other goals. One of the most important of these goals is to allow users to write specifications that document detailed designs of existing code. This motivates the choice of making JML a BISL, as described above. Moreover, we would like JML to be useful for documenting code regardless of whether it was designed according to any particular design method or discipline. This is important because the cost of specification is high enough that it is not always justified until one knows that the design and the code have stabilized enough to make the documentation potentially useful to other people. In general, JML’s design adheres to the goal of being able to document existing designs; however, there is one significant aspect of JML’s design that departs from this goal—JML imposes the specifications of supertypes on subtypes, a property termed specification inheritance, in order to achieve behavioral subtyping [19]. JML’s use of specification inheritance is justified by another of our goals: we want JML to support modular reasoning, that is, reasoning about the behavior of a compilation unit using just the specifications of the compilation units that it references (as opposed to the details of their implementations). Modular reasoning is important because without it, the difficulty of understanding an object-oriented program increases much more rapidly than the size of the program, and thus the benefits of the abstraction mechanisms in object-oriented languages are lost. Consequently, modular reasoning is also important for formal verification, because then the scope of the verification problem is limited. Specification inheritance, and the resulting behavioral subtyping, allows modular reasoning to be sound, by allowing one to reason based on the static types of references. Subsumption in Java allows a reference to a subtype object to be substituted for a supertype reference. The requirements of behavioral subtyping [2, 3, 19, 57, 59, 63, 69] guarantee that all such substituted objects will obey the specifications inherited from the static type of the reference [19, 60, 61]. Because of the benefits of modular reasoning to programmers and verifiers, we favor specification inheritance over the conflicting goal of being able to document existing designs that do not follow behavioral subtyping. In any case, it is possible to work around the requirements of behavioral subtyping for cases in which a subtype does not obey the inherited specifications of its supertype(s). One simply underspecifies each supertype enough to allow all of the subtypes that
How the Design of JML Accommodates
267
are desired [63, 69]. Note that this work-around does not involve changing the code or the design, but only the specification, so it does not interfere with the goal of documenting existing code. 1.5
Outline
The remainder of this paper is organized as follows. The next section discusses our solution to the notational problem described above. Having described the notation in general terms, Section 3 provides more background on JML. The subsequent three sections treat the remaining problems discussed above. The paper ends with a discussion of related work and some conclusions.
2
Solving the Notational Problem
To solve the notational problem described in Section 1.3.1, JML generally follows Eiffel, basing the syntax of specification expressions on Java’s expression syntax. However, because side effects are not desired in specification expressions, JML’s specification expressions do not include Java expressions that can cause obvious side effects, i.e., assignment expressions and Java’s increment and decrement operators (++ and --). Furthermore, to make JML suitable for formal verification efforts, JML includes a number of operators that are not present in Java [55, Section 3]. The syntax of these operators comes in two flavors: those that are symbolic and those that are textual. We did not want to introduce excess notation that would cause difficulties for programmers when reading specifications, so JML adds just five symbolic operators. Four of these are logical operators: forward and reverse implication, written ==> and <==, respectively, and logical equivalence and inequivalence, written <==> and <=!=>, respectively. The inclusion of symbols for logical operators is inspired by the calculational approach to formal methods [17, 20, 31]. The other symbolic operator is <:, which is used to compare types to see if they are in a subtype relationship [65]. All the other operators added to Java and available in JML’s specification expressions use a textual notation consisting of a backslash (\) followed by an English word or phrase. For example, the logical quantifiers in JML are written as \forall and \exists [55]. Besides these quantifiers, JML also has several other operators using this backslash syntax. One of the most important is \old(), which is used in method postconditions to indicate an expression whose value is taken from the pre-state of a method call. For example, \old(i-1) denotes the value of i-1 evaluated in the pre-state of a method call. This notation is borrowed from the old operator in Eiffel. Other JML expressions using the backslash syntax include \fresh(o), which says that o was not allocated in the pre-state of a method call, but is allocated (and not null) in the post-state, and \result, which denotes the normal result returned by a method.
268
G.T. Leavens et al.
The backslashes in the syntax of these operators serve a very important purpose—they prevent the rest of the operator’s name from being interpreted as a Java identifier. This allows JML to avoid reserving Java identifiers in specification expressions. For example, result can be used as a program variable and is distinguished from \result. This trick is useful in allowing JML to specify arbitrary Java programs. Indeed, because a goal of JML is to document existing code, it cannot add new reserved words to Java.
3
Background on JML
In this section we provide additional background on JML that will be useful in understanding our solutions to the remaining problems. 3.1
Semantics of Specification Expressions
Just as JML adopts much of Java’s expression syntax, it attempts to keep JML’s semantics similar to Java’s. In particular, the semantics of specification expressions is a reference semantics. That is, when the name of a variable or field is used in an expression, it denotes either a primitive value (such as an integer) or a reference to an object. References themselves are values in the semantics, which allows one to directly express aliasing or the lack of it. For example, the expression arg != fieldVal says that arg and fieldVal are not aliased. Java also allows one to compare the states of objects using the equals method. For example, in the postcondition of a clone method, one might write the following to say that the result returned by clone is a newly allocated object that has the same state as the receiver (this): \fresh(\result) && this.equals(\result); Note that the exact meaning of the equals method for a given type is left to the designer of that type, as in Java. Thus, if one only knows that o is an Object, it is hard to conclude much about x from o.equals(x). Because JML uses this reference semantics, specifiers must show the same care as Java programmers when choosing between the == and equals equality tests. And like Eiffel, but unlike Larch-style interface specification languages, JML does not need “state functions” to be applied to extract the value of an expression from a reference. Values are implicitly extracted as needed by methods and operators. Besides being easier for programmers, this lends some succinctness to the notation. Currently, JML adopts all of the Java semantics for integer arithmetic. Thus types such as int use two’s complement arithmetic and are finite. Although Java programmers are, in theory, aware of the nature of integer arithmetic, it seems that JML’s adoption of Java’s semantics causes some misunderstandings; for example, some published JML specifications are inconsistent because of this semantics [10]. Chalin has suggested adding new primitive value types for infinite precision arithmetic to JML; in particular, he suggests a type \bigint for infinite precision integers [9, 10]. He is currently implementing and experimenting with this idea.
How the Design of JML Accommodates
3.2
269
Method and Type Specifications
To explain JML’s semantics for method specifications, we use the example in Figure 1. JML uses special comments, called annotations, to hold the specification of behavior; these are added to the interface information contained in the Java code. A specifier writes these annotation comments by inserting an at-sign (@) following the usual characters that signify the start of a comment. In multi-line annotation comments, at-signs at the beginnings of lines are ignored. Figure 1 starts with a “model import” directive, which says that JML will consider all types in the named package, org.jmlspecs.models, to be imported for purposes of the specification. This allows the JML tools to find the type JMLObjectSequence (see the third line) in that package. The type JMLObjectSequence is used as the type of the model instance field, named absVal. In this declaration, the model keyword says that the field is not part of the Java code, but is used solely for purposes of specification. The instance keyword says that the field is imagined, for purposes of specification, to be a non-static field in every class that implements this interface3 . Following the declaration of the two model instance fields is an invariant. It says that the field absVal is never null. Following the invariant are the declarations and specifications of three methods. In JML, a method’s specifications are typically written, as they are in Figure 1, before the header of the method they specify. This makes the scope of the formal parameters of a method a bit strange, because it extends backward into the method’s specification. However, it works best with Java tools, which expect comments related to a method, such as javadoc comments, to precede the method’s header. Consider the specification of the first method, push. This shows the general form of a “normal behavior” specification case. A specification case includes a precondition, indicated by the keyword requires, and some other specification clauses. A specification case is satisfied if, whenever the precondition is satisfied, the other clauses are also satisfied. Additionally, in a normal behavior specification case, the method must not throw an exception when the precondition is satisfied. The specification case given for push includes, besides the requires clause, a frame axiom, introduced by the keyword assignable, and a normal postcondition, following the keyword ensures. As with specification languages in the Larch family, a precondition that is just true can be omitted. In the Larch family, an omitted frame axiom means “assignable \nothing;”, which is a very strong specification that says that the method has no side effects. Following a suggestion of Erik Poll, we decided that such a specification was too strong for a default. So in JML, an omitted frame axiom allows assignment to all locations. This agrees with most of the defaults for omitted clauses in JML, which impose no restrictions. JML also allows specifiers to write “exceptional behavior” specification cases, which say that, when the precondition is satisfied, the method must not return 3
Omitting instance makes fields static and final, which is Java’s default for fields declared in interfaces.
270
G.T. Leavens et al.
//@ model import org.jmlspecs.models.*; public interface Stack { //@ public model instance JMLObjectSequence absVal; //@ public instance invariant absVal != null; /*@ public normal_behavior @ requires true; @ assignable absVal; @ ensures absVal.equals(\old(absVal.insertFront(x))); void push(Object x);
@*/
/*@ public normal_behavior @ requires !absVal.isEmpty(); @ assignable absVal; @ ensures absVal.equals(\old(absVal.trailer())) @ && \result == \old(absVal.first()); @ also @ public exceptional_behavior @ requires absVal.isEmpty(); @ assignable \nothing; @ signals (Exception e) e instanceof IllegalStateException; @*/ Object pop(); //@ ensures \result <==> absVal.isEmpty(); /*@ pure @*/ boolean isEmpty(); } Fig. 1. The specification and code for the interface Stack.
normally but must instead throw an exception. An example appears in the specification of the pop method. This specification has two specification cases connected with also. The meaning of the also is that the method must satisfy both of these specification cases [84, 86]. Thus, when the value of the model instance field absVal is not empty, a call to pop must return normally and must satisfy the given ensures clause. But when the value of the model instance field absVal is empty, a call to pop must throw an IllegalStateException. This kind of case analysis can be desugared into a single specification case, which can be given a semantics in the usual way [38, 41, 53, 76]. The specification cases given for push and pop are heavyweight specification cases [55, Section 1]. Such specification cases are useful when one wants to give a relatively complete specification, especially for purposes of formal verification. For runtime assertion checking or documentation, one may want to specify only part of the behavior of a method. This can be done using JML’s lightweight specification cases, which are indicated by the absence of a behavior keyword
How the Design of JML Accommodates
271
(like normal behavior). Figure 1 gives an example of a lightweight specification case in the specification of the method isEmpty.
4
Dealing with Undefinedness
As discussed in Section 1.3.2, a fundamental problem in using the underlying language for specification expressions is dealing with expressions whose value is undefined. In Java, undefinedness in expressions is typically signaled by the expression throwing an exception. For example, when one divides an integer by 0, the expression throws an ArithmeticException. Exceptions may also be thrown by methods called from within specification expressions. Specification languages have adopted several different approaches to dealing with undefinedness in expressions [4,32]. We wanted a semantics that would not be surprising to either Java programmers or to those doing formal verification. Typically, a Java programmer would try to write the specification in a way that “protects” the meaning of the expression against any source of undefinedness [62]. This can be accomplished by using the short-circuit boolean operators; for example, a specifier might write denom > 0 && num/denom > 1 to be sure that the division would be defined whenever it was carried out. However, we would like specifications to be meaningful even if they are not protective. Hence, the semantics of JML does not rely on the programmer writing protective specifications but, instead, ensures that every expression has some value. To do this, we adopted the “underspecified total functions” approach favored in the calculational style of formal methods [31,32]. That is, an expression that would not have a value in Java is given an arbitrary, but unspecified, value. For example, num/0 has some integer value, although this approach does not say what the value is, only that it must be uniformly substituted in any surrounding expression. In JML all expressions have an implicit argument of the program’s state; thus, the uniform substitution of values need only be carried out within a given assertion. An advantage of this substitution approach is that it validates the rules for standard logic. For example, in JML, E1 && E2 is equivalent to E2 && E1 . Consider what happens if E1 throws an exception; in that case, one may chose some unspecified boolean value for E1 , say b. This means that E1 && E2 equals b && E2 , which is equal to E2 && b, as can be seen by a simple case analysis on E2 ’s value. The case where E2 throws an exception is similar. Furthermore, if programmers write protective specifications, they will never be surprised by the details of this semantics. The JML assertion checking compiler takes advantage of the semantics of undefinedness to attempt, as much as possible, to detect possible assertion violations [13]. That is, assertion checking attempts to use a value that will make the overall assertion false, whenever the undefinedness of some subexpression allows it to do so. In this way, the assertion checker can both follow the rules of standard logic and detect places where specifications are not sufficiently protective. This is a good example of how JML caters to the needs of both runtime assertion checking and formal verification.
272
5
G.T. Leavens et al.
Preventing Side Effects in Assertions
As discussed in Section 1.3.3, it is important to prevent side effects in assertions, for both practical and theoretical reasons. JML is designed to prevent such side effects statically. It does this using an effect-checking type system [30, 81]. This type system is designed to be as simple as possible. Although it allows specification expressions to call Java methods and constructors, it only allows such calls if the called method or constructor is declared with the modifier pure. The semantics of JML must thus assure that pure methods and constructors are side-effect free. 5.1
JML’s Purity Restrictions
JML’s semantic restrictions on pure methods and constructors are as follows: – A pure method implicitly has a specification that includes the following specification case [55, Section 2.3.1]: assignable \nothing; This ensures that a correct implementation of the method has no side effects. – “A pure constructor implicitly has a specification that only allows it to assign to the instance fields of the class in which it appears” (including inherited instance fields) [55, Section 2.3.1]. This ensures that, if the constructor is correctly implemented, then a new expression that calls it has no side effects. – Pure methods and pure constructors may only invoke other methods and constructors that are pure. This makes the type system modular, as it allows the purity of a method or a constructor to be checked based only on its code and the specifications of the other methods and constructors that it calls. – All methods and constructors that override a pure method or constructor must also be pure. This inheritance of purity is a consequence of specification inheritance and is necessary to make the type system modular in the presence of subtyping. The first restriction implies that a pure method may not perform any input or output, nor may it assign to any non-local variables. Similarly, by the second restriction, a pure constructor may not do any I/O and may not assign to non-local storage other than the instance fields of the object the constructor is initializing. Note that, in JML, saying that a method may not assign to non-local storage means precisely that—even benevolent side effects are prohibited [55, Section 2.1.3.1]. This seems necessary for sound modular reasoning [64]. It is also a useful restriction for reasoning about supertypes from their specifications [78] and for reasoning about concurrent programs. The last two restrictions are also motivated by modularity considerations. Inheritance of purity has as a consequence that a method cannot be pure if any overriding method has side effects. In particular, a method in Object can be specified as pure only if every override of that method, in any Java class, obeys JML’s purity restrictions.
How the Design of JML Accommodates
273
The type system of JML is an important advance over languages like Eiffel, which trust programmers to avoid side effects in assertions rather than statically checking this property. However, as we will see in the following subsection, JML’s purity restrictions give rise to some practical problems. 5.2
Practical Problems with JML’s Purity Restrictions
An initial practical problem is how to decide which methods in Java’s libraries should be specified as pure. One way to start to answer this question is to use a static analysis to conservatively estimate which methods in Java’s libraries have side effects. A conservative analysis could count a method as having side effects if it assigns to non-local storage or calls native methods (which may do I/O), either directly or indirectly. All other methods can safely be specified as pure, provided they are not overridden by methods that the analysis says have side effects. Researchers from Purdue have provided a list of such methods to us, using their tools from the Open Virtual Machine project4 . We hope to integrate this technology into the JML tools eventually. Declaring a method to be pure entails a very strong specification, namely that the method and all possible overriding methods have no side effects. Thus, finding that a method, and all known methods that override it, obey JML’s purity restrictions is not the same as deciding that the method should be specified as pure. Such a decision affects not just all existing overrides of the method, but all future implementations and overrides. How is one to make such a decision? This problem is particularly vexing because there are many methods that seem intuitively to be side-effect free, but that do not obey JML’s purity restrictions. Methods with benevolent side effects are common examples. A benevolent side effect is a change in the internal state of an object in a way that is not externally visible. Two examples from the protocol of Object will illustrate the importance of this problem. First, consider computing a hash code for an instance of a class. Because this may be computationally costly, an implementation may desire to compute the hash code the first time it is asked for and then cache the result in a private field of the object. When the hash code is requested on subsequent occasions, the cached result is returned without further computation. For example, this is done in the hashCode method of Java’s String class. However, in JML, storing the computed hash code into the cache is considered to be a side effect. So String’s hashCode method cannot be specified as pure. Second, consider computing object equality. In some implementations, an object’s fields might be lazily initialized or computed only on first access. If the equals method happens to be the first such method to be called on such an object, it will trigger the delayed computation. We found such an example in our work on the MultiJava compiler [15, 16]; in this compiler, the class CClassType has such delayed computations, and its override of Object’s equals method can trigger a previously delayed computation with side effects. It seems very difficult to rewrite this method to be side-effect free, because to do so one would probably 4
See http://www.ovmj.org/.
274
G.T. Leavens et al.
need to change the compiler’s architecture. (Similar kinds of lazy initialization of fields occur in implementations of the Singleton pattern, although these usually do not affect the equals method.) We have shown two cases where methods in the protocol of Object are overridden by methods that cannot be pure. By purity and specification inheritance, these examples imply that neither hashCode nor equals can be specified as pure in Object. Object is typically used in Java as the type of the elements in a collection. Hence, in the specification of a collection type, such as a hash table, one cannot use the hashCode or equals methods on elements. Without changes, this would make JML unsuitable for specifying collection types. (This problem is mostly a problem for collection types, because one can specify many subclasses of Object with pure hashCode and equals methods. Specifications operating on instances of such subclasses can use these methods without violating JML’s type system.) 5.3
Solving the Problems
The desire to use intuitively side-effect free methods in specifications, even if they are not pure according to JML’s semantics, is strong enough that we considered changing the semantics of the assignable clause in order to allow benevolent side effects. However, we do not know how to do that and still retain sound modular reasoning [64]. In any case, the use of such methods in runtime assertion checking would still be problematic because of the side effects they might cause. In addition, we would like to prevent problems when a programmer wrongly believes that side effects are benevolent; it is not clear whether an automatic static analysis could prevent such problems, and even if so, whether such a tool could be modular. Thus far, the only viable solution we have identified is to refactor specifications by adding pure model (i.e., specification-only) methods that are to be used in specifications in place of program methods that cannot be pure. That is, whenever one has an intuitively side-effect free program method, m, that is not pure according to JML’s semantics, one should create a pure model method m , which returns the same result as m but without its side effects. Then one replaces calls to m by calls to m in assertions. We are currently experimenting with this solution. The most important part of this experiment is to replace uses of Object’s equals method, which cannot be pure, with calls to a new pure model method in Object, called isEqualTo. The specifications of these methods are shown in Figure 2. The assignable clause in the specification of the equals method permits benevolent side effects; it is also specified to return the same result as would a call to isEqualTo. Thus, whenever someone overrides equals, they should also override the isEqualTo method. When an override of equals is specified as pure, then an override of isEqualTo in the same class can be specified in terms of this pure equals method, and the implementation of the model isEqualTo method can simply call equals as well. However, an implementation of equals can never call isEqualTo, because program code cannot call model methods (since model methods can only be used in specifications). Therefore, to avoid code duplication when equals is not
How the Design of JML Accommodates
275
/*@ public normal_behavior @ assignable objectState; @ ensures \result <==> this.isEqualTo(obj); @*/ public boolean equals(Object obj); /*@ public normal_behavior @ requires obj != null; @ assignable \nothing; @ ensures (* \result is true when obj is equal to this object *); @ also @ public normal_behavior @ requires obj != null && \typeof(this) == \type(Object); @ assignable \nothing; @ ensures \result <==> this == obj; @ also @ public normal_behavior @ requires obj == null; @ assignable \nothing; @ ensures \result <==> false; public pure model boolean isEqualTo(Object obj) { return this == obj; } @*/ Fig. 2. The refactored specification for Object’s equals method and the pure model method isEqualTo. The text between (* and *) in the first specification case of isEqualTo’s specification is an “informal description”, which formally is equivalent to writing true [53].
declared to be pure but the two methods share some common implementation code, one can introduce a (non-model) pure, private method that both equals and isEqualTo can call. We have also applied this refactoring to all the collection classes in java.util (and in other packages) that we had previously specified, in order to check that the solution is viable. So far the results seem satisfactory. However, as of May 2003, this restructuring is not part of the JML release, because the JML tools are not yet able to handle some of the details of this approach. In particular, the runtime assertion checker is not yet able to compile the model methods added to Object without having all of Object’s source code available. (And we cannot legally ship Sun’s source code for Object in the JML release.) However, we working on solutions to this problem that will allow us to obtain more experience with this approach and to do more case studies.
276
5.4
G.T. Leavens et al.
Future Work on Synchronized Methods and Purity
JML currently permits synchronized methods to be declared pure if they meet all the criteria described in Section 5.1. Given that obtaining a lock is a side effect that can affect control flow in a program, does allowing synchronized methods to be pure violate the intent of JML’s purity restrictions? On the surface it would seem so, because when a synchronized method gains a lock, it may change the outcome of other concurrent threads. Furthermore, execution of such a method might block, conceivably even causing a deadlock between concurrent threads that would not occur if one was not doing assertion checking. However, since we have largely ignored concurrency thus far in JML’s design, we leave resolution of this issue for future work.
6
Mathematical Libraries
As described in Section 1.3.4, we need to provide a library of mathematical concepts with JML in a way that does not overwhelm programmers, and yet is useful for formal verification. 6.1
Hiding the Mathematics
It is sometimes convenient to use mathematical concepts such as sets and sequences in specification, particularly for collection classes [36, 68, 85]. For example, the specification of Stack in Figure 1 uses the type JMLObjectSequence, which is part of JML’s org.jmlspecs.models package. This package contains types that are intended for such mathematical modeling. Besides sequences, these include sets, bags, relations, and maps, and a few other convenience types. Most types in org.jmlspecs.models have only pure methods and constructors5 . For example, JMLObjectSequence’s insertFront method returns a sequence object that is like the receiver, but with its argument placed at the front; the receiver is not changed in any way. JMLObjectSequence’s trailer method similarly returns a sequence containing all but the first element of the receiver, without changing the receiver. Because such methods are pure, they can be used during runtime assertion checking without changing the underlying computation. JML gains two advantages from having these mathematical modeling types in a Java package, as opposed to having them be purely mathematical concepts. First, these types all have Java implementations and thus can be used during runtime assertion checking. Second, using these types in assertions avoids the introduction of special mathematical notation; instead, normal Java expressions (method calls) are used to do things like concatenating sequences or intersecting sets. This is an advantage for our main audience, which consists of programmers and not mathematicians. 5
The org.jmlspecs.models package does have some types that have non-pure methods. These are various kinds of iterators and enumerators. The methods of these iterators and enumerators that have side effects cannot be used in specification expressions.
How the Design of JML Accommodates
6.2
277
Use by Theorem Provers
The second part of the mathematical libraries problem described in Section 1.3.4 is that the library of mathematical modeling types should be useful for formal verification. The types in the org.jmlspecs.models package are intended to correspond (loosely) to the libraries of mathematical concepts found in theorem provers, such as PVS. As we gain experience, we can add additional methods to these types to improve their correspondence to these mathematical concepts. It is also possible to add new packages of such types tailored to specific theorem provers or to other notations, such as OCL. When translating specification expressions into theorem prover input, the Loop tool currently treats all methods in the same way — it does not make a special case for pure methods in the org.jmlspecs.models package. This makes the resulting proof obligations more complex than is desirable. Since the types in the models package are known, it seems that one should be able, as a special case, to replace the general semantics of such a method call with a call to some specific function from the theorem prover’s library of mathematical concepts. To facilitate this, it may be that these model types should all be declared to be final, which is currently not the case.
7
Related Work
We have already discussed how JML differs from conventional formal specification languages, such as Z [80, 79, 87], VDM [6, 27, 74, 43], the Larch family [33, 48, 52, 85] and RESOLVE [22, 73]. To summarize, the main difference is that JML’s specification expressions are based on a subset of the Java programming language, a design that is more congenial to Java programmers. The Alloy Annotation Language (AAL) offers a syntax similar to JML for annotating Java programs [46]. AAL supports extensive compile-time checking based on static analysis techniques. Unlike similar static analysis tools such as ESC/Java [18], AAL also supports method calls and relational expressions in assertions. However, AAL’s assertion language is based on a simple first-order logic with relational operators [39] and not on a subset of Java expressions. We believe that a Java-based syntax is more likely to gain acceptance among Java programmers. However, JML could adopt some of AAL’s features for specifying sets of objects using regular expressions. These would be helpful in using JML’s frame axioms, where they would allow JML to more precisely describe locations that can be assigned to in the method. (Another option that would have similar benefits would be to use the approach taken in DemeterJ [67].) We have also discussed how JML differs from design by contract languages, such as Eiffel [70, 71], and tools, such as APP [77]. Summarizing, JML provides better support for complete specifications and formal verification by – extending the set of specification expressions with more expressive mathematical constructs, such as quantifiers, – ensuring that specification expressions do not contain side effects, and – providing a library of types corresponding to mathematical concepts.
278
G.T. Leavens et al.
JML’s specification-only (model) declarations and frame axioms also contribute to its ability to specify types more completely than is easily done with design by contract tools. We know of several other design by contract tools for Java [5,21,24,45,47,75]. The approaches vary from a simple assertion mechanism similar to the assert macros of C and C++ to full-fledged contract enforcement capabilities. Jass [5], iContract [47], and JContract [75] focus on the practical use of design by contract in Java. Handshake and jContractor focus on implementation techniques such as library-based on-the-fly instrumentation of contracts [21, 45]. Contract Java focuses on properly blaming contract violations [24, 25]. These notations and tools suffer from the same problems as Eiffel. That is, none of them guarantee the lack of side effects in assertions, handle undefinedness in a way that would facilitate formal verification and reasoning, support more expressive mathematical notations such as quantifiers, or provide a set of immutable types designed for use in specifications. In sum, they all focus on runtime checking, and thus it is difficult to write complete specifications for formal verification and reasoning.
8
Conclusion
JML synthesizes the best from the worlds of design by contract and more mathematical specification languages. Because of its expressive mathematical notations, its specification-only (model) declarations, and library of mathematical modeling types, one can more easily write complete specifications in JML than in a design by contract language, such as Eiffel. These more complete specifications, along with JML’s purity checking, allow JML to be useful for formal verification. Thus, JML’s synthesis of features allows it to serve many roles in the Java formal methods community. Our experience so far is that this approach has had a modest impact. Release 3.7 of JML has been downloaded almost 400 times. JML has been used in at least 5 universities for teaching some aspects of formal methods. It is used somewhat extensively in the Java Smart Card industry and has been used in at least one company outside of that industry (Fulcrum). In the future, we would like to extend the range of tools that JML supports to include tools for model checking and specification of concurrent Java programs [1]. We invite others to join us in this effort to furnish Java programmers with a single notation that can be used by many tools.
Acknowledgments The work of Leavens, Cheon, Clifton, and Ruby was supported in part by the US National Science Foundation, under grants CCR-0097907 and CCR-0113181. Thanks to Robyn Lutz, Sharon Ryan, and Janet Leavens for comments on earlier drafts of this paper. Thanks to all who have contributed to the design and implementation of JML including Al Baker, Erik Poll, Bart Jacobs, Joe Kiniry, Rustan Leino, Raymie Stata, Michael Ernst, Gary Daugherty, Arnd PoetzschHeffter, Peter M¨ uller, and others acknowledged in [55].
How the Design of JML Accommodates
279
References 1. E. Abraham-Mumm, F.S. de Boer, W.P. de Roever, , and M. Steffen. A toolsupported proof system for mutlithreaded java. In Frank de Boer, Marcello Bonsangue, Susanne Graf, and Willem-Paul de Roever, editors, FMCO 2002: Formal Methods for Component Objects, Proceedings, Lecture Notes in Computer Science. Springer-Verlag, 2003. 2. Pierre America. Inheritance and subtyping in a parallel object-oriented language. In Jean Bezivin et al., editors, ECOOP ’87, European Conference on ObjectOriented Programming, Paris, France, pages 234–242, New York, NY, June 1987. Springer-Verlag. Lecture Notes in Computer Science, volume 276. 3. Pierre America. Designing an object-oriented programming language with behavioural subtyping. In J. W. de Bakker, W. P. de Roever, and G. Rozenberg, editors, Foundations of Object-Oriented Languages, REX School/Workshop, Noordwijkerhout, The Netherlands, May/June 1990, volume 489 of Lecture Notes in Computer Science, pages 60–90. Springer-Verlag, New York, NY, 1991. 4. H. Barringer, J. H. Cheng, and C. B. Jones. A logic covering undefinedness in program proofs. Acta Informatica, 21(3):251–269, October 1984. 5. D. Bartetzko, C. Fischer, M. Moller, and H. Wehrheim. Jass - Java with assertions. In Workshop on Runtime Verification held in conjunction with the 13th Conference on Computer Aided Verification, CAV’01, 2001. Published in Electronic Notes in Theoretical Computer Science, K. Havelund and G. Rosu (eds.), 55(2), 2001. Available from www.elsevier.nl. 6. Juan Bicarregui, John S. Fitgerald, Peter A. Lindsay, Richard Moore, and Brian Ritchie. Proof in VDM: A Practitioner’s Guide. Springer-Verlag, New York, NY, 1994. 7. Grady Booch, James Rumbaugh, and Ivar Jacobson. The Unified Modeling Language User Guide. Object Technology Series. Addison Wesley Longman, Reading, Mass., 1999. 8. Chandrasekhar Boyapati, Sarfraz Khurshid, and Darko Marinov. Korat: Automated testing based on Java predicates. In Proceedings International Symposium on Software Testing and Analysis (ISSTA), pages 123–133. ACM, July 2002. 9. Patrice Chalin. Back to basics: Language support and semantics of basic infinite integer types in JML and Larch. Technical Report CU-CS 2002-003.1, Computer Science Department, Concordia University, October 2002. 10. Patrice Chalin. Improving JML: For a safer and more effective language. Technical Report 2003-001.1, Computer Science Department, Concordia University, March 2003. 11. Yoonsik Cheon and Gary T. Leavens. The Larch/Smalltalk interface specification language. ACM Transactions on Software Engineering and Methodology, 3(3):221– 253, July 1994. 12. Yoonsik Cheon and Gary T. Leavens. A quick overview of Larch/C++. Journal of Object-Oriented Programming, 7(6):39–49, October 1994. 13. Yoonsik Cheon and Gary T. Leavens. A runtime assertion checker for the Java Modeling Language (JML). In Hamid R. Arabnia and Youngsong Mun, editors, Proceedings of the International Conference on Software Engineering Research and Practice (SERP ’02), Las Vegas, Nevada, USA, June 24-27, 2002, pages 322–328. CSREA Press, June 2002.
280
G.T. Leavens et al.
14. Yoonsik Cheon and Gary T. Leavens. A simple and practical approach to unit testing: The JML and JUnit way. In Boris Magnusson, editor, ECOOP 2002 — ObjectOriented Programming, 16th European Conference, M´ aalaga, Spain, Proceedings, volume 2374 of Lecture Notes in Computer Science, pages 231–255, Berlin, June 2002. Springer-Verlag. 15. Curtis Clifton. MultiJava: Design, implementation, and evaluation of a Javacompatible language supporting modular open classes and symmetric multiple dispatch. Technical Report 01-10, Department of Computer Science, Iowa State University, Ames, Iowa, 50011, November 2001. Available from www.multijava.org. 16. Curtis Clifton, Gary T. Leavens, Craig Chambers, and Todd Millstein. MultiJava: Modular open classes and symmetric multiple dispatch for Java. In OOPSLA 2000 Conference on Object-Oriented Programming, Systems, Languages, and Applications, volume 35(10) of ACM SIGPLAN Notices, pages 130–145, New York, October 2000. ACM. 17. Edward Cohen. Programming in the 1990s: An Introduction to the Calculation of Programs. Springer-Verlag, New York, NY, 1990. 18. David L. Detlefs, K. Rustan M. Leino, Greg Nelson, and James B. Saxe. Extended static checking. SRC Research Report 159, Compaq Systems Research Center, 130 Lytton Ave., Palo Alto, Dec 1998. 19. Krishna Kishore Dhara and Gary T. Leavens. Forcing behavioral subtyping through specification inheritance. In Proceedings of the 18th International Conference on Software Engineering, Berlin, Germany, pages 258–267. IEEE Computer Society Press, March 1996. A corrected version is Iowa State University, Dept. of Computer Science TR #95-20c. 20. Edsger W. Dijkstra and Carel S. Scholten. Predicate Calculus and program semantics. Springer-Verlag, NY, 1990. 21. Andrew Duncan and Urs Holzle. Adding contracts to Java with Handshake. Technical Report TRCS98-32, Department of Computer Science, University of California, Santa Barbara, CA, December 1998. 22. Stephen H. Edwards, Wayne D. Heym, Timothy J. Long, Murali Sitaraman, and Bruce W. Weide. Part II: Specifying components in RESOLVE. ACM SIGSOFT Software Engineering Notes, 19(4):29–39, Oct 1994. 23. Michael Ernst, Jake Cockrell, William G. Griswold, and David Notkin. Dynamically discovering likely program invariants to support program evolution. IEEE Transactions on Software Engineering, 27(2):1–25, February 2001. 24. Robert Bruce Findler and Matthias Felleisen. Contract soundness for objectoriented languages. In OOPSLA ’01 Conference Proceedings, Object-Oriented Programming, Systems, Languages, and Applications, October 14-18, 2001, Tampa Bay, Florida, USA, pages 1–15, October 2001. 25. Robert Bruce Findler, Mario Latendresse, and Matthias Felleisen. Behavioral contracts and behavioral subtyping. In Proceedings of Joint 8th European Software Engineering Conference (ESEC) and 9th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE), September 10-14, 2001, Vienna, Austria, September 2001. 26. Kate Finney. Mathematical notation in formal specification: Too difficult for the masses? IEEE Transactions on Software Engineering, 22(2):158–159, February 1996. 27. John Fitzgerald and Peter Gorm Larsen. Modelling Systems: Practical Tools in Software Development. Cambridge, Cambridge, UK, 1998.
How the Design of JML Accommodates
281
28. Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge, Greg Nelson, James B. Saxe, and Raymie Stata. Extended static checking for Java. In Cindy Norris and Jr. James B. Fenwick, editors, Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation (PLDI-02), volume 37, 5 of SIGPLAN, pages 234–245, New York, June 17–19 2002. ACM Press. 29. Lisa Friendly. The design of distributed hyperlinked programming documentation. In S. Fra¨iss`e, F. Garzotto, T. Isakowitz, J. Nanard, and M. Nanard, editors, Proceedings of the International Workshop on Hypermedia Design (IWHD’95), Montpellier, France, 1–2 June 1995, pages 151–173. Springer, 1995. 30. David K. Gifford and John M. Lucassen. Integrating functional and imperative programming. In ACM Conference on LISP and Functional Programming, pages 28–38. ACM, August 1986. 31. David Gries and Fred B. Schneider. A Logical Approach to Discrete Math. Texts and Monographs in Computer Science. Springer-Verlag, New York, NY, 1994. 32. David Gries and Fred B. Schneider. Avoiding the undefined by underspecification. In Jan van Leeuwen, editor, Computer Science Today: Recent Trends and Developments, number 1000 in Lecture Notes in Computer Science, pages 366–373. Springer-Verlag, New York, NY, 1995. 33. John V. Guttag, James J. Horning, S.J. Garland, K.D. Jones, A. Modet, and J.M. Wing. Larch: Languages and Tools for Formal Specification. Springer-Verlag, New York, NY, 1993. 34. C. A. R. Hoare. An axiomatic basis for computer programming. Communications of the ACM, 12(10):576–583, October 1969. 35. C. A. R. Hoare. Notes on data structuring. In E. Dijkstra Ole-J. Dahl and C. A. R. Hoare, editors, Structured Programming, pages 83–174. Academic Press, Inc., New York, NY, 1972. 36. C. A. R. Hoare. Proof of correctness of data representations. Acta Informatica, 1(4):271–281, 1972. 37. Marieke Huisman. Reasoning about Java Programs in higher order logic with PVS and Isabelle. Ipa dissertation series, 2001-03, University of Nijmegen, Holland, February 2001. 38. Marieke Huisman and Bart Jacobs. Java program verification via a Hoare logic with abrupt termination. In T. Maibaum, editor, Fundamental Approaches to Software Engineering (FASE 2000), volume 1783 of LNCS, pages 284–303. Springer-Verlag, 2000. An earlier version is technical report CSI-R9912. 39. Daniel Jackson. Alloy: A lightweight object modeling notation. ACM Transactions on Software Engineering and Methodology, 11(2):256–290, April 2002. 40. Bart Jacobs, Joseph Kiniry, and M. Warnier. Java program verification challenges. In Frank de Boer, Marcello Bonsangue, Susanne Graf, and Willem-Paul de Roever, editors, FMCO 2002: Formal Methods for Component Objects, Proceedings, Lecture Notes in Computer Science. Springer-Verlag, 2003. 41. Bart Jacobs and Eric Poll. A logic for the Java modeling language JML. In Fundamental Approaches to Software Engineering (FASE’2001), Genova, Italy, 2001, volume 2029 of Lecture Notes in Computer Science, pages 284–299. SpringerVerlag, 2001. 42. Bart Jacobs, Joachim van den Berg, Marieke Huisman, Martijn van Berkum, Ulrich Hensel, and Hendrik Tews. Reasoning about Java classes (preliminary report). In OOPSLA ’98 Conference Proceedings, volume 33(10) of ACM SIGPLAN Notices, pages 329–340. ACM, October 1998. 43. Cliff B. Jones. Systematic Software Development Using VDM. International Series in Computer Science. Prentice Hall, Englewood Cliffs, N.J., second edition, 1990.
282
G.T. Leavens et al.
44. H. B. M. Jonkers. Upgrading the pre- and postcondition technique. In S. Prehn and W. J. Toetenel, editors, VDM ’91 Formal Software Development Methods 4th International Symposium of VDM Europe Noordwijkerhout, The Netherlands, Volume 1: Conference Contributions, volume 551 of Lecture Notes in Computer Science, pages 428–456. Springer-Verlag, New York, NY, October 1991. 45. Murat Karaorman, Urs Holzle, and John Bruno. jContractor: A reflective Java library to support design by contract. In Pierre Cointe, editor, Meta-Level Architectures and Reflection, Second International Conference on Reflection ’99, SaintMalo, France, July 19–21, 1999, Proceedings, volume 1616 of Lecture Notes in Computer Science, pages 175–196. Springer-Verlag, July 1999. 46. Sarfraz Khurshid, Darko Marinov, and Daniel Jackson. An analyzable annotation language. In Proceedings of OOPSLA ’02 Conference on Object-Oriented Programming, Languages, Systems, and Applications, volume 37(11) of SIGPLAN Notices, pages 231–245, New York, NY, November 2002. ACM. 47. Reto Kramer. iContract – the Java design by contract tool. TOOLS 26: Technology of Object-Oriented Languages and Systems, Los Alamitos, California, pages 295– 307, 1998. 48. Leslie Lamport. A simple approach to specifying concurrent systems. Communications of the ACM, 32(1):32–45, January 1989. 49. Craig Larman. Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and the Unified Process. Prentice Hall PTR, Upper Saddle River, NJ, second edition edition, 2002. 50. Gary T. Leavens. An overview of Larch/C++: Behavioral specifications for C++ modules. In Haim Kilov and William Harvey, editors, Specification of Behavioral Semantics in Object-Oriented Information Modeling, chapter 8, pages 121–142. Kluwer Academic Publishers, Boston, 1996. An extended version is TR #96-01d, Department of Computer Science, Iowa State University, Ames, Iowa, 50011. 51. Gary T. Leavens. Larch/C++ Reference Manual. Version 5.41. Available in ftp: //ftp.cs.iastate.edu/pub/larchc++/lcpp.ps.gz or on the World Wide Web at the URL http://www.cs.iastate.edu/˜leavens/larchc++.html, April 1999. 52. Gary T. Leavens. Larch frequently asked questions. Version 1.110. Available in http://www.cs.iastate.edu/˜leavens/larch-faq.html, May 2000. 53. Gary T. Leavens and Albert L. Baker. Enhancing the pre- and postcondition technique for more expressive specifications. In Jeannette M. Wing, Jim Woodcock, and Jim Davies, editors, FM’99 — Formal Methods: World Congress on Formal Methods in the Development of Computing Systems, Toulouse, France, September 1999, Proceedings, volume 1709 of Lecture Notes in Computer Science, pages 1087– 1106. Springer-Verlag, 1999. 54. Gary T. Leavens, Albert L. Baker, and Clyde Ruby. JML: A notation for detailed design. In Haim Kilov, Bernhard Rumpe, and Ian Simmonds, editors, Behavioral Specifications of Businesses and Systems, pages 175–188. Kluwer Academic Publishers, Boston, 1999. 55. Gary T. Leavens, Albert L. Baker, and Clyde Ruby. Preliminary design of JML: A behavioral interface specification language for Java. Technical Report 98-06v, Iowa State University, Department of Computer Science, May 2003. See www.jmlspecs.org. 56. Gary T. Leavens and Yoonsik Cheon. Preliminary design of Larch/C++. In U. Martin and J. Wing, editors, Proceedings of the First International Workshop on Larch, July, 1992, Workshops in Computing, pages 159–184. Springer-Verlag, New York, NY, 1993.
How the Design of JML Accommodates
283
57. Gary T. Leavens and Krishna Kishore Dhara. Concepts of behavioral subtyping and a sketch of their extension to component-based systems. In Gary T. Leavens and Murali Sitaraman, editors, Foundations of Component-Based Systems, chapter 6, pages 113–135. Cambridge University Press, 2000. 58. Gary T. Leavens, K. Rustan M. Leino, Erik Poll, Clyde Ruby, and Bart Jacobs. JML: notations and tools supporting detailed design in Java. In OOPSLA 2000 Companion, Minneapolis, Minnesota, pages 105–106. ACM, October 2000. 59. Gary T. Leavens and Don Pigozzi. A complete algebraic characterization of behavioral subtyping. Acta Informatica, 36:617–663, 2000. 60. Gary T. Leavens and William E. Weihl. Reasoning about object-oriented programs that use subtypes (extended abstract). In N. Meyrowitz, editor, OOPSLA ECOOP ’90 Proceedings, volume 25(10) of ACM SIGPLAN Notices, pages 212–223. ACM, October 1990. 61. Gary T. Leavens and William E. Weihl. Specification and verification of objectoriented programs using supertype abstraction. Acta Informatica, 32(8):705–778, November 1995. 62. Gary T. Leavens and Jeannette M. Wing. Protective interface specifications. Formal Aspects of Computing, 10:59–75, 1998. 63. Gary Todd Leavens. Verifying object-oriented programs that use subtypes. Technical Report 439, Massachusetts Institute of Technology, Laboratory for Computer Science, February 1989. The author’s Ph.D. thesis. 64. K. Rustan M. Leino. A myth in the modular specification of programs. Technical Report KRML 63, Digital Equipment Corporation, Systems Research Center, 130 Lytton Avenue Palo Alto, CA 94301, November 1995. Obtain from the author, at [email protected]. 65. K. Rustan M. Leino, Greg Nelson, and James B. Saxe. ESC/Java user’s manual. Technical note, Compaq Systems Research Center, October 2000. 66. K. Rustan M. Leino, James B. Saxe, and Raymie Stata. Checking Java programs via guarded commands. Technical Note 1999-002, Compaq Systems Research Center, Palo Alto, CA, May 1999. 67. Karl Lieberherr, Doug Orleans, and Johan Ovlinger. Aspect-oriented programming with adaptive methods. Communications of the ACM, 44(10):39–41, October 2001. 68. Barbara Liskov and John Guttag. Abstraction and Specification in Program Development. The MIT Press, Cambridge, Mass., 1986. 69. Barbara Liskov and Jeannette Wing. A behavioral notion of subtyping. ACM Transactions on Programming Languages and Systems, 16(6):1811–1841, November 1994. 70. Bertrand Meyer. Eiffel: The Language. Object-Oriented Series. Prentice Hall, New York, NY, 1992. 71. Bertrand Meyer. Object-oriented Software Construction. Prentice Hall, New York, NY, second edition, 1997. 72. Jeremy W. Nimmer and Michael D. Ernst. Static verification of dynamically detected program invariants: Integrating Daikon and ESC/Java. In Proceedings of RV’01, First Workshop on Runtime Verification. Elsevier, July 2001. To appear in Electronic Notes in Theoretical Computer Science. 73. William F. Ogden, Murali Sitaraman, Bruce W. Weide, and Stuart H. Zweben. Part I: The RESOLVE framework and discipline — a research synopsis. ACM SIGSOFT Software Engineering Notes, 19(4):23–28, October 1994.
284
G.T. Leavens et al.
74. International Standards Organization. Information technology – programming languages, their environments and system software interfaces – Vienna Development Method – specification language – part 1: Base language. ISO/IEC 13817-1, December 1996. 75. Parasoft Corporation. Using design by contractTM to automate JavaTM software and component testing. Available from http://www.parasoft.com/jsp/ products/tech papers.jsp?product=Jcontract, as of Feb. 2003. 76. Arun D. Raghavan and Gary T. Leavens. Desugaring JML method specifications. Technical Report 00-03c, Iowa State University, Department of Computer Science, August 2001. 77. D. S. Rosenblum. Towards a method of programming with assertions. In Proceedings of the 14th International Conference on Software Engineering, pages 92–104, May 1992. 78. Clyde Ruby and Gary T. Leavens. Safely creating correct subclasses without seeing superclass code. In OOPSLA 2000 Conference on Object-Oriented Programming, Systems, Languages, and Applications, Minneapolis, Minnesota, volume 35(10) of ACM SIGPLAN Notices, pages 208–228, October 2000. 79. J. Spivey. An introduction to Z and formal specifications. Software Engineering Journal, January 1989. 80. J. Michael Spivey. The Z Notation: A Reference Manual. International Series in Computer Science. Prentice-Hall, New York, NY, 1989. ISBN 013983768X. 81. Jean-Pierre Talpin and Pierre Jouvelot. The type and effect discipline. Information and Computation, 111(2):245–296, June 1994. 82. Jos Warmer and Anneke Kleppe. The Object Constraint Language: Precise Modeling with UML. Addison Wesley Longman, Reading, Mass., 1999. 83. Jos Warmer and Anneke Kleppe. OCL: The constraint language of the UML. Journal of Object-Oriented Programming, 12(1):10–13,28, March 1999. 84. Alan Wills. Capsules and types in Fresco: Program validation in Smalltalk. In P. America, editor, ECOOP ’91: European Conference on Object Oriented Programming, volume 512 of Lecture Notes in Computer Science, pages 59–76. Springer-Verlag, New York, NY, 1991. 85. Jeannette M. Wing. Writing Larch interface language specifications. ACM Transactions on Programming Languages and Systems, 9(1):1–24, January 1987. 86. Jeannette Marie Wing. A two-tiered approach to specifying programs. Technical Report TR-299, Massachusetts Institute of Technology, Laboratory for Computer Science, 1983. 87. Jim Woodcock and Jim Davies. Using Z: Specification, Refinement, and Proof. Prentice Hall International Series in Computer Science, 1996.
Finding Implicit Contracts in .NET Components Karine Arnout1 and Bertran Meyer1,2 1 Chair
of Software Engineering, Swiss Federal Institute of Technology (ETH) CH-8092 Zurich, Switzerland 2 Eiffel Software, 356 Storke Road, Santa Barbara CA 93117, USA [email protected] http://se.inf.ethz.ch, http://www.eiffel.com
Abstract. Are contracts inherent in reusable libraries, or just one design technique among others? To help answer this question, we performed an empirical study of library classes from the .NET Collections library, which doesn’t use Design by Contract™, to look for unexpressed contracts. This article reports on the buried contracts we have found, and discusses improvements to the architecture – especially to the libraries’ ease of learning and ease of use – that may result from making the contracts explicit. It extends previous reports [3,4,5,6] with an analysis of the benefits of an a posteriori addition of contracts for the library users. Keywords: Design by Contract™, Library design, Reuse, Implicit contracts, .NET, Metadata, Contract Wizard, Eiffel.
1 Introduction Equipping libraries with contracts has become a second nature to designers working with Eiffel. Many commonly used libraries, however, don’t show any contracts at all. The resulting style is very different, and, to someone used to Design by Contract [21, 23,25,31], deficient. Because the benefits of contracts are so clear to those who use them, it’s natural to suspect that non-Eiffel programmers omit contracts because they have no good way to express them, or haven’t even been taught the concepts, but that conceptually contracts are there all the same: that inside every contract-less specification there is a contract wildly signaling to be let out. For an Eiffel programmer this is the natural interpretation. But when you are doing something different from the rest of the world, it’s good to check your own sanity. Are we wrong in seeing contracts around libraries, and the rest of the world – including the most recent general-purpose development frameworks – right in continuing to act as if contracts had never been invented? This article is such a sanity check. The basic conjecture that it explores may be stated more precisely: Resolving the Closet Contract Conjecture is interesting for several reasons: − The answer can shed light on important issues of reusable component design, one of the keys to progress in software engineering. − An answer can help library users (application programmers) choose between competing libraries. F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 285–318, 2003. © Springer-Verlag Berlin Heidelberg 2003
286
K. Arnout and B. Meyer Box 1. The Closet Contract Conjecture
Eiffel libraries have contracts; most others don’t. Which is the right explanation? − The contract-rich style of Eiffel libraries is but an artefact of the support for contracts in the Eiffel method, language and tools. Remove contract mechanisms, and the contracts just go away. − Contracts are inherent in library design; if not explicitly stated, as in C++/Java/.NET libraries, they are lurking anyway under the cover, either suppressed or replaced by comments in the program, explanations in the documentation, exceptions and other ersatz techniques. − On a more specific point, the answer would help ascertain the potential usefulness of a “Contract Wizard”. Such a tool, of which a first version has been implemented by Eiffel Software [1], takes advantage of the reflection facilities of .NET — “Metadata” — to let its users work interactively on compiled classes, coming from any non-contracted language such as C#, C++, Visual Basic, Cobol or Java, and add contracts to them a posteriori. But this is only interesting if the second answer holds for the Closet Contract Conjecture. If not, the Wizard’s user wouldn’t find any interesting contracts to add. − If that second answer indeed holds, we may use it to improve our understanding of the libraries, and even to improve the library themselves by turning the implicit contracts that we have elicited into explicit elements of the software. To help answer the conjecture, we have started a study of non-contracted libraries to see if we could spot implicit contracts. The .NET collection library [27], a comprehensive set of data structure implementations, has been one of the first targets. We examined some commonly used .NET collection classes, sleuthing around for hidden contracts, and trying to uncover language or documentation techniques used to make up for the absence of proper contract mechanisms such as precondition clauses, postcondition clauses and class invariants. Where we spotted closet contracts, we proceeded to out them, by producing class variants that retain the original APIs but make the contracts explicit. We compared the result both with the originals and with some of the classes’ closest counterparts in the EiffelBase library. The rest of this presentation describes the analysis and its outcomes: − Section 2 provides more details on why we have engaged in this study and explains the method of analysis. − Section 3 recalls the principles of Design by Contract and their application to library design. − Section 4 summarizes the contributions of .NET and its Collections library. − Section 5 presents the results of analyzing an important .NET collection class: ArrayList [28]. − Section 6 introduces a variant of ArrayList where the implicit contracts detected in the official version have been made explicit, and gives measurements of properties of the contracted class.
Finding Implicit Contracts in .NET Components
287
− Section 7 extends the analysis to some other classes and interfaces to assess the consistency of the results across the library. − Section 8 compares the effect of the two design styles, with and without contracts, on the development of applications using a library: ease of use, ease of learning, bug avoidance. − Section 9 presents related work about contract extraction and evaluates the possibility of automating the extraction of hidden preconditions by analyzing the CIL code. − Section 10 concludes with an assessment of the lessons learned. The focus of this work is on design and programming methodology, in particular design methodology for the construction of libraries of reusable components; we are looking for techniques that help component producers turn out better components, help component consumers learn to use the components, and reduce the potential for errors arising from incorrect, incomplete or misunderstood specifications. We were surprised, when presenting early versions, that some listeners were mostly interested in possibilities of extracting the contracts automatically from non-contracted components. Although section 9 indeed describes possibilities in this direction, building on earlier work on contract extraction, we must warn the reader not to expect any miracles; no computer tool can divine the intent behind a programmer’s work without some help from the programmer. The potential for automatic contract inference should not obscure the concrete and immediate benefits that good design methodology can bring to both the producers and consumers of reusable software.
2 The Context 2.1 A Distinctive Design Style Applying to reusable libraries the ideas of Design by Contract [21,23,25,31] means equipping each library class with precise specifications, or “contracts”, governing its interaction with the library's clients. Contracts include the class invariant, stating general consistency conditions to be maintained by every exported routine of the class, and, for every routine, preconditions stating the clients' obligations, and postconditions stating guarantees to the clients. Systematic application of these principles leads to a distinctive design style, immediately visible in Eiffel frameworks such as EiffelBase [10] [22] covering fundamental data structures and algorithms, EiffelVision for portable graphics, and others. These Eiffel libraries have been in wide use for many years and seemingly appreciated by their users as easy to learn, convenient to use, and beneficial to the reliability of applications built with them. A recent report by the Software Engineering Institute [37] confirms that for components in general — not just classes — the use of contracts appears to be a key condition of any effort to improve “composability” and scale up the application of component-based technology. Design by Contract as it has been applied to libraries so far, mostly in Eiffel, is not an a posteriori addition to the design of a library; it is an integral part of the design process. The resulting contract-rich library APIs are markedly different from more traditional, contract-less designs. One might argue that criticism of current libraries [38] becomes partly unjustified when libraries are built according to this style. The
288
K. Arnout and B. Meyer
difference is clear, for example, in a comparison of two libraries that cover some of the same ground: EiffelBase [10] [22], which is based on Design by Contract, and the .NET Collections library [27], which is not. Most non-Eiffel libraries, such as the .NET framework’s libraries, have indeed been built without explicit consideration to the notion of contract. Three possible explanations come to mind: − The library authors do not know about Design by Contract. − They know about the concepts, but don’t find them particularly useful. − They know about the concepts and find them interesting but too cumbersome to apply without built-in Eiffel-style support in the method, language and supporting tools. Regardless of the reason, the difference in styles is so stark that we must ask what happened, in these contract-less libraries, to the properties that the Eiffel designer would have expressed in preconditions, postconditions and class invariants. It’s this question that leads to the Closet Contract Conjecture: are the contracts of Eiffel libraries a figment of the Eiffel programmer’s obsession with this mechanism? Or are they present anyway, hidden, in non-Eiffel libraries as well? The only way to find out is to search contract-less libraries for closet contracts. In performing this search we have been rummaging through interface specifications, source code when available, documentation, even – since any detective knows not to overlook the household’s final output – generated code, which in .NET and Java still retains significant high-level information. 2.2 .NET Libraries and the Contract Wizard A property of the .NET libraries that makes them particularly interesting for such a study is the flexibility of the .NET component model, which has enabled the development of a “Contract Wizard” [1], a tool that enables a user to examine a compiled module (“assembly” in .NET), typically coming from a contract-less language such as C#, Visual Basic, C++, Cobol etc., and interactively add contracts to its classes and routines, producing a proxy assembly that is contracted as if it had been written in Eiffel, but calls the original. The Contract Wizard relies on the reflection capabilities provided in .NET by the metadata that every assembly includes, providing interface information such as the signature of each routine, retained from the source code in the compiling process. By nature, however, the Contract Wizard is only interesting if the Closet Contract Conjecture holds. This observation provides one of the incentives for the present study: as we consider further developments of the Contract Wizard, we must first gather empirical evidence confirming or denying its usefulness. If we found that .NET and other non-contracted libraries do very well without contracts, thank you very much, and that there are no useful closet contracts to be added, it would be a waste of time to continue working on the Contract Wizard. 2.3 Method of Work Our library analyses have so far not relied on any automatic tools. Because we are looking for something that officially isn’t there, we have to exercise our own interpretation to claim and authenticate our finds. It’s incumbent on us to state why we think a
Finding Implicit Contracts in .NET Components
289
particular class characteristic, such as an exception, is representative of an underlying contract. Having to rely on a manual extraction process puts a natural limit on future extensions of this article’s analysis to other libraries. Beyond facilitating the analysis, automated extraction tools could help users of the Contract Wizard by suggesting possible contract additions. The results of this article indeed suggest certain patterns, in code or documentation, that point to possible contracts, as certain geological patterns point to possible oil deposits. However, the final process of contract elicitation, starting from non-contracted libraries, requires subjective decisions.
3 Building Libraries with Design by Contract The ideas of Design by Contract are inspired by commercial relationships and business contracts, which formally express the rights and obligations binding a client and a supplier. Likewise, software contracts are a way to specify the roles and constraints applying to a class as a whole (class invariants) or to the routines of the class (preconditions and postconditions). 3.1 Why Use Contracts? Many programmers who have heard of contracts think they are just a way to help test and debug programs through conditionally compiled instructions of the form if not “Some condition I expect to hold here” “Scream”
then
end where “Scream” might involve triggering an exception, or stopping execution altogether. Such a use — similar to the “assert” of C — is only a small part of the application of contracts, and wouldn’t by itself justify special language constructs. Contracts address a wider range of issues in the software process, for general application development as well as library design: − Correctness: Contracts help build software right in the first place by avoiding bugs rather than correcting once they are there. Designing with contracts encourages the designer to think about the abstract properties of each software element, and build the observance of these properties into the software. − Documentation: From contracted software, automatic tools can extract documentation that is both abstract and precise. Because the information comes from the software text, this approach saves the effort of writing documentation as a separate product, and lowers the risk of divergence between software and documentation. It underlies the basic form of Eiffel documentation for Eiffel software: the contract form, produced by tools of the Eiffel environment and retaining interface information only. − Debugging and testing: Run-time monitoring of contracts permits a coherent, focused form of quality assurance based on verifying that the run-time state of the software satisfies the properties expected by the designers.
290
K. Arnout and B. Meyer
− Inheritance control: Design by Contract principles provide a coherent approach to inheritance, limiting the extent to which new routine definitions may affect the original semantics (preconditions may only be weakened, postconditions strengthened). − Management: Contracts allow project managers and decision makers to understand the global purpose of a program without going into the depth of the code. The principles are particularly relevant to library design. Eiffel libraries are thoroughly equipped with contracts stating their abstract properties, as relevant to clients. 3.2 Kinds of Contract Elements Contracts express the semantic specifications of classes and routines. They are made of assertions: boolean expressions stating individual semantic properties, such as the property, in a class representing lists stored in a container of bounded capacity, that the number count of elements in a list must not exceed the maximum permitted, capacity. Uses of contracts include: − Preconditions: Requirements under which a routine will function properly. A precondition is binding on clients (callers); the supplier (the routine) can turn it to its advantage to simplify its algorithm by assuming the precondition. − Postconditions: Properties guaranteed by the supplier to the client on routine exit. − Class invariants: Semantic constraints characterizing the integrity of instances of a class; they must be ensured by each constructor (creation procedure) and maintained by every exported routine. − Check instructions: “Assert”-like construct, often used on the client side, before a call, to check that a precondition is satisfied as expected. − Loop variants and invariants: Correctness conditions for a loop. Check instructions, loop variants and loop invariants address implementation correctness rather than properties of library interfaces and will not be considered further here. Although preconditions and postconditions are the best known forms of library contracts, class invariants are particularly important in an object-oriented context since they express fundamental properties of the abstract data type (ADT) underlying a class, and the correctness of the ADT implementation chosen for the class (representation invariant [9]). We must make sure that our contract elicitation process doesn’t overlook them. 3.3 Contracts in Libraries Even a very simple example shows the usefulness of contracts in library design. Consider a square root function specified, in a first approach, as sqrt (x: REAL): REAL This specification tells us that the function takes a REAL argument and returns a REAL result. That is already a form of contract, specifying the type signature of the
Finding Implicit Contracts in .NET Components
291
function. We can call it a signature contract. (Regrettably, some of the Java and .NET reference documentation uses the term “contract”, without qualification, for such signature contracts, creating confusion with the well established use of the term as used in the rest of this article.) A more complete contract — semantic contract if we need to distinguish it from mere signature contracts — should also specify properties of the argument and result that can’t just be captured by type information, but is just as important to the library client. The most obvious example is what happens for a negative argument, with at least four possible answers: − The function might silently return a default value, such as zero. (Not very good!) − It might return a default value, and set a special flag that it is the caller’s responsibility to examine after a call. − It might trigger an exception, which it is the caller’s responsibility to handle (otherwise execution will probably terminate abnormally). − It might produce aberrant behavior, such as entering an infinite loop or crashing the execution. (Not good in the absence of contracts.) A fifth would be to return a COMPLEX result, but that is not permitted by statically typed languages if the specification, as above, declares the type of the result as REAL. A contract — here a precondition and a postcondition — will express which of these specifications the function implements. In Eiffel the function would appear as sqrt (x: REAL): REAL is -- Mathematical square root of x, within epsilon require non_negative: x >= 0 do ... Square root algorithm here ... ensure good_approximation: abs (Result ^2 – x) <= 2 * x * epsilon end
where epsilon is some appropriate value expressing the requested precision, abs gives the absolute value, and ^ is the power operator. The assertion tags non_negative and good_approximation are there for documentation purposes and will also appear in error messages if the contracts are checked at run time during debugging and testing. The contract form of the enclosing class, as produced by the environment tools, will show the above without the do… part and the is keyword (but with the header comment). This is the basic documentation that any application programmer wishing to use an Eiffel class will receive. That documentation is, essentially, the set of contracts associated with the class. Here we find direct support for the contract clauses — require, ensure, and the yet to be encountered invariant — in the language and the associated documentation standard, but the contract is inherent to the routine, regardless of its language of implementation. If a library is to provide a usable square root routine it must be based on such a contract. Unless you know under what conditions a square root
292
K. Arnout and B. Meyer
function will operate and what properties you may expect of its result, you couldn’t use it properly. The general questions are: − If there is no explicit contract discipline, comparable to the Eiffel practice of documenting all libraries through their contract form, where will we find the implicit contract? − Will, as in this example, a contract always exist, whether expressed or not? The rest of this study provides material for answering these questions.
4 Why .NET Libraries? .NET libraries suggested themselves for our study not only because they are one of the most recent and widely publicized collections of general-purpose reusable components, but also because the innovative .NET concept of metadata equips them with specification information that appears directly useful to contract elicitation. 4.1 The Scope of .NET The .NET libraries are part of the .NET framework [24], a major Microsoft endeavor. Overall, .NET addresses the needs of companies and the general public through advances in Web services infrastructure, new tools for business-to-business and business-to-consumer interactions (Universal Description, Discovery and Integration, MyServices, Biztalk), advanced Web mechanisms (through the ASP.NET framework), new security mechanisms and other innovations. In support of these goals, .NET includes new techniques and products of direct interest to developers. The most significant advance here is extensive support for multilanguage programming with full interoperability between languages. Beyond Microsoft’s own languages, C# and Visual Basic .NET, implementations exist for thirdparty languages, in particular Eiffel. (The Eiffel implementation on .NET includes the full language as on other platforms; in particular it supports Design by Contract, genericity and multiple inheritance without restrictions [2] [36].) All such language implementations benefit from common mechanisms including a versioning scheme, an extensive security framework, a component model considerably simpler to use than Microsoft’s previous COM technology, facilities for memory management, debugging and exception handling, a multi-language development environment (Visual Studio .NET), all supported by a Common Language Runtime. They also benefit from a set of libraries covering many areas of applications, which the interoperability infrastructure makes available to programs written in all languages supported on .NET. 4.2 The Role of Metadata The basic compilation unit in .NET, covering for example a library or a section of a library, is the assembly. The key to the framework’s support for component-based development is the presence, in every assembly, of documentary information known as metadata, making the assembly self-describing in accordance with the selfdocumentation principle [23].
Finding Implicit Contracts in .NET Components
293
The metadata of an assembly — accessible to programs through the Reflection library, to users through various tools, and to the world at large in XML — provides information on the assembly’s contents, including: − A “manifest” describing the assembly name, version, culture and public key (if the assembly is signed). − The list of dependencies on other .NET assemblies. − For each class, the list of its parents (interfaces, and at most one class) and of its features (routines, attributes, properties and events). − For each class member, the signature (including arguments and return type). In addition to these predefined categories, developers can define, assuming proper source language support, their own specific kinds of metadata in the form of custom attributes. Metadata of both kinds – predefined and custom – opens attractive new possibilities. The Contract Wizard is one of them: by relying on the metadata, it enables users to examine a class and its features interactively, and add any appropriate contracts – all without having access to the source code.
5 Analysis of a Collection Class Our first search for closet contract will target the class ArrayList [28], part of the core .NET library (mscorlib.dll). This choice of class is almost arbitrary; in particular it was not based on any a priori guess that the class would suggest more (or fewer) contracts than any other. Rather, the informal criteria were that the class, describing lists implemented through arrays: − Is of obvious practical use. − Seems typical, in its style, of the Collections library. − Has a direct counterpart, ARRAYED_LIST, in the EiffelBase library, opening the possibility of comparisons once we’ve completed the contract elicitation process. 5.1 Implicit Class Invariants Documentation comments first reveal properties of ArrayList that fall into the category of class invariants. We find our first leads in the specification of class constructors, which states that “The default initial capacity for an ArrayList is 16” This comment implies that the capacity of the created object is greater than zero. Taking up this lead, we notice that all three constructors of ArrayList set the initial list’s capacity to a positive value. This suggests an invariant, since it is part of the Design by Contract rules that an invariant property must be guaranteed by all the creation procedures of a class. To test our intuition, we examine the other key property of an invariant: that it must be preserved by every exported routine of the class. Examining all such routines
294
K. Arnout and B. Meyer
confirms this and suggests that we indeed have the germ of an invariant, which in Eiffel would be expressed by the clause invariant positive_capacity: capacity >= 0
Continuing our exploration of the documentation, we note that two of the three constructors of ArrayList “initialize a new instance of the ArrayList class that is empty”. The count of elements of an array list created in such a way must then be zero. The third constructor, which takes a collection c as parameter “initializes a new instance of the ArrayList class that contains elements copied from the specified collection” So the number of elements of the new object equals the number of elements in the collection received as parameter, expressed by the assertion count = c.count (which in Eiffel would normally appear in a postcondition). Can then c.count be negative? Most likely not. Checking the documentation further reveals that the argument c passed to the constructor may denote any non-void collection, represented through one of the many classes inheriting from the ICollection interface [29]: arrayed list, sorted list, queue etc. Without performing an exhaustive examination, we note a hint in ArrayList itself, in the specification of routine Remove: “The average execution time is proportional to Count. That is, this method is an O(n) operation, where n is Count” which implies that count must always be non-negative. This evidence is enough to let us add a clause to the above invariant: positive_count: count >= 0 These first two properties are simple but already useful. For our next insights we examine the specification of class members. Documentation on the Count property reveals interesting information: “Count is always less than or equal to Capacity”. The self-assurance of this statement indicates that this property of the class always holds, suggesting that it is a class invariant. Hence a third invariant property for class ArrayList yielding the accumulated clause invariant positive_capacity: capacity >= 0 positive_count: count >= 0 valid_count: count <= capacity
Finding Implicit Contracts in .NET Components
295
5.2 Implicit Routine Preconditions Aside from implicit class invariants, the documentation also suggests preconditions. To get our clues we may look at documented exception cases. The specification of the routine Add of class ArrayList states that Add throws an exception of type NotSupportedException if the arrayed list on which it is called is read-only or has a fixed size. This suggests that the underlying implementation of Add first checks that the call target is writable (not read-only) and extendible (does not have a fixed size) before actually adding elements to the list. Such a requirement for having the method do what it is expected to is the definition of a routine precondition in terms of Design by Contract. An Eiffel specification of Add would then include the following two preconditions: require writable: not is_read_only extendible: not is_fixed_size
is_read_only and is_fixed_size are the Eiffel counterparts of the .NET properties IsReadOnly and IsFixedSize of class ArrayList. This example — one of many to be found in the reference documentation of the .NET Framework — suggests a scheme for extracting preconditions, applicable systematically, with some possibility of tool support: − Read the exception condition; e.g. the array list is read-only. − Take the opposite; for ArrayList the condition would be not is_read_only
− Infer the underlying routine precondition; here: writable: not is_read_only
5.3 Implicit Routine Postconditions
Does the .NET documentation also reveal closet postconditions? For an answer we consider the example of the query IndexOf. More precisely, since it is an overloaded method, we choose a specific version identified by its signature: public virtual int IndexOf (Object value);
The documentation explains that the return value is “the zero-based index of the first occurrence of value within the entire ArrayList, if found; otherwise, -1”. We may rephrase this specification more explicitly: − If value appears in the list, the result is the index of the first occurrence, hence greater than or equal to zero (.NET list indexes are indexed starting at zero) and less than Count, the number of elements in the list. − If value is not found, the result is –1.
296
K. Arnout and B. Meyer
Such a property is a guarantee on routine exit, a condition incumbent on the supplier on completion of the task — the definition of a postcondition. In Eiffel we would add the corresponding clause to the routine: ensure valid_index_if_found: contains (value) implies Result>0 and Result
This simple analysis suggests that routine postconditions do exist in .NET libraries, although not explicitly expressed because of the lack of support from the underlying environment. Unlike preconditions — for which it may be possible to devise supporting tools — postconditions are likely to require case-by-case human examination since they are scattered across the reference documentation. 5.4 Contracts in Interfaces Class ArrayList implements three interfaces (completely abstract specification modules) of the .NET Collections library: IList, ICollection, and IEnumerable. It is interesting to subject such interfaces to the same analysis as we have applied to the class. This analysis (see sections 6 and 7 for general statistics about the contract rates of .NET collection classes and interfaces) indicates that IList, ICollection, IEnumerable, and IEnumerator (of which IEnumerable is a client), do have routine preconditions and postconditions similar to those of ArrayList. We have not, however, found class invariants in these interfaces. This is probably because of the more limited scope of interfaces in .NET (coming from Java) as compared to “deferred classes”, their closest counterpart in the object-oriented model embodied by Eiffel. Deferred classes may have a mix of abstract and concrete features; in particular, they may include attributes. Interfaces, for their part, are purely abstract and may not contain attributes. The Eiffel policy provides a continuous spectrum from totally deferred classes, the equivalent of .NET and Java interfaces, to fully implemented classes, supporting the aims of object-oriented development with a seamless process from analysis (which typically uses deferred classes) to design and implementation (which make the classes progressively more concrete). Class invariants in the Eiffel libraries [22] often express consistency properties binding various attributes together. One can imagine, however, finding properties that hold for all the classes implementing the interface and that would be relevant candidates for “interface invariants”. But our non-exhaustive analysis of the .NET Collections library did not reveal such a case.
Finding Implicit Contracts in .NET Components
297
6 Adding Contracts a Posteriori The discovery of closet contracts in the .NET arrayed list class suggests that we should build a “contracted variant” of this class, ARRAY_LIST, that has the same interface as the original ArrayList plus the elicited contracts. We now present a sketch of this class and compare it with its EiffelBase counterpart: ARRAYED_LIST. Rather than modifying the original class we may produce the contracted variant — here in Eiffel — as a new class whose routines call those of the original. This is the only solution anyway when one doesn’t have access to the source code. The Contract Wizard is intended to support such a process, although for this discussion we have produced the result manually. 6.1 A Contracted Form of the .NET Arrayed List Class The original class, ArrayList, has 57 features (members in the .NET terminology). In presenting the contracted version we limit ourselves to 12 features, discussed in the preceding analysis of the class. The notation require -- from MY_CLASS
or ensure -- from MY_CLASS
shows that the assertion clauses that follow (respectively preconditions or postconditions) are inherited from the parent class MY_CLASS. This is the convention applied by EiffelStudio documentation tools when displaying the assertions of a class, for the parts inherited from ancestors. The difference is that in Eiffel assertions clauses are automatically inherited from parents; in .NET there is no such convention, so the .NET documentation has to repeat the same comments and exception conditions for an ancestor class or interface and all its descendants. The feature keyword introduces a “feature clause”, which groups a set of features (members) that have a common purpose. For example, the feature clause feature -- Initialization lists all the creation procedures of class ARRAY_LIST: make, make_from_capacity and make_from_collection. indexing description: "[ Implementation of a list using an array, whose size is dynamically increased as required. ]" class interface ARRAY_LIST create -- Note for non-Eiffelists: This is the list of -- creation procedures (constructors) for the -- class; the procedures’ definitions appear below.
298
K. Arnout and B. Meyer
make, make_from_capacity, make_from_collection feature -- Initialization make -- Create empty list with capacity -- Default_capacity. ensure empty: count = 0 default_capacity_set: capacity = Default_capacity writable: not is_read_only extendible: not is_fixed_size make_from_capacity (a_capacity: INTEGER) -- Create empty list with capacity a_capacity. require positive_capacity: a_capacity >= 0 ensure empty: count = 0 positive_capacity_implies_capacity_set: a_capacity > 0 implies capacity = a_capacity capacity_is_zero_implies_default_capacity_set: a_capacity =0 implies capacity =Default_capacity writable: not is_read_only extendible: not is_fixed_size make_from_collection (c: ICOLLECTION) -- Create list containing elements copied from c -- and the corresponding capacity. require collection_not_void: c /= Void ensure capacity_set: capacity = c.count count_set: count = c.count writable: not is_read_only extendible: not is_fixed_size feature -- Access capacity: INTEGER -- Number of elements the list can store count: INTEGER
Finding Implicit Contracts in .NET Components -- Number of elements in the list Default_capacity: INTEGER is 16 -- Default list capacity index_of (value: ANY): INTEGER -- Zero-based index of the first occurrence of -- value ensure -- from ILIST not_found_implies_minus_one: not contains (value) implies Result = - 1 found_implies_valid_index: contains (value) implies Result >= 0 and Result < count found_implies_correct_index: contains (value) implies item (Result) = value item (index: INTEGER): ANY -- Entry at index require -- from ILIST valid_index: index >= 0 and index < count feature -- Status report contains (an_item: ANY): BOOLEAN -- Does list contain an_item? is_fixed_size: BOOLEAN -- Has list a fixed size? is_read_only: BOOLEAN -- Is list read-only? feature -- Status setting set_capacity (value: like capacity) -- Set list capacity to value. require valid_capacity: value >= count ensure capacity_set: value > 0 implies capacity = value default_capacity_set: value = 0 implies capacity = Default_capacity feature -- Element change add (value: ANY): INTEGER -- Add value to the end of the list (double list
299
300
K. Arnout and B. Meyer -- capacity if the list is full) and return the -- index at which value has been added. require -- from ILIST writable: not is_read_only extendible: not is_fixed_size ensure -- from ILIST value_added: contains (value) updated_count: count = old count + 1 valid_index_returned: Result = count - 1 ensure then capacity_doubled: (old count = old capacity)
implies (capacity = 2 * (old capacity)) invariant positive_capacity: capacity >= 0 positive_count: count >= 0 valid_count: count <= capacity end
6.2 Metrics Fig. 1 shows measurements of properties of the contracted class ARRAY_LIST, produced with by the Metrics Tool of EiffelStudio. The measurements apply to the full class, with all 57 features, not to the abbreviated form shown above. A feature is “immediate” if it is new in the class, as opposed to a feature inherited from a parent (and possibly redefined in the class). The Eiffel metric tool’s output uses the following terminology: − Feature is the general name for class members. Features include attributes (“fields”) and routines (“methods”). A routine is a computation (algorithm) applicable to instances of the class; an attribute is stored in memory. If a routine returns a result, it is called a procedure; otherwise, it is a function. − Eiffel also distinguishes between commands and queries: A command returns no result; a query returns a result. If a query is computed at run time, it is a function; if it is stored in memory, it is an attribute. We note the following conclusions from these measurements: − 62% of the routines now have a contract (a precondition or a postcondition, usually both): 33 out of 52. − The 33 routines with preconditions tend to have more than one precondition clause: 2.5 on the average (82 total). − The 33 routines with postconditions tend to have more than one postcondition clause: 2 on average (67 total).
Finding Implicit Contracts in .NET Components
301
Fig. 1. Metrics about the contracted arrayed list class
7 Extending to Other Classes and Interfaces Equipped with our first results on ArrayList and its contracted Eiffel counterpart ARRAY_LIST, we now perform similar transformations and measurements on a few other classes and interfaces, to probe how uniform the results appear to be across the .NET Collections library. Since the assumptions and techniques are the same, we won’t repeat the details but go directly to results and interpretations. 7.1 Interfaces First, consider Eiffel deferred classes obtained by contracting the .NET interfaces from which ArrayList inherits: − ILIST, ICOLLECTION, IENUMERABLE, from which ARRAY_LIST inherits. − IENUMERATOR, of which IENUMERABLE is a client. Table 1 shows some resulting measurements. The statistics highlight three trends: − Absence of class invariant in these .NET interfaces, as already noted. − Presence of routine contracts: both preconditions and postconditions. The figures about IENUMERABLE and ICOLLECTION involve too few routines to bring valuable information — only one for class IENUMERABLE and four for ICOLLECTION. The figures about ILIST and IENUMERATOR are more significant in that respect — class ILIST has eleven routines and IENUMERATOR has six. Both classes (ILIST and IENUMERATOR) have at least one half of their routines with contracts. − Presence of multiple routine contracts: most routines have several preconditions and postconditions.
302
K. Arnout and B. Meyer Table 1. “Contract rate” of some .NET collection interfaces ILIST
ICOLLECTION
IENUMERABLE
IENUMERATOR
Routines
11
4
1
6
Routines with preconditions
7
1
0
3
Routines with postconditions
7
1
1
3
Number of preconditions
14
7
0
4
Number of postconditions
11
1
2
3
Precondition rate
64 %
25 %
0%
50 %
Postcondition rate
64 %
25 %
100 %
50 %
0
0
0
0
Class invariants
The last two points are consistent with the properties observed for class ARRAY_LIST. 7.2 Other Classes: Stack and Queue To test the generality of our first results on ArrayList, we consider two other classes of the Collections library. We choose Stack and Queue because they: − Are concrete collection classes. − Have no relation to ArrayList, except that all three implement the .NET interfaces ICollection and IEnumerable. − Have a direct counterpart in the EiffelBase library. Three classes is still only a small sample of the library. Any absolute conclusion would require exhaustive analysis, and hence a larger effort than the present study since the analysis is manual. So we have to be careful with any generalization of the results. We may note, however, that none of the three choices has been influenced by any a priori information or guess about the classes’ likelihood of including contracts. The same approach was applied to these classes as to ArrayList and its parents. Fig. 2 shows some of the resulting measurements for classes Stack and Queue. The figures confirm the trends previously identified: − Preconditions and postconditions are present. Class STACK has a 29% precondition rate (17 routines, of which 5 have preconditions) and a 59% postcondition rate (10 postcondition-equipped out of 17); class QUEUE has 42% and 58%.
Finding Implicit Contracts in .NET Components
303
Fig. 2. Metrics about the contracted stack and queue classes
− Preconditions and postconditions usually include several assertions. For example, STACK has 16 postcondition assertions for 10 contract-equipped routines. − Concrete classes have class invariants. For example, all three classes ArrayList, Stack, and Queue have an invariant clause positive_count: count >= 0
involving one attribute: count. This case-by-case analysis of 3 concrete classes and 4 interfaces of the .NET Collections library (out of 13 concrete classes and 8 interfaces) supports the second answer of the “Closet Contract Conjecture” – that contracts are inherent. We will now explore the benefits and limitations of such an a posteriori addition of contracts.
8 Effect on Library Users To appreciate the value of the results of the preceding analysis, we should assess their effect on the only constituency that matters in the end: library users – application
304
K. Arnout and B. Meyer
developers who take advantage of library classes to build their own systems. This issue is at the core of the Closet Contract Conjecture, since it determines whether we are doing any good at all by uncovering implicit contracts in contract-less libraries. By producing new versions of the library that make the contracts explicit, are we actually helping the users? To answer this question, we may examine the effect of the different styles on the library user (in terms of ease of learning and ease of use) and on the likely quality of the applications they develop. We take arrayed lists as an example and consider three variants: − The original, non-contracted class ArrayList from the .NET Collections library. − The contracted version ARRAY_LIST discussed above. − Finally, the corresponding class in the EiffelBase library, called ARRAYED_LIST, which was built with Design by Contract right from the start, rather than contracted a posteriori, and uses some other design ideas as well. 8.1 Dealing with Abnormal Cases in a Contract-Less Style The chapter in the .NET documentation devoted to class ArrayList provides a typical example of dealing with arrayed lists in that framework: using System; using System.Collections; public class SamplesArrayList { public static void Main() { // Creates and initializes a new ArrayList. ArrayList myAL = new ArrayList(); myAL.Add("Hello"); myAL.Add("World"); myAL.Add("!"); // Displays the properties and values of the // ArrayList. Console.WriteLine("myAL"); Console.WriteLine("\tCount: {0}",myAL.Count); Console.WriteLine("\tCapacity: {0}",myAL.Capacity); Console.Write ("\tValues:"); PrintValues (myAL); } public static void PrintValues (IEnumerable myList) { System.Collections.IEnumerator myEnumerator = myList.GetEnumerator(); while (myEnumerator.MoveNext()) Console.Write("\t{0}", myEnumerator.Current); Console.WriteLine(); } }
Finding Implicit Contracts in .NET Components
305
Running this C# program produces the following output: myAL Count: 3 Capacity: 16 Values: Hello
World
!
One striking point of this example is the absence of any exception handling — not even one if instruction in the class text — although our analysis of class ArrayList (see section 5) has revealed a non-trivial number of implicit contracts. For example, we have seen that the .NET method Add can only work properly if the targeted arrayed list is writable and extendible. But there is no such check in the class text above. This is likely to be on purpose since the property always holds at this point of the method execution: the .NET constructor ensures that the created list is not read-only and does not have a fixed size (see the contracted version of class ArrayList introduced in section 6), which allows calling the method Add on it. ArrayList myAL = new ArrayList(); /* Implicit check: (!myAL.IsFixedSize) && (!myAL.IsReadOnly) */ myAL.Add ("Hello"); myAL.Add ("World"); myAL.Add ("!");
If harmless in this simple example, such code may become dangerous if part of a reusable component. As a matter of fact, a novice programmer may overlook such a subtlety and reuse this code to create and add elements to a fixed-size arrayed list, which would cause the program execution to terminate on an unhandled exception of type NotSupportedException. This becomes even clearer if we encapsulate the calls to Add in a separate method FillArrayList that would look like the following: public void FillArrayList( ArrayList AL ){ AL.Add ("Hello"); AL.Add ("World"); AL.Add ("!"); }
and use FillArrayList in the Main routine: public static void Main() { ArrayList myAL = new ArrayList(); /* Implicit check: (!myAL.IsFixedSize) && (!myAL.IsReadOnly) */ FillArrayList (myAL); }
The previous program would work; the following one would not:
306
K. Arnout and B. Meyer
public static void Main() { ArrayList myAL = new ArrayList(); ArrayList.FixedSize (myAL); // The following call would throw an exception // because myAL is now a fixed-size arrayed list, // to which no element can be added. FillArrayList (myAL); }
Having Design by Contract support would be the right solution here (as discussed in the next sections). But because the .NET Common Language Runtime does not have native knowledge of contracts, .NET users have to rely on other techniques: − Using a “defensive” style of programming: checking explicitly for the routine requirements even if it can be inferred directly from the previous method statements (relying on the motto: “better check too much than too less”), hence adding redundant checking: ArrayList myAL = new ArrayList(); if ((!myAL.IsFixedSize) && (!myAL.IsReadOnly)) FillArrayList (myAL);
with: public void FillArrayList (ArrayList AL){ if ((!myAL.IsFixedSize) && (!myAL.IsReadOnly)){ AL.Add ("Hello"); AL.Add ("World"); AL.Add ("!"); } }
This style, however, leads to needless complexity by producing duplicate errorchecking code. The Design by Contract method goes in the opposite direction by avoiding redundancy and needless (Non-Redundancy principle, [20] p 343). − Relying on the exception handling mechanism of the .NET Common Language Runtime (typically, by using try…catch...finally… clauses): public static void Main() { try { // Creates and initializes a new ArrayList. ArrayList myAL = new ArrayList(); FillArrayList (myAL); // Prints list values. } catch (NotSupportedException e) { Console.WriteLine (e.Message); } }
Finding Implicit Contracts in .NET Components
307
with: public void FillArrayList (ArrayList AL) throws NotSupportedException { AL.Add ("Hello"); AL.Add ("World"); AL.Add ("!"); }
− Adding comments in the code to make implicit checks explicit and avoid misleading the library users: public static void Main() { ArrayList myAL = new ArrayList(); /* Implicit check: (!myAL.IsFixedSize) && (!myAL.IsReadOnly) */ FillArrayList (myAL); }
with: /* This method can only be called if AL does not * have a fixed size and is not read-only. */ public void FillArrayList (ArrayList AL) { AL.Add ("Hello"); AL.Add ("World"); AL.Add ("!"); }
Such an approach is efficient in the sense that there is no redundant check, thus no performance penalty, which gets closer to the ideas of Design by Contract, but it is not enforced at run time since it just relies on comments. This suggests the next approach, a posteriori contracting of classes. 8.2 Dealing with Abnormal Cases in a Contract-Rich Style A posteriori addition of contracts to a .NET component is likely to simplify the task of clients: rather than testing for a routine’s successful completion, they can just rely on the contracts, yielding to a lighter programming style (no redundant checking): indexing description: "[ Typical use of contracted class ARRAY_LIST ]"
308
K. Arnout and B. Meyer
class ARRAY_LIST_SAMPLE create make feature -- Initialization make is -- Create an arrayed list, fill it with -- Hello World!, and print its content. local my_list: ARRAY_LIST do create my_list.make fill_array_list (my_list) print_values (my_list) end feature -- Element change fill_array_list (an_array_list: ARRAY_LIST) is -- Fill an_array_list with Hello World!. require an_array_list_not_void: an_array_list /= Void is_extendible: not an_array_list.is_fixed_size is_writable: not an_array_list.is_read_only local index: INTEGER do index := an_array_list.add ("Hello ") index := an_array_list.add ("World”) index := an_array_list.add (“!") ensure array_list_filled: an_array_list.count = 3 end feature -- Output print_values (an_array_list: ARRAY_LIST) is -- Print content of an_array_list. require an_array_list_not_void: an_array_list /= Void local my_enumerator: IENUMERATOR do from my_enumerator := an_array_list.enumerator until
Finding Implicit Contracts in .NET Components
309
not my_enumerator.move_next loop print (my_enumerator.current_element) end end end
Since we know from the postconditions is_extendible and is_writable of creation procedure make of ARRAY_LIST that the preconditions of fill_array_list will be satisfied at this point of the routine execution, we do not need to add tests before calling the procedure. For readability or to facilitate debugging — when executing the software with assertion monitoring on — we might want to use an additional check instruction: create my_list.make check non_void: my_list /= Void is_extendible: not my_list.is_fixed_size is_writable: not my_list.is_read_only end fill_array_list (my_list) print_values (my_list)
although this is not required. If the creation routine make of class ARRAY_LIST had no such postconditions as is_extendible and is_writable, an explicit if control would have been needed in the client class ARRAY_LIST_SAMPLE to guarantee that the requirements of feature fill_array_list actually hold: create my_list.make if not my_list.is_fixed_size and not my_list.is_read_only then fill_array_list (my_list) end
But this is different from the “defensive” programming style used in a contract-less environment, since the test only affects the client side, not both client and supplier; the latter simply has preconditions. We call such a use of routine preconditions the a priori scheme: the client must act beforehand — before calling the routine — and ensure that the contracts are satisfied (either by testing them directly with an if control, or by relying on the postconditions of a previously called routine or on the class invariants). With this approach, any remaining run-time failure signals a design error. Such a design may not always be applicable in practice for either of three reasons: − Performance: Testing for a precondition before a routine call may be similar to the task of the routine itself, resulting in an unacceptable performance penalty. − Lack of expressiveness of the assertion languages: The notation for assertions might not be powerful enough.
310
K. Arnout and B. Meyer
− Dependency on external events: It is impossible to test for requirements if a routine involves interaction with the outside world, for example with a human user: there is no other choice than attempting to execute it, hence no way to predict abnormal cases. To address these limitations of the a priori scheme, it is possible to apply an a posteriori scheme — try the operation first and find out how it went — if a failed attempt has no irrecoverable consequences. Performance overhead – the first case above – is not a problem when the test being repeated is checking that a number is positive or a reference is not void. But the inefficiency might be more significant. An example from numerical computation [23] is a matrix equation solver: an equation of the form AX = B, where A is a matrix, and X (the unknown) and B are vectors, has a unique solution of the form X = A•¹ B only if matrix A is not singular. (A matrix is singular if one of the rows is a linear combination of others.) Applying the a priori scheme would lead the client to write code looking like the following: if a.is_singular then -- Report error. else x := a.inverse (b) end using a function inverse with precondition non_singular_matrix: inverse (b: VECTOR): VECTOR -- Solve equation of the form ax = b. require non_singular_matrix: not is_singular
This code does the job but is inefficient since determining whether a matrix is singular is essentially the same operation as solving the associated linear equation. Hence the idea of applying the a posteriori scheme; the client code would be of the form: a.invert (b) if a.inverted then x := a.inverse else -- Process erroneous case. end
Procedure invert replaces the previous function inverse. A call to this procedure (for which a more accurate name might be attempt_to_invert) sets a boolean attribute inverted to True or False to indicate whether inverting the matrix was possible, and if it was, makes the result available through attribute inverse. (A class invariant may state that inverted = (inverse /= Void).) This technique, which splits any function that may produce errors into a procedure that attempts to perform an operation and two attributes, one reporting whether the
Finding Implicit Contracts in .NET Components
311
operation was successful and the other giving access to the result of the operation if any, is compliant with the Command-Query Separation principle ([23], p 751). This example highlights one basic engineering principle for dealing with abnormal cases: whenever available, a method for preventing failures to occur is usually preferable to methods for recovering from failures. The techniques seen so far do not, however, provide a solution in three cases: − When abnormal events — such as a numerical failure or memory exhaustion — can cause the hardware or the operating system to interrupt the program execution abruptly (which is intolerable for systems with continuous availability requirements). − When abnormal situations, although not detectable through preconditions, must be diagnosed at the earliest possible time to avoid disastrous consequences — such as destroying the integrity of a database or even endangering human lives, as in an airplane control system. (One must keep in mind that such situations can appear in a contract-rich environment as well, since the support for assertions may not be rich enough to express complex properties.) − When there is a requirement for software fault tolerance, protecting against the most dramatic consequences of any remaining errors in the software. 8.3 “A Posteriori Contracting” vs. “Contracting from the Start” We have seen that clients of a .NET library are likely to benefit from an a posteriori addition of contracts: instead of having to test whether a routine successfully went to completion (with the risk of forgetting to check and getting an exception at run time), they could just rely on the contracts. What about contracting “from the start” now? Is the EiffelBase class ARRAYED_LIST more convenient to use than the contracted class ARRAY_LIST? To help answer this question, let’s consider a variant of the previous class ARRAY_LIST_SAMPLE, representing a typical client use of arrayed lists, this time using the EiffelBase ARRAYED_LIST: indexing description: "Typical use of EiffelBase ARRAYED_LIST" class ARRAYED_LIST_SAMPLE create make feature -- Initialization make is -- Create a list with two elements and print the -- list contents. local my_list: ARRAYED_LIST [STRING] do create my_list.make my_list.extend ("Hello ")
312
K. Arnout and B. Meyer my_list.extend ("World”) my_list.extend (“!") from my_list.start until my_list.after loop io.put_string (my_list.item) my_list.forth end end
end
This example highlights three characteristics of the EiffelBase ARRAYED_LIST: − A clear separation between commands and queries: the routine extend returns no result (on the contrary to the .NET feature Add, which returns an integer, yielding a useless local variable index in the ARRAY_LIST code example). − The usefulness of genericity: we know that my_list.item is of type STRING, thus we can use a more appropriate I/O feature to print it: put_string, rather than the general print. − A user-friendly interface to traverse the list through features start, after, item, and forth, relying on an internal cursor stored in class ARRAYED_LIST. Another interesting property to look at is the easiness of switching to other list implementations. As shown on Fig. 3, ARRAYED_LIST inherits from both ARRAY and DYNAMIC_LIST. It suffices to remove the relationship with class ARRAY to obtain a LINKED_LIST. The a-posteriori-contracted class ARRAY_LIST (section 6) just mapped the original .NET hierarchy, which makes extensive use of interfaces to compensate the lack of multiple inheritance (Fig. 4). Not surprisingly in light of how it was obtained, this class does not fully benefit from the power of Eiffel, in matter of design, reusability, and extendibility. Another obvious difference between ARRAY_LIST (the contracted class) and ARRAYED_LIST (the EiffelBase class) is the use of genericity in the Eiffel version. Although this falls beyond the scope of the present discussion, we may note that the lack of genericity in the current .NET object model leads to code duplication as well as to run-time check (casts) that damage both the performance and the reliability of the software. Future versions of .NET are expected to provide genericity [17] [18]. The next difference is the use of enumerators in ARRAY_LIST whereas ARRAYED_LIST stores a cursor as an attribute of the class. − In the first approach, enumerators become irrecoverably invalidated as soon as the corresponding collection changes (addition, modification, or deletion of list elements). The approach, on the other hand, allows multiple concurrent traversals of the same list through multiple cursors. − The EiffelBase approach solves the problem of invalid cursors: addition or deletion of list elements change the cursor position, and queries before and after take care of the cursor position’s validity. But the use of internal cursors requires care to avoid endless loops.
Finding Implicit Contracts in .NET Components
313
Fig. 3. Inheritance hierarchy of the EiffelBase arrayed list class
Fig. 4. Inheritance hierarchy of the .NET arrayed list class
These differences of style should not obscure the fundamental difference between a design style that has contracts from the start and one that adds them to existing contract-less library components. The examples illustrate that the right time to put in the contracts is during design. When that is not possible, for example with a library produced by someone who didn’t trouble himself with contracts, it may still be worthwhile to add them a posteriori, as a way to understand the library better, improve its documentation, and make it safer to use.
9 Automatic Extraction of Closet Contracts Our analysis of the .NET Collections library has shown interesting “patterns” about the nature and location of hidden contracts. In particular, routine preconditions tend to be buried under exception conditions. Our goal is to estimate the highest degree of automation we can achieve in extracting closet contracts from .NET libraries. We first report on the technique of dynamic contract detection and then describe our approach of inferring preconditions from the CIL code [32] of .NET assemblies.
314
K. Arnout and B. Meyer
9.1 Dynamic Contract Inference Dynamic contract inference, working from source code, seeks to deduce assertions from captured variable traces by executing the program with various inputs, relying on a set of possible assertions to deduce contracts from the execution output. The next step is to determine whether the detected assertions are meaningful and useful to the users, typically by computing a confidence probability. Ernst’s Daikon tool discovers class invariants, loop invariants and routine pre- and postconditions. Its first version [11] was limited to finding contracts over scalars and arrays; the next one (Daikon 2) [14] enables contract discovery over collections of data, and computes conditional assertions. Daikon succeeds in finding the assertions of a formally-specified program, and can even find some more, revealing deficiencies in the formal specification. Daikon also succeeds in inferring contracts from a C program, which helps developers performing changes to the C program without introducing errors [12]. It appears that the large majority of reported invariants are correct, and that Daikon extracts more invariants from high-quality programs [11]. Daikon still needs improvement in terms of: − Performance: “Invariant detection time grows approximately quadratically with the number of variables over which invariants are checked” [12]. Some experiments using incremental processing have shown promising results in improving Daikon’s performance [14]. − Relevance of the reported invariants: Daikon still reports irrelevant — meaning useless, but not necessary incorrect — invariants. Polymorphism can help increase the number of desired invariants reported to the users [14]. − Richness of inferred invariants: Currently, most cover simple properties. Ernst et al. suggest examining the techniques and algorithms used in the research fields of artificial intelligence [12] and information retrieval [13] to improve dynamic inference of invariants for applications in software evolution. The Daikon detector is not the sole tool available to dynamically extract contracts. Some Java detectors also exist; some of them do not even require the program source code to infer contracts: they can operate directly on bytecode files (*.class). 9.2 Extracting Routine Preconditions from Exception Cases “Human analysis is sometimes more powerful than either, allowing deep and insightful reasoning that is beyond hope for automation” [16]. When comparing static and dynamic techniques of program analysis, Ernst et al. admit that automatic tools fail in some cases where a human being would succeed. Does extraction of closet contracts fall into the category of processes that cannot be fully automated? If so, can we at least automate part of the effort? The analysis reported in this article has shown some regularity in the form and location of the closet contracts we can find in existing .NET components. In particular, preconditions tend to be buried in exception cases. Since method exception cases are not kept into the assembly metadata, we are currently exploring another approach: inferring routine preconditions from a systematic analysis of the CIL (Common Intermediate Language) code [32] of the .NET assemblies provided as input. More precisely, we are parsing the CIL code of .NET libraries — using Gobo Eiffel Lex and
Finding Implicit Contracts in .NET Components
315
Gobo Eiffel Yacc [9] — to list the exceptions a method or a property may throw to infer the corresponding routine preconditions. The first results are promising.
10 Conclusion This discussion has examined some evidence from the .NET libraries relevant to our basic conjecture: do existing libraries designed without a clear notion of contract contain some “contracts” anyway? This analysis provides initial support for the conjecture. The contracts are there, expressed in other forms. Preconditions find their way into exceptions; postconditions and class invariants into remarks scattered across the documentation, hence more difficult to extract automatically. The analysis reported here provides a first step in a broader research plan, which we expect to expand in the following directions: − Applying the same approach to other .NET and non-.NET libraries, such as C++ STL (a first informal look at [26] suggests that there are contracts lurking there too). − Investigating more closely the patterns that help discover each type of contract — class invariants, routine preconditions and postconditions — to facilitate the work of programmers interested in adding contracts a posteriori to existing libraries, with a view to providing an interactive tool that would support this process. − Turning the Eiffel Contract Wizard into a Web service to allow any programmers to contribute contracts to .NET components. This area of research opens up the possibility of various generalizations of this work in a broad investigation of applications of Design by Contract. (We are looking forward to seeing the evolution of the project conducted by Kevin McFarlane and aiming at providing a Design by Contract framework for use in .NET projects [19], of a current project at Microsoft Research about adding contracts into the C# language — see the “Assertions” section of [30] — and also of the new eXtensible C#© [35]; the outcome of these projects are likely to influence our research direction.)
Acknowledgements This paper takes advantage of extremely valuable comments and insights from Éric Bezault (Axa Rosenberg), Michael D. Ernst (MIT), Tony Hoare (Microsoft) and Emmanuel Stapf (Eiffel Software). Opinions expressed are of course our own. References [3] to [6] are previous versions of this work. Reference [7] is a summary version.
316
K. Arnout and B. Meyer
References 1. Karine Arnout, and Raphaël Simon. “The .NET Contract Wizard: Adding Design by Contract to languages other than Eiffel”. TOOLS 39 (39th International Conference and Exhibition on Technology of Object-Oriented Languages and Systems). IEEE Computer Society, July 2001, p 14-23. 2. Karine Arnout. “Eiffel for .NET: An Introduction”. Component Developer Magazine, September-October 2002. Available from http://www.devx.com/codemag/Article/8500. Accessed October 2002. 3. Karine Arnout, and Bertrand Meyer. “Extracting implicit contracts from .NET components”. Microsoft Research Summer Workshop 2002, Cambridge, UK, 9-11 September 2002. Available from http://se.inf.ethz.ch/publications/arnout/workshops/microsoft_summer_research_workshop_2002/contract_extraction.pdf. Accessed September 2002. 4. Karine Arnout. “Extracting Implicit Contracts from .NET Libraries”. 4th European GCSE Young Researchers Workshop 2002, in conjunction with NET.OBJECT DAYS 2002. Erfurt, Germany, 7-10 October 2002. IESE-Report No. 053.02/E, 21 October 2002, p 20-24. Available from http://www.cs.uni-essen.de/dawis/conferences/Node_YRW2002/papers/ karine_arnout_gcse_final_copy.pdf. Accessed October 2002. 5. Karine Arnout. “Extracting Implicit Contracts from .NET Libraries”. OOPSLA 2002 (17th ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications), Posters. Seattle USA, 4-8 November 2002. OOPSLA'02 Companion, ACM, p 104105. 6. Karine Arnout, and Bertrand Meyer. “Contrats cachés en .NET: Mise au jour et ajout de contrats a posteriori”. LMO 2003 (Langages et Modèles à Objets). Vannes, France, 3-5 February 2003. 7. Karine Arnout, and Bertrand Meyer. “Spotting hidden contracts: the .NET example”. Submitted for publication. 8. Mike Barnett, and Wolfram Schulte. “Contracts, Components, and their Runtime Verification on the .NET Platform”. Microsoft Research Technical Report TR 2002-38, April 2002. Available from ftp://ftp.research.microsoft.com/pub/tr/tr-2002-38.pdf. Accessed April 2002. 9. Éric Bezault. Gobo Eiffel Lex and Gobo Eiffel Yacc. Retrieved September 2002 from http://www.gobosoft.com. 10. Eiffel Software Inc. EiffelBase. Retrieved October 2002 from http://docs.eiffel.com/libraries/base/index.html. 11. Michael D. Ernst. “Dynamically Detecting Likely Program Invariants”. Ph.D. dissertation, University of Washington, 2000. Available from http://pag.lcs.mit.edu/~mernst/pubs/invariants-thesis.pdf. Accessed August 2002. 12. Michael D. Ernst, Jake Cockrell, William G. Griswold, and David Notkin. “Dynamically Discovering Likely Program Invariants to Support Program Evolution”. IEEE TSE (Transactions on Software Engineering), Vol.27, No.2, February 2001, p: 1-25. Available from http://pag.lcs.mit.edu/~mernst/pubs/invariants-tse.pdf. Accessed August 2002. 13. Michael D. Ernst, Adam Czeisler, William G. Griswold, and David Notkin. “Quickly Detecting Relevant Program Invariants”. ICSE 2000 (International Conference on Software Engineering), Limerick, Ireland, 4-11 June 2000; Available from http://pag.lcs.mit.edu/~mernst/pubs/invariants-icse2000.pdf. Accessed August 2002. 14. Michael D. Ernst, William G. Griswold, Yoshio Kataoka, and David Notkin. “Dynamically Discovering Program Invariants Involving Collections”, Technical Report, University of Washington, 2000. Available from http://pag.lcs.mit.edu/~mernst/pubs/invariantspointers.pdf. Accessed August 2002. 15. C.A.R. Hoare. “Proof of Correctness of Data Representations”. Acta Infomatica, Vol. 1, 1973, p: 271-281.
Finding Implicit Contracts in .NET Components
317
16. Yoshio Kataoka, Michael D. Ernst, William G. Griswold, and David Notkin: “Automated Support for Program Refactoring using Invariants”. ICSM 2001 (International Conference on Software Maintenance), Florence, Italy, 6-10 November 2001. Available from: http://pag.lcs.mit.edu/~mernst/pubs/invariants-refactor.pdf. Accessed August 2002. 17. Andrew Kennedy, and Don Syme. “Design and Implementation of Generics for the .NET Common Language Runtime”. PLDI 2001 (Conference on Programming Language Design and Implementation). Snowbird, Utah, USA, 20-22 June 2001. Available from http://research.microsoft.com/projects/clrgen/generics.pdf. Accessed September 2002. 18. Andrew Kennedy, and Don Syme. Generics for C# and .NET CLR, September 2002. Retrieved September 2002 from http://research.microsoft.com/projects/clrgen/. 19. Kevin McFarlane. Design by Contract Framework for .Net. February 2002. Retrieved October 2002 from http://www.codeproject.com/csharp/designbycontract.asp and http://www.codeguru.com/net_general/designbycontract.html. st 20. Bertrand Meyer: Object-Oriented Software Construction (1 edition). Prentice Hall International, 1988. 21. Bertrand Meyer. “Applying ‘Design by Contract’”. Technical Report TR-EI-12/CO, Interactive Software Engineering Inc., 1986. Published in IEEE Computer, Vol. 25, No. 10, October 1992, p 40-51. Also published as “Design by Contract” in Advances in ObjectOriented Software Engineering, eds. D. Mandrioli and B. Meyer, Prentice Hall, 1991, p 150. Available from http://www.inf.ethz.ch/personal/meyer/publications/computer/contract. pdf. Accessed April 2002. 22. Bertrand Meyer: Reusable Software: The Base Object-Oriented Component Libraries. Prentice Hall, 1994. 23. Bertrand Meyer: Object-Oriented Software Construction, second edition. Prentice Hall, 1997. 24. Bertrand Meyer, Raphaël Simon, and Emmanuel Stapf: Instant .NET. Prentice Hall (in preparation). 25. Bertrand Meyer: Design by Contract. Prentice Hall (in preparation). 26. Scott Meyers: Effective STL. Addison Wesley, July 2001. 27. Microsoft. .NET Collections library. Retrieved June 2002 from http://msdn.microsoft.com/library/default.asp?url=/library/enus/cpref/html/frlrfsystemcollections.asp. 28. Microsoft. .NET ArrayList class. Retrieved June 2002 from http://msdn.microsoft.com/library/default.asp?url=/library/enus/cpref/html/frlrfsystemcollectionsarraylistclasstopic.asp. 29. Microsoft. .NET ICollection interface. Retrieved October 2002 from http://msdn.microsoft.com/library/default.asp?url=/library/enus/cpref/html/frlrfsystemcollectionsicollectionclasstopic.asp. 30. Microsoft Research. Current research, Programming Principles and Tools. Retrieved November 2002 from http://research.microsoft.com/research/ppt/. 31. Richard Mitchell, and Jim McKim: Design by Contract, by example. Addison-Wesley, 2002. 32. NET Experts. ECMA TC39 TG2 and TG3 working documents. Retrieved September 2002 from http://www.dotnetexperts.com/ecma/index.html. 33. Jeremy W. Nimmer, and Michael D. Ernst. “Invariant Inference for Static Checking: An Empirical Evaluation”. FSE ’02 (10th International Symposium on the Foundations of Software Engineering). Charleston, SC, USA. November 20-22, 2002. Proceedings of the ACM SIGSOFT. Available from http://pag.lcs.mit.edu/~mernst/pubs/esc-annotate.pdf. Accessed October 2002. 34. Jeremy W. Nimmer, and Michael D. Ernst. “Automatic generation of program specifications”. ISSTA 2002 (International Symposium on Software Testing and Analysis). Rome, Italy, 22-24 July 2002. Available from http://pag.lcs.mit.edu/~mernst/pubs/invariantsspecs.pdf. Accessed October 2002.
318
K. Arnout and B. Meyer
35. ResolveCorp. eXtensible C#© is here! Retrieved May 2003 from http://www.resolvecorp.com/products.htm. 36. Raphaël Simon, Emmanuel Stapf, and Bertrand Meyer. “Full Eiffel on .NET”. MSDN, July 2002. Available from http://msdn.microsoft.com/library/default.asp?url=/library/enus/dndotnet/html/pdc_eiffel.asp. Accessed October 2002. 37. Software Engineering Institute. “Volume II: Technical Concepts of Component-Based Software Engineering”. CMU/SEI-2000-TR-008, 2000. Available from http://www.sei.cmu.edu/publications/documents/00.reports/00tr008.html. Accessed June 2002. 38. Dave Thomas. “The Deplorable State of Class Libraries”. Journal of Object Technology (JOT), Vol.1, No.1, May-June 2002. Available from http://www.jot.fm/issues/issue_2002_05/column2. Accessed June 2002.
From Co-algebraic Specifications to Implementation: The Mihda Toolkit Gianluigi Ferrari, Ugo Montanari, Roberto Raggi, and Emilio Tuosto Dipartimento di Informatica, Universit´ a di Pisa, Italy. {giangi,ugo,raggi,etuosto}@di.unipi.it
Abstract. This paper describes the architecture of a toolkit, called Mihda, providing facilities to minimize labelled transition systems for name passing calculi. The structure of the toolkit is derived from the co-algebraic formulation of the partition-refinement minimization algorithm for HD-automata. HD-automata have been specifically designed to allocate and garbage collect names and they provide faithful finite state representations of the behaviours of π-calculus processes. The direct correspondence between the coalgebraic specification and the implementation structure facilitates the proof of correctness of the implementation. We evaluate the usefulness of Mihda in practice by performing finite state verification of π-calculus specifications.
1
Introduction
Finite state automata (e.g. labelled transition systems) provide a foundational model underlying effective verification techniques of concurrent and distributed systems. From a theoretical point of view, many behavioural properties of concurrent and distributed systems can be naturally defined directly as properties over automata. From a practical point of view, efficient algorithms and verification techniques have been developed and widely applied in practice to case studies of substantial complexity in several areas of computing such as hardware, compilers, and communication protocols. We refer to [2] for a review. A fundamental property of automata is the possibility, given an automaton, to construct its canonical form: The minimal automaton. The theoretical foundations guarantee that the minimal automaton is indistinguishable from the original one with respect to many behavioural properties (e.g., bisimilarity of automata and behavioural properties expressed in suitable modal or temporal logics). Minimal automata are very important also in practice. For instance, the problem of deciding bisimilarity is reduced to the problem of computing the minimal transition system [8]. Moreover, it is often convenient, from a computational point of view, to verify properties on the minimal automaton rather than on the original one. Indeed, minimization algorithms can be used to attack state explosion: They yield a small state space, but still retain all the relevant information for the verification.
This work has been supported by EU-FET project PROFUNDIS IST-2001-33100 and by MIUR project NAPOLI.
F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 319–338, 2003. c Springer-Verlag Berlin Heidelberg 2003
320
G. Ferrari et al.
Global computing systems consists of networks of stationary and mobile components. The primary features of a global computing systems is that components are autonomous, software versioning is highly dynamic, the network coverage is variable and often components reside over the nodes of the network (WEB services), membership is dynamic and often ad hoc without a centralized authority. Global computing systems must be made very robust since they are intended to operate in potentially hostile environments. Moreover, they are hard to construct correctly and very difficult to test in a controlled way. Although significant progresses have been made in providing formal models and effective verification techniques to support verification of global computing systems, current software engineering technologies provide limited solutions to some of the issues discussed above. The problem of formal verification of global computing systems still requires considerable research and dissemination efforts. History Dependent automata (HD-automata shortly) have been proposed in [14, 11, 4] as a new effective model for name passing calculi. Name passing calculi (e.g. the π-calculus [10, 9, 16]) are basically the best known and probably the most acknowledged models of mobility. Moreover, they provide a rich set of techniques for reasoning about mobile systems. Similarly to ordinary automata, HD-automata are made out of states and labelled transitions; their peculiarity resides in the fact that states and transitions are equipped with names which are no longer dealt with as syntactic components of labels, but become explicit part of the operational model. This allows one to model explicitly name creation/deallocation or name extrusion: These are the distinguished mechanisms of name passing calculi. HD-automata have been abstractly understood as automata over a permutation model, whose ingredients are a set of names and an action of its group of permutations (renaming substitutions) on an abstract set. This framework is sufficient to describe and reason about formalisms with name-binding operations. It has been incorporated into various kinds of transition systems that aim at providing syntax-free models of name-passing calculi [5, 6, 12, 15]. It is important to emphasize that names of states of HD-automata have local meaning. For instance, assume that A(x, y, z) denotes an agent having three (free) names x, y and z. Then agent A(y, x, z), obtained through the transformation which swaps names x and y, is syntactically different from A(x, y, z). However, these two agents can be semantically represented by means of a single state q of a HD-automaton simply by considering a “swapping” operation on the local names corresponding to names x and y. More generally, states that differs only for renaming of their local names are identified in HD-automata. This property allows for a very compact representation of name passing calculi. Local meaning of names requires a mechanism for describing how names correspond each other along state transitions. Graphically, we can represent such correspondences using “wires” that connect names of labels, source and target states of transitions. For instance, Figure 1 depicts a transition from source state s to destination state d. The transition exposes two names: Name 2 of s and a fresh name 0. State s has three names, 1, 2 and 3 while state d has two names
From Co-algebraic Specifications to Implementation: The Mihda Toolkit
lab 2
3
321
0 5
2 1
4 s
σ
d
Fig. 1. A HD-automaton transition
4 and 5 which correspond to the old name 1 of s and to the fresh name 0, respectively. Notice that names 3 is discharged along such state transition. HD-automata have a natural representation as coalgebras on a category of named sets and named functions. Elements of named sets are equipped with names which are defined up to groups of name permutations called symmetries [12]. General results concerning coalgebras guarantees the existence of the minimal HD-automata up to bisimilarity. In [4] two of the authors describe a declarative coalgebraic procedure to perform minimization of finite state HDautomata. In this paper, we review the coalgebraic description of the minimization algorithm for HD-automata, and we illustrate its prototype implementation. This yields a toolkit, called Mihda, providing general facilities to minimize labelled transition systems for name passing calculi. The usefulness of the Mihda toolkit will be shown by performing finite state verification of π-calculus specifications. The minimization algorithm has been specified by exploiting a type-theoretic notation rather than standard coalgebraic notations. The implementation data structures have been obtained by refining with efficiency considerations the typetheoretic formulation. The direct correspondence between the semantical structures and the implementation structures facilitates the design and the implementation of the toolkit. Moreover, it provides the formal machinery to perform the proof of correctness of the implementation. Recently, several software engineering technologies have been introduced to support a programming paradigm where the web is exploited as a service distributor. By service we do not mean a monolithic web server but rather a component available over the web that others might use to develop other services. Conceptually, web services are stand-alone components that reside over the nodes of the network. Each web service has an interface which is network accessible through standard network protocols and describes the interaction capabilities of the service. The Mihda toolkit have been designed and made available as a WEB service. By a few clicks in a browser at the URL http://jordie.di.unipi.it:8080/pweb/ the Mihda toolkit can be accessed remotely and its facilities can be evaluated directly over the WEB.
2
Preliminaries
This section sketches the main concepts on the coalgebraic representation of automata as a basis for finite state verification by semantic minimization. We
322
G. Ferrari et al.
Set idA
/ F(A) t
,A f
f ;g
+3 Set
F
F (f )
/ F(B)
B g
F(idA ) = idF(A)
F(f ; g) = F(f ); F(g)
F (g)
/ F(C)
C
Fig. 2. Functor over Set
illustrate the approach by providing the coalgebraic specification of the minimization algorithm for ordinary labelled transition systems. Hereafter, we will use terms ’automaton’ and ’labelled transition system’ interchangeably. An automaton A is a triple (S, L, →) where S is the set of states, L is the set of actions or labels and →⊆ S × L × S is the transition relation. Usually, one writes s − → d to indicate (s, , d) ∈− →; s is the source state and d is the destination or target state. Let idA denote the identity function on set A and f ; g be the composition of functions f and g (when it is defined). An endo-functor F (over category Set) is a mapping from sets to sets and from function to functions that preserves identity functions and function composition. Figure 2 gives a graphical representation of how a functor acts on sets and functions. If A is mapped on F (A) then idA is associated to idF (A) and, if f and g can be composed, then F(f ) and F (g) can be composed as well. Moreover, the image through F of the function composition f ; g is obtained by composing the images of functions f and g. Definition 1 (F -coalgebra). Let F be an endo-functor on the category Set. A F-coalgebra consists of a pair (A, α) such that α : A → F (A). The duality between F -coalgebras and F -algebras (a function F(A) → A) consists in the fact that domain and codomain are “reversed”, namely, are arrows between the same objects but with opposite directions. Different directions can be interpreted as “construction” (induction) and “observation” (coinduction). The interested reader is referred to [7, 1]. Before specifying the coalgebraic description of the minimization algorithm we introduce some notations. – Expression Q : Set denotes a set and q : Q is synonym of q ∈ Q; – Fun is the collection of functions among sets (the arrows of category Set). The function space over sets has the following structure: Fun = {H | H = S : Set , D : Set , h : S − → D}.
From Co-algebraic Specifications to Implementation: The Mihda Toolkit bij
323
inj
– h : A − → B (h : A − → B) explicitly states that function h is bijective (injective). We shall use SH , DH and hH to denote domain, codomain and mapping of an element of Fun,respectively. A similar convention will be used throughout the paper to denote components of tuples. Let H, K ∈ Fun be two functions, then the composition of H and K (H; K) is defined provided that SK = DH and it is the function such that SH;K = SH , DH;K = DK , and hH;K = hK ◦ hH . Sometimes, we shall need to work with be the function given by S = SH , D = surjective functions. Hence we let H H H {q : DH | ∃q : SH , hH (q) = q } and hH = hH . Finite-state transition systems have been coalgebraically described by employing two ingredients: A set Q, that represents the state space, together with a function K : Q − → ℘fin (L × Q) giving the transition relation; K(q) is the set of
→ q . pairs (, q ) such that q − In this paper, we shall work on a more concrete representation. In particular, we introduce a mathematical structure, called bundle, whose rˆole is to provide a declarative specification of the concrete data structure storing all the transitions out of a given state. Indeed, each bundle details which states are reachable by performing certain actions. Definition 2 (Bundles). Let L be the set of labels, then a bundle β over L is a structure D : Set, Step : ℘fin (L × D). Set D is the support of β. Given a fixed set of labels L, B L denotes the collection of bundles and β : B L indicates that β is a bundle over L. We now introduce the functor A over the universe of sets and functions. The following clauses define A: – A(Q) = {β : B L | Dβ = Q}, for each Q : Set; – For each H : Fun, A(H) is defined as follows: • SA(H) = A(SH ) and DA(H) = A(DH ); • hA(H) : β → DH , {, hH (q) | , q : Stepβ }. Definition 3 (Transition systems as coalgebras). Let L be a set of labels. Then a labelled transition system over L is a coalgebra for functor A, namely it is a function K such that DK = A(SK ). Example 1. A coalgebra K for functor A represents a transition system where SK is the set of states, and hK (q) = β, with Dβ = SK . Let us consider a finite-state automaton and its coalgebraic formulation via the mapping hK . 0> >> >> b
a
a
/1i >> >> a > b b 4 3> >> > > c c 5
) b
2
Sk , {a, 1, b, 3} hK (0) = hK (1) = Sk , {a, 2, b, 3, b, 4} hK (2) = Sk , {a, 1, b, 4} hK (3) = Sk , {c, 5} hK (4) = Sk , {c, 5} hK (5) = Sk , ∅
324
G. Ferrari et al.
Note how, for each state q ∈ {0, ..., 5}, hK (q) yields all the immediate successor states of q and the corresponding labels. In other words, (, q ) ∈ StephK (q) if,
→ q . and only if, q − General results on coalgebras ensure the existence of the final coalgebra for a large class of functors. These results apply to our formulation of labelled transition systems. In particular, it is interesting to see the result of the iteration along the terminal sequence of functor A. Let K be a transition system, and let H0 , H1 , . . . , Hi+1 , . . . be the sequence of functions computed by Hi+1 = K; A(Hi ), where H0 is the unique function from SK to the one-element set {∗} given by SH0 = SK ; DH0 = {∗}; and hH0 (q : SH0 ) = ∗. Finiteness of ℘fin ensures convergence of the iteration along the terminal sequence. We can say much more if the transition system is finite state. Indeed, if K is a finite-state transition system, then – The iteration along the terminal sequence converges in a finite number of steps, i.e. DHn+1 ≡ DHn (for some natural number n), – The isomorphism mapping F : DHn − → DHn+1 yields the minimal realization of transition system K. Comparing the co-algebraic construction with the standard algorithm [8, 3] which constructs the minimal labelled transition system we can observe: – at each iteration i the elements of DHi are the blocks of the minimization algorithm (i.e. the i-th partition). Notice that the initial approximation DH0 contains a single block: in fact H0 maps all the states of the transition system into {∗}. – at each step the algorithm creates a new partition by identifying the splitters for states q and q . This corresponds in our co-algebraic setting to the fact that Hi (q) = Hi (q ) but Hi+1 (q) = Hi+1 (q ). – the iteration proceeds until a stable partition of blocks is reached: then the iteration along the terminal sequence converges. We now apply the iteration along the terminal sequence to the coalgebraic formulation of the transition system of Example 1. The initial approximation is the function H0 defined as follows H0 = SH0 = SK , DH0 = {∗}, hH0 (q) = ∗ and
→ q } the first approximation H1 is the map hH1 : q → DH0 , {, hH0 (q ) : q − obtained by T (H0 ){1, 2, 3, 4, 5}, {a, 2, b, 3, b, 4} = {∗}, {a, ∗, b, ∗}
From Co-algebraic Specifications to Implementation: The Mihda Toolkit
325
We obtain the function hH1 and the destination state DH1 = {β1 , β2 , β3 } as detailed below. hH1 (0) hH1 (1) hH2 (2) hH1 (3) hH1 (4) hH1 (5)
= {∗}, {a, ∗, b, ∗} = {∗}, {a, ∗, b, ∗} β1 = {∗}, {a, ∗, b, ∗} = {∗}, {a, ∗, b, ∗} {∗}, {c, ∗} β2 = = {∗}, {c, ∗} β3 = ∅ = {∗}, {c, ∗} = {∗}, ∅
A further iteration yields: hH2 (0) hH2 (1) hH2 (2) hH2 (3) hH2 (4) hH2 (5)
= DH1 , {a, β1 , b, β2 } = DH1 , {a, β1 , b, β2 } = DH1 , {a, β1 , b, β2 } = DH1 , {c, ∅} = DH1 , {c, ∅} = DH1 , ∅
Since DH2 ≡ DH1 the iterative construction converges, thus providing the minimal labelled transition system depicted as
a
•M 1
b
/ •2
c
/ •3
where •1 = {0, 1, 2}, •2 = {3, 4} and •3 = {5}.
3
HD-Automata for π-Agents
This section outlines the representation of HD-automata as coalgebras over the concrete permutation algebra of named sets. Let N be an infinite countable set of names ranged over by v and let N be the set N ∪ ∗ where ∗ ∈ N is a distinguished name. The distinguished name ∗ will be used to model name creation. We also assume ta total order < on N (for instance, < can be the lexicographic order on N and ∀v ∈ N : ∗ < v). Table 1 displays the definitions of named sets, named functions, and composition of named functions. In Table 1, the general product is employed (as usual in type theory) to type functions f such that the type of f (q) is dependent on q. Intuitively, a named set represents a set of states equipped with a mechanism to give local meaning to names occurring in each state. In particular, function | | yields the number of local names of states. Moreover, the permutation group GA (q) allows one to describe directly the renamings that do not affect the behaviour of q, i.e., symmetries on the local names of q. For technical reasons, we assume that states are totally ordered. By convention we write {q : QA }A to indicate the set {v1 , ..., v|q|A } and we use NSet to denote the universe of named sets. As in the case of standard transition systems, named functions are used to determine the possible transitions of a given state. Intuitively, hH (q) yields the
326
G. Ferrari et al.
Table 1. Named sets, Named Functions and Composition of Named Functions Named set A named set A is a structure A = Q : Set, | |: Q −→ ω, ≤: ℘(Q × Q), G :
bij
℘({v1 ..v|q| } −→ {v1 ..v|q| })
q∈Q
where ∀q : QA , GA (q) is a permutation group and ≤A is a total ordering.
Named function A named function H is a structure inj
H = S : NSet, D : NSet, h : QS −→ QD , Σ : QS −→ ℘({h(q)}D −→ {q}S ) where ∀q : QSH , ∀σ : ΣH (q), 1. GDH (hH (q)); σ = ΣH (q) and 2. σ; GSH (q) ⊆ ΣH (q). Composition of named functions Named functions can be composed in the obvious way. Let H and K be named functions. Then H; K is defined only if DH = SK , and SH;K = SH ,
DH;K = DK ,
hH;K : QSH −→ QDK = hH ; hK ,
ΣH;K (q : QSH ) = ΣK (hH (q)); ΣH (q)
denotes the surjective component of H: Let H be a named function, H = {q : QDH | ∃q : QSH .hH (q) = q }, – SH = SH and QDH – |q|D = |q|DH , GD (q) = GDH (q), hH (q) = hH (q) and ΣH (q) = ΣH (q)
H
H
behaviour of state q : SH , i.e. the transitions departing from q. Since states are equipped with local names, a name correspondence is needed to describe how names in the destination state are mapped into names of the source state, therefore we must equip H with a set ΣH (q) of injective functions. However, names of corresponding states (q, hH (q)) in hH are defined up to permutation groups and name correspondence must not be “sensible” to the local meaning of names. Therefore, the whole set ΣH (q) must be generated by saturating any of its elements by the permutation group of hH (q), and the result must be invariant with respect to the permutation group of q. Condition (1) in Table 1 states that the group of hH (q) does not change the meaning of names in hH (q), while Condition (2) states that the group of q does not “generate meanings” for local names of q that are outside hH (q).
From Co-algebraic Specifications to Implementation: The Mihda Toolkit
327
Table 2. Bundles: the π-calculus case Bundles A bundle β consists of the structure β = D : NSet, Step : ℘(qd D) where qd D is the set of quadruples of the form , π, σ, q given by inj
qd D = { : Lπ , π : N −→ {v1 ..}, σ :
inj
{q}D → Q, q : QD }.
∈Lπ
and Q =
N if ∈ {BOU T, BIN } N if ∈ {BOU T, BIN }
under the constraint that GDβ (q); Sq = Sq , where Sq = {, π, σ, q ∈ Stepβ } and ρ; , π, σ, q = , π, ρ; σ, q.
Bundle names Let β be a bundle. Function {| |} : B → N , mapping each bundle to the set of its names, is defined by {| β |} =
rng(π) ∪ rng(σ) \ {∗}
,π,σ,q∈Stepβ
where rng yields the range of functions. We only consider bundles β such that {| β |} is finite and we let β to indicate the number of names which occur in the bundle β (i.e. β = |{| β |}|).
3.1
Bundles over π-Calculus Actions
To represent the minimization algorithm for the early semantics of π-calculus [10], the notion of bundle must be enriched. Labels of transitions must distinguish among the different meanings of names occurring in π-calculus actions, namely synchronization, bound/free output and bound/free input. The set of π-calculus labels Lπ is {T AU, BOU T, OU T, BIN, IN }. We specify two different labels for input actions: Label BIN is used when the input transition exposes a fresh name, while label IN handles the case of an input transition that exposes a name of the source state of the transition. Labels in Lπ have weights. The weight map | | : Lπ → {∅, {1}, {1, 2}} is defined as: |T AU | = ∅
|BOU T | = |BIN | = {1}
|OU T | = |IN | = {1, 2}
and associates the set of indexes of distinct names the label refers to. A bundle on π-labels is defined as in Table 2. Table 2 illustrates definitions of bundles and names of bundles. As it is the case for ordinary automata, the Step component of a bundle specifies the data structure that contains the set of successor states for a given source state. More precisely, if , π, σ, q ∈ qd D,
328
G. Ferrari et al.
then q is the destination state; is the label of the transition; π associates to the label the names observed in the transition; and σ states how names in the destination state are related to the names in the source state. According to the definition of σ in Table 2, a name in a destination state of a quadruple is mapped on the distinguished name ∗ only on transitions where a new name is created (i.e. transitions labelled by BOU T or BIN ). In order to exploit named functions for representing HD-automata it is necessary to equip the set of bundles B with a named set structure. In other words we must define – a total order on bundles, – a function that maps a bundle to its number of names, – a group of permutations over those names. The names of a bundle are the names (different from ∗) that appear either in the labels or in the range of σ’s of the quadruples of the bundle. Without loss a generality, we can assume that a total order on states and labels exist. Hence, quadruples are totally ordered1 . The order over quadruples yields an ordering over bundles. bij The group of β : B Lπ is the set of permutations θ : {| β |} −→ {| β |} such that β; θ = β where β; θ is defined as Dβ , {, π; θ, σ; θ, q | , π, σ, q : β}. 3.2
Normalizing Bundles
In the minimization algorithm two states belong to the same block (partition) whenever they have the “same” bundles. Hence, the most crucial construction on bundles is the normalization operation. This operation is necessary for two different reasons. The first reason is that there are different equivalent ways for picking up the step components (i.e. quadruples , π, σ, q) of a bundle. The second (more important) reason consists of removing from the step component of a bundle all the redundant input transitions. Indeed, redundant transitions occur when a HD-automaton is built from a π-calculus agent. During this phase, it is not possible to decide which free input transitions are required, and which transitions are irredundant2 . The solution to this problem consists of adding a superset of the required free input transitions and to exploit a reduction function to remove the unnecessary ones during the minimization phase. Consider for instance the case of a state q having only one name v1 and assume that the following two tuples appear in a bundle: IN, xy, {v1 → y}, q
and
BIN, x, {v1 → ∗}, q.
Then, the IN transition is redundant if y is not active in q as it expresses exactly the same behaviour of the second tuple, except that a “free” input transition 1 2
For instance, we can assume the lexicographic order of labels, states and names. In the general case, to decide whether a free input transition is required it is as difficult as to decide the bisimilarity of two π-calculus agents.
From Co-algebraic Specifications to Implementation: The Mihda Toolkit
329
is used rather than a “bound” one. Hence, the transformation removes the first tuple from the bundle. During the iterative execution of the minimization algorithm, bundles are split; hence the set of redundant transitions of bundles decreases. Hence, when the iterative construction terminates, only those free inputs that are really redundant have been removed from the bundles. The normalization of a bundle β is done in different steps. First, the bundle is reduced by removing all the possibly redundant input transitions. Reduction function red(β) on bundles is defined as follows: – Dred(β) = Dβ , – Stepred(β) = Stepβ \ {IN, xy, σ, q | BIN, x, σ , q : Stepβ ∧ σ = σ; {y → ∗}}.
where σ; {y → ∗} is the function equal to σ on any name different from y and that assigns ∗ to y. Second, the normalization function norm(β) is defined as follows: – Dnorm(β) = Dβ – Stepnorm(β) = min Stepβ \ {IN, xy, σ, q | y ∈ anβ } ,
where anβ = {| red(β) |} is the active names of β and min is the function that, when applied to Stepβ , returns the step of the minimal bundle (with respect to order ) among those obtained by permuting names of β in all possible ways. More precisely, given a bundle β, min β is the minimal bundle in the set bij
{β; θ | θ : {| β |} −→ {| β |}}, with respect to the total ordering of bundles over D. The order relation is used to define the canonical representatives of bundles and relies on the order of quadruples. Hereafter, we use perm(β) to denote the canonical permutation that associates Stepnorm(β) and Stepβ \ {IN, xy, σ, q | y ∈ anβ }. We remark that, while all IN transitions covered by BIN transitions are removed in the definition of red(β), only those corresponding to the reception of non-active names are removed in the definition of norm(β). In fact, even if an input transition is redundant, it might be the case that it corresponds to the reception of a name that is active due to some other transitions. Finally, we need a construction which extracts in a canonical way a group of permutations out of a bundle. Let β be a bundle, define Gr β to be the set {ρ | Stepβ ; (ρ[∗ /∗ ]) = Stepβ }. It can be proved that Gr β is a group of permutations. 3.3
The Minimization Algorithm
We are now ready to give the definition of the functor T that states the coalgebras for HD-automata. The action of functor T over named sets is given by: – – – –
QT (A) = {β : Bundle | Dβ = A, β normalized}, |β|T (A) = β, GT (A) (β) = Gr β, β1 ≤T (A) β2 iff Stepβ1 Stepβ2 ,
while the action of functor T over named functions is given by:
330
G. Ferrari et al.
– ST (H) = T (SH ), DT (H) = T (DH ), – hT (H) (β : QT (SH ) ) : QT (DH ) = norm(β ), : → {| norm(β ) |}{β}T (SH ) – ΣT (H)(β : QT (SH ) ) = Gr(norm(β ));(perm(β ))−1;inj − where β = DH , {, π, σ ; σ, hH (q) | , π, σ, q : Stepβ , σ : ΣH (q)}.
Notice that functor T maps every named set A into the named set T (A) of its normalized bundles. A named function H is mapped into a named function T (H) in such a way that every corresponding pair (q, hH (q)) in hH is mapped into a set of corresponding pairs (β, norm(β )) of bundles in hT (H) . The quadruples of bundle β are obtained from those of β by replacing q with hH (q) and by saturating with respect to the set of name mappings in ΣH (q). The name mappings in ΣT (H) (β) are obtained by transforming the permutation group of bundle norm(β ) with the inverse of the canonical permutation of β and with a fixed injective function inj mapping the set of names of norm(β ) into the set of names of β, defined as i < j, inj(vi ) = vi and inj(vj ) = vj implies i < j . Without bundle normalization, the choice of β among those in β ; θ would have been arbitrary and not canonical with the consequence of mapping together fewer bundles than needed. Definition 4 (Transition systems for π-agents). A transition system over named sets and π-actions is a named function K such that DK = T (SK ). HD-automata are particular transition systems over named sets. An HD- automaton A is given by: – – – –
the elements of QA are π-agents and ≤A is the lexicographic order on QA ; |p(v1 , ..., vn )|A = n; GA (q) = {id : {q}A −→ {q}A }, where id denotes the identity function, h : QA −→ {β | Dβ = A} is such that , π, σ, q ∈ Steph(q) represent the π-calculus transitions from agent q. ,π,σ
We will often use the notation q −−→ q to denote the “representative” transitions from agent q that are used in the construction of the HD-automaton. We can now define the function K. – SK = A, – hK (q) = norm(h(q)), – ΣK (q) = Gr(hK (q)); (perm(h(q)))−1 ; inj : {| h(q) |} −→ {q}A The minimal HD-automata is built by an iterative procedure on K: the iteration along the terminal sequence. The formula which details the iterative construction is given by Hi+1 = K;T (Hi ). If K is a finite state HD-automaton. Then The initial approximation, H0 , is defined as follows:
From Co-algebraic Specifications to Implementation: The Mihda Toolkit
331
Block
Domination
Bundle
Automaton
Transitions
Labels
States Fig. 3. Mihda Software Architecture
– SH0 = SK , DH0 = unit where Qunit = {∗}, |∗|unit = 0 (and hence {∗} = φ), Gunit ∗ = φ, and ∗ ≤unit ∗, – hH0 (q : QsH0 ) = ∗, – ΣH0 (q) = {φ} We recall that the iteration along the terminal sequence converges in a finite number of steps: i exists such that DHi+1 ≡ DHi , and the isomorphism mapping F : DHi → DHi+1 yields the minimal realization of the transition system K up to strong early bisimilarity.
4
The Mihda Toolkit
The previous sections outlined the coalgebraic foundation for the finite state verification of name passing process calculi. It remains to show that this theory can be effectively used as a basis for the design and development of effective and usable verification toolkits. This section and the one following explore this issue by describing our experience in designing, implementing and experimenting a minimization toolkit, called Mihda, for verifying finite state mobile systems represented in the π-calculus. The Mihda toolkit cleanly separates facilities that are language-specific (parsing, transition system calculation) from those that are independent from the calculus notation (bisimulation) in order to ease modifications. The toolkit has been implemented in ocaml. Indeed, the partition refinement algorithm has been specified in a “type-theoretic” style and the underlying type system makes use of parametric polymorphism. The type system of ocaml offers all the necessary features for handling these kind of types. Figure 3 illustrates the modules of Mihda and their dependencies.
332
G. Ferrari et al.
For instance, State is the module which provides all the structures for handling states and its main type defines the type of the states of the automata. Domination is the module containing the structures underlying bundle normalization. The connections express typing relationships among the modules. For instance, since states in bundles and transitions must have the same type, then a connection exists between modules Bundle and Transitions. Notice that the iterative construction of the minimal automaton is parameterized with respect to the modules of Figure 3. Indeed, the same algorithm can be applied to different kind of automata and bisimulation, provided that these automata match the constraints on types imposed by the software architecture. For instance, the architecture of Mihda has been exploited to provide minimization of both HD-automata and ordinary automata (up to strong bisimilarity). 4.1
The Main Cycle
We have already pointed out that the iterative step of the minimization algorithm can be represented in a functional style form as follows: ,π,σ
hHi+1 (q) = norm DHi , {, π, σ ; σ, hHi (q ) | q −−→ q , σ : ΣHi (q )}.
(1)
We compute hHi+1 (q) through the following steps: (a) determine the bundle of state q; (b) for each quadruple , π, σ, q in this bundle, apply hHi to q , the target state of the quadruple (yielding the bundle of q in the previous iteration of the algorithm); (c) left-compose symmetry σ ∈ Σ(q ) with σ; (d) normalize the resulting bundle. In the Mihda implementation the value of the i-th iteration (i.e. hHi ) is stored in a list of blocks which are the crucial data structures of Mihda. Blocks implements the action of the functor on states of the automata and contain all those information needed to compute the iteration steps of the algorithm expressed in a set theoretic framework. Blocks represent both (finite) named functions and partitions of an automaton (at each iteration of the algorithm). When the algorithm terminates, each block will correspond to a state of the minimal automaton. A block has the following structure: type Block t = Block of id : string ∗ states : State t list ∗ norm : Bundle t ∗ names : int list ∗ group : int list list ∗ Σ : (State t → (int ∗ int) list list) ∗ Θ −1 : (State t → (int ∗ int) list)
From Co-algebraic Specifications to Implementation: The Mihda Toolkit
333
q
θq
x
Fig. 4. Graphical representation of a block
Field id is the name of the block and is used to identify the block in order to construct the minimal automaton at the end of the algorithm. Field states contains the states which are considered equivalent. The remaining fields represent – – – –
the normalized bundle with respect to the block considered as state (norm), names is the list of names of the bundle in norm, group is its group, function Θ −1 , given a state q, maps the names appearing in norm into the name of q. Basically, Θ−1 (q) is the function which establishes a correspondence between the bundle of q and the bundle of the corresponding representative element in the equivalence class of the minimal automaton.
We pictorially represent (some components of) a block as in Figure 4: The upper elements are the states in the block, while the element x is the “representative state”, namely it is a graphical representation of the block as a state. For each state q a function θq maps names of x into the names of q. Function θq describes “how” the block approximates the state q at a given iteration. The circled arrow on x aims at recording that a block also has symmetries on its names. Bundle norm of block x is computed by exploiting the ordering relations over names, labels and states. A graphical representation of steps (a)-(d) above in terms of blocks is illustrated in Figure 5. Step (a) is computed by the facility Automaton.bundle that filters all transitions of the automaton whose source corresponds to q. Figure 5(a) shows that a state q is taken from a block and its bundle is computed. Step (b) is obtained by applying facility Block.next to the bundle of q. The operation Block.next substitutes all target states of the quadruples with the corresponding current block and computes the new mappings (see Figure 5(b)). Step (c) does not seem to correctly adhere to the corresponding step of equation 1. However, if we consider that θ functions are computed at each step by composing symmetries σ’s we can easily see that θ functions exactly play the rˆole of σ’s. Finally, step (d) is represented in Figure 5(d) and is obtained via the function Bundle.normalize.
334
G. Ferrari et al.
The main step of the minimization algorithm is the function split that computes, at each iteration, the current partition (the list of blocks). let split blocks block = try let minimal = (Bundle.minimize red (Block.next (h n blocks) (state of blocks) (Automaton.bundle aut (List.hd (Block.states block))))) in Some (Block.split minimal (fun q → let normal = (Bundle.normalize red (Block.next (h n blocks) (state of blocks) (Automaton.bundle aut q))) in Bisimulation.bisimilar minimal normal) block) with Failure e → None
Fig. 5. Computing hHi+1
From Co-algebraic Specifications to Implementation: The Mihda Toolkit
335
let blocks = ref [ (Block.from states states) ] in let stop = ref false in while not ( !stop ) do begin let oldblocks = !blocks in let buckets = split iter (split oldblocks) oldblocks in begin blocks := (List.map (Block.close block (h n oldblocks)) buckets); stop := (List.length !blocks) = (List.length oldblocks) && (List.for all2 (fun x y → (Block.compare x y) == 0) !blocks oldblocks) end end done ; !blocks Fig. 6. The main cycle of Mihda
Let block be a block in the list blocks, function split computes minimal by minimizing the reduced bundle of the first state of block. The choice of the state for computing minimal is not important: Without loss of generality, given two equivalent states q and q’, it is possible to map names of q into names of q’ preserving their associated normalized bundle if, and only if, a similar map from names of q’ into names of q exists. Once minimal has been computed, split invokes Block.split with parameters minimal and block, while the second argument of Block.split is a function that computes the current normalized bundle of each state in block and checks whether or not it is bisimilar to minimal. This computation is performed by function Bisimulation.bisimilar. If bisimilarity holds through θq then Some θq is returned, otherwise None is returned. We are now ready to comment on the main cycle of Mihda reported in Figure 6. Let k = (start, states, arrows) be a HD-automaton. When the algorithm starts, blocks is the list that contains a single block collecting all the states of the automata k. At each iteration, the list of blocks is splitted, as much as, possible by split iter that returns a list of buckets which have the same fields of a block apart from the name, symmetries and the functions mapping names of destination states into names of source states. Essentially, the split operation checks if two states in a block are equivalent or not. States which are no longer equivalent to the representative element of the block are removed and inserted into a bucket. Then, by means of Block.close block, all buckets are turned into blocks which are assigned to blocks. Finally, the termination condition stop is evaluated. This condition is equivalent to say that a bijection can be established between oldblocks (that corresponds to Di ) and blocks (corresponding to Di+1 ). This
336
G. Ferrari et al.
condition reduces to test whether blocks and oldblocks have the same length and that blocks at corresponding positions are equal.
5
Verifying Mobile Systems with Mihda
In this section we discuss some experimental results of Mihda in the analysis of mobile systems. In particular, we consider the π-calculus specification of the Handover Protocol for Mobile Telephones borrowed from that given in [17] (which has been in turn derived from that in [13]). The π-calculus specification of the GSM is define GSM(in,out) = (tca)(ta)(ga)(sa)(aa)(tcp)(tp)(gp)(sp)(ap) |( Car(ta,sa,out), Base(tca,ta,ga,sa,aa), IdleBase(tcp,tp,gp,sp,ap), Centre(in,tca,ta,ga,sa,aa,tcp,tp,gp,sp,ap))
Centre receives messages from the environment on channel in; these input actions are the only observable actions performed by Centre. Module Car sends the messages to the end user along the channel out; these outputs are the only visible actions performed by the Car. Modules Centre and Car interact via the base corresponding to the cell in which the car is located. The specification of modules Car, Base, IdleBase and Centre is reported in Table 3. The behaviour of the four modules is briefly summarized below: – Car brings a MobileStation and travels across two different geographical areas that provide services to end users; – Base and IdleBase are Base Station modules; they interconnect the MobileStation and the MobileSwitching Centre. – Centre is a MobileSwitching centre which controls radio communications within the whole area composed by the two cells; The protocol starts when Car moves from one cell to the other. Indeed, Centre communicates to the MobileStation the name of the base corresponding to the new cell. The communication of the new channel name to the MobileStation is performed via the current base. All the communications between the MobileSwitching centre and the MobileStation are suspended until the MobileStation receives the names of the new transmission channels. Then the base corresponding to the new cell is activated, and the communications between the MobileSwitching centre and the MobileStation continue through the new base. In Table 4 we report the results of Mihda on two different versions of the protocols. The first row of the table corresponds to the version discussed above. The second line gives the figures on a version of the GSM protocol that models the MobileSwitching and MobileStation modules in a more realistic way. Indeed, the ’full’ version exploits a protocol for establishing whether or not the car is crossing the boundary of a cell and entering the other cell.
From Co-algebraic Specifications to Implementation: The Mihda Toolkit
337
Table 3. π-calculus specification of GSM modules define Car(talk,switch,out) = talk?(msg).out!msg.Car(talk,switch,out) + switch?(t).switch?(s).Car(t,s,out) define Base(talkcentre,talkcar,give,switch,alert) = talkcentre?(msg).talkcar!msg. Base(talkcentre,talkcar,give,switch,alert) + give?(t).give?(s).switch!t.switch!s.give!give. IdleBase(talkcentre,talkcar,give,switch,alert) define IdleBase(talkcentre,talkcar,give,switch,alert) = alert?(empty).Base(talkcentre,talkcar,give,switch,alert) define Centre(in,tca,ta,ga,sa,aa,tcp,tp,gp,sp,ap) = in?(msg).tca!msg.Centre(in,tca,ta,ga,sa,aa,tcp,tp,gp,sp,ap) + tau.ga!tp.ga!sp.ga?(empty).ap!ap. Centre(in,tcp,tp,gp,sp,ap,tca,ta,ga,sa,aa)
Table 4. Mihda at work Protocol Time to compile States Transitions Time to minimize States Transitions GSM small 0m 0.931s 211 398 0m 4.193s 105 197 GSM full 0m 8.186s 964 1778 0m 54.690s 137 253
The results are obtained by running Mihda on an AMD AthlonTM XP 1800+ dual processor with 1Giga RAM. The time for minimizing the automata is very contained. The results on the GSM seem very promising. Indeed, the size of the minimal automata in terms of states and transitions is smaller than their nonminimized version. In the case of GSM small the size of the minimal automaton is the half or the automaton obtained by compiling the original specification; while, in version GSM full, states and transitions are reduced of a factor 8.
6
Conclusion
This paper has provided an overview of a foundational model for the finite state verification of global computing systems and has showed how efficient tool supports can be derived from it. We are currently extending the Mihda toolkit with facilities to handle other notions of equivalences (e.g. open bisimilarity) and other foundational calculi for global computing (e.g. the asynchronous π-calculus, the fusion calculus). To improve efficiency, we plan to incorporate software supports for symbolic approaches based on Binary Decision Diagrams.
338
G. Ferrari et al.
References 1. Peter Aczel. Algebras and coalgebras. In Roy Backhouse, Roland Crole and Jeremy Gibbons, editors, Algebraic and Coalgebraic Methods in the Mathematics of Program Construction, volume 2297 of LNCS, chapter 3, pages 79–88. Springer Verlag, April 2002. Revised Lectures of the Int. Summer School and Workshop. 2. Edmund M. Clarke and Jeanette M. Wing. Formal methods: state of the art and future directions. ACM Computing Surveys, 28(4):626–643, December 1996. 3. Jean Claude Fernandez. An implementation of an efficient algorithm for bisimulation equivalence. Science of Computer Programming, 13:219–236, 1990. 4. GianLuigi Ferrari, Ugo Montanari, and Marco Pistore. Minimizing transition systems for name passing calculi: A co-algebraic formulation. In Mogens Nielsen and Uffe Engberg, editors, FOSSACS 2002, volume LNCS 2303, pages 129–143. Springer Verlag, 2002. 5. Marcelo Fiore, Gordon G. Plotkin, and Daniele Turi. Abstract syntax and variable binding. In 14th Annual Symposium on Logic in Computer Science. IEEE Computer Society Press, 1999. 6. Murdoch J. Gabbay and Andrew M. Pitts. A new approach to abstract syntax involving binders. In 14th Annual Symposium on Logic in Computer Science. IEEE Computer Society Press, 1999. 7. Bart Jacobs and Jan Rutten. A tutorial on (co)algebras and (co)induction. Bulletin of the EATCS, 62:222–259, 1996. 8. Paris C. Kanellakis and Scott A. Smolka. Ccs expressions, finite state processes and three problem of equivalence. Information and Computation, 86(1):272–302, 1990. 9. Robin Milner. Commuticating and Mobile Systems: the π-calculus. Cambridge University Press, 1999. 10. Robin Milner, Joachim Parrow, and David Walker. A calculus of mobile processes, I and II. Information and Computation, 100(1):1–40,41–77, September 1992. 11. Ugo Montanari and Marco Pistore. History dependent automata. Technical report, Computer Science Department, Universit` a di Pisa, 1998. TR-11-98. 12. Ugo Montanari and Marco Pistore. π-calculus, structured coalgebras and minimal hd-automata. In Mathematical Foundations of Computer Science 2000, volume 1893. Springer, 2000. 13. Fredrik Orava and Joachim Parrow. An algebraic verification of a mobile network. Formal Aspects of Computing, 4(5):497–543, 1992. 14. Marco Pistore. History dependent automata. PhD thesis, Computer Science Department, Universit` a di Pisa, 1999. 15. Andrew M. Pitts and Murdoch J. Gabbay. A metalanguage for programming with bound names modulo renaming. In Mathematics of Program Construction, 5th International Conference, MPC2000, volume 1837. Springer, 2000. 16. Davide Sangiorgi and David Walker. The π-calculus: a Theory of Mobile Processes. Cambridge University Press, 2002. 17. Bj¨ orn Victor and Faron Moller. The Mobility Workbench — a tool for the πcalculus. In David Dill, editor, Proceedings of CAV ’94, volume 818 of Lecture Notes in Computer Science, pages 428–440. Springer-Verlag, 1994.
A Calculus for Modeling Software Components Oscar Nierstrasz and Franz Achermann Software Composition Group University of Bern, Switzerland http://www.iam.unibe.ch/∼scg
Abstract. Many competing definitions of software components have been proposed over the years, but still today there is only partial agreement over such basic issues as granularity (are components bigger or smaller than objects, packages, or application?), instantiation (do components exist at run-time or only at compile-time?), and state (should we distinguish between components and “instances” of components?). We adopt a minimalist view in which components can be distinguished by composable interfaces. We have identified a number of key features and mechanisms for expressing composable software, and propose a calculus for modeling components, based on the asynchronous π calculus extended with explicit namespaces, or “forms”. This calculus serves as a semantic foundation and an executable abstract machine for Piccola, an experimental composition language. The calculus also enables reasoning about compositional styles and evaluation strategies for Piccola. We present the design rationale for the Piccola calculus, and briefly outline some of the results obtained.
1
Introduction
What is a software component? What are the essential aspects of ComponentBased Software Development? What is a suitable foundation for modeling and reasoning about CBSD? To the first question, one of the most robust and appealing answers has been: “A software component is a unit of independent deployment without state.” [43] This simple definition captures much that is important, though it leaves some very important aspects implicit. First, CBSD attempts to streamline software development and evolution by separating what is stable from what is not. That is, components are not just “independently deployable”, but they must encapsulate a stable unit of functionality. This, of course, begs the question, “If components are the stable stuff, what makes up the rest?” Second, “independent deployment” of components actually entails compliance with some well-defined component model in which components present their services as a set of interfaces or “plugs”: “A software component is a static abstraction with plugs.” [31] F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 339–360, 2003. c Springer-Verlag Berlin Heidelberg 2003
340
O. Nierstrasz and F. Achermann
This leads us to answer the question, “What makes up the rest?” as follows: Applications = Components + Scripts [6] that is, component-based applications are (ideally) made up of stable, off-theshelf components, and scripts that plug them together. Scripts (ideally) make use of high-level connectors that coordinate the services of various components [3,29,42]. Furthermore, complex applications may need services of components that depend on very different architectural assumptions [16]. In these cases, glue code is needed to adapt components to different architectural styles [40,41]. Returning to our original questions, then, we conclude that it is not really possible to define software components without taking these complementary aspects of CBSD into account. At a purely technical level, i.e., ignoring methodological and software process aspects, these aspects include styles (plugs and connectors), scripts, coordination and glue code. A formal foundation for any reasonable notion of software components must address these aspects. We claim that most of these aspects can be adequately addressed by the notion of forms — first-class, extensible namespaces. The missing aspect (coordination) can be addressed by agents and channels. We propose, therefore, a calculus for modeling composable software which is based on the asynchronous π calculus [25,36] extended with first-class namespaces [5]. This calculus serves both as the semantic target and as an executable abstract machine for Piccola, an experimental composition language for implementing styles, scripts, coordination abstractions and glue code [4,6]. The Piccola calculus is described in greater detail in Achermann’s PhD dissertation [2]. In this paper we first motivate the calculus by establishing a set of requirements for modeling composition of software components in section 2. Next, we address these requirements by presenting the syntax and semantics of the Piccola calculus in section 3. In section 4 we provide a brief overview of Piccola, and summarize how the calculus helps us to define its semantics, reason about composition, and optimize the language bridge by partial evaluation while preserving its semantics. Finally, we conclude with a few remarks about related and ongoing work in sections 5 and 6.
2
Modeling Software Composition
As we have seen, a foundation for modeling software components must also be suitable for expressing compositional styles, scripts, coordination abstractions and glue code. Let us examine each of these in turn to see which requirements they pose. Figure 1 summarizes these requirements, and illustrates how Piccola and the Piccola calculus support them.
A Calculus for Modeling Software Components
341
Glue Piccola • extensible, immutable records • first-class, monadic services • language bridging • introspection • explicit namespaces • services as operators • dynamic scoping on demand • agents & channels
• generic wrappers • component packaging • generic adaptors
Styles • primitive neutral object model • meta-objects • HO plugs & connectors • default arguments • encapsulation • component algebras
Scripts Coordination • coordination abstractions
• sandboxes • composition expressions • context-dependent policies
Fig. 1. How Piccola supports composition.
2.1
Compositional Styles
A compositional style allows us to express the structure of a software application in terms of components, connectors and rules governing their composition (cf. “architectural style” [42]). – Neutral object model: There exists a wide variety of different object and component models. Components may also be bigger or smaller than objects. As a consequence, a general foundation for modeling components should make as few assumptions about objects, classes and inheritance as possible, namely, objects provide services, they may be instantiated, and their internal structure is hidden. – Meta-objects: On the other hand, many component models depend on runtime reflection, so it must be possible to express dynamic generation of metaobjects. – Higher-order plugs and connectors: In general, connectors can be seen as higher-order operators over components and other connectors. – Default arguments: Flexibility in plugging together components is achieved if interface dependencies are minimized. Keyword-based rather than positional arguments to services enable both flexibility and extensibility. – Encapsulation: Components are black-box entities that, like objects, provide services, without exposing their structure. At the same time, the components and connectors of a particular style can be encapsulated as a module, or namespace within which components may be scripted.
342
O. Nierstrasz and F. Achermann
Fig. 2. Evaluating the helloButton script.
– Component algebras: Compositional styles are most expressive when compositions of components and connectors again yield components (or connectors). (The composition of two filters is again a filter.) Based on these requirements, we conclude that we need (at least) records (to model objects and components), higher-order functions, reflection, and (at some level) overloading of operators. Services may be monadic, taking records as arguments, rather than polyadic. To invoke a service, we just apply it to a record which bundles together all the required arguments, and possibly some optional ones. These same records can serve as first-class namespaces which encapsulate the plugs and connectors of a given style. For this reason we unify records and namespaces, and call them “forms”, to emphasize their special role. A “form” is essentially a nested record, which binds labels to values. Consider, for example, the following JPiccola script [30]: makeFrame title = "AWT Demo" x = 200 y = 100 hello = "hello world" sayHello: println hello component = Button.new(text=hello) ? ActionPerformed sayHello This script invokes an abstraction makeFrame, passing it a form containing bindings for the labels title, x, and so on. The script makes use of a compositional style in which GUI components (i.e., the Button) can be bound to events (i.e., ActionPerformed) and actions (i.e., sayHello) by means of the ? connector. When we evaluate this code, it generates the button we see in figure 2. When we click on the button, hello world is printed on the Java console. 2.2
Glue
Glue code is needed to package, wrap or adapt code to fit into a compositional style. – Generic wrappers: Wrappers are often needed to introduce specific policies (such as thread-safe synchronization). Generic wrappers are hard to specify for general, polyadic services, but are relatively straightforward if all services are monadic.
A Calculus for Modeling Software Components
343
– Component packaging: Glue code is sometimes needed to package existing code to conform to a particular component model or style. For this purpose, a language bridge is needed to map existing language constructs to the formal component model. – Generic adaptors: Adaptation of interfaces can also be specified generically with the help of reflective or introspective features, which allow components to be inspected before they are adapted. The JPiccola helloButton script only works because Java GUI components are wrapped to fit into our compositional style. In addition to records and higher-order functions over records, we see that some form of language bridging will be needed, perhaps not at the level of the formal model, but certainly for a practical language or system based on the model. 2.3
Scripts
Scripts configure and compose components using the connectors defined for a style. – Sandboxes: For various reasons we may wish to instantiate components only in a controlled environment. We do not necessarily trust third-party components. Sometimes we would like to adapt components only within a local context. For these and other reasons it is convenient to be able to instantiate and compose namespaces which serve as sandboxes for executing scripts. – Composition expressions: Scripts instantiate and connect components. A practical language might conveniently represent connectors as operators. Pipes and filters connections are well-known, but this idea extends well to other domains. – Context-dependent policies: Very often, components must be prepared to employ services of the dynamic context. Transaction services, synchronization or communication primitives may depend on the context. For this reason, pure static scoping may not be enough, and dynamic scoping on demand will be needed for certain kinds of component models. So, we see that explicit, manipulable namespaces become more important. 2.4
Coordination
CBSD is especially relevant in concurrent and distributed contexts. For this reason, a foundation for composition must be able to express coordination of interdependent tasks. – Coordination abstractions: Both connectors and glue code may need to express coordination of concurrent activities. Consider a readers/writers synchronization policy as a generic wrapper. We conclude that we not only need higher-order functions over first-class namespaces (with introspection), but also a way of expressing concurrency and communication [40].
344
O. Nierstrasz and F. Achermann Table 1. Syntax of the Piccola Calculus. A, B, C ::= | | | | | |
A; B → x L λx.A νc.A c?
empty form sandbox bind inspect abstraction restriction input
| | | | | | |
F, G, H ::= →F | x
empty form binding
|S service | F · G extension
S ::= F ; λx.A closure → | x bind | c output
3
R x hide x A·B AB A|B c
current root variable hide extension application parallel output
|L inspect | hide x hide
The Piccola Calculus
As a consequence of the requirements we have identified above, we propose as a foundation a process calculus based on the higher-order asynchronous π calculus [25,36] in which tuple-based communication is replaced by communication of extensible records, or forms [5]. Furthermore, forms serve as first-class namespaces and support a simple kind of introspection. The design of the Piccola calculus strikes a balance between minimalism and expressiveness. As a calculus it is rather large. In fact, it would be possible to express everything we want with the π calculus alone, but the semantic gap between concepts we wish to model and the terms of the calculus would be rather large. With the Piccola calculus we are aiming for the smallest calculus with which we can conveniently express components, connectors and scripts. 3.1
Syntax
The Piccola calculus is given by agents A, B, C that range over the set of agents A in 1. There are two categories of identifiers: labels and channels. The set of labels L is ranged over by x, y, z. (We use the term “variables” and “labels” interchangeably.) Specific labels are also written in the italic text font. Channels are denoted by a, b, c, d ∈ N . Labels are bound with bindings and λ-abstractions, and channels are bound by ν-restrictions. The operators have the following precedence: application > extension > restriction, abstraction > sandbox > parallel Agent expressions are reduced to static form values or simply forms. Forms are ranged over by F, G, H. Notice that the set of forms is a subset of all agents. Forms are the first-class citizens of the Piccola calculus, i.e., they
A Calculus for Modeling Software Components
345
are the values that get communicated between agents and are used to invoke services. Forms are sets of bindings and services. The set of forms is denoted by F. Certain forms play the role of services. We use S to range over services. User-defined services are closures. Primitive services are inspect, the bind and hide primitives, and the output service. Before considering the formal reduction relation, we first give an informal description of the different agent expressions and how they reduce. – The empty form, , does not reduce further. It denotes a form without any binding. – The current root agent, R, denotes the current lexical scope. – A sandbox A; B evaluates the agent B in the root context given by A. A binds all free labels in B. If B is a label x, we say that A; x is a projection on x in A. – A label, x, denotes the value bound by x in the current root context. →A – The primitive service bind creates bindings. If A reduces to F , then x →F . reduces to the binding x → · y →) reduces – The primitive service hide x removes bindings. So, hide x (x →. to y – The inspect service, L, can be used to interate over the bindings and services of an arbitrary form F . The result of LF is a service that takes as its argument a form that binds the labels isEmpty, isService and isLabel to services. One of these three services will then be selected, depending on whether F is , contains some bindings, or is only a service. – The values of two agents are concatenated by extension. In the value of A · B the bindings of B override those for the same label in A. – An abstraction λx.A abstracts x in A. – The application AB denotes the result of applying A to B. Piccola uses a call-by-value reduction order. In order to reduce AB, A must reduce to a service and B to a form. – The expression νc.A restricts the visibility of the channel name c to the agent expression A, as in the π calculus. – A | B spawns off the agent A asynchronously and yields the value of B. Unlike in the π calculus, the parallel composition operator is not commutative, since we do not wish parallel agents to reduce to non-deterministic values. – The agent c? inputs a form from channel c and reduces to that value. The reader familiar with the π-calculus will notice a difference with the input prefix. Since we have explicit substitution in our calculus it is simpler to specify the input by c? and use the context to bind the received value instead of defining a prefix syntax c(X).A as in the π-calculus. – The channel c is a primitive output service. If A reduces to F , then cA reduces to the message cF . The value of a message is the empty form . (The value F is only obtained by a corresponding input c? in another agent.)
346
O. Nierstrasz and F. Achermann Table 2. Free Channels. fc() fc(x) →) fc(x fc(A; B) fc(λx.A) fc(νc.A) fc(c?)
3.2
= = = = = = =
∅ ∅ ∅ fc(A) ∪ fc(B) fc(A) fc(A)\{c} {c}
fc(R) fc(L) fc(hide x ) fc(A · B) fc(AB) fc(A | B) fc(c)
= = = = = = =
∅ ∅ ∅ fc(A) ∪ fc(B) fc(A) ∪ fc(B) fc(A) ∪ fc(B) {c}
Free Channels and Closed Agents
As in the π-calculus, forms may contain free channel names. An agent may create a new channel, and communicate this new name to another agent in a separate lexical scope. The free channels fc(A) of an agent A are defined inductively in table 2. αconversion (of channels) is defined in the usual way. We identify agent expressions up to α-conversion. We omit a definition of free variables. Since Piccola is a calculus with explicit environments, we cannot easily define α-conversion on variables. Such a definition would have to include the special nature of R. Instead, we define a closed agent where all variables, root expressions, and abstractions occur beneath a sandbox: Definition 1. The following agents A are closed: →, hide x , L, c and c? are closed. – , x – If A and B are closed then also A · B, AB, A | B and νc.A are closed. – If A is closed, then also A; B is also closed for any agent B.
Observe that any form F is closed by the above definition. An agent is open if it is not closed. Open agents are R, variables x, abstractions λx.A and compositions thereof. Any agent can be closed by putting it into a sandbox with a closed context. Sandbox agents are closed if the root context is closed. In lemma 1 we show that the property of being closed is preserved by reduction. 3.3
Congruence and Pre-forms
As in the π calculus, we introduce structural congruence over agent expressions to simplify the reduction relation. The congruence allows us to rewrite agent expressions to bring communicating agents into juxtapositions, as in the Chemical Abstract Machine of Berry and Boudol [8]. The congruence rules constitute three groups (see table 3). The first group (from ext empty right to single service) deals with congruence over forms. It specifies that extension is idempotent and associative on forms. The rules single service and single binding specify that extension overwrites services and bindings with the same label. We define labels(F ) as follows:
A Calculus for Modeling Software Components
347
Table 3. Congruences. ≡ is the smallest congruence satisfying the following axioms:
x =y
implies
x =y
implies
c∈ / fc(A) c∈ / fc(A) c∈ / fc(A) c∈ / fc(A) c∈ / fc(A) c∈ / fc(A) c∈ / fc(A) c∈ / fc(A)
implies implies implies implies implies implies implies implies
F · ·F (F · G) · H →F ) S · (x →F · y →G x →F · x →G x S · S F;A · B F ; AB A; (B; C) F;G F;R →G) hide x (F · x →G) hide y (F · x hide x hide x S (F · S)G (A | B) | C (A | B) | C (A | B) · C F · (A | B) (A | B)C F (A | B) (A | B); C F ; (A | B) F |A cF νcd.A A | νc.B (νc.B) | A (νc.B) · A A · νc.B A; νc.B (νc.B); A (νc.B)A A(νc.B)
≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡ ≡
F (ext empty right) F (ext empty left) F · (G · H) (ext assoc) →F ) · S (x (ext service commute) →G · x →F y (ext bind commute) →G x (single binding) S (single service) (F ; A) · (F ; B) (sandbox ext) (F ; A)(F ; B) (sandbox app) (A; B); C (sandbox assoc) G (sandbox value) F (sandbox root) hide x F (hide select) →G hide y F · x (hide over) (hide empty) S (hide service) SG (use service) A | (B | C) (par assoc) (B | A) | C (par left commute) A | B·C (par ext left) A | F ·B (par ext right) A | BC (par app left) A | FB (par app right) A | B; C (par sandbox left) F;A | F;B (par sandbox right) A (discard zombie) cF | (emit) νdc.A (commute channels) νc.(A | B) (scope par left) νc.(B | A) (scope par right) νc.(B · A) (scope ext left) νc.(A · B) (scope ext right) νc.(A; B) (scope sandbox left) νc.(B; A) (scope sandbox right) νc.BA (scope app left) νc.AB (scope app right)
Definition 2. For each form F , the set of labels(F ) ⊂ L is given by: labels() = ∅ →G) = {x} labels(x
labels(S) = ∅ labels(F · G) = labels(F ) ∪ labels(G)
348
O. Nierstrasz and F. Achermann
Using the form congruences, we can rewrite any form F into one of the following three cases: F ≡ F ≡S →G F ≡ F · x
where x ∈ labels(F )
This is proved by structural induction over forms [2]. This formalizes our idea that forms are extensible records unified with services. A form has at most one binding for a given label. The second group (from sandbox ext to use service) defines preforms. These are agent expressions that are congruent to a form. For instance, the agent hide x is equivalent to the empty form . The set of all preforms is defined by: F ≡ = {A|∃F ∈ F with F ≡ A} Clearly, all forms are preforms. The last group (from par assoc to scope app right) defines the semantics of parallel composition and communication for agents. Note how these rules always preserve the position of the rightmost agent in a parallel composition, since this agent, when reduced to a form, will represent the value of the composition. In particular, the rule discard zombie garbage-collects form values appearing to the left of this position. The rule emit, on the other hand, spawns an empty form as the value, thus enabling the message to move around freely. For instance in →c() ≡ x →(c() | ) x → ≡ c() | x
by emit by par ext right
→. the message c() escapes the binding x
3.4
Reduction
We define the reduction relation → on agent expressions to reduce applications, communications and projections (see table 4). ⇒ is the reflexive and transitive closure of →. Especially noteworthy is the rule reduce beta. This rule does not substitute G for x in the agent A as in the classical λ-calculus. Instead, it extends the environment in which A is evaluated. This is essentially the beta-reduction rule found in calculi for explicit substitution [1,32]: →G; A (F ; λx.A)G → F · x
The application of the closure F ; λx.A to the argument G reduces to a sandbox →G. Free expression in which the agent A is evaluated in the environment F · x occurrences of x in A will therefore be bound to G. The property of being closed is respected by reduction: Lemma 1. If A is a closed agent and A → B or A ≡ B then B is closed as well. Proof. Easily checked by induction over the formal proof for A → B.
A Calculus for Modeling Software Components
349
Table 4. Reduction rules.
→G; A (F ; λx.A) G → F · x
(reduce beta)
cF | c? → F
(reduce comm)
→G; x → G F · x
(reduce project)
L → ; λx.(x; isEmpty)
(reduce inspect empty)
LS → ; λx.(x; isService)
(reduce inspect service)
→G) → ; λx.(x; isLabel )label x L(F · x
(reduce inspect label)
A≡A
A →B
A→B
B ≡B A→B
E[A] → E[B]
(reduce struct) (reduce propagate)
→(; λx.(x; x)) · hide →hide x · bind →(x →) and E is an evaluation where label x = project context defined by the grammar: E ::= [ ] E · A F · E E; A F ; E EA F E A|E E|A νc.E
3.5
Encoding Booleans
The following toy example actually illustrates many of the principles at stake when we model components with the Piccola calculus. We can encode booleans by services that either project on the labels true or false depending on which boolean value they are supposed to model (cf. [13]). (This same idea is used by the primitive service L to reflect over the bindings and services of a form.) def
True = ; λx.(x; true) def
False = ; λx.(x; false)
(1) (2)
Consider now: →1 · false →2) = (; λx.(x; true))(true →1 · false →2) True(true →(true →1 · false →2); (x; true) → · x
by reduce beta →(true →1 · false →2); x); true by sandbox assoc ≡ ( · x →1 · false →2); true → (true by reduce project →2 · true →1); true ≡ (false by ext bind commute →1
by reduce project
350
O. Nierstrasz and F. Achermann
Note how the bindings are swapped to project on true in the last step. A →1 · false →2) ⇒ 2. similar reduction would show False(true One of the key points of forms is that a client can provide additional bindings which are ignored when they are not used (cf. [13]). This same principle is applied to good effect in various scripting languages, such as Python [22]. For instance →F for arbitrary we can use True and provide an additional binding notused form F : →1 · false →2 · notused →F ) True(true →1 · false →2 · notused →F ); true ⇒ (true →2 · true →1 · notused →F ); true ≡ (false →2 · notused →F · true →1); true ≡ (false →1
by ext bind commute by ext bind commute by reduce project
Extending forms can also be used to overwrite existing bindings. For instance instead of binding the variable notused a client may override true: →1 · false →2 · true →3) ⇒ 3 True(true
A conditional expression is encoded as a curried service that takes a boolean and a case form. When invoked, it selects and evaluates the appropriate service in the case form: def
→(v; then) · false →(v; else)) if = ; λuv.u(true
Now consider: →(F ; λx.A) · else →(G; λx.B)) if True (then →; A ⇒ F · x
The expression if True has triggered the evaluation of agent A in the envi→. ronment F · x The contract supported by if requires that the cases provided bind the labels then and else. We can relax this contract and provide default services if those bindings are not provided by the client. To do so, we replace in the definition of if the sandbox expression v; else with a default service. This service gets triggered when the case form does not contain an else binding: def
→(v; then) · false →(else →(λx.) · v; else)) ifd = ; λuv.b(true (F ; λx.A)) ⇒ . Now ifd False(then →
3.6
Equivalence for Agents
Two agents are equivalent if they exhibit the same behaviour, i.e., they enjoy the same reductions. We adopt Milner and Sangiorgi’s notion of barbed bisimulation
A Calculus for Modeling Software Components
351
[26]. The idea is that an agent A is barbed similar to B if A can exhibit any reduction that B does and if B is a barb, then A is a barb, too. If A and B are similar to each other they are bisimilar. The advantage of this bisimulation is that it can be readily be given for any calculus that contains barbs or values. For the asynchronous π-calculus, barbs are usually defined as having the capability of doing an output on a channel. A Piccola agent reduces to a barb, i.e., it returns a form. During evaluation the agent may spawn off new subthreads which could be blocked or still be running. We consequently define barbs as follows: Definition 3. A barb V is an agent expression A that is congruent to an agent generated by the following grammar: V ::= F A|V νc.V We write A ↓ for the fact that A is a barb, and A ⇓ when a barb V exists such that A ⇒ V . The following lemma relates forms, barbs and agents: Lemma 2. The following inclusion holds and is strict: F ⊂ F ≡ ⊂ {A|A ↓} ⊂ A Proof. The inclusions hold by definition. To see that the inclusion are strict, consider the empty form , the agent hide x , the barb 0 | hide x and the agent 0 (where 0 = νc.c? is the deadlocked null agent). The following lemma gives a syntactical characterization of barbs. Lemma 3. For any form F , agent A, and label x, the following terms are barbs, given V1 and V2 are barbs. V 1 · V2 V1 ; V 2 →V1 x
νc.V1 A | V1
Proof. By definition we have V ≡ ν˜ c.A | F . The claim follows by induction over F. We now define barbed bisimulation and the induced congruence: Definition 4. A relation R is a (weak) barbed bisimulation, if A R B, i.e., (A, B) ∈ R implies: – – – –
If If If If
A → A then there exists an agent B with B ⇒ B and A R B . B → B then there exists an agent A with A ⇒ A and A R B . A ↓ then B ⇓. B ↓ then A ⇓.
352
O. Nierstrasz and F. Achermann
˙ B, if there is some (weak) Two agents are (weakly) barbed bisimilar, written A ≈ barbed bisimulation R with A R B. Two agents are (weakly) barbed congruent, ˙ C[B]. written A ≈ B, if for all contexts C we have C[A] ≈ We define behavioural equality using the notion of barbed congruence. As usual we can define strong and weak versions of barbed bisimulation. The strong versions are obtained in the standard way by replacing ⇒ with → and ⇓ with ↓ in Definition 4. We only concentrate on the weak case since it abstracts internal computation. 3.7
Erroneous Reductions
Not all agents reduce to forms. Some agents enjoy an infinite reduction [2]. Other agents are stuck. An agent is stuck if it is not a barb and can reduce no further. Definition 5. An agent A is stuck, written A ↑, if A is not a barb and there is no agent B such that A → B. Clearly it holds that 0 ↑ and R ↑. The property of being stuck is not compositional. For instance c? ↑ but obviously, c() | c? can reduce to . We can put R into a context so that it becomes a barb, for instance F ; R ≡ F . Note that if an agent is stuck it is not a preform: F ≡ ∩ {A|A ↑} = ∅ by definition. Although 0 is arguably stuck by intention, in general a stuck agent can be interpreted as an error. The two typical cases which may lead to errors are (i) projection on an unbound label, e.g., ; x, and (ii) application of a non-service, e.g., . 3.8
π-Calculus Encoding
One may well ask what exactly the Piccola calculus adds over and above the asynchronous π calculus. In Achermann’s thesis it is shown that the Piccola calculus can be faithfully embedded into the localized π-calculus Lπ of Merro and Sangiorgi [23,36]. The mapping [[]]a encodes Piccola calculus agents as π-calculus processes. The process [[A]]a evaluates A in the environment given by the empty form, and sends the resulting value along the channel a. A form (value) is encoded as a 4-tuple of channels representing projection, invocation, hiding and selection. The main result is that the encoding is sound and preserves reductions. We do not require a fully abstract encoding since that would mean that equivalent Piccola agents translated into the π-calculus could not be distinguished by any π-processes. Our milder requirement means that we consider only π-processes which are translations of Piccola agents themselves and state that they cannot distinguish similar agents: Proposition 1 (Soundness). For closed agents A, B and channel a the congruence [[A]]a ≈ [[B]]a implies A ≈ B.
A Calculus for Modeling Software Components
353
Although it is comforting to learn that the π-calculus can serve as a foundation for modeling components, it is also clear from the complexity of the encoding that it is very distant from the kinds of abstractions we need to conveniently model software composition. For this reason we find that a richer calculus is more convenient to express components and connectors.
4
From the Piccola Calculus to Piccola
Piccola is a small composition language that supports the requirements summarized in figure 1, and whose denotational semantics is defined in terms of the Piccola calculus [2]. Piccola is designed in layered fashion. At the lowest level we have an abstract machine that implements the Piccola calculus. At the next level, we have the Piccola language, which is implemented by translation to the abstract machine, following the specification of the denotational semantics.
Applications: Composition styles: Standard libraries:
Piccola language:
Piccola calculus:
Piccola Layers Components + Scripts Streams, GUI composition, ... Coordination abstractions, control structures, basic object model ... Host components, user-defined operators, dynamic namespaces Forms, agents and channels
Piccola provides a more convenient, Python-like syntax for programming than does the calculus, including overloaded operators to support algebraic component composition. It also provides a bridge to the host language (currently Java or Squeak). Piccola provides no basic data types other than forms and channels. Booleans, integers, floating point numbers and strings, for example, must be provided by the host language through the language bridge. Curiously, the syntax of the Piccola calculus is actually larger than that of Piccola itself. This is because we need to represent all semantic entities, including agents and channels, as syntactic constructs in the calculus. In the Piccola language, however, these are represented only by standard library services, such as run and newChannel. The next layer provides a set of standard libraries to simplify the task of programming with Piccola. Not only does the Piccola language provide no built-in data types, it does not even offer any control structures of its own. These, however, are provided as standard services implemented in Piccola. Exceptions and
354
O. Nierstrasz and F. Achermann
try-catch clauses are implemented using agents, channels, and dynamic namespaces [5]. The first three layers constitute the standard Piccola distribution. The fourth layer is provided by the component framework designer. At this level, a domain expert encodes a compositional styles as a library of components, connectors, adaptors, coordination abstractions, and so on. Finally, at the top level, an application programmer may script together components using the abstractions provided by the lower layers [3,29]. 4.1
Reasoning about Styles
We have also explored how to reason about Piccola programs at the language level [2]. We have studied two extended examples. First, we considered synchronization wrappers that express the synchronization constraints assumed by a component. We can use synchronization wrappers to make components safe in a multithreaded environment. The wrappers separate the functionality of the component from their synchronization aspects. If the constraints assumed by the component hold in a particular composition the wrapper is not needed. In particular the wrapper is not necessary when the component is already wrapped. This property is formally expressed by requiring that the wrappers be idempotent. The second study compares push- and pull-flow filters. We demonstrate how to adapt pull-filters so that they work in a push-style. We have constructed a generic adapter for this task in two iterations. The first version contains a racecondition that may lead to data being lost. The formal model of Piccola is used to analyze the traces of an adapted filter and helps to detect the error. To fix the problem we specify the dynamics of a push-style filter, namely that push and close calls be mutually exclusive, that no further push calls may be attempted after a close, and that no “air-bubble” elements (filter slots holding an empty form) may be pushed downstream. Having clarified the interaction protocol as a wrapper, we present an improved version of the generic adapter. We show that the adapter ensures these invariants. 4.2
Partial Evaluation
Another interesting application of the calculus was to enable the efficient implementation of the language bridge. Since Piccola is a pure composition language, evaluating scripts requires intensive upping and downing [24] between the “down” level of the host language and the “up” level of Piccola. If the language bridge were implemented na¨ıvely, it would be hopelessly inefficient. Instead, Piccola achieves acceptable performance by adopting a partial evaluation scheme [2,38,39]. Since the language has a denotational semantics, we can implement it efficiently while proving that we preserve the intended semantics. The partial evaluation algorithm uses the fact that forms are immutable. We replace references
A Calculus for Modeling Software Components
355
to forms by the forms referred to. We can then specialize projections and replace applications of referentially transparent services by their results. However, most services in Piccola are not referentially transparent and cannot be inlined since that would change the order in which side-effects are executed. We need to separate the referentially transparent part from the non-transparent part in order to replace an application with its result and to ensure that the order in which the side-effects are evaluated is preserved. At the heart of the proof is that we can separate form expressions into sideeffects and referentially transparent forms [2].
5
Related Work
The Piccola calculus extends the asynchronous π-calculus with higher-order abstractions and first-class environments. π-calculus. The π-calculus [25] is a calculus of communicating systems in which one can naturally express processes with a changing structure. Its theory has been thoroughly studied and many results relate other formalisms or implementations to it. The affinity between objects and processes, for example, has been treated by various authors in the context of the π-calculus [18,44]. The Pict experiment has shown that the π-calculus is a suitable basis for programming many high-level construct by encodings [33]. For programming and implementation purposes, synchronous communication seems uncommon and can generally be encoded by using explicit acknowledgments (cf. [18]). Moreover, asynchronous communication has a closer correspondence to distributed computing [45]. Furthermore, in the π-calculus the asynchronous variant has the pleasant property that equivalences are simpler than for the synchronous case [14]. Input-guarded choice can be encoded and is fully abstract [27]. For these reasons we adopt asynchronous channels in Piccola. Higher-order abstractions. Programming directly in the π-calculus is often considered like programming a concurrent assembler. When comparing programs written in the π-calculus with the lambda-calculus it seems like lambda abstractions scale up, whereas sending and receiving messages does not scale well. There are two possible solutions proposed to this problem: We can change the metaphor of communication or we can introduce abstractions as first-class values. The first approach is advocated by the Join-calculus [15]. Communication does not happen between a sender and a receiver, instead a join pattern triggers a process on consumption of several pending messages. The Blue calculus of Boudol [9] changes the receive primitive into a definition which is defined for a scope. By that change, the Blue calculus is more closely related to functions and provides a better notion for higher-order abstraction. Boudol calls it a continuation-passing calculus. The other approach is adopted by Sangiorgi in the HOπ-calculus. Instead of communicating channels or tuples of channels, processes can be communicated
356
O. Nierstrasz and F. Achermann
as well. Surprisingly, the higher-order case has the same expressive power as the first-order version [35,36]. In Piccola we take the second approach and reuse existing encodings of functions into the π-calculus as in Pict. The motivation for this comes from the fact that the HOπ-calculus itself can be encoded in the first-order case. Asymmetric parallel composition. The semantics of asynchronous parallel composition is used in the concurrent object calculus of Gordon and Hankin [17] or the (asymmetric) Blue calculus studied by Dal-Zilio [12]. In the higher-order π-calculus the evaluation order is orthogonal to the communication semantics [36]. In Piccola, evaluation strategy interferes with communication, therefore we have to fix one for meaningful terms. For Piccola, we define strict evaluation which seems appropriate and more common for concurrent computing. Record calculus. When modeling components and interfaces, a record-based approach is the obvious choice. We use forms [20,21] as an explicit notion for extensible records. Record calculi are studied in more detail for example in [11,34]. In the λ-calculus with names of Dami [13] arguments to functions are named. The resulting system supports records as arguments instead of tuples as in the classical calculus. The λN -calculus was one of the main inspiration for our work on forms without introspection. An issue omitted in our approach is record typing. It is not clear how far record types with subtyping and the runtime acquisition can be combined. An overview of record typing and the problems involved can be found for example in [11]. Explicit environments. An explicit environment generalizes the concept of explicit substitution [1] by using a record like structure for the environment. In the environment calculus of Nishizaki, there is an operation to get the current environment as a record and an operator to evaluate an expression using a record as environment [32,37]. Projection of a label x in a record R then corresponds to evaluating the script x in an environment denoted by R. The reader may note that explicit environments subsume records. This is the reason why we call them forms in Piccola instead of just records. Handling the environment as a first-class entity allows us to define concepts like modules, interfaces and implementation for programming in the large within the framework. To our knowledge, the language Pebble of Burstall and Lampson was the first to formally show how to build modules, interfaces and implementation, abstract data types and generics on a typed lambda calculus with bindings, declarations and types as first-class values [10]. Other approaches A very different model is offered by ρω (AKA Reo) [7], a calculus of component connectors. Reo is algebraic in flavour, and provides various connectors that coordinate and compose streams of data. Primitives connectors can be composed using the Reo operators to build hiigher-level connectors. In contrast to process calculi, Reo is well-suited to compositional reasoning, since connectors can be composed to yield new connectors, and properties of
A Calculus for Modeling Software Components
357
connectors can be shown to compose. Data communicated along streams are uninterpreted in Reo, so it would be natural to explore the application of Reo to streams of forms.
6
Concluding Remarks
We have presented the Piccola calculus, a high-level calculus for modeling software components that extends the asynchronous π-calculus with explicit namespaces, or forms. The calculus serves as the semantic target for Piccola, a language for composing software components that conform to a particular compositional style. JPiccola, the Java implementation of Piccola, is realized by translation to an abstract machine that implements the Piccola calculus. The Piccola calculus is not only helpful for modeling components and connectors, but it also helps to reason about the Piccola language implementation and about compositional styles. Efficient language bridging between Piccola and the host language (Java or Squeak) is achieved by means of partial evaluation of language wrappers. The partial evaluation algorithm can be proved correct with the help of the Piccola calculus. Different compositional styles make different assumptions about software components. Mixing incompatible components can lead to compositional mismatches. We have outlined how the Piccola calculus can help to bridge mismatches by supporting reasoning about wrappers that adapt component contracts from one style to another. One shortcoming of our work so far is the lack of a type system. We have been experimenting with a system of contractual types [28] that expresses both the provided as well as the required services of a software component. Contractual types are formalized in the context of the form calculus, which can be seen as the Piccola calculus minus agents and channels. Contractual types have been integrated into the most recent distribution of JPiccola [19].
Acknowledgments We gratefully acknowledge the financial support of the Swiss National Science Foundation for projects No. 20-61655.00, “Meta-models and Tools for Evolution Towards Component Systems”, and 2000-067855.02, “Tools and Techniques for Decomposing and Composing Software”.
References 1. Mart´ın Abadi, Luca Cardelli, Pierre-Louis Curien, and Jean-Jacques L´evy. Explicit substitutions. Journal of Functional Programming, 1(4):375–416, October 1991. 2. Franz Achermann. Forms, Agents and Channels - Defining Composition Abstraction with Style. PhD thesis, University of Berne, January 2002.
358
O. Nierstrasz and F. Achermann
3. Franz Achermann, Stefan Kneubuehl, and Oscar Nierstrasz. Scripting coordination styles. In Ant´ onio Porto and Gruia-Catalin Roman, editors, Coordination ’2000, volume 1906 of LNCS, pages 19–35, Limassol, Cyprus, September 2000. SpringerVerlag. 4. Franz Achermann, Markus Lumpe, Jean-Guy Schneider, and Oscar Nierstrasz. Piccola – a small composition language. In Howard Bowman and John Derrick, editors, Formal Methods for Distributed Processing – A Survey of Object-Oriented Approaches, pages 403–426. Cambridge University Press, 2001. 5. Franz Achermann and Oscar Nierstrasz. Explicit Namespaces. In J¨ urg Gutknecht and Wolfgang Weck, editors, Modular Programming Languages, volume 1897 of LNCS, pages 77–89, Z¨ urich, Switzerland, September 2000. Springer-Verlag. 6. Franz Achermann and Oscar Nierstrasz. Applications = Components + Scripts – A Tour of Piccola. In Mehmet Aksit, editor, Software Architectures and Component Technology, pages 261–292. Kluwer, 2001. 7. Farhad Arbab and Farhad Mavaddat. Coordination through channel composition. In F. Arbab and C. Talcott, editors, Coordination Languages and Models: Proc. Coordination 2002, volume 2315 of Lecture Notes in Computer Science, pages 21– 38. Springer-Verlag, April 2002. 8. G´erard Berry and G´erard Boudol. The chemical abstract machine. Theoretical Computer Science, 96:217–248, 1992. 9. G´erard Boudol. The pi-calculus in direct style. In Conference Record of POPL ’97, pages 228–241, 1997. 10. Rod Burstall and Butler Lampson. A kernel language for abstract data types and modules. Information and Computation, 76(2/3), 1984. Also appeared in Proceedings of the International Symposium on Semantics of Data Types, Springer, LNCS (1984), and as SRC Research Report 1. 11. Luca Cardelli and John C. Mitchell. Operations on records. In Carl A. Gunter and John C. Mitchell, editors, Theoretical Aspects of Object-Oriented Programming. Types, Semantics and Language Design, pages 295–350. MIT Press, 1993. 12. Silvano Dal-Zilio. Le calcul bleu: types et objects. Ph.D. thesis, Universit´e de Nice - Sophia Antipolis, July 1999. In french. 13. Laurent Dami. Software Composition: Towards an Integration of Functional and Object-Oriented Approaches. Ph.D. thesis, University of Geneva, 1994. 14. C´edric Fournet and Georges Gonthier. A hierarchy of equivalences for asynchronous calculi. In Proceedings of ICALP ’98, pages 844–855, 1998. 15. C´edric Fournet, Georges Gonthier, Jean-Jacques L´evy, Luc Maranget, and Didier R´emy. A calculus of mobile agents. In Proceedings of the 7th International Conference on Concurrency Theory (CONCUR ’96), volume 1119 of LNCS, pages 406–421. Springer-Verlag, August 1996. 16. David Garlan, Robert Allen, and John Ockerbloom. Architectural mismatch: Why reuse is so hard. IEEE Software, 12(6):17–26, November 1995. 17. Andrew D. Gordon and Paul D. Hankin. A concurrent object calculus: Reduction and typing. In Proceedings HLCL ’98. Elsevier ENTCS, 1998. 18. Kohei Honda and Mario Tokoro. An object calculus for asynchronous communication. In Pierre America, editor, Proceedings ECOOP ’91, volume 512 of LNCS, pages 133–147, Geneva, Switzerland, July 15–19 1991. Springer-Verlag. 19. Stefan Kneubuehl. Typeful compositional styles. Diploma thesis, University of Bern, April 2003. 20. Markus Lumpe. A Pi-Calculus Based Approach to Software Composition. Ph.D. thesis, University of Bern, Institute of Computer Science and Applied Mathematics, January 1999.
A Calculus for Modeling Software Components
359
21. Markus Lumpe, Franz Achermann, and Oscar Nierstrasz. A Formal Language for Composition. In Gary Leavens and Murali Sitaraman, editors, Foundations of Component Based Systems, pages 69–90. Cambridge University Press, 2000. 22. Mark Lutz. Programming Python. O’Reilly & Associates, Inc., 1996. 23. Massimo Merro and Davide Sangiorgi. On asynchrony in name-passing calculi. In Kim G. Larsen, Sven Skyum, and Glynn Winskel, editors, 25th Colloquium on Automata, Languages and Programming (ICALP) (Aalborg, Denmark), volume 1443 of LNCS, pages 856–867. Springer-Verlag, July 1998. 24. Wolfgang De Meuter. Agora: The story of the simplest MOP in the world — or — the scheme of object–orientation. In J. Noble, I. Moore, and A. Taivalsaari, editors, Prototype-based Programming. Springer-Verlag, 1998. 25. Robin Milner, Joachim Parrow, and David Walker. A calculus of mobile processes, part I/II. Information and Computation, 100:1–77, 1992. 26. Robin Milner and Davide Sangiorgi. Barbed bisimulation. In Proceedings ICALP ’92, volume 623 of LNCS, pages 685–695, Vienna, July 1992. Springer-Verlag. 27. Uwe Nestmann and Benjamin C. Pierce. Decoding choice encodings. In Ugo Montanari and Vladimiro Sassone, editors, CONCUR ’96: Concurrency Theory, 7th International Conference, volume 1119 of LNCS, pages 179–194, Pisa, Italy, August 1996. Springer-Verlag. 28. Oscar Nierstrasz. Contractual types. submitted for publication, 2003. 29. Oscar Nierstrasz and Franz Achermann. Supporting Compositional Styles for Software Evolution. In Proceedings International Symposium on Principles of Software Evolution (ISPSE 2000), pages 11–19, Kanazawa, Japan, Nov 1-2 2000. IEEE. 30. Oscar Nierstrasz, Franz Achermann, and Stefan Kneubuehl. A guide to jpiccola. Technical report, Institut f¨ ur Informatik, Universit¨ at Bern, Switzerland, 2003. Available from www.iam.unibe.ch/∼scg/Research/Piccola. 31. Oscar Nierstrasz and Laurent Dami. Component-oriented software technology. In Oscar Nierstrasz and Dennis Tsichritzis, editors, Object-Oriented Software Composition, pages 3–28. Prentice-Hall, 1995. 32. Shin-ya Nishizaki. Programmable environment calculus as theory of dynamic software evolution. In Proceedings ISPSE 2000. IEEE Computer Society Press, 2000. 33. Benjamin C. Pierce and David N. Turner. Pict: A programming language based on the pi-calculus. In G. Plotkin, C. Stirling, and M. Tofte, editors, Proof, Language and Interaction: Essays in Honour of Robin Milner. MIT Press, May 2000. 34. Didier R´emy. Typing Record Concatenation for Free, chapter 10, pages 351–372. MIT Press, April 1994. 35. Davide Sangiorgi. Expressing Mobility in Process Algebras: First-Order and HigherOrder Paradigms. Ph.D. thesis, Computer Science Dept., University of Edinburgh, May 1993. 36. Davide Sangiorgi. Asynchronous process calculi: the first-order and higher-order paradigms (tutorial). Theoretical Computer Science, 253, 2001. 37. Masahiko Sato, Takafumi Sakurai, and Rod M. Burstall. Explicit environments. In Jean-Yves Girard, editor, Typed Lambda Calculi and Applications, volume 1581 of LNCS, pages 340–354, L’Aquila, Italy, April 1999. Springer-Verlag. 38. Nathanael Sch¨ arli. Supporting pure composition by inter-language bridging on the meta-level. Diploma thesis, University of Bern, September 2001. 39. Nathanael Sch¨ arli and Franz Achermann. Partial evaluation of inter-language wrappers. In Workshop on Composition Languages, WCL ’01, September 2001. 40. Jean-Guy Schneider. Components, Scripts, and Glue: A conceptual framework for software composition. Ph.D. thesis, University of Bern, Institute of Computer Science and Applied Mathematics, October 1999.
360
O. Nierstrasz and F. Achermann
41. Jean-Guy Schneider and Oscar Nierstrasz. Components, scripts and glue. In Leonor Barroca, Jon Hall, and Patrick Hall, editors, Software Architectures – Advances and Applications, pages 13–25. Springer-Verlag, 1999. 42. Mary Shaw and David Garlan. Software Architecture: Perspectives on an Emerging Discipline. Prentice-Hall, 1996. 43. Clemens A. Szyperski. Component Software. Addison Wesley, 1998. 44. David Walker. Objects in the π-calculus. Information and Computation, 116(2):253–271, February 1995. 45. Pawel T. Wojciechowski. Nomadic Pict: Language and Infrastructure Design for Mobile Computation. PhD thesis, Wolfson College, University of Cambridge, March 2000.
Specification and Inheritance in CSP-OZ Ernst-R¨ udiger Olderog and Heike Wehrheim Department of Computing Science University of Oldenburg 26111 Oldenburg, Germany {olderog,wehrheim}@informatik.uni-oldenburg.de
Abstract. CSP-OZ [16,18] is a combination of Communicating Sequential Processes (CSP) and Object-Z (OZ). It enables the specification of systems having both a state-based and a behaviour-oriented view using the object-oriented concepts of classes, instantiation and inheritance. CSP-OZ has a process semantics in the failures divergence model of CSP. In this paper we explain CSP-OZ and investigate the notion of inheritance. Behavioural subtyping relations between classes introduced in [50] guarantee the inheritance of safety and “liveness” properties. Keywords: CSP, Object-Z, failure divergence semantics, inheritance, safety and “liveness” properties, model-checking, FDR
1
Introduction
In contrast to the wide-spread use of object-oriented programming and specification languages, little is known about the properties enjoyed by systems constructed in the object-oriented style. Research on verification of object-oriented descriptions takes often place in the setting of object-oriented programming languages, for instance Java. The methods range from Hoare-style verification supported by theorem provers [25,36] via static checkers [30] to model-checking techniques [20]. Verification of object-oriented modeling languages focusses on UML (e.g. [28,40]). These approaches check properties of UML state machines by translating them into existing model-checkers. Although UML is an integrated formalism allowing the specification of data and behaviour aspects, existing model-checking techniques most often focus on the behavioural view. For a semantics of UML integrating different types of diagrams (including dynamic object creation and dynamically changing communication topologies) and its verification see [8,9]. Reasoning about object-oriented specifications represents a challenge in its own right. To describe the preservation of behavioural properties of classes under change the concept of subtyping has been lifted from data types to objects by [1,31]. Whereas these approaches are restricted to state-based specifications (using e.g. Object-Z) [34] proposed definitions suitable to deal with behaviouroriented specifications (using e.g. CSP). A first systematic study of subtyping for specifications integrating state-based and behaviour-oriented views is [50].
This research is partially supported by the DFG under grant Ol/98-3.
F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 361–379, 2003. c Springer-Verlag Berlin Heidelberg 2003
362
E.-R. Olderog and H. Wehrheim
Given a proof made with respect to one data-model, its reuse in another data-model extended by inheritance represents a major problem that must be overcome in order to build up libraries that support proofs about non-trivial applications. Existing approaches, based on shallow representations of subtyping via parametric polymorphism, are either geared towards abstractions of programs (like Eiffel, e.g. [21]) or specific data-models (like UML/OCL, [5]). So far, these techniques have not been applied to the combination of data-oriented and behavioral specifications. In this paper we study specification and inheritance in the language CSP-OZ [16,18] combining two existing specification languages for processes and data. Specification of processes. Communicating Sequential Processes (CSP) were introduced by Hoare [22,23]. The central concepts of CSP are synchronous communication via channels between different processes, parallel composition and hiding of internal communication. c!e
c
c?x
For CSP a rich mathematical theory comprising operational, denotational and algebraic semantics with consistency proofs has been developed [4,35,38]. Tool support comes through the FDR model-checker [37]. The name stands for Failure Divergence Refinement and refers to the standard semantic model of CSP, the failures divergence model, and its notion of process refinement. Specification of data. Z was introduced in the early 80’s in Oxford by Abrial as a set-theoretic and predicate language for the specification of data, state spaces and state transformations. It comprises of the mathematical tool kit, a collection of convenient notations and definitions, and the schema calculus for structuring large state spaces and their transformations. A Z schema has a name, say S, and consists of variable declarations and a predicate constraining the values of these variables. It is denoted as follows: S declarations predicate The first systematic description of Z is [46]. Since then the language has been published extensively (e.g. [52]) and used in many case studies and industrial projects. Object-Z is an object-oriented extension of Z [11,42,44]. It comprises the concepts of classes, instantiation and inheritance. Z and Object-Z come with the concept of data refinement. For Z there exist proof systems for establishing properties of specifications and refinements such as Z/EVES [39] or HOL-Z based on Isabelle [27]. For Object-Z type checkers exist. Verification support is less developed except for an extension of HOL-Z [41].
Specification and Inheritance in CSP-OZ
363
Combination. CSP-OZ, developed by C. Fischer at the University of Oldenburg, is a combination of CSP and Object-Z. Object-Z is used to describe all data-dependent aspects of classes, viz. attributes and methods. The dynamic behaviour of classes, i.e. their protocols of interaction, are specified within CSP. CSP-OZ has been used in a number of case studies from the area of telecommunication systems, production automatisation and satellite technology [3,49,33]. Verification of CSP-OZ specifications is supported by the FDR model-checker for CSP [14]. Structure of this paper. In Section 2 the combination CSP-OZ is introduced by way of an example. In Section 3 the semantics of CSP-OZ is reviewed. Section 4 is devoted to the inheritance operator in CSP-OZ and its semantics. In Section 5 inheritance of properties is studied in depth, and Section 6 concludes the paper.
2
The Combination CSP-OZ
There are various specialised techniques for describing individual aspects of system behaviour. But complex systems exhibit various behavioural aspects. This observation has led to research into the combination and semantic integration of specification techniques. One such combination is CSP-OZ [16,13,14,18] integrating CSP and ObjectZ. Central to CSP-OZ is the notion of a class. The specification of a class C has the following format: C I P Z
[interface] [CSP part] [OZ part]
The interface I declares channels names and types to be used by the class. The CSP part uses a CSP process P to describe the desired sequencing behaviour on these channels. The OZ part is of the form Z st : State Init(st) com c(st, in?, out!, st )
[state space] [initial condition] [communication schemas]
where the state space of C and its transformation is specified in the style of Object-Z. The state space itself is given by a schema State, here with the symbolic variable st ranging over State. The initial state of C is described by the Init schema that restricts the value of st. For the channels c declared in the interface communication schemas com c describe the state transformation induced by communicating along c with input values in? and output values out!. In Z and hence Object-Z the prime decoration is used for denoting the new values after the transformation. Thus com c depends on st, in?, out!, st .
364
E.-R. Olderog and H. Wehrheim
Example 1. To illustrate the use of CSP-OZ we present here part of the specification of a till [50] for the problem “cash point service” defined in [10]. Informally, the till is used by inserting a card and typing in a PIN which is compared with the PIN stored on the card. In case of a successful identification, the customer may withdraw money from the account. The till is only one component of a larger system including banks, cards, customers, and money dispensers. Global definitions. The CSP-OZ class Till makes use of two basic Z types: [Pin, CardID] Pin represents the set of personal identification numbers and CardID the set of card identification numbers. Interface. The Till is connected to its environment by several typed channels: chan chan chan chan
getCustPin : [ p? : Pin ] getCardPin : [ p? : Pin ] pay : [ a! : N ] updateBalance : [ a! : N; c! : CardID ]
The channel getCustPin expects a value p? of type Pin as its input, the channel getCardPin inputs a value p? of type Pin and outputs a value c! of type CardID, the channel pay outputs a natural number a! (the amount of money to be paid out to the customer), the channel updateBalance is intended for communication with the bank and outputs an amount a! of money to be withdrawn form the account belonging to card with identity c!. The full definition of the till comprises of more interface channels (see [50]). CSP part. This part specifies the order in which communications along the interface channels can occur. To this end, a set of recursive process equations is c given. The main equation starts with the process identifier main. The symbol = is used instead of an ordinary equals symbol to distinguish between CSP process equations and Z equations. c
main = insertCard → Ident c Ident = getCustPin → getCardPin → (idFail → Eject 2 idSucc → Service) c
Eject = eject → main c Service = (stop → Eject 2 withdraw → getAmount → pay → updateBalance → Eject)
Specification and Inheritance in CSP-OZ
365
This process specifies the following behaviour. First the till gets the CardID via a communication along channel insertCard . Then the till checks the customers identity by getting the customer’s PIN via getCustPin, retrieving the PIN stored on the card via getCardPin, and comparing both. The communications idFail signals that this comparison failed, the communication idSucc signals that it was successful. In case of failure the card is ejected. In case of success that service mode is entered where the customer has the choice of stopping the interaction or withdrawing money. In the latter case the communication getAmount inputs the amount of money the customer wishes to withdraw, the communication pay initiates the money dispenser to pay out this amount, and the communication updateBalance informs the bank about the change. Note that in this particular CSP process no communication values are specified. They will be dealt with in the OZ part. OZ part. This part specifies the state of the till and the effect of the communications on this state. The state consists of the following typed variables:
currCard : CardID currPin, typedPin : Pin amount : N
[state space]
The effect of communications along the interface channels are specified using communication schemas. Here we give only some examples. The schema com insertCard ∆(currCard ) c? : CardId currCard = c? specifies that a communication along insertCard may only change the state variable currCard . This is the meaning of the ∆-list in the first line of the declaration part of this schema. The second line specifies that a communication along insertCard has an input value c? of type CardID. The predicate of the schema specifies that the effect of the schema is an assignment of this input value to the variable currCard . The schema com idSucc ∆() currPin = typedPin specifies that a communication idSucc does not change any variable of the state (empty ∆-list) and is enabled only if the current PIN of the card is identical to the PIN typed in by the customer. The schema
366
E.-R. Olderog and H. Wehrheim
com updateBalance ∆() a! : N c! : CardId a! = amount ∧ c! = CardId specifies that the amount a! of money to withdrawn and the identity c! of the card are sent to the bank where the balance of the account is updated. Instances. An instance (object) t of the class Till is specified by a declaration t : Till . The instance t behaves like the class Till but with all channels ch renamed to ch.t with its own identity t. A customer using till t might perform the following interaction with it expressed as a CSP process: c
Customer = insertCard .t.765 → getCustPin.t.4711 → withdraw .t → getAmount.t.100 → SKIP To model the behaviour of several instances t1 , ..., tn of the class Till the interleaving operator ||| of CSP can be used: t1 ||| ... ||| tn
or
|||i=1,...,n ti
To be able to be connected to a finite sets of tills the class Bank will have a parameter adr of type F Till for the addresses of the different tills: Bank [adr : F Till ] chan updateBalance : [ t : adr , a! : N; c! : CardID ] ........................... For example, a system comprising one bank b connected to tills t1 , ..., tn can be specified by the process expression (b : Bank ; t1 , ..., tn : Till • b({t1 , ..., tn }) ||{updateBalance} (|||i=1,...,n ti )) where b is instantiated with the set {t1 , ..., tn } and ||{updateBalance} is the parallel composition enforcing synchronisation on the channel updateBalance between the bank b and the tills t1 , ..., tn .
3
Semantics
Each class of a CSP-OZ specification denotes a process obtained by transforming the OZ part into a process that runs in parallel with the CSP part. First we briefly review the semantics of CSP and Object-Z.
Specification and Inheritance in CSP-OZ
3.1
367
Semantics of CSP
The standard semantics of CSP is the FD-semantics based on failures and divergence [38]. A failure is a pair (s, X ) consisting of a finite sequence or trace s ∈ seq Comm over a set Comm of communications and a so-called refusal set X ∈ P Comm. Intuitively, a failure (s, X ) describes that after engaging in the trace s the process can refuse to engage in any of the communications in X . Refusal sets allow us to make fine distinctions between different nondeterministic process behaviour; they are essential for obtaining a compositional definition of parallel composition in the CSP setting of synchronous communication when we want to observe deadlocks. Formally, we need the following sets of observations about process behaviour: Traces = seq Comm, Refusals = P Comm, Failures = Traces × Refusals. A divergence is a trace after which the process can engage in an infinite sequence of internal actions. The FD-semantics of CSP is then given by two mappings F : CSP → P Failures and D : CSP → P Traces. For a CSP process P we write FD[[P ]] = (F[[P ]] , D[[P ]]). Certain well-formedness conditions relate the values of F and D (see [38], p.192). The FD-semantics induces a notion of process refinement denoted by F D . For CSP processes P and Q this relation is defined as follows: P F D Q iff F[[P ]] ⊇ F[[Q]] and D[[P ]] ⊇ D[[Q]] Intuitively, P F D Q means that Q refines P , i.e. Q is more deterministic and more defined than P . 3.2
Semantics of Z and Object-Z
In Z an operation schema Op x : Dx x : Dx P( x, x
)
describes a transformation on state space x specified by the predicate P ( x , x ). Z comes with the usual notion of data refinement and operation refinement [52]. Given a relation ρ between an abstract and a concrete data domain, a concrete operation schema C Op refines an abstract operation schema A Op, denoted by A Op
ρ C Op,
368
E.-R. Olderog and H. Wehrheim
if C Op is more defined and more deterministic than A Op. For Object-Z a history semantics comprising of sequences of states and events (operation calls) as well as some more abstract semantics are defined [42]. We do not need these semantics here because we only use the state transformation view of the communication schemas in the OZ part. 3.3
Semantics of CSP-OZ
The semantics of CSP-OZ is defined in [16,18]. Each CSP-OZ class denotes a process in the failures divergence model of CSP. This is achieved by transforming the OZ part of such a class into a CSP process that runs in parallel and communicates with the CSP part of the class. The OZ part can hence be seen as describing all data-dependent orderings of communications. The process successively chooses one of the enabled methods, executes it (thereby changing the state space) and afterwards starts again. Consider a CSP-OZ class C I P Z
[interface] [CSP part] [OZ part]
also written horizontally as U = spec I L P Z end with an OZ part Z st : State Init(st) ...com c(st, in?, out!, st )...
[state space] [initial condition] [one communication schema for each c in I]
where the notation com c(st, in?, out!, st ) indicates that this communication schema for c relates the state st to the successor state st and has input parameters in? and output parameters out!. Transformation. The OZ part of the class is transformed into a CSP process OZMain defined by the following system of (parametrised) recursive equations for OZpart using the (indexed) CSP operators for internal nondeterministic choice () and alternative composition (2): OZMain = OZPart(st) =
st OZPart(st) 2c, in? out!, st
c.in?.out! → OZPart(st )
where st ranges over all states in State : Exp satisfying Init(st). Thus the process OZMain can nondeterministically choose any state st satisfying Init(st) to start
Specification and Inheritance in CSP-OZ
369
with. Further on, c ranges over all channels declared in I , and in? ranges over the set Inputs(c) such that the precondition of the communication schema for c holds, i.e. ∃ out! : Outputs(c); st : State • com c(st, in?, out!, st ). Finally, for any chosen c and in?, the value out! ranges over the set Outputs(c), and st ranges over State such that com c(st, in?, out!, st ) holds. So the OZPart(st) is ready for every communication event c.in?.out! along a channel c in I where for the input values in? the communication schema com c(st, in?, out!, st ) is satisfiable for some output values out! and successor state st . For given input values in? any such out! and st can be nondeterministically chosen to yield c.in?.out! and the next recursive call OZPart(st ). Thus input and output along channels c are modelled by a subtle interplay of the CSP alternative and nondeterministic choice. Semantics of a class. A CSP-OZ class defines a template for all of its instances. The semantics of an instance is derived from the semantics of its class by filling in an instance name. Using parallel composition we define the semantics of a class in the failures divergence model: FD[[C ]] = FD[[ P ||{commonEvents} ZMain ]] D = FD[[P ]] ||F {commonEvents} FD[[Main]] D where ||F denotes the semantic counterpart in the failure divergence model A of the parallel composition ||A with synchronisation on the event set A (see e.g. [38]).
Semantics of an instance. Suppose an instance o of class C is declared by o : C . Then o denotes a process which is obtained from FD[[C ]] by applying a renaming operator that renames every channel ch in the interface I of C into ch.o. Thus events ch.d for some data element d become now ch.o.d as described by the following definition due to [12]: FD[[o]] = FD[[ C [{ch : Chans; d : Data | ch.d ∈ I • ch.d → ch.o.d }] ]] Refinement compositionality. By the above process semantics of CSP-OZ, the refinement notion F D is immediately available for CSP-OZ. In [18] it has been shown that CSP-OZ satisfies the principle of refinement compositionality, i.e. refinement of the parts implies refinement of the whole. Formally: – Process refinement P1 F D P2 implies refinement in CSP-OZ: spec I P1 Z end F D spec I P2 Z end – Data refinement Z1 ρ Z2 for a refinement relation ρ implies refinement in CSP-OZ: spec I P Z1 end F D spec I P Z2 end
370
4
E.-R. Olderog and H. Wehrheim
Inheritance
Process refinement P F D Q in CSP stipulates that P and Q have the same interface. Often one wishes to extend communication capabilities of a process or the operation capabilities of a class. This can be specified using the notion of inheritance, a syntactic relationship on classes: a superclass (or abstract class) A is extended to a subclass (or concrete class) C . The subclass should inherit the parts of the superclass. In CSP-OZ this is denoted as follows. Given a superclass A of the form A IA PA ZA
[interface] [CSP part] [OZ part]
we obtain a subclass C of A by referring to A using the inherit clause: C inherit A IC PC ZC
[superclass] [interface] [CSP part] [OZ part]
The semantics of the inherit operator is defined in a transformational way, i.e. by incorporating the superclass A into C yielding the following expanded version of C : C I P Z
[interface] [CSP part] [OZ part]
where I = IA ∪ IC and P is obtained from PA and PC by parallel composition and Z is obtained from ZA and ZC by schema conjunction. More precisely, to obtain the CSP part P we first replace in PA and PC the process identifiers main by new identifiers mainA and mainC respectively, then collect the resulting set of process equations, and add the equation c
main = mainA ||{commonEvents} mainC modelling parallel composition of PA and PC . To obtain the OZ part Z the corresponding schema of ZA and ZC are conjoined: State = StateA ∧ StateC , Init = InitA ∧ InitC , com c = com cA ∧ com cC for all channels c in IA ∩ IC , com c = com cA for all channels c in IA \ IC , com c = com cC for all channels c in IC \ IA .
Specification and Inheritance in CSP-OZ
371
Note that a subclass constructed in this way is not necessarily a refinement of its superclass.
5
Inheritance of Properties
In this section we study preservation of properties under inheritance. The scenario we are interested in is the following: suppose we have a superclass A for which we have already verified that a certain property P holds. Now we extend A to a subclass C and like to know under which conditions P also holds for C . This would allow us to check the conditions on the subclass and avoid re-verification. In principle we are thus interested in a reasoning which is similar to that used for refinement in CSP. In CSP, properties are preserved under refinement, i.e. P F D A ∧ A F D C ⇒ P F D C holds. If inheritance is employed instead of refinement this is in general not true. Inheritance allows to modify essential aspects of a class and thus may destroy every property proven for the superclass. We thus have to require a closer relationship between super- and subclass in order to achieve inheritance of properties. A relationship which guarantees a certain form of property inheritance is behavioural subtyping [31]. Originally studied in state-based contexts, this concept has recently been extended to behaviour-oriented formalisms (see [15]) and is thus adequate for CSP-OZ with its failure divergence semantics. Behavioural subtyping guarantees substitutability while also allowing extension of functionality as introduced by inheritance. In the following we will look at two forms of behavioural subtyping and show that one of them preserves safety and the other also a form of liveness properties. 5.1
Safety: Trace Properties
Since CSP-OZ has a failure-divergence semantics we use the CSP way of property specification. In CSP, properties are formalised by CSP processes. A property holds for a class A if the class refines the property. Since CSP offers different forms of refinement there are also different forms of satisfaction. We say that a class satisfies a property with respect to safety issues if it is a trace refinement of the property; when failure-divergence refinement is used a (limited form of) liveness is checked (see next section). Definition 2. Let A be a class and P a CSP property (process). A satisfies the trace property P (or A satisfies P in the trace semantics) iff traces(A) ⊆ traces(P ) (or equally P T A). We illustrate this by means of the cash point example. Consider the following class A0 with a behaviour as specified in Figure 1. For reasons of readability we only consider a very simple form of till.
372
E.-R. Olderog and H. Wehrheim A0
R idSucc
stop
? withdraw
pay
R
Fig. 1. The simple till A0 .
We want to specify that money is only paid out after the correct PIN code has been entered. As a CSP property process this gives: Seq = idSucc → pay → Seq Safe = Seq ||| CHAOS ( Σ\{idSucc, pay} ) Here the process CHAOS (S ), where S is a set of events, is the chaotic process which can always choose to communicate as well as refuse events of S . It is ev → CHAOS (S )). A0 satisfies the defined by CHAOS (S ) = STOP ( ev ∈S trace property Safe since Safe T A0 . Next we like to know whether such a trace property P can be inherited to a subclass (or more specifically, a subtype) C . As a first observation we notice that C potentially has traces over a larger alphabet than A since it can have a larger interface. This might immediately destroy the holding of a trace property. Nevertheless, a trace property might still hold in the sense that, as far as the operations of A are concerned, the ordering of operations as specified in P also hold in C . Definition 3. Let A, C be classes with α(A) ⊆ α(C ). C satisfies a trace property P w.r.t. α(A) iff traces(C ) ↓ α(A) ⊆ traces(P ) ↓ α(A). Here α(A) denotes the alphabet of A, i.e. the set of events A may communicate, and ↓ denotes projection. The question we ultimately like to answer can now be precisely formulated: if A satisfies P , does C satisfy P wrt. α(A)? This is in fact the case when C is a trace subtype of A. Definition 4. Let A, C be classes with α(A) ⊆ α(C ) and N = α(C )\α(A). Then C is a trace subtype of A, abbreviated A tr −st C , iff A ||| CHAOS (N ) T C .
Specification and Inheritance in CSP-OZ
373
Intuitively, parallel composition with the chaotic process over the set of methods N says that C may have new methods in addition to A and these can at any time be executed as well as refused but have to be independent from (interleaved with) the A-part. Safety properties are inherited to trace subtypes. Theorem 5. Let A, C be processes with A tr −st C , let P be a process formalising a trace property. If A satisfies the trace property P then C satisfies P w.r.t. α(A). As an example we look at an extension of class A0 . The till C0 depicted in Figure 2 extends A0 with a facility of viewing the current balance of the account. C0 is a trace subtype of A0 and thus inherits property Safe, i.e. C0 satisfies Safe wrt. α(A0 ). A0
C0
R
R
idSucc
idSucc
stop display
? withdraw
pay
R
stop
-?
view
withdraw
pay
R
Fig. 2. A class and a trace subtype.
This completes the section on inheritance of safety properties. Next we study liveness properties. 5.2
“Liveness”: F D Properties
Liveness properties are checked in CSP by comparing the property process and the class with respect to their failure divergence set, i.e. checking whether the class is a failure divergence refinement of the property. This yields a form of bounded liveness check: it can be proven that methods can be refused or conversely, are always enabled after certain traces. Unbounded liveness, as expressible in temporal logic, cannot be specified in CSP. Definition 6. Let A be a class and P a CSP property (process). A satisfies the liveness property P (or A satisfies P in the FD semantics) iff FD[[A]] ⊆ FD[[P]] (or equally P F D A).
374
E.-R. Olderog and H. Wehrheim
We illustrate this again on our till example. The property we like to prove for class A0 concerns service availability: Money can be withdrawn immediately after the PIN code has been verified. Formalised as a CSP property this is: Live = idSucc → withdraw → Live pay → Live stop → Live Class A0 satisfies the liveness property Live since Live F D A0 . Analogous to trace properties we now first have to define what preservation of a liveness property to a subclass should mean. Again, we have to face the fact that the subclass has additional functionality and thus failures and divergences range over a larger alphabet. In principle we apply the same technique as for trace properties. We project the failures and divergences of the subclass down to the alphabet of the superclass and on this projection carry out the comparison with the property P . Definition 7. Let A, C be classes with α(A) ⊆ α(C ). C satisfies a liveness property P w.r.t. α(A) iff ∀(s, X ) ∈ failures(C ) ∃(t, Y ) ∈ failures(P ) : s ↓ α(A) = t ↓ α(A) ∧ X ∩ α(A) = Y ∩ α(A) and ∀ s ∈ divergences(C ) ∃ t ∈ divergences(P ) : s ↓ α(A) = t ↓ α(A) . Liveness properties can be shown to be inherited to classes which are optimal subtypes of the superclass. Definition 8. Let A, C be classes with α(A) ⊆ α(C ) and N = α(C )\α(A). Then C is an optimal substype of A, abbreviated A ost C , iff A ||| CHAOS (N ) F D C . This definition lifts the idea of trace subtyping to the failure divergence semantics: in class C anything ”new” is allowed in parallel with the behaviour of A as long as it does not interfere with this ”old” part. Looking at class C0 in comparison with A0 we find that C0 is not an optimal subtype of A0 . For instance, C0 has the pair (idSucc view , {Σ \ {display}) in its failure set for which projected down to the alphabet of A0 no corresponding pair can be found in failures(A0 ) (the crucial point is refusal of withdraw ). Class C1 as depicted in Figure 3 on the other hand is an optimal subtype of class A0 and it indeed inherits property Live. Theorem 9 (Wehrheim). Let A, C be classes with A ost C , let P be a process formalising a liveness property. If A satisfies P in the FD semantics, then C satisfies P w.r.t. α(A) in the FD semantics.
Specification and Inheritance in CSP-OZ A0
375
C1
R
R
idSucc
idSucc
stop
?
?
test withdraw
stop
pay
withdraw
R
pay
R Fig. 3. A class and an optimal subtype.
stop
english
idSucc
? pay
withdraw
C2
german
german
R idSucc
stop
?
english
withdraw
pay
R
Fig. 4. Another optimal subtype.
A more complex extension of the simple till which is also an optimal subtype is shown in Figure 4. The till allows to switch between different languages but switching does not effect the basic functionality. Since C2 is an optimal subtype of A0 we get by the previous theorem that C2 satisfies Live.
6
Conclusion
In this paper we took the combined specification formalism CSP-OZ to define and study the inheritance of properties from superclasses to subclasses. Semantically, classes, instances, and systems in CSP-OZ denote processes in the standard failures divergence model of CSP. This allowed us to make full use of the well established mathematical theory of CSP. In case of systems with finite state CSP parts and finite data types in the OZ parts the FDR model-checker for CSP can be applied to automatically verify refinement relations between CSPOZ specifications [14] and subtyping relations [51]. Optimal subtyping is a strong requirement for subclasses to satisfy. As future work it would be interesting to study conditions under which specific properties are inherited. Related Work. A number of other combinations of process algebra with formalisms for describing data exist today. A comparison of approaches for com-
376
E.-R. Olderog and H. Wehrheim
bining Z with a process algebra can be found in [17]. Such integrations include Timed CSP and Object-Z (TCOZ) [32], B and CSP [6] and Z and CCS [47,19]. A similar type of integration which has been developed much earlier than these approaches is LOTOS [2]. LOTOS adopts a number of operators from CCS and CSP. The first design of LOTOS contained ACT-One for describing data, a language for specifying abstract data types. This proved to be unacceptable for industrial users, and the newest development called E-Lotos [26] (standardised by ISO) uses instead elements from functional languages. Closest to the combination CSP-OZ is Object-Z/CSP due to Smith [43]. There, CSP operators serve to combine Object-Z classes and instances. Thus to Object-Z classes a semantics in the failures divergence model of CSP is assigned just like done here for CSP-OZ. This semantics is obtained as an abstraction of the more detailed history semantics of Object-Z [42]. In contrast to CSP-OZ there is no CSP-part inside of classes. As we have seen in the example, the CSP part is convenient for specifying sequencing constraints on the communications events. Both CSP-OZ and Object-Z/CSP have been extended to deal with realtime [24,45]. The issue of inheritance of properties to subtypes has been treated by van der Aalst and Basten [48]. They deal with net-specific properties like safety (of nets), deadlock freedom and free choice. Leavens and Weihl [29] show how to verify object-oriented programs using a technique called “supertype abstraction”. This technique is based on the idea that subtypes need not to be re-verified once a property has been proven for their supertypes. In their study they have to take particular care about aliasing since in object-oriented programs several references may point to the same object, and thus an object may be manipulated in several ways. Subtyping for objectoriented programs has to avoid references which are local to the supertype but accessible in the subtype. Preservation of properties is also an issue in transformations within the language UNITY proposed by Chandy and Misra [7]. The superposition operator in Unity is a form of parallel composition which requires that the new part does not make assignments to underlying (old) variables. This is close to the non-modification condition we used in one pattern. Superposition preserves all properties of the original program.
References 1. P. America. Designing an object-oriented programming language with behavioural subtyping. In J.W. de Bakker, W.P. de Roever, and G. Rozenberg, editors, REX Workshop: Foundations of Object-Oriented Languages, number 489 in LNCS. Springer, 1991. 2. T. Bolognesi and E. Brinksma. Introduction to the ISO specification language LOTOS. Computer Networks and ISDN Systems, 14:25–59, 1987. 3. J. Bredereke. Maintaining telephone switching software requirements. IEEE Communications Magazine, 40(11):104–109, 2002.
Specification and Inheritance in CSP-OZ
377
4. S.D. Brookes, C.A.R. Hoare, and A.W. Roscoe. A theory of communicating sequential processes. Journal of the ACM, 31:560–599, 1984. 5. A. Brucker and B. Wolff. A proposal for a formal OCL semantics in Isabelle/HOL. In Proc. International Conference on Theorem Proving in Higher Order Logics (TPHOLs), LNCS. Springer, 2002. 6. M. Butler. csp2B: A practical approach to combining CSP and B. In J. Wing, J. Woodcock, and J. Davies, editors, FM’99: Formal Methods, number 1708 in Lecture Notes in Computer Science, pages 490–508. Springer, 1999. 7. K.M. Chandy and J. Misra. Parallel Program Design – A Foundation. AddisonWesley, 1988. 8. W. Damm, B. Josko, A. Pnueli, and A. Votintseva. Understanding UML: A Formal Semantics of Concurrency and Commu nication in Real-Time UML. In F.S. de Boer, M. Bonsangue, S. Graf, and W.P. de Roever, editors, Formal Methods of Components and Objects (FMCO’02), LNCS. Springer, 2003. (this volume). 9. W. Damm and B. Westphal. Live and Let Die: LSC-based Verification of UMLModels. In F.S. de Boer, M. Bonsangue, S. Graf, and W.P. de Roever, editors, Formal Methods of Components and Objects (FMCO’02), LNCS. Springer, 2003. (this volume). 10. B. T. Denvir, J. Oliveira, and N. Plat. The Cash-Point (ATM) ‘Problem’. Formal Aspects of Computing, 12(4):211–215, 2000. 11. R. Duke, G. Rose, and G. Smith. Object-Z: A specification language advocated for the description of standards. Computer Standards and Interfaces, 17:511–533, 1995. 12. C. Fischer, E.-R. Olderog, and H. Wehrheim. A CSP view on UML-RT structure diagrams. In H. Husmann, editor, Fundamental Approaches to Software Engineering, volume 2029 of LNCS, pages 91–108. Springer, 2001. 13. C. Fischer and G. Smith. Combining CSP and Object-Z: Finite or infinite tracesemantics? In T. Mizuno, N. Shiratori, T. Higashino, and A. Togashi, editors, Proceedings of FORTE/PSTV’97, pages 503–518. Chapmann & Hall, 1997. 14. C. Fischer and H. Wehrheim. Model-checking CSP-OZ specifications with FDR. In K. Araki, A. Galloway, and K. Taguchi, editors, Integrated Formal Methods, pages 315–334. Springer, 1999. 15. C. Fischer and H. Wehrheim. Behavioural subtyping relations for object-oriented formalisms. In T. Rus, editor, AMAST 2000: International Conference on Algebraic Methodology And Software Technology, number 1816 in Lecture Notes in Computer Science, pages 469–483. Springer, 2000. 16. C. Fischer. CSP-OZ: A combination of Object-Z and CSP. In H. Bowman and J. Derrick, editors, Formal Methods for Open Object-Based Distributed Systems (FMOODS’97), volume 2, pages 423–438. Chapman & Hall, 1997. 17. C. Fischer. How to combine Z with a process algebra. In J. Bowen, A. Fett, and M. Hinchey, editors, ZUM’98 The Z Formal Specification Notation, volume 1493 of LNCS, pages 5–23. Springer, 1998. 18. C. Fischer. Combination and Implementation of Processes and Data: From CSPOZ to Java. PhD thesis, Bericht Nr. 2/2000, University of Oldenburg, April 2000. 19. A. J. Galloway and W. Stoddart. An operational semantics for ZCCS. In M. Hinchey and Shaoying Liu, editors, Int. Conf. of Formal Engineering Methods (ICFEM). IEEE, 1997. 20. J. Hatcliff and M. Dwyer. Using the Bandera tool set to model-check properties of concurrent Java software. In K.G. Larsen, editor, CONCUR 2001, LNCS. Springer, 2001.
378
E.-R. Olderog and H. Wehrheim
21. S. Helke and T. Santen. Mechanized analysis of behavioral conformance in the Eiffel base libraries. In M. Butler, L. Petre, and K. Sere, editors, Proceedings of the FME 2001, LNCS. Springer, 2001. 22. C.A.R. Hoare. Communicating sequential processes. CACM, 21:666–677, 1978. 23. C.A.R. Hoare. Communicating Sequential Processes. Prentice Hall, 1985. 24. J. Hoenicke and E.-R. Olderog. Combining specification techniques for processes, data and time. In M. Butler, L. Petre, and K. Sere, editors, Integrated Formal Methods (IFM 2002), volume 2335 of LNCS, pages 245–266. Springer, 2002. 25. M. Huisman and B. Jacobs. Java Program Verification via a Hoare Logic with Abrupt Termination. In T. Maibaum, editor, Fundamental Approaches to Software Engineering (FASE 2000), volume 1783 of LNCS, pages 284–303. Springer, 2000. 26. ISO. Final comittee draft on enhancements to LOTOS. ISO/IEC JTC1/SC21, WG7 Enhancements to LOTOS, 1998. ftp://ftp.dit.upm.es/pub/lotos/elotos/Working.Docs/. 27. Kolyang. HOL-Z – An Integrated Formal Support Environment for Z in Isabelle/HOL. PhD thesis, Univ. Bremen, 1997. Shaker Verlag, Aachen, 1999. 28. D. Latella, I. Majzik, and M. Massink. Automatic verification of a behavioural subset of UML statechart diagrams using the SPIN model-checker. Formal Aspects of Computing, 11:430–445, 1999. 29. G.T. Leavens and W.E. Weihl. Specification and verification of object-oriented programs using supertype abstraction. Acta Informatica, 32:705–778, 1995. 30. K. R. M. Leino. Extended static checking: A ten-year perspective. In R. Wilhelm, editor, Informatics – 10 Years Back, 10 Years Ahead, volume 2000 of LNCS, pages 157–175. Springer, 2001. 31. B. Liskov and J. Wing. A behavioural notion of subtyping. ACM Transactions on Programming Languages and Systems, 16(6):1811 – 1841, 1994. 32. B. P. Mahony and J.S. Dong. Blending Object-Z and Timed CSP: An introduction to TCOZ. In The 20th International Conference on Software Engineering (ICSE’98), pages 95–104. IEEE Computer Society Press, April 1998. 33. A. Mota and A. Sampaio. Model-checking CSP-Z: strategy, tool support and industrial application. Science of Computer Programming, 40(1), 2001. 34. O. Nierstrasz. Regular types for active objects. In O. Nierstrasz and D. Tsichritzis, editors, Object-oriented software composition, pages 99 – 121. Prentice Hall, 1995. 35. E.-R. Olderog and C.A.R. Hoare. Specification-oriented semantics for communicating processes. Acta Inform., 23:9–66, 1986. 36. A. Poetzsch-Heffter and J. Meyer. Interactive verification environments for objectoriented languages. Journal of Universal Computer Science, 5(3):208–225, 1999. 37. A.W. Roscoe. Model-checking CSP. In A.W. Roscoe, editor, A Classical Mind — Essays in Honour of C.A.R.Hoare, pages 353–378. Prentice-Hall, 1994. 38. A.W. Roscoe. The Theory and Practice of Concurrency. Prentice-Hall, 1997. 39. M. Saaltink. The Z/EVES system. In J. Bowen, M. Hinchey, and D. Till, editors, ZUM’97, volume 1212 of LNCS, pages 72–88. Springer, 1997. 40. T. Sch¨ afer, A. Knapp, and S. Merz. Model Checking UML State Machines and Collaborations. In Workshop on Software Model Checking, volume 55 of ENTCS, 2001. 41. G. Smith, F. Kamm¨ uller, and T. Santen. Encoding Object-Z in Isabelle/HOL. In D. Bert, J.P. Bowen, M.C. Henson, and K. Robinson, editors, ZB 2002: Formal Specification and Development in Z and B, volume 2272 of LNCS, pages 82–99. Springer, 2002. 42. G. Smith. A fully abstract semantics of classes for Object-Z. Formal Aspects of Computing, 7:289–313, 1995.
Specification and Inheritance in CSP-OZ
379
43. G. Smith. A semantic integration of Object-Z and CSP for the specification of cocurrent systems. In J. Fitsgerald, C.B. Jones, and P. Lucas, editors, Foraml Methods Europe (FME’97), volume 1313 of LNCS, pages 62–81. Springer, 1997. 44. G. Smith. The Object-Z Specification Language. Kluwer Academic Publisher, 2000. 45. G. Smith. An integration of real-time Object-Z and CSP for specifying concurrent real-time systems. In M. Butler, L. Petre, and K. Sere, editors, Integrated Formal Methods (IFM 2002), volume 2335 of LNCS, pages 267–285. Springer, 2002. 46. J.M. Spivey. The Z Notation: A Reference Manual. Prentice-Hall International Series in Computer Science, 2nd edition, 1992. 47. K. Taguchi and K. Araki. Specifying concurrent systems by Z + CCS. In International Symposium on Future Software Technology (ISFST), pages 101–108, 1997. 48. W.M.P. van der Aalst and T. Basten. Inheritance of Workflows – An approach to tackling problems related to change. Theoretical Computer Science, 270(1-2):125– 203, 2002. 49. H. Wehrheim. Specification of an automatic manufacturing system – a case study in using integrated formal methods. In T. Maibaum, editor, FASE 2000: Fundamental Aspects of Software Engineering, number 1783 in LNCS, pages 334–348. Springer, 2000. 50. H. Wehrheim. Behavioural subtyping in object-oriented specification formalisms. University of Oldenburg, Habilitation Thesis, 2002. 51. H. Wehrheim. Checking behavioural subtypes via refinement. In A. Rensink B. Jacobs, editor, FMOODS 2002: Formal Methods for Open Object-Based Distributed Systems, pages 79–93. Kluwer, 2002. 52. J. Woodcock and J. Davies. Using Z — Specification, Refinement, and Proof. Prentice-Hall, 1996.
Model-Based Testing of Object-Oriented Systems Bernhard Rumpe IRISA-Université de Rennes 1, Campus de Beaulieu, Rennes, France and Software & Systems Engineering, TU München, Germany
Abstract. This paper discusses a model-based approach to testing as a vital part of software development. It argues that an approach using models as central development artifact needs to be added to the portfolio of software engineering techniques, to further increase efficiency and flexibility of the development as well as quality and reusability of results. Then test case modeling is examined in depth and related to an evolutionary approach to model transformation. A number of test patterns is proposed that have proven helpful to the design of testable object-oriented systems. In contrast to other approaches, this approach uses explicit models for test cases instead of trying to derive (many) test cases from a single model.
1 Portfolio of Software Engineering Techniques Software has become a vital, but often invisible part of our lives. Embedded forms of software are part of almost any technical device. The average household uses several computers, and the internet and telecommunication world has considerably changed our lives. Software is used for a variety of jobs. It can be as small as a simple script or as complex as an entire operating or enterprise resource planning system. For the near future, we can be rather sure that we will not have a single notation or process that can cover the diversity of today’s development projects. Projects are too different in their application domain, size, need for reliability, time-to-market pressure, and the skills and demands of the project participants. Even the UML [OMG02], which is regarded as a de-facto standard, is seen as a family of languages rather than a single notation and by far doesn’t cover all needs. This leads to an ongoing proliferation of methods, notations, principles, techniques and tools in the software engineering domain that is at least partly influenced from practical applications of Formal Methods. On the one hand, methods like Extreme Programming [Bec99] and Agile Software Development [Coc02] even discourage the long well known distinction between analysis, design and implementation activities and abandon all documentation activities in favor of rigorous test suites. On the other hand, upcoming development tools allow to generate increasing amounts of code from UML models, thus supporting the OMG’s initiative on “Model Driven Architecture” (MDA) [OMG01]. MDA’s primary purpose is to decouple platform-independent models from platform-specific, F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 380–402, 2003. © Springer-Verlag Berlin Heidelberg 2003
Model-Based Testing of Object-Oriented Systems
381
technical information. This should increase the reusability of both. Code generation, however, is today focusing pretty much on the generation of the productive system. Generation of test code from models is still a side issue. In particular the question, what a good test model should look like, is to be examined in this paper. In general, we can observe that in the foreseeable future we will have a portfolio of software engineering techniques that enables developers and managers to select appropriate processes and tools for their projects. Today, however, it is not quite clear which elements the portfolio should have, how they relate, when they are applicable, and what their benefits and drawbacks are. The software and systems engineering community therefore must reconsider and extend its portfolio of software engineering techniques incorporating new ideas and concepts, but also try to scientifically assess the benefits and limits of new approaches. For example: • Lightweight projects that don’t produce requirement and design documentation need intensive communications and can hardly be split into independent subprojects. Thus they don’t scale up to large projects. But where are the limits? A guess is, around 10 people, but there have been larger projects reportedly “successful” [RS02]. • Formal methods have built a large body of knowledge (see for example [ABRS03,JKW03,LCCRC03]), but how can this knowledge successfully and in a goal-oriented way be applied in today’s projects? A guess seems to be, formal methods apply best, if embodied in practical tools, using practical and well known notations without exposing the user directly to the formal method. • Product reliability often need not be 100% for all developments and in the first iteration already. But how to predict reliability from project metrics and how to adapt the project to increase reliability and accuracy to the desired level while minimizing the project/product costs? Thus in contrast to applying formal methods for verification purposes, the use of formal techniques for test case specification and metrics of test coverage does not give 100% reliability, but in practice has a much better cost/benefit ratio. Based on this observation we will in the following examine the modeling of test cases using several of the UML-like notations, arguing that this technique should be a new item in the broad portfolio of SE techniques. For this purpose, we develop our interpretation of the used UML notation in the context of test case modeling, which should give us the justification to regard the used notation as being backed up by a formal technique (without explicitly referring to that formal method). Section 2 discusses synergies and problems of using models for a variety of activities, including programming. Section 3 establishes the general needs of successful testing strategies. In Section 4 the scenario of a model-based test approach is discussed. Sections 5 and 6 present several test patterns that are useful to make an object-oriented design testable. While Section 5 concentrates on basic test patterns, Section 6 presents more specific test patterns for distributed systems. Section 7 finally discusses the benefits of an evolutionary approach to modeling in combination with an intensive, model-based test approach. In particular, the usability of tests as invariant observations for model-transformations is explored. For sake of conceptual discussions, technical details are omitted, but can be found in [Rum03].
382
B. Rumpe
2 Modeling Meets Programming UML [OMG02] undoubtedly has become the most popular modeling language for software intensive systems used today. Models can be used for quite a variety of purposes. Besides informal sketches that are used for communication, e.g. by being drawn on paper and posted on a wall the most common are: • Semi-precisely defined diagrams are used for documentation of that part of the requirements that is not written in plain English. • Architecture and designs are captured and documented with models. In practice, these models are increasingly often used for code generation. More sophisticated and therefore less widespread uses of models are analysis of certain quality attributes (such as message throughput, responsiveness or failure likelihood) or development of tests from models. Many UML-based tools today offer functionality to directly simulate models or generate at least parts of the code. As tool vendors work hard on continuous improvement of this feature, this means a sublanguage of UML will become a high-level programming language and modeling at this level becomes identical to programming. This raises a number of interesting questions: • Is it critical for a modeling language to be also used as programming language? For example analysis and design models may become overloaded with details that are not of interest yet, because modelers are addicted to executability. • Is a future version of the UML expressive enough to describe systems completely or will it be accompanied by conventional languages? How well are these integrated? • How will the toolset of the future look like and how will it overcome round trip engineering (i.e. mapping code and diagrams in both directions)? • What implications does an executable UML have on the development process? In [Rum03,Rum02] we have discussed these issues and have demonstrated, how the UML in combination with Java may be used as a high-level programming language. But, UML cannot only be used for modeling the application, but more importantly for modeling tests on various levels (class, integration, and system tests) as well. For this purpose we need executable test models, as testing is in general the process of executing a program with the intention to identify faults [Mye79,Bin99]. Executable models are usually less abstract than design models, but they are still more compact and abstract than the implementation. The same holds for test models versus manually implemented tests. One advantage of using models for test case description is that application specific parts are modeled with UML-diagrams and technical issues, such as connection to frameworks, error handling, persistence, or communication are handled by the parameterized code generator. This basically allows us to develop models that are independent of any technology and platform, as for example proposed in [SD00]. Only during the generation process platform dependent elements are added. When the technology changes, we only need to update the generator, but the application defining models as well as test models can directly be reused. This concept also directly
Model-Based Testing of Object-Oriented Systems
383
supports the above mentioned MDA-Approach [Coc02] of the OMG. Another important advantage is that both, the production code and automatically executable tests at any level, are modeled by the same UML diagrams. Therefore, developers use a single homogeneous language to describe implementation and tests. This will enhance the availability of tests already at the beginning of the coding activities and leads to a development method similar to the “test first approach” [Bec01,LF02]. Some of the UML-models (mainly class diagrams and statecharts) are used constructively, others are used for test case definition (mainly OCL, sequence and enhanced object diagrams). Fig. 1 illustrates the key mappings. object diagrams
statecharts
__:
class diagrams
__: sequence diagrams
Java
OCL
__:
WKLFNQHVVRIDUURZ LQGLFDWHVLPSRUWDQFH RIJHQHUDWLRQ
FRGHDQG WHVWFDVH JHQHUDWLRQ production code
test code
Fig. 1. Mapping of UML-models to code and test code.
As a consequence of the various possible forms of model use, we will identify the notion of diagram and model. Thus a model is a coherent piece of information, denoted in a diagrammatic or textual notation that describes an abstraction of the desired system. Multiple models can describe various aspects of the system. We also allow models to be composed of sub-models, as is the case with test models. This is in slight contrast to approaches, where a model is a virtual something in a tool and the user manipulates it indirectly through (diagrammatic) views. The latter approach has shown some difficulties when using models for test case definition.
3 Testing Strategies Not only [Bin99,Mye79] show that there is a huge variety of testing strategies and testing goals. While tests in the small usually try to identify faults, tests suites and coverage metrics can be used to estimate the quality of the system in terms of absence of faults. The “success” of a test can therefore be seen twofold, but we follow the
384
B. Rumpe
general line and define a test to fail, when an abnormal behavior (failure) shows that there is at least one fault in the system. This means the test and the system do not fit together. Testing can be done manually or automated. The widespread use of JUnit [BG99] shows that the use of automated tests has gained considerable attention in recent years, because it allows to “reuse” tests in form of regression tests on evolving systems without actually knowing what the test does. This allows very small iterations with continuous integration and the use of refactoring techniques [Fow99] to improve the code structure. Automated tests ensure a low defect rate and continuous progress, whereas manual tests would very rapidly lead to exhausted testers. To summarize the characteristics of tests we are aiming at: • Tests run the system – in contrast to static analyses. • Tests are automatic to prevent project members to get bored with tests (or alternatively to prevent a system that isn’t tested enough) • Automated tests build the test data, run the test and examine the result automatically. Success resp. failure of the test are automatically observed during the test run. • A test suite also defines a system that is running together with the tested production system. The purpose of this extended system is to run the tests in an automated form. • A test is exemplaric. A test uses particular values for the input data, the test data. • A test is repeatable and determined. For the same setup the same results are produced. In particular the last point is tricky to achieve, when the system under test (SUT) is distributed or has side effects. Specific care has to be taken to deal with these situations. Faults may also occur without being observable, because they are hidden in the SUT or cannot be traced in a distributed system. That is why the TTCN standard [TTCN92] also allows “inconclusive” and “none” as test results. We instead strongly demand that systems need to be built in a way that testing can be properly achieved. At least in the domain of object-oriented software systems this is a realistic demand. After discussing the structure of a model-based test in the following section, we will discuss, how object-oriented systems should be designed to assist automated testing.
4 Model-Based Testing The use of models for the definition of tests and production code can be manifold: • Code or at least code frames can be generated from a design model. • Test cases can be derived from an analysis or design model that is not used/usable for constructive generation of production code. For example behavioral models, such as statecharts, can be used to derive test cases that cover states, transitions or even larger subsets of its paths. • The models itself can be used to describe test cases or at least some part thereof.
Model-Based Testing of Object-Oriented Systems
test data
o1 o3
expected result and/or OCL-contract as test oracle
test driver
o2
OD
SD or method call o2
o1
objects under test o5
o4
385
o3
OD
o4
+
OCL
Fig. 2. Structure of a test modeled with object diagrams (OD), sequence diagram (SD) and the Object Constraint Language (OCL).
The first two uses are already discussed e.g. in [Rum02] and [BL01]. Therefore, in this section we concentrate on the development of models that define tests. A typical test, as shown in Fig. 2 consists of a description of the test data, the test driver and an oracle characterizing the expected test result. In object-oriented environments, test data can usually be described by an object diagram (OD). The object diagram in Fig. 3 shows the necessary objects as well as concrete values for their attributes and the linking structure. OD Ready a1213:Auction
…
long auctionIdent = 1213 String title= “Copper 413tons“ /int numberOfBids = 0
timePol:ConstantTimingPolicy int status = TimingPolicy.READY_TO_GO boolean isInExtension = false int extensionTime = 180
Fig. 3. Object diagram (OD) describing the test data as a particular situation in an online auction, that has not yet started (see the full example in [Rum03,Rum03b]).
The test driver can be defined using a simple method call or, if more complex, modeled by a sequence diagram (SD). An SD has the considerable advantage that not only the triggering method calls can be described, but it is possible to model desired interactions and check object states during the test run. For this purpose, the Object Constraint Language (OCL, [WK98]) is used. In the sequence diagram in Fig. 4, an OCL constraint at the bottom ensures that the new closing time of the auction is set to the time when the bid was submitted (bid.time) plus the extension time to allow competitors to react (the auction system using this structure is partly described in [Rum03, Rum03b]). It is effective to model the test oracle using a combination of an object diagram and reuse globally valid OCL properties. The object diagram in this case serves as a property description and can therefore be rather incomplete, just focusing on desired effects. The OCL constraints can also be taken from the set of general invariants or can be defined as specific properties. In practice, it turns out that there is a high amount of reuse possible through the following techniques:
386
B. Rumpe SD copper: Auction «trigger» handleBid(bid)
:BidPolicy
:TimingPolicy
validateBid(bid) return OK
test driver getNewClosingTime(bid) return t
OCL constraints describe properties during the test run
t.time == bid.time + extensionTime
Fig. 4. A sequence diagram (SD) describing the trigger of a test driver and predicts some interactions as well as an OCL property that holds at that point in the test.
•
Well prepared test data can be reused for many tests. There is often only a handful of basic test structures necessary. From those the specifically desired test data can be derived by small adaptations, e.g. replacing a single attribute value or adding a certain object. Having an explicit, graphically depicted basic model of the data structure at hand increases its reusability for specific adaptations. • Test data can be composed of several object diagrams, describing different parts of the data. • Test oracles can be defined using a combination of an object diagram and reuse globally valid OCL properties. The resulting object diagram can be rather small, describing deltas only and can be derived from the test data diagram. The OCL properties can be reused as they are usually globally valid invariants. As already mentioned, being able to use the same, coherent language to model the production system and the tests allows for a good integration between both tasks. It allows the developer to immediately define tests for the constructive model developed. It is therefore feasible that in a kind of “test-first modeling approach” the test data in form of possible object structures is developed before the actual implementation.
5 Test Pattern In the last few years a number of Agile Methods have been defined that share a certain kind of characteristics, described in [AM03]. Among these Extreme Programming (XP) [Bec99] is the most widely used and discussed method. One of the most important XP characteristics is that it uses automated tests at all stages. Practical experience shows that when this is properly done, the defect rate is considerably low [RS02]. Furthermore, automation allows to repeat tests continuously in form of regression tests. Thus the quality of the result is ensured through strong emphasis on testing activities, ideally on development of the tests before the production code (“test
Model-Based Testing of Object-Oriented Systems
387
first approach” [LF02]). When using UML models for test design the development project should become even more efficient. However, practical experience shows that there are a number of obstacles that need to be overcome to enable model-based testing. In particular, there are object-oriented architectures that exhibit serious problems that prevent tests. It is therefore important to identify those problems and offer appropriate and effective solutions. In the remainder of this section, we provide several solutions for a number of problems that typically occur and that we also experienced e.g. in the online auction system. These solutions are defined in form of applicable test patterns similar to the design patterns of [GHJV94]. Indeed, some of the test patterns are based on design patterns, such as singleton, adapter or factory to achieve a testable design. Unlike [Bin99] we only provide the essential structure and a short explanation of the patterns in this article and refer to [Rum03] to a more detailed description. A test pattern description typically consists of several parts, describing intention, how to apply, the resulting structure, example implementations and a discussion of the pros and cons. Often the structure itself appears as a simple concept and it’s the method part, describing practical knowledge of its applicability that makes a pattern useful. auctions Auction *
1 «interface» BiddingPolicy
BidPolDummy
participants
1 «interface» TimingPolicy
TimePolDummy
CD
bidder Person *
PersonDummy
dummies replace ordinary objects through subclassing
Fig. 5. Class diagram showing how dummies are added to the system.
Dummies for the Test Context It has become a primary testing technique to use dummies (also “stubs” or “mocks”) to replace parts of the system and thus better expose the tested part to the test driver. Only object-oriented concepts, namely inheritance and dynamic binding (also known as polymorphic replacement of objects), comfortably allows to build object structures that are testable with dummies. Fig. 5 shows the principle in one class diagram that allows to isolate an auction object, by replacing its context completely by dummies. Sometimes a dummy just does nothing, but often it is also necessary to feed back specific values to keep the test going in the desired direction. Practical experience shows that this should normally not be achieved through various dummy-subclasses,
388
B. Rumpe
but through parameterized dummies, whose behavior during the test can be determined via constructor parameters. This for example allows to predefine and store results of queries given back to the calling SUT just to see what the SUTs reaction will be on that data. Remembering Interaction and Results A typical application of a dummy is to prevent side effects that a system otherwise has on the environment. Such side effects may affect files, the data base, the graphical user interface etc. An object responsible for logging activities that provides a method “write” may be replaced by a subclass object (say “LogDummy”) where the “write” method simply stores the line to write in a local attribute where it can be examined after the test. Sequence diagrams, however, already allow access to this kind of values during the test. Fig. 6 describes the effect of a test on the log object directly.
«Testclass» :AuctionTest
kupfer912: Auction
SD :ProtocolSimpleDummy
«trigger» start() «trigger» handleBid(bid1) «trigger» handleBid(bid2) «trigger» finish()
write(“Auction 912 opened at 14:00:00“)
write(“Bid 55.200 $US in Auction 912 from Theo accepted“)
effects of the test are captured in the method arguments
write(“Bid 55150 $US in Auction 912 from Joe accepted“)
write(“Auction 912 closed at 15:30:00“)
Fig. 6. Describing the effects for the log in a sequence diagram.
Static Elements As explained two concepts of object-oriented languages, namely inheritance and dynamic binding, allow to setup tests in a form that was not possible in procedural and functional languages. However, typical OO languages also provide concepts that make testing difficult. These are in particular static concepts, such as static attributes, static methods and constructors. Static attributes are mainly used to allow easy sharing of some global resources, such as the log object or database access. Static attributes, however, should be avoided anyway. If necessary e.g. for access to generally known objects, a static attribute can at least be encapsulated by a static method. The problem with a static method results from the inability to redefine it for testing purposes. For example if the static method “write” does have side effects, these cannot be prevented during a test. Unfortunately, there are often at least some static
Model-Based Testing of Object-Oriented Systems
389
methods necessary. We have therefore used the technique shown in Fig. 7 to provide a static interface to the customer and at the same time to allow the effect of the static method to be adaptable, through using an internal delegation to a singleton object that is stored in a static attribute. With proper encapsulation of the initialization of that attribute this is a safe and still efficient technique to make static methods replaceable by dummies without changing their signature. Thus the side effects of static methods can be prevented. CD
Singleton #Singleton singleton = null +initialize() #initialize(Singleton s) +method(Arguments) #doMethod(Arguments)
SingletonDummy #doMethod(Arguments)
singleton initialisation (default resp. with an object of a subclass) the (underlined) static method only calls the dynamic method „doMethod“ dynamic method contains the real functionality and can be redefined.
Fig. 7. Singleton object behind a static method.
Constructors With respect to testing purposes a constructor shares a number of similarities with static methods, because a constructor is not dynamically bound and can therefore not be redefined. Furthermore, a constructor creates new objects and thus allows the object under test to change its own environment. For example the tested object may have the idea to create its own log object and write the log through it. The standard solution for this problem is to force any object creation to be done by a factory. So instead “new class(arg)” a method “getClass(arg)” might be called. This method may be static using the approach above to encapsulate the factory object, but still make it replaceable for tests. A factory dummy can then create objects of appropriate subclasses that serve as dummies with certain predefined test behavior. In practice, we found it useful to model the newly created objects using object diagrams. The factory dummy that replaces the factory then doesn’t really create new objects, but returns one of the predefined objects each time it is called. Fig. 8 shows the factory part of a test data structure where three different person objects shall be “created” during the test. The data structure and the attribute values can be described using the same object diagram as is used when describing the basic test data structure. Further advantages are (1) that the newly “created” objects are known by name and can thus easily be checked after the test, even if the objects were disposed during the test and (2) the order of creation can also be checked.
390
B. Rumpe OD
:TestFactoryDummy index=0
:PersonBidderDummy
…
personIdent = 1783 name = “Theo Smith“ isActive = true
index=1
index=2
:PersonGuestDummy personIdent = 20544 name = “Otto Mair“ isActive = false
…
:Person
…
personIdent = 19227 name = “Jo Miller“ isActive = false
Fig. 8. Replacing a constructor by a factory and modeling the factory behavior through a series of objects to be “created” during a test.
Frameworks and Components Frameworks are commonly used in object-oriented systems. Most prominent examples are the AWT and Swing from Java, but basic classes, such as the containers partly also belong to that category. Software that uses predefined frameworks is particularly hard to test: • The control flow of the framework makes tests tricky to run. • Form and class of newly created objects within the framework is predefined through the used constructors. • Static variables and methods of the framework cannot be controlled, in particular if they are encapsulated. • Encapsulation prevents to check the test result. • Sometimes subclasses cannot be built properly, because the class or methods are declared “final”, there is no public constructor, the constructor does have side effects, or the internal control flow is unknown. Today there doesn’t exist a single framework that is directly suited for tests. Such a framework should allow to replace framework objects by self-defined subclass objects and should provide its own default dummy subclasses. Furthermore, the framework should use factories for object creation and give the test developer a possibility to replace these factories. The white-box adaptation principles that frameworks usually provide through subclassing are indeed helpful and sometimes sufficient, but if not, a more general technique, the adapter, is needed to separate application and framework. This is a recommended technique for application developers anyway to decouple application and framework and can be reused for improvement of testability as well. Fig. 9 shows how a JSP “ServletRequest” class is adapted. A “ServletRequest” basically contains the contents of a web form filled by the user in form of pairs (parametername, content). Unfortunately “ServletRequest”-objects can only be created by handling actual requests through the web. Therefore, an adapter is used, which is called “OwnServletRequest”. In an adapter normally simple delegation is used. But, as framework classes are strongly interconnected method calls often require other framework objects as parameters or reveal access to other framework objects. For example the method “get-
Model-Based Testing of Object-Oriented Systems
391
Session()” needs an additional wrapping to return the proper object of class “OwnSession”. This adapter technique allows us to completely decouple application and framework and even to run the application part without the framework as it may be desired in tests. For testing purposes “OwnServletRequestDummy” may now overwrite all methods and use a “Map” to store a predefined set of “user” inputs. However, there must be noted that this kind of wrapping may need additional infrastructure to ensure that each time “getSession()” is called on the same “ServletRequest”, the same corresponding session object is returned. This can be solved through an internal Map from Session to OwnSession that keeps track.
«Adapter» OwnServletRequest
…
OwnServletRequest() OwnServletRequest(HttpServletRequest hsr) +Enumeration getParameterNames() +String getParameter(String name) +OwnSession getSession()
OwnServletRequestDummy
HttpServletRequest 0..1
…
CD
+Enumeration getParameterNames() +String getParameter(String name) +HttpSession getSession()
result with the type of a framework class are also wrapped in an adapter
…
Map/*String,String*/ parameter OwnServletRequestDummy(Map /*String,String*/ p) +Enumeration getParameterNames() +String getParameter(String name) +OwnSession getSession()
…
0..1
«Adapter» OwnSession
… HttpSession
… OwnSessionDummy
Fig. 9. Adapters for framework classes.
So far we have discussed a number of easy to apply, but effective techniques to make object-oriented systems testable. We have used class, sequence and object diagrams to model the test pattern and demonstrated how to use these diagrams to model the test data and dummies. It is an interesting question, how to present such methodological knowledge and its individual parts. Basically the technical principles, such as pattern structure can be formally defined. This has the advantage that at least the structural part of such a pattern can automatically be applied using an appropriate tool. However, the most important part of a test pattern, namely the methodical experience cannot be formalized, but needs to be presented to the user in an understandable way. The user needs then to adapt a pattern to his specific situation. Therefore, we have chosen to largely use examples instead of precise descriptions for patterns.
6 Test Pattern for Distributed Systems Testing a distributed system may become hard, as distribution naturally involves concurrent processes with interactions and timeouts thus leading to nondeterministic behavior. Furthermore, it is normally not possible to stop the system and obtain a global system state for consistency checking. There exists a variety of approaches to
392
B. Rumpe
deal with these problems, in particular in the hardware and embedded systems area. Through the distribution of web services in particular in the E-Commerce domain, it becomes increasingly important to be able to deal with distributed object systems in this domain as well. In our example application, the online auction system, timing and distribution are very important, as auctions last only a very restricted time (e.g. one hour) and in the final phase bids are submitted within seconds. Therefore, it is necessary that auctions are handled synchronously over the web. The test patterns discussed in this section tackle four occurring problems: (1) simulation of time and progress, (2) handling concurrency through threads, (3) dealing with distribution and (4) communication. As already mentioned, the test patterns concentrate on functional tests. Additional effort is necessary to test quality of service attributes, such as throughput, mean uptime, etc. The proposed techniques have already been used in other approaches, the novelty basically comes from the combination of modeling techniques and these concepts in form of methodical test patterns. Handling Time and Progress An online auction usually takes about one hour. However, a single test may not take that time, but needs to be run in milliseconds, as hundreds of tests shall finish quickly. So it is necessary to simulate time. This becomes even more important, when distributed processes come into play that do not agree on a global time as is usually the case in the internet. Thus instead of calling the time routine of the operating system directly, an adapter is used. The adapter can be replaced by a parameterized dummy that allows us to freely set time. For many tests, a fixed time is sufficient, for tests of larger series of behaviors, however, it is also necessary that progress happens. Thus in the time pattern two more concepts can be established: First, each query of the current time increases time for one tick. Second, we use explicit time stamps on sequence diagrams to adapt time during the test. The time stamps as shown in Fig. 10 therefore correspond to statements that update the timing dummy. This active use of time stamps contrasts other approaches, where a passive interpretation regards a time stamp as maximum durations that a signal may take. The principle used here to simulate time also allows to simulate the behavior of times that trigger certain events regularly or after timeouts. Concurrency Using Threads Concurrency within one processing unit is often used to increase reactivity and to delegate regularly occurring tasks to specific units. In web applications, threads deal with polling of TCP/IP-data from sockets and with GUI interactions. However, those threads are normally encapsulated in the frameworks and use “callbacks” to the application code to handle a request. For a functional test of this type of concurrency it is necessary to simulate these callbacks. This can be done by defining a fixed scheduling for callbacks to obtain the necessary determinism and repeatability. Fig. 11 shows a test driver in form of a sequence diagram, where the driving object
Model-Based Testing of Object-Oriented Systems
393
shows a test driver in form of a sequence diagram, where the driving object submits several requests to a number of objects that are normally running within different threads. time stamps define the time where the interaction takes place and are used to set the time dummy «Testclass» :AuctionTest
{time=Feb 12 2000, 13:00:00}
{time=14:42:22}
{time=14:44:18}
{time=14:47:18}
SD
copper912: Auction
timePol: TimingPolicy
«trigger» start() «trigger» handleBid(bid1)
«trigger» handleBid(bid2)
newCurrentClosingTime(copper912, bid1)
{time+=100msek}
return t1 newCurrentClosingTime(copper912, bid2)
{time+=40msek}
return t2
«trigger» finish()
Fig. 10. A sequence diagram with time stamps to describe the progress of time.
SD
three objects usually active in different threads of the auction client
:ClockDisplay {time+=1s}
«Testclass» :ClientTest
:WebBidding
:BiddingPanel
updateDisplay() updateMessages() foreignBid(Money m1)
{time+=1s}
updateDisplay() m1.full=“553.000,00 $US“ updateDisplay()
{time+=1s}
actionPerformed(...)
AWT callback bid(“552 400“)
updateMessages()
ownBid(Money m2) m2.full=“552.400,00 $US“
updateDisplay() {time+=1s} method calls are completed before next one is issued, there is no true concurrency
Fig. 11. A test driver schedules calls to objects that normally reside in different threads.
This approach only works for sequential calls and therefore doesn’t test whether the simulated threads would behave similar, if running in parallel resp. a machine scheduled interleaving. Thus we just do functional tests. On one hand interleaving can be checked through additional stress tests and reviews. On the other hand Java e.g. provides a synchronization concept that if properly used is a powerful technique to make programs thread-safe. In practice concurrency problems have been considerably reduced since the concept of thread-safeness was introduced. In more involved situations, where interaction between two active threads is actually desired and therefore shall be tested, it might be necessary to slice methods into smaller portions and do a more fine grained scheduling. However, the possibilities of interactions easily
394
B. Rumpe
explode and efficient testing strategies are necessary. It is also possible to set up test drivers that run large sets of generated tests to explore at least a part of the interaction space. In an application where the developers define threads on their own, these threads usually have the form of a loop with a regularly repeated activity and a sleep statement. If not, they usually can and for applicability of the test pattern also should be reformulated in such a form. The repeating activity then can easily be added to the test scheduling, whereas the Thread object itself should be so simple, that a review is sufficient to ensure its correctness. In Java the Thread class provides additional functionality, such as a join or termination of threads which causes additional effort to simulate. As some of those methods cannot be redefined in subclasses it might be necessary to use an adapter. Distributed Systems Based on the test pattern defined so far, it now becomes feasible to test distributed systems. Real or at least conceptually distributed systems have subsystems with separated storage and enforce explicit communication. With CORBA, DCOM, RMI or even plain socket handling there is a variety of communication techniques available. Of course it is possible to run distributed tests, but it is a lot more efficient to simulate the distribution within one process. Again this technique only works for tests of the functionality. One cannot expect to get good data on reactivity and efficiency of the system when several subsystems are mapped into one process. As each object in the distributed system resides in exactly one part, we introduce a new tag, called location that allows to model in the test, where the object resides. Fig. 12 shows a test driver with an interleaving of activities in distributed locations. To simulate a distributed system it is necessary to ensure that the distributed threads are mapped into one process in such a way that no additional interactions occur. But interactions usually occur when static state is involved, because e.g. static attributes can be globally accessed. In a distributed system every subsystem had its own static attribute, after the mapping only one attribute exists. Our encapsulation of static attributes in singleton objects, however, can easily be adapted to simulate a multiple static attribute. Actually the delegation mechanism explained earlier is extended to use a map from location to attribute content instead of a single attribute. The location is set by the test driver accordingly thus allowing to distinguish the respective context of each tested object. This for example allows to handle multiple logs, etc. The location tag is therefore crucial to setup virtually distributed systems and run them in an interleaved manner in such a way that they believe they run on their own. The virtually distributed system, however, gives us an interesting opportunity: It allows to stop the system during the run and check invariants and conditions across subsystem borders by simply adding globally valid OCL-constraints to the sequence diagram that drives the test run. To be furthermore able to talk about local invariants, we have extended the OCL in [Rum03] to also allow localized constraints.
Model-Based Testing of Object-Oriented Systems th ree o b jects from d istin g uish ed lo catio n s
{location=S erver} s:A uctio n S erver
«Testclass» :C om m Test
395 SD
{location=C 2} w 2:W eb B id d in g
{location=C 3} w 3:W eb B id ding
subm itB id(b) updateM essages()
updateM essages() subm itB id(b) updateM essages()
updateM essages()
Fig. 12. A test driver schedules calls to objects in different locations.
Distributed Communication The remaining problem for tests of distributed systems is to simulate communication in an efficient way. The standard technique here is to build layers of respective communicating objects and use proxies (stubs) on each layer where appropriate. If for example CORBA is used, we build an adapter system around the CORBA API to encapsulate it in the same way as for ordinary frameworks. Thus replacement of the communication part through a dummy becomes feasible. In Fig. 13 we see two object diagrams showing layers of a subset of the communication mechanism that directly deals with sockets in Java. A bid arrives at the AuctionServerProxy in the client. It is transformed into a string and transferred via the MessageHandleProxy, the BufferedWriter and the URLConnection to the socket on the server side. There a thread that resides in the HttpConnection sleeps until a string is received on the socket. The received strings is transferred to the actual MessageHandler that un-marshalls the object into the original bid and gives it to the actual auction server. as:AuctionServer
…
OD Server
submitBid(Bid b)
…
asp:AuctionServerProxy
OD Client
submitBid(Bid b)
mh:MessageHandler
…
send(String s)
…
mhp:MessageHandlerProxy send(String s) send(String s, Writer w)
:HttpConnection
…
handleRequest(Socket s) handleRequest(Reader r)
«java.net» :Socket
/
«java.io» :BufferedReader
«java.net» :URLConnection
/
«java.io» :BufferedWriter
Internet: Browser, Caches, Proxies, Web-Server
Fig. 13. The layers of communication objects in the original system.
396
B. Rumpe
{location=Server} as:AuctionServer
…
{location=Client} asp:AuctionServerProxyDummy
submitBid(Bid b)
OD
submitBid(Bid b) delegation and copying of the arguments
…
{location=Server} as:AuctionServer submitBid(Bid b)
OD
{location=Client} asp:AuctionServerProxy submitBid(Bid b)
{location=Server} mh:MessageHandler
…
test of marshalling included
send(String s)
as:AuctionServer
…
submitBid(Bid b)
…
asp:AuctionServerProxy
OD
submitBid(Bid b)
mh:MessageHandler
…
send(String s)
…
mhp:MessageHandlerProxy send(String s) send(String s, Writer w)
…
:HttpConnection handleRequest(Socket s) handleRequest(Reader r)
«java.io» :PipedReader
«java.io» :PipedWriter
Fig. 14. Three shortcuts for the communication layer.
The trick is that both proxies on the right hand side resemble the same signature by sharing the same interface as their real counterparts on the left hand side. Therefore in a test we may simply shortcut the communication structure. Depending on the objects, we want to test, we might shortcut at the AuctionServer-layer already or go as deep as the Reader/Writer-pair. Fig. 14 shows three variants of possible connections. In the first two configurations it is important to model the location of each object, because the test generator needs to ensure the location is changed, when a client object calls a server object. In the third configuration it is unnecessary, as a transfer of a bid now consists of two parts. First, the bid is submitted at the AuctionServerProxy on the client side and stored at the PipedReader and then the HttpConnection is activated in a second call on the server side. In the preceeding two sections, we have discussed a number of test patterns using models to describe the basic structure. The test patterns on one hand allow us to actually define functional tests for almost any kind of object-oriented and in particular distributed system in a systematic way. On the other hand the used examples show how easy it can be to define and understand test setups that are based on models. These models are a lot more compact and can more easily be developed, read and understood than code. Increased usability of these models for several development stages becomes feasible, because of a better understanding what these models can be used for. Therefore, model-based development as proposed by the MDA-approach [OMG01] becomes applicable.
Model-Based Testing of Object-Oriented Systems
397
7 Model Evolution Using Automated Tests Using models for test and application development is only one side of the medal. Automated testing is the primary enabler for an evolutionary approach of developing systems. Therefore, in this section, we give a sketch of how model-based, automated testing, and model evolution fit together. In the development approach sketched so far, an explicit architectural design phase is abandoned and the architecture emerges during design. Architectural shortcomings are resolved through the application of refactoring techniques [OJ93,Fow99]. These are transformational techniques to evolve a system in small, systematic steps to enhance its structure. The concept isn’t new (see [PR03] for a discussion), but through availability of tools and its embedding in XP [Bec99], transformational development now becomes widely used. Nowadays, it is expected that the development and maintenance process is capable of being flexible enough to dynamically react on changing requirements. In particular, enhanced business logic or additional functionality should be added rapidly to existing systems, without necessarily undergo a major re-development or reengineering phase. This can be achieved at best if techniques are available that systematically evolve the system using transformations. To make such an approach manageable, the refactoring techniques for Java [Fow99] have proven that a comprehensible set of small and systematically applicable transformation rules seems optimal. Transformations, however, cannot only be applied to code, but to any kind of model. A number of possible applications are discussed in [PR03]. Having a comprehensible set of model transformations at hand, model evolution becomes a crucial step in software development and maintenance. Architectural and design flaws can then be more easily corrected, superfluous functionality and structure removed, structure for additional functionality or behavioral optimizations be adapted, because models are more abstract, exhibit higher-level architectural and design information in a better way. CD
CD
Person
Person transformation
Bidder
Guest
checkPasswd()
checkPasswd()
long ident
checkPasswd() long ident
Bidder
Guest
Fig. 15. Two transformational steps moving an attribute and a method along the hierarchy.
Two simple transformation rules for a class diagram are shown in Fig. 15. The figure shows two steps that move a method and an attribute upward in the inheritance hierarchy at once. The upward move of the attribute is accompanied by the only context condition that the other class “Guest” does not have an attribute with the same name
398
B. Rumpe
yet. In contrast, moving the method may be more involved. In particular, if both existing method bodies are different, there are several possibilities: (1) Move up one method implementation and have it overridden in the other class. (2) Just add the methods signature in the superclass. (3) Adapt the method implementations in such a way that distinguishing parts are factored out into other sub-methods and the remainder of the method bodies is identical in both methods. Many of the necessary transformation steps are as simple as the upward move of an attribute. However, others are more involved and their application comes with a larger set of context conditions. These of course need automated assistance. The power of these rather simple and manageable transformation steps comes from the possibility to combine them and evolve complex designs in a systematic and traceable way. Following the definition on refactoring from [Fow99], we use transformational steps for structure enhancement that do not affect “externally visible behavior”. For example both transformations shown in Fig. 15 do not affect the external behavior if made properly. By “externally visible behavior” Fowler in [Fow99] basically refers to behavioral changes visible to the user. This can be generalized by introducing an abstract “system border” that may also act as interface to other systems. Furthermore, in a hierarchically structured system, we may enforce behavioral equivalence for “subsystem borders” already. It is therefore necessary to explicitly describe, which kind of behavior is regarded as externally visible. For this purpose tests are the appropriate technique to describe behavior, because (1) tests are already available as result of the development process and (2) tests are automated which allows us to check the effect of a transformation through inexpensive, automated regression testing. A test case thus acts as an “observer” of the behavior of a system under a certain condition. This condition is also described by the test case, namely through the setup, the test driver and the observations made by the test. Fig. 16 illustrates this situation. test = driver and “observer” setup & call
observe observe creation interaction
check property
compare with expected result
snapshots of the test run
time axis
Fig. 16. A test case acts as observation.
Fig. 16 also shows that tests do not necessarily constrain their observation to “externally visible behavior”, but can make observations on local structure, internal interactions or state properties even during the system run. Therefore, we distinguish “inter-
Model-Based Testing of Object-Oriented Systems
399
nal” test that evolve together with the transformed system and “external” tests which need to remain unchanged, because they describe external properties of the system. Unit and integration tests focus on small parts of the system (classes or subsystems) and usually take a deep look into system internals. These tests are usually transformed together with the code models. Unit and integration tests are usually provided by the developer or test teams that have access to the systems internal details. Therefore, these are usually “glass box tests”. Acceptance tests, instead, are “black box” tests that are provided by the user (although again realized by developers) and describe external properties of the system. These tests must be a lot more robust against changes of internal structure. Fig. 17 illustrates a diagram that illustrates how an observation remains invariant under a test. To achieve robustness, acceptance tests should be modeled against the published interfaces of a system. In this context “published” means that parts of the system that are explicitly marked as externally visible and therefore usually rather stable. Only explicit changes of requirements lead to changes of these tests and indeed the adaptation of requirements can very well be demonstrated through adaptation of these test models followed by the transformations necessary to meet these tests afterwards in a “test-first-approach”.
test = driver and “observer”
observation
transformation
system run
modified system run
Fig. 17. The transformed system model is invariant under a test observation.
To increase stability of acceptance tests in transformational development, it has proven useful to follow a number of standards for test model development. These are similar to coding standards and have been found useful already before the combination with the transformational approach: • In general an acceptance test should be abstract, by not trying to determine every detail of the tested part of the system. • A test oracle should not try to determine every part of the output and the resulting data structure, but concentrate on important details, e.g. by ignoring uninteresting objects and attribute values (e.g. in object diagrams and OCL constraints). • OCL property descriptions can often be used to model a range of possible results instead of determining one concrete result.
400
B. Rumpe
• Query-methods can be used instead of direct attribute access. This is more stable when the data structure is changed. • It should not be tried to observe internal interactions during the system run. This means that sequence diagrams that are used as drivers for acceptance tests concentrate on triggers and on interactions with the system border. • Explicitly published interfaces that are regarded as highly stable should be introduced and acceptance tests should focus on these interfaces.
8 Conclusions The proposal made in this paper is part of a pragmatic approach to model-based software development. This approach uses models as primary artifact for requirements and design documentation, code generation and test case development and includes a transformational technique to model evolution for efficient adaptation of the system to changing requirements and technology, to optimize architectural design and fix bugs. To ensure the quality of such an evolving system, intensive sets of test cases are an important prerequisite. They are modeled in the same language, namely UML, and thus exhibit a good integration and allow us to model system and tests in parallel. The paper demonstrates that it is feasible to use various kinds of models to explicitly define automated tests. For use in object-oriented systems, however, the design of the system has to some extent to be adapted in order to allow testable systems. A series of basic and enhanced test patterns lead to a better testable design. In particular test patterns for distributed systems are a necessary prerequisite to allow testability. However, there are some obstacles for the proposed approach. (1) Currently, tool assistance is still in its infancy. (2) More experience is needed to come up with effective testing techniques in the context of model evolution, which must also involve coverage metrics. (3) These new techniques, namely an executable sub-language of the UML as well as a lightweight methodological use of models in a development process are both a challenge to current practice in software engineering. They exhibit new possibilities and problems. Using executable UML allows to program in a more abstract and efficient way. This may finally downsize projects and decrease costs. The free resources can alternatively be used within the project for additional validation activities, such as reviews, additional tests or even a verification of critical parts of the system. Therefore, we can conclude that techniques such as model-based development, model evolution and test-first design will change software engineering and add new elements to its portfolio.
Acknowledgements I would like to thank Markus Pister, Bernhard Schätz, Tilman Seifert and Guido Wimmel for commenting an earlier version of the paper as well as for valuable dis-
Model-Based Testing of Object-Oriented Systems
401
cussions. This work was partially supported by the Bayerisches Staatsministerium für Wissenschaft, Forschung und Kunst and through the Bavarian Habilitation Fellowship, the German Bundesministerium für Bildung und Forschung through the Virtual Software Engineering Competence Center (ViSEK).
References [ABRS03]
E. Abraham-Mumm, F.S. de Boer, W.P. de Roever, and M. Steffen. A Toolsupported Proof System for Mutlithreaded Java. (in this volume) LNCS. Springer, 2003. [AM03] Agile Manifesto. http://www.agilemanifesto.org/. 2003. [Bec99] Beck, K. Extreme Programming explained, Addison-Wesley. 1999. [Bec01] Beck K. Aim, Fire (Column on the Test-First Approach). IEEE Software, 18(5):87-89, 2001. [BG99] Beck K., Gamma E. JUnit: A Cook’s Tour, JavaReport, August, 1999. [Bin99] Binder, R. Testing Object-Oriented Systems. Models, Patterns, and Tools, Addison-Wesley, 1999. [BL01] Briand L. and Labiche Y. A UML-based Approach to System Testing. In M. Gogolla and C. Kobryn (eds): «UML» - The Unified Modeling Language, 4th Intl. Conference, pages 194-208, LNCS 2185. Springer, 2001. [Coc02] Cockburn, A. Agile Software Development. Addison-Wesley, 2002. [Fow99] Fowler M. Refactoring. Addison-Wesley. 1999. [GHJV94] Gamma E., Helm R., Johnson R., Vlissides J. Design Patterns, Addison-Wesley, 1994. [JKW03] B. Jacobs, J. Kiniry, and M. Warnier. Java Program Verification Challenges. (in this volume) LNCS. Springer, 2003. [LCCRC03] G.T. Leavens, Y. Cheon, C. Clifton, C. Ruby and D.R. Cok. How the Design of JML Accommodates Both Runtime Assertion Checking and Formal Verification. (in this volume) LNCS. Springer, 2003. [LF02] Link J., Fröhlich P. Unit Tests mit Java. Der Test-First-Ansatz. dpunkt.verlag Heidelberg, 2002. [Mye79] Myers, G. The Art of Software Testing, John Wiley & Sons, New York, 1979. [OJ93] Opdyke W., Johnson R. Creating Abstract Superclasses by Refactoring. Technical Report. Dept. of Computer Science, University of Illinois and AT&T Bell Laboratories. 1993 [OMG01] OMG. Model Driven Architecture (MDA). Technical Report OMG Document ormsc/2001-07-01, Object Management Group, 2001. [OMG02] OMG - Object Management Group. Unified Modeling Language Specification. V1.5. 2002. [PR03] Philipps J., Rumpe B.. Refactoring of Programs and Specifications. In: Practical foundations of business and system specifications. H.Kilov and K.Baclawski (Eds.), 281-297, Kluwer Academic Publishers, 2003. [Rum02] Rumpe, B. Executable Modeling with UML. A Vision or a Nightmare? In: Issues & Trends of Information Technology Management in Contemporary Associations, Seattle. Idea Group Publishing, Hershey, London, pp. 697-701. 2002. [Rum03] Rumpe, B. Agiles Modellieren mit der UML. Habilitation Thesis. To appear 2003.
402
B. Rumpe
[Rum03b]
[RS02]
[SD00] [TTCN92]
[WK98]
Rumpe B. E-Business Experiences with Online Auctions. In: Managing ECommerce and Mobile Computing Technologies, Julie Mariga (Ed.) Idea Group Inc., 2003. Rumpe B., Schröder A. Quantitative Survey on Extreme Programming Projects. In: Third International Conference on Extreme Programming and Flexible Processes in Software Engineering, XP2002, May 26-30, Alghero, Italy, pg. 95-100, 2002. Siedersleben J., Denert E. Wie baut man Informationssysteme? Überlegungen zur Standardarchitektur. Informatik Spektrum, 8/2000:247-257, 2000. ISO/IEC, Information Technology - Open Systems Interconnection - Conformance Testing Methodology and Framework - Part 3: The Tree and Tabular Combined Notation (TTCN), ISO/IEC International Standard 9646, 1992. Warmer J., Kleppe A. The Object Constraint Language. Addison-Wesley. 1998.
Concurrent Object-Oriented Programs: From Specification to Code Emil Sekerinski McMaster University Department of Computing and Software Hamilton, Ontario, Canada [email protected]
Abstract. In this paper we put forward a concurrent object-oriented programming language in which concurrency is tightly integrated with objects. Concurrency is expressed by extending classes with actions and allowing methods to be guarded. Concurrency in an object may be hidden to the outside, thus allowing concurrency to be introduced in subclasses of a class hierarchy. A disciplined form of intra-object concurrency is supported. The language is formally defined by translation to action systems. Inheritance and subtyping is also considered. A theory of class refinement is presented, allowing concurrent programs to be developed from sequential specifications. Our goal is to have direct rules for verification and refinement on one hand and a practical implementation on the other hand. We briefly sketch our implementation. While the implementation relies on threads, the management of threads is hidden to the programmer.
1 Introduction The reason for having concurrency in programs is that concurrency occurs naturally when modeling the problem domain, is to make programs more responsive, and is to exploit the potential speedup offered by multiple processors. It has been argued that objects can be naturally thought of as evolving independently and thus concurrently; objects are a natural “unit” of concurrency. Yet, current mainstream object-oriented languages treat concurrency independently of objects: typically concurrency is expressed in terms of threads that have to be created separately from objects. In this paper we put forward a notation for writing truly concurrent object-oriented programs. Sequential object-oriented programs are expressed in terms of classes featuring attributes and methods. We keep this paradigm and extend it by augmenting classes by actions and adding guards to methods. While methods need to be invoked, actions are executed autonomously. Atomicity of attribute access is guaranteed by allowing only one method or action to be active in an object at any time. Concurrency is achieved by having active methods and actions in several objects. We also suggest a theory for developing concurrent object-oriented programs out of sequential ones, recognizing that concurrent programs often arise from sequential specifications. Class hierarchies are commonly used to express specification-implementation relationships. We envisage continuing to do this with concurrent classes by treating concurrency in the same way an implementation issue as the choice of a data structure. F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 403–423, 2003. c Springer-Verlag Berlin Heidelberg 2003
404
E. Sekerinski
Thus we may have a class serving as a specification and subclasses of it being sequential or concurrent implementations. For a general overview of concurrent object-oriented languages we refer to [10]. Our work shares with the πoβλ approach by Jones et al. [14, 16] the use of synchronous communication between objects and the use of objects for restricting interference. While πoβλ is defined in terms of the π calculus, a process algebra, the definition of our language is in terms of action systems. We do not directly support early return and delegate statements as πoβλ does, but we do support inheritance and subtyping. Earlier related work includes the POOL family of languages [1], where communication between concurrent objects is done by rendezvous. Hoare-style verification rules for a language that includes statements for sending and receiving synchronous messages are given in [2]. Here we consider instead only (synchronous) method calls, where entrance to objects is regulated by method guards. Several approaches have emerged from extending action systems to model concurrent objects, as there is an established theory of data refinement and atomicity refinement of action systems [3, 6]. Action systems with procedures by Back and Sere [5] and Sere and Walden [23] resemble concurrent objects, except that action systems cannot be created dynamically like objects. Bosangue, Kok and Sere [8, 9] apply action systems to model dynamically created objects. B¨uchi and Sekerinski [11] take this further by defining inheritance and subtyping and justify the refinement rules with respect to observable traces. However, both approaches enforce strict atomicity of actions: if an action (or method) contains several method calls that may block, either all are executed or the whole action is not enabled. Thus, these approaches do not allow direct translation to efficient code. The Seuss approach of Misra [20] is also action-based but additionally considers fairness between actions. Guarded methods are distinguished from unguarded methods, with the syntactic restriction that there can be only one call to a guarded method per action and this must be the first statement. Other restrictions are that object cannot be created dynamically and there is no inheritance. The goal of the presented work is on one hand to have a simple theory of program development and on the other hand to have an efficient and practical implementation. This paper is the result of several iterations towards this goal, starting with [4]. To test our ideas, we have developed a prototypical compiler for our language [17]. A key concept is to weaken the strict atomicity of methods and actions: when a method call transfers control to another object, the lock to the first object is released and a new activity in that object can be initiated. Section 2 introduces the language and gives some examples. Our approach to making the theory simple is to start with a formal model of concurrent modules and to express all other constructs by translations into this “core”. Only translations that are needed have to be applied and all formal reasoning is done in the core. The formalization is done within the Simple Theory of Types. Section 3 formalizes that core, on top of which Section 4 defines classes, objects, inheritance, subtyping, and dynamic binding and discusses verification and refinement. Section 5 extends this to concurrent objects. Section 6 sketches the implementation of the language. We conclude with a discussion of the proposed model and the kind of concurrency it leads and with observations of the limitations of the current work.
Concurrent Object-Oriented Programs: From Specification to Code
405
2 A Concurrent Object-Oriented Language We start by giving the (slightly simplified) formal syntax of the language in extended BNF. The construct a | b stands for either a or b, [a] means that a is optional, and {a} means that a can be repeated zero or more times: class
::= class identifier [ inherit identifier ] [ extend identifier ] { attribute | initialization | method | action } end attribute ::= attr variableList initialization ::= initialization ( variableList ) statement method ::= method identifier ( variableList , res variableList ) [ when expression do ] statement action ::= action [ identifier ] [ when expression do ] statement statement ::= assert expression | identiferList := expressionList | identiferList :∈ expressionList | identifier.identifier ( expressionList , identifierList ) | identifier := new identifier ( expressionList ) | begin statement { ; statement } end | if expression then statement [ else statement ] | while expression do statement var variableList • statement variableList ::= identifierList : type { , identifierList : type } identifierList ::= identifier { , identifier } expressionList ::= expression { , expression } A class is declared by giving it a name, optionally stating the class being inherited or extended, and then listing all the attributes, initializations, methods, and actions. Initializations have only value parameters, methods may have both value and result parameters, and actions don’t have parameters. Both methods and actions may optionally have a guard, a boolean expression. Actions may be named, though the name does not carry any meaning. The assertion statement assert b checks whether boolean expression b holds. If it holds, it continues, otherwise it aborts. The assignment x := e assigns simultaneously the values of the list e to the list x of variables. The nondeterministic assignment statement x :∈ s selects an element of the set s and assigns it to the list x of variables. This statement is not part of the programming language, but is included here for use in abstract programs. A method call o.m(e, z) to object o takes the list e as the value parameters and assigns the result to the list z of variables. The object creation o := new C(e) creates a new object of class C and calls its initialization with value parameter e. We do not further define identifier and expression. We illustrate the constructs of the language by a series of examples. Consider the problem of ensuring mutual exclusion of multiple users accessing two shared resources. A user can perform a critical section cs only if that user has exclusive access to both resources. We assume each resource is protected by a semaphore. Semaphores and users are represented by objects, with a semaphore having a guarded method:
406
E. Sekerinski
class Semaphore attr n : integer initialization n := 1 method P when n > 0 do n := n − 1 method V n := n + 1 end
class User attr s, t : Semaphore initialization (a, b : Semaphore) s, t := a, b method doCriticalSection begin s.P ; t.P ; cs ; s.V ; t.V end end
We assume that all statements only access and update attributes of the object itself and local variables, except for method calls that may access and update the state of other objects. All statements are executed atomically up to method calls. Thus in class Semaphore the method V is always executed atomically, as is the initialization. The method P may block if the guard is not true, but once the method is enabled, it is also executed atomically. The method doCriticalSection may block at the calls s.P and t.P. In this case some other activity must first call the V method of the corresponding semaphore before execution can resume. The next example is about merging the elements of two bounded buffers into a third buffer. Buffers and mergers are represented by objects: class Buffer attr b : array of Object attr in, out, n, max : integer initialization (m : integer) in, out, n, max := 0, 0, 0, m ; b := new Object[m] method put(x : Object) when n < max do in, b[in], n := (in + 1) mod max, x, n + 1 method get( res x : Object) when n > 0 do out, x, n := (out + 1) mod max, b[out], n − 1 end class Merger attr in1, in2, out : Buffer attr a1, a2 : boolean attr x1, x2 : Object initialization (i1, i2, o : Buffer) in1, in2, out, a1, a2 := i1, i2, o, false, false action copy1 when a1 do begin a1 := false ; in1.get(x1) ; out.put(x1) ; a1 := true end action copy2 when a2 do begin a2 := false ; in2.get(x2) ; out.put(x2) ; a2 := true end end After creating a new merger object m, the actions of m can execute in parallel with the remaining program, including other Merger objects. Actions cannot be called, they can be initiated automatically whenever they are enabled. Action copy1 is enabled if a1 is true. Once copy1 is initiated, it may block at either the call in1.get(x1) or the call
Concurrent Object-Oriented Programs: From Specification to Code
407
out.put(x1). In this case another activity in the same object may be initiated or may resume (if it was blocked). Initiating an activity here means starting either copy1 or copy2 again. Since a1 is false at these points, copy1 is disabled and cannot be initiated a second time. On the other hand, copy2 may be initiated and come to conclusion or block at the call in2.get(x2) or the call out.put(x2). Hence for example the situation may arise that both actions are blocked at the out.put calls. Thus Merger can buffer two elements. The last example is the observer design pattern, expressed as an abstract program. The pattern allows that all observers of one subject perform their update methods in parallel: class Observer attr sub : Subject initialization (s : Subject) begin sub := s ; s.attach(this) end method update . . . end class Subject attr obs, notifyObs : set of Observer initialization obs, notifyObs := {}, {} method attach(o : Observer) obs := obs ∪ {o} method notify notifyObs := obs action notifyOneObserver when notifyObs = {} do var o : Observers • begin o :∈ notifyObs ; notifyObs := notifyObs − {o} ; o.update end end As soon as execution of the action notifyOneObserver in a subject s reaches the call o.update, control is passed to object o and another activity in s may be initiated or may resume. In particular, the action notifyOneObserver may be initiated again, as long as notifyObs is not empty, i.e. some observers have not been notified. Thus at most as many notifyOneObserver actions are initiated as there are observers and all notified observers can proceed concurrently. New observers can be added at any time and will be updated after the next call to notify.
3 Statements, Procedures, Modules, and Concurrency We introduce the “core” language into which the object-oriented constructs are translated. The definition is done in terms of higher order logic, as the type system of higher order logic is close to that of Pascal-like languages. We assume there are some basic types like boolean and integer. New types can be constructed as functions X → Y and
408
E. Sekerinski
products X × Y, for given types X and Y. Function application is written as f (x) or simply f x and a pair as (x, y) or simply x, y. For convenience we also assume that further type constructors like set of T and bag of T are available. Statements The core statements are as follows. Let X be the type of the program state and p : X → boolean be state predicate. The assertion {p} does nothing if p is true and aborts otherwise. The guard [p] does nothing if p holds and blocks otherwise. If S and T are statements then S ; T is their sequential composition. The choice S T selects either S or T nondeterministically. If Q is a relation, i.e. a function of type X → Y → boolean, then [Q] is a statement that updates the state according to relation Q, choosing one state nondeterministically if several final states are possible and blocking it no final state according to Q exists. All further statements are defined in terms of these five core statements. These five statements can for example be defined by higher order predicate transformers, i.e. function mapping predicates (the postconditions) to predicates (the preconditions) as done by Back and von Wright [7]. States are typically tuples and program variables are used to selects components of the state tuple. For example, if the state space is X = integer × integer and variables x, y are used to refer to the two integer components, then a state predicate p can be defined as p(x, y) = (x > y). We assume the state space is understood from context, allowing us to write boolean expressions instead of state predicates in assertions and guards, for example the assertion {x > y}. We define skip = {true} = [true] to be the statement that does nothing, abort = {false} to be the statement that always aborts, and wait = [false] to be the statement that always blocks. Assume b is a boolean expression. The assertion statement assert b is synonymous to {b}. The guarded statement when b do S and the conditional statements if b then S and if b then S else T are defined as: when b do S = [b] ; S if b then S = ([b] ; S) [¬b] if b then S else T = ([b] ; S) ([¬b] ; T) Suppose x : X and y : Y are the only program variables. The assignment statement x := e updates x and leaves y unchanged. The nondeterministic assignment statement x :∈ s assigns x an arbitrary element of the set s and leaves y unchanged. If s is the empty set then the statement blocks. Both are defined in terms of an update statement: x := e = [Q] x :∈ s = [Q]
where where
Q(x, y)(x , y ) = (x = e) ∧ (y = y) Q(x, y)(x , y ) = (x ∈ s) ∧ (y = y)
The declaration of a local variable var x :∈ s • S extends the state space by x, executes S, and reduces the state space again. The initial value of x is chosen nondeterministically from the set s. If s is the empty set then the statement blocks. We write var x : X • S or simply var x • S if an arbitrary element of type X is chosen initially. [Q] ; S ; [R] var x :∈ s • S =
where
Q y (x , y ) = (x ∈ s) ∧ (y = y) R (x, y) y = (y = y)
Following theorem gives laws for transforming statements into equivalent ones. We let e[x\f ] stand for simultaneously substituting variables x by expressions f in e:
Concurrent Object-Oriented Programs: From Specification to Code
409
Theorem (Equational Laws). Assume x, y are disjoint lists of variables. x := e = x :∈ {x | x = e}
(1)
x :∈ {x | b} ; y :∈ {y | c} = x, y :∈ {x , y | b ∧ c[x\x ]} var x • x, y :∈ {x , y | b} = y :∈ {y | ∃x, x • b}
(2) (3)
For a statement S and predicate b, we let wp(S, b) be the weakest precondition for S to terminate and to establish postcondition b. The enabledness domain or guard of statement S is defined by grd S = ¬wp(S, false) and the termination domain by trm S = wp(S, true). The weakest liberal precondition wlp(S, b) is the weakest precondition for S to establish b provided S terminates. We give selected laws: Theorem (Weakest Preconditions). wlp(x := e, b) = b[x\e]
(4)
wlp(x :∈ s, b) = ∀x ∈ s • b wlp(S ; T, b) ⇐ wlp(S, wlp(T, b))
(5) (6)
The refinement of statement S by T, written S T, means that T terminates whenever S does, T is disabled whenever S is, and T is “more deterministic” than S. In the predicate transformer model, S T holds if for any postcondition q, whenever S establishes q so does T. Data refinement S R T generalizes (algorithmic) refinement by relating the initial and final state of S and T with relation R. We allow R to refine only part of the state, i.e. if the (initial and final) state space of S is X × Z, the state space of T is Y × Z, then it is sufficient for R to relate X to Y. We write Id for the identity relation and × for the parallel composition of relations: S R T = S ; [R × Id] [R × Id] ; T We give selected laws about data refining statements; they naturally generalize when only a specific component of a larger state space is refined. Theorem (Refinement Laws). Assume that relation R relates X to Y and the state space includes Z. Variables x, y, z refer to the corresponding state components: x := e R y := f if R x y ⇒ R e f {a} ; x := e R {b} ; y := f iff a ∧ R x y ⇒ b and a ∧ R x y ⇒ R e f x := e R y :∈ {y | d} if R x y ∧ d ⇒ R e y {a} ; x := e R {b} ; y :∈ {y | d} iff a ∧ R x y ⇒ b and a ∧ R x y ∧ d ⇒ R e y
(7) (8) (9) (10)
x :∈ {x | c} R y :∈ {y | d} if R x y ∧ d ⇒ ∃ x • c ∧ R x y (11) (12) {a} ; x :∈ {x | c} R {b} ; y :∈ {y | d} iff a ∧ R x y ⇒ b and a ∧ R x y ∧ d ⇒ ∃ x • c ∧ R x y z :∈ {z | c} R z :∈ {z | d} if R x y ∧ d ⇒ c {a} ; z :∈ {z | c} R {b} ; z :∈ {z | d} if a ∧ R x y ⇒ b and a ∧ R x y ∧ d ⇒ c
(13) (14)
410
E. Sekerinski
S1 ; S2 R T1 ; T2 if S1 R T1 and S2 R T2
(15)
S1 S2 R T1 T2 if S1 R T1 and S2 R T2
(16)
The iteration statement Sω repeats S an arbitrary number of times, as long as S is enabled. If S never becomes disabled, then Sω aborts. Iteration Sω is defined as the least fixed point (with respect to the refinement relation) of the equation X = (S ; X) skip. The while statement while b do S is defined in terms of iteration, with the additional restriction that upon termination ¬b must hold: while b do S = ([b] ; S)ω ; [¬b] Modules. A module declares a number of variables with initial values as well as a number of procedures. The procedures operate on the local variables and possibly variables declared in other modules either directly or by calling other procedures. Formally a module is a pair (init, proc) where init is the initial local state and proc is a tuple of statements. The syntax for defining a module with a two variables p, q of types P, Q with initial values p0 , q0 and a single procedure m is as follows: module K var p : P := p0 var q : Q := q0 procedure m(u : U, res v : V) M end Formally we have K = (init, proc) with init = (p0 , q0 ) and proc = M. The (initial and final) state space of the body M of m is U × V × X, where X is the state space of the whole program, which includes P and Q as components. Again, K.p or simply p is the name used to select the corresponding state component. Procedure names are used for selecting the components of proc: we write K.m or simply m in order to refer to statement M. A procedure call m(e, z) extends the state space by the formal value and result parameters, copies the actual value parameters to the formal parameters, executes the procedure body, and copies the formal result parameters to the actual result parameters: m(e, x) = var u, v • u := e ; m ; x := v Within modules other modules may be referred. The state space of the whole program is the combined state space of all modules of that program. Concurrency. Concurrency is introduced by adding actions to modules. These actions may access variables of that module and variables of other modules, either directly or through procedures. Actions that access disjoint sets of variables may be executed in any order or in parallel. Module actions are executed atomically, i.e. either an action is enabled and can be carried to completion or it is not enabled (in contrast to class actions that are atomic only up to method calls). Formally a concurrent module is a triple (init, proc, act) where in addition act is the combined action of the module. We use following syntax for defining a module with actions a and b:
Concurrent Object-Oriented Programs: From Specification to Code
411
module K var p : P := p0 var q : Q := q0 procedure m(u : U, res v : V) M action a A action b B end We have K = (init, proc, act) with init = (p0 , q0 ), proc = M, and act = A B. All actions are combined into a single action and the names of the actions do not carry any meaning. The state space of act is the state space of the whole program, which includes P and Q as components.
Definition (Module Refinement). Module K = (init, proc) with variables p is refined by module K = (init , proc , act) with variables p through relation R, written K R K , if: (a) for the initialization: R init init (b) for every procedure m: (b.1) procedure refinement: K.m R K .m (b.2) procedure enabledness: grd K.m ∧ R p p ⇒ grd K .m ∨ grd act (c) for the action: (c.1) action refinement: skip R act (c.2) action termination: R p p ⇒ trm( do act od ) The loop do S od repeats S as long as it is enabled. It is defined as Sω ; [¬grd S]. Compared to the while loop, the guard is implicit in the body. Condition (a) requires that the initializations are in the refinement relation. Condition (b.1) requires that each procedure of K is refined by the corresponding procedure of K . While refinement by itself allows the guard to be weakened, condition (b.2) requires that whenever K.m is enabled, either K .m or act must be enabled. Condition (c.1) requires that the effect of the action act is not visible when viewed from K. Finally, condition (c.2) requires that act eventually disables itself, hence cannot introduce non-termination. The definition can be applied when K has no action by taking act = wait. As grd wait = false condition (b.2) simplifies to grd K.m ∧ r ⇒ grd K .m and condition (c) holds by default. Module refinement can be generalized in several ways: K may also be allowed to have an action, allowing to increase the concurrency of an already concurrent module [23]. Both K and K can exhibit finite stuttering and the generalized rule can be shown to be correct with respect to trace refinement [11]. We have restricted the refinement relation to relate only the local variables. The refinement relation can be generalized to include global variables, at the expense of losing compositionality [11, 23].
412
E. Sekerinski
4 Objects We distinguish between the class and the type of an object. The class defines the attributes and the methods of objects. We define a class in terms of a module with one variable for each attribute, one procedure for each method, and an extra variable for the objects populating that class. The variables map each object of the class to the corresponding attribute values. Each procedure takes an additional value parameter, this, for the object to which the procedure is applied. We assume the type Object is infinite and contains the distinguished element nil. All objects are of type Object. We write x :∈ /s as a shorthand for x :∈ s: = module C class C var C : set of Object := {} attr p : P var p : Object → P initialization (g : G) procedure new(g : G, res this : Object) I this :∈ / C ∪ {nil} ; C := C ∪ {this} ; I method l(s : S, res t : T) procedure l(this : Object, s : S, res t : T) L {this ∈ C} ; L method m(u : U, res v : V) procedure m(this : Object, u : U, res v : V) M {this ∈ C} ; M end end Within a method body attribute p is referred to by this.p. In general, referencing x.p amounts to applying the function p to x. Creating a new object x of class C with initialization parameter e amounts to calling the new procedure of class C. Calling the method m of an object x of class C amounts to calling the procedure m of class C with x as the additional parameter that is bound to this in m: x.p = p(x) x := new C(e) = C.new(e, x) x.m(f , z) = C.m(x, f , z) We follow the practice of using class names as if they were types in variable declarations, e.g. c : C. While the type of c is Object, the class name C is used to determine the module to which method calls to c go. The class name can also be used by the compiler to forbid certain assignments. We illustrate these concepts by an example of points in a plane. class Point attr x : integer attr y : integer initialization (x : integer, y : integer) this.x, this.y := abs(x), abs(y) method distance(p : Point, res d : integer) d := abs(this.x − p.x) + abs(this.y − p.y) method copy( res p : Point) p := new Point(this.x + 2, this.y + 2) end
Concurrent Object-Oriented Programs: From Specification to Code
413
Class Point translates to following module. We write f [a ← b] for modifying function f to return b for argument a. The assignment x.p := e, or equivalently p(x) := e, stands for p := p[x ← e]. For convenience we continue to write x.p instead of p(x): module Point var Point : set of Object := {} var x : Object → integer var y : Object → integer procedure new(x : integer, y : integer, res this : Object) this :∈ / Point ∪ {nil} ; Point := Point ∪ {this} ; this.x, this.y := abs(x), abs(y) procedure distance(this : Object, p : Object, res d : integer) {this ∈ Point} ; d := abs(this.x − p.x) + abs(this.y − p.y) procedure copy(this : Object, res p : Point) {this ∈ Point} ; new(this.x + 2, this.y + 2, p) end We sketch how to verify invariance properties of classes. For example, consider showing that (this.x ≥ 0) ∧ (this.y ≥ 0) is an invariant of class Point: this requires proving that I defined as ∀ this ∈ Point • (x(this) ≥ 0) ∧ (y(this) ≥ 0) is an invariant of the module Point. This holds if the initial values imply the invariant, (Point = {}) ⇒ I, and each procedure preserves the invariant, I ⇒ wlp(Point.new, I), I ⇒ wlp(Point.distance, I), and I ⇒ wlp(Point.copy, I). For new we have by using (4), (5), and (6): wlp(Point.new, I) = wlp(this :∈ / Point ∪ {nil} ; Point := Point ∪ {this} ; x(this), y(this) := abs(x), abs(y), ∀ p ∈ Point • (x(p) ≥ 0) ∧ (y(p) ≥ 0)) ⇐ wlp(this :∈ / Point ∪ {nil} ; Point := Point ∪ {this}, ∀ p ∈ Point • (x[this ← abs(x)](p) ≥ 0) ∧ (y[this ← abs(y)](p) ≥ 0)) ⇐ ∀this :∈ / Point ∪ {nil} • ∀ p ∈ Point ∪ {this} • (x[this ← abs(x)](p) ≥ 0) ∧ (y[this ← abs(y)](p) ≥ 0)) ⇐I While we allow references this.a to attributes of the object itself to be abbreviated by a, care has to be taken as this involves a hidden function application, which is the source of aliasing. For example, consider adding method tile to class Point: method tile(p : Point) p.x := x + 2 ; p.y := y We might be tempted to conclude that the postcondition p.x = x + 2 is always established. Expanding the body to x(p) := x(this)+ 2 ; y(p) := y(this) and the postcondition to x(p) = x(this) + 2 makes it evident that this is only true if initially this = p, i.e. the postcondition does not hold for the call p.tile(p), with p ∈ Point. We turn our attention to inheritance. Suppose C is as earlier and class D inherits from C, while adding attributes and methods and redefining the initialization and some methods. We call C the superclass of D and D the subclass of C. This corresponds to defining a module D that uses module C:
414
E. Sekerinski
class D inherit C attr q : Q initialization (h : H) J method m(u : U, res v : V) M method n(w : W, res y : Y) N end
=
module D var D : set of Object := {} var q : Object → Q procedure new(h : H, res this : Object) this :∈ / C ∪ {nil} ; C := C ∪ {this} ; D := D ∪ {this} ; J procedure l(this : Object, s : S, res t : T) {this ∈ D} ; C.l(this, r, s) procedure m(this : Object, u : U, res v : V) {this ∈ D} ; M procedure n(this : Object, w : W, res y : Y) {this ∈ D} ; N end
Those methods that are not explicitly redefined in D are defined in D as forwarding the call to C. Method bodies may contain calls to other methods of the same class, either to the same object, this.m(e, z) or to another object, x.m(e, z). The call this.m(e, z) is also written m(e, z). A method body in D may also contain a super-call super.m(e, z). In this case the call goes to the inherited class, i.e. the immediate superclass. This also applies to inheritance hierarchies with more than two classes: super.m(e, z) = C.m(e, z) We illustrate these issues with classes Point1D and Point2D: class Point1D attr x : integer method setX(x : integer) this.x := x method scale(s : integer) this.x := this.x × s end
class Point2D inherit Point1D attr y : integer method setY(y : integer) this.y := y method setXY(x, y : integer) this.setX(x) ; this.setY(y) method scale(s : integer) super.scale(s) ; this.y := this.y × s end
These classes translate to following modules: module Point1D var Point : set of Object := {} var x : Object → integer procedure new( res this : Object) this :∈ / Point1D ∪ {nil} ; Point1D := Point1D ∪ {this} procedure setX(this : Object, x : integer) {this ∈ Point1D} ; this.x := x procedure scale(this : Object, s : integer) {this ∈ Point1D} ; this.x := this.x × s end
Concurrent Object-Oriented Programs: From Specification to Code
415
module Point2D var Point2D : set of Object := {} var y : Object → integer procedure new( res this : Object) this :∈ / Point2D ∪ {nil} ; Point1D := Point1D ∪ {this} ; Point2D := Point2D ∪ {this} procedure setX(this : Object, x : integer) {this ∈ Point2D} ; Point1D.setX(this, x) procedure setY(this : Object, y : integer) {this ∈ Point2D} ; this.y := y procedure setXY(this : Object, y : integer) {this ∈ Point2D} ; setX(this, x) ; setY(this, y) procedure scale(this : Object, s : integer) {this ∈ Point2D} ; Point1D.scale(this, s) ; this.y := this.y × s end
Inheritance does not affect the creation of objects, i.e. if D inherits from C then x := new D(e) = D.new(e, x). A key point of the definition of inheritance is that a new object of class D becomes also a member of class C, that is D is a subtype of C. Subtypes correspond to subsets between the members of the class, C ⊆ D. Assuming c, d are objects, the type test c is D tests whether c is indeed an object of class D. The type cast d := c as D aborts if c is not an object of class D and assigns c to d otherwise. Assuming D is a subtype of C and c is declared to be of class C, the method call c.m(e, z) is bound dynamically, i.e. the actual class of c rather than the declared class determines the module to which the call goes. This generalizes to class hierarchies involving more than two classes accordingly: c is D = c∈D d := c as D = {c ∈ D} ; d := c c.m(e, z) = if c ∈ D then D.m(c, e, z) else C.m(c, e, z) Within the bodies of the methods of class D attributes of class C may be referred to. The type system would either allow or forbid this according to visibility declarations; we do not explicitly indicate visibility here. However, we note that if modification of C attributes is allowed in D, then an invariant shown to hold for C objects does not necessarily hold for D objects. Such an invariant has also to be shown to be preserved by D methods. We can also define inheritance without subtyping, which we call extension. When class E extends class C, the methods of E may refer to the attributes of C, but creating an E object does not make it a C object. Methods are not inherited and no super-calls are possible (although one could generalize this). Hence this only allows sharing of attribute declarations. In case of extension the type system would forbid assignments between E and C objects:
416
E. Sekerinski
class E extend C attr q : Q initialization (h : H) J method m(v : U, res v : V) M method n(w : W, res y : Y) N end
=
module E var E : set of Object := {} var q : Object → Q procedure new(h : H, res this : Object) this :∈ / D ∪ {nil} ; D := D ∪ {this} ; J procedure m(this : Object, u : U, res v : V) {this ∈ D} ; M procedure n(this : Object, w : W, res y : Y) {this ∈ D} ; N end
Now we show how class refinement translates to module refinement. We give an example that involves creation of auxiliary objects and creation of garbage—objects to which there is no reference. Consider following class S for defining a store in which we only record whether the store is empty or full: = class S attr f : boolean initialization f := false method full( res r : boolean) r := f method store f := true end
module S var S : set of Object := {} var f : Object → boolean procedure new( res this : Object) this :∈ / S ∪ {nil} ; S := S ∪ {this} ; this.f := false procedure full(this : Object, res r : boolean) {this ∈ S} ; r := this.f procedure store(this : Object) {this ∈ S} ; this.f := true end
In the refinement LS the boolean attribute f becomes a link l to another object of class LS. Initially l is nil and is set to some object of class LS in store. Hence, repeated calls to store will generate garbage: = class LS attr l : LS initialization f := nil method full( res r : boolean) r := l = nil method store l := new LS end
module LS var LS : set of Object := {} var l : Object → Object procedure new( res this : Object) this :∈ / LS ∪ {nil} ; LS := LS ∪ {this} ; this.l := nil procedure full(this : Object, res r : boolean) {this ∈ LS} ; r := this.l = nil procedure store(this : Object) {this ∈ LS} ; new(this.l) end
We show refinement between modules S and LS with relation R defined by: R(S, f )(LS, l) = (S ⊆ LS) ∧ (∀ s ∈ S • f (s) = (l(s) = nil))
Concurrent Object-Oriented Programs: From Specification to Code
417
Condition (a) of module refinement, R({}, f )({}, l), holds immediately. To show condition (b.2) for new we rewrite the bodies using (1) and (2): S.new = this, S, f := {this , S , f | / S ∪ {nil}) ∧ (S = S ∪ {this }) ∧ (f = f [this ← false])} (this ∈ LS.new = this, LS, l := {this , LS , l | / LS ∪ {nil}) ∧ (LS = LS ∪ {this }) ∧ (l = l[this ← nil])} this ∈ Refinement is now established by first applying (11) and then eliminating LS , l , S , f by the one-point rule: S.new R LS.new = (S ⊆ LS) ∧ (∀ s ∈ S • f (s) = (l(s) = nil)) ∧ (this ∈ / LS ∪ {nil}) ∧ (LS = LS ∪ {this }) ∧ (l = l[this ← nil]) ⇒ (∃ S , f • (this ∈ / S ∪ {nil}) ∧ (S = S ∪ {this }) ∧ (f = f [this ← false]) ∧ (S ⊆ LS ) ∧ (∀ s ∈ S • f (s) = (l (s) = nil))) = (S ⊆ LS) ∧ (∀ s ∈ S • f (s) = (l(s) = nil)) ∧ (this ∈ / LS ∪ {nil})∧ ⇒ (this ∈ / S ∪ {nil}) ∧ (S ∪ {this} ⊆ LS ∪ {this }) ∧ (∀ s ∈ S ∪ {this } • f [this ← false](s) = (l[this ← nil](s) = nil)) = true For procedure full we immediately apply (8): S.full R LS.full = ((this ∈ S) ∧ (S ⊆ LS) ∧ (∀ s ∈ S • f (s) = (l(s) = nil)) ⇒ (this ∈ LS)) ∧ ((this ∈ S) ∧ (S ⊆ LS) ∧ (∀ s ∈ S • f (s) = (l(s) = nil)) ⇒ (f (this) = (l(this) = nil))) = true We rewrite procedure procedure S.store using the definitions. For procedure LS.store we expand the call, rename the local variable to t, apply (1) and (2) to merge the assignments, and apply (3) to eliminate the local variable: S.store = {this ∈ S} ; this, S, f := this, S, f [this ← false])} LS.store = {this ∈ LS} ; this, LS, l := {this , LS , l | (this = this) ∧ / LS ∪ {nil}) ∧ (LS = LS ∪ {t}) ∧ (∃ t • (t ∈ (l = l[t ← nil][this ← t])) Refinement of store is established by applying (10); we leave out the details of the proof. To show condition (b.2) we first observe that grd LS.new = true, grd LS.full = true, and grd LS.store = true, i.e. all procedures are always enabled. Therefore condition (b.2) is immediately satisfied for all procedures. This completes the proof.
5 Concurrent Objects Classes with actions are translated to modules with actions, such that there is one action for each object of the class. This is formally expressed by nondeterministically assigning any element of C to this before executing the action body. If C is empty, no action is
418
E. Sekerinski
enabled. For the time being we make the restriction that method calls can appear only as the first statement in methods and actions. = module C class C var C : set of Object := {} attr p : P var p : Object → P initialization (g : G) procedure new(g : G, res this : Object) I this :∈ / C ∪ {nil} ; C := C ∪ {this} ; I method l(s : S, res t : T) procedure l(this : Object, s : S, res t : T) L {this ∈ C} ; L method m(u : U, res v : V) procedure m(this : Object, u : U, res v : V) M {this ∈ C} ; M action a action a A var this :∈ C • A action b B action b end var this :∈ C • B end Inheritance and subtyping works as for classes without actions. Refinement of classes with actions translates to refinement of modules with actions. We give an example that illustrates the concept of delaying a computation by enabling a background action. Class Doubler allows to store an integer and to retrieve its double. Class DelayedDoubler doubles the integer in the background and blocks if the integer to be retrieved is not yet doubled: class DelayedDoubler class Doubler attr y : integer attr x : integer attr d : boolean method store(u : integer) initialization d := true this.x := 2 × u method store(u : integer) method retrieve( res u : integer) y, d := u, false u := this.x method retrieve( res u : integer) end when d do u := y action double when ¬d do y, d := 2 × y, true end These classes translate to modules in the same fashion as previous examples. We give immediately the refinement relation needed to prove that Doubler is refined by DelayedDoubler: R(Doubler, x)(DelayedDoubler, y, d) = (Doubler = DelayedDoubler) ∧ (∀ o ∈ Doubler • (d(o) ∧ y(o) = x(o)) ∨ (¬d(o) ∧ (2 × y(o) = x(o))) We conclude this example by noting that we can alternatively express DelayedDoubler as a subtype of Doubler, thus arriving at a class hierarchy in which concurrency is introduced in a subclass:
Concurrent Object-Oriented Programs: From Specification to Code
419
class DelayedDoubler inherit Doubler attr d : boolean initialization d := true method store(u : integer) x, d := u, false method retrieve( res u : integer) when d do u := x action double when ¬d do x, d := 2 × x, true end Statements in classes are atomic only up to method calls. If method calls appear not only as the first statement in methods and actions, the class has to be normalized first. A method or action body with such a call has to be split in order to model that execution can block at that point. If at the point of the method call there are no local variables, then we introduce an auxiliary integer variable that is initialized to zero and incremented at the point of the method call. For every call we also introduce an action that contains the call and the remainder of the body. This action is enabled if the counter for that call is positive and the action decrements the counter first. We illustrate this by an example of a faulty merger: class FaultyMerger attr in1, in2, out : Buffer attr x1, x2 : integer initialization (i1, i2, o : Buffer) in1, in2, out := i1, i2, o action begin in1.get(x1) ; out.put(x1) end action begin in2.get(x2) ; out.put(x2) end end Class FaultyMerger is normalized as follows: class FaultyMerger attr in1, in2, out : Buffer attr x1, x2 : integer attr at1, at2 : integer initialization (i1, i2, o : Buffer) in1, in2, out := i1, i2, o action begin in1.get(x1) ; at1 := at1 + 1 end action when at1 > 0 do begin at1 := at1 − 1 ; out.put(x1) end action begin in2.get(x2) ; at2 := at2 + 1 end action when at2 > 0 do begin at2 := at2 − 1 ; out.put(x2) end end If in1 contains sufficiently many elements, the action in1.get(x1) ; at1 := at1 + 1 can be taken several times and overwriting x1 before the action for placing x in out is taken. The class Merger avoids this problem with the help of an extra variable.
420
E. Sekerinski
¬inPool thread pool
inPool
inPool
object pool
Fig. 1. Illustration of the implementation. Boxes with the inPool attribute represent active objects, the other passive objects. A thin arrow between boxes represents a reference, a thick arrows from a thread to an object represents a reference with a lock.
Suppose there are local variables at the point of the method call. These local variables form the context in which execution may resume, after possible interleaving with other methods or action. This is modelled by storing the context in an attribute with each object. As multiple activities may create local contexts, but the order of creation is ignored, the contexts are stored in a bag. We illustrate this by normalizing the action notifyOneObserver of class Subject: attr at1 : bag of Observer action notifyOneObserver when notifyObs ={} do var o : Observers • begin o :∈ notifyObs ; notifyObs := notifyObs − {o} ; at1 := at1 + [o] end action notifyOneObserver var o :∈ at1 • begin at1 := at1 − [o] ; o.update end This normalization step is required before verification and refinement can be carried out by translating classes to modules.
6 Implementation In order to test our ideas, we have developed a prototypical compiler for our language, see [17] for details. The compiler currently translates to the Java Virtual Machine. We sketch the principles of the implementation, see Fig. 1 for an illustration. The implementation relies on the restriction that method and action guards may refer only to attributes of the object itself and may not contain method calls. An object that has guarded methods is called a guarded object. An object that has actions is called an active object, otherwise a passive object. An active object that has at least one enabled action is called an enabled object, otherwise a disabled object. At runtime a thread pool and an object pool are maintained. The object pool is initially empty. When an active object is created, a pointer to it is placed in the object
Concurrent Object-Oriented Programs: From Specification to Code
421
pool and only active objects are placed in the object pool. Each active object has an extra boolean attribute inPool indicating whether a pointer to it is in the object pool. Threads request a reference to an active object from object pool. If the object is disabled, the thread resets the inPool attribute and removes it from the object pool. If the object is enabled, the thread executes an enabled action and leaves the object in the object pool. Each thread obtains a lock to an object when entering one of its methods or actions and releases the lock when exiting the method or action. The lock is also released at a call to another object and obtained again at re-entry from the call. If a guarded method is called the guard is evaluated and the thread waits if the guard is false. At the exit from a guarded object all waiting threads are notified to reevaluate the guards. Fairness among the actions of an object is ensured by evaluating the guards in a cyclic fashion. This is done with one additional attribute for the index of the last evaluated action guard in every active object. The object pool is implemented as a dynamic array. Fairness among the objects is ensured by retrieving active objects in a cycling fashion. The object pool grows and shrinks like a stack: new objects are added at the end and when an object is retrieved, its position is filled with the last object. Hence adding objects and retrieving objects take constant time. Active objects are garbage collected like passive objects, i.e. when there is no reference from any other object and no reference from the object pool. With this scheme action guards are only evaluated when a thread is searching for an action to execute. Method guards are only re-evaluated when another thread has exited the object and thus possibly affected the guard. The memory overhead is that every active object requires one bit for the inPool attribute, one integer for the index to the last evaluated action guard, and one pointer in the object pool. We are currently experimenting with techniques to control the creation and termination of threads.
7 Discussion A number of attempts have been made in formalizing objects with records, initiated by the work of Cardelli [12] and leading to various type systems incorporating objectoriented concepts. Our experience in using one such type system is that in verification and refinement it is more in the way than helpful [22]. Understanding attributes as mappings from object identities to their values emerges naturally in object modeling techniques like [21], and is used in a number of formalizations of object models, e.g. [15]. The approach of viewing a method as a procedure with an additional this parameter is also taken in Modula-3 and Oberon-2. We find that this combination leads to a simple model with a clear distinction between classes and types. A consequence is that objects can only be allocated on the heap, an approach also taken in several mainstream objectoriented languages. One may argue about releasing the lock to an object when a method call in that object goes to another object, hence allowing other methods to be called or actions to be initiated. Indeed in our first implementation we retained the lock. However we found programs to be difficult to analyze, as it is necessary to keep track of which objects are locked by which actions. The model of releasing the lock allows some disciplined intra-object concurrency: while several actions or methods can be initiated, only one
422
E. Sekerinski
can progress, thus still guaranteeing atomicity of attribute updates. In order for a class invariant to be preserved, the class invariant has not only to be established at the end of every method, but also before each call to another object. The need for doing so is already recognized in sequential programs when re-entrance is possible [19]. The model presented does not define (indirectly) recursive method calls; doing so would require taking a fixed point. The model also does not accurately capture how self- and super-calls are resolved when methods are redefined: a super-call will always remain in the superclass, even if calling other methods that are redefined in the subclass. In order to model this, methods calls must not be resolved immediately, but when objects of the classes are created. A model of inheritance that delays resolution of method calls to the time when objects are created was proposed by Cook and Palsberg [13] and applied to studies of class refinement by Mikhajlov and Sekerinski [18]. While our implementation follows this model, the presented theory does not capture this.
Acknowledgement The author’s understanding profited from the interaction with participants of FMCO 02; discussions with Rustan Leino are particularly acknowledged. The comments of the reviewer lead to a significant improvement.
References 1. Pierre America. Issues in the design of a parallel object-oriented language. Formal Aspects of Computing, 1(4):366–411, 1989. 2. Pierre America and Frank de Boer. Reasoning about dynamically evolving process structures. Formal Aspects of Computing, 6(3):269–316, 1994. 3. Ralph Back. Refinement calculus, part II: Parallel and reactive programs. In J. W. deBakker, W.-P. deRoever, and G. Rozenberg, editors, REX Workshop on Stepwise Refinement of Distributed Systems - Models, Formalisms, Correctness, Lecture Notes in Computer Science 430, pages 67–93, Mook, The Netherlands, 1989. Springer Verlag. 4. Ralph Back, Martin B¨uchi, and Emil Sekerinski. Action-based concurrency and synchronization for objects. In T. Rus and M. Bertran, editors, Transformation-Based Reactive System Development, Fourth AMAST Workshop on Real-Time Systems, Concurrent, and Distributed Software, Lecture Notes in Computer Science 1231, pages 248–262, Palma, Mallorca, Spain, 1997. Springer-Verlag. 5. Ralph Back and Kaisa Sere. Action systems with synchronous communication. In E.-R. Olderog, editor, IFIP Working Conference on Programming Concepts, Methods, Calculi, pages 107–126, San Miniato, Italy, 1994. North-Holland. 6. Ralph Back and Joakim von Wright. Trace refinement of action systems. In B. Jonsson and J. Parrow, editors, CONCUR ’94: Concurrency Theory, Lecture Notes in Computer Science 836. Springer-Verlag, 1994. 7. Ralph Back and Joakim von Wright. Refinement Calculus – A Systematic Introduction. Springer-Verlag, 1998. 8. Marcello M. Bonsangue, Joost N. Kok, and Kaisa Sere. An approach to object-orientation in action systems. In Mathematics of Program Construction, Lecture Notes in Computer Science 1422, Marstrand, Sweden, 1998. Springer-Verlag.
Concurrent Object-Oriented Programs: From Specification to Code
423
9. Marcello M. Bonsangue, Joost N. Kok, and Kaisa Sere. Developing object-based distributed systems. In P. Ciancarini, A. Fantechi, and R. Gorrieri, editors, 3rd IFIP International Conference on Formal Methods for Open Object-based Distributed Systems (FMOODS’99), pages 19–34. Kluwer, 1999. 10. Jean-Pierre Briot, Rachid Guerraoui, and Klaus-Peter Lohr. Concurrency and distribution in object-oriented programming. ACM Computing Surveys, 30(3):291–329, 1998. 11. Martin B¨uchi and Emil Sekerinski. A foundation for refining concurrent objects. Fundamenta Informaticae, 44(1):25–61, 2000. 12. Luca Cardelli. A semantics of multiple inheritance. In G. Kahn, D. MacQueen, and G. Plotkin, editors, International Symposium on the Semantics of Data Types, Lecture Notes in Computer Science 173, pages 51–67. Springer-Verlag, 1984. 13. William Cook and Jens Palsberg. A denotational semantics of inheritence and its correctness. In ACM Conference Object Oriented Programming Systems, Languages and Applications, ACM SIGPLAN Notices, Vol 14, No 10, pages 433–443, 1989. 14. Steve J. Hodges and Cliff B. Jones. Non-interference properties of a concurrent object-based language: Proofs based on an operational semantics. In Burkhard Freitag, Cliff B. Jones, Christian Lengauer, and Hans-Joerg Schek, editors, Object Orientation with Parallelism and Persistence, pages 1–22. Kluwer Academic Publishers, 1996. 15. Daniel Jackson. Alloy: A lightweight object modelling notation. ACM Transactions on Software Engineering and Methodology, 11(2):256–290, 2002. 16. Cliff B. Jones. Accomodating interference in the formal design of concurrent object-based programs. Formal Methods in System Design, 8(2):105–122, March 1996. 17. Kevin Lou. A Compiler for an Action-Based Object-Oriented Programming Language. Master’s thesis, McMaster University, 2003. 18. Leonid Mikhajlov and Emil Sekerinski. A study of the fragile base class problem. In Eric Jul, editor, ECOOP’98 – 12th European Conference on Object-Oriented Programming, Lecture Notes in Computer Science 1445, pages 355–382, Brussels, Belgium, 1998. Springer-Verlag. 19. Leonid Mikhajlov, Emil Sekerinski, and Linas Laibinis. Developing components in presence of re-entrance. In J. Wing, J. Woodcock, and J. Davis, editors, World Congress on Formal Methods, FM’99, Lecture Notes in Computer Science 1709, Toulouse, France, 1999. Springer-Verlag. 20. Jayadev Misra. A simple, object-based view of multiprogramming. Formal Methods in System Design, 20(1):23–45, 2002. 21. James Rumbaugh, Michael Blaha, William Premerlani, Frederick Eddi, and William Lorensen. Object-Oriented Modeling and Design. Prentice-Hall, 1991. 22. Emil Sekerinski. A type-theoretic basis for an object-oriented refinement calculus. In S. J. Goldsack and S. J. H. Kent, editors, Formal Methods and Object Technology, pages 317–335. Springer-Verlag, 1996. 23. Kaisa Sere and Marina Wald´en. Data refinement of remote procedures. Formal Aspects of Computing, 12(4):278–297, 2000.
Design with Asynchronously Communicating Components J. Plosila1 , K. Sere2 , and M. Wald´en2,3 1
University of Turku Turku Centre for Computer Science (TUCS) 2 ˚ Abo Akademi University Turku Centre for Computer Science (TUCS) FIN-20520 Turku, Finland 3 Financing via the Academy of Finland
Abstract. Software oriented methods allow a higher level of abstraction than the often quite low-level hardware design methods used today. We propose a component-based method to organise a large system derivation within the B Method via its facilities as provided by the tools. The designer proceeds from an abstract high-level specification of the intended behaviour of the target system via correctness-preserving transformation steps towards an implementable architecture of library components which communicate asynchronously. At each step a pre-defined component is extracted and the correctness of the step is proved using the tool support of the B Method. We use Action Systems as our formal approach to system design.
1
Introduction
When carrying out formal specification and derivation of systems we can apply methods that allow a high-level abstract specification of a system to be stepwise developed into a more concrete version by correctness- preserving transformations. Hence, these methods provide a top-down approach to system design where the initial specification is a very abstract view of the system to be built. Details about the intended functionality and hardware/software components of the system are stepwise added to the specification during the design while preserving the intended behaviour of the original description. While this approach is very appealing from the designer’s point of view as it allows the system to be developed and verified in manageable tasks, it still lacks e.g. good tool support. In this paper, we concentrate on formal specification and derivation of asynchronous systems within the Action Systems formalism. Such systems may contain locally synchronous components, but the components interact with each other via asynchronous communication channels. This kind of architecture provides a promising, modular and reliable approach to implement modern large digital systems. Action systems [3] and the associated refinement calculus [4], which provides a mathematical reasoning basis for the stepwise development of action F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 424–442, 2003. c Springer-Verlag Berlin Heidelberg 2003
Design with Asynchronously Communicating Components
425
systems, have shown their value in design of reliable and correct systems in many ways [18]. The formal design methods supporting action systems and reasoning about them are heavily influenced by approaches to parallel and distributed program design as well as approaches to object-oriented programming [3,6]. Recently, methods to derive systems from an abstract specification all the way down to VLSI circuits within action systems have received considerable attention [15,17]. An action systems-based design process of an asynchronous system starts from an abstract specification which is then decomposed into asynchronously communicating library components. Each component can be implemented as an asynchronous (self-timed) [15] or a synchronous (clocked) [17] hardware module, or developed into an executable piece of software which is run in a standard microprocessor. Utilisation of formal methods is particularly important when designing such complex embedded systems, as it potentially reduces the design costs by eliminating errors and wrong design decisions at an early stage of the design process, before the involved costly hardware modules, e.g. VLSI chips, have been manufactured. The focus in this paper is on the correctness preserving stepwise decomposition of a system specification into an asynchronous network of action system components pre-defined in a module library available to the designer. This is a very important design phase as it determines the basic structure of the system. The next phases, which include for example the software/hardware partitioning and the decision on the type of the hardware modules (clocked/self-timed), are out of the scope of this paper. In order to get more confidence in the development of asynchronous systems we need mechanical tool support. Recently, Wald´en et al. [8,21] have shown how action systems and, hence, parallel and distributed systems can be modelled with the B Method, an approach that with its tool support [1,13,20] has been accepted by many industrial organisations. The B Method, similarly to action systems, supports stepwise derivation of programs. Moreover, the B Method offers automatic proof-support for the verification of the correctness of each step as well as tools to manage and administrate a derivation task. Hence, the very well-developed and rich theory for Action Systems can utilise the mechanical tool support provided by the B Method. In this paper we will further extend the applicability area of the B Method to provide support for formal asynchronous system design. Our goal is component based design, where we can utilise the component libraries provided by the tools. We propose a method to administrate a large system derivation within the B Method via stepwise decomposition. We start in Section 2 by formalising the concepts of asynchronous systems in the B Method using the Action Systems approach. In Section 3 we develop the refinement and the component extraction ideas and associated tool support needed in design of asynchronous systems in a quite high level of abstraction. Section 4 is then devoted to a case study on component- based design. We end in Section 5 with comparisons to related work and some concluding remarks.
426
2
J. Plosila, K. Sere, and M. Wald´en
Asynchronous Systems in B
Asynchronous interfacing provides a viable approach to build modern large digital systems, composed of several hardware and software units, in a modular and reliable manner. In asynchronous communication, a data transfer event between two system components consists of two phases: request and acknowledgement (Section 2.4). Depending on the application, the duration of each phase may be either unbounded or bounded. Asynchronously communicating components form an asynchronous system architecture in which a component module, taken separately, can internally be either an asynchronous (self-timed) or synchronous (clocked) hardware block, or a software module running in a standard or application-specific processor. When decomposing an abstract functional system specification stepwise into an asynchronous composition of dedicated modules within the formal framework of Action Systems, a correctness-preserving transformation accompanied with a number of proof obligations is applied at each refinement step. In order to give more confidence in the correctness proof of a refinement step we want to use a mechanical tool. Atelier B [20] and B-Toolkit [13] provide this kind of tool. They both comprise a set of tools which support the B Method [1], a formal framework for stepwise system development. Action Systems and the B Method have essentially the same structure, both are state-based methods as opposed to event-based process calculi, and both support the stepwise refinement paradigm to system construction. This gives a solid background for combining the two methods. The designer supplies the tool, Atelier B or B-Toolkit, with the specification and a number of refinements of this specification. The verification conditions, the proof obligations, needed for proving the correctness of the refinement steps can be automatically generated. Furthermore, these verification conditions can be automatically or interactively proved using these tools. Hence, by using the B Method for designing asynchronous systems we will have tool support with proving facilities for correct design. 2.1
Action Systems in B
We model asynchronous systems in B via abstract machines. The main components of an abstract machine are the state variables, the operations on the state variables and the invariant giving the properties of these variables. For specifying the operations we use substitutions, for example, a skip-substitution, a simple substitution (x := e), a multiple substitution (x := e || y := f ), preconditioned substitution (PRE P THEN S END), an action (also called guarded substitution) (SELECT P THEN S END), or a non-deterministic substitution (ANY x WHERE THEN P ENDS ), where x and y are variables, e and f are expressions, P is a predicate, and S is a substitution. Each substitution S is defined as a predicate transformer which transforms a postcondition Q into the weakest precondition for S to establish Q, wp(S, Q) [10], the initial states from which S is guaranteed to terminate. The substitutions above are defined as follows:
Design with Asynchronously Communicating Components wp(skip, Q) wp(x := e, Q) wp(x := e || y := f, Q) wp(PRE P THEN S END, Q) wp(SELECT P THEN S END, Q) wp(ANY x WHERE P THEN S END, Q)
= = = = = =
427
Q Q[x := e] Q[x, y := e, f ] P ∧ wp(S, Q) P ⇒ wp(S, Q) (∀x.P ⇒ wp(S, Q))
The abstract machine A given below MACHINE A VARIABLES x INVARIANT I(x) INITIALISATION x := x0 OPERATIONS ˆ SELECT P1 THEN S1 END; A1 = ... ˆ SELECT Pm THEN Sm END Am = END
where every operation in the operations-clause is an action, is called an action system. An action system is identified by a unique name, here A. The state variable(s) x of the action system are given in the variables-clause. The invariant I(x) in the invariant-clause gives types and other properties for the variables. The variables are assigned initial values in the initialisation-clause. In the operationsclause each action Ai is given as a named operation. Each operation might have value parameters and/or return a result. An action A with the value parameter a and the result parameter b is denoted b ← A(a). This feature is used later to model communication in the form of message-passing between action systems. The types of the value parameters should be given as a preconditioned substituˆ PRE type(a) THEN SELECT P THEN S END END . tion in the operation b ← A(a) = For readablility of the action systems we have excluded the preconditions from the operations in this paper. Action systems are used as a model for parallel and distributed systems [3,21] with the basic idea that actions are selected for execution in a non-deterministic way. Hence, there is a non-deterministic choice between the actions A1 , . . . , Am of A, A1 [] . . . [] Am . The non-deterministic choice of the actions A and B is defined as wp(A [] B, Q) = wp(A, Q) ∧ wp(B, Q). Only actions that are enabled, i.e., when the predicate P holds for the action in a given state, are considered for execution. The execution terminates when there are no enabled actions. If two actions do not share any variables, they can be executed in any order or in parallel. Hence, we have an interleaving semantics for action systems. Example. The machine E below is a high-level action system specification of a system unit which computes a new value for data dout whenever the action E is enabled and selected for execution. The parameters l and r act as two- directional communication signals between E and its environment. In other words, when E receives values for l and r from the environment, it responds by assigning other values which are then detected by the environment. The values l and r can be assigned are req and ack corresponding to the request and acknowledgement phases of asynchronous communication (Section 2.4). The machine E receives
428
J. Plosila, K. Sere, and M. Wald´en
input data din when l = req and sends output data dout to the environment by setting r to req. MACHINE E OPERATIONS l, r, dout ← E(l, din, r, dout) = ˆ SELECT l = req ∧ r = ack THEN ANY dout WHERE F (dout , din, dout) THEN dout := dout END || r := req || l := ack END END
2.2
Scheduling of Actions
Implicitly there is a nondeterministic choice between enabled actions in the operations-clause as explained above. This can be stated explicitly in a scheduling-clause of an action system. In this clause we can also give other more specific scheduling policies like sequential composition or some parallel, exclusive, prioritised, or probabilistic composition between actions. The clause is optional, but in case a scheduling-clause appears in an action system, all the actions of the machine must be included. The scheduling has the same name as the abstract machine. If the actions have parameters, these will also be parameters of the scheduling. MACHINE A ... SCHEDULING A= ˆ A1 * . . . * A m END
In this paper we focus on the two most common composition operators between actions and consider each occurrence of * to be either a nondeterministic choice, [], or sequential composition, ;. The sequential composition is frequently needed in asynchronous modeling to sequence communication events on asynchronous communication channels discussed later in Section 2.4. The non-deterministic choice was explained above. The sequential composition of two actions can also be interpreted in terms of the non-deterministic ˆ SELECT P THEN S END and choice. Let us consider the two actions A = B = ˆ SELECT Q THEN T END . Their sequential composition, A ; B, can then be interpreted as the non-deterministic choice, A [] B , between the two actions A and B , where a variable pc (initially set to 1) for scheduling the actions has been added:
= ˆ SELECT P ∧ pc = 1 THEN S || pc := 2 END
= ˆ SELECT Q ∧ pc = 2 THEN T || pc := 1 END
A
B
Hence, B is only enabled after A has been executed, setting pc to 2. We can note that the scheduling-clause provides us with a convenient way of rewriting the scheduling of the actions, which otherwise should be coded within the actions. As an example of the scheduling, let the action system A have three actions ˆ ((A1 ; A2 ) [] A3 ). The execution of A1 , A2 and A3 and the scheduling clause A =
Design with Asynchronously Communicating Components
429
A is restricted so that A1 and A2 always are executed in a sequence interleaved with A3 . In case the actions have parameters these have to be taken into account in the scheduling. For example, the sequential execution of the two actions B1 (b) and d ← B2 (c) with associated value and result parameters b, c and d, is given as B(b, c, d) = ˆ B1 ; B2 in the scheduling clause of B. Example. In the machine Reg below the operations Reg1 and Reg2 are sequentially composed. We return to this machine later. MACHINE Reg OPERATIONS ˆ SELECT a = req THEN dout := din || b := req END; b, dout ← Reg1 (a, din) = ˆ SELECT b = ack THEN a := ack END a ← Reg2 (b) = SCHEDULING Reg(a, din, b, dout) = Reg1 ; Reg2 END
2.3 Modularisation Action systems can be composed/decomposed into parallel systems [5]. The parallel composition of action systems A and B can be presented in the B Method using the extends-clause. This also provides an efficient way to model system hierarchy, i.e., a system can be presented as a composition of subsystem modules listed in its extends-clause. MACHINE A EXTENDS B VARIABLES x INVARIANT I(x) INITIALISATION x := x0 OPERATIONS A(a) = ˆ SELECT P THEN S END SCHEDULING A(a) = ˆ A [] B(a, x) END
MACHINE B VARIABLES y INVARIANT J(y) INITIALISATION y := y0 OPERATIONS B(b, c) = ˆ SELECT Q THEN T END SCHEDULING B(b, c) = ˆ B END
Here the action system A extends the system B indicating that A is considered to be composed in parallel with B. We can also say that A contains the component (i.e. subsystem) module B. The scheduling of A is then A [] B(a, x), where a and x are the actual parameters which should be variables of A and/or formal parameters of the actions in A. The result of composing A and B in parallel is given as the system AB below. MACHINE AB VARIABLES x, y INVARIANT I(x) ∧ J(y) INITIALISATION save x := x0 || y := y0 OPERATIONS A(a) = ˆ SELECT P THEN S END B(a) = ˆ SELECT Q[a/b, x/c] THEN T [a/b, x/c] END SCHEDULING AB(a) = ˆ A [] B END
430
J. Plosila, K. Sere, and M. Wald´en
The variables, the invariants and the actions of the two action systems A and B are simply merged in the composed action system AB. The formal parameters, b and c, in the action of B are substituted with the actual parameters, a and x, in A. Since a is a formal parameter of the scheduling A, it should also be a formal parameter of the action B after the substitution. Example. To exemplify modularisation, let us consider the below action system E 1 which contains the system Reg (Section 2.2) as a component. The scheduling of E 1 is then ((E11 ; E13 ) [] E12 ) [] Reg(c1 , dm, c2 , dout), where Reg(c1 , dm, c2 , dout) stands for (Reg1 ; Reg2 ), indicating that the parallel composition of the action systems E 1 and Reg is actually modelled. The types of the involved variables are given as the sets com ({req, ack}) and data (represesents any data type). We will return to this machine later. MACHINE E 1 EXTENDS Reg VARIABLES c1 , c2 , dm INVARIANT c1 ∈ com ∧ c2 ∈ com ∧ dm ∈ data INITIALISATION c1 := ack || c2 := ack || dm :∈ data OPERATIONS ˆ E11 (l, din, dout) = SELECT l = req ∧ r = ack THEN ANY dm WHERE F (dm , din, dout) THEN dm := dm END || c1 := req END; ˆ SELECT c2 = req THEN c2 := ack || r := req END; r ← E12 = ˆ SELECT c1 = ack THEN l := ack END l ← E13 = SCHEDULING 1 E (l, din, r, dout) = ˆ (E11 ; E13 ) [] E12 [] Reg(c1 , dm, c2 , dout) END
2.4
Modelling Asynchronous Components
In this paper, we consider systems which are organizations of asynchronously communicating components. Such building blocks with asynchronous interfaces are here collectively called asynchronous components, independently of the intended internal structure of each component. As an example, the abstract system E discussed in Section 2.1, acts as an asynchronous component towards its environment. Asynchronous Communication Channels. Interaction between asynchronous components is arranged via communication channels composed of the value and result parameters of the actions. In our formal design framework, a communication channel c(d), or a channel c in short, is defined to be a tuple (c, d), where c is a communication variable and d the list of those data variables whose values are transfered from a system module to another by communicating via c. In the case
Design with Asynchronously Communicating Components
431
when the list d is empty, c is called a control channel . Otherwise we have a data channel c(d). Furthermore, when refering to a single party of communication, we talk about communication ports rather than channels. Generally, a communication variable c is of the enumerated type comm,n defined by {req1 , . . . , reqm , ack1 , . . . , ackn } comm,n = where req1 , . . . , reqm and ack1 , . . . , ackn are request and acknowledgement states (values), respectively. A variable c ∈ comm,n is initialized to one of its acknowledgement states ackj . If m = 1 or n = 1, the default value is just req or ack, respectively. Hence, the simplest and the most usual type com1,1 is equivalent to {req, ack} by default. We denote com1,1 simply by com. A communication channel connects two action system components, one of which acts as the master and the other as the slave. The master side of a channel is called an active communication port, and the slave side is referred to as a passive communication port. A communication cycle on a channel c(d) includes two main phases. When the active party, the master, initiates the cycle by setting c to a request state reqi , the cycle is said to be in the request phase. Correspondingly, when the passive party, the slave, responds by setting c to an acknowledgement state ackj , the communication cycle on c is said to be in the acknowledgement phase. The data d can be transfered either in the request phase from the master to the slave (push channel ), in the acknowledgement phase from the slave to the master (pull channel ), or in both phases bidirectionally (biput channel ) [14]. Example. The above machine E 1 (Section 2.3) and its component Reg (Section 2.2) communicate asynchronously via the push channels c1 (dm) ∈ com and c2 (dout) ∈ com. When the system E 1 transfers data dm to Reg by setting c1 to the request state req, it acts as the master towards the machine Reg which sets c1 to the acknowledgement state ack as a response. On the other hand, when data dout is transfered from Reg to E 1 via the channel c2 , the system Reg is the active party and E 1 acts as the slave.
3
Deriving Asynchronous Systems with B
Top-down design of an asynchronous system starts from a high-level action system specification of the basic functionality of the target system. The initial abstract specification is then stepwise implemented, within the refinement calculus framework, as an asynchronous architecture of action system modules representing pre-defined library components available to the designer. Such a module could be, for example, an arithmetic-logical unit, a computational algorithm, an interface module, a memory block, a controller of a set of other units, or even a very complex microprocessor core. After this component extraction process, the main system does not contain any operations of its own, but all of them come from the component modules listed in the extends-clause of the system. The
432
J. Plosila, K. Sere, and M. Wald´en
components interact via asynchronous communication channels created during the stepwise extraction process. In this section, we first discuss the refinement in the B Method generally, and then we formulate a specific transformation rule for component extraction in asynchronous system design. 3.1
Abstract Machine Refinement
Refinement is a viable method for stepwise derivation of systems. Let us consider an abstract machine A and its refinement C given as MACHINE A VARIABLES x INVARIANT I(x) INITIALISATION x := x0 OPERATIONS ˆ SELECT P1 THEN S1 END; A1 = ... ˆ SELECT Pm THEN Sm END Am = SCHEDULING A= ˆ A1 * . . . * Am END
REFINEMENT C REFINES A VARIABLES y INVARIANT R(x, y) INITIALISATION y := y0 OPERATIONS ˆ SELECT Q1 THEN T1 END; C1 = ... ˆ SELECT Qn THEN Tn END Cn = SCHEDULING C = ˆ C1 * . . . * C n MAPPINGS ... END
The machine refinement states in the refines-clause what it refines, an abstract machine or another machine refinement. Above, the refinement C refines the abstract machine A. The invariant R(x, y) of the refinement gives the relation between the variable(s) x in the action system A and the variable(s) y in its refinement C for replacing abstract statements with more concrete ones. The refined and more concrete actions Ci are given in the operations-clause and the scheduling-clause indicates how these actions are composed. The mappings-clause states the refinement relation between the actions in A and C and will be dealt with in more detail below. In order to prove that the refinement C on the variables y is a refinement of the action system A on the variables x using the invariant R(x, y), a number of proof obligations must be satisfied [1]. The invariant R of the refinement should not contradict the invariant I of the specification. (∃(x, y). I ∧ R)
(1)
Furthermore, the initialisation y := y0 in C establishes a situation where the initialisation x := x0 in A cannot fail to establish the invariant R. wp(y := y0 , ¬wp(x := x0 , ¬R))
(2)
Moreover, we need to prove that the actions in A are data refined by the actions in C. In case each action Ai in A corresponds to one action Ci in C,
Design with Asynchronously Communicating Components
433
(n = m), we have an entry Ai ≤ Ci for each action Ai of A indicating that action Ai is data refined by Ci under invariant R. MAPPINGS A1 ≤ C1 , ... Am ≤ Cm
Hence, it should be proven that for each action Ai in A there is an action Ci in C such that the action Ci establishes a situation where Ai cannot fail to maintain R. (3) (∀(x, y). I ∧ R ⇒ wp(Ci , ¬wp(Ai , ¬R))) where 1 ≤ i ≤ m. In this case the scheduling of the corresponding actions should be the same in A and C, i.e. if A = ˆ A1 [] A2 then C = ˆ C1 [] C2 . In asynchronous system design, especially in the component extraction process, we often rely on atomicity refinement [12,19], where we split an atomic action in a system into several actions in order to increase the degree of parallelism in the system. Hence, we may need to introduce new actions during the refinement process. In case we introduce new actions in C, (n > m), we have the case that an action in A is refined by a composition of actions in C, e.g., A2 ≤ C3 [] C4 . Furthermore, we allow a composition of actions in A to be refined by a composition of actions in C. MAPPINGS Ai ∗ . . . ∗ Ak ≤ Cj ∗ . . . ∗ Cl , ...
where 1 ≤ i, k ≤ m and 1 ≤ j, l ≤ n and each * is [] or ; depending on the scheduling of the corresponding actions in A and C. Each action of A and C should appear once and only once in the mappings-clause. If we denote the composed actions Ai ∗ . . . ∗ Ak as Di and Cj ∗ . . . ∗ Cl as Ei , we can write the proof obligation for refinement using composed actions as (∀(x, y). I ∧ R ⇒ wp(Ei , ¬wp(Di , ¬R))),
(4)
for each entry Di ≤ Ei in the mappings-clause. Hence, for each composed action Di of A the composed actions Ei of C should establish such a situation that Di cannot fail to maintain R. Notice that in case the refined action system C is composed in parallel with another action system B. i.e., C has B as a component module, and B contains the scheduling B = ˆ B1 ; B2 , then either B, or both B1 and B2 should appear within the composed actions Ei in the mappings-clause of C. If action A1 in A is refined by (C1 [] B) in C then we actually consider A1 to be refined by C1 and the composition of all the actions in B, A1 ≤ (C1 [] (B1 ; B2 )). The proof obligations (1), (2) and (3) can be generated automatically and checked using the theorem-proving environments associated with the B Method [1,13]. See Wald´en and Sere [21] for further details on refining action systems within B. The B Method supports one-to-one refinement corresponding to proof obligation (3). This is, however, too restrictive for derivation of asynchronous
434
J. Plosila, K. Sere, and M. Wald´en
systems. Therefore, we have introduced the mappings-clause providing us with the needed flexibility for the refinement. The mappings-clause makes an automatic generation and check of proof obligation (4) possible as well. In Event B [9] for developing distributed systems in B several operations can refine the same operation and one operation can in turn refine several opertions. However, in Event B sequential composition of these operations is not allowed. In this paper we propose only to add new variables and not to change the abstract variables in each refinement step. We, therefore, have the case that the parallel composition of action systems is monotonic with respect to this refinement. Due to this and to the transitivity of the refinement relation, if action system A is refined by A and action system B is refined by B , then the parallel composition of A and B is refined by the parallel composition of A and B . This means that the subsystems in a parallel composition may be refined independently. Example. Studying the examples given in Section 2 we can note that the action system E 1 , which has Reg as a component, is actually a refinement of the action system E. Since E has fewer actions than E 1 , the refinement of the actions is given in the mappings-clause as follows. REFINEMENT E 1 REFINES E EXTENDS Reg VARIABLES ... SCHEDULING ˆ (E11 ; E13 ) [] E12 [] Reg(c1 , dm, c2 , dout) E 1 (l, din, r, dout) = MAPPINGS E ≤ ((E11 ; E13 ) [] E12 [] Reg(c1 , c2 , dm, dout)) END
The tool support for the B Method is then used to generate the proof obligations (1), (2) and (4) for E 1 as well as to automatically or interactively prove them. Hence, the tool assists us in proving that E 1 is a correct refinement of E. 3.2
Component Extraction
The stepwise component extraction process of an asynchronous system is based on decomposing the system by introducing new asynchronous communication channels. Each decomposition step creates at least one channel through which the extracted component is accessed. Let us now sketch a tranformation rule for component extraction within B. We study here one of the most usual cases. More application-specific rules can be derived when required. First, assume that we have a module library at our disposal and that this library contains a component M of the form
Design with Asynchronously Communicating Components
435
MACHINE M ... OPERATIONS x ← M (x, y) = ˆ SELECT x = req THEN S(y) || x := ack END SCHEDULING M(x, y) = ˆ M END
where S is an arbitrary substitution on the data parameter y. The component has the passive communication port x(y) (x ∈ com) through which it can be accessed. Consider an action system A defined as: MACHINE A ... OPERATIONS A = ˆ SELECT P THEN S1 (d); S(d); S2 END; ... SCHEDULING A= ˆ A... END
Here S1 and S2 are two arbitrary substitutions, and S is the same substitution as in the library component M given above, and the data variable d, shared by S1 and S, is of the same type as the parameter x of M. Let us now assume that we want to extract the library component M from the machine A, i.e., our goal is to implement a part of A as an instance of the pre-defined module M. For this, a fresh communication channel c(d) (c ∈ com) is created. The result of this refinement is the system C given as MACHINE C EXTENDS M ... OPERATIONS ˆ SELECT P THEN S1 (d) || c := req END; C1 = ˆ SELECT c = ack THEN S3 END; C2 = ... SCHEDULING C = ˆ (C1 ; C2 ) [] M(c, d) . . . END
The initial value of the communication variable c is ack. Observe how the variables c and d are given to the component module M as actual parameters in the scheduling-clause. The system C acts as a master towards the component M and initiates a communication cycle on the channel c(d) by setting c to the request state req in its operation C1 . The acknowledgement ack from the slave M is detected in the operation C2 which is composed sequentially with C1 . Notice that the substitutions S1 , S, and S2 are executed exactly in the same order as in the initial machine A, but in C and M this execution sequence is non-atomic. The component M takes care of the substitution S, while S1 and S2 belong to C. Extraction as Refinement. To obtain the above system C from the initial machine A the communication variable c ∈ com needs to be first introduced. Then the operation A of A is transformed into three separate atomic actions C1 , C2 , and
436
J. Plosila, K. Sere, and M. Wald´en
M using the variable c and sequential scheduling. The sequence of the involved substitutions S1 , S, and S2 is preserved. Hence, the initial machine A is refined into A which has the form REFINEMENT A REFINES A ... OPERATIONS ˆ SELECT P THEN S1 (d) || c := req END; C1 = M = ˆ SELECT c = req THEN S(d) || c := ack END; ˆ SELECT c = ack THEN S2 END; C2 = ... SCHEDULING ˆ (C1 ; C2 ) [] M . . . A = MAPPINGS A ≤ (C1 ; C2 ) [] M, ... END
The transformation is correct provided we can show the correctness of refining A into (C1 ; C2 ) [] M as required by the proof obligation (4). For this, we need to find a suitable invariant R which is application dependent. Intuitively, the machine A is semantically equivalent to the above machine C which contains the component module M. The structural difference is that in A the action M is a local operation of the system, while in C it belongs to the pre-defined library component M mentioned in the extends-clause. This means that the action instance M in the scheduling-clause of A is turned into the component instance M(c, d) in the scheduling- clause of C. We can write REFINEMENT C REFINES A EXTENDS M ... OPERATIONS ... SCHEDULING C = ˆ (C1 ; C2 ) [] M(c, d) . . . MAPPINGS M ≤ M(c, d), ... END
In order to show the correctness of this transformation as a refinement step we need to verify the proof obligations (1)-(4) from the Section 3.1. The proof obligations can be automatically generated from C and proved with the tools supporting the B Method. Note that the above transformation rule applies to the most straightforward extraction step. In practice, the procedure can be more complicated requiring several refinement steps, depending on the complexity of the library component which is to be extracted. The communication variable c, for instance, might have more values than the two (req, ack) needed above, and the introduction of c might involve more than one (A) action in the original system. Furthermore, an extraction operation can require several distinct communication variables rather than just one, and maybe a number of fresh data variables as well. However, the basic idea remains the same, i.e., atomic actions are split into two or more separate parts using the introduced channels in order to form a model of a known
Design with Asynchronously Communicating Components
437
library component into the system. This embedded model is then replaced with a reference to the actual library component. Naturally, the idea of component extraction discussed above can be as well used for creating new library components which are to be re-used in future design projects.
4
Component Extraction Example
As an example of the component extraction process, consider the action system E of Section 2.2. We assume that E operates within an environment, modeled by another abstract machine, which instantiates E in its scheduling-clause as the component E (l, din, r, dout), where (l ∈ com)∧(r ∈ com)∧(dout ∈ data)∧(din ∈ data), and the communication variables l and r are both initialized to ack. The machine E is an abstract model of an asynchronous system. It has one passive input port l(din) and one active output port r(dout), and its behavior is the following. First, the environment selects a value for the data input din and activates the machine E by setting the channel l to the request state req. Then E computes a new value for the data output dout using the relation F . Observe that F is not explicitly specified in this generic example. The machine E sends the data dout to the environment via the channel r by setting r to the request state req. Simultaneously, an acknowledgement ack is issued through the channel l. This indicates that the environment may send new data din to E via l, but computation in E is blocked until the environment has stored the value of dout and issued an acknowledgement through r. Let us now assume that we have a library of pre-defined asynchronous componets available and that we intend to implement the abstract system specification E stepwise as a composition of 4 components belonging to this library: register Reg, function Func, release R, and suspend Susp. Below we discuss these componets and related extraction steps separately. 4.1 Register Component The predicate F in the operation E of the machine E refers also to the variable dout itself. In other words, the next value of dout depends on the current value of dout. Furthermore, new input data din can arrive from the environment before communication on the channel r has been completed. This indicates that a storage element, a register, is needed for the variable dout. Hence, the register component Reg, defined in Section 2.2, is extracted as the first decomposition step. As the result we obtain the machine E 1 , given in Sections 2.3 and 3.1, which is a refinement of the initial abstract machine E, containing Reg as a component. REFINEMENT E 1 REFINES E EXTENDS Reg ... SCHEDULING ˆ (E11 ; E13 ) [] E12 [] Reg(c1 , dm, c2 , dout) E 1 (l, din, r, dout) = MAPPINGS E ≤ ((E11 ; E13 ) [] E12 [] Reg(c1 , dm, c2 , dout)) END
438
J. Plosila, K. Sere, and M. Wald´en
The extraction procedure splits the operation E of E into five separate parts, two of which (Reg1 , Reg2 ) belong to the register Reg and the others (E11 , E12 , E13 ) to the refined machine E 1 . For this, we have introduced two fresh communication variables c1 , c2 ∈ com, through which E 1 and Reg communicate, and applied twice the transformation rule discussed in Section 3.2. Furthermore, the refinement step includes the introduction of the intermediate data variable dm which acts as the data input of the extracted register component, so that Reg carries out the copying assignment dout := dm. The machine E 1 activates Reg by computing a new value for the data variable dm and setting the channel c1 to the request state in the operation E11 . Then Reg copies the value of dm to the variable dout and performs a communication cycle on the channel c2 , which activates E 1 to send dout to the environment via the channel r in the operation E12 . After the cycle on c2 , Reg sets c1 to the acknowledgement state, and E 1 executes finally the operation E13 , where the channel l is set to the acknowledgement state. 4.2
Function Component
The second extraction step places the computation of the data variable dm, the input of Reg, to the component machine Func defined by MACHINE F unc OPERATIONS ˆ b, dout ← F unc1 (a, din1 , din2 ) = SELECT a = req THEN ANY dout WHERE F (dout , din1 , din2 ) THEN dout := dout END || b := req END; ˆ SELECT b = ack THEN a := ack END a ← F unc2 (b) = SCHEDULING F unc(a, din1 , din2 , b, dout, F ) = F unc1 ; F unc2 END
This library component has the passive input port a(din1 , din2 ) and the active output port b(dout). Notice that also the predicate F is viewed as a parameter in the scheduling-clause. In the extraction procedure, the formal ports a(din1 , din2 ) and b(dout) are replaced with the actual push channels c3 (din, dout) and c1 (dm), respectively, where c3 ∈ com is the new communication variable introduced in the transformation step. The resulting machine E 2 , which is a refinement of E 1 , is given below. In order to extract the component Func, the operations E11 and E13 of the system E 1 are split into two parts each: E11 into E21 and F unc1 , and E13 into E23 and F unc2 . The system E 2 activates Func by executing the operation E21 , where the current values of the variables dout and din are sent to Func via the new channel c3 . The function component then carries out the data assignment ANY dm WHERE F (dm , din, dout) THEN dm := dm END
Design with Asynchronously Communicating Components
439
and sends the result data dm to the register component Reg via the channel c1 which was created in the first decomposition step. REFINEMENT E 2 REFINES E 1 EXTENDS Reg, F unc VARIABLES c1 , c2 , c3 , dm INVARIANT c1 ∈ com ∧ c2 ∈ com ∧ dm ∈ data ∧ c3 ∈ com DEFINITIONS F (· · ·) == . . . INITIALISATION c1 := ack || c2 := ack || c3 := ack || dm :∈ data OPERATIONS ˆ SELECT l = req ∧ r = ack THEN c3 := req END; E21 (l, r) = ˆ SELECT c2 = req THEN c2 := ack || r := req END; r ← E22 = ˆ SELECT c3 = ack THEN l := ack END l ← E23 = SCHEDULING 2 ˆ (E21 ; E23 ) [] E22 [] Reg(c1 , dm, c2 , dout) E (l, din, r, dout) = [] F unc(c3 , din, dout, c1 , dm, F ) MAPPINGS (E11 ; E13 ) ≤ ((E21 ; E23 ) [] F unc(c3 , din, dout, c1 , dm, F )), E12 ≤ E22 END
4.3
Final Extraction
We complete our example by extracting two distinct library components based on the three operations of E 2 . The component machines are called R (release) and Susp (suspend), defined as follows: MACHINE R OPERATIONS ˆ SELECT a = req THEN b := req || a := ack || bsy := true END; a, b, bsy ← R1 (a) = ˆ SELECT b = ack THEN bsy := false END bsy ← R2 (b) = SCHEDULING R(a, b, bsy) = R1 ; R2 END
MACHINE Susp OPERATIONS b ← Susp1 (a, bsy) = ˆ SELECT a = req ∧ ¬bsy THEN b := req END; ˆ SELECT b = ack THEN a := ack END a ← Susp2 (b) = SCHEDULING Susp(a, b, bsy) = Susp1 ; Susp2 END
They both have two communication ports: the passive port a and the active port b. Furthermore, R outputs the boolean signal bsy setting it to true whenever a communication cycle on the active port b begins, and back to f alse when a cycle on b ends. The component Susp, in turn, reads the signal bsy, allowing a new communication cycle on its active port b to start only when bsy = false. In the system derivation, the formal parameters a and b of R are replaced with the
440
J. Plosila, K. Sere, and M. Wald´en
actual variables c2 and r, respectively. In the case of Susp, a and b are replaced with l and c3 , respectively. The parameter bsy becomes a variable of the same name, shared by the components R and Susp. The final system E 3 is then a refinement of E 2 , given as REFINEMENT E 3 REFINES E 2 EXTENDS Reg, F unc, R, Susp VARIABLES c1 , c2 , c3 , bsy, dm INVARIANT c1 ∈ com ∧ c2 ∈ com ∧ c3 ∈ com ∧ bsy ∈ BOOL ∧ dm ∈ data DEFINITIONS F (· · ·) == . . . INITIALISATION c1 := ack || c2 := ack || c3 := ack || bsy := false || dm :∈ data SCHEDULING ˆ Reg(c1 , dm, c2 , dout) E 3 (l, din, r, dout) = [] F unc(c3 , din, dout, c1 , dm, F ) [] R (c2 , r, bsy) [] Susp(l, c3 , bsy) MAPPINGS ((E21 ; E23 ) [] E22 ) ≤ (Susp(l, c3 , bsy) [] R (c2 , r, bsy) END
Observe that the resulting system does not have operations of its own, but the functionality comes completely from the four component machines. In the final transformation, we do not insert new communication variables of the type com. Instead a boolean variable bsy (“busy”) is introduced in order to move the detection of the condition r = ack to a different location, making extraction of R and Susp possible. Basically, the component Susp implements the operations E21 and E23 of E 2 . The component R, in turn, implements the operation E22 . However, a new action has to be created for detecting the condition r = ack and setting bsy to false. This action is the operation R2 of R. The component R is enabled by the register component Reg via the channel c2 . It initiates a communication cycle with the environment on the channel r and sends immediately an acknowledgement back to Reg setting the introduced boolean variable bsy to true. Hence, R releases the communication cycles on the channels c1 , c2 , c3 , and l to be completed while output data dout is transferred from the system E 3 to the environment through r. The component Susp reads the variable bsy and suspends computation on new input data din, sent by the environment via the channel l, until the component R has received an acknowledgement on r and set the control signal bsy back to false.
5
Conclusions
We have proposed an approach to component-based asynchronous system design within the B Method and its supporting tools. The main idea is to extract system components in a stepwise manner from the initial specification. At each extraction step new asynchronous communication channels are introduced. There has been a lot of work on composition and refinement in the context of formal component-based design approaches, see e.g. [2]. However, these seldom come with a practical methodology and tool support, which is our main focus here.
Design with Asynchronously Communicating Components
441
The B Method was used almost as such with minor extensions mainly needed to specify alternative scheduling strategies for actions in a scheduling-clause and explicitly giving refinement relations in the mappings-clause. The first clause is mainly syntactic sugaring as every scheduling could be coded in the actions of the operations-clause. The second clause is a real extension, but also it can be easily supported by the tools. The scheduling-clause is inspired by the work of Butler [7] who studies a more general process-concept for the B Method to tie it together with the CSP-approach [11]. The Refinement Calculator [22] is a tool that supports development of correct programs within the refinement calculus framework. It is a tool for program refinement based on the HOL theorem prover. Until now, the tool does not have proper support for action systems. However, a method to prove the correctness of the implementations of asynchronous modules has been mechanized within the HOL theorem prover [16] which supports more general verification techniques than the tools studied in this paper. In the future we can envisage a situation where the verification of the low level implementations are carried out within HOL and the high-level design and component libraries are supported by the tools of the B Method.
References 1. J.-R. Abrial. The B-Book. Cambridge University Press, Cambridge, Great Britain, 1996. 2. L. de Alfaro and T.A. Henzinger. Interface Theories for Component-based Design. Proc. of the 1st International Workshop on Embedded Software, Springer-Verlag, 2001. 3. R. J. R. Back and R. Kurki-Suonio. Decentralization of process nets with centralized control. In Proc. of the 2nd ACM SIGACT–SIGOPS Symp. on Principles of Distributed Computing, pages 131–142, 1983. 4. R. J. R. Back and K. Sere. Stepwise refinement of action systems. Structured Programming, 12:17–30, 1991. 5. R. J. R. Back och K. Sere. From action systems to modular systems. In Proc. of FME’94: Industrial Benefit of Formal Methods. LNCS 873, pp. 1–25, Barcelona, Spain, October 1994. Springer–Verlag. 6. M. M. Bonsangue, J. N. Kok, and K. Sere. Developing object-based distributed system. In Formal Methods for Open Object-based Distributed Systems (FMOODS’99), Florence, Italy, February 1999. Kluver Academic Publishers. 7. M. J. Butler. csp2B: A practical approach to combining CSP and B. In J. Wing, J. Woodcock and J. Davies (Eds.) Proc. of FM’99 - Formal Methods. LNCS 1708, pages 490 – 508, Toulouse, France, September 1999. Springer-Verlag. 8. M. J. Butler and M. Wald´en. Distributed System Development in B. In H. Habrias (Ed.) Proc. of the First Conference on the B Method. pages 155 – 168, IRIN, Nantes, France, November 1996. 9. ClearSy. Event B Reference Manual v1., 2001. 10. E. W. Dijkstra. A Discipline of Programming. Prentice–Hall International, Englewood Cliffs, New Jersey, 1976. 11. C.A.R. Hoare. Communicating Sequential Processes. Series in Computer Science, Prentice-Hall Int, 1985.
442
J. Plosila, K. Sere, and M. Wald´en
12. R. J. Lipton. Reduction: A method of proving properties of parallel programs. Communications of the ACM, 18(12):717–721, 1975. 13. D. S. Neilson and I. H. Sorensen. The B-Technologies: A system for computer aided programming. Including the B-Toolkit User’s Manual, Release 3.2. B-Core (UK) Ltd., Oxford, U.K., 1996. 14. A. Peeters. Single-Rail Handshake Circuits. PhD Thesis, Eindhoven University of Technology, The Netherlands, 1996. 15. J. Plosila. Self-Timed Circuit Design – The Action Systems Approach. PhD thesis, University of Turku, Turku, Finland, 1999. 16. R. Ruksenas. Tool Support for Data Refinement. Ph.D. Thesis. Forthcoming. 17. T. Seceleanu. Systematic Design of Synchronous Digital Circuits. PhD thesis, Turku Centre for Computer Science (TUCS), Turku, Finland, 2001. 18. E. Sekerinski and K. Sere (Eds.). Program Development by Refinement. FACIT, Springer Verlag 1998. 19. K. Sere and M. Wald´en. Data Refinement of Remote Procedures. Formal Aspects of Computing, Volume 12, No 4, pp. 278 - 297, December 2000. 20. St´eria M´editerran´ee. Atelier B. France, 1996. 21. M. Wald´en and K. Sere. Reasoning about action systems using the B-Method. In Formal Methods in System Design, Vol. 13, No 1, pages 5 - 35, May 1998. Kluwer Academic Publishers. 22. J. von Wright. Program refinement by theorem prover. In Proc. of Sixth BCS-FACS Refinement Workshop, January 1994.
Composition for Component-Based Modeling Gregor G¨ ossler1 and Joseph Sifakis2 1 INRIA Rhˆ one-Alpes [email protected] 2 VERIMAG [email protected]
1
Introduction
Component-based engineering is of paramount importance for rigorous system design methodologies. It is founded on a paradigm which is common to all engineering disciplines: complex systems can be obtained by assembling components (building blocks). Components are usually characterized by abstractions that ignore implementation details and describe properties relevant to their composition e.g. transfer functions, interfaces. Composition is used to build complex components from simpler ones. It can be formalized as an operation that takes in components and their integration constraints. From these, it provides the description of a new, more complex component. Component-based engineering is widely used in VLSI circuit design methodologies, supported by a large number of tools. Software and system componentbased techniques have known significant development, especially due to the use of object technologies supported by languages such as C++, Java, and standards such as UML and CORBA. However, these techniques have not yet achieved the same level of maturity as has been the case for hardware. The main reason seems to be that software systems are immaterial and are not directly subject to the technological constraints of hardware, such as fine granularity and synchrony of execution. For software components, it is not as easy to establish a precise characterization of the service and functionality offered at their interface. Existing component technologies encompass a restricted number of interaction types and execution models, for instance, interaction by method calls under asynchronous execution. We lack concepts and tools allowing integration of synchronous and asynchronous components, as well as different interaction mechanisms, such as communication via shared variables, signals, rendez-vous. This is essential for modern systems engineering, where applications are initially developed as systems of interacting components, from which implementations are derived as the result of a co-design analysis. The development of a general theoretical framework for component-based engineering is one of the few grand challenges in information sciences and technologies. The lack of such a framework is the main obstacle to mastering the complexity of heterogeneous systems. It seriously limits the current state of the practice, as attested by the lack of development platforms consistently integrating design activities and the often prohibitive cost of validation. F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 443–466, 2003. c Springer-Verlag Berlin Heidelberg 2003
444
G. G¨ ossler and J. Sifakis
The application of component-based design techniques raises two strongly related and hard problems. First, the development of theory for building complex heterogeneous systems. Heterogeneity is in the different types of component interaction, such as strict (blocking) or non strict, data driven or event driven, atomic or non atomic and in the different execution models, such as synchronous or asynchronous. Second, the development of theory for building systems which are correct by construction, especially with respect to essential and generic properties such as deadlock-freedom or progress. In practical terms, this means that the theory supplies rules for reasoning on the structure of a system and for ensuring that such properties hold globally under some assumptions about its constituents e.g. components, connectors. Tractable correctness by construction results can provide significant guidance in the design process. Their lack leaves a posteriori verification of the designed system as the only means to ensure its correctness (with the well-known limitations). In this paper, we propose a framework for component-based modeling that brings some answers to the above issues. The framework uses an abstract layered model of components. It integrates and simplifies results about modeling timed systems by using timed automata with dynamic priorities [5,1]. A component is the superposition of three models: a behavioral model, an interaction model, and an execution model. – Behavioral models describe the dynamic behavior of components. – Interaction models describe architectural constraints on behavior. They are defined as a set of connectors and their properties. A connector is a maximal set of compatible component actions. The simultaneous occurrence of actions of a connector is an interaction. – Execution models reduce non determinism resulting from parallel execution in the lower layers. They are used to coordinate the execution of threads so as to ensure properties related to the efficiency of computation, such as synchrony and scheduling. An associative and commutative composition operator is defined on components, which preserves deadlock-freedom. The operator defines a three-layered component by composing separately the corresponding layers of its arguments. As a particular instance of the proposed framework, we consider components where behaviors are transition systems and both interaction and execution models are described by priority relations on actions. Our framework differs from existing ones such as process algebras, semantic frameworks for synchronous languages [4,11,3,17] and Statecharts [12], in two aspects. First, it distinguishes clearly between three different and orthogonal aspects of systems modeling: behavior, interaction (architecture) and execution. This distinction, apart from its methodological interest, allows solving technical problems such as associativity of a unique and powerful composition operator. The
Composition for Component-Based Modeling
445
proposed framework has concepts in common with Metropolis [2] and Ptolemy [16] where a similar separation of concerns is advocated. Second, parallel composition preserves deadlock-freedom. That is, if the arguments can perform some action from any state then their product does so. This is due to the fact that we replace restriction or other mechanisms used to ensure strong synchronization between components, by dynamic priorities. Nevertheless, our composition is a partial operation: products must be interaction safe, that is, they do not violate strong synchronization assumptions. In that respect, our approach is has some similarity to [7]. The paper is organized as follows. Section 2 discusses three requirements for composition in component-based modeling. The first is support for two main types of heterogeneity: heterogeneous interaction and heterogeneous execution. The second is that it provide results for ensuring correctness by construction for a few essential and generic system properties, such as deadlock-freedom. The third is the existence of a composition operator that allows abstraction and incremental description. Section 3 presents a general notion of composition and its properties for components with two layers: behavior and interaction models. Interaction models relate concepts from architecture (connectors) to actions performed by components via the notion of interaction. Interaction models distinguish between complete and incomplete interactions. This distinction induces the concept of interaction safety for models, meaning that only complete or maximal interactions are possible. We show associativity and commutativity of the composition operator. The section ends with a few results on correctness by construction for interaction safety of models and deadlock-freedom. Section 4 presents two examples illustrating the use of execution models. We assume that execution models can be described by priority orders. The first example shoes how synchronous execution can be enforced by a priority order on the interactions between reactive components. The order respects the causality flow relation between component actions. The second example shows how scheduling policies can be implemented by an execution model. Section 5 presents concluding remarks about the presented framework.
2 2.1
Requirements for Composition General
We consider a very simple and abstract concept of components that is sufficient for the purpose of the study. A component can perform actions from a vocabulary of actions. The behavior of a component describes the effect of its actions. A system of interacting components is a set of components integrated through various mechanisms for coordinating their execution. We assume that the overall effect of integration on the components of a system is the restriction of their behavior and it can be abstractly described by integration constraints. The latter describe the environment of a component. A component’s actions may be blocked until the environment offers actions satisfying these constraints.
446
G. G¨ ossler and J. Sifakis
We distinguish two types of integration constraints: interaction and execution constraints. Interaction constraints characterize mechanisms used in architectures such as connectors, channels, synchronization primitives. Interactions are the result of composition between actions. In principle, all the actions of a component are “visible” from its environment. We do not consider any specific notion of interface. Execution constraints restrict non determinism arising from concurrent execution, and ensure properties related to the efficiency of computation, such as synchronous execution and scheduling. There exists a variety of formalisms proposing concepts for parallel execution of sequential entities, such as process algebras (CCS [19], CSP [13]), synchronous languages (Esterel, Lustre, Statecharts), hardware description languages (VHDL), system description languages (SystemC [20], Metropolis), and more general modeling languages (SDL [14], UML [10]). In our terminology, we use the term “component” to denote any executable description whose runs can be modeled as sequences of actions. Component actions can be composed to produce interactions. Tasks, processes, threads, functions, blocks of code can be considered as components provided they meet these requirements. The purpose of this section is to present concept requirements for composition in component-based modeling and to discuss the adequacy of existing formalisms with respect to these requirements. 2.2
Heterogeneity
Heterogeneity of Interaction. It is possible to classify existing interaction types according to the following criteria: Interactions can be atomic or non atomic. For atomic interactions, the behavior change induced in the participating components cannot be altered through interference with other interactions. Process algebras and synchronous languages assume atomic interactions. In languages with buffered communication (SDL, UML) or in multi-threaded languages (Java), interactions are not atomic, in general. An interaction is initialized by sending a message or by calling a method, and between its initiating action and its termination, components non participating in the interaction can interfere. Interactions can involve strict or non strict synchronization. For instance, atomic rendez-vous of CSP is an interaction with strict synchronization in the sense that it can only occur if all the participating actions can occur. Strict synchronization can introduce deadlocks in systems of interacting deadlock-free components, that is, components that can always offer an action. If a component persistently offers an action and its environment is unable to offer a set of actions matching the interaction constraints, then there is a risk of deadlock. In synchronous languages, interactions are atomic and synchronization is non strict in the sense that output actions can occur whether or not they match with some input. Nevertheless, for inputs to be triggered, a matching output is necessary.
Composition for Component-Based Modeling
447
Finally, interactions can be binary (point to point) or n-ary for n 3. For instance, interactions in CCS and SDL are binary (point to point). The implementation of n-ary interactions by using binary interaction primitives is a non trivial problem. Clearly, there exists no formalism supporting all these types of interaction. Heterogeneity of Execution. There exist two well-known execution paradigms. Synchronous execution is typically adopted in hardware, in the synchronous languages, and in many time triggered architectures and protocols. It considers that a system run is a sequence of global steps. It assumes synchrony, meaning that the system’s environment does not change during a step, or equivalently “that the system is infinitely faster than its environment”. In each execution step, all the system components contribute by executing some “quantum” computation, defined through the use of appropriate mechanisms such as timing mechanisms (clocks, timers) or a notion of stable states. For instance, in synchronous languages, an execution step is a reaction to some external stimulus obtained by propagating the reactions of the components according to a causality flow relation. A component reaction is triggered by a change of its environment and eventually terminates at some stable state for this environment. The synchronous execution paradigm has built-in a very strong assumption of fairness: in each step all components execute a quantum computation defined using either quantitative or logical time. The asynchronous execution paradigm does not adopt any notion of a global computation step in a system’s execution. It is used in languages for the description of distributed systems such as SDL and UML, and programming languages such as Ada and Java. The lack of a built-in mechanism for sharing resources between components is often compensated by using scheduling. This paradigm is also common to all execution platforms supporting multiple threads, tasks, etc. Currently, there is no framework encompassing the diversity of interaction and execution models. Figure 1 classifies different system description languages in a three-dimensional space with coordinates corresponding to execution (synchronous/asynchronous) and to interaction: atomic/non atomic and strict/nonstrict. It is worth noting that synchronous languages use non strict and atomic interactions. This choice seems appropriate for synchronous execution. On the contrary, for asynchronous execution there is no language using this kind of interaction. 2.3
Correctness by Construction
It is desirable that frameworks for component-based modeling provide results for establishing correctness by construction for at least a few common and generic properties such as deadlock-freedom or stronger progress properties. In practical terms, this implies the existence of inference rules for deriving system and com-
448
G. G¨ ossler and J. Sifakis
Fig. 1. About composition: heterogeneity. A: atomic, S: strict interaction.
ponent properties from properties of lower-level components. In principle, two types of rules are needed for establishing correctness by construction. Composability rules allow to infer that, under some conditions, a component will meet a given property after integration. These rules are essential for preserving across integration previously established component properties. For instance, to guarantee that a deadlock-free component (a component that has no internal deadlocks) will remain deadlock-free after integration. Composability is essential for incremental system construction as it allows building large systems without disturbing the behavior of their components. It simply means stability of established component properties when the environment changes by adding or removing components. Property instability phenomena are currently poorly understood e.g. feature interaction in telecommunications, or non composability of scheduling algorithms. Results in composability are badly needed. Compositionality rules allow to infer a system’s properties from its components’ properties. There exists a rich body of literature for establishing correctness through compositional reasoning [15,9,8]. However, most of the existing results deal with the preservation of safety properties. 2.4
Abstraction and Incrementality
A basic assumption of component-based engineering is that components are characterized by some external specification that abstracts out internal details. However, it is often necessary to modify the components according to the context of their use, at the risk of altering their behavior. Such modifications may be necessary to adapt components to a particular type of composition. For instance, to model non strict synchronization using strict synchronization, a common transformation consists in modifying both the action vocabularies (interfaces) and the behavior of components by adding for each action a of the interface a “com-
Composition for Component-Based Modeling
449
plementary” a ¯ action that will be executed from all the states from which a is not possible. To model strict synchronization using non strict synchronization, similar modifications are necessary (see for instance Milner’s SCCS [18]). We currently lack sufficiently powerful and abstract composition operators encompassing different kinds of interaction. Another important requirement for composition is incrementality of description. Consider systems consisting of sets of interacting components, the interaction being represented as usual, by connectors or architectural constraints of any kind. Incrementality means that such systems can be constructed by adding or removing components and that the result of the construction is independent of the order of integration. Associative and commutative composition operators allow incrementality. Existing theoretical frameworks such as CCS, CSP, SCCS, use parallel composition operators that are associative and commutative. Nevertheless, these operators are not powerful enough. They need to be combined with other operators such as hiding, restriction, and renaming in system descriptions. The lack of a single operator destroys incrementality of description. For instance, some notations use hiding or restriction to enforce interaction between the components of a system. If the system changes by adding a new component, then some hiding or restriction operators should be removed before integrating the new component. Graphical formalisms used in modeling tools such as Statecharts or UML do not allow incremental description as their semantics are not compositional. They are defined by functions associating with a description its meaning, as a global transition system (state machine), i.e., they implicitly use n-ary composition operators (n is equal to the number of the composed components). It is always easy to define commutative composition, even in the case of asymmetric interactions. On the contrary, the definition of a single associative and commutative composition operator which is expressive and abstract enough to support heterogeneous integration remains a grand challenge.
3
Composition
We present an abstract modeling framework based on a unique binary associative and commutative composition operator. Composition operators should allow description of systems built from components that interact by respecting constraints of an interaction model. The latter characterizes a system architecture as a set of connectors and their properties. Given a set of components, composition operations allow to construct new components. We consider that the meaning of composition operations is defined by connectors. Roughly speaking, connectors relate actions of different components and can be abstractly represented as tuples or sets of actions. The related actions can form interactions (composite actions) when some conditions are met. The conditions define the meaning of the connector and say when and how the interaction can take place depending on the occurrence of the related actions.
450
G. G¨ ossler and J. Sifakis
For instance, interactions can be asymmetric or symmetric. Asymmetric interactions have an initiator (cause) which is a particular action whose occurrence can trigger the other related actions. In symmetric interactions all the related actions play the same role. The proposed composition operator differs from existing ones in automata theory and process algebras in the following. – First, it preserves deadlock-freedom. This is not the case in general, for existing composition operators except in very specific cases. For instance, when from any state of the behavioral model any action offered by the environment can be accepted. – Second, deadlock-freedom preservation is due to systematic interleaving of all the actions of the composed components, combined with the use of priority rules. The latter give preference to synchronization over interleaving. In existing formalisms allowing action interleaving in the product such as CCS and SCCS, restriction operators are used instead of priorities to prevent occurrence of interleaving actions. For instance, if a and a ¯ are two synchronizing actions in CCS, their synchronization gives an invisible action τ = aa ¯. The interleaving actions a and a ¯ are removed from the product system by using restriction. This may introduce deadlocks at product states from which no matching actions are offered. Priority rules implement a kind of dynamic restriction and lead to a concept of “flexible” composition. 3.1
Interaction Models and Their Properties
Consider a set of components with disjoint vocabularies of actions Ai for i ∈ K, K a set of indices. We put A = i∈K Ai . A connector c is a non empty subset of A such that ∀i ∈ K . |Ai ∩ c| 1. A connector defines a maximally compatible set of interacting actions. For the sake of generality, our definition accepts singleton connectors. The use of the connector {a} in a description is interpreted as the fact that action a cannot be involved in interactions with other actions. Given a connector c, an interaction α of c is any term of the form α = a1 . . . an such that {a1 , . . . , an } ⊆ c. We assume that is a binary associative and commutative operator. It is used to denote some abstract and partial action composition operation. The interaction a1 . . . an is the result of the simultaneous occurrence of the actions a1 , . . . , an . When α and α are interactions we write α α to denote the interaction resulting from their composition (if its is defined). Notice that if α = a1 . . . an is an interaction then any term corresponding to a sub-set of {a1 , . . . , an } is an interaction. By analogy, we say that α is a sub-interaction of α if α = α α for some interaction α . Clearly, actions are minimal interactions. The set of the interactions of a connector c = {a1 , . . . , an }, denoted by I(c), consists of all the interactions corresponding to sub-sets of c (all the subinteractions of c). We extend the notation to sets of connectors. If C is a set
Composition for Component-Based Modeling
451
of connectors then I(C) is the set of its interactions. Clearly for C1 , C2 sets of connectors, I(C1 ∪ C2 ) = I(C1 ) ∪ I(C2 ). Definition 1 (Interaction model). The interaction model of a system composed of a set of components K with disjoint vocabularies of actions Ai for i ∈ K, is defined by ; – the vocabulary of actions A = i∈K Ai – the set of its connectors C such that c∈C c = A, and if c ∈ C then there exists no c ∈ C and c ⊂ c . That is, C contains only maximal connectors; – the set of the complete interactions I(C)+ ⊆ I(C), such that ∀b, b ∈ I(C), b ∈ I(C)+ and b ⊆ b implies b ∈ I(C)+ . We denote by I(C)− the set of the incomplete (non complete) interactions. Notice that all actions appear in some connector. The requirement that C contains only maximal sets ensures bijective correspondence between the set of connectors C and the corresponding set of interactions I(C). Given I(C), the corresponding set of connectors is uniquely defined and is C. To simplify notation, we write IC instead of I(C). The distinction complete/incomplete is essential for building correct models. As models are built incrementally, interactions are obtained by composing actions. It is often necessary to express the constraint that some interactions of a sub-system are not interactions of the system. This is typically the case for binary strict synchronization (rendez-vous). For example, send and receive should be considered as incomplete actions but sendreceive as complete. The occurrence of send or receive alone in a system model is an error because it violates the assumption about strict synchronization made by the designer. Complete interactions can occur in a system when all the involved components are able to perform the corresponding actions. The distinction between complete/incomplete encompasses many other distinctions such as input/output, internal/external, observable/controllable used in different formalisms. It is in our opinion, the most relevant concerning the ability of components to interact. Clearly, internal component actions should be considered as complete because they can be performed by components independently of the state of their environment. In some formalisms, output actions are complete (synchronous languages, asynchronous buffered communication). In some others, with strict synchronization rules, all actions participating in interactions are incomplete. In that case, it is necessary to specify which interactions are complete. For instance, if a1 a2 a3 is complete and no sub-interaction is complete, this means that a strong synchronization between a1 , a2 , a3 is required. A requirement about complete interactions is closedness for containment that is, if α is a complete interaction then any interaction containing it, is complete. This requirement follows from the assumption that the occurrence of complete interactions cannot be prevented by the environment. Very often it is sufficient to consider that the interactions of IC+ are defined from a given set of complete actions A+ ⊆ A. That is, IC+ consists of all the interactions of IC where at least one complete action (element of A+ ) is involved.
452
G. G¨ ossler and J. Sifakis
In the example of figure 2, we give sets of connectors and complete actions to define interaction models. By convention, bullets represent incomplete actions and triangles complete actions. In the partially ordered set of the interactions, full nodes denote complete interactions. The interaction between put and get represented by the interaction putget is a rendez-vous meaning that synchronization is blocking for both actions. The interaction between out and in is asymmetric as out can occur alone even if in is not possible. Nevertheless, the occurrence of in requires the occurrence of out. The interactions between out, in1 and in2 are asymmetric. The output out can occur alone or in synchronization with any of the inputs in1 , in2 .
Fig. 2. Flexible composition: interaction structure.
In general, completeness of interactions need not be the consequence of the completeness of some action. For instance, consider a connector {a1 , a2 , a3 , a4 } and suppose that the set of the minimal complete interactions of I{a1 , a2 , a3 , a4 } is a1 a2 and a3 a4 . That is, the actions a1 , a2 , a3 , a4 are incomplete and only interactions containing a1 a2 or a3 a4 are complete. This specification requires strict synchronization of at least one of the two pairs (a1 , a2 ), (a3 , a4 ). 3.2
Incremental Description of Interaction Models
Consider the interaction model IM = (IC, IC+ ) of a set of interacting components K with disjoint vocabularies of actions Ai for i ∈ K. IC and IC+ denote the sets of interactions and complete interactions, respectively on the vocabulary of actions A = i∈K Ai .
Composition for Component-Based Modeling
453
For given K ⊆ K the interaction model IM[K ] of the set of interacting components K is defined as follows: – A[K ] = i∈K Ai is the vocabulary of actions of IM[K ]; ∃c ∈ C . c c ∩ A[K ]} is the – C[K ] = {c | ∃c ∈ C . c = c ∩ A[K ] ∧ set of the connectors of IM[K ]; – IM[K ] = (IC[K ], IC[K ]+ ) is the interaction model of IM[K ] where IC[K ] is the set of the interactions of C[K ] and IC[K ]+ = IC[K ] ∩ IC+ . Definition 2. Given a family of disjoint sets of components K1 , . . . , Kn subsets of K, denote by C[K1 , . . . , Kn ] the set of the connectors having at least one action in each set, that is, C[K1 , . . . , Kn ] = {c = c1 ∪ · · · ∪ cn | ∀i ∈ [1, n] . ci ∈ C[Ki ]}. Clearly, C[K1 , . . . , Kn ] is the set of the connectors of IM[K1 ∪ · · · ∪ Kn ] which are not connectors of any IM[K ] for any subset K of at most n − 1 elements from {K1 , . . . , Kn }. Proposition 1. Given K1 , K2 two disjoint subsets of K. IC[K1 ∪ K2 ] = IC[K1 ] ∪ IC[K2 ] ∪ IC[K1 , K2 ] IC[K1 ∪ K2 ]+ = IC[K1 ]+ ∪ IC[K2 ]+ ∪ IC[K1 , K2 ]+ IM[K1 ∪ K2 ] = (IC[K1 ∪ K2 ], IC[K1 ∪ K2 ]+ ) = IM[K1 ] ∪ IM[K2 ] ∪ IM[K1 , K2 ] where IC[K1 , K2 ]+ = IC[K1 , K2 ] ∩ IC+ . Proof. The first equality comes from the fact that C[K1 ] ∪ C[K2 ] ∪ C[K1 , K2 ] contains all the connectors of C[K1 ∪ K2 ] and other connectors that are not maximal. By definition, IC contains all the sub-sets of C. Thus, IC[K1 ∪ K2 ] = I(C[K1 ] ∪ C[K2 ] ∪ C[K1 , K2 ]) = IC[K1 ] ∪ IC[K2 ] ∪ IC[K1 , K2 ]. Remark 1. The second equality says that the same interaction cannot be complete in an interaction model IM[K1 ] and incomplete in IM[K2 ], for K1 , K2 ⊆ K. This proposition provides a basis for computing the interaction model IM[K1 ∪K2 ] from the interaction models IM[K1 ] and IM[K2 ] and from the interaction model of the connectors relating components of K1 and components of K2 . Property 1. For K1 , K2 , K3 three disjoint subsets of K, IC[K1 ∪ K2 , K3 ] = IC[K1 , K3 ] ∪ IC[K2 , K3 ] ∪ IC[K1 , K2 , K3 ] IM[K1 ∪ K2 , K3 ] = IM[K1 , K3 ] ∪ IM[K2 , K3 ] ∪ IM[K1 , K2 , K3 ] Proof. The first equality comes from the fact that C[K1 , K3 ] ∪ C[K2 , K3 ] ∪ C[K1 , K2 , K3 ] contains all the connectors of C[K1 ∪ K2 , K3 ] and in addition, other connectors that are not maximal. By definition, IC contains all the subsets of C. Thus, IC[K1 ∪ K2 , K3 ] = I(C[K1 , K3 ] ∪ C[K2 , K3 ] ∪ C[K1 , K2 , K3 ]) from which we get the result by distributivity of I over union.
454
G. G¨ ossler and J. Sifakis
This property allows computing the connectors and thus the interactions between IM[K1 ∪ K2 ] and IM[K3 ] in terms of the interactions between IM[K1 ], IM[K2 ], and IM[K3 ]. By using this property, we get the following expansion formula: Proposition 2 (Expansion formula). IM[K1 ∪ K2 ∪ K3 ] =IM[K1 ] ∪ IM[K2 ] ∪ IM[K3 ] ∪ IM[K1 , K2 ] ∪ IM[K1 , K3 ] ∪ IM[K2 , K3 ] ∪ IM[K1 , K2 , K3 ] . 3.3
Composition Semantics and Properties
We consider that a system S is a pair S = (B, IM) where B is the behavior of S and IM is its interaction model. As in the previous section, IM is the interaction model of a set of interacting components K with disjoint action vocabularies Ai , i ∈ K. For given K ⊆ K, we denote by S[K ] the sub-system of S consisting of components of K , S[K ] = (B[K ], IM[K ]), where IM[K ] is defined as before. We define a composition operator allowing to obtain for disjoint sub-sets K1 , K2 of K, the system S[K1 ∪K2 ] as the composition of the sub-systems S[K1 ], S[K2 ] for given interaction model IM[K1 , K2 ] connecting the two sub-systems. The operator composes separately the behavior and interaction models of the sub-systems. Definition 3. The composition of two systems S[K1 ] and S[K2 ] is the system S[K1 ∪ K2 ] = (B[K1 ], IM[K1 ]) (B[K2 ], IM[K2 ]) = (B[K1 ] × B[K2 ], IM[K1 ] ∪ IM[K2 ] ∪ IM[K1 , K2 ]) where × is a binary associative behavior composition operator such that B[K1 ] × B[K2 ] = B[K1 ∪ K2 ]. Due to proposition 1 we have (B[K1 ], IM[K1 ]) (B[K2 ], IM[K2 ]) = (B[K1 ∪ K2 ], IM[K1 ∪K2 ]), which means that composition of sub-systems gives the system corresponding to the union of their components. Notice that under these assumptions composition is associative: (B[K1 ], IM[K1 ]) (B[K2 ], IM[K2 ]) (B[K3 ], IM[K3 ]) = = (B[K1 ∪ K2 ], IM[K1 ∪ K2 ]) (B[K3 ], IM[K3 ]) = (B[K1 ] × B[K2 ] × B[K3 ], IM[K1 ∪ K2 ] ∪ IM[K3 ] ∪ IM[K1 ∪ K2 , K3 ]) = (B[K1 ∪ K2 ∪ K3 ], IM[K1 ∪ K2 ∪ K3 ]) by application of proposition 2. Transition Systems with Priorities. As a rule, interaction models constrain the behaviors of integrated components. We consider the particular case where interactions are atomic, component behaviors are transition systems, and the constraints are modeled as priority orders on interactions. Transition systems with dynamic priorities have already been studied and used to model timed systems. The interested reader can refer to [6,1].
Composition for Component-Based Modeling
455
Fig. 3. The composition principle.
Definition 4 (Transition system). A transition system B is a triple (Q, I(A), →) where Q is a set of states, I(A) is a set of interactions on the action vocabulary A, and →⊆ Q × I(A) × Q is a transition relation. α
As usual, we write q1 → q2 instead of (q1 , α, q2 ) ∈→. Definition 5 (Transition system with priorities). A transition system with priorities is a pair (B, ≺) where B is a transition system with set of interactions I(A), and ≺ is a priority order, that is, a strict partial order on I(A). Semantics: A transition system with priorities represents a transition system: if B = (Q, I(A), →), then (B, ≺) represents the transition system B = α α (Q, I(A), → ) such that q1 → q2 if q1 → q2 and there exists no α and q3 such α
that α ≺ α and q1 → q3 . Definition 6 (⊕). The sum ≺1 ⊕ ≺2 of two priority orders ≺1 , ≺2 is the least priority order (if it exists) such that ≺1 ∪ ≺2 ⊆≺1 ⊕ ≺2 . Lemma 1. ⊕ is a (partial) associative and commutative operator.
456
G. G¨ ossler and J. Sifakis
Definition 7 ( ). Consider a system S[K] with interaction model IM[K] = (IC[K], IC[K]+ ). Let S[K1 ] = (B[K1 ], ≺1 ) and S[K2 ] = (B[K2 ], ≺2 ) with disjoint K1 and K2 be two sub-systems of S[K] such that their priority orders do not allow domination of complete interactions by incomplete ones, that is for all α1 ∈ IC[K]+ and α2 ∈ IC[K]− , ¬(α1 ≺ α2 ). The composition operator is defined as follows. If Bi = (Qi , IC[Ki ], →i ) for i = 1, 2, then S[K1 ] S[K2 ] = (B1 × B2 , ≺1 ⊕ ≺2 ⊕ ≺12 ), where B1 × B2 = (Q1 × Q2 , IC[K1 ∪ K2 ], →12 ) with α
α
α
α
α
α α
q1 →1 q1 implies (q1 , q2 ) →12 (q1 , q2 ) q2 →2 q2 implies (q1 , q2 ) →12 (q1 , q2 ) α
1 2 q1 →1 1 q1 and q2 →2 2 q2 implies (q1 , q2 ) → 12 (q1 , q2 ) if α1 α2 ∈ IC[K1 ∪ K2 ].
≺12 is the minimal priority order on IC[K1 ∪ K2 ] such that – α1 ≺12 α1 α2 for α1 α2 ∈ IC[K1 , K2 ] (maximal progress priority rule); – α1 ≺12 α2 for α1 ∈ IC[K1 ∪ K2 ]−− and α2 ∈ IC[K1 ∪ K2 ]+ (completeness priority rule), where IC[K1 ∪ K2 ]−− denotes the elements of IC[K1 ∪ K2 ]− that are non-maximal in IC[K1 ∪ K2 ]. The first priority rule favors the largest interaction. The second ensures correctness of the model. It prevents the occurrence of incomplete interactions if they are not maximal. The occurrence of such interactions in a model is a modeling error. If a component can perform a complete action, all non maximal interactions of the other components are prevented. By executing complete actions the components may reach states from which a maximal incomplete interaction is possible. Proposition 3. is a total, commutative and associative operator. Proof. Total operator: prove that for K1 ∩ K2 = ∅, ≺1 ⊕ ≺2 ⊕ ≺12 is a priority order, that is, the transitive closure of the union of ≺1 , ≺2 , and ≺12 does not have any circuits. The maximal progress priority rule defines a priority order identical to the set inclusion partial order, and is thus circuit-free. The completeness priority rule relates incomplete and complete interactions and is circuit-free, too. The only source of a priority circuit could be the existence of interactions α1 , α2 , α3 ∈ IC[K1 ∪ K2 ] such that α1 = α2 α3 , α1 ∈ IC[K1 ∪ K2 ]−− , and α2 ∈ IC[K1 ∪ K2 ]+ . This is impossible due to the monotonicity requirement of definition 1. Associativity: (B[K1 ], ≺1 ) (B[K2 ], ≺2 ) (B[K3 ], ≺3 ) = = (B[K1 ∪ K2 ], ≺1 ⊕ ≺2 ⊕ ≺12 ) (B[K3 ], ≺3 ) = (B[K1 ∪ K2 ∪ K3 ], ≺1 ⊕ ≺2 ⊕ ≺12 ⊕ ≺3 ⊕ ≺[12],3 ) where ≺[12],3 is the least priority order defined by
Composition for Component-Based Modeling
457
Fig. 4. Composition: producer/consumer.
– α1 ≺[12],3 α1 α2 for α1 α2 ∈ IC[K1 ∪ K2 , K3 ], and – α1 ≺[12],3 α2 for α1 ∈ IC[K1 ∪ K2 ∪ K3 ]−− and α2 ∈ IC[K1 ∪ K2 ∪ K3 ]+ . It can be shown that the order ≺=≺12 ⊕ ≺[12],3 is the one defined by – α1 ≺ α1 α2 for α1 α2 ∈ IC[K1 , K2 ] ∪ IC[K1 , K3 ] ∪ IC[K2 , K3 ] ∪ IC[K1 , K2 , K3 ], and – α1 ≺ α2 for α1 ∈ IC[K1 ∪ K2 ∪ K3 ]−− and α2 ∈ IC[K1 ∪ K2 ∪ K3 ]+ . So the resulting priority order is the same independently of the order of composition. Example 1. Consider the system consisting of a producer and a consumer. The components interact by rendez-vous. The actions put and get are incomplete. We assume that the actions prod and cons are internal and thus complete. Figure 4 gives the interaction model corresponding to these assumptions. The product system consists of the product transition system and the priority order defined from the interaction model. The priority order removes all incomplete actions (crossed transitions). 3.4
Correctness by Construction
We present results allowing to check correctness of the models with respect to two properties: interaction safety and deadlock-freedom.
458
G. G¨ ossler and J. Sifakis
Interaction Safety of the Model. As explained in section 3.1, the distinction between complete and incomplete interactions is essential for building correct models. In existing formalisms, undesirable incomplete interactions are pruned out by applying restriction operators to the model obtained as the product of components [19]. In our approach, we replace restriction by priorities. This allows deadlock-freedom preservation: if an interaction is prevented from occurring, then some interaction of higher priority takes over. Nevertheless, it is necessary to check that our “flexible” composition operator does not allow illegal incomplete actions in a system model. For this we induce a notion of correctness called interaction safety. Interaction safety is a property that must be satisfied by system models at any stage of integration. Notice however, that legality of incomplete interactions depends on the set of integrated components. Sub-systems of a given system may perform incomplete interactions that are not legal interactions of the system. For instance, consider a system consisting of three components with a connector {a1 , a2 , a3 } such that all its interactions are incomplete. The interaction a1 a2 is legal in the sub-system consisting of the first two components while it is illegal in the system. In the latter, a1 a2 is incomplete and non maximal. It must synchronize with a3 to produce the maximal incomplete interaction a1 a2 a3 . For a given system, only complete and maximal incomplete interactions are considered as legal. Definition 8 (Interaction safety). Given an interaction model IM = (IC, IC+ ), define the priority order ≺ on incomplete interactions such that α1 ≺ α2 if α1 ∈ IC−− and α2 ∈ IC− IC−− . A system with interaction model IM is interaction safe if its restriction with ≺ can perform only complete or maximal incomplete interactions. Notice that the rule defining the priority order ≺ is similar to the completeness priority rule of definition 7. For a given system, incomplete interactions that are maximal in IC have the same status as complete interactions with respect to non maximal incomplete interactions. Nevertheless, the priority order ≺ depends on the considered system as legality of incomplete actions depends on the interaction model considered. We give below results for checking whether a model is interaction safe. Dependency graph: Consider a system S[K] consisting of a set of interacting components K with interaction model IM = (IC, IC+ ). For c ∈ C (C is the set + (c) the set of the minimal complete of the connectors of IC ) we denote by Imin + + interactions of c, and write Imin (C) for {i ∈ Imin (c)}c∈C . The dependency graph of S[K] is a labelled bipartite graph with two sets of nodes: the components of K, and nodes labelled with elements of the set + + (c) = ∅} ∪ {(c, α) | c ∈ C ∧ α ∈ Imin (c)}, where α(c) is {(c, α(c)) | c ∈ C ∧ Imin the maximal interaction of c (involving all the elements of c). The edges are labelled with actions of A as follows: Let (c, α) = ({a1 , . . . , an }, α) be a node of the graph and assume that for an action ai of c, owner(ai ) ∈ K is the component which is owner of action ai . For
Composition for Component-Based Modeling
459
all actions ai of c occurring in α, add an edge labelled with ai from owner(ai ) to (c, α). For all actions ai of c, add an edge labelled with ai from (c, α) to owner(ai ) if ai is offered in some incomplete state of owner(ai ), that is, a state in which no complete or maximal action is offered. The graph encodes the dependency between interacting actions of the components in the following manner. If a component has an input edge labelled ai from a node ({a1 , . . . , an }, α), then for ai to occur in some interaction of {a1 , . . . , an } containing α it is necessary that all the actions labelling input edges of ({a1 , . . . , an }, α) interact. We call a circuit in the dependency graph non trivial if it encompasses more than one component node. Example 2 (Producer/consumer). Consider a producer providing data to two consumers. Interaction is by rendez-vous and takes place if at least one of the two consumers can get an item. The interaction model is described by C = {put, get1 , get2 } and IC+ = {put get1 , put get2 , put get1 get2 }. The dependency graph is shown in figure 5.
Fig. 5. Dependency graph for the producer/two consumer example.
Definition 9 (Cooperativity). Let a and b be labels of input and output edges of a component k in the dependency graph of S[K]. We say that a component k ∈ K is cooperative with respect to (a, b) if from any state of B[k] with a transition labelled a there exists a transition labelled b. k ∈ K is cooperative in a circuit γ in the dependency graph if it is cooperative wrt. (a, b), where a and b are the arcs of γ entering and leaving k, respectively. Theorem 1 (Interaction safety). A system model is interaction safe if its dependency graph contains a non-empty sub-graph G such that (1) G contains all
460
G. G¨ ossler and J. Sifakis
its predecessors, (2) any component in G is deadlock-free, and in any elementary circuit γ of G, either (3a) there exists a component k that is cooperative in γ and whose successor node in γ is a binary interaction, or (3b) the set of components k in γ whose successor node is not a binary interaction, is not empty, and all components in this set are cooperative in γ. Proof. Assume that the system is in an incomplete state, that is, a state from which only incomplete actions are possible. Then each component in G offers some incomplete action since it is deadlock-free. We consider the sub-graph G of G that represents dependencies in the current state: G has an edge from an interaction node (c, α) to a component node k if k is actually waiting for α in the current state; G has the same edges from component to interaction nodes as G. G has the same set of components as G since any component of G is awaiting at least one incomplete action. If according to (3a) one of the components k is cooperative in some non trivial elementary circuit γ G , and the successor node (c, α) of k in γ is a binary interaction, then k and the successor of (c, α) can interact via the complete or maximal interaction α. Otherwise, all non trivial circuits in G satisfy condition (3b). Let k be some component in a strongly connected sub-graph of G not having any predecessors. Such a sub-graph exists since any component is node of some non-trivial circuit. Let γ be a non-trivial circuit in G containing k, and consider some non-binary interaction node (c, α) in γ. Let k be an arbitrary predecessor node of (c, α) in G . By the choice of k, k and (c, α) are in some non-trivial circuit γ of G . γ satisfies (3b), which implies that k is cooperative in γ . That is, all predecessors of (c, α) are cooperative, such that the complete or maximal interaction α is enabled. In both cases, at least one complete or maximal interaction is enabled, which means that any non-maximal incomplete interaction is disabled in (B, ≺). Intuitively, the hypotheses of Theorem 1 make sure that any circular dependency between the occurrence of strict interactions is broken by some cooperative component. Notice that by definition components are cooperative with respect to (a, a) for any action a. If the dependency graph has a backwards-closed subgraph all of whose elementary circuits are self-loops with the same label then the model is interaction safe. Example 3 (Producer/consumer). For example 2, the only subgraph G satisfying the backward closure requirement is the whole dependency graph. Let n1 = ({put, get1 , get2 }, put get1 ) and n2 = ({put, get1 , get2 }, put get2 ). ΓG contains two non-trivial elementary circuits γ1 = (producer, n1 , consumer2 , n2 ) and γ2 = (producer, n2 , consumer1 , n1 ). Since the producer is trivially cooperative wrt. the pair (put, put), condition (3a) is satisfied. If all three components are deadlockfree, the system is interaction safe.
Composition for Component-Based Modeling
461
Deadlock-Freedom. We give some results about deadlock-freedom preservation for transitions systems with priorities. Similar results have been obtained for timed transition systems with priorities in [5]. Definition 10 (Deadlock-freedom). A transition system is called deadlockfree if it has no sink states. A system is deadlock-free if the transition system with priorities representing it is deadlock-free. Proposition 4 (Composability). Deadlock-freedom is preserved by priority orders that is, if B is deadlock-free then (B, ≺) is deadlock-free for any priority order ≺. Proposition 5 (Compositionality). Deadlock-freedom is preserved by composition that is, if (B1 , ≺1 ) and (B2 , ≺2 ) are deadlock-free then (B1 , ≺1 ) (B2 , ≺2 ) is deadlock-free. Proof. Follows from the fact that composition of behaviors preserves deadlockfreedom and from the previous proposition. Proposition 6. Any system obtained by composition of deadlock-free components is deadlock-free.
4
Execution Model
Execution models constitute the third layer. They implement constraints which superposed to interaction constraints further restrict the behavior of a system by reducing non determinism. They differ from interaction models from a pragmatic point of view. Interaction models restrict behavior so as to meet global functional properties, especially properties ensuring harmonious cooperation of components and integrity of resources. Execution models restrict behavior so as to meet global performance and efficiency properties. They are often timed and specific to execution platforms. In that case, they describe scheduling policies which coordinate system activities by taking into account the dynamics of both the execution platform and of the system’s environment. We assume that execution models are also described by priority orders, and discuss two interesting uses of execution models. Asynchronous vs. Synchronous Execution. As explained in 2.2, synchronous execution adopts a very strong fairness assumption as in all computation steps components are offered the possibility to execute some quantum of computation. Our thesis is that synchronous execution can be obtained by appropriately restricting the first two layers. Clearly, it is possible to build synchronous systems by using specific interaction models to compose behaviors. This is the case for Statecharts, and synchronous languages whose semantics
462
G. G¨ ossler and J. Sifakis
use parallel composition operators combined with unary restriction operators [17]. Nevertheless, their underlying interaction model uses non strict interaction and specific action composition laws which are not adequate for asynchronous execution. In the proposed framework, systems consisting of the first two layers are not synchronous, in general. Interactions between components may be loose. Components keep running until they reach some state from which they offer a strongly synchronizing action. Thus, executions are rich in non-determinism resulting from the independence of computations performed in the components. This is the case for formalisms which point to point interaction, such as SDL and UML. We believe that it is possible to define synchronous execution semantics for appropriate sub-sets of asynchronous languages. Clearly, these sub-sets should include only reactive components, that is, components with distinct input and output actions such that when an input occurs some output(s) eventually occur. The definition of synchronous execution semantics for asynchronous languages is an interesting and challenging problem. Consider the example of figure 6, a system which is the serial composition of three strongly synchronized components with inputs ij and outputs oj , j = 1, 2, 3. Assume that the components are reactive in the sense that they are triggered from some idle (stable) state when an input arrives and eventually produce an output before reaching some idle state from where a new input can be accepted. For the sake of simplicity, components have simple cyclic behaviors alternating inputs and outputs. The interaction model is specified by {o1 , i2 , o2 , i3 } ≺ {i1 , o3 }, {o1 , i2 } ≺ o1 i2 , {o2 , i3 } ≺ o2 i3 . That is, we assume that i1 and o3 are complete as the system is not connected to any environment. In the product of the behaviors restricted with the interaction model each component can perform computation independently of the others provided the constraints resulting from the interaction model are met. This corresponds to asynchronous execution. The behavior of the two layers can be further constrained by an execution model to become synchronous in the sense that a run of the system is a sequence of steps, each step corresponding to the treatment of an input i1 until an output o3 is produced. This can be easily enforced by the order i1 ≺ o1 i2 ≺ o2 i3 ≺ o3 . This order reflects the causality order between the interactions of the system. In fact, if all the components are at some idle state then all the components are awaiting for an input. Clearly, only i1 can occur to make the first component evolve to a state from which o1 i2 can occur. This will trigger successively o2 i3 and finally o3 . Notice that i1 cannot be executed as long as a computation takes place in some component.
Scheduling Policies as Execution Models. We have shown in [1] that general scheduling policies can be specified as timed priority orders. The following example illustrates this idea for untimed systems.
Composition for Component-Based Modeling
463
Fig. 6. Enforcing synchronous execution.
We model fixed priority scheduling with pre-emption for n processes sharing a common resource (figure 7). The scheduler gives preference to low index processes. The states of the i-th process are si (sleeping), wi (waiting), ei (executing), and ei (pre-empted). The actions are ai (arrival), bi (begin), fi (finish), pi (preempt), ri (resume). To ensure mutual exclusion between execution states ei , we assume that begin actions bj are complete and synchronize with pi for all 1 i, j n, i = j. By the maximal progress priority rule, an action bj cannot occur if some interaction bj pi is possible. Similarly, we assume that finish = j. An actions fj are complete and synchronize with ri for all 1 i, j n, i action fj cannot occur if some interaction fj ri is possible. The system is not interaction safe, since the structural properties of theorem 1 cannot exclude the case where the system is in the incomplete state (e1 , . . . , en ), that is, all processes are preempted. However, this is the only incomplete state of the system, and it is easy to show that it is not reachable from any other state: as all actions pi are incomplete, they are disabled by the completeness priority rule of definition 7 giving priority to the complete actions. Interactions bi pj are complete but keep component i, and thus the whole system, in a complete state. Therefore, initialized in a complete state the system always remains in a complete state, where interaction safety is guaranteed. Scheduling constraints resolve conflicts between processes (bi and ri actions) competing for the acquisition of the common resource. They can be implemented
464
G. G¨ ossler and J. Sifakis
Fig. 7. Fixed-priority preemptive scheduling of processes.
by adding a third layer with the priority rules bi ≺ bj , bi pk ≺ bj pk , and fk ri ≺ fk rj for all k, and i > j. It is easy to check that these constraints preserve mutual exclusion, in the sense that if the initial state respects mutual exclusion then mutual exclusion holds at any reachable state. Notice that as the components are deadlock-free and the composition of the interaction and execution priority orders is a priority order, the obtained model is deadlock-free.
5
Discussion
The paper proposes a framework for component composition encompassing heterogeneous interaction and execution. The framework uses a single powerful associative and commutative composition operator for layered components. Component layering seems to be instrumental for defining such an operator. Existing formalisms combine at the same level behavior composition and unary restriction operators to achieve interaction safety. Layered models allow separation of concerns. Behaviors and restrictions (represented by priority orders) are composed separately. This makes technically possible the definition of a single associative operator. Interaction models describe architectural constraints on component behavior. Connectors relate interacting actions of different components. They naturally define the set of interactions of a system. The distinction between complete and incomplete interactions is essential for the unification of existing interaction
Composition for Component-Based Modeling
465
mechanisms. It induces the property of interaction safety characterizing correctness of a model with respect to modeling assumptions about the possibility for interactions to occur independently of their environment. Such assumptions are implicit in existing formalisms. Their satisfaction is enforced on models at the risk of introducing deadlocks. The proposed composition operator preserves deadlock-freedom. Theorem 1 can be used to check interaction safety of models. The distinction between interaction and execution models is an important one from a methodological point of view. Priority orders are a powerful tool for describing the two models. Their use leads to a semantic model consisting of behaviors and priorities which is amenable to correctness by construction. This is due to the fact that priorities are restrictions that do not introduce deadlocks to an initially deadlock-free system. More results about deadlock-freedom and liveness preservation can be found in [5].
References 1. K. Altisen, G. G¨ ossler, and J. Sifakis. Scheduler modeling based on the controller synthesis paradigm. Journal of Real-Time Systems, special issue on ”controltheoretical approaches to real-time computing”, 23(1/2):55–84, 2002. 2. F. Balarin, L. Lavagno, C. Passerone, A. Sangiovanni-Vincentelli, M. Sgroi, and Y. Watanabe. Modeling and Designing Heterogeneous Systems, volume 2549 of LNCS, pages 228–273. Springer-Verlag, 2002. 3. A. Benveniste, P. LeGuernic, and Ch. Jacquemot. Synchronous programming with events and relations: the SIGNAL language and its semantics. Science of Computer Programming, 16:103–149, 1991. 4. G. Berry and G. Gonthier. The ESTEREL synchronous programming language: Design, semantics, implementation. Science of Computer Programming, 19(2):87– 152, 1992. 5. S. Bornot, G. G¨ ossler, and J. Sifakis. On the construction of live timed systems. In S. Graf and M. Schwartzbach, editors, Proc. TACAS’00, volume 1785 of LNCS, pages 109–126. Springer-Verlag, 2000. 6. S. Bornot and J. Sifakis. An algebraic framework for urgency. Information and Computation, 163:172–202, 2000. 7. L. de Alfaro and T.A. Henzinger. Interface theories for component-based design. In T.A. Henzinger and C. M. Kirsch, editors, Proc. EMSOFT’01, volume 2211 of LNCS, pages 148–165. Springer-Verlag, 2001. 8. W.-P. de Roever, F. de Boer, U. Hannemann, J. Hooman, Y. Lakhnech, M. Poel, and J. Zwiers. Concurrency Verification: Introduction to Compositonal and Noncompositional Methods. Cambridge University Press, 2001. 9. W.-P. de Roever, H. Langmaack, and A. Pnueli, editors. Compositionality: The Significant Difference, volume 1536 of LNCS. Springer-Verlag, 1997. 10. OMG Working Group. Response to the omg rfp for schedulability, performance, and time. Technical Report ad/2001-06-14, OMG, June 2001. 11. N. Halbwachs, P. Caspi, P. Raymond, and D. Pilaud. The synchronous dataflow programming language lustre. Proceedings of the IEEE, 79(9):1305–1320, September 1991. 12. D. Harel. Statecharts: A visual formalism for complex systems. Science of Computer Programming, 8:231–274, 1987.
466
G. G¨ ossler and J. Sifakis
13. C. A. R. Hoare. Communicating Sequential Processes. Prentice Hall, 1985. 14. ITU-T. Recommendation Z.100. Specification and Design Language (SDL). Technical Report Z-100, International Telecommunication Union — Standardization Sector, Geneva, 1999. 15. L. Lamport. Specifying concurrent program modules. ACM Trans. on Programming Languages and Systems, 5:190–222, 1983. 16. E.A. Lee et al. Overview of the Ptolemy project. Technical Report UCB/ERL M01/11, University of California at Berkeley, 2001. 17. F. Maraninchi. Operational and compositional semantics of synchronous automaton compositions. In proc. CONCUR, volume 630 of LNCS. Springer-Verlag, 1992. 18. R. Milner. Calculi for synchrony and asynchrony. Theoretical Computer Science, 25(3):267–310, 1983. 19. R. Milner. Communication and Concurrency. Prentice Hall, 1989. 20. SystemC. http://www.systemc.org.
Games for UML Software Design Perdita Stevens and Jennifer Tenzer Software Engineering Programme and Laboratory for Foundations of Computer Science School of Informatics, University of Edinburgh [email protected] Fax: +44 131 667 7209 [email protected] Fax: +44 131 667 7209
Abstract. In this paper we introduce the idea of using games as a driving metaphor for design tools which support designers working in UML. We use as our basis a long strand of work in verification and elsewhere. A key difference from that strand, however, is that we propose the incremental development of the rules of a game as part of the design process. We will argue that this approach may have two main advantages. First, it provides a natural means for tools to interactively help the designer to explore the consequences of design decisions. Second, by providing a smooth progression from informal exploration of decisions to full verification, it has the potential to lower the commitment cost of using formal verification. We discuss a simple example of a possible game development.
1
Introduction
The Unified Modeling Language (UML)[10] is a widely adopted standard for modelling object-oriented software systems. It consists of several diagram types providing different views of the system model. UML is a semi-formal language defined by a combination of UML class diagrams, natural language and formal constraints written in the object constraint language (OCL). There are many tools available which support, or claim to support, design with UML. They aid in drawing UML diagrams, generation of code fragments in different object-oriented languages and documentation of software systems. Some of them are able to perform consistency checks, for example, checking accessibility of referenced packages. However, these features seem to be useful for the recording and verifying of a chosen design, rather than for the design activity itself. There is nothing available to the mainstream object-oriented business software developer which will actively help him or her to explore different design decisions and work out which is the best option. It is natural to look to verification to fill this gap. Ideally, a designer would be able to make use of verification technology whenever s/he is faced with a difficult decision, say between two design solutions. S/he might take two models representing the design with each of the solutions applied, F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 467–486, 2003. c Springer-Verlag Berlin Heidelberg 2003
468
P. Stevens and J. Tenzer
together with some desired properties, and check in each case whether the desired properties are fulfilled. Then s/he would adopt the design that best met the requirements. Unfortunately this situation is far from the truth at present. The decision to verify cannot be taken so lightly, for several reasons. It generally requires the building of a special purpose model, which has to be complete in an appropriate sense. (There are a few tools which can work directly with UML models, but most work only with one diagram type, typically state diagrams.) Writing properties which should be fulfilled by a design is a specialist job requiring knowledge of an appropriate logic: even for someone with that knowledge, identifying the properties at an appropriate level of granularity can sometimes be even harder than the design problem itself. One of the issues is that a good design does not only meet the external, customer requirements. It also does so in a clear, understandable way so that the software will be maintainable in future. The designer’s choice often involves understanding the implications of each decision and then making essentially aesthetic judgements about them. That is, the desirable characteristics are often a combination of technical, formalisable features and features pertaining to how future human maintainers will most easily think. For this reason, it is unlikely that the process of design will ever be fully automated: verification and synthesis will not replace design. Instead tools should support the human designer in making use of formal technology alongside his/her own skills. Thus the ideal tool support should do two things: first, it should help make verification in the traditional sense, against explicit requirements, available at any stage of design and even, as far as possible, in the presence of incomplete models; second, it should help the designer to explore consequences of design decisions so that choices can be made even when the criteria for the choices have not been formalised.
2
Games in Verification
Our thesis is that mathematical, formal games may be a suitable basis for improving tool support for software design. In order to support this, we need to introduce the formal games, before explaining the relevance of games to software design. In this section we introduce games by means of a simple example, the bisimulation game as explained by Stirling in [12]. Such a game is used for checking whether two processes are equivalent under the equivalence relation known as bisimulation. Essentially this captures the idea that two processes can each simulate the other, and that during the simulation their states remain equivalent, so that either process can “lead” at any time. A bisimulation game is defined over two processes E and F and played by players Refuter (abbreviated R) and Verifier (abbreviated V ). The aim of player R is to show that E and F are different from each other while player V wants to prove that E is equivalent to F. At the beginning of the game player R picks one of the two processes and chooses a transition. After that player V has to
Games for UML Software Design
469
respond by choosing a transition with the same label from the other process. This procedure is repeated and each time player R can choose a transition from either process. Furthermore all moves in the game, no matter by which player they were made, can be seen by both players. If one of the players is stuck and cannot choose a suitable transition, the other player wins the game. In the case that the game is infinite player V wins. A player can play the game according to a strategy, which is a set of rules. These rules tell the player how to move and may depend on earlier decisions taken in the game. A strategy is called a winning strategy if a player wins every game in which he or she uses it. Figure 1 shows the classic example of two vending machines. Imagine a machine E which has only one coin slot, and a machine F which has separate slots for tea and coffee. E and F are not equivalent because player R has a winning strategy consisting of the following rules: 1. Pick transition 20p from E. 2. If player V responds with the left transition 20p in F, choose selCoffee in E. Otherwise select transition selTea in E. If player R follows this strategy, player V gets stuck and thereby player R wins the game. Notice that playing the game yields a counter-example, i.e. a particular sequence of moves from which we can see that E and F are not equivalent, which is an advantage of using formal games for verification tasks. E
F
20p getTea
getCoffee
selTea
getTea
20p
20p
getCoffee
selCoffee selTea
selCoffee
Fig. 1. Bisimulation game for two vending machines E and F.
In this game a winning strategy for Refuter can be viewed as a proof that no bisimulation could exist. Similarly, a winning strategy for player V is a bisimulation relation, that is, it is a proof that the answer to the question “are these processes bisimilar?” is Yes. Similarly, model-checking can be expressed as a game: this one is somewhat more complicated, and we will not go into details here (the interested reader is referred to [3]). Two differences are particularly worth noting. First, in a modelchecking game it is not necessarily the case that the players alternate turns. The
470
P. Stevens and J. Tenzer
current game position determines who is to move. Depending on the position and the move chosen by a player, the next game position may be one from which the same player is to move again. Second, the winning conditions are more complex than the simple fact that V wins all infinite plays. Instead, there is a predicate on plays which determines which player wins a given play. In fact, it seems that all verification questions can be expressed as games, although sometimes the game is “trivial” in the sense that a play will always be over after two moves. Tools such as the Edinburgh Concurrency Workbench1 exploit this game view of verification questions. The user asks the question; the tool calculates a winning strategy for the game; it then offers to take the winning part in a game against the user. The user finds that, no matter which moves s/he chooses, the tool always has an answer: the user can only lose. This seems to be an effective way of getting an intuition about why the answer to the question is as it is.
3
Beyond Verification Games
When we use verification games, we notice a curious fact. A typical scenario is as follows: we have one process, representing a system, and we want to verify that it is correct. We may choose to do this in any one of several ways. We may develop a second process, which is supposed to stand in some formal relation to our system process; perhaps the two are supposed to be bisimilar or perhaps one is supposed to be a refinement of the other according to one of the many different refinement relations used in process algebra. Alternatively, we may choose to develop a logical formula in a temporal logic such as the modal mu calculus. In either case, the verification problem can be expressed as a game. The positions of the game incorporate some information about the “current state” of the model, and also some information about the “current state” of the specification. If the two specifications, by process and by logic, intuitively succeed in expressing the same notion of correctness, they correspond to (essentially) the same game. Now, games are rather natural: the idea of defining a game between Verifier and Refuter by describing the valid challenges that Refuter may make, how Verifier may respond, and who wins under which circumstances, is quite easy to grasp. It is easier than understanding the semantics of one’s chosen relation between two processes, or understanding the semantics of the modal mu calculus. (Indeed in many institutions, including Edinburgh, this fact is often used to help students grasp the concepts of bisimulation and the semantics of the mu calculus.) Thus the central idea of this work is to allow the user to define, directly, a verification game. The game should involve the existing design of the system, together with the information that determines whether the design is correct. It should be the case that the player Verifier has a winning strategy for the game if and only if the design is correct. The rules of the game incorporate 1
http://www.lfcs.ed.ac.uk/cwb/
Games for UML Software Design
471
the challenging circumstances in which the design must work as challenges by Refuter; correct and incorrect behaviour is captured by the winning conditions. Once we have decided to let the user define a game, we may observe that the game can in fact be defined incrementally. For example, suppose that the design is complete, but that there is only limited understanding of what it means for the design to be correct. (We assume that, as usual, there is no formal specification. Perhaps it has not yet been understood how the informal specification of overall system requirements should be translated down into precise requirements; or perhaps the informal specification is itself incomplete or incorrect. In mainstream business software development, which is our main focus of concern, both are likely to be the case.) In this case the game as initially defined by the user will incorporate only a small amount of information about what it is for the design to be correct: it will be “too easy” for Verifier to win the game. The user should be able to explore the game and improve the rules to make it a better reflection of the correctness of the design. This might include, for example, changing the moves of the game to permit Refuter to make new challenges, or changing the winning conditions so that plays which would have been won by Verifier are won by Refuter in the new game. At the same time, it is likely that the design itself is too incomplete to permit full verification. The user should also be able to change the game by adding more information about the design. In order to work more formally with this basic idea, let us begin by defining what is meant by a game in general. 3.1
Game Terminology and Formal Definition
For the purposes of this paper, a game is always played between two players Verifier (abbreviated V ) and Refuter (abbreviated R). We refer to players A and B to mean Verifier and Refuter in either order. Definition 1. A game G is (Pos, I, moves, λ, WR , WV ) where: – P os is a set of positions. We use u, v, . . . for positions. – I ⊆ P os is a set of starting positions: we insist that λ(i) = λ(j) for all i, j ∈ I. – moves ⊆ P os × P os defines which moves are legal. A play is in the obvious way a finite or infinite sequence of positions starting at some p0 ∈ I where pj+1 ∈ moves(pj ) for each j. We write pij for pi . . . pj . – λ : P os → {Verifier, Refuter} defines who moves from each position. – WR , WV ⊆ P osω are disjoint sets of infinite plays, and (for technical reasons to do with working with abstractions of games) WA includes every infinite play p such that there exists some i such that for all k > i, λ(pk ) = B. Player A wins a play p if either p = p0n and λ(pn ) = B and moves(pn ) = ∅ (you win if your opponent can’t go), or else p is infinite and in WA . Notice that our general definition does not insist that WR ∪ WV = P osω ; that is, it is possible for a play to be a draw. The games we consider in this
472
P. Stevens and J. Tenzer
paper will have no draws, but when we go on to consider abstractions of games (see e.g. [11]) it is necessary to permit them. Definition 2. A (nondeterministic) strategy S for player A is a partial function from finite plays pu with λ(u) = A to sets of positions (singletons, for deterministic strategies), such that S(pu) ⊆ moves(u) (that is, a strategy may only prescribe legal moves). A play q follows S if whenever p0n is a proper finite prefix of q with λ(pn ) = A then pn+1 ∈ S(p0n ). Thus an infinite play follows S whenever every finite prefix of it does. It will be convenient to identify a strategy with the set of plays following the strategy and to write p ∈ S for p follows S. S is a complete strategy for Player A if whenever p0n ∈ S and λ(pn ) = A then S(p0n ) = ∅. It is a winning strategy for A if it is complete and every p ∈ S is either finite and extensible or is won by A. It is non-losing if it is complete and no p ∈ S is won by B. It is history-free (or memoryless) if S(pu) = S(qu) for any plays pu and qu with a common last position. A game is determined if one player has a winning strategy. All the games we need to consider are determined, and this is an assumption of this work. In this paper we focus on the informal presentation of the idea of using and modifying games for software design. It should be clear, however, that there is scope for defining relationships between games in which the existence of a winning strategy for one game implies the existence of a winning strategy for a related game. Some early work in this direction was reported in [11] in the context of verification games. The study of these relationships between games will be important in the context of tools to support games for software design. A simple example is that increasing the number of options open to Refuter – e.g., adding requirements that a design should satisfy – should make it harder for Verifier to win the game: if Refuter had a winning strategy for the original game, the same strategy will work in the extended game. 3.2
How to Manage Games and Their Improvement in a Tool
A tool will always have a notion of “current game”. Part of what the tool should do is to allow the user to play the current game. The tool could operate in two modes: 1. Tool as referee. The user chooses moves both for Refuter and for Verifier (indeed, there might be several users playing against one another). The tool’s role is simply to ensure fair play and declare who wins (in the case of a finite play which is won at all). 2. Tool as opponent. The user chooses whether to play Verifier or Refuter and the tool takes the other player’s part. If it is possible for the tool to calculate a winning strategy for the current game, then the tool might play this winning strategy, or use it to suggest better moves to the user, as appropriate. Otherwise, the tool might use random choices and/or heuristics to play as well as possible.
Games for UML Software Design
473
It is not immediately obvious how to incorporate the improvement of games into the tool use. Should improving a game be thought of as part of the game, or as a separate process? It is undoubtedly easier for formal reasoning to regard the improvement of the game as a separate process: then we do not have to worry about what’s true of strange composite games in which the rules are changed part way through a play. For practical purposes though, if the user plays a significant number of moves and then realises that there is a problem with the rules which affects how play will proceed from then on, but not what has happened up to this point, it would be annoying not to be allowed to alter the rules and carry on. A suitable compromise is probably to say that a play in which the user changed the rules cannot be formally won by either player. (The tool might announce “You won, but you cheated so it doesn’t count”.) It is possible to formalise such rule changes as being the actions of a third player, but we have not so far found this very helpful.
4
Example: Incremental Definition of Simple Game
The overall aim of this work is to show how games akin to verification games can be used for software design, and incrementally designed as part of the software design process. Thus we have two aspects to address: the incremental definition of games, and the use of games for software design. In the interests of clarity we devote this section to demonstrating how a simple game, without any special relation to software design, can be defined incrementally. Our example is deliberately very simple. We will follow a hypothetical user through the design of a vending machine process. At any stage, there will be both a current system design and a current game, which will include the currently permissible challenges and winning conditions. Our “system design” will be simply a labelled transition system. We initialise this to the LTS shown in Figure 2. A position in the game may take either of two forms: 1. A state (si say) in the system design: Refuter to move. We notate such a position simply si . 2. A state si in the system design, plus a challenge c from Refuter. Verifier to move. We notate such a position (si , c). Note that it is immediate from the form of the position who is to move: we do not have to specify λ separately. Initially the winning conditions are that Verifier wins any infinite plays; other than this, we have only the default rules, that any player wins if their opponent is supposed to move but cannot legally do so. Thus in order to define the game we need to specify, for each state of the system design, what the legal challenges that Refuter may make are, and for each state and challenge, what the legal responses from Verifier are. Initially, we decide simply to record the requirements that from the initial state, s0, it is possible to insert 20p and (maybe many events later) receive tea, respectively coffee. We express this as the challenge to “pick a trace” with certain characteristics.
474
P. Stevens and J. Tenzer F
s0 20p
20p
s1
s2 selTea
selCoffee
s3
s4 getTea
getCoffee
s5
s6 Fig. 2. Initial system design.
State Challenges 20p
getTea
20p
getCoffee
s0 pick a trace s0 −→ . . . −→ si pick a trace s0 −→ . . . −→ si Note that such a challenge is easy to express in a table as above, and it would be easy for a tool to provide an interface for defining such challenges; however, expressing the property required to hold at s0 in a temporal logic is already slightly tricky, because of the need to permit actions other than those named in the traces. For example, the existence of the first trace from state s0 is equivalent to state s0 satisfying the mu calculus formula 20pµX. getTeaT ∨ −X. After Verifier successfully picks such a trace, we specify that the new game position will be si (i.e. system state si , with Refuter to move). There are (so far) no legal challenges from any system state other than s0, so if Verifier picks such a trace, which will necessarily not end in s0, she will win because it will be Refuter’s turn to pick a move, but there will be no legal moves. This is a boring game so far: Verifier easily wins. Suspending disbelief in the inability of the user to understand this, suppose that the user plays the game, taking either Verifier or Refuter’s part, and finally is satisfied of this. However, the user realises that the system should not simply stop after the tea or coffee is collected. S/he decides to capture the liveness of the system by permitting Refuter a new challenge, valid from any system design state, which is simply “pick a transition”. At this point, playing the game confirms that Refuter has a winning strategy. So the user refines the system design by merging s5 and s6 with s0 . We get the game described by Figure 3 and the table below.
Games for UML Software Design
475
F
s0 getTea
20p
20p
s1
s2 selTea
getCoffee
selCoffee
s4
s3
Fig. 3. Revised system design.
State Challenges 20p
getTea
20p
getCoffee
s0 pick a trace s0 −→ . . . −→ si si
pick a trace s0 −→ . . . −→ si pick a transition si −→
We could continue in various ways, with the user gradually improving both the system design and the challenges and winning conditions which (in this example) constitute its specification. So far we have only described adding challenges. Changing the winning conditions is the other main way of altering the game. We could imagine, for example, that instead of winning every infinite play, we might want to say that Verifier won every infinite play on which the number of 20p actions so far was infinitely often equal to the number of getTea actions plus the number of getCoffee actions so far, thus capturing the idea that the machine should not systematically swindle the beverage-buyer, or vice-versa. (A technical point is that we cannot specify arbitrary winning conditions without losing known determinacy of the game, that is, the existence of a winning strategy for one player: however, in practice, any reasonably specifiable winning condition will fall within the class known to lead to determined games.)
5
Games in the Context of Object Orientation/UML
In this section we address the question of how games for software design using UML may differ from standard verification games; we do not consider the incremental definition of such games. In the next section we consider an example that brings together aspects of the incremental definition considered in the previous section with the games for design issues considered here. The chief question is what constitutes the “system” being designed, the part analogous to the LTS in the previous example. The design artifact being produced by the designer is the UML model, so obviously the UML model is part of the system. Does it suffice? In order to explore the dynamic behaviour which is implied by a UML model, we will need to be able to record a “current state” of a prototypical system described by the model, and here we meet interesting issues. The UML model is
476
P. Stevens and J. Tenzer
most unlikely to define a unique system, complete in all detail. This is of course the usual challenge of doing any formal work with an incomplete specification. In the game framework, a promising approach is to allow one or both players, when faced with an incompleteness in the specification, to resolve the nondeterminacy. Different choices about exactly how this is done will yield different games with different strengths. For example, if a transition in a UML state diagram is guarded by an informal guard which cannot be formally evaluated, perhaps the player choosing the transition may decide to assume that the guard is satisfied. If the game is to be maximally powerful for detecting flaws, we probably wish to make sure that we record enough information to ensure that subsequent choices about whether that guard holds are consistent. For example, we might insist that the player writes a formal specification of the guard, and use this in place of the original informal guard to annotate the model for the rest of the play. On the other hand, a weaker treatment in which the rules of the game did not ensure consistency could still be useful for exploring possibilities. There are many such game-design choices to be made. More concretely, any particular game for design with UML will draw information from particular parts of a UML model – maybe a complete model, maybe just the parts of it considered most relevant. If the focus is to be on exploring consequences of design decision, it seems likely, though perhaps not essential, that some dynamic diagrams will be involved. The section following gives an example using a simple form of state diagrams, protocol state machines (PSMs)[10] in which transitions are labelled by events, not by actions. (In earlier work we attempted to use more general state machines with both events and actions labelling transitions. However, we found that in normal sequential systems where events and actions are the receiving and sending (respectively) of normal synchronous messages, the UML semantics has anomalies in the presence of recursion. This is discussed in [13], where we propose as a solution that PSMs without actions be used for providing (loose) specifications of the overall behaviour of classes, whilst method state machines are used to define the behaviour of operations.) Another issue which may arise is the possibility of using parts of the UML for more than simply describing the current system model. For example, it might be convenient to let the user describe certain kinds of challenges as sequence diagrams. We do not consider this further here.
6
Example: Definition of a Game for UML Software Design
We now show how a game for software design with UML could be defined. For this purpose we assume that a class diagram and state diagrams for some of its classes are given. As an example consider the class diagram shown in figure 4 with classes Customer, TravelAgent, Hotel and Flight which models a (very much simplified and not realistic) software system for a travel agency. The most interesting class is
Games for UML Software Design Customer
477
TravelAgent
−startDate: Date −endDate: Date +bookHoliday(l:String) +changeHotel(s:int)
ta +findFlight(d:String, sd:Date, ed:Date):Flight +findHotel(l:String, sd:Date, ed:Date, s:int):Hotel
−setFlight(f:Flight) −setHotel(h:Hotel)
−flight
−hotel
Flight
Hotel
...
...
+ destination: String ...
+location: String +nearestAirport:String ...
Fig. 4. Example class diagram.
Customer which has attributes for the holiday start and end dates of a customer. It contains public methods for booking a holiday, changing a hotel booking and private methods for linking hotel and flight objects. The parameter l to Customer::bookHoliday and TravelAgent::findHotel represents the desired holiday location and the parameter s to Customer::changeHotel and TravelAgent::findHotel represents the requested hotel quality given by the number of stars. The remaining parameters d, sd, ed in TravelAgent provide values for the flight destination (an airport name), and the start and end date of the holiday. A protocol state machine for Customer is given in Figure 5. Only the effects of booking a holiday, changing a hotel and setting hotel and flight on the object state are shown in this diagram. For the other classes of the system it is assumed that their objects can only be in a state default which is not changed by any of the methods, i.e. the corresponding state machines have only one state and a loop transition for each method of the class. For simplicity we only use classes and associations from the class diagram and the (finite) set of abstract states specified in the state machines for the definition of a state in the UML system design. As we will see later this is restrictive with respect to how the game can be incremented. For a given class diagram CD and a set of state machines SM , where S is the set of all states occurring in SM and Sc the set of states in the state machine for class c, a state in the UML system design consists of – a collaboration given by a set of objects O and a relation Links ⊆ O × R × O respecting the associations in CD where R is the set of role names in CD, including the empty string ε. A tuple (o, r, p) means that o can access p via rolename r.
478
P. Stevens and J. Tenzer Customer setHotel(h)
hotel only
setFlight(f) changeHotel(s)
no booking
bookHoliday(l)
setFlight(f)
hotel and flight
setHotel(h) flight only
Fig. 5. State machine for Customer.
– a function state : O → S such that state(o) ∈ Sc for an object o of class c. – a callstack cs = a1 : . . . : an whose elements ai are of the form o.m(p1 , . . . pn ) where o ∈ O is an object of class c and m is a method of c with parameters p1 , . . . pn according to the method definition in CD. We refer to o in this context as the target object of m. A position in the game consists of a state in the UML system design and depending on the callstack it is either – a state in the UML system design with an empty callstack: Refuter to move. – a state in the UML system design with a non-empty callstack: Verifier to move. As initial state S0 for our example we choose – O = {c:Customer, t:TravelAgent, h:Hotel, f:Flight} and Links0 = {(c, ta, t), (t, ε, c)}. For simplicity the choice of objects is not changed during the game, i.e. we do not consider creation and deletion of objects. – state(c) =no booking and state(t) = state(h) = state(f) = default – cs is the empty stack. Refuter challenges by invoking a method that has public visibility on any of the currently available objects. The choice of the method and its parameters has to conform to the specification by the class diagram, i.e. the method has to be defined in the class of the target object and the parameters have to be accessible and of suitable type. Furthermore the object has to be in a state where an invocation is possible. A challenge is independent of the current linking of objects and the states of t, h, f never change, so table 1 only shows the mapping of state for c. A challenge by Refuter is pushed on the callstack. Since the callstack is then non-empty that means Verifier has to make the next move. Verifier can respond in two different ways:
Games for UML Software Design
479
Table 1. Challenges by Refuter. State any state
Challenge t.findFlight(d,sd,ed) t.findHotel(l,sd,ed,s)
any state where c.bookHoliday(l) state(c) = no booking any state where c.changeHotel(s) state(c) = hotel and flight
– pick a transition in the state machine for the class of the target object of the method on top of the callstack. The call on top of the stack is popped and the state of the target object is updated according to the state machine. The response may also have an effect on how the objects are linked with each other. – call n other (possibly non-public) methods on objects that are reachable from the target object of the method call on top of the stack. The first new method call is pushed on the callstack. We assume that Verifier’s responses are at some point of the first kind, which leads to the call being popped from the callstack. After that Verifier pushes the next method call which is specified in his/her response onto the callstack and again we assume that it is at some point removed from the callstack. This procedure continues until all n new methods calls have been one by one pushed onto and then later popped from the callstack. Finally, after the response has been completed, the call on top of the stack, which is the one that caused the response, is popped. Notice that in general Verifier and Refuter do not take alternate turns. Verifier responds to the method invocation that is on top of the callstack which can either come from Refuter or Verifier. For our example we could have responses as shown in table 2. Of particular interest are the responses to bookHoliday and changeHotel which are of the more complicated kind explained above. Notice that by the choice of parameters for the methods some properties are fixed. The last parameter “3” of findHotel in the response to c.bookHoliday, for instance, has the effect that h is always set to a 3-star-hotel. Furthermore the usage of nearestAirport in findFlight within the same response ensures that the flight destination fits well with the chosen hotel. The specification of responses can be regarded as a strategy for meeting Refuter’s challenges. Since the sequence of method calls in the response is pushed on the stack one by one and Verifier has to memorise which ones s/he has already handled, it is a strategy with a history. Tool support for the creation of a game as described here should allow the user to manipulate and refine the strategy for Verifier in a comfortable way. For each system state the tool could display all reachable objects and let the user pick a target object. The user could then proceed by selecting one of the object’s methods, which are again displayed by
480
P. Stevens and J. Tenzer Table 2. Responses by Verifier.
State any state
Top of callstack Response t.findFlight(d,sd,ed) pick loop transition findFlight in state machine for TravelAgent any state t.findHotel(l,sd,ed,s) pick transition findHotel in state machine for TravelAgent h = c.ta.findHotel(l,c.startDate, c.endDate, 3); any state where f = c.ta.findFlight(h.nearestAirport, c.bookHoliday(l) state(c) = no booking c.startDate,c.endDate); c.setFlight(f); c.setHotel(h) c.hotel = any state where c.ta.findHotel(c.hotel.location, c.changeHotel(s) state(c) = hotel and flight c.startDate,c.endDate,s); pick transition setFlight in state maany state where chine for Customer whose source state(c) = no booking or c.setFlight(f) is state(c) and add (c,flight,f) and state(c) = hotel only (f, ε, c) to links pick transition setHotel in state maany state where chine for Customer whose source state(c) = no booking or c.setHotel(h) is state(c) and add (c,hotel,h) and state(c) = flight only (h,ε,c) to links
the tool, and stepwise create a valid response. The tool could also contain a functionality to record the chosen strategy in a diagram, such as for example a method state machine (see [13]). In order to complete our definition of a software design game we have to declare the winning conditions. We assume that a game is won by one player if the other one cannot make a move and that all infinite plays are won by Verifier. We can now finally play a game, starting with our initial system state S0 . An extract of a play is shown in table 3. The play will be won by Verifier because it is infinite: after the first two challenges Refuter can still continue to make challenges, but Verifier can always respond by simply picking a loop transition. The table does not record the full system state but only the parts that are relevant for this example (callstack, links, state of c) and the moves of the players. The parameters with which a method is called are sometimes left out in the callstack to save space and because they are specified in the preceding move. Moreover Refuter is abbreviated by R and Verifier by V.
7
Example: Incrementing a Software Design Game
There are several ways in which the example game from the previous section could be incremented. One way is to permit Refuter to call a public method from additional states, for instance we could permit a challenge by changeHotel when the object is in state hotel only. Another possibility is to add a completely
Links {(c, ta, t), (t, , c)} {(c, ta, t), (t, , c)} {(c, ta, t), (t, , c)} {(c, ta, t), (t, , c)} {(c, ta, t), (t, , c)} {(c, ta, t), (t, , c)} {(c, ta, t), (t, , c)}
Move R: c.bookHoliday(l) V: h=c.ta.findHotel(l,c.startDate,c.endDate,3) V: pick loop transition findHotel V: f=c.ta.findFlight(h.nearestAirport,c.startDate,c.endDate) V: pick loop transition findFlight V: c.setFlight(f) V: pick transition setFlight from no booking to flight only
c.bookHoliday(l) V: c.setHotel(h)
{(c, ta, t), (t, , c), (c, flight, f), (f, , c)} c.setHotel(h) {(c, ta, t), (t, , c), V: pick transition setHotel from flight only to hotel and flight c.bookHoliday(l) (c, flight, f), (f, , c)} {(c, ta, t), (t, , c), (c, flight, f), (f, , c), c.bookHoliday(l) response to bookHoliday completed (c, hotel, h), (h, , c)} {(c, ta, t), (t, , c), (c, flight, f), (f, , c), empty R: c.changeHotel(4) (c, hotel, h), (h, , c)} {(c, ta, t), (t, , c), c.changeHotel(4) V: c.hotel=c.ta.findHotel(c.hotel.location,c.startDate,c.endDate,4) (c, flight, f), (f, , c), (c, hotel, h), (h, , c)} {(c, ta, t), (t, , c), t.findHotel(...) (c, flight, f), (f, , c), V: pick loop transition for findHotel c.changeHotel(4) (c, hotel, h), (h, , c)} {(c, ta, t), (t, , c), (c, flight, f), (f, , c), c.changeHotel(4) response to changeHotel completed (c, hotel, h), (h, , c)} {(c, ta, t), (t, , c), (c, flight, f), (f, , c), empty ... (c, hotel, h), (h, , c)}
Callstack empty c.bookHoliday(l) t.findHotel(...) c.bookHoliday(l) c.bookHoliday(l) t.findFlight(...) c.bookHoliday(l) c.bookHoliday(l) c.setFlight(f) c.bookHoliday(l)
Table 3. Example play.
hotel and flight
hotel and flight
hotel and flight
hotel and flight
hotel and flight
hotel and flight
flight only
flight only
no booking
no booking
no booking
no booking
no booking
State of c no booking no booking
Games for UML Software Design 481
new public method to the class diagram, such as for example a method for the cancellation of a holiday booking. Notice that this would be both an incrementation of the system design and its specification at the same time. That is caused by our decision that challenges are method calls and to require methods to be picked in accordance with the class diagram. The game could be further incre-
482
P. Stevens and J. Tenzer
mented by adding states and transitions to the protocol state machines which offer new possibilities of responding to the Verifier. As soon as we want to increment the game in a more sophisticated manner it becomes clear that working with abstract states is often not enough. For our example we could require that c.hotel.nearestAirport always equals c.flight.destination when c is in state hotel and flight. In order to express this as an additional winning condition which makes it more difficult for Verifier to win the game, attribute values have to be part of the system state. A more detailed object state also leads to increased expressiveness in state machines since we could for instance specify guards whose evaluation depends on the current attribute values. However, introducing a concrete object state has the disadvantage that we in general have to handle an infinite state space.
8
Discussion and Future Work
This paper has described very early steps in a programme of work. Much remains to be done. Most obviously, we are developing tools to support the use of these techniques in practice. In an academic environment these tools will necessarily be limited prototypes, but the experience of building them should help in our explorations and dissemination of the ideas. One possible objection to the proposal is that it is not obvious that there is a specification of the system separate from its design. If the specification is incorporated into the game definition, which also incorporates a particular design, does this not lose clarity and prevent a separate evaluation of the specification? We can answer this objection in a variety of ways. One answer would be that given appropriate prototype tool support we could investigate what games people build in practice, and see whether there is in fact a clear distinction between design and specification. For example, the challenges in our examples so far can be seen to be separable and could be expressed as a separate specification. Is this typical, or in real examples would the design and specification be more intertwined? Another answer would be to say that it is not important to have a separate specification at the design level. A specification that is independent of design is a realistic aim at a high level, where the user requirements are being expressed in the user’s vocabulary. Inevitably, though, the verification of a design is done against a more detailed, technical specification that always incorporates many assumptions about the design, even when it is presented as a separate document. In this paper we have considered incrementing a game as a process outside the game framework. For software design games this process corresponds to incremental software development processes used in practice. Alternatively game improvement could itself be regarded as a game. One way of defining such an incrementation game is to couple it closely to verification. If Refuter wins a play of a verification game, this play can be used as a challenge in the incrementation game. Verifier has to respond by improving
Games for UML Software Design
483
the game in a suitable way. In order to prove that her/his modifications are a valid response, Verifier has to play the (incremented) verification game again with the same challenges by Refuter as in the play that induced the move in the incrementation game. A different way of defining Refuter’s challenges in an incrementation game is to regard them as independent proposals for system changes which are not related to plays of verification games. In this more general version incrementation games can for instance be used to explore and verify the evolvability of a system. In this case the incrementation is hypothetical and serves to show that a game can be extended as desired without breaking functionality that was present in the initial design. However it is not yet clear how incrementation games could be used in practice. Though they might be a suitable way of thinking it is not obvious if and how playing this kind of games can be reasonably supported by tools.
9
Related Work
There is, of course, a large amount of work on applying verification techniques to UML models. We do not attempt a representative discussion of this crowded field; since our focus here is on highly interactive tool support for design using games as a basis, we will instead discuss the related games-based work. Two-player games of the kind we consider here have a long history in mathematics (see for example [6]) and informatics (see for example [3,12]). In controller synthesis for open systems two-player games where one player (control) has a strategy which enforces the system to behave according to a given specification independent of what the other player (environment) does are of interest. Finding a winning strategy (the controller) for player control is known as the control problem. The control problem has been studied for different kinds of open systems such as, for example, discrete systems [9] and systems in reactive environments [7]. The system specification can be given as linear or branching time temporal logic formula. Results on the complexity of solving the control problem depend on the chosen logic and the kind of system that is investigated. In case that a controller exists for a given combination of system and specification it is also of relevance whether the controller is finite and how big it is [9]. Games in this context are often infinite and there are some classes of specifications which are of particular interest. An example for such a kind of specification or “game objective” is that eventually an element of a given target set of system states has to be reached. Some frequently occurring game objectives, corresponding winning strategies and complexity results are presented in [14]. The relation between games in control and verification is pointed out in [1], where a translation of a game objective from control into a fixpoint formula written in the mu-calculus such as used in verification, is defined. A closely related area is that of system synthesis. Given a specification, the task of constructing a system that satisfies it is known as the synthesis problem. It can be formulated in terms of a game between environment and system, and a
484
P. Stevens and J. Tenzer
winning strategy for such a game represents the desired system. If such a strategy exists we say that the given specification is realisable. Like the control problem system synthesis and realisability have been examined for different kinds of logics and systems, such as for systems in reactive environments [7] and distributed systems [8]. Another kind of two-player games called combinatorial games is used to represent, analyse and solve interesting problems in areas such as complexity, logic, graph theory and algorithms [2]. The players move alternately and cannot hide information from each other. The game ends with a tie or the win of one player and lose of the other. An example for a combinatorial game with a lot of literature within the area of artificial intelligence is chess. A different, and more complex style of game appears in economics. Here, rather than simply winning or losing, a player receives a variable payoff. A game is played between two or more players with possibly conflicting interests, i.e. a move which leads to a better payoff for one player may have a negative effect on another player’s payoff. Thus a strategy which optimises one player’s payoff can depend on the strategies followed by other players. The issue of payoff optimisation can be considered under different circumstances: the players can have complete, partial or zero knowledge of each others moves. It is also possible for players to collaborate in order to gain a better payoff for a group of players rather than for the individual player. These kinds of games reflect situations in economics such as, for example, competition between different companies, and were first introduced within this context in [15]. Since then a large amount of further work has been published in this area. It would be interesting to explore the applicability of this kind of game to software design, but for now we prefer the simpler games described here. The work of Harel et. al. on “play-in play-out scenarios” [4], [5] has a similar flavour to our work, and is motivated by similar concerns about the interactivity of tools to support design. Play-in scenarios allow the capture of requirements in a user-friendly way. The user specifies what behaviour s/he expects of a system by operating the system’s graphic user interface (GUI) – or an abstract version thereof – which does not have any behaviour or implementation assigned to it yet. A tool which is called the play-engine transforms the play-in of the user into live sequence charts (LSCs), which are used as formal requirements language. The user does not have to prepare or modify the LSCs directly but only interacts with the GUI. LSCs are a powerful extension of message sequence charts (MSCs). In contrast to sequence diagrams – the variant of MSCs which is part of UML, and which is implicitly existential – they can be either existential or universal. A universal LSC defines restrictions that have to hold over all system runs, while an existential LSC represents a sample interaction which has to be realised by at least one system run. Using play-out scenarios we can verify whether a set of LSCs – created by play-in scenarios or in any other way – meets the system requirements. Thereby the user feeds the GUI with external environment actions rather as though s/he
Games for UML Software Design
485
were testing the final system. For each user input the tool computes the response of the system on the basis of the LSCs in terms of a sequence of events which are carried out. The system response is called a superstep and it is correct if no universal LSC is violated during its execution. The task of finding the desired superstep can be formulated as a verification problem. In [5] a translation of LSCs into transition systems which allows the usage of model checking tools for the computation of the supersteps is given. Similarly model checking can provide the answer to the question whether an existential LSC can be satisfied. This approach differs form ours in that its focus is on capturing and testing the requirements while we are mainly interested in helping the user to design a system. Thus play-in play-out scenarios do not aim to help in defining intraobject behaviour, as our games do, but remain on the higher level of interaction between objects and environment. Since our work concentrates on UML we use the diagram types provided by it, i.e. UML sequence diagrams instead of the more expressive LSCs.
10
Conclusion
We have suggested the use of games as a driving metaphor for highly interactive design tools to support designers working in UML. We have proposed that the user of such a tool should define incrementally a game which captures not only the evolving state of the system design but also the properties that the design should satisfy; the tool should support the user both in informal explorations of the resulting game at each stage, and in verification, that is, in the finding of a winning strategy. We have given simple examples of how a design tool based on this idea might operate. We hope that eventually this work may contribute to allowing mainstream business software developers to take advantage of verification technology without giving up their existing incremental development practices.
Acknowledgements We are particularly grateful to a reviewer of this paper for useful comments on incrementation games as mentioned in the discussion, and to the British Engineering and Physical Sciences Research Council for funding (GR/N13999/01, GR/A01756/01).
References 1. Luca de Alfaro, Thomas A. Henzinger, and Rupak Majumdar. From verification to control: Dynamic programs for omega-regular objectives. In Proceedings of the 16th Annual Symposium on Logic in Computer Science (LICS), pages 279–290. IEEE Computer Society Press, 2001.
486
P. Stevens and J. Tenzer
2. A.S. Fraenkel. Selected bibliography on combinatorial games and some related material. The Electronic Journal of Combinatorics, (DS2), 2002. Available from http://www.combinatorics.org/Surveys/ds2.ps. 3. E. Gr¨ adel. Model checking games. In Proceedings of WOLLIC 02, volume 67 of Electronic Notes in Theoretical Computer Science. Elsevier, 2002. 4. D. Harel. From play-in scenarios to code: An achievable dream. IEEE Computer, 34(1):53–60, January 2001. 5. D. Harel, H. Kugler, R. Marelly, and A. Pnueli. Smart play-out of behavioral requirements. In Proceedings of the 4th International Conference on Formal Methods in Computer-Aided Design (FMCAD 2002), pages 378–398, November 2002. 6. W. Hodges. Model theory, volume 42 of Encyclopedia of Mathematics. Cambridge University Press, Cambridge, 1993. 7. O. Kupferman, P. Madhusudan, P.S. Thiagarajan, and M.Y. Vardi. Open systems in reactive enfironments: control and synthesis. In Catuscia Palamidessi, editor, Proceedings of the 11th International Conference on Concurrency Theory (CONCUR 2000), volume 1877 of Lecture Notes in Computer Science, pages 92–107. Springer, August 2000. 8. O. Kupferman and M.Y. Vardi. Synthesising distributed systems. In Proceedings of the 16th Annual IEEE Symposium on Logic in Computer Science (LICS 2001). IEEE Computer Society, June 2001. 9. P. Madhusudan and P.S. Thiagarajan. Branching time controllers for discrete event systems. Theoretical Computer Science, 274(1-2):117–149, 2002. 10. OMG. Unified Modeling Language Specification version 1.4, September 2001. OMG document formal/01-09-67 available from http://www.omg.org/technology/documents/formal/uml.htm. 11. Perdita Stevens. Abstract interpretations of games. In Proc. 2nd International Workshop on Verification, Model Checking and Abstract Interpretation, VMCAI’98, number CS98-12 in Venezia TR, 1998. 12. Colin Stirling. Model checking and other games. Notes for Mathfit Workshop on finite model theory, University of Wales, Swansea, July 1996. 13. Jennifer Tenzer and Perdita Stevens. Modelling recursive calls with UML state diagrams. In Proc. Fundamental Approaches to Software Engineering, number 2621 in LNCS, pages 135–149. Springer-Verlag, April 2003. 14. W. Thomas. On the synthesis of strategies in infinite games. In E.W. Mayr and C. Puech, editors, Proceedings of the 12th Annual Symposium on Theoretical Aspects of Computer Science, STACS ’95, volume 900 of Lecture Notes in Computer Science, pages 1–13, Berlin, 1995. Springer. 15. J. von Neumann and O. Morgenstern. Theory of Games and Economic Behavior. Princeton University Press, Princeton, third edition, 1953.
Making Components Move: A Separation of Concerns Approach Dirk Pattinson and Martin Wirsing Institut f¨ur Informatik, LMU M¨unchen
Abstract. We present a new calculus for mobile systems, the main feature of which is the separation between dynamic and topological aspects of distributed computations. Our calculus realises the following basic assumptions: (1) every computation executes in a uniquely determined location (2) processes modify the distributed structure by means of predefined operations, and (3) the underlying programming language can be changed easily. This paper introduces our calculus, and shows, that this separation of concerns leads to a perfect match between the logical, syntactical and algebraic theory. On the methodological side, we demonstrate by means of two examples, that the strict distinction between topological and computational aspects allows for an easy integration of features, which are missing in other calculi.
1 Introduction With the success of the Internet, mobile systems have been promoted as new computational paradigm in which computation can be distributed over the net and highly dynamic, with the network itself changing continuously. In practice, however, such systems are not well accepted since users fear security problems, or more generally, the problems with controlling the behaviour of mobile systems. As a remedy, process calculi, modal logics and other formal techniques have been proposed and studied which provide theoretical foundations for mobile systems and allow one to analyse and verify properties of such systems. The most well-known example is the π-calculus [8] of Milner which provides an abstract basis for mobility where communicating systems can dynamically change the topology of the channels. The Ambient calculus [5] of Cardelli and Gordon focuses on the handling of administrative domains where mobile processes may enter a domain or exit from a domain and in this way may change the topology of the network. Similarly, the Seal calculus [17] of Vitek and Castagna aims at describing secure mobile computations in a network that is hierarchically partitioned by localities. In this paper we continue this line of research by proposing a new basic calculus for mobile processes called BasicSail with focus on explicit localities and dynamic reconfiguration of networks. A configuration is a hierarchy of administrative domains, each of which is controlled by a process and which may contain other subconfigurations. Configurations may be dynamically reconfigured by entering another configuration or by exiting from a configuration. This is similar to the Ambient calculus; in contrast
This work has been partially sponsored by the project AGILE, IST-2001-39029.
F.S. de Boer et al. (Eds.): FMCO 2002, LNCS 2852, pp. 487–507, 2003. c Springer-Verlag Berlin Heidelberg 2003
488
D. Pattinson and M. Wirsing
to other approaches we aim at a clear separation between processes and configurations: processes show behaviour, whereas the configurations provide the topological structure. BasicSail abstracts from a concrete process calculus: We aim at studying the dynamic reconfiguration of configurations independently of the underlying notion of process. Our approach is centred around three assumptions, which we now briefly discuss: Assumption 1. Every computation takes place in a uniquely determined location. This assumption in particular forces a two-sorted approach: We need to distinguish between elements which relate to the spatial structure and those, which drive the computation process. Since our primary interest is the study of mobile computation, we would like to be as independent as possible from the concrete realisation of processes, and therefore make Assumption 2. The distributed part of the calculus is independent of the underlying programming language or process calculus. However, a computation needs some means to change the distributed and spatial structure (otherwise our study would end here). That is, we need a clean mechanism, through which the distributed structure can be modified : Assumption 3. Processes modify the distributed structure of the computation through interfaces only. Our calculus is modelled after these assumptions. Regarding independence of the underlying programming language, we assume that the processes, which control the computations, already come with a (fixed) operational semantics, in terms of a labelled transition system; this allows us to realise interfaces as a particular set of distinguished labels. As already mentioned before, the separation between processes and locations is taken care of by using a two sorted approach. The main technical contribution of the paper is the study of the algebraic and logical properties of the basic calculus, and of its extension with local names. We introduce the notion of spatial bisimulation and give an algebraic and a logical characterisation of the induced congruence. Our main result here is, that if one abstracts from the concrete realisation of the computations, we obtain a perfect match between structural congruence, logical equivalence and spatial congruence. Methodologically, we want to advocate the separation between the concepts “mobility” and “computation” on a foundational basis; we try to make this point by giving two extensions of the calculus, which are missing in other calculi and can be smoothly integrated into BasicSail, thanks to the separation between spatial structure and computation. We introduce the basic calculus, that is, the calculus without local names, in Section 2. The algebraic theory of he calculus is investigated in Section 3, and Section 4 transfers these results to a logical setting. We then extend the calculus with local names (Section 5). Further extensions, which demonstrate the versatility of our approach, are discussed in Section 6. Finally, Section 7 compares our approach to other calculi found in the literature.
Making Components Move: A Separation of Concerns Approach
489
2 Basic Sail: The Basic Calculus This section introduces BasicSail, our testbed for studying mobile components. In order to ensure independence from the underlying programming language (cf. Assumption 1), BasicSail consists of two layers. The lower layer (which we assume as given) represents the programming language, which is used on the component level. The upper level represents the distributed structure, which is manipulated through programs (residing on the lower level) by means of pre-defined interfaces. Technically, we assume that the underlying programming language comes with a labelled transition system semantics, which manipulates the distributed structure (on the upper level) by means of a set of distinguished labels. The distinction between processes (or programs) and the locations, in which they execute (and the structure of which they modify), forces us to work in a two-sorted environment, where we assume the programs (and their operational semantics) as given, and concentrate on the distributed structure. Our basic setup is as follows: Notation 1. Throughout the paper, we fix a set N of names and the set L = {in , out , open } × N of labels and a transition system (P, −→), where P is a set (of processes) and −→⊆ P × L × P. We assume that (P, −→) is image finite, that is, l
for every (P, l) ∈ P × L, the set {P | P −→ P } is finite. We write in n for the pair (in , n) ∈ L and similarly for out , open and call the elements of L basic labels. The set P is the set of basic processes. The prototypical example of transition systems, which can be used to instantiate our framework, are of course process calculi. We present one such calculus, which will also be used in later examples, next. Example 1. Take P to be given as the least set according to the following grammar: P P, Q ::= 0 | P Q | α.P |!P where α ∈ L ranges over the basic labels. The transition relation −→ is generated by the following rules α P −→ P , α α α.P −→ P P Q −→ P Q modulo structural congruence ≡, given by the axioms P Q ≡ Q P , P 0 ≡ P , P (Q R) ≡ (P Q) R and !P ≡ P !P . For convenience, we often omit the trailing inert process and write α for α.0. Intuitively, α.P is a process which can perform an α action and continue as P ; the term P Q represents the processes P and Q running concurrently and !P represents a countable number of copies of P . Note that we use this concrete syntax for processes just in order to illustrate our approach; the general theory is independent of the syntactical presentation and just assumes that processes form a set and come with a transition system over the set L of labels. Given such a transition system (P, −→), the distributed structure (which is our primary interest) is built on top of (P, −→) as follows:
490
D. Pattinson and M. Wirsing
Definition 1. The set C of basic configurations is the least set according to the grammar C A, B ::= 0 | nP [A] | A, B where P ∈ P is a process and n ∈ N is a name, modulo structural congruence ≡, given by the equations A, B ≡ B, A A, 0 ≡ A A, (B, C) ≡ (A, B), C We call the configuration building operator “,” spatial composition. Here, 0 is the empty configuration, nP [A] is a configuration with name n, which is controlled by the process P and has the subconfiguration A. Finally, A, B are two configurations, which execute concurrently. The next definition lays down the formal semantics of our calculus, which is given in terms of the reduction semantics −→ of the underlying process calculus: Definition 2. The operational semantics of BasicSail is the relation given by the following rules the following rules in n
P −→ P mP [A],nQ[B] ⇒nQ[mP [A],B] out n
P −→ P nQ[mP [A],B] ⇒ mP [A],nQ[B] open n
P −→ P mP [A], nQ[B] ⇒ mP [A], B together with the congruence rules A =⇒ A A, B =⇒ A , B A =⇒ A nP [A] =⇒ nP [A ] where we do not distinguish between structurally congruent configurations. The relation =⇒ is called spatial reduction. In the examples, we often omit the empty configuration, and write nP [] instead of nP [0]. Using the above definition, we can study phenomena, which arise in a distributed setting, without making a commitment to any kind of underlying language. In particular, we do not have to take internal actions of processes into account; these are assumed to be incorporated into the reduction relation −→ on the level of processes. We cannot expect to be able to embed the full ambient calculus [5] into our setting, due to the fact that in the (original) ambient calculus, there are no sorts available. However, we can nevertheless treat many examples:
Making Components Move: A Separation of Concerns Approach
491
Example 2. We use the set of basic processes from Example 1. 1. An agent, which has the capability to enter and exit its home location to transport clients inside can be modelled as follows: Put agent = aP [] client = cQ[] home = h0[agent] where P =!(out h.in h.0) and Q = in a.out a.0. In the configuration home, client, we have the following chain of reductions (where P = in h.0 P and Q = out a.0): home, client =⇒ h0[], aP [], cQ[] =⇒ h0[], aP [cQ []] =⇒ h0[aP [cQ []] =⇒ h0[aP [], c0[]]. This sequence of reductions shows a guarded form of entry into h: The client has to enter the mediating agent a, which then transports it into h, where the client then exits. Note that in the basic calculus, c could enter h directly, if c’s controlling process were different. This can be made impossible if one adds local names, as we shall do later. 2. We model an agent, which repeatedly visits two network nodes, as follows: agent ≡ aP [] with P =!(in n1 .out n1 .0) !(in n2 .out n2 .0). The activity of a once it is at either n1 or n2 is not modelled (but imagine a checks, whether a node has been corrupted or is otherwise non-functional). In the presence of two nodes n1 and n2 , we have the (spatial) reductions, where we write N1 and N2 for the controlling processes of n1 and n2 : n1 N1 [], n2 N2 [], aP [] =⇒n1 N1 [aP1 []], n2 N2 [] =⇒n1 N1 [], n2 N2 [], aP [] =⇒n1 N1 [], n2 N2 [aP2 []] =⇒ . . . In the above, we have abbreviated P1 = out n1 .0 P and P2 = out n2 .0 P . Here, the program P controlling a does not force a to visit n1 and n2 in any particular order, and a could for example choose to enter and leave n1 continuously, without ever setting foot into n2 .
3 Algebraic Theory of the Basic Calculus This section is devoted to the algebraic theory of the basic calculus; extensions of the calculus, in particular with local names, are deferred until Section 5. In this section, we
492
D. Pattinson and M. Wirsing
show that the algebraic and the logical theory of the basic calculus fit together seamlessly. In more detail, we discuss the relationship between three relations on processes: spatial bisimulation (which we introduce shortly), the induced spatial congruence and structural congruence. 3.1 Basic Definitions and Examples Spatial bisimulation will defined as binary relation on configurations, subject to some closure properties; the precise meaning of which is given as follows: Terminology 2. Suppose R ⊆ A × A is a binary relation on a set A and S ⊆ A × · · · × A is n + 1-ary. We say that R is closed under S, if, whenever (a, b) ∈ R and (a, a1 , . . . , an ) ∈ S, there are b1 , . . . , bn ∈ A with (b, b1 , . . . , bn ) ∈ S and (ai , bi ) ∈ R for i = 1, . . . , n. If R is closed under S, it is often helpful to think of R as an equivalence on processes and of S as a reduction relation. In this setting, R is closed under S if, whenever a and b are equivalent (i.e. (a, b) ∈ R) and a reduces to a (i.e. (a, a ) ∈ S), there is some b such that a and b are again equivalent (i.e. (a , b ) ∈ R) and b reduces to b (that is, (b, b ) ∈ R). So if R is closed under S, we think of R as being some bisimulation relation and R the corresponding notion of reduction. Definition 3 (Spatial Bisimulation). Consider the following endorelations on C: 1. Subtree reduction ↓⊆ C ×C, where C ↓ D iff C ≡ nP [D] for some n ∈ N and P ∈P 2. Forest reduction ⊆ C × C × C where C (A, B) iff C ≡ A, B and A is of the form A ≡ nP [D] for some n ∈ N , P ∈ P and D ∈ C. 3. Top-level names @n ⊆ C, where n ∈ N and C ∈ @n iff C ≡ nP [A] for some P ∈ P and A ∈ C. The largest relation ⊆ C × C, which is closed under spatial reduction =⇒, subtree reduction ↓, forest reduction and top-level names @n, for all n ∈ N , is called spatial bisimulation. Furthermore, spatial congruence ∼ = is the largest spatial bisimulation, which is a congruence with respect to construction of configurations. Note that, in the previous definition, we just require the congruence property wrt. the construction of configurations, that is we require 1. A0 ∼ = A1 , B0 ∼ = B1 =⇒ A0 , A1 ∼ = B0 , B1 and ∼ 2. A = B, n ∈ N , P ∈ P =⇒ nP [A] ∼ = nP [B]. This not only justifies the name spatial congruence – it furthermore allows us to study the evolution of the tree structure of (a set of) mobile processes without reference to the underlying process calculus. Note that the spatial congruence is not the largest congruence contained in the spatial bisimulation (corresponding to closure under contexts). Our notion of spatial congruence follows the approach of dynamic bisimulation [9].
Making Components Move: A Separation of Concerns Approach
493
In a nutshell, two configurations are spatially bisimilar, if they have bisimilar reducts, bisimilar subtrees, and the same top-level names. If two configurations are spatially congruent, one can furthermore substitute them for one another, obtaining spatially congruent processes. Although spatial bisimulation is a very strong notion of bisimilarity, it is not a congruence: Example 3. Take n, m ∈ N with n = m and let A ≡ nin m.0[] and B ≡ n0[]. Then A B (since neither A nor B can perform a spatial reduction), but A ∼
B, since = A, m0[] does reduce, whereas B, m0[] does not. Since we clearly want equivalent configurations to be substitutable for one another (which allows us to build large systems in a compositional way), spatial congruence is the notion of equivalence we are interested in. By definition, spatial congruence involves the closure under all configuration constructing operators, and is therefore not easy to verify. Our first goal is therefore an alternative characterisation of spatial congruence. As it turns out, we only need to add one closure property to the definition of spatial bisimulation in order to obtain spatial congruence. 3.2 Spatial Congruence and Spatial Bisimulation We start on our first characterisation of spatial congruence. The approach is as follows: We consider labelled reduction, introduced in the next definition, and show that (i) spatial congruence is closed under labelled reduction, and (ii) that spatial bisimulation + labelled reduction is a congruence. This immediately entails that spatial congruence is spatial bisimulation plus closure under labelled reductions. We begin with the definition of labelled reduction: l
Definition 4. Let l ∈ L. Define the relation =⇒⊆ C × C by the rules l
P −→ P l
nP [A] =⇒ nP [A] l
C =⇒ C l
C, D =⇒ C , D l
and call a relation B ⊆ C × C closed under labelled reduction, if B is closed under =⇒ for all l ∈ L. We use the name “labelled bisimulation” for the closure of spatial bisimulation under labelled reductions. Definition 5. We take labelled bisimulation to be the largest symmetric relation ⊆ C × C which is closed under forest reduction, spatial reduction, subtree reduction, labelled reduction and top level names.
494
D. Pattinson and M. Wirsing
In order to be able to compare spatial congruence and labelled bisimulation, we need a proof principle, which allows us to reason about labelled bisimulation using induction on reductions. This principle works for finitely branching systems only, and is the content of the following two lemmas: Lemma 1. Suppose (P, −→) is finitely branching. Then the relations =⇒, , ↓ and l
=⇒ (for all l ∈ L) are image finite. Proof. By structural induction using the respective definitions. Proposition 2. Assume that (P, −→) is image finite and define a sequence of relations ∼i ⊆ C × C inductively as follows: 1. ∼0 = C × C 2. C ∼i+1 D is the largest symmetric relation s.t. – C ∈ @n implies D ∈ @n – (C, C ) ∈ R implies ∃D .(D, D ) ∈ R and C ∼i D where R is one of =⇒ or ↓ – C (C1 C2 ) implies ∃D1 , D2 .D (D1 , D2 ) and C1 ∼i D1 , C2 ∼i D2 l l – C =⇒ C implies ∃D .D =⇒ D and C ∼i+1 D for l ∈ L Then, if C and D ∈ C, we have C D iff C ∼i D for all i ∈ N. Proof. We abbreviate ∼= i∈N ∼i . In order to see that C D whenever C ∼i D, one shows that ∼ is a spatial bisimulation, which is closed under labelled reduction. The converse follows from the fact that all relations used in the definition of ∼i are image finite (Lemma 1). We note two easy consequences of the above characterisation: in particular, controlling processes, which are bisimilar (in the ordinary sense) do not destroy the relations ∼i and therefore preserve labelled bisimulation. That is, if we call the largest symmetric relation B ⊆ P × P, which is a (strong) labelled bisimulation in the ordinary sense a process bisimulation, we have the following: Lemma 3. 1. ∼i+1 ⊆∼i for all i ∈ N. 2. Let n ∈ N , A, B ∈ C and P, Q ∈ P rop. Then for all i ∈ N nP [A] ∼i+1 nQ[B] iff P, Q are process-bisimilar and A ∼i B. The relationship between labelled bisimulation and process bisimulation can be formalised as follows: Corollary 4. Let n ∈ N , A, B ∈ C and P, Q ∈ P rop. Then nP [A] and nQ[B] are labelled bisimilar iff P, Q are process-bisimilar and A and B are labelled bisimilar. We are now ready to tackle the first step of our comparison between labelled bisimulation and spatial congruence.
Making Components Move: A Separation of Concerns Approach
495
Lemma 5. Spatial congruence is closed under labelled reduction. l
Proof. Suppose n ∈ N , C, D ∈ C are spatially congruent and C =⇒ C . Then C is of l the form C ≡ C0 , C1 with C0 ≡ mP [E] and P −→ P for some P ∈ P and E ∈ C. We proceed by case distinction on l ∈ L, where we use a fresh name k ∈ N , i.e. k does not occur as the name of a location either in C or in D, and some arbitrary R ∈ P. Case l = in n: Consider the context K[ ] = nR[kR[]], . Then K[C] =⇒ C with C ≡ C1 , nR[mP [E], kR[]]. Since C ∼ = D, we have K[D] =⇒ D with ∼ C = D . Since spatial congruence is closed under forest reduction and top-level names, we can split D ≡ D1 , nR [F ] for some R ∈ P and F ∈ C, where D1 ∼ = C1 and nR [F ] ∼ = nR[mP [E], kR[]]. Using closure under subtree reduction, we obtain F ∼ = mQ [E ], kR[] (since k is fresh) with mQ [E ] ∼ = mP [E]. Again using in n that k is fresh, we have D ≡ D1 , mQ[E ] for some Q ∈ P with Q −→ Q with D1 ∼ = C1 and mP [E] ∼ = mQ [E ]; since spatial congruence is a congruence we in n finally obtain D =⇒ D1 , mQ [E ] ∼ = C1 , mP [E]. Case l = out n: Similar, using the context nR[ , kR[]]. Case l = open n: Similar, using the context nR[kR[]], . The converse of Lemma 5 needs the proof principle of Proposition 2. Lemma 6. Labelled bisimulation is a congruence. Proof. We have to show that labelled bisimulation is a congruence wrt. the construction of configurations, that is, wrt. “putting in a box” and spatial composition. Congruence wrt. spatial composition: We show that the relation Ri = {(C, E), (D, E)|C, D, E ∈ C and C ∼i D} is a subset of ∼i for all i ∈ N. The case i = 0 is trivial; for the inductive step we show that any pair ((C, E), (D, E)) ∈ Ri+1 satisfies the properties defined in Prop. 2 with ∼i+1 replaced by Ri . The cases of top level names, forest reductions and labelled reductions follow directly from the definitions of the Ri+1 and the fact that ∼i+1 ⊆∼i . For spatial reduction suppose C, E =⇒ C . If either C =⇒ C0 and C ≡ C0 , E or E =⇒ E0 and C ≡ C, E0 the result follows easily from the induction hypothesis. For all other cases we have to show that C, E and D, E have the same spatial reductions, resulting in configurations, which are ∼i equivalent. We only consider the in -rule; the other cases are similar. If C, E =⇒ C by virtue of the in -rule, either a component of C enters into a component of E, or vice versa. That is, we have one of the following two cases: in n
1. C ≡ C0 , C1 with C0 ≡ mP [F ] and P −→ P and E ≡ E0 , E1 with E0 ≡ nQ[G], or in n 2. E ≡ E0 , E1 with E0 ≡ mP [F ] and P −→ P and C ≡ C0 , C1 with C0 ≡ nQ[G]. We only treat the first case; the second can be treated along similar lines (using Lemma 3). From the assumption C ∼i+1 D we obtain (using forest reduction and preservation of top level names), that we can split D ≡ D0 , D1 with
496
D. Pattinson and M. Wirsing
D0 ≡ mR[H] and Cj ∼i Dj for j = 0, 1. Using closure under labelled rein n duction, we have R −→ R with mP [F ] ∼i mR [H]. Since C, E =⇒ C we obtain C ≡ nQ[mP [F ], G], C1 , E1 and D, E =⇒ D with D ≡ nQ[mR [H], G], D1 , E1 , from which we obtain C ∼i D using that ∼i is a congruence. Congruence wrt. putting in a box: Suppose C, D ∈ C with C ∼i+1 D and n ∈ N , P ∈ P. We have to show that nP [C] i+1 nP [D]. As before, the only interesting cases arise through spatial reductions. So suppose nP [C] =⇒ C . If this is because C =⇒ C and C ≡ nP [C ], we find D ∼i C with D =⇒ D , since C ∼i+1 D. In this case nP [D] =⇒ D with D ≡ nP [D ] and by ind. hyp. C ∼i D . Now assume nP [C] =⇒ C using the out -rule. That is C ≡ C0 , C1 with C1 of out n out n the form C1 ≡ mQ[E] and Q −→ Q . With C0 ≡ mQ [E] we thus have C0 =⇒ C0 . Using forest reduction, we can split D ≡ D0 , D1 with Dj ∼i Cj for j = 0, 1. In out n particular, D0 =⇒ D0 and D0 ∼i C0 . By assumption, we have C ≡ C0 , nP [C1 ]. Putting D ≡ D0 , nP [D1 ], we obtain nP [D] =⇒ D and D ∼i C . From the previous lemma, we obtain the desired characterisation of spatial congruence: Corollary 7. Spatial congruence and labelled bisimulation coincide. Proof. By Lemma 5, spatial congruence is contained in spatial bisimulation. Lemma 6 proves the other inclusion. This result is our first characterisation of spatial congruence in the basic calculus. Spatial congruence allows us to observe the dynamic behaviour of controlling processes plus the tree structure of configurations. One therefore suspects, that spatial congruence is a very intensional notion of equivalence. In the following, we show that spatial congruence is very intensional indeed, by comparing it to the relation of structural congruence on configurations. 3.3 Spatial Congruence vs Structural Congruence Depending on the underlying labelled transition system (P, −→), which controls the behaviour of processes (which in turn control the evolution of configurations), it is clear that structural congruence is strictly contained in spatial congruence: If P, Q ∈ P are bisimilar but not identical, we have that nP [] and nQ[] are not structurally congruent, but spatially congruent. This example relies on the existence of equivalent, but non-identical processes in P. In this section, we show, that this is indeed the only possible way in which we can have configurations, which are spatially congruent, but not structurally congruent. We now proceed to show that spatial congruence coincides with structural congruence modulo process bisimilarity. We start with the following: Definition 6. Weak structural congruence is the least relation R generated by the rules of Definition 1, plus the rule C≡D P, Q process bisimilar nP [A] ≡ nQ[B] where n ∈ N , A, B ∈ C and P, Q ∈ P.
Making Components Move: A Separation of Concerns Approach
497
Thus weak structural congruence not only identifies structurally congruent configurations, but also configurations with bisimilar controlling processes. We think of weak structural congruence as structural congruence up to process bisimilarity. Note that – coming back to the example at the beginning of the section – that nP [A] and nQ[A] are weakly congruent for P, Q process bisimilar. We have argued that this is an example of a pair of configurations, which are spatially congruent, but not structurally congruent. Extending structural congruence to include those configurations, which only differ in the controlling process, structural and spatial congruence can be shown to coincide: Proposition 8. Weak structural congruence and spatial congruence coincide. Proof. It follows directly from the definitions that weak structural congruence (which we denote by ≡ for the purpose of this proof) is contained in spatial congruence. We prove the converse inclusion by contradiction: assume that the set F = {(C, D) ∈ C | C∼ = D, C ≡ D} of felons is non empty. For C ∈ C, we define the height of C, ht(C), by induction as follows: ht(0) = 0, ht(C, D) = ht(C) + ht(D), ht(nP [C ]) = 1 + ht(C ). Since the standard ordering on natural numbers is a well-ordering, there is a pair (C, D) of felons, such that ht(C) is minimal, that is, for all (C , D ) ∈ F we have ht(C ) ≥ ht(C). We discuss the different possibilities for C. Case C ≡ C0 , C1 with C0 ≡ 0 ≡ C1 : Using forest reduction, we can split D ≡ D0 , D1 with Dj ∼ = Cj for j = 0, 1. Since ht(C0 ) < ht(C) and ht(C1 ) < ht(C), neither (C0 , D0 ) nor (C1 , D1 ) are felons, that is, C0 ≡ D0 and C1 ≡ D1 , hence C ≡ C0 , C1 ≡ D0 , D1 ≡ D, contradicting (C, D) ∈ F . Case C ≡ nP [C0 ]: By subtree reduction, D ≡ mQ[D0 ] with C0 ∼ = D0 . Since ht(C0 ) < ht(C), the pair (C0 , D0 ) is not a felon, hence C0 ≡ D0 . By closure under top-level names, furthermore n = m, and closure under labelled reduction implies that P and Q are process bisimilar. Hence nP [C0 ] and mQ[D0 ] are weakly congruent, contradicting (C, D) ∈ F . Case C ≡ 0: From C ∼ = D we conclude D ≡ 0, contradicting C ≡ D. This concludes our investigation of the algebraic properties of BasicSail, which we summarise as follows: Theorem 9. Suppose C, D ∈ C. The following are equivalent: 1. C and D are spatially congruent 2. C and D are labelled bisimilar 3. C and D are weakly structurally congruent
4 The Logical Theory of BasicSail In the previous section, we have looked at spatial congruence from an algebraic viewpoint and have given three different characterisations. This section adopts a logical view and gives a further characterisation of spatial bisimulation in terms of a (modal style) logic. Using our setup from the previous section, this task is not overly difficult, we
498
D. Pattinson and M. Wirsing
just have to make the (standard) assumption that the underlying processes are finitely branching. Making this assumption, we obtain a logic, which is completely standard except for one binary modal operator, which plays a role similar to the linear implication used in [4, 2], except for the fact that linear implication in loc. cit. is the logical version of parallel composition, whereas the modal operator we are about to introduce, is the logical dual to “extending a parallel composition with one more process”. As before, our definitions and results are parametric in a set N of names and the associated set L of labels (cf. Notation 1). We begin with introducing spatial logic. In essence, this definition is modelled after the characterisation given in Corollary 7. Definition 7. The language L of spatial logic is the least set of formulas according to the grammar L φ, ψ ::= | @n | ff | φ → ψ | Rφ | φψ l
where n ∈ N , l ∈ L ∪ {τ } and R ranges over the relations ↓, =⇒ and =⇒ for l ∈ L. Intuitively, the formula allows us to speak about the empty context and @n allows us to observe the names of locations. Formulas of type Rφ allow us (as in standard modal logic) to reason about the behaviour of a process after evolving according to the relation R. In our case, we can specify properties of sub-configurations (using ↓), tranl sitions (using =⇒) and labelled reductions (using =⇒). The most interesting formula is of type φψ: it asserts that we can split a process into a single node satisfying φ and a remainder, satisfying ψ. Definition 8. The semantics of propositional connectives is as usual. For the modal operators, we put, for C ∈ C: C |=
iff C ≡ 0
C |= @n C |= Rφ
iff C ∈ @n iff ∃C .(C, C ) ∈ R
C |= φψ
and C |= φ iff ∃C , C .C (C , C ) and C |= φ, C |= ψ
where R is as above. As usual, Th(C) = {φ ∈ L | C |= φ} denotes the logical theory of C ∈ C. Two configurations C, D are logically equivalent, if Th(C) = Th(D). Note that we use the expression “@n” above both as an atomic formula of the logic and as a unary relation. In this section, we show that logical equivalence gives yet another characterisation of spatial congruence, provided the underlying set of processes is finitely branching. This follows from the characterisation of spatial congruence as spatial bisimulation + labelled reduction by appealing to Proposition 2. We then obtain a characterisation of spatial congruence in the sense of Hennessy and Milner [7]. The main result of this section is as follows:
Making Components Move: A Separation of Concerns Approach
499
Theorem 10. Suppose (P, −→) is image finite. Then spatial congruence and logical equivalence coincide. Proof. We use the characterisation of spatial congruence as labelled bisimulation and Proposition 2. It follows directly from the definition of spatial logic, that formulas of spatial logic cannot distinguish states, which are labelled bisimilar, hence labelled bisimilarity is contained in logical equivalence. For the converse, we use the method of Hennessy and Milner [7] and a variant of Proposition 2, replacing “i + 1” by “i” in the last clause of the assumption (the meticulous reader is invited to check that the Proposition remains valid). Suppose for a contradiction that there is a pair of configurations (C, D) ∈ C × C such that C and D are logically equivalent, but not labelled bisimilar. Let i be minimal such with the property that C ∼i D but C ∼k D for all k < i (such an n exists because of Proposition 2). Since C and D are not labelled bisimilar, we have – up to symmetry – one of the following cases: 1. C ∈ @m but D ∈ @m for some m ∈ N . Then C |= @m but D |= @m, contradicting Th(C) = Th(D). 2. There is C ∈ C such that (P, P ) ∈ R but there is no D ∈ C with (D, D ) ∈ R l and C ∼i−1 D , where R is one of ↓, =⇒ or =⇒ (for l ∈ L). Since i is minimal, this means that for all D with(D, D ) ∈ R there is a formula φD such that D |= φD but C |= φD . Take φ = D :(D,D )∈R RφD , which is well defined by Lemma 1. Then C |= φ but D |= φ, contradicting Th(C) = Th(D). 3. There are C0 , C1 with C (C0 , C1 ) but there is no (D0 , D1 ) ∈ C × C with Dj ∼i−1 Cj (j = 0, 1) and D (D0 , D1 ). The argument is as above, using formulas of the form φ ψ. Summing up, we have shown that Spatial congruence = spatial bisimulation + labelled reduction = structural congruence up to process bisimilarity = logical equivalence Before extending these correspondences to a more general setting, we give some examples. Example 4. We use the same setup as in Example 2. 1. Consider the configuration C ≡ home, client from Example 2. We have C |= )(@home, tt), corresponding to the statement that there is a top level node with the name “home”. Also, C |= (↓@agent, tt), which expresses that C has a subtree, one node of which has the name “agent”. 2. Consider the configuration C ≡ n1 P [], n2 Q[], similar to Example 2. Here, C |= (@n1 , tt), i.e. there is a location in C with the name “n1 ”. Also, C |= (@n1 , ((@n2 , ))), which says that all top level processes contained in C have either the name n1 or n2 .
500
D. Pattinson and M. Wirsing
5 Local Names In the calculus of mobile ambients, local names are essential for many examples. The treatment of local names is derived from the π-calculus, i.e. governed by structural rule of scope extrusion (νnP ) | Q ≡ νn(P | Q) whenever n is not a freely occurring name of Q. In the ambient calculus, local names cut across dynamics and spatial structure, by adopting a second structural rule: νn(k[P ]) ≡ k[νnP ] if n = k, which allows to move the restriction operator up and down the tree structure, induced by the nesting of the ambient brackets. If we want to remain independent from the underlying process calculus, we cannot adopt the latter rule. However, we can look at a calculus with local names, where local names obey scope extrusion a la π-calculus. The next definition extends the syntax as to incorporate local names. In order to deal with scope extrusion, we also have to introduce the concept of free names. Definition 9. The set C of configurations in the calculus with local names is given by C C, D ::= 0 | nP [C] | C, D | (νn)C for n ∈ N and P ∈ P. Given P ∈ P and n ∈ N , we say that n is free in P , if there lk l1 l2 l are l1 , . . . , lk and P1 , . . . , Pk such that P −→ P1 −→ · · · −→ Pk −→ Q, where l is one of in n, out n and open n. We let fn(P ) = {n ∈ N | n free in P }. For C ∈ C, the set fn(C) is defined by induction on the structure of C as follows: – – – –
fn() = ∅ fn(C, D) = fn(C) ∪ fn(D) fn(nP [C]) = {n} ∪ fn(P ) ∪ fn(C) fn(νnC) = fn(C) \ {n}
where structural congruence is as in Definition 1, augmented with α-equivalence and the rule (νn)(A, B) ≡ (νnA), B whenever n does not occur freely in B. The operational semantics is given as in Definition 1, augmented with the rule C =⇒ C (νn)C =⇒ (νn)C for C, C ∈ C and n ∈ N . Note that, in order to be able to state the rule for α-equivalence, we need a notion of substitution on the underlying processes, which we do not make explicit here. Before investigating the logical and algebraic theory of the calculus with local names, we give a short example. Recall that in Example 2, we had an agent in a home location, the sole purpose of which was to transport clients inside the home-location. However, as we remarked when discussing this example, nothing prevents the client process to enter the home-location directly. This shortcoming can now be remedied in the calculus with local names.
Making Components Move: A Separation of Concerns Approach
501
Example 5. We can now model an agent, which has the capability to enter and exit its home location and to transport clients inside with local names as follows: We let “client” and “agent” as in Example 2 and put home = (νh)h0[agent] Using scope extrusion, we have the same chain of reductions as in Example 2. However, since h is a private name now, the client cannot enter “home” without the help of “agent”. The next issue we are going to discuss is the algebraic and the logical theory of the calculus with local names. In order to obtain a similar characterisation as in the calculus without local names, we have to extend the definition of spatial bisimulation, and demand closure under name revelations. Definition 10. Suppose C ∈ C and n, k ∈ N . We put rev n
C =⇒ C
iff C ≡ (νk)C and C ≡ C[n/k]
whenever n ∈ / fn(C). The definition of spatial bisimulation is modified as follows: Spatial bisimulation is the largest symmetric relation which is closed under spatial reduction =⇒, forest reduction , subtree reduction ↓, top level names @n and under rev n revelation =⇒ (for all n ∈ N ). As before, spatial congruence is the largest congruence, which is a spatial bisimulation. We now turn to the impact of local names on the equivalences, which we have discussed previously. Since we make revelation an explicit part of spatial bisimulation, everything goes through as before, once the equivalences are transferred (without changes) to the calculus with local names. We obtain: – labelled bisimulation is the largest spatial bisimulation, which is closed under labelled reduction – weak structural congruence is the least relation, which contains structural congruence and all pairs of the form (nP [C], nQ[C]) for P, Q ∈ P process bisimilar. Comparing these equivalences, we obtain Theorem 11. In the calculus with local names, spatial congruence coincides with labelled bisimulation and with weak structural congruence. Proof. We extend the respective results for the calculus without local names. The arguments used in Lemma 5 remain valid, showing that spatial congruence is closed under labelled reduction, implying that spatial congruence is contained in labelled bisimilarity. In order to see that labelled bisimulation is a congruence, one has to consider revelarev n tion reductions, that is, reductions of the form =⇒ on top of the reductions considered in Lemma 6, but they do not pose any problems. The comparison of spatial congruence and weak structural congruence is as in Proposition 8.
502
D. Pattinson and M. Wirsing
In order to transfer the characterisation result to a logical setting, we introduce a hidden name quantifier a la Gabbay / Pitts [6]: Definition 11. The language of spatial logic with local names is the least set according to the following grammar L φ, ψ ::= | @n | ff | φ → ψ | Rφ | φψ | Hn.φ Given C ∈ C and φ ∈ L, satisfaction C |= φ is as in Definition 7, plus the clause rev n
C |= Hn.φ iff C =⇒ C and C |= φ for the hidden name quantifier. As before, Th(C) = {φ ∈ L | C |= φ} for C ∈ C, and C, D ∈ C are called logically equivalent, if Th(C) = Th(D). Since the relation rev n (for n ∈ N ) is image-finite, Lemma 1 and Proposition 2 remain valid in the calculus with local names. We thus obtain Theorem 12. In the calculus with local names, spatial congruence and logical equivalence coincide.
6 Further Extensions This section shows, that the separation of dynamic and spatial aspects of mobile components allows for seamless integration of extensions, which are more difficult to model in other calculi. First, we demonstrate that multiple names can easily be handled, since every process runs in precisely one location. It is therefore a straightforward extension to allow the controlling process to change the name of that location. The second extension can be seen as orthogonal: Since the behaviour of every location is governed by precisely one process, new controlling processes can easily be substituted into configurations. This section intends to give an idea regarding extensions of the BasicSail calculus; we leave the investigation of the algebraic and logical theory for further work. 6.1 Change of Names and Multiple Names In the ambient calculus, each ambient has precisely one name, which does not change throughout the reduction process. One can argue that this does not reflect the real world in a faithful manner, since computing devices can also have no names, multiple names or change their names over time. The explicit reference to the enclosing location allows to model the change of names elegantly in the Sail-calculus by extending the set of labels, which influence the spatial reduction relation. Since we want to keep the separation of the dynamical from the spatial structure, we let the controlling processes change the names of locations through an interface (a set of distinguished labels) as before. This necessitates to extend the set of labels for the underlying process calculus:
Making Components Move: A Separation of Concerns Approach
503
Convention 3. We extend the set L of labels to include primitives for name changing as follows: L = {in , out , open , up , down } × N ; as before, (P, −→) is a labelled transition system with −→⊆ P × L × P. Definition 12. In the calculus with multiple names, configurations are given by C A, B ::= 0 | A, B | νnA | (n1 , . . . , nk )P [B] where k ∈ N and n1 , . . . , nk ∈ N . The axioms and rules of structural congruence are those of Definition 9 augmented with (n1 , . . . , nk )P [B] ≡ (nσ(1) , . . . , nσ(k) )P [B] (n, n, n1 , . . . , nk )P [B] ≡ (n, n1 , . . . , nk )P [B] whenever σ : {1, . . . , k} → {1, . . . , k} is a bijection. The operational semantics is that of Definition 9, augmented with the rules up n
P −→ P nP [A] −→ n+n (P )[A] down n
P −→ P nP [A] −→ n−n (P )[A] where n−n deletes n from the list n of names; n+n adds n to the list n of names. The idea of a term (n, m)P [A] is that of a location with two names, n and m, running the programme P and which has A as sub-locations. The additional rule of structural congruence captures the fact that there is no order on the names. The gained expressivity allows us to treat the following: Example 6. 1. Anonymous locations are modelled by an empty set of names. Take for example ()P [A] for P ∈ P and A ∈ C. Note that anonymous locations are anonymous also for processes from within, that is, the same effect cannot be achieved using local names. Indeed, the processes νn(n)P [kout n[]] and ()P [kout n[]] differ in that the former can perform a reduction under the name binder, whereas the latter cannot. 2. Consider the configuration (n)down n.0[A], ()in n.0[B]. First, this shows that unnamed locations can perform movements. Second, this example illustrates that the movement only succeeds, if the unnamed agent is lucky enough to enter into his partner before the name disappears.
504
D. Pattinson and M. Wirsing
6.2 Dynamic Reconfiguration We conclude by demonstrating the strength of our approach by discussing dynamic reconfiguration, another extension of the basic calculus. Here, we use the one-to-one relation between locations and controlling processes to model dynamic reconfiguration, i.e. locations, which dynamically change the programme they run. Sloppily speaking, this allows for downloading a new programme, which is then run in an already existing location. As with multiple names and the change of names, the explicit reference to the enclosing location allows for a concise and elegant formulation of dynamic reconfiguration. Note that this in particular necessitates the transmission of programmes (processes). The extension of the calculus follows the same scheme as the above extension with multiple names: in order to keep dynamic and spatial structure apart, we introduce new labels, which act as an interface, through which the controlling process manipulates the spatial structure. Convention 4. We extend the set L of labels to include primitives for dynamic reconfiguration as follows: L = {in , out , open } × N ∪ {send , rec , run } × P; as before, (P, −→) is a labelled transition system with −→⊆ P × L × P. Note that this requires the underlying transition system to have processes occurring in the labels, since processes need to be transmitted. Except for the absence of channel names, this is for example realised in the higher order π-calculus (see [13, 16]). For our purposes, it suffices that processes can be transmitted and received; we leave the concrete (syntactical) mechanism abstract. Definition 13. In the calculus with dynamic reconfiguration, configurations are given as in the calculus with local names (but recall the extended set of labels). The operational semantics for the calculus with dynamic reconfiguration is given by the rules of Definition 9, augmented with send R
rec R
P −→ P Q −→ Q nP [C],mQ[D] ⇒ nP [C],mQ [D] run R
P −→ P nP [C] =⇒ nR[C] Note that in the action run R, R is a process and the reduct of P after the run R reduction is forgotten. Using dynamic reconfiguration and communication, we can now model a location, which updates the process it executes: Example 7. We model an electronic device, which attempts to update the code it is running (its operating system). That is, it tries to replace the programme which it is running by another (newer) version. In order to model this behaviour, we first have to be more precise about the underlying set of processes: We let P, Q P ::= 0 | P Q | α.P |!P | X | run Q.P | send Q.P | rec Q.P
Making Components Move: A Separation of Concerns Approach
505
where X ∈ X ranges over a set of (process valued) variables. The process level transition relation from Example 1 is augmented with rec Q
rec X.P −→ P [Q/X] and the usual rules send Q
send Q.P −→ P run Q
run Q.P −→ P Note that in particular process variables X ∈ X do not generate reductions. Now consider P = (rec X.run X) O running inside location n, that is, the configuration C = nP [B], where B are n’s sub-locations. In the vicinity of a location which sends updates, e.g. U = u!(send (rec X.run X N ))[], where N stands for the “new” firmware, we have U, C =⇒ U, nrun (rec X.run X N ) O[B] which, executing the run -operation, reduces to U, nrec X.run X N [B], that is, a process which (again) waits for an update, but now running the new firmware N. As already mentioned in the introductory remark of this section, both extensions, multiple names and dynamic reconfiguration, are to demonstrate the extensibility of the calculus; the study of the algebraic and logical properties is left for further research.
7 Conclusions and Related Work As discussed above the first calculus for mobile computation was the π-calculus [8]. Further calculi are the Fusion calculus [12], Nomadic Pict [18] and the distributed coordination language KLAIM [10]. The study of hierarchical re-configurable administrative domains was introduced by the Ambient [5] and the Seal calculus [17]. BasicSail follows these lines but distinguishes processes and configurations in an a priori way and concentrates on a even simpler set of operations for reconfiguration. The basic calculus and its variations were inspired by the Seal-Calculus. [17]. However, the Seal-Calculus is quite involved syntactically; the present calculus is a simplification in order to study the effect of the separation of dynamics from the underlying topological structure, which is also present in Seal. The second source of inspiration was the calculus of mobile ambients [5]. As we have pointed out before, our principal
506
D. Pattinson and M. Wirsing
design decisions do not allow to embed the full ambient calculus into our framework. Spatial logics were studied by Cardelli and Caires [2, 3], although to our knowledge not wrt. a clear characterisation of the expressive power. Such a characterisation (called “intensional bisimulation”) was considered by Sangiorgi for a variant of the ambient calculus [14, 15]. Separation of Concerns in Models of software architecture has also been addressed – albeit not in the context of mobile code – in [1, 11]. There the authors differentiate between components, which provide certain services, and an additional layer, which describes the composition of components. In the context of explicit code mobility, this approach can be seen as orthogonal to ours; and it would certainly be interesting to have coordination and mobility in a single framework. Of course, there remains a wealth of open problems: Most pressingly, we have investigated neither the logical nor the algebraic theory of the calculus with multiple names or the calculus with reconfiguration.
References 1. F. Arbab. Abstract behaviour types: A foundation model for components and their composition. This Volume. 2. L. Caires and L. Cardelli. A spatial logic for concurrency (part i). In N. Kobayashi and B. Pierce, editors, Proc. TACS 2001, volume 2215 of Lecture Notes in Computer Science, pages 1–37. Springer, 2001. 3. L. Caires and L. Cardelli. A spatial logic for concurrency (part i). In L. Brim, P. Jan`car, M. K`ret`ınsk`y, and A. Ku`cera, editors, Proc. CONCUR 2002, volume 2421 of Lecture Notes in Computer Science. Springer, 2002. 4. L. Cardelli and A. Gordon. Anytime, anywhere: Modal logics for mobile ambients. In Proc. POPL 2000, pages 365–377. ACM, 2000. 5. L. Cardelli and A. Gordon. Mobile ambients. Theor. Comp. Sci., 240(1):177–213, 2000. 6. D. Gabbay and A. Pitts. A new approach to abstract syntax involving binders. In 14th IEEE Symposium on Logic in Computer Science (LICS 1999), pages 214–224. IEEE Computer Society, 1999. 7. M. Hennessy and R. Milner. Algebraic Laws for Non-determinism and Concurrency. Journal of the ACM, 32:137–161, 1985. 8. R. Milner. Communicating and Mobile Systems: the π-Calculus. Cambridge University Press, 1999. 9. U. Montanari and V. Sassone. Dynamic congruence vs. progressing bisimulation for CCS. Fundamenta Informaticae, 16(2):171–199, 1992. 10. R. De Nicola, G. Ferrari, and R. Pugliese. Klaim: a kernel language for agents interaction and mobility. IEEE Trans. Software Engineering, 24(5):315–330, 1998. 11. O. Nierstrasz and F. Achermann. A calculus for modelling software components. This Volume. 12. J. Parrow and B. Victor. The fusion calculus: Expressiveness and symmetry in mobile processes. In Thirteenth Annual Symposium on Logic in Computer Science (LICS 1998), pages 176–185. IEEE, IEEE Computer Society, 1998. 13. D. Sangiorgi. From π-calculus to Higher-Order π-calculus — and back. In M.-C. Gaudel and J.-P. Jouannaud, editors, Proc. TAPSOFT 93, volume 668 of Lect. Notes in Comp. Sci., pages 151–166, 1993.
Making Components Move: A Separation of Concerns Approach
507
14. D. Sangiorgi. Extensionality and intensionality of the ambient logics. In Proc. POPL 2001, pages 4–13. ACM, 2001. 15. D. Sangiorgi. Separability, expressiveness, and decidability in the ambient logic. In 17th IEEE Symposium on Logic in Computer Science (LICS 2002). IEEE Computer Society, 2002. 16. Davide Sangiorgi and David Walker. The π-calculus: a Theory of Mobile Processes. Cambridge University Press, 2001. 17. J. Vitek and G. Castagna. Seal: A framework for secure mobile computation. Internet Programming, 1999. 18. P. Wojciechowski and P. Sewell. Nomadic pict: Language and infrastructure design for mobile agents. IEEE Concurrency, 8(2):42–52, 2000.
This page intentionally left blank
Author Index
´ Abrah´ am, E. 1 Achermann, F. 339 Arbab, F. 33 Arnout, K. 285
Meyer, B. 285 Montanari, U. 319
Batson, B. 242 Boer, F.S. de 1
Olderog, E.-R.
Cheon, Y. 262 Clifton, C. 262 Cohen, I.R. 136 Cok, D.R. 262 Damm, W. 71, 99 Deng, X. 154 Dwyer, M.B. 154 Efroni, S.
136
Ferrari, G.
319
G¨ ossler, G.
443
Harel, D. 136 Hatcliff, J. 154 Hooman, J. 182 Jacobs, B. 202 Jong, H. de 220 Josko, B. 71 Jung, G. 154 Kiniry, J. 202 Klint, P. 220 Lamport, L. 242 Leavens, G.T. 262
Nierstrasz, O.
339 361
Pattinson, D. 487 Plosila, J. 424 Pnueli, A. 71 Pol, Jaco van de 182 Raggi, R. 319 Robby, 154 Roever, W.-P. de Ruby, C. 262 Rumpe, B. 380
1
Sekerinski, E. 403 Sere, K. 424 Sifakis, J. 443 Singh, G. 154 Steffen, M. 1 Stevens, P. 467 Tenzer, J. 467 Tuosto, E. 319 Votintseva, A.
71
Wald´en, M. 424 Warnier, M. 202 Wehrheim, H. 361 Westphal, B. 99 Wirsing, M. 487
